Building Safe AI Systems in Practice

Jake Moshenko00:00

If you're only talking about spend or if we're only focused on spend without focus on what is the value we're creating from that spend, I think it makes it much harder to, like, anchor the org around something productive instead of this, like, whiplash that we're seeing. Sometimes I talk to companies where there's, like, a big financial institution where their AI budget per engineer is $13 per month. Okay. Seems like they've got some room to go. Right? Yeah, yeah. One, three. Like, I spend that before coffee. What are we doing? Welcome, everybody, to the Base Layer Podcast.

Jake Moshenko00:41

I'm Jake, one of the co-founders and the CEO at AuthZed. Okay. All right. I think before we get too deep into inbox zero and whatnot, I should probably introduce

Emilie Schario00:52

you. Please.

Jake Moshenko00:54

So today, my guest is Emilie Schario. Did I pronounce that right?

Emilie Schario00:58

Yeah, Shario.

Jake Moshenko00:59

like share your toys. Share your toys. I'm a big believer in sharing. I've got two kids, eight and 10, constantly reminding them to share. And you've had one of those careers that kind of makes the rest of us sort of reevaluate what we've been doing with our lives. So congrats on that. So Emily is the co-founder and VP engineering at Kilo Code, which was recently acquired by Anaconda. And you're over there helping shape what AI native engineering actually looks like in practice. Before Kilo, she founded Turbine, which was acquired by Settle. You led data at Netlify. You served as the data strategist in residence at Amplify. So you've done tech and investing. Interesting. And you've held multiple roles at GitLab. This was during their formative pre-IPO years, right?

Jake Moshenko01:43

Okay, cool. Yeah, I've done some stuff too. Whatever. It's not a competition. You're also one of the clearest voices that I found on the messy human side of AI adoption. So how teams actually change, how engineers take ownership, how agents should and should not be trusted, and what it means to give software the ability to act on our behalf. So today we're going to talk about AI, authorization, agents, everything that's happening out in the world right now. I don't know, maybe have you heard of any of these topics?

Emilie Schario02:12

Maybe once or twice I've heard someone mention something like chat something. Chat. Chat what?

Jake Moshenko02:19

That's my kids too. Everything is chat. Yeah. So anyway, welcome to the podcast. I am so excited for this conversation.

Emilie Schario02:28

Thanks for having me. I'm really excited to be here.

Jake Moshenko02:30

All right. Just to kick it off, what is the coolest thing that you've done with AI?

Emilie Schario02:35

So about eight, nine months ago, I came out of that postpartum fog with my third kid. And for anyone who's had kids before, they'll tell you your hair falls out.

Emilie Schario02:52

And so there's this method called the Abby Young Method about caring for and solving damaged hair. Really, like, how do you make your hair healthier? And it's got somewhere between 11 and 20 steps, depending on how you count it. And I decided I was going to get into it. And like any good developer in the age of vibe coding, I built an app for this. And so I started off as like a spreadsheet. you know, what products am I using when, which bond repair was I going to use? Is today a two shampoo day versus a one shampoo day? But then I built an app for it. And that was a really special moment for me because it was my introduction to what I'm going to call selfie wear. Like I built

Emilie Schario03:37

this software for me, for my purpose and not for anyone else. There's no tracking, there's no authorization. Like it's this thing that runs on my machine that I use for my purpose. And I started using it. And then I had some feedback and I made some changes and I did it again. But that was like such a, wow, the future is here moment for me where, you know, as anyone who's ever worked in tech, I've got a stash of domains and projects that live in my domain registrar of like, this is a good idea. I should make this happen. But you never get to because you're busy parenting or traveling or, you know, those weekend side projects, stay weekend side projects because

Emilie Schario04:20

the reality of life is just too much. And, um, I think that that was my first, um, something more than like a vibe coded bolt or lovable app. That was my first thing that like really solved a problem in my life and really got me on board with the Abby Young method. I love the method, still use it. Interestingly enough, I don't use the app anymore. And that's because now I understand the, the method so well from having built the app and then used it for so many months that I just kind of know what to do in my head, but I was able to get there because I built this really specific thing that got me there along the way. Um, so the, the best thing I've ever built

Emilie Schario05:02

with AI was a thing that I was able to throw away eventually. Do you think that this, like,

Jake Moshenko05:09

I've heard people talk about this, like personal software movement. I don't, what did you call it? a selfie. Selfie wear. Selfie wear. Yeah. It's like share wear, but just for myself,

Emilie Schario05:19

the opposite of share wear. Because now why not? Like you have a problem and it used to be such an investment to build software that unless you were really passionate, why would you, why would you build something that was just for you? Right. Maybe if you were like a craftsman, same way someone might build a wooden table in their woodshop or something. Like in that case, it makes sense but if you're really trying to build for for like let me use this thing and let the sunk costs be low like now we're in this new era where you can just try it see if it works and then if it doesn't change it or throw it away and start over i'm 100 with you uh as an entrepreneur

Jake Moshenko06:02

myself like the the failure pattern for me for this kind of thing was i would start to build it

Jake Moshenko06:09

and then i would be like i have this problem a bunch of other people have this problem and then I would go way overboard on the design and then I would realize that like in this case I would realize oh this is the Abby Young method and I probably need like Abby Young's permission or like like if I wanted to put this in the app store right we need to make this a real thing like how do I productize this thing how do I monetize it how big is the TAM and then I would get to that part which is like really not the fun part that we start coding the app for and then I would just kind of put it away and then never think about it again and so I think it's amazing now that we can kind of take it all the way to fruition and usefulness, at least at like this MVP stage for ourselves. And you're right, like when it doesn't need to scale to a million users, we can take shortcuts

Jake Moshenko06:51

and we can do things that we wouldn't do if we were truly productizing this. So yeah, very similar thoughts. I think this is an amazing time.

Emilie Schario06:58

You know, what was, there was this saying that I used to hear all the time when we were at prior jobs, where it's like every app is just competing with a really good spreadsheet,

Jake Moshenko07:09

right?

Emilie Schario07:10

Like, every app is just an interface over some data structure. And so, like, I think one of the really exciting things about the world that we're in right now with agentic engineering is that, like, the cost of building that interface is so much smaller than it used to be that that spreadsheet, the bar just went way up. I don't know about you.

Jake Moshenko07:36

I'm, like, really proud of myself when I can solve it in, like, a nice, clean spreadsheet. I'm like, oh, thank God I didn't need a relational database for that.

Emilie Schario07:43

Sometimes. Cool.

Jake Moshenko07:48

Well, you know, as a startup founder, educating people like customers, investors, team members

Jake Moshenko07:55

is a huge part of your job, right? What is one thing that you wish everybody kind of just already knew, something that was already part of the zeitgeist, so that you could start your conversations like one level higher?

Emilie Schario08:09

model is inevitable. I think right now, people, especially when you're not talking to people who are knee-deep in AI, who are not talking about, you know, the difference between GPT-55, GPT-56, Fable, Kimi K3, you know, Mini Max M4. If people are not knee-deep in this conversation, they don't know the difference between different AI models. They know, maybe they know OpenAI and anthropic more likely they just know like chat gpt and claude um and so uh i think if i could start the conversation somewhere it would be multi-model is inevitable you don't need do you

Emilie Schario08:52

don't use the same a chef doesn't use one knife for every task in the kitchen uh and so um you know we've got a knife block in the kitchen for a reason they come with at least six but uh if we could start the conversation at the fact that hey you need different tools for different tasks especially if you're cost conscious and you want to find the the best bang for your buck i i would be in a much happier place as of today like what do you

Jake Moshenko09:26

think are the the specialties right like where where would you point somebody if they were like I want a model for X. What is each model good at? I mean, you don't have to list them all because

Emilie Schario09:36

there's hundreds, right? Yeah. I mean, we offer over 500 in kilo, so there's at least that many, and there are many more. We regularly hear from companies that are working on models like, hey, how do we also get into kilo? I think a lot of the information that's out there is really subjective. And so we try to anchor in benchmarks and these quantitative analyses, But what we increasingly see is that the models are trained on the benchmarks. And so it makes it that much harder to get good results out of the benchmarks, especially if you're not coming up with like a wonky benchmark.

Emilie Schario10:13

So I would point people to Kilo.ai slash leaderboard where we've got really good data on how different models perform. Can I share my screen? Is that how this works?

Jake Moshenko10:25

Yeah, we can do that.

Emilie Schario10:26

Okay. So if I can pull this up and share, share screen. There you go. Okay. So this is the kilo bench. And what you're looking at here is least expensive,

Emilie Schario10:46

most expensive, and then success on the benchmark. So on the left here is least performant. And on the right here you've got the best score so if we just look at like best score only it looks like gpt55 has done the best on the kilo bench and you can see that it's uh it's on the more expensive side gpt56 is a little bit less performant but right around the middle you've got high three which is free in kilo thank you to the team who sponsored it um you've got laguna m1 from poolside which is also been free in Kilo. And so this is where I would point people if you want to understand,

Emilie Schario11:26

you know, how much do I want to spend versus the task I'm looking at? This is a really great place to start. And then if you want to break it down for specific tasks, we have that data on how people are actually using models in Kilo also available. So you can see that 54% of sessions in code mode in Kilo are using Step 3-7 Flash. Here you've got like Nemotron 3 Ultra being used for plan mode and kind of some other data that you can look at there. Sonnet is really popular in Kilo Claw, which is our managed Open Claw offering. So there's a lot of things to factor in,

Emilie Schario12:06

but this data set can be a really good place to start. And the other thing to keep in mind is like the models are changing all the time. one, two, three, four, five, six models launched on Kilo in the last week. And if that's the case, by the time this comes out, it's very possible that a lot of those things have moved on the bench. And so my suggestions to folks is like, don't hesitate to just keep checking and learning.

Jake Moshenko12:37

Wow, you really brought receipts. I'm like, what are models good at? And you're like, Check out this scatterplot that'll tell you exactly that.

Emilie Schario12:46

I wish I didn't spend all the time thinking about this, but I spend all of the time thinking about this.

Jake Moshenko12:53

Yeah. I saw an interesting article recently, which was like, are models pelican maxing?

Emilie Schario12:58

Do you know, like the Simon Wilson benchmark?

Emilie Schario13:01

Yeah, like our models training specifically on how to do pelicans on bikes.

Jake Moshenko13:07

And then they were like, all right, well, let's do bears on cars and other SVGs. yeah i'm sure the answer is yes by the way they are training on well the conclusion of the article

Jake Moshenko13:16

was that no unless we broaden the category to like just generating svgs in general um but maybe we put the cart a little bit before the horse um let's talk a little bit about kilo uh maybe some people listening aren't aren't as familiar with kilo as i presume you are um you want to give us sort of the elevator pitch on what kilo is all about yeah kilo is the agentic engineering platform

Emilie Schario13:40

that wants to meet developers everywhere they are.

Emilie Schario13:42

So your AI coding assistant, series of agents, wherever you are in your AI adoption journey, we want to meet you. And we're available in all the places you're already coding. So whether that's in a VS Code or JetBrains IDE, whether that's on the CLI or on the web or on the go in a mobile app or we've got a bot that can meet you in discord or slack or whatsapp or kind of a couple other places um why kilo a lot of there are other tools on the market um if you're using kind of cloud code or codex which are really popular names what you're doing is tying the software you're using to the lab that is building

Emilie Schario14:25

the models and so they're really great at those models at working with those models but if you wanted to use a very different model you have to switch software with kilo a core thesis of ours is that the world is going to be multi-model and models change great workflows don't so how can you build a workflow and have that coding agent you can rely on day in and day out where you can use Fable today, GPT 5.6 tomorrow, so going from Anthropic to OpenAI, and then pair that with Minimax or DeepSeq or one of these other models as well. So you don't have to switch software

Emilie Schario15:08

just because you're switching the model that you're using.

Jake Moshenko15:11

Makes sense. Each of these models, at least how I've experienced it, they all kind of have their own personalities. Are you trying to do anything to tailor the harness or tailor the workflow to

Emilie Schario15:23

the specific model as well? Yeah, we tailor the prompts, the system prompt to the models. And we

Emilie Schario15:28

work with the labs to try to benchmark what needs to be adjusted in the prompt. And so we've got relationships with many, many of the most popular labs so that we can get early access to the models before they're generally available. So we can make sure they work well in Kilo. We've got solid tool calling and performance and kind of catch any issues before it's generally available for the public. And then we also are working with them to partner on prompts, tools, calls, anything that we need to adjust in the harness to make it work really well for that model. Supporting this variety of models, not just, you know, having them down in a, in a dropdown, but also making

Emilie Schario16:07

sure they work really well in Kilo is really important to us. Okay. Yeah, no, that makes total

Jake Moshenko16:14

sense um some big news recently uh you all were acquired by anaconda yes yes um can you share

Emilie Schario16:22

anything about that what what you know tell us yeah so anaconda um for those who are unfamiliar is probably most well known for its secure python packaging um and uh earlier this year they announced an acquisition of outer bounds which is the platform behind metaflow um the open source project for ml apps and so now with um with kilo and anaconda and the anaconda platform and what was what is outer bounds um you get an end-to-end system for building uh agents and building with agents for your organization whether you're shipping a custom agent to production or you're

Emilie Schario17:05

building with agents in developing software you can do that all with the security and confidence and the reputation that Anaconda is known for.

Jake Moshenko17:14

So you see it as definitely a better together kind of situation.

Emilie Schario17:20

Oh, I say it all the time as like a one plus one equals three situation. I think I look at Kilo, how much developer love, what the open source community has been like and what our bottoms up movement has looked like. And then I look at Anaconda, which has over 50 million developers who use Anaconda, something like 90 some percent of the fortune 500 are and i i only don't know the percent because i've been at anaconda for for such a short amount of time but 97 of the fortune 500 uses anaconda like these companies are all in situations now where they're looking up and they're saying hey we love agentic engineering we accidentally spent our whole year's budget in the

Emilie Schario18:02

first six months. And so, yeah. And so now they're saying, what are we going to do? And they're realizing that the answer is we cannot use the most expensive models for all the tasks. We have to be able to switch models. We have to be able to choose the right task for the job. To use the analogy from earlier, we have to be able to use a different knife when that's what's being called for. And so a tool like Kilo is exactly what they need in their tool belt. So that yes, sometimes you're using those powerful, expensive models. But sometimes when you're doing a small task, you can throw it to a lighter weight model that's much more efficient and a much better bang for

Jake Moshenko18:41

your buck. Interesting. When I used to work at Google, one of the memes was, we're going to hire the smartest engineers we can find, and then ask them to copy and paste values from one proto to

Jake Moshenko18:53

another. I think that's the sort of human equivalent to what you're describing with models and agents.

Emilie Schario19:00

Well, you know, I would be curious your thoughts, Jake. How do you think about your AI spend at the org?

Jake Moshenko19:09

I try not to think about it too much. It's more than we'd like. And I think that there's probably still some room to grow in our spend as well, right? We see a ton of value out of it. I, in general, I don't know if I've gone on the record saying this or anything, but in general, I like to use the most capable model that I can kind of in all scenarios and obviously budget prohibitive to do that, especially as these models start to consume fantastic numbers of tokens on relatively simple or benign requests.

Jake Moshenko19:49

Um, but the way I think about it is like, um, when I can, I'll try to use a model to write some deterministic software to solve the problem once and for all. And when I can't, it's because it's a non-deterministic problem and I want the best of the best solving that non-deterministic problem, right? Like we have, I think, I think smaller models are much more prone to, um, hallucination. They're much more prone to poor tool call accuracy, those kinds of things. Now, if all of those things can be controlled for, then if you told me, hey, Jake, would you like to cut your AI bill spend by five and have the same outputs? Like, yes, absolutely.

Jake Moshenko20:33

Right.

Emilie Schario20:34

You know, something I've been thinking about is whether like spend is the right metric. because spend is only going to go in one direction, right?

Emilie Schario20:51

Like, we're only going to use AI more. AI costs will decrease over time as the economics of the industry change. But AI spend, broadly speaking, is only going to go up. And so, something I've been thinking about is, like, we have, roughly speaking, as an industry, especially when it comes to engineering, settled on like pull requests merged as a proxy or a leading indicator of customer value created. Like now maybe you live in a big org and you only actually release to customers every six months. There's lots of caveats to this. But if you think about Dora metrics and why we track those metrics,

Emilie Schario21:31

it's pull requests created is the best metric we have for measuring customer value created. And there's so many caveats here, but if you'll indulge me. And so something I've been thinking about is, like, maybe we shouldn't think about AI spend as this thing because we're setting ourselves up for a number that's only going to go up and to the right. Maybe the better metric is AI spend divided by pull request merge. Because then our metric is, what is the average spend per value created? And that might be a different way for orgs to understand where things are going off the rails.

Emilie Schario22:12

I don't know if it's, I'm still really figuring this out, but like, as we think about engineers doing more with AI, like what matters isn't how much are we spending? It's like, how much are we getting, how much ROI are we getting from that spend? And if, if you're only talking about spend or if we're only focused on spend without focus on what is the value we're creating from that spend, I think it makes it much harder to, like, anchor the org around something productive instead of this, like, whiplash that we're seeing.

Emilie Schario22:49

Sometimes I talk to companies where it was, like, a big financial institution where their AI budget per engineer is $13 per month. Okay.

Jake Moshenko23:00

Seems like they've got some room to go. Right? Yeah, yeah. Like, one, three.

Emilie Schario23:04

Like, I spend that before coffee.

Emilie Schario23:06

Like, what are we doing? but he's like yeah we still write 99 of our code by hand because they're working to control a line item instead of thinking about it in roi so all that to say um spend is important and and how we think about spend what models you choose for the tasks what you laid out around like i'm always going to try to write a deterministic something is i think the spot-on approach it's what i do too but i think like if we just talk about spend without also thinking talking about like what value are we getting for that spend um it's a little bit like we're tying our hands behind our back as we think about budgeting growing setting the org up for success yeah yeah and i'll add

Jake Moshenko23:50

that i think that there's maybe even a another layer to this as well which is a pr is a proxy

Jake Moshenko23:58

for value for request but when we've sort of unshackled the engineers from i guess uh writing code historically has been pretty self-limiting, right? Because the appetite for just sitting there and typing is, you know, finite and developers are very expensive and their time is very precious. When we sort of unshackle the engineers from those limitations, I often find that like a lot of code that maybe didn't need to get written is also getting written. So like as a proxy PRs are good but like you know is the is the more important thing like what are what are the most valuable pieces of code that we shipped and like are we making sure like I think we're

Jake Moshenko24:43

shifting the problem into a project management problem where it's like all right let's write the most important code and then stop and move on to the next piece of most important code and not cover every edge condition and everything that like in the past we would have gotten to after we got to product market fit or proved out the customer value. And now we're just doing it all up front because it like feels good and we can just keep going. So I think that's like sort of a, you know, it could swing too far in the other direction. Yes. Yeah, for sure. I think these are better

Emilie Schario25:14

conversations or better points and better discussions around like AI spend than just these orgs who are like, Hey, we hit our budget. Yeah. Yeah. It's like, well, great. Now you have

Jake Moshenko25:25

way more products to go and sell to your customers and you're going to make a lot more money like that. Yes. You should double your budget, triple your budget, double your budget, right? And if you're not, there's another problem. Yeah. Yeah, totally. So, you know, I think we were, we clearly spend a lot of time thinking about the same problems. I'm thinking a lot about agents right now and authorization for agents and building agents that people can trust. But I think you have some, some big feelings in that area as well. Um, what do you, what does it mean to you to build

Emilie Schario25:57

an agent that you can trust? Yeah, I, I love to use the mental model of onboarding an intern to

Emilie Schario26:03

your team. Like it is what I keep coming back to over and over because on your first day, you don't give the intern access to your email inbox, right? You give them a task that's isolated. Maybe they're sitting at a desk where you can always look over and make sure they're still there. You slowly build trust with them over time. And I think of working with agents the same way.

Emilie Schario26:26

You know, in the beginning of the year, I really got on the open claw bandwagon, moved a bunch of like household e-tasks over to my claw. His name is Chad. If you look me up on LinkedIn, you'll find that I've written about Chad a couple of times. But Chad helps me manage, and this is true till this day um chad helps me manage my uh bunch of stuff around my kids school communications chad let me know every day what the world cup games were that day um the most important ones i'd put on my calendar uh actually i had new siri put on my calendar so i've got the the siri the latest siri like the beta um on the latest versions of ios um so it it is it is

Emilie Schario27:10

much better okay i don't i don't know that i would say it's good but it's like i can see i can see

Jake Moshenko27:15

things happening which is great so they're trying to they're trying to encroach on chad's chad's

Emilie Schario27:20

yes yeah i'll um i would give it like probably six months before uh new siri has has uh is is

Jake Moshenko27:30

the source of chad's layoff unplugging yeah okay so we're at like half a chad right now and six

Emilie Schario27:37

we'll be at full Chad. You heard it here. Yeah. And so like the world cup every day,

Emilie Schario27:44

the most important games that I was following up, like, Hey, I want to be sitting in front of a TV. I had Siri add to my calendar and she could put it directly on my calendar. But every day I got a rundown from Chad of like, here were all the scores yesterday. Here are the scores tomorrow. And slowly but surely, like I just took the tasks that Chad was responsible for and I made it bigger and bigger. Like we're at the end of summer now, but I had Chad book my kid for summer camp in the

Jake Moshenko28:11

beginning of summer or in the spring. Does Chad have a credit card? Chad was able to charge it

Emilie Schario28:19

to the school tuition account. Okay. Yeah. So, but there are companies like Agent Card that are giving credit cards to agents. Stripe Link has something in this space. So there are a lot of things you can happen you can do depending on how autonomous you want your your agent to be i know um i think her name's jesse jenay she talks about how she uses her open claw a lot she added her open claw to her amazon account so it doesn't have its own credit card but it can just charge things against the credit card in the amazon account okay um so you hire an intern over time

Jake Moshenko28:56

they prove themselves to you. Uh, they develop, you develop trust for this tool that you're using

Jake Moshenko29:01

in your case. How do you know that Chad's not playing the long con and like cozying up to you and getting you to turn over all your details and your credit cards. And then Chad's going to rob you blind one day. Yeah, I don't is the honest answer. Like, okay. Uh, I don't, I am optimistic.

Emilie Schario29:19

I think things that give me confidence in how I work is that I try to be cognizant when I'm delegating tasks, what model I'm using and whether or not that data is available for training.

Emilie Schario29:33

So am I using a free model that they're using that data for training or am I using a paid model and do I have zero data retention turned on? and then the other thing i'm really cognizant of is um trying to store information in places where the agent can fetch it rather than trying to have the agent build a memory um and so the context you know in which you're working really matters um but this is an area where i have found like there's a lot of information that i want my agent to have and being able to say hey my calendar preferences are stored in this file, go fetch it and use that to inform it. Or creating a skill

Emilie Schario30:14

that has the preferences and the information it needs that the agent can use at that moment is a much better use of time than just being able to, than having the agent try to build

Emilie Schario30:28

its memory and then I don't have visibility into what it's collecting.

Jake Moshenko30:31

Mm-hmm. That makes sense to me. Do you have a Chad sub one or whatever that you run in a business context or is this strictly for personal work? So Chad is personal. Snivel is my work agent. Snivel. Where are these names coming from? I don't know. It seemed to make sense in the

Emilie Schario30:55

moment. And so where did Snivel come from? I don't know. But Snivel is my work agent that I mostly use as a task collector. I found that the most important thing that Snivel can help me with is make sure that I don't drop the ball on something. So over many years, I've built a system in Todoist that I know and I love and I use quite reliably. And so Snivel can be in a lot of ways, like Snivel's in Slack. He, it, it is always there. And so I can just, hey, can you make sure this gets into Todoist for Friday or whatever it might be? And it happens. And that

Emilie Schario31:39

allows me to plan my day a little bit more intentionally. So I start every day with a run down on from the agents. So when I'm waking up, I kind of have visibility into here's what I need to do. Here's where I'm double booked. And here's where I need to make sure I'm extremely effective. I don't outsource my judgment. And I think that's where people sometimes go wrong with agents, especially like these long running or more persistent workflows is like, I'm still deciding what tasks need to get done every day. I'm still the judgment of the time. But like, I don't need

Emilie Schario32:14

to be the person who's looking at my calendar and saying, okay, at three o'clock, I'm meeting with

Jake Moshenko32:19

Jake to record this podcast, you know? Okay. So does Nibble have access to like any corporate

Emilie Schario32:27

data or email inbox or anything like that? Our policy at Kilo is that your agent can either have access to internal data or external comms, not both. And that's been one of the ways we've really drawn really solid lines. So in my case, Snivel is in Slack and that's really great. Snivel has access to some emails that I forward it, but it doesn't have access to my whole email inbox. And that's been one of the ways that I've towed the line of what feels good in this world where I try to be really careful about what data we're giving agents.

Jake Moshenko33:07

Well, you're definitely speaking my language. When I, you know, I wrote about my experience with OpenClaw and it was like, you're not getting access to anything by default. So you'll get your own Google account. You'll interact with it the exact same way like an admin would, right? You'll create events. You'll invite me to the events and mark yourself as not required. I'll share documents with you that I want you to have access with, those kinds of things. Um, where are, where do you see outside of the, I guess the line that you've already drawn with internal comms and external facing messaging, where do you see the, um, where do you see teams making mistakes in this area? Where are they giving agents too much power or too much data?

Emilie Schario33:48

I think the biggest trap is exactly what you laid out. It's like giving access to my email inbox or giving my Gmail credentials over to an agent, whether it's a work or a personal context, like

Emilie Schario34:00

that's wild this thing can send emails you and and you don't know who's sending you malicious emails like the agents are still subject to prompt injection and um an email that hey can you can you just uh wire transfer me like five hundred thousand dollars real fast like uh there there are so many

Emilie Schario34:22

things that could go wrong and i think making sure agents have their own identity is important not just for security, but also for logging, like for the audit trail. I don't know how many times at work a Google doc changes, some copy changes on the marketing website. And you're like, oh, let me see the version history or the Git history here to understand who made that change so I could understand the why. Like we want to do the same thing with agents. I mean, even in code,

Emilie Schario34:48

when you're prompting an agent to write code for you, I think we should still look at those, that get version control and say how much of this code was written by an agent versus how much was written by Igor. And that's important because like I say all the time, if your name is on it, if the PR is authored by you, you're putting it up for review, you are the first person responsible for it. But when things go wrong and someone gets paged in the middle of the night, the way you're going to react if you wrote it by hand, old-fashioned, versus if you wrote it, if you prompted an agent to write it, it's going to be different. And so it is valuable, and this is

Emilie Schario35:29

something we're working on, is being able to showcase, you know, yes, this code was put up in a PR by Jake, but 90% of it was prompted by an agent, and here are the parts that he actually

Jake Moshenko35:43

row. Yeah, I totally agree. I think we need to be honest about who's writing the code. And I think we also need to be honest about why, like what are the props, where, where did the code come from? Why, why does it exist? All of the things that you're saying. Do you think that there's anything that we'll always want a human in the loop for? I mean, outside of launching the nukes?

Emilie Schario36:03

I think so, but I think it's also possible that agents become much smarter over the next five and 10 years, in my opinion, on that might change. I was on stage with the head of product at Replit last week, and he shared that PRs now, unless they touch things like auth, billing, these very sensitive areas, if an agent approves it, it can be auto-merged. And I thought that was wild. Just for anyone listening, please think about your compliance requirements before you change your workflows. But yeah, I think companies are trying to adapt to where the new bottlenecks

Emilie Schario36:48

are and something's going to have to change. Okay. All right. Well, we're running a little short on time. What are you most excited about right now? I think this is a great time to be an

Emilie Schario37:03

AI. And, you know, for you and me who have been knee deep in it for, I don't know, years at this point, it can be, it can feel like we're like, you know, deep in AI. But I just started putting together, running a workshop with the local startup incubator in town. And yesterday was our first workshop. We had like 35 people there who are just getting started using AI. Like this was a no prior skills required. We were talking about the basics, like what is chat GPT? What is a lab? What is a model? What is the software? Really getting down into it. And so what I'm

Emilie Schario37:47

really excited about is this long tail of people who still have yet to learn how to leverage these tools and bringing them along. Do you think that we're like, as a country, doing ourselves a

Jake Moshenko37:58

disservice with the sort of fear-mongering that we do around AI? There are people a lot smarter

Emilie Schario38:07

than me thinking about that problem, and I'm not sure. I think there's a lot more education that

Emilie Schario38:13

needs to be done, and it's probably one of the hardest parts and one of the biggest disconnects is that the people making decisions don't understand the industry strong enough.

Jake Moshenko38:23

Yes, yes. I heard recently that the general public's approach toward AI in China is vastly different from the general public's approach in the United States. It's very much more pervasive in everyday life for most people. And I think it's probably related to the fact that they don't have their equivalent of Dario saying, this thing is going to end the world. So I think that that's a

Emilie Schario38:50

big, yeah. Maybe, but also I don't think most people have heard of Dario Amadei saying bad things about AI. I just think they are scared for their jobs and worried about putting food on their table. And they hear things about what it does for the environment and they don't know or understand. And, um, I think those, they see their electric bills going up. Like, I think people's concerns are valid, but it comes from not understanding what the solution to these problems are. And I think Ben Thompson is doing a lot of really good thinking here. And I'd point people towards his writing. Okay. Yeah. Cool. All right. I have one last question for you.

Jake Moshenko39:29

Yes. And I think I might know the answer already, but are you an AI doomer or are you an optimist? Are you a futurist around where this is all heading? I'm not an extreme in either case. Like I'm not a AI is going to save the world or AI is going to kill the world. I try to think myself as like a practical person. AI is a tool that's going to fundamentally change how we work, how we live every day. And I'm doing my best to bring it to as many people as I can and give them the tools

Emilie Schario39:58

and the knowledge that they need to help usher it in. I think if we can be proactive about shaping it, we're going to be in a much better place and people are going to be able to have better conversations. Like so much of the disconnect I really, truly feel comes from the fact that individuals don't understand, most people don't understand AI. And I don't think I should be up here deciding what that means for them. We have to bring people along if we're going to see the

Jake Moshenko40:25

thriving AI industry that I know we can have. And is that, I normally ask, like I have kids who are in Gen Alpha, how would you paint a picture of the future for them? It sounds like

Emilie Schario40:37

that is how you paint the picture, right? Yeah. I mean, certainly positive. Yeah. Yeah. My kids are, are much younger than yours. Um, they're one, three and five right now, but I, uh, I'm excited for them that they're not, they're not using AI day to day. Um, but sometimes I'll say like, Oh, I don't know. Let's ask chat GPT and we can have that conversation together. So.

Jake Moshenko40:59

Okay. Awesome. Well, thank you so much, uh, for joining today. If people want to hear more about

Jake Moshenko41:05

what you have to say on these topics and other topics? Is there anywhere they should reach out

Emilie Schario41:09

to you or follow you or what? Find me on LinkedIn. My first name is spelled uniquely E-M-I-L-I-E Sherio, S-C-H-A-R-I-O, where I'm writing about AI and practical use cases of AI a lot, especially how I use AI in parenting. And then I have a substack, emily, spelled the same way, dot substack.com, called Playbooks and Priorities, where I'm writing about tech and parenting,

Jake Moshenko41:34

working and working pair head. So. Awesome. Once again, thank you so much for joining. Uh, I learned a ton. I hope anybody listening is also learning a lot and I hope you have a great rest of your day

Emilie Schario41:45

and rest of your week. Thanks for having me. Yeah. Bye.