Over 120 girl geeks joined networking and talks at the sold-out Sierra Girl Geek Dinner to learn from women working in AI at Sierra.
Speakers shared insights on thriving in the AI-forward era and moving toward roles that inspire a bit of “fear” as the pace of strategic change is more intense than deadline crunch. As the cost of building software drops, the value of “knowing what to build” (product sense) increases, reflects Beth Nakamura (Agent Engineering Lead at Sierra).
The boundaries between disciplines are blurring as AI tools like Claude Code and Ghostwriter allow non-engineers to build functional prototypes.
- Welcome: Girl Geek X Founder Angie Chang & CTO Sukrutha Bhadouria & Sierra Sales Engineer Kelly Kitagawa
- Introduction to Sierra: Building the Future of AI with Sierra Sales Engineer Kelly Kitagawa
- How to Interview at Sierra: Recruiting Insights with Sierra Recruiter Alice Alvarez
- Connect with Sierra Recruiting Chat Agent
- Tech Talk: The Evolution of AI Memory with Sierra Product Manager Linden Schrage
- Leadership Panel: Meet Sierra Agent Engineering Lead Beth Nakamura, Sierra Product Manager Sachi Shah, Sierra VP of Sales Erin Rudwall, moderated by Sierra Sales Engineer Kelly Kitagawa
Transcript of Sierra Girl Geek Dinner – Lightning Talks:
Kelly Kitagawa: I love seeing a group of badass women just talking and learning and asking really hard questions together.
Linden Schrage: And today I want to talk about how for the first time in history AI is unlocking personalization at scale through memory.
Beth Nakamura: It’s just changing at a rate that I think has been truly incredible. The cost of building is going down and the value of knowing what to build is going up.
Audience Member: Or are you reevaluating which model gets priority?
Kimberly Patron: Fantastic question. In fact, our own employees ask that question. This is a really great chance to interact with an agent that we build.
Sachi Shah: I use the example of Ghostwriter because no one asked us to build Gosha. And if we’d had a six month roadmap that was completely fixed, we would have picked up on the momentum of look at how Codex and Opus have changed the landscapes.
Erin Rudwall: The ways I feel most fulfilled professionally, it’s not so much the title, but the impact that I’m able to have in that role.
Kelly Kitagawa: It’s more important than ever to have these events to show up and also to show who’s actually building in this industry.
Beth Nakamura: Everyone wants to use this, but we want to use it well and it’s not clear yet what will be sustainable.
Angie Chang welcomes the sold-out crowd to our Sierra Girl Geek Dinner in San Francisco! (Watch on YouTube)
Angie Chang: Hi, welcome everyone. My name is Angie Chang, the founder of Girl Geek X. We’ve been organizing Girl Geek Dinners here in the Bay Area for many years, so I want to say thank you for coming back. And if you’re new here, thank you for coming to a first Girl Geek Dinner.
Angie Chang: I really love hosting them and really appreciate all the companies that are supporting women in tech and sponsoring them and inviting us into their office so that we can enjoy some food, conversation, talk to each other, as well as people who work here and learn about all the exciting roles here at Sierra. And now I want to introduce my co-organizer, Sakrutha.
Sukrutha Bhadouria: Hi, everyone. There’s still some seats up here, by the way, for those of you who are figuring this out, but there’s a lot of spots and there’s spots down here too. So yeah, hi, I’m Sukrutha.
Sukrutha Bhadouria: Thank you all for coming and thank you to Kelly for making this happen. We’ve been starved of in-person networking events. I’ll tell you a little bit of history about Girl Geek X, even though Angie has been doing it forever. I’ll tell you the story. Basically, Angie was looking around. I’m going to talk about you as if you’re not here.
Sukrutha Bhadouria: Angie was walking down the street and she was like, “What if I created this environment for women to get a sneak peek into companies and learn about what the work-life balance is, what the work culture is and what kind of cool work that they do through speakers who happen to be women, who are not just talking about what it’s like to be a woman in tech, but actually talking about the cool work that they’re doing.” And so she got Google to sponsor and overnight over 400 people signed up that should just tell you how starved everyone was of events like this.
Sukrutha Bhadouria: And this was by the way, back in 2008. Cut to a few years later, I just moved to the Bay Area and I was like, “Gosh, I need to meet more cool people. ” Because I was new to the city and then I found out that it was called Bay Area Girl Geek Dinners back then. Angie had gotten the most recent event that had happened was at Genentech. Genentech had sponsored the dinner and the model Angie had was really cool because it was like, let’s do the event at the company. They sponsored it. They showcased their amazing female employees and they all talk about the cool work that they’re doing. And then we will provide the audience and the attendees. And this was like started, like I said, with that event with 400 odd people. I saw that Genentech had just had one.
Sukrutha Bhadouria: The dinner tables had centerpieces made out of test tubes and cool things and they had fun games. So I reached out and I was like, “Angie, I want to help you. ” Then it just took off after that. And then 10 years later, Angie had this genius idea to rebrand and call it Girl Geek X. Why was X added, Angie? The 10th year. Yeah. The 10th year and then X is a variable. So then we started to do podcasts. You can still find us on Spotify or any podcast tool of your choice. We thought about webinars.
Sukrutha Bhadouria: We created the virtual conference, which is called ELEVATE – it’s free, so we have people joining from all over the world. It’s very thrilling. But I think the most important thing was that we started to see our membership base grow like crazy. At one point we were on Wall Street Journal, or featured on Wall Street Journal, and they called us the toughest tickets to get in Silicon Valley.
Sukrutha Bhadouria: We were covered all over the media. Even my parents were reading about it in Economic Times. It was just like a wild, wild journey. Obviously when the pandemic hit, that put a screeching halt to in- person events. I’m really glad that this is picking back up because at one point we were booked up for a year and a half in advance of an event every single week so much so that by the time the event came, the person we were working with had invariably changed jobs. That’s like how long the waiting time was. But I’m glad this is happening. Thank you so much and I’m most excited to hear about the amazing talks that the cool speakers are going to talk about. So thank you. Thank you, Kelly.
Kelly Kitagawa: Thank you. Actually, fun fact, the first Girl Geek Dinner that I went to was in 2010 and it was so impactful to me and I loved it so much, hence me doing these and this is the second one that I’ve hosted at a company that I’ve worked for.
Kelly Kitagawa: I always like to say why I like putting these on. One, it’s selfish reasons that I love seeing a group of badass women just talking and learning and asking really hard questions together because for many of us that work in tech, representation is still tough and especially for women of color and other diversities as well. I always think you can never get enough of it.
Kelly Kitagawa: I really believe that if you can’t see it, you can’t be it. And so these events are so important for the younger folks coming out of college, but also just to, especially with these political times where diversity is a bad word as well, it’s more important than ever to have these events to show up and also to show who’s actually building in this industry and that it’s not just certain profile of people.
Kelly Kitagawa: And the one way that we do that is to get to show off and really highlight women that are building in this industry. And of course with AI, which is one of the most consequential times that for people in tech that this is, right? It’s one of the fastest growing, biggest, most impactful, but also like low representation for women and people of other diversity, so it is more important than ever that our voices are heard and that we take up space and that we are building all of us here and experimenting and sharing and supporting each other. That’s why I love doing these.
Kelly Kitagawa: Thank you so much for investing your time, especially post- COVID coming to networking events can feel a little like vomit inducing. And so I’m just very grateful for all of you taking the time to be here. It means a lot to me, but also it’s so wonderful that Sierra gets to host you all, so please enjoy the time.
Kelly Kitagawa: In terms of our agenda, you guys all could read. We’ll do kickoff. We have three lightning talks as you saw on the Eventbrite page and then we’ll finish with a panel. And I just wanted to talk two minutes about what Sierra does. I joined Sierra about four months ago and not everyone really knows what we do, including myself at the time. I thought I would just really briefly talk about what Sierra does.
Kelly Kitagawa: We were co-founded by Bret Taylor, who’s the chairman of the board of OpenAI. So he has quite the side hustle as we say. And then Clay Bavor, who grew his career all extensively at Google and ran VR and labs there as well. We have some really great industry titans that are very familiar with the enterprise space and our mission is to build better, more human experiences with AI, which I think is such a fascinating intersection when we just hear about AI being foundational models like OpenAI and Anthropic, but we are an applied AI company.
Kelly Kitagawa: What that means is that you don’t talk about LLMs when we talk about like customers getting value from it. And so I think that you’re going to see a lot more surgeons of more applied AI companies and we’re one of them. And then I think this part is interesting too from like a history of time perspective. Bret and Clay basically started this company because they’re seeing similar patterns in the industry.
Kelly Kitagawa: In 1995, the way that customers were to find your brand or your companies, you had to have a website. And then fast forward to 2015, mobile apps were the same thing, right? Most of you interact with the technology today through mobile apps and now 2025, we’ve hit this inflection point and now in 2026 we believe that every brand and every product will have an AI agent. You’ll see more of this as we go through.
Kelly Kitagawa: And lastly, when we think about AI agents, it’s such a vast and broad spectrum. And what we really specialize in is customer experience, helping you figure out where your order is, any kind of billing questions. Nordstrom’s is a great one of like, where is my order? Is it delayed?
Kelly Kitagawa: Another interesting amount of rocket mortgages, we help customers with the intake process. We have an AI agent doing all of the intake boring intake process and then we hand it over to an in- person financial professional. And so with things like that, they said that it’s four times the conversion rate to get a customer from a lead where you have AI doing a lot of the legwork and then the humans really handling the complex tasks. Really interesting use cases. And I would love to talk more about Sierra at any time, so come find us. There’s a bunch of other sales engineers here too, or any of us that work at Sierra, but we thought it’d be great to just talk a little bit about our recruiting for any of you that are on the job hunt now and I’m going to hand it off to Alice.
Alice Alvarez: Hi everyone. I’m Alice and as Kelly mentioned, I’m on the recruiting team here at Sierra. I’m actually joined by two of my colleagues, Eleni and Alishan. So if you have any questions about recruiting after, feel free to find us. We’ll have name fabs. There’s Eleni there. Yeah, just a quick overview. I’m here to talk through open roles, teams that we’re hiring for best practices. It can be nerve-racking to interview, so here to shed some light on how to ace your interviews and keep up with what we’re up to. We built recruiting agents, so excited for you all to interact with that.
Alice Alvarez: First is just an overview of the teams that we’re hiring for right now. Obviously we’re going kind of across the board, but these are the main teams that we’re really focused on. Agent development is one of the teams that I actually help hire for. This is a team that focuses on scoping, building, voice chat, agentic solutions. It’s more than just building. It’s a lot of like crafting the logics, workflows, and getting really deep into behaviors and making our agents outcome-based. In terms of the types of roles.
Kelly Kitagawa:
Also, Alice built the agent for today and I was her first agent, so give her a shout out. Okay.
Kimberly Patron: Thank you. Feel free to give me feedback afterwards. But yeah, anyways, so in regards to agent development, we’re hiring across the board. Engineers, product managers, strategists, if you want to know what a strategist is, come find me. I’m happy to share more details. With the platform team, so this is really the team that develops data systems, infrastructure, all the tooling that we use to build agents. So similarly here across the board, engineers, designers, and product managers.
Alice Alvarez: And finally, sales wouldn’t be able to do it without our incredible sales team. This is a team that works closely with our customers to shape business outcome, find solutions, and really drive high impact AI adoption. And the teams that we’re hiring for are sales engineers and enterprise AEs.
Alice Alvarez: Interviewing at Sierra, definitely lots of tips and tricks to share. This is just a general high level overview. As is with any interview, we’re looking to assess your ability to drive adoption, excellence, and your character, so when it comes to best practices, I always recommend leading with a clear structure, being really thoughtful about how you approach your answers.
Alice Alvarez: I don’t know if you’ve heard of the STAR method, but if not, it is situation task, action results. Result is a big one and just taking a methodical approach, really laying out your information, your answers, and sharing your interviewer is following along and being reflective.
Alice Alvarez: I think that is a key tenant to being successful here at Sierra is reflecting on the challenges you’ve helped solve or feedback and taking that and applying that to future situations and reflecting on that as well. Connecting your work to Impact, we’re still a fairly lean startup building for impact.
Alice Alvarez: There’s a lot of opportunities to build here, so it’s just important to highlight those areas and bonus points, but I always think it’s important to come with research prepared, review our values, come prepared with questions. It’s just as much as important it is for us to get to know you as it is for you to get to know us.
Alice Alvarez: Generally, when it comes to interviewing, we do frame it from two different types of interviews. We’ve got behavioral situational questions. This is really framed around your experience, how you might handle ambiguity or situational questions and really just grounding yourself in concrete examples, scope and measurable outcomes.
Alice Alvarez: When it comes to the technical side, that will obviously depend upon the team that you’re interviewing for, but generally knowing how to dive deep in on solving technical challenges, coming prepared, think out loud, that’s a big one, just walking through your approach.
Alice Alvarez: And then finally, this is our agent. You’re welcome to take out your phones if you like. We also have flyers throughout after, but this is a really great chance to interact with an agent that we build. Great opportunity for you to share what types of roles you’re looking for, your background, and also a great way for us to follow up with you after. Thank you.
Kelly Kitagawa:
Thank you. And again, that was Alice’s first agent, so please check it out. All right, we’re going to move over to the Lightning Talks. We’ll have about 10 minutes of presentation, then feel free to ask Q&A for about two or three minutes after the talk as well so you don’t have to wait till the end. So without further ado, I’m going to introduce Linden. Welcome.
Linden Schrage: Getting situated. Okay, thank you. I actually brought a prop. Hi, everybody. You might have seen this yellow book on the way to the auditorium. It’s Unreasonable Hospitality by Will Guidara. Guidara is the famous owner of the restaurant, 11 Madison Park, and he’s also an advisor to Sierra. His whole idea is that hospitality isn’t about being fancy or efficient. It’s about specific personal acts of care that makes someone feel seen.
Linden Schrage:The most famous example in the book is he talks about a couple that goes to his restaurant. He overhears that they mention they’re on their trip to New York and they haven’t been able to try a New York City street hotdog so he runs out, he buys the hotdog, the Michelin staff plates it up and they serve it to the couple and that’s the whole idea. Unreasonable hospitality is personalization to its extreme and for most of business history, that kind of thing has been economically impossible at scale.
Linden Schrage: Either you hire an army of people who can understand each individual customer or you’re shipping a general product to a general audience. And when you think about what separates good customer experience from a bad one, the answer almost always comes down to personalization. Does the agent know who you are? Does it know what you want and what you’re trying to do?
Linden Schrage: Today I want to talk about how for the first time in history AI is unlocking personalization at scale through memory. My name’s Linden. I’m a product manager on the agent memory team at Sierra. I joined Sierra as part of their APX program, which is a rotational program for engineering and product management. For the past few months, I’ve been thinking about how we use personalization to create delightful AI experiences. Oh, forgive me. Well, this is my first slide and I’ll go to the next.
Linden Schrage: Most of you probably have an intuition for AI memory and consumer products. I think what’s really interesting here though is that the two biggest players in the AI consumer space have taken opposing approaches to how they’ve implemented memory. ChatGPT’s memory is generous and automatic. Every conversation starts preloaded with context from your past chats. The design philosophy is the user shouldn’t have to think about memory at all.
Linden Schrage: Claude takes the opposite approach. The memory is precise and invoked. Every conversation starts blank and then when you say, remember when we talked about X to the agent, Claude searches through your raw chat history, pulls in relevant bits and exposes that in the UI. The takeaway here is that memory isn’t one size fits all. How you build the product depends on what you’re trying to get out of it.
Linden Schrage: It’s also really easy to make memory bad. An OpenAI co-founder recently had a tweet that really struck me. He said, “One common issue with personalization in all LLMs is how distracting memory seems to be for the models. And I think anyone who has used ChatGPT or Claude has had an experience like this. I off-handedly mentioned in one conversation with ChatGPT how I had an allergy to peanuts and now anytime I reference food at all, it injects that into the conversation in a way that is weird and off-putting.
Linden Schrage: Memory really is this two-edged thing. On the one hand, it’s the basis for unreasonable hospitality and on the other end it can be really creepy and really distracting, and so most of the interesting product work is in how we bridge that gap.
Linden Schrage: Imagine you’re calling an airline because your flight got canceled. You spend 15 minutes on the line with the agent, you’re finally getting somewhere, you’re about to rebook your flight, you’ve mentioned that you’re traveling with your cat and then the phone call drops. You call back a little while later and the question becomes, what should the agent remember? A live agent would remember nothing. A live agent would start the conversation from scratch, but an AI agent has the ability to remember. In this scenario, let’s walk through three problems that make enterprise memory harder than it might seem.
Linden Schrage: The first is where the memory is coming from. In ChatGPT, everything the model knows about you is from information you’ve provided yourself. This is first-party data. In enterprise, first-party data is just the tip of the iceberg. Sure, the agent needs to know that you have a pet because you mentioned it in your past conversation, but the airline also already knew that you’re platinum and that you fly from JFK to LAX every month because they’re pulling from their personal booking management systems.
Linden Schrage: This is third party data and in enterprise customers, this is often the richer data source. An enterprise agent needs to be able to gracefully handle both. Second, memory isn’t one kind of thing. Some of it is structured like your status, your preferred seat, your in progress booking, but some of the most important memory is also unstructured. What was the sentiment on the last call? What were the troubleshooting issues that you took to get the issue resolved?
Linden Schrage: Structured memory is what lets the agent act. Unstructured memory is what gives the agent context to act. In an enterprise agent, again, you need both. Third is when do you surface any of it? This is a Karpathi problem from earlier. You can have all the right signal and still ruin the customer experience by dumping it all at the start of the conversation. No one wants an agent that starts the call with, “Hi, I see your platinum. I see you’re gluten-free. I see you have a cat. How can I help you with your problem today?
Linden Schrage: What you want is each of these showing up where it’s relevant. Platinum, when it affects your rebook priority, gluten-free when you’re confirming a meal and the fact that you have a cat when you ask about pet travel options. But memory is how the agent knows you. On its own, it’s passive. The next layer is the agent doing something because of what it knows about you proactively. Because you travel JFK to LAX every month and you always pick a window seat and you always order the gluten-free option, the agent shouldn’t have to wait for you to call.
Linden Schrage: It can reach out a week before your trip with a message, “Hey, I’ve held seat 12A for you. I’ve pre-ordered your gluten-free meal and here’s an upgrade that we think you might like. Say yes to confirm. That’s not memory anymore. That’s intelligence.
Linden Schrage: Memory is what the agent knows about you. Intelligence is what it does because of it. In the next few years, every company is going to have an AI agent. The ones that win aren’t going to be on the fanciest models. They’re the ones that figure out how to use what they know about you to show up better and this is what we’re building at Sierra. I don’t know if I have time now, but I’d be happy to take any questions about memory here or afterwards.
Audience Member: Thanks. Thank you. This was great. I have a quick question. How do you guys handle cold start use cases where there is limited or no data on the 1P or 3P side?
Linden Schrage: Yeah, that’s a great question. It’s something we think a lot about because in the case of enterprise agents, a lot of your contacts are the first time they’re contacting the platform. And so in those cases, we look for third party data sources. Even if they haven’t engaged with the agent before, we might have a pool of information sitting on that customer. We also look for similarities between different industries. You can learn a lot about what a customer might want based on what similar profiles have done. I would say we kind of take a two-step approach there.
Audience Member: I’m curious about the differences between B2C and B2B interactions and data and memory and intelligence there, mainly because you mentioned OpenAI, which is B2C primarily, just like Google and then Anthropic, which is B2B. Did I say B2C initially? Absolutely. Yeah. B2C and there are different data sets and you would think that there’d be a wealth of data just to piggyback off of the first question. There’d be a wealth of data on the enterprise side, but I’m curious about how you think about that and what are the leverage points for quality in intelligence and delivery given whether or not it’s a B2B or a B2C model.
Linden Schrage: That’s a great question and something we think a lot about. Like I mentioned, I think it’s common in industry to think about memory in terms of B2C because most people working on memory agents are working or thinking about the ChatGPT, the anthropic model. But in B2C, we really do have a different paradigm because we’re operating with different data sources. Not only do we have that first party information, but we have much less of it because often the recontact rates are low. I’d say the focus for a B2B model like Sierra is to think about leveraging third party data in interesting ways.
Kelly Kitagawa:
Okay. Last one.
Audience Member:
Mine’s a super quick one. Thank you by the way for talking us through that. I think memory’s super interesting. I’m just curious, how do you on the memory team think about measuring success?
Linden Schrage:
That’s another great question. We think of success in terms of the customer experience, but of course we have secondary metrics that measure that. All of our agents have a containment rate. All of our agents have a greeting acceptance rate. How quickly did a user ask like human agent, human agent. And so you can see, you can compare in the conversations where we use memory, where we leverage our first and third party memory systems, you see the metrics that matter to the agent improve. The way that we think about memory success is in terms of agent success, which ultimately is a reflection of customer experience success.
Kelly Kitagawa: All right. Well, round of applause for Linden, please. All right. Okay. So at this point we are going to finish with a panel so if my panelists could come up here. Wonderful. Okay. This is usually my favorite because we get to ask all the juicy real questions and so I always love to start a panel with just introductions, but we want the introduction to be not the polished LinkedIn version. We want to hear the pivots, the pitfalls in 90 seconds. So Sachi, why don’t you go first?
Sachi Shah: Hi everyone. I’m Sachi. Thank you. Hi everyone. I’m Sachi. I work on the product management team at Sierra. I have been at the company for a year, year and a half, which in the company’s history is quite a long time, not as long as Beth has been here though. I started my career within engineering. I worked at a small company, mostly on the backend side of things for about a year.
Sachi Shah: From there, I went on to join the Forward Deployed Engineering team for several years at Palantir and obviously working with large enterprises within specific industries, and it’s kind of nice to see a lot of the similarities and the way we operate with what I saw 10, 15 years ago at Palantir. From there, I kind of moved on. I went to business school, tried my hand to starting my own company while I was there and then eventually moved into product roles.
Sachi Shah: I spent the last six years or so within developer tools. I worked in an observability company that was acquired by ServiceNow within products, spent a couple of years at ServiceNow after that as well. And then I led product at an application security company. A couple of reasons why I joined here. One is I had a bunch of former coworkers who work at the company, which just for me spoke a lot to the talent par at the company and the culture at the company as well and is what reeled me in. And secondly, I think once I started meeting more and more people at the company and actually thinking about the product, listening to talks like these, I was fascinated by the underlying technology and the pace of change and coming from kind of the platform building side where I was like working on products that help you build really reliable software and software at scale and testing that software.
Sachi Shah: I was really excited to start thinking about the parallels in agent development and agent who’s a new kind of software. It’s a new kind of production software. How do you think about building it? How do you think about putting it in the hands of the right users, developing it at scale, testing it rigorously and so on. Those are some of the challenges I work on at Sierra as well.
Kelly Kitagawa: Wow, you went through a lot in 90 seconds. Hold on. All right, Erin, who’s a VP of Sales. We’d love to hear your backstory.
Erin Rudwall: Hi, everyone. Erin. I lead one of our go-to-market teams here at Sierra. I did not start my career in tech. I majored in economics and actually started my career working at a hedge fund in a client-facing role. Loved the client-facing part, did not like working in the hedge fund industry and wanted to pivot to something more dynamic and fluid, so a pivot, I took a step back in terms of responsibility, in terms of comp, in terms of role, and pivoted into a tech company.
Erin Rudwall: Prior to joining Sierra, I’ve been here about six months, which I think makes me old school on the sales team here, I was leading the strategic sales team at Figma. I had a pivot there as well where I actually decided to stay on the strategic side, leading that organization, shrank my reporting line by two thirds, but really felt convicted on staying in that role with a smaller remit in terms of headcount. So it’s great to meet everyone. Excited to be here.
Kelly Kitagawa: Great. And Beth, why don’t you finish us off with career journey?
Beth Nakamura: Will do. I’m Beth. I’m a Bay Area native and I went out to college out in Ann Arbor, Michigan. It was really cold, go blue. It was awesome. I will say I went out there thinking I was going to do consulting or iBanking and I found out really quickly that was likely not going to be a fit. I finished degrees in math and business there, but actually upon graduation, I realized that the biggest risk was not that I graduate unemployed, but I would graduate and do something that I did not like for 40 years, and so I actually went and did Hackbright before my senior year, which is where I met Angie way back in the day. Learned to code, did a lot of teaching myself and entered the industry into a GE software development leadership program, which was a rotational. After working at a huge company and realizing GE is awesome at building a lot of things, but is understandably not software focused.
Beth Nakamura: I actually really wanted to go to a place that was super focused and only had one product. I went to Lyft. I was there for five years. I was a staff engineer there building out in their infrastructure org, bailing out tools in cloud capacity and efficiency. As one does, at one point I did have a little bit of an existential crisis and I was like, maybe I actually do want to go back to school. I remember I signed up for the GRE and it just so happened I took the GRE the same day that Lyft IPOs. I remember I went, I took the exam and I went back to the office and drank champagne with my coworkers. All in all, a great day. I ended up going and getting my MBA at Stanford GSB. That is where I met Natalie Muir, who was my co-founder.
Beth Nakamura: We did a Web3 startup for a couple of years after school and we decided to turn that down at the end of 2023. I took a little bit of a break and Natalie was pretty immediately like, “I just interviewed at this amazing place. You should come. It’s everything you’re looking for. It’s Bret Taylor’s company.” I said, “I’m not sure who Bret Taylor is, but do you like it? ” And she’s like, “Yes, it’s amazing.” And so I came and interviewed and I agreed with her.
Beth Nakamura: The thing that really stuck out to me was like I wanted to be at a place that was going to be opinionated about how AI was actually going to be in the world. As soon as I used ChatGPT, it was so clear how it was going to change my day-to-day. I really wanted to be at a place where I could figure out how is it going to change my grandmother’s day-to-day, if at all.
Beth Nakamura: And then the other thing is I wanted in particular to be in a role where I could build product, grow as a software engineer and see what it looked like to actually execute on new customer segments because that go to market cycle I was really, really terrible at as a co-founder. And so agent development at Sierra was sort of the perfect place for me. Been here for a couple of years and I’ll let Kelly kick us off.
Kelly Kitagawa: All right, thanks. Very diverse backgrounds from different walks of past, but I think also one of the common strings between the three is that all of you have led large teams. All of you have had very fancy roles and titles too. And coming to Sierra, somebody even asked me, “Do you guys even have managers at Sierra?”
Kelly Kitagawa: We have a very flat org structure and I thought maybe Erin, you could start us off with why would you leave your high ranking large organization position that arguably was very hard to get to and why would you all leave that for a small startup in a flat org again? Pros and cons.
Erin Rudwall: Yeah, I would say there’s a lot of ways to be a leader outside of a title. And when I think about the ways I feel most fulfilled professionally, it’s not so much the title but the impact that I’m able to have in that role. This is a frontier technology so from a go-to-market lens, most people haven’t sold this yet. We don’t really know how to do it and so starting from the ground up made a lot of sense to me personally. But I think my personal career path, my philosophy, it’s more damaging to be at the wrong company than to take a lesser title. And you want to be somewhere where you feel motivated and connected to the work and Sierra was that for me. And I think it’s a place where title doesn’t matter. You have a huge impact in your day-to-day and I think every single person at this company has a huge impact.
Kelly Kitagawa: Would you think you would have said that same answer five years ago or 10 years ago or did you feel like you needed to get to a certain part in order to feel like, oh.
Erin Rudwall: I think this is my second time taking a step down in terms of title and the first time I have zero regrets doing it so maybe I think I probably would have, but that’s maybe unique to pivoting out of the hedge fund industry as well.
Kelly Kitagawa: What about you, Beth? Any thoughts on …
Beth Nakamura: Yeah, this might be a very engineer’s take, but hardly more prestige than being a rockstar IC. Kim is living proof of that. I will say for me, it was really about getting to grow skills and build in a space that I felt just was so important, truly. I honestly, I knew AI was going to change everything. I’m not the biggest model head. I have friends that are, but I was less interested in kind of that research and particularly with my background so many places were interested in me working in an info and capacity and I really wanted to figure out how this was going to actually solve real problems in the world. And so it was much more about being at a place that would let me do that, being surrounded by people who had that same curiosity and were willing to jump in. I will say Sierra is growing fast.
Beth Nakamura: I’m now leading a team that is larger than the company was when I joined. So all in good time if management is something that you want to make sure to get experience in.
Kelly Kitagawa: Very fair. On that same note, Sachi, I wanted to ask you this as well, because you also came from a much higher position, but you have a really good perspective on how Sierra operates different than some of the other companies you’ve been at, large, small, and also started your own as well. I’d love to hear your thoughts on how is Sierra different than some of the other companies you’ve seen and pros and cons.
Sachi Shah: It’s vastly different in so many ways and still fundamentally the same in other ways as well. A few ways in which I think things are quite different is no surprise. Everyone is shipping stuff really, really fast now because the technology actually enables us to get there. With that in mind, the way we think about product is also just a lot more nimble, I would say.
Sachi Shah: This is the only company I’ve been at that there’s no form of dedicated planning at specific points in time in the quarter. Even the smaller companies I was at, we would have some kind of planning cadence. And it’s not that no one knows what they’re doing and people are just constantly building. There are substitutes for these kinds of things. I was just talking with someone else earlier. I was like, when you’re shipping this quickly, think about like if you’re driving at 100 miles an hour, you need to look further ahead and know where you’re going, otherwise it’s just like one wrong turn and you’re kind of like off on the side pretty much.
Sachi Shah: What we do instead of these fixed planning cycles that timing doesn’t really make sense for a lot of teams and all of these things is we just try and have visions and the main problems you want to go after per product area. For me, it’s like, what is my vision around what collaboration should look like if there’s hundreds of people trying to build an agent in our product? What’s my vision for the next thing? How do you actually release these agents at scale and so on and so forth? And then I think you’re just like constantly getting customer input. You’re learning from the technology, you’re learning from the market and it’s not just kind of like customers ask for something. The way they’re interacting with AI, their concerns with AI might be super different today than they were three months ago and it’s like might delay from like one year ago to now.
Sachi Shah: And so you have to keep all of that in mind and maybe to just give like a concrete example, one of the products that we launched earlier this year is something called Ghostwriter, which is an agent that lets you build agents. You can build an entire agent using natural language. Of course, you want to kind of review the output and so on, just like you use Codex or CloudCode, you’re not going to be someone who’s never seen a line of code building a way and building a production piece of software, reading authentication. You kind of want to know what’s coming out and use it as kind of like this assistant that’s helping you and expediting you and giving you a lot of leverage.
Sachi Shah: I use the example of Ghostwriter because no one asked us to build Ghostwriter and if we’d had a six month roadmap that was completely fixed, we wouldn’t have picked up on the momentum of like, look at how Codex and Opus have changed kind of the landscape, the advancements at least on December, the power of harnesses and so on. And because we saw all of that happening, we were able to kind of think through what does a harness look like for building agents within the Sierra platform as well. If you were sticking to our regular roadmap, we would have had wait another four months to actually pick up the next thing, right?
Kelly Kitagawa: Yeah. And you brought up such a good point about like Claude Code and Codex in the last six months completely transforming our industry. How many of you all have used Claude Code or Codex? Yeah, incredible.
Kelly Kitagawa: And how many of you have built your own app at some point, right? And how many of you all have had near zero coding experience before that? That’s incredible, right? That’s so incredibly powerful. Also love that you guys are all building, but on that note too, I mean, the nimbleness is obviously required with how fast the industry is changing.
Kelly Kitagawa: I don’t know if you have any perspectives on all of you as well, what it’s like working in the AI industry compared to other industries. And I think from a technology standpoint, you have a really great example, but curious from you two as well, how is the AI industry different than what you’ve seen before?
Beth Nakamura: Yeah, it’s just changing at a rate that I think has been truly incredible. The way I characterize it, particularly in my role is the cost of building is going down and the value of knowing what to build is going up. And so the ability is to spend time directly with customers or as Saji said, very, very nimble to trust your instinct and to test what verify you can actually build. I think that comes with some other …
Beth Nakamura: We’re going to see a change in coordination costs already we’ve seen a number of great ideas and part of what Ghostwriter was, was to productize the best practices that we knew internally. And so that’s what I’ve seen. And I think in particular, one of the changes in terms of some of the intensity we’re seeing in an industry is with the advances coming from so many players in the broader industry is you’re going from not just internal crunch, which is, oh, we have this roadmap, these deadlines, but also strategic crunch.
Beth Nakamura: Ad like this idea, you see an announcement about Cursor and XAI and you’re rethinking what that means for both how you are thinking about coding, how your organization is thinking about coding, what that means in terms of how they’re thinking even six months from now. And that has been both incredibly exciting. It’s also been where being surrounded by people who, whether they’re model heads or just very, very deep in industry news or building has been incredibly helpful just because everything is moving so fast. I know I can’t keep up with all the pace of industry and so it’s so helpful to have the entire team, the entire company really, really getting rigorous about thinking about what’s happening, what it means, not just for us, but for our industry, our careers, our professions.
Kelly Kitagawa: Yeah, Erin, and I would love to hear from the sales perspective, buying customer perspective too, like how has it been for you and your experience?
Erin Rudwall: Yeah, I would say from the sales lens, the pace of innovation at Sierra is so rapid. It’s a highly competitive space and so I think it has really forced the bar up for salespeople to be immersed in technological advancements, be able to go levels deeper on talking to the complexities of the product. I think it’s been kind of a fun change. And then the pace of innovation, obviously when you’re customer facing and you get to deliver things like Ghostwriter to people is so fun because it’s just such a big unlock for them and how they’re thinking about their own workflows and adding value to their business.
Kelly Kitagawa: Yeah. And it’s so fascinating for someone that’s been in the business for a long time for things to feel so different, but yet the sales cycle is similar, but to your point, the way that we position, the way how fast the product is moving, it is hard to keep up and it really keeps you on your toes, I would say. But there’s something that you said, Beth, too, about since the cost of building is dropping and what do you all think, and Sachi, I’ll hand this over to you, what are the skills and roles that you feel are we should be positioning ourselves more in the room given those types of changes and rapid development and maybe engineering or coding not being as … Since it’s so much more accessible. But yeah, I would love to hear your advice on how we in the room should be positioning ourselves.
Sachi Shah: Yeah. I think with the last couple of points that we spoke about, we spoke a lot about creativity in a certain way as well, right? For me, I come from a heavy B2B background and Sierra in a lot of ways feels extremely creative in comparison and part of it is because our customers are some of the biggest consumer brands in the world. So there is this kind of consumer experience that we’re designing for as well at the end of the day. Part of it is the industry is moving so quickly and so you’re always having to keep up to date and so on. But also part of it is like the pace of change is just so rapid that you have to build that kind of intuition to actually see what’s around the corner and start building towards that. And so it comes back to some of these player coach examples and moving into IC roles or finding that builder time as well, because unless you’re actually working with the technology hands on using different kinds of products out there, getting intuition for different things.
Sachi Shah: We speak a lot in product about developing product sense and I think that’s kind of like an equivalent in the AI age where you’re using agentic products, you’re developing model sense or agent sense or whatever you want to call it in a certain way as well. So I guess my first piece of advice is if you do want to build up that intuition and build creative solutions in all of this white space that we have right now, carve out Friday afternoons, Thursday afternoons, move into a role we are getting to actually be on the ground floor building things. And also as we’re seeing organizations flatten and so on, it’s so important to have that player coach mindset even if you are managing teams. It’s kind of more of like a coach than a manager these days right now. So that I think would be my biggest piece of advice right now.
Kelly Kitagawa: Thanks. Yeah. And I think the building aspect is so much more accessible now and especially like I was just talking about how can we use Claude Code for non-engineering roles or like non-technical roles, quote unquote. But even in the sales side, there’s so many ways that you could build with the Claude Code or Codex to help you if you’re in a non-technical role. We were just talking about someone in marketing wanting to create more Slack automation and things like that.
Kelly Kitagawa: Building has never been more accessible now than ever. I loved that you said that. Do you two have any tips on like what things should people start doing with AI tools or what tools would you recommend for people to look at from helping you in your day-to-day job to upskilling yourself or anything on broad spectrum?
Beth Nakamura: I know we touched on it, but the ability to really build apps quickly that I already bespoke, I honestly think has also changed the viability of certain businesses and microservices as well, which I think has been really, really interesting to see, and so a big part of things like not just Ghostwriter, but even what I’m seeing people build is like, where are places that you have true subject matter expertise and why is that not scaling and can you build something that can help scale it?
Beth Nakamura: The other thing that I’ve seen myself lean on more and more is actually my systems design expertise because these tools are very powerful, but they’re often limited by the systems that they’re in. And so thinking about what are those right interfaces so that you can safely and securely integrate and like build across more systems, including taking as the reasoning models get better, what does that mean in terms of enclosed reasoning and where do you want the human in the loop?
Beth Nakamura: Where is the human in the loop the most leveraged? And how can you take more and more of like the places where you see subject matter expertise being really high leveraged and coupling it with AI to truly produce something new because I think the other thing I’ve seen is like, and a lot of what you’ll see publicly on our website is known inbound customer service use cases agents are going to change the way brands interact with their communities and like I’m really excited over the next couple of years it’ll be really design partnering with these flagships in industry to figure out what that looks like. And so just building out like what are your workflows, where are you realizing that like, wait a minute, I’m doing this thing, this is my side of my exercise, how do I scale myself? I think is a really, really interesting thought exercise.
Kelly Kitagawa: What about you, Erin, when you were upskilling yourself or getting your hands dirty into this industry, do you have any recommendations or tips that helped you?
Erin Rudwall: Yeah, I think of some of the AI. The most helpful moments for me using AI in my day-to-day job is I just see it as a force multiplier for me to spot patterns and trends. I sit across a team that’s working with hundreds of different accounts to evaluate Sierra and some of the most compelling things we can do in a sales cycle is actually connect the dots on how this relates to a problem that someone else has had and give a real world proof point. I can leverage AI to help me research across the entire company’s work, and so I feel like it’s given me the ability to provide better insights to my team, to our prospective customers, to our existing customers so I can see it as it’s a way that helps me scale myself across a lot of different things. On a personal note, I’m trying to get my husband to build me a summer camp scheduler with Claude Code. He’s like 50% done, but ways like that, things you can do just to scale your life.
Kelly Kitagawa: Yeah. There’s an incredible woman, Helen, who started Women Defining AI and she just, one of her inspirations of starting was like using AI to help be a better mom and how much coordination and to your points, summer camps, but it’s also like how can we hack our life? How can we use these everyday tools for whatever we need? And like I said, the accessibility has never been there more than ever so completely agree with you and love that use case. All right, have one more question and then I want you all to start thinking about what questions you want to ask the group here, but I always love to end on more of a anecdotal note, but what’s one thing that you wish that someone had told you earlier in your career? I’ll let you think and then whoever goes comes to you or what comes to you?
Erin Rudwall: Something that’s always resonated with me, if your next move doesn’t inspire a little bit of fear, it’s not the right move and always bet on yourself.
Kelly Kitagawa: I like that one.
Beth Nakamura: Yeah. Something that I feel just so, so fortunate is that in undergrad, I had such an allergic reaction to the undergrad business school that I sort of came to that realization like, oh my God, it’s an amazing feature. I was like, “It’s not for me. ” And I didn’t actually, at that time, I wasn’t really able to articulate with precision why it wasn’t for me, but I just like-
Kelly Kitagawa: Fear.
Beth Nakamura: Yeah. I was just like this- Fear. All of a sudden my new biggest fear was, what if I do something that I really don’t like for a long time? And so what I always tell folks is like, “Hey, you know when you feel it, honestly, and I think it comes in waves and you’re ready for something new.” I think a lot of times because it’s very good in professional and personal settings to be able to communicate with precision, but there’s a point when you’re like, if it’s not a fit, it’s okay to just be like, “It’s not a fit. I’m going to figure out what that is later, but I know right now it’s not a fit and I’m going to start looking at other things.” Because I remember every time that I was ready for something next, actually going out and seeing comparison points helped me articulate, “Oh, actually I do like this.
Beth Nakamura: I don’t like this. I do like this. I don’t like this.” And so that’s my recommendation is just like it is important to reflect and be able to look back because that will help you from making those same mistakes again, although I’ve done much of that too, but when you know that it’s not and you start to feel that, it’s okay to just not have to decide right away, but at least start trying to figure out what’s next.
Kelly Kitagawa: I like that one. Getting closer to your fears and understanding the better. Sachi?
Sachi Shah: For me, it’s nothing matters other than working on a problem you care about and finding inspiration from the people you work with on a day-to-day. We spend more time with our coworkers than pretty much anyone else in our lives to some extent, and so I say all of this because I think ultimately I think role, title, all of these things don’t really matter.
Sachi Shah: In fact, one thing that I really enjoy about Sierra is that the roles are so fluid. I’m sure Beth in a given day or an engineer, a PM, a salesperson, or whatever else needs to get done kind of thing. And it just doesn’t matter as long as it’s a problem that you care about and you’re willing to solve it holistically. I would let everything else kind of go away. And I think the moment in tech is here where people are just recognizing good work and those people who can actually just be super high leverage folks at the company and just solve important problems regardless of where in the org they sit and so on.
Kelly Kitagawa: I love that. All right. Well, round of applause for these incredible women sharing all of their experiences and learnings. I’d love to open the floor and pass the mic if any of you have any questions you want to ask.
Audience Member: Yeah. Thank you to the amazing panelists today. My question is actually for Sachi and product management. So you mentioned we don’t really do planning anymore in this new era of AI because things are moving incredibly fast. I’m curious how the coordination’s done across multiple product managers who are also in the same boat of, “Hey, I’m building something. I have to move 100 miles per hour, but you’re also moving 100 miles per hour. So how do we make sure that we’re all striving for the right goal and connection points and things look seamless from the customer
Sachi Shah: Perspective?” Yeah, I think we’re pretty small product team over here, number one and I think so naturally, I think some of the communication panels might be bigger than a 500% PM team or something like that. I definitely give us that as a cost if you will. But I think ultimately, I think the way you set up the team is so important to make sure that people have problems they’re going after and it’s kind of obvious.
Sachi Shah: For example, if we’re thinking through how do we provide more out of the box capability for multilingual agents, I kind of already know I need to talk to the PM who works on voice and then I work on the agent building side so the two of us will jam on stuff and figure out the plan. There may be something she’s already thinking about. And so it kind of becomes like once you have slightly crisp definitions and people again don’t think, even though you have crisp definitions, people don’t really care about the boundaries that much because it’s like, okay, this is a problem if someone else is solving it, great, here’s my input.
Sachi Shah: Otherwise, let me pick it up because it’s like high priority for all of these reasons. And then I also think that that overarching kind of strategy at the company and the transparency in sharing that strategy is also really, really important and that can kind of be a guiding glide to make sure people are working on the right things and thinking about the right problems.
Kelly Kitagawa: Another question hand raised. Oh, yes, go ahead.
Audience Member: Thank you. My questions for Erin, in this new world, you guys are still talking to customers, they have their problems, their goals. Do you find that they’re extremely fluent in AI already and do they say, “I heard about Opus 4.7, can we use that? ” Or are you able to direct them more? How does that product versus hype cycle work in your process?
Erin Rudwall: I would say it’s highly dependent. I was on a call today with a gym, a fitness studio. They had one of their largest franchise owners who owns 35, 40 franchises and he had his own dedicated AI department deploying and evaluating AI, which was surprising, so it really depends.
Erin Rudwall: I would say we see all flavors, but I think one of the things that’s really great about Sierra is and the way we partner with customers is we’re there to help them build and help them along that journey and that path no matter where they are and their expertise. So we see all sorts of variants on that curve.
Audience Member:
Thank you. Oops. Sorry.
Audience Member: Hey, if you can answer this one, what’s been your favorite customer impact story here at Sierra so far? It seems all of you joined Sierra for that same reason.
Erin Rudwall :Oh, I didn’t know if they were directing … Is that directed at me? I would say one of the things that I think is really interesting in terms of customer impact, this is kind of one of our marquee customer stories and it’s kind of maybe an odd choice for this, but we work with SiriusXM and one of the things that I think is really interesting there is their user base skews a little bit older. There was some hesitation on … They were concerned that people weren’t going to adopt AI and people love their agent Harmony and find so much value in it. They say, “Thank you. Thank you for helping me. ” And so I think that that’s an impact where maybe there was a bit of hesitation on the customer side to see what the adoption amongst their customers would be and it’s just provided a ton of value for their customers and their agent is really robust and does a lot of different types of workflows, so that’s a story I really like.
Beth Nakamura: Yeah. A troubleshooting story comes to mind, which is we do troubleshooting across a lot of our agents. For one major consumer electronics company, we had a troubleshooting call that was 90 minutes. To me, whether I was the caller, the person responding to the call, that would have been terrible, but one, we were able to resolve this person’s issue. She was so thankful. It was like the agent is endlessly patient. And then for the customer, normally a 90 minute call, again, would have been really bad for average handle time. When it’s an AI agent, totally fine. I mean, yes, inference costs, Kim’s on it, Ken’s working on that, but that really changed my thinking around what is important. Some of the ways that we measured really great experience and resolution and what’s the fastest path to get there, it made me really think, “Oh, all of a sudden this is going to change quite a bit.” And then the other thing that comes to mind is I get to work a lot with retail folks.
Beth Nakamura: It is so awesome to work with customers who are obsessed about their brand. And so in particular, they all have this amazing, amazing view of their product, their ecosystem, their users, how to make it an amazing experience. And it’s so much fun to help give them AI tools so that they can leverage all of that expertise that they have and turn it into results a lot more quickly. Particularly, this is a group where folks don’t always have access to engineering resources. I was honestly surprised how quickly they trusted Sierra to be their AI expert, but then I realized once you get to work with a lot of folks, you do have a good sense of these leverage points and can equip them to be really, really powerful.
Kelly Kitagawa (00:59:26):
There we go. There.
Audience Member:
Thank you. This is more towards Erin or Sachi. So early on you guys mentioned that you guys collect a lot of third party data. So how does CRS performance depend on the quality of the third party data and where in your cycle do you try to influence this to make it better? Do you give feedback to your customers saying the third party data you’re getting is not good and it’s actually impacting the quality of the results you guys are able to generate? I How do you guys work around this?
Sachi Shah: I can take a pass, but Linden, if you want to add anything. With the third party data, it’s interesting because it’s not really like you don’t need it. You can have an entire conversation and get the job done with first party data and first party memory. I think third party data is useful for powering more of some of these intelligent decisions. Say you’re someone who calls in to SiriusXM or subscription business to cancel every single week and try and get a deal. We should have knowledge of that so we don’t keep giving you offers. Or if you’re genuinely a longtime customer, high LTP customer and the business doesn’t want to let you churn, we can actually use all of the data that we have at hand to actually help make a better decision. So I think if the quality of the third party data, of course it kind of varies company by company, but it is something that companies feel like true ownership of their AI agents.
Sachi Shah: They’ re branded according to Harmony for Sirius XM. It’s not branded as a CR agent. They own the agent at the end of the day and we’re helping them build it and use our platform to build it and so on. And so when they feel such ownership of that, when they’re testing through their simulations or even starting to roll out the agent a subset of traffic, they kind of start spotting, okay, where’s the data week? Maybe a simpler example would even be a knowledge base that they might be maintaining outside of CRR that we kind of sync with. And we often say the problem with the solution to AI is more AI. So we use products to actually detect in the conversation should there have been a knowledge article that didn’t exist, so if the user signs in, they can see like, “Okay, these are all of the things that I should actually go ahead and improve.” It’s not like it’s completely up to the … I mean, it’s in the customer’s hands, but we’re trying to steer them and show them where the gaps are as well.
Audience Member: Okay. Thank you.
Audience Member: Hi, I have a question to Sachi. So as a company who builds the AI agents, I want to know how actually employees are working with AI agents. I can imagine that every single people person can make their own AI agent to streamline their workflow, but also I can also imagine that there should be some duplicated efforts about this. So for example, as a product manager, there should be some standardized product development process. So I was wondering if there is this kind of shared library so that you can share your AI agent to help your colleagues. So I want to ask about that.
Sachi Shah: Yeah, it’s a great question. I think we do a lot of experimentation and knowledge sharing and also things like lunch and learns at our team meetings and so on. We locationly be like, “Hey, how are we actually using products?” Or like someone sitting next to me in a meeting, I’ll look over and I’m like, “Hey, what’s that? ” And I’m always learning things and that’s what makes it fun because everyone has different products that they use and so on. But we do try and find ways to kind of, if someone’s found something that works, they have the avenue to actually share that. I think one of my favorite things is also we have an AI acceleration team. We have an entire engineering team focused on AI acceleration within the company. And so if you walk around the office in the middle of the afternoon, you’ll see on everyone’s monitors, they have multiple cloud agents running and so on and we’re building the technology to actually enable these and productize some of the things that have come out of one engineer tried something and it worked really well.
Sachi Shah: I guess on the same note about I understand how important it is to adopt AI for internal use and I can imagine different use cases across sales, across finance, across onboarding HR, anything else and engineering of course it’s kind of natural doesn’t need to be mentioned really. But how do you balance the costs of this versus productivity output? Sorry, I just imagine that it can sometimes one developer that’s not very careful can maybe spend $10,000 in an hour with stockens just hypothetical number.
Beth Nakamura: For the record, I did that accidentally once at Lyft too with infrastructure, so not necessarily a new problem. And I think being very thoughtful about how do you want to set up some of the structures and when does it make sense to help have more support?
Beth Nakamura: One thing I love about Sierra is it’s incredibly engineering friendly. That means that you can do a lot without friction. I mean, they trust their engineers a lot. That means you need really robust CICD, you need really robust ability to roll back. It’s an entire principle to think like, oh, we trust and give agency to individuals to go out and win their space, win these problems. That means you also have to give them the right support and that includes a lot of observability, a lot of tooling, a lot of signals around what might you have missed and not expecting that every single thing that is built has completely thought about all those signals, and so instead building out systems that will help you do that. So that’s kind of how I think about it from the engineering lens.
Audience Member: This is the build off of the last two questions, but with this whole AI agent emergence, how are you guys thinking about governance? I mean, you talked about it in the context of engineering or individuals trying to execute and use tokens, but there’s a broader resource used, right? There’s like tokens, there’s finances, there’s water, there’s energy. And when you lift that up and think about it in the scope of like an enterprise or a startups or an organization’s broader philosophy on agents, there are a lot of values that go into that. So I’m curious about the values that go into AI agent deployment because we want to use them, but we want to use them well. And I struggle with this on a daily basis. Anyway, that’s my question if that makes sense.
Beth Nakamura: 100%. And I love what you said, everyone wants to use this, but we want to use it well and it’s not clear yet what will be sustainable. One of the things that drew me to Sierra is at the time I was looking, I was really looking for a place that would be able to successfully jump from product to platform and I didn’t actually know what that meant. I just had seen a lot of great teams and companies in tech kind of get stuck at that product phase often with like really good businesses but hadn’t necessarily been able to jump out of that. And I actually think that what I realized it actually looks like is being able to choose the right abstractions so that you can optimize in parallel. We saw a litle bit with the voice pipeline, but to your point, we are spinning up a lot of agents, but the fact that we are thinking about it as a platform means that we are also, we have teams thinking about the optimization, the paralyzation, and there’s a component of like not just does it work, but is this actually going to work at scale?
Beth Nakamura: And that’s like that having that thinking upfront from the founding of the company was really important just in terms of seeing the technical decisions be made, but also what are the principles that we actually build on.
Beth Nakamura: One of our values is customer obsession so I think it’s like very much we’re always thinking about doing the customer first, but a huge part of it too is just the ability of like, we want to be a durable company, a durable platform.
Beth Nakamura: We are trusted by customers who already have durable businesses to help them, and so it’s really important that we’re able to do that knowing that like we have to have a backbone. Kim leads a lot of that backbone to actually serve across different regions, different use cases, and then during these companies’ most important events. All
Kelly Kitagawa: Right. I think we have time for one more question. Yeah, I’m there. You got to come get the mic though. I’ll be the mic runner.
Audience Member: Hi, I have a question maybe for Sachi and Beth, but like in the age of a coding agents, there’s two views that I’m seeing like talking to friends that work in other industries is one is, okay, so the code agents are going to empower the product people and then they’re going to need maybe less engineers and the product people come up with like the prototypes and the engineers productize them. And the other view is like, no, we need maybe less product and the engineers become also product people and then the code agents help these engineers be more productive. I’m really curious about how you guys balance that dynamic nowadays and if you just had lessons about having these code agents helping both product and engineering.
Sachi Shah: Yeah. I think from what we’re seeing, I think what I’m seeing at other companies work as well is what you want is a product manager who’s technically curious enough to actually design the right solutions with the technology at hand and be afloat of all of the changes and all of these things and an engineer that’s product minded.
Sachi Shah: You’re thinking not just about like shipping the feature, but does this make sense as part of the product product? What do the workflows look like? Is the solution actually fitting the needs and solving for the problem that was outlined to start with and all of these things, so in a lot of ways the roles are becoming more fluid and coming together, but maybe that’s how I’ll summarize it as more technical PMs and more product-minded engineers and also just kind of like taking that lens because I think a lot of us have it in us and so it’s just kind of like having the right mindset around it.
Sachi Shah: And then I also, maybe one thing I’ll add is the way I think about using AI to deliver true outcomes and not just for like using AI for the sake of using AI is it’s making people a lot more productive. It doesn’t mean like, hey, you need fewer people. It just means that you’re producing so much more value in the world and for the business is the way I like to think about it.
Beth Nakamura: Yeah. I would add power to the proactive. I love it. I think the roles are going to converge or at least have a lot more overlap. Sachi and I are both former founders. Something that a lot of people can talk about is like founder market fit. I actually was not as into that because I was like, who cares if there’s a better person out in the world? What matters is who is willing to build it and who is willing to stick with the problem and think about it rigorously and learn from it.
Beth Nakamura: I think now, especially in this space, the only competitive moat is the ability to execute and compound on your learnings from what you learned by just getting about a litle bit further. Everything I’ve seen in industry, places where Sierra’s product is best in class, places where we improve, just so many really smart people chasing down that moat.
Beth Nakamura: That key of just being proactive, really building something, knowing that it might be wrong, but at least then you can be precisely wrong instead of generally right and figure out what’s next is like really, really powerful.
Kelly Kitagawa: Power to the productive.
Beth Nakamura: Exactly.
Kelly Kitagawa: All right. Well, we’re going to wrap up. Can I please have one more round of applause for our amazing panelists and also all of our speakers, Linden, Kimberly as well for our speakers and Beth, of course. And I also wanted to give one more round of applause for everyone who helped me and my amazing team, Alishan, there you are.
Kelly Kitagawa: Shout out to Alishan and Alice and Eleni and Sama and Sasha and everyone that really helped put this together, especially Angie and Sukrutha as well, so all of this takes a village and yeah, we are so grateful. And with that, I’ll close.
Kelly Kitagawa: Please take as much food as you can on the way out, but we hoped you enjoyed and we hope you get home safe.
What is the difference between traditional software engineering and AI agent engineering?
Answer: Traditional software engineering relies on deterministic, predictable code roadmaps. In contrast, AI agent engineering focuses heavily on prompt orchestration, managing non-deterministic system behaviors, and implementing robust semantic “AI memory” so agents can handle complex, multi-turn human interactions contextually.
How does an AI conversational agent ensure personalization at scale?
Answer: Conversational agents achieve personalization by integrating specialized retrieval systems with an “AI memory” layer. This architecture allows the LLM to instantly recall user preferences, past interactions, and unique customer data points to dynamically tailor responses in real-time, moving far beyond static, rule-based chatbots.
What skills are most critical for Product Managers (PMs) transitioning to AI products?
Answer: While technical understanding of LLM orchestration is helpful, the most critical skill for AI PMs is exceptional “product sense”—knowing exactly what to build and why. Because AI dramatically lowers the cost of writing code, a PM’s value shifts from managing rigid development timelines to quickly iterating on user experiences and evaluating agent behaviors in real-world environments.
How should candidates prepare for technical interviews at an AI-forward company like Sierra?
Answer: Candidates should be prepared to showcase problem-solving agility over rote memorization. For technical roles, be ready to discuss how you handle non-deterministic systems, data retrieval pipelines, and edge cases. Using structured frameworks like the STAR method (Situation, Task, Action, Result) to explain how you navigated past engineering ambiguity is highly recommended.
What are the main challenges of moving an AI agent from a prototype to production?
Answer: The primary hurdle in productionizing AI agents is balancing conversational flexibility with reliable system constraints. Engineering teams must design strict evaluation guardrails to monitor for hallucinations, optimize data intake pipelines for latency, and ensure the agent can gracefully pass a customer to a human agent when a request falls outside its sandbox.
How do fast-paced AI engineering teams maintain collaboration and alignment?
Answer: Leading AI teams skip prolonged, theoretical planning cycles in favor of rapid prototyping and cross-functional feedback loops. Engineering, product, and design sit closely together to continuously test live agent outputs, allowing them to iterate instantly on user experience shifts driven by evolving model capabilities.
How does context window management impact AI agent performance?
Answer: The context window defines how much text an LLM can process at one time. If an agent passes too much irrelevant data or entire conversational histories into the window, latency increases, operational costs rise, and the model may lose track of core instructions (often referred to as “lost in the middle”). Efficient agents use smart retrieval strategies to pass only the most critical, real-time data snippets required for that specific turn.
What role does customer data privacy play when deploying conversational AI?
Answer: Deploying enterprise-grade AI agents requires strict data governance and compliance architecture. Companies must implement secure data pipelines that mask or strip Personally Identifiable Information (PII) before it reaches public LLMs, utilize dedicated private cloud instances, and ensure customer data is never used to train foundational third-party models without explicit consent.
During an inspiring keynote at 
