Girl Geek X OpenAI Lightning Talks (Video + Transcript)

openai girl geek dinner san francisco mission district residency

Over 120 girl geeks joined networking and talks at the sold-out OpenAI Girl Geek Dinner on September 14, 2022 in San Francisco’s Mission district.

Hear lightning talks from OpenAI women working in AI with music and deep learning, sharing the power of trying and trying again, how to make language models useful, and much more at the OpenAI Girl Geek Dinner video on YouTube!

OpenAI Residency applications are open! OpenAI is looking for engineers and researchers who are interested in applying their skills to AI and machine learning. Please apply for OpenAI jobs here!

If you have an unconventional educational background, we encourage you to apply to OpenAI Residency (applications are open through September 30, 2022).

Table of Contents

  1. Welcome – Elena Chatziathanasiadou, Talent Programs Lead at OpenAI, Recruiting & People – watch her talk or read her words

  2. Multimodal Research: MuseNet & JukeboxChristine McLeavey, Member of Technical Staff at OpenAI, Multimodal – watch her talk or read her words

  3. If At First You Don’t Succeed, Try Try Again – Alethea Power, Member of Technical Staff at OpenAI watch them talk or read their words

  4. Making Language Models Useful Tyna Eloundou, Member of Policy Staff at OpenAI, Policy Research – watch her talk or read her words

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Transcript of OpenAI Girl Geek Dinner – Lightning Talks:

Angie Chang: Hello. Thank you everyone for coming tonight. My name’s Angie Chang and I’m one of the founders of Girl Geek X. We started over a decade ago as, Bay Area Girl Geek Dinners, and we’re still going strong. Thank you to OpenAI for hosting us for a second time. We’re really excited to see the new office and invite a bunch of Girl Geeks over to hear these lightning talks on AI and policy and all these things that we’re so excited to learn about tonight!

Sukrutha Bhadouria: Hi. I know you all were still chatting when Angie introduced herself, but she’s Angie and Girl Geek X is basically her brainchild. It started off with Angie looking to bring women together, I’m doing your pitch, Angie for you because I have a louder voice. Some people, they ask me if I swallowed a mic as a child because I’m so loud and I don’t need a mic.

OpenAI Girl Geek Dinner Facebook Cover

Sukrutha Bhadouria: Anyway, I’m Sukrutha, so Angie started Girl Geek and it was back then called Bay Area Girl Geek Dinners, this was over 10 years ago. And when I had just moved to the Bay Area, looking for ways to meet new people and I found out about Bay Area Girl Geek Dinners dot com at that time, and I tried really hard to meet with Angie, but she was a busy bee doing all sorts of cool things, trying to change the world. And this was way before ERGs existed, right? So people didn’t have a way to connect with the community until they went to meetups.

Sukrutha Bhadouria: And Girl Geek Dinners, at that time, was the one way you could also get an insight into what these sponsoring companies worked on, what life was like. And so it also allowed people to get an opportunity to speak and a lot of the speakers at Girl Geek Dinners were first time speakers. They were too afraid to sign up for conferences. If you go to our website (, you’ll see all these amazing stats on how since Angie started, there’s been a real shift in the environment in how people are more willing to speak at conferences, due to some of the chances they’ve gotten as a result of speaking at an event sponsored by their company. This organization exists.

Sukrutha Bhadouria: I joined Angie and we tried to change the world together. I’m happy to report that I think we actually did. We rebranded to Girl Geek X, and that’s when the organization hit 10 years. It was a sizable number of people working on it, it was Angie and me and it was just the two of us. And then Angie had this idea to really evolving into a company and so that’s when she started to bring on contractors, more people such as somebody who could take video of our events to make us look a little bit more professional and somebody else to do our website besides me. And we started to do podcasts.

Sukrutha Bhadouria: We started to do virtual annual conferences and we really, really, really were always consistently sold out for our in-person events that would happen at various companies that we partnered with through the Bay Area. Then COVID hit and the good thing is that we had already started to have a global presence through the virtual conferences that we had and we’ve now had four? Five, yeah.

Sukrutha Bhadouria: We used to be carpooling all around the Bay Area together to these events after work and now we are moms. So it’s amazing. We would look up and see amazing people working at these sponsoring companies speak and we’d be like, “Wow, look at them managing their mom life and parent life and coming to these events.” But I just think that it’s now become such a common thing that it’s not as isolated anymore. And I’m hopeful that, you all can come back again and again, because this in person event has really made me really happy.

Sukrutha Bhadouria: I’ve been holed up in my home office today, which is basically a room which also has my… What’s it called? A bike that stays in one place, stationary bike, so it has too many things going on in the room, but I wanted to give a big thanks to OpenAI for hosting us for the second time, for sponsoring for the second time. And I hope that we can keep doing this. So please do get your companies to sponsor and encourage them to do it in person. That’s all I will say. I know I said a lot more than I had planned, but thank you again, and Angie.

Angie Chang: Thank you Sukrutha, for the intro. I guess I should talk up Sukrutha a little more. When I first met her, she was a software engineer in test, and now she is at Salesforce as a Senior Director of Engineering there, so I’m very proud of her. And over the years we… She mentioned we have a podcast, we have annual virtual conferences!

Angie Chang: We’ll be launching a career fair virtually as well, to be announced. And I don’t want to say too much. We have an amazing line up of speakers tonight and we’re going to invite up first, Elena, who is our host for the night from OpenAI.

Elena Chatziathanasiadou: Hi everyone, I’m Elena. I work here and I’m on the recruiting team, I’m leading the Residency program right now. I’m very excited that you’re all here and have joined us together. Really want to thank Angie and Girl Geek X. We’re very excited to deepen our partnership together and to be back in the office here all together, in the new space and to experience this tonight.

openai girl geek dinner Elena Chatziathanasiadou

Elena Chatziathanasiadou: We’re very excited about having you here and in terms of what we’ll see tonight, we’ll have a series of lightning talks and then that will be followed by Q&A and then we’ll get some dessert in the area that we were before and then we’ll wrap up at 8:30. But before we get started, I did want to take a moment to make a quick plug and share that…

Elena Chatziathanasiadou: We’re actively hiring for our Residency program and that includes both research and engineering roles and the goal of it is really to help develop AI talent. The program, it offers a pathway to a full-time role at OpenAI for folks that are currently not focusing on AI and are already researchers or engineers in a different field.

Elena Chatziathanasiadou: We’re really excited to hear from you. If you do have an interest in making this career switch, come talk to me after. And we’ll also have full time recruiting team members and positions that we’re hiring for across research product and engineering that we can tell you more about. Please come find us and learn more about the interview process, but also what the program offers.

Elena Chatziathanasiadou: With that I wanted to introduce our first speaker, Christine, who’s currently managing our multimodal team and previously worked on music generation research, created MuseNet and was collaborating on Jukebox. And before that was a classical pianist who transitioned into a researcher as well. I’ll hand it over to Christine. Thank you so much.

Christine McLeavey: Thank you. So yes, it’s really an honor to be here tonight. Thank you all for being here. And this Residency program is near and dear to my own heart, because I first joined OpenAI through, what was then the Scholars Program and the Fellows Program and those are the programs which have since evolved into this Residency program. I’ll put a plug in for anyone who’s considering it.

openai girl geek dinner Christine McLeavey

Christine McLeavey: I want to talk this evening about my own path through OpenAI, but especially about the two music models that I worked on during the time here. I thought I’d start by just going ahead and playing an example of each of the models. The first one, this is the one I worked on when I was doing the Scholars and Fellows program. This is MuseNet, which works in the MIDI domain, so this is the model trying to generate in the style of jazz. Okay, I’ll cut that off and then after I joined full time, I was lucky enough to collaborate with some amazing researchers here to work on a model that was instead working in the raw audio domain. The fun of that is you get to imitate human voices. This is trying to do the style of Elvis with lyrics by Heewoo. Okay.

Christine McLeavey: Elena mentioned before being at OpenAI, I was actually working as a pianist, I had done some math and physics in college, but obviously it had been a long time and so I think I took a good year of self studying before I applied to anything. And I thought I would just give a shout out to three of the online programs that I particularly liked at that point. They’re all amazing. But then I was lucky enough to join the first cohort of scholars that we had here. And at that point I was just trying to do this process of learning about all these different models. And I had this feeling that instead of just copying a model or copying what someone else has done, let me just try to translate it into a field that I know well, which was music. And so what became MuseNet was really my attempt to take all of the stuff I was learning and then apply it to the music domain instead.

Christine McLeavey: MIDI format is this really nice representation of music. I think of it as the way that a composer thinks of music, so it’ll do things like it tells you what notes it plays when, the timing of it, the volume of it, things like that, which instrument is supposed to play. But it loses all the actual detail of when a human takes it and performs it. You don’t get a person’s voice, you don’t get the sound of a great cellist, anything like that.

Christine McLeavey: The nice thing is it’s what you trade in expressivity, you get in this nice really meaningful representation. It does sound pretty terrible when you try to render materials. As a musician, just thinking about the structure of music, this was a nice simplification for a scholars project. What I did is I took a bunch of MIDI files and I tried to pull them out and turned them into a sort of language to make them look as much the sort of thing that you could get in your own net to predict as possible.

Christine McLeavey: I did things like I would always tell the model which composer or which band was going to be first and then things like what tempo was going to be when notes would turn on and off, and a wait token, which would tell the model how long to wait, things like that. And then what you end up doing is you translate that tokenization into just a dictionary of numbers and the model sees something like this. Which I think that this is the first page of a Chopin bellade or something.

Christine McLeavey: What the model is faced with is this task of given the very first number, what number do you think is going to come next? And then given the first two numbers, what number is going to come next? And when you first look at the first thing and when the model first sees it’s like how do you do this? What does that even mean? It feels like an impossible task. But what happens is the model sees many, many, many examples of this.

Christine McLeavey: And over time it starts to pick up on, ah, if I see 4,006 somehow I tend to see 586 more often after that or something. It starts to pick up on these patterns, which we know because we know the tokenization was like, oh, if a piano plays the note G, then probably soon after it’s going to turn off the note G or something. It has real musical meaning to us. But the model is just seeing these numbers like that. The nice thing is the model gets really good at this job and then you can turn it into a generator just by sampling based on, I thinks there’s like a 20% chance this token’s going to come next, so 20% of the time take that.

Christine McLeavey: The other really fun thing you can do is you can then study the sort of mathematical representation you’ve gotten for these tokens. So I was always giving it the composer or band token in the beginning and now you can look at the vectors or the sort of embedding that it learns through these composers.

Christine McLeavey: And as a musician it’s really fun because I would clearly think that Da Vinci and Ravel, for all these French guys are related and the model just picked up on the same thing, which is cool. But the other really fun thing is that you can mix and match those [inaudible]. So here is the start of one of my very favorite Chopin, Nocturnes. So I actually just gave the model the first six notes of that and this is what the model thought, if instead it was being written by [inaudible] It was a bunch of VPs. It goes on for a while, but I’ll cut it off there. And that was MuseNet.

Christine McLeavey: And then I ended up joining full time after that and I was lucky enough to collaborate with Prafulla and Heewoo on taking music generation over to the raw audio domain. And so in a way this is a much harder problem because now whereas in MIDI world you have just nice tokens which are meaningful in a musical way, raw audio is just literally 22,000 or 44,000 times per second.

Christine McLeavey: You’re recording how loud the sound is at that moment in time and the nice thing about it is it gives you all this expressive freedom, right? Literally any sound you can imagine you can represent as a sound wave, just audio recording to that. The trouble is there are just so many ways for those waves to go wrong or those patterns to go wrong. If you mess up on the short scale, it’s just like crazy hissing noise. If you mess up on long scale, your piece sadly starts getting out of tune or the rhythm drifts or so many ways it can go wrong, it’s really an unforgiving sort of medium. And the problem is now in order to get a minute of music, it’s no longer maybe 3000 tokens you have to do, it’s maybe a million numbers that you have to get correct.

Christine McLeavey: We approached this by looking at ways that we could compress the music to make it more tractable because at that point a transformer could maybe deal well with the context of 4,000 tokens or something. We used an auto encoder to do three different layers or levels of compression and the sort of least compressed on the bottom. The nice thing about that is it’s very easy to translate it back to the regular raw audio. If you put some original song in and then back out, you don’t notice any loss at all. Whereas if you put it through the most compressed version, the nice thing is now it’s super compressed, like 3000 tokens might get you half a minute of music or something. But if you go through this simple just trying to reconstruct the raw audio, it sounds really bad. You can sort of tell that someone’s singing but you’ve lost most of the detail.

Christine McLeavey: The nice thing about it is when you work in that top layer of tokens, now this looks a lot like the MuseNet problem or even just a lot language problem where you’re just predicting tokens. So we train a transformer on that. We sort of added in the same which person was singing, which band was playing, and then we also added in where you can write the lyrics in, so the model conditions on the lyrics and then generates these tokens. And then I won’t get into the details, but we had to train extra transformers to do this upsampling process so that you could get back to raw audio without totally losing all the detail.

Christine McLeavey: The fun thing is you can do things like ask it to generate in the style of Sinatra singing Hot Tub Christmas and I have to put in a book, these were lyrics by at, that point, GPT-2. All right. It’s a Christmas classic now. And then last I wanted to wrap up by talking a little bit about the multimodal team, which is the team that I’m really excited to be managing these days. It’s this really, really great group of people. Unfortunately, our current projects are all internal and I can’t talk about them, although stay tuned, we’ll be publishing them to the blog when we can. You might recognize Clip, which was work done by Alec and Jong Wook both on our team. This is, I guess, nearly two years ago already, but made a really big impact on the image work at that point. And then just to put in a plug for the team, we’re about a group of 10 at this point and we will be hosting a resident in 2023.

Christine McLeavey: Please reach out if anyone’s interested to talk more. And then we’re doing all sorts of projects in the sort of image, audio and video domains both on the sort of understanding side and generation side. And we end up working really closely with algorithms, which is the other team that tends to do a lot of awesome multimodal projects. But then also anytime we get close to things that we’re looking at putting out tech customers, we end up working with applied through that and then also obviously scaling because at OpenAI we believe deeply in this, get a good pattern and then scale it up and it becomes awesome. So thank you so much for your attention.

Elena Chatziathanasiadou: Thank you so much, Christine. That was awesome. So now next we’ll have Alethea. Alethea has spent the last couple of years at OpenAI working on getting neural networks to do math. Before that, they built large infrastructure health system, studied math and philosophy and spent lots of time singing karaoke. Welcome, Alethea.

Alethea Power: Thank you. So this talk is called If At First You Don’t Succeed, Try Try Again. It’s been a wild few years. I decided I wanted to give an uplifting and encouraging talk. It’s a short talk so it doesn’t get too deep into technical details, but if you’re interested in it, please find me afterwards. I will talk your ear off about it.

openai girl geek dinner Alethea Power

Alethea Power: Okay, my name is Alethea Power and yes, Patience is actually my middle name, which will be very relevant for this talk. Okay, so about 10 years ago I was a software engineer and site reliability engineer and my dream was to get into artificial intelligence, but I didn’t know how to do it. I didn’t have a degree in AI, I didn’t have any background in AI, I didn’t have any idea how to break in. So I thought, ah, I probably need to take some time off to study this before I can get into the field.

Alethea Power: I started saving up some money so that I could take time off to study. But by the time I had enough money saved up, I realized I needed to handle my gender issues. So I took that time off to go through a gender transition instead of studying AI. Eventually though I was finally ready to try and break into AI in some form or fashion and that was about the time that OpenAI hosted their last Girl Geek Dinner, that was in 2019. And I came to that talk and I met one of the recruiters who stunned me by telling me I didn’t need to have a degree in AI and I didn’t need to have a background in AI to be able to work here.

Alethea Power: She introduced me to the Scholars Program, the same program that Christine went through, which today is called the Residency Program. And I applied to that and I got in and I had the best mentor in the entire program, Christine. I’m second generation scholar up here. But there were in addition to the obstacles before, there were obstacles after joining the program as well, about three weeks after I joined, there was a pandemic, you may have heard about it. But despite spending a lot of time fearing that I might die or people I love might die for some reason or another, health or political, Christine was very kind and understanding and supportive and she helped me get to the point where I had learned a ton about artificial intelligence and managed to do a great project and I ended up applying full-time and I got three offers here. Thank you. I wasn’t trying to brag, but thank you. This is more to encourage you.

Alethea Power: I ended up taking a job on a team that was trying to teach neural networks to reason and do math. And what I want to talk about here is about a year after I joined that team, I released my first research paper called Grokking: Generalization Beyond Overfitting on Small Datasets. I’m going to give you a very basic introduction to what all that jargon means. And like I said, if you want more technical details, come talk to me afterwards. So first I need to explain how training neural networks works. If you have a background in ML, this is going to be very basic 101. If you don’t, it’s going to be exciting.

Alethea Power: Okay, so usually when we’re trying to train a neural network, we’ve got some amount of data that captures a pattern that we want that neural network to recreate in the future. And often if we’re doing what’s called supervised training, we’ll break that data up into training data and evaluation data. And you can think of this, the training data is sort of what we actually teach the neural network, what it learns from. This is like classroom education and evaluation data is basically like pop quizzes to see how much the neural network learned. And neural networks have this nice property where you can pop quiz them. They don’t learn anything from the pop quiz, they just tell you how they did and then five minutes later you can pop quiz them again and the questions are all new again, they have no memory of them. Throughout the course of training, we measure the performance of the neural network on both the training data, the classroom instruction and the evaluation data, the pop quizzes.

Alethea Power: And there’s two main ways we measure this. One is called loss. I won’t go into details right now about what loss is, but the short version is it’s a differentiable function calculus derivatives that we use to actually figure out how to modify the network, so it learns, when loss goes down. The network is learning. Accuracy is exactly what you would think of being like a test score, so 0% accuracy means you got every question wrong. A hundred percent accuracy means you got every question right. This is what a very successful neural network training looks like. You can see, oh, the x axis here on both of these graphs is steps of training. You can see that as we train this neural network along the loss on both the training and evaluation go down. It’s learning what it’s supposed to learn from and it’s able to generalize that to the pop quizzes.

Alethea Power: It’s doing well on the tests as well and then this is what it’s actually scoring. So by the end of this training it gets up to 90% accuracy, so it’s got an A. Sometimes though, if you train a neural network for too long, it starts to do what’s called overfitting. You might remember the word overfitting from the title of the paper. In this case, the neural network learns too much detail from the training set that doesn’t really generalize to the rest of the world. And so its performance on the quizzes starts to get worse. So an example of this in this paper, I was training neural networks to do math, basic mathematical equations. For instance, if it happened to be the case that the training data had more even numbers than odd numbers, and if it was trying to learn addition, then it might learn that usually the answer is going to be even. Well, in reality that’s not true in addition.

Alethea Power: In reality, you want to actually know how to add and the number’s going to be whatever it is. So that would be an example where it learned some sort of incorrect, non-generalizable information from the training set and that made it start performing worse on the evaluation set. And you can see here in this situation, the accuracy on evaluation would go back down. Sometimes, and this is very common when you’re trying to get a neural network to do math, you have an even worse situation where the same thing happens with your loss, but it consistently fails the pop quiz every time. Gets to a 100% percent accuracy on the training data and fails the pop quiz. This means the network and we were using similar kinds of networks to the ones Christine was talking about, just math instead of music, this means the network never really understood what it was learning, it just memorized it.

Alethea Power: This is like the kid who knows that when you say six plus four, you’re supposed to respond with 10 but has no idea how to actually add. So this was a common scenario when training neural networks to do math. They’re really good at pattern recognition, but they’re not always good at understanding a deep analytical precise truth underneath the pattern. Well then one day we got lucky and by lucky I mean forgetful. So one of my coworkers was running an experiment like this and he went on vacation and forgot to stop it. And so a week later he came back and it had just kept studying and studying and studying and studying and studying and studying and studying and studying and studying. And it learned. So what happened here was, it went into this overfitting regime where usually we’d say, ah, it’s learned all it can learn from this training data.

Alethea Power: There’s no more to learn and see, it still had zero accuracy and it just kept getting worse and worse and worse. And then suddenly long after it memorized all of the training data, it had an ‘aha’ moment and it was like, oh, all this stuff that I memorized actually makes a pattern and the pattern is addition or division or S5 composition or whichever task we had it working on. And then the loss started coming back down on the pop quizzes and it went up and it got a 100%. This is weird, this never happens in neural networks. We dug in and recreated this many times, implemented it twice, saw the same behavior with two completely independent implementations on a wide variety of tasks and there’s all sorts of other interesting stuff about when this happens and when it doesn’t, ask me in the questions afterwards.

Alethea Power: The point here is at first the network didn’t succeed, but it just kept trying the same way I did when at first I couldn’t get into AI, but I just kept trying. We named this phenomenon where it finally figures it out Grokking, and we named this after Robert Heinlein’s novel Stranger in a Strange Land. It’s a science fiction book and Grok is a Martian word in that book, which means, “To understand so thoroughly that the observer becomes a part of the observed to merge, blend, intermarry, lose identity in group experience.” And it turns out this is exactly what these neural networks do. I’m going to let you take pictures before I change the slide.

Alethea Power: This network was trying to learn modular addition and modular addition you can think of is adding hours on a clock. Also, thank you to Christine for that analogy. If you have 11 and you add 3 to it, you don’t end up with 14, you end up with 2 because that’s what happens on the clock. The clock is modular 12, we were having it learn modular 97, and then we tore open the network that had grokked afterwards to see what was going on inside of it and it had actually built internally this circular structure of the numbers. It had created the mathematical structure we were trying to get it to learn that allowed it to actually solve the problem. Did this with all different kinds of problems, so we had one network learning to compose permutations and it found what are called subgroups and co-sets out of that, details later. But the point is, it worked so hard for so long through so much failure that it became the knowledge it was trying to get.

Alethea Power: The point here is, that if your dream is to get into AI, even if you have no background in AI or whatever your dream is, it doesn’t matter. Keep trying and keep trying and keep trying and keep trying and maybe you can get there eventually. And in particular, if your dream is to work at OpenAI, which I highly recommend because this place is fabulous, then try, even if it’s not the background you have already, even if you feel like you have a weird background or you’re not like the people here or like the people in this field.

Alethea Power: We’re a humanitarian organization. Our core mission embodied in our legal structure and our financial structure is to make sure that artificial intelligence benefits all of humanity instead of just a small number of rich people in Silicon Valley. And to be a humanitarian organization with a humanitarian mission, we need a wide diversity of perspectives here. If you have a different life story, a different path, different perspectives than we’ve seen before, that makes you more valuable here, not less, so please consider applying.

Elena Chatziathanasiadou: Thank you so much, Alethea, That was awesome. And now next we’ll have Tyna, who’s on the policy research team currently doing our rotation on applied research and she participated in the OpenAI Scholars Program, has spent some time researching economic impacts of our models, building safety evaluations, and collaborated on web GPT and moderation API. Let’s hear from Tyna.

Tyna Eloundou: Wow, so many of you. Let’s see. Okay, this works. Hi, everyone, thank you so much for coming. I’m Tyna Eloundou, I’ll be speaking to you today about making language models useful. A bit about myself, let’s see, wow, I’m also a former scholar. I can’t make the claim to third generation because Alethea was not my mentor, but they were super helpful in making my experience here amazing. And part of that culture and that welcoming environment was a reason I chose to stay on after the scholars program [now the Residency program].

openai girl geek dinner Tyna Eloundou

Tyna Eloundou: Today we’re going to be talking about language models and by language model, I mean any model that has language as input and output. So that could mean GPT-3, CODE-X, or BigScience’s Bloom, what have you. Okay, this is going to be the only equation you see throughout this talk and it’s really not that important, but I think it gives us some context as to where we’re going.

Tyna Eloundou: Looking back at this, this is the training objective for GPT-3 and for all GPT like models. Given a corpus of tokens, right? We define the objective to maximize this likelihood, L, which is defined as a conditional log probability over a sequence of tokens that is modeled by a neural network with parameters data that is trained by gradient descent. Now you can forget everything I just said.

Tyna Eloundou: Essentially this optimization produces these models that are trained to predict tokens, but that in itself may not be that useful on its own. I don’t think I’m giving away any secret sauce by revealing this equation to you, but it is remarkable that somehow we go from this to models that can produce, oh sorry, that can do that, right? Write prose, write code or parse data and so on.

Tyna Eloundou: I’d like to talk a bit about the notion of usefulness itself. One way to think about whether language models are useful in the first place is in the pragmatic sense. In the ideal scenario, we would be able to succinctly communicate our goals and preferences to a language agent without having to laboriously explain and list what to do and what not to do.

Tyna Eloundou: How did we initially get usefulness out of language models? When these models were first being developed in research labs, some researchers came with some ideas about how to really get them to do what it is that you want them to do. And these are two of the most prominent ones. One was few shot prompting, which is a method by which you really tell the model what the task is and before putting it on the spot, so to speak, you give it some examples of what you like to do, some demonstrations, right? For translate English to French, you could have a pen to [foreign language], I’m hungry to [foreign language], et cetera. And the translation that you actually want, you say, I would like to eat ice cream and hopefully with that same formatting you get the model to translate to French.

Tyna Eloundou: The other method is supervised fine tuning, which involves essentially just having examples for the model and then kicking off another round of training so the model can become hyper focused on your task and hopefully improve its performance on that task. So as many of you probably know, OpenAI has since then adapted this iterative deployment approach, which helps us put models in the hands of real people and understand how they interact with them. At the time of GPT-3 release, it was doing great by research standards, right? And unfortunately a lot of these research metrics are designed around these methods that we’d spoke about before, which are to prompt with few shot prompting or perhaps to do supervised fine tuning. Once we deployed, we really quickly learned that people don’t like prompt engineering. In fact, they don’t really like to do a lot to communicate their goals to the model, which is fine. It’s a feature, not a bug.

Tyna Eloundou: At its most helpful, a language agent can infer what we want without lots of specification and carry out those inferred goals effectively and efficiently. Unlike researchers, people were using natural language instructions to ask GPT-3 for what they wanted. But because of the training objective that we saw previously, the model was really tempted to just pattern match, right? If you gave it a prompt of write a short poem about a wise frog, it would very helpfully give you similar types of prompts instead of following your intent. This spurred a research effort within our alignment team to teach the models how to follow direct instructions. They did this using two insights. The first is borrowing from the supervised fine tuning or supervised learning literature where you can train the model based on examples or demonstrations, right?

Tyna Eloundou: You have a prompt and you tell them what you would ideally like it to do. And the second insight came from the reinforcement learning literature where you have some humans compare outputs. And so this model learns to generate, that model learns to compare, right? That model learns to tell this is good, this is bad. And so now with these two things, you can kick off this joint training process where you have a model that’s generating and then a model that’s critiquing, and this is good, this is not so good.

Tyna Eloundou: Over the course of training, the model learns to get better at pursuing this objective, which is no longer the pure language model laying objective and now it’s the instruction following objective. So the resulting model was InstructGPT, which is presented here. Well, yeah, you can see the output. It’s a poem, it’s about a frog, mentions wisdom, and it’s pretty short. I feel like all the requirements were met for following instructions there.

Tyna Eloundou: This was a plot that was quite striking to me. This is one of the main results from the InstructGPT paper. When I first saw this, it didn’t make a ton of sense until I really understood the research behind it. But I think that you can think of the Y axis as a proxy for usefulness and the X axis. We have model size and conventional wisdom has it that… We’re at OpenAI as you scale things, things get in general better. But you can see that even at its smaller size, right here, if you can’t see it’s 1.5 billion parameters, even at its smallest size InstructGPT was deemed to be more useful than any permutation of the base GPT model. So I started this discussion by talking about how research based approaches were not pushing far enough in terms of getting us usefulness out of these models. There’s now this emerging literature focused on helping models be more effective in tasks.

Tyna Eloundou: Broadly speaking, this literature involves having models break big problems up into smaller problems or things step by step before coming up with a final answer. And this does not need to be at odds with our human alignment driven research. In fact, right here you see a result by Kojima et al. and although their results are great overall across the board, we do see that they make the Instruct models even greater. There’s such a huge gap, a huge gain that we see with the Instruct series of models.

Tyna Eloundou: I would like to conclude by thinking about the next steps in this line of research. We know that there can be some instructions that can be malicious or exploitative or deceptive. If language models were to pursue usefulness at all costs, they might become dangerous in the pursuit of dangerous instructions or dangerous intent. Could there be other objectives we pursue along with usefulness that get us helpful but not dangerous models, perhaps kindness or hopefulness?

Tyna Eloundou: And lastly, with instructions, we’re mainly in the driver’s seat and we initiate interactions. As language models become smarter, perhaps kinder, more capable, it may be appropriate to think of them as collaborators and they may be capable of initiating ideation, creation among other things. What are the different modes of interaction we would like to have with these models? Would we want them to advise us? Would we want them to inspire us? Perhaps at Girl Geek X 2042, it’ll be a language model presenting about something new. Thank you.

Elena Chatziathanasiadou: Thank you so much all for joining. I guess with that note, I did want to mention that we’ll kick off mingling time and dessert in the area that we were before and our speakers will be available for you to ask them questions. We also have some of our recruiting team members here tonight. If you all want to come up to the front to just quickly introduce yourself or just say hi so that people can see you and then you can all come find us.

Elena Chatziathanasiadou: As I mentioned in the beginning, I’m Elena, I’m also hiring for the Residency program, so come talk to me, come find me. And then we also have some demo stands of our Dolly product and also our GPT-3, if you want to check them out. Jessica and Natalie will be doing those demos. So yeah, go find them as well.

Elena Chatziathanasiadou: Thank you all for being here. I hope you enjoyed it. Thank you to our lovely speakers and to Girl Geek X, to Cory and to all of our ops team and everyone who helped put this together and let’s go enjoy some dessert!

openai girl geek dinner networking
openai girl geek dinner organic straus soft serve dessert
openai girl geek dinner networking after talks
OpenAI Girl Geek Dinner

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Girl Geek X New Relic Lightning Talks + Panel (Video + Transcript)

Women at New Relic discuss observability, metrics, monitoring, community, APIs, React, and leadership at the New Relic Girl Geek X event with over 190 girl geeks joining the lightning talks and leadership panel discussion online.

Table of Contents

  1. What is Observability? Padmaja Gohil, Senior Solutions Consultant at New Relic watch her talk or read her words

  2. Customer Success and Value Realization Through Value MetricsKate Kordnejad, Lead Principal Technical Account Manager at New Relic – watch her talk or read her words

  3. How Browser Monitoring Can Be Used To Improve Website UX and UI? Carolina Rotstein, Solutions Consultant at New Relic – watch her talk or read her words

  4. DE&I – Finding a Community with New Relic ERGsSolmaira Flores-Valadez, Senior Technical Account Manager at New Relic – watch her talk or read her words

  5. Observability in the Age of Web3Nora Shannon Johnson, Solutions Consultant II – LATAM at New Relic – watch her talk or read her words

  6. APIs: Get Your Data When You Want It and How You Want ItSarah Hudspeth, Solutions Consultant at New Relic – watch her talk or read her words

  7. The Power of React.jsJo Ann de Leon, Senior Technical Account Manager at New Relic – watch her talk or read her words

  8. Leadership PanelAriane Evans, DEI Manager at New Relic, Nada Da Veiga, GVP, Technical Solutions Sales at New Relic, Erin Dieterich, Senior Director, Social Impact & ESG at New Relic, Kim Camacho, Director, DE&I at New Relic, Tracy Ravenscraft, Director, Technical Account Management at New Relic, Stefanie Smith, Senior Manager, Talent Acquisition at New Relic – watch the panel or read their words

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Transcript of Girl Geek X New Relic – Lightning Talks:

Angie Chang: We’re going to give people a chance to join us, but in the meantime, I guess I’ll start with some introductions. Hi. My name is Angie Chang. My pronouns are she, her, hers. I wanted to say, thank you so much for joining us for our Girl Geek X New Relic event. I want to encourage us to connect with each other. If you can, I would invite you to put in the chat, your name, your location, your job title company, and your LinkedIn URL, so we can all get connected. Feel free you to connect with me. I wanted to introduce myself and give you some background as to what Girl Geek Dinners is about.

Angie Chang: I started Girl Geek Dinners in San Francisco when I started working in engineering, and I felt a bit lonely on the team as the only female engineer. And I go to all these tech events, but I wanted to go to tech events where the gender ratio was flipped. These didn’t exist in 2008. I decided to start my own series of Girl Geek Dinners. It turns out, after five days of posting about online, we had over 400 girl geeks that were interested in joining us for our first Girl Geek Dinner. And then the next one was sponsored by Facebook. And then we just snowball from there.

Angie Chang: And now today we have over 200… I think we’re at 300 Girl Geek events. We’ve also started things like a virtual conference every year, celebrating international women’s day. We really have also filled out our product portfolio of this podcast. You can go on YouTube. All the talks that you will hear today will also be on our YouTube channel. I invite you to subscribe to that. It’s at And you can find all the videos from our previous events, and today’s event on there as well. [inaudible] chatting.

Angie Chang: I wanted to share how much I love learning from going to all these events over the years, because from listening to the women working in the diverse corners of male dominated industries, from engineering to sales, we have heard from people share their expertise. And we also learned things like, that job titles are constantly evolving. I remember thinking that this was a really interesting part of engineering and tech that we often don’t think about, of the first thought of big tech or tech companies.

Angie Chang: When I used to work at Hackbright Academy, a coding bootcamp for women, there was some women that I met at New Relic who were sales engineering leaders. And I thought they were so cool, because they not only knew engineering, but they were also very savvy on the business side. It’s because of the sales stuff. I remember thinking that this was a really interesting part of engineering and tech that we often don’t think about, of the first thought of big tech or tech companies.

Angie Chang: The sales engineering side is overlooked. I’m glad that we have heard from people like Tracy, and all the solutions consultants and technical account managers, who are interested in sharing the projects they’ve been working on and their passion for technology, today with us. We are excited to partner with New Relic, a company leading in full stack observability. We’ll hear from the solutions consultants. And they’re formerly called solutions engineers, sales engineers, and technical account managers. I think what I’ve learned is that solutions consultants are pre-sales, and technical account managers are post-sales, but that’s something that you can have a conversation with people about afterwards in networking.

Angie Chang: These lightning talks will be discussing observability metrics, ReactGraphQL for APIs and more.

Angie Chang: Now our first speaker on customer facing technical roles at New Relic is Padmaja Gohil. Padmaja is a senior solutions consultant at New Relic, and loves being a sales engineer, because it not only helps her stay at the cutting edge of technology, and she gets to work with a multitude of customers using these technologies. In her free time, she loves listening to music and adventure parks. Welcome, Padmaja.

Padmaja Gohil: Thank you, Angie. Hey, everyone. Very nice to you. Is everyone able to see my screen? Angie, can you just give me a thumbs up?

Angie Chang: Yay.

girl geek x new relic padmaja gohil observability code phrases

New Relic solutions engineer Padmaja Gohil talks about observability in software development, the phases of observability, and observability as code at Girl Geek X New Relic virtual event. (Watch the talk)

Padmaja Gohil: Okay. Awesome. I’m Padmaja Gohil. I’m currently a senior solutions engineer with New relic. Today we’re going to be talking about all things observability. Quick disclaimer, please not hold me accountable to any sort of overlooking statements. Before we dive into the presentation itself, I would like to give you guys a quick glimpse into my journey so far. Growing up, I’ve always wanted to be an engineer, but once I started my engineering degree, I realized that my interest lay somewhere at that nexus of tech and business, which led me to do my masters in engineering management, where I studied business concepts, but focused in high tech industry. I’ve also previously dabbled in consulting, venture capital and data privacy.

Padmaja Gohil: I’ve been a solutions engineer with New Relic for the last three years. I absolutely love what I do. New Relic is an observability platform, and because of which I’m going to be talking about observability today. But at the same time, in my day to day, I get to work with a lot of different customers. Understand how they’re using technology, and I help them achieve their goals using New Relic. If you guys have any questions about what solutions consulting, solutions engineering, sales engineering is all about, feel free to reach out to me on LinkedIn or my email address, and I would love to have a chat. The way I’ve structured the presentation today is, we’re going to talk about what were the changes that we saw in the software development space, that led to observability. Why do we really need it? What it is, and the different phases in which you can implement it.

Padmaja Gohil: And finally, we’re going to touch very quickly on observability as code. We’re going to be covering a lot of ground. Again, feel free to get in touch with me if you have more questions, or if you would like to learn more. Now let’s take a look at how has the model software industry evolved. If you look at the left, on the left side of this screen, you’re looking at our past. Our past was primarily Monoliths. They were stood up on on-premise servers. Usually scaled vertically, very static operations based scenario. We would release once or twice a year. I still remember the days when we would have to manually update our softwares. Now, fast forward to today. Today’s architectures are more microservice based. They’re open sourced. They’re more complicated. They’re usually hosted on Kubernetes cluster.

Padmaja Gohil: We went from releasing once or twice a year, to releasing maybe multiple times a day. This has been great in terms of the business. We’re able to push out new code, push out your releases and update our software faster, but it on-boards with it a level of complexity when it comes to troubleshooting, detecting issues and finding resolutions for it. This alongside other reasons is why we need observability. In the days of mainframes and static operations, when things went wrong, what would happen is, we would have maybe a couple of dashboards, that we would get alerted on. Usually these dashboards were static. We had run books for all of them, to figure out what’s going wrong and to fix issues. Now, typically, these systems would fail in the same manner over and over and again and again.

Padmaja Gohil: It was a little more simplistic than maybe today. Now, today if things were to go wrong, I’d be staring at my screen, wondering what’s going wrong. Is my cloud provider seeing an outage? Is someone deploying code? Is that the reason why I’m seeing some sort of an issue. Or I could be staring at the symptoms and not the root cause. There is so many ways in which things could break, that it’s really hard and complicated in how we do troubleshooting today. Also there has been an increased frequency of CodeDeploys. We went from once or twice a year, to multiple times a day, which can increase the chances of things going wrong. We no longer have discrete application owners.

Padmaja Gohil: We have distributed systems, but at the same time, we also have distributed teams working on things. There is a need for contextualized data in case of… if a person were to just come in blind, not knowing the history of the systems, they can quickly take a look at things and start fixing. These are just some of the reasons why we need observability today. But let’s take a look at what the definition is. There are a lot of definitions out there. The way I like to think about it is, how well do you understand your system from the work it does? It enables you to do a lot of things. For example, it enables you to collect and alert on the telemetry data types. There’s four telemetry data types, and these are the pillars of observability.

Padmaja Gohil: I’ll speak to those further in the presentation as well, but it’s metrics, events, logs, and traces. These are the four pillars of observability. Observability allows you to focus on your day to day. As software engineers, your job is to, let’s say, deploy code faster, come out with newer features. Your job is not to spend a lot of time in fixing issues. Observability also allows you to focus on that. It enables you to troubleshoot faster. It makes sure that you are ensuring up time and performance while you push out this newer code. It also gives you the confidence to push out new code, because let’s say if things were to go wrong when you were deploying, you have the confidence that yes, I have the system in place to fix those things. It builds that culture of innovation as well.

Padmaja Gohil: In real life scenario, there are so many different ways in which you can implement observability, but there are three phases, three broad phases in how we implement it. I would like to talk to you about it. The first phase is the reactive phase. All of us might have heard the saying that you cannot improve what you cannot measure. The first phase is where you start instrumenting your entire tech stack to collect data. You’re collecting metrics, events, logs and traces from all of the tech stack. You are then understanding how your applications are behaving. A lot of times you might not know what normal looks like for your applications. What does your normal response time look like? What does the normal error rate look like? The first phase is when you are establishing the normals and the baselines, and then you’re setting up foundational alerts on it.

Padmaja Gohil: That’s what the first phase is about. The second phase is now codifying your team’s work. Now, when I say that, what I mean is, you are setting up service level objectives for your application, because what happens is you’re seeing plethora signals coming at you. And you now need to understand how do you measure the success of your application? One of the ways to do that is by setting up service level objectives, and service level indicators, which are SLIs. Let me give you an example of what an SLO can look like. For a web application, an SLO could be that the videos should start playing within the two seconds, and 499% of the time during that one week period. That is your SLO. Now, the service level indicator, which is the SLI, measures the proportion of videos on the website that start playing in less than two seconds.

Padmaja Gohil: You start setting up these kinds of SLOs, SLIs. You measure them over time in the second phase. Now, lastly, the data driven phase. The ultimate aim of observability is to help teams within a company make data driven decisions. You are doing a lot of trend analysis of the SLOs and the SLIs that you set up. But at the same time, you’re evangelizing this to the teams beyond, let’s say, site reliability, DevOps, or application engineers. You’re pulling in folks from, let’s say, customer support, product. Everyone’s looking at the same data, and you’re making decisions. Eventually, you want to get to a stage where you can figure out, how is it that your digital operations are impacting business KPI. For example, if you were an eCommerce website, if the page load of that eCommerce website increases by, let’s say, 10%, are you seeing a drop in the number of users on the website?

Padmaja Gohil: Are you seeing lesser number of things in your card? These are the kinds of relationships you want to start visualizing and measuring. That’s the last phase of observability. One of the things of last phase, is also being able to automate processes. That’s where observability as code comes into the picture. Now, observability as code can again, mean a lot of things. It could mean that the way you are interacting with your observability platform, you’re automating it, but it can also mean Gitops, config as code, infrastructure as code, CICD. Whenever you hear these things, know that these are observability as code. Now, what we’re doing essentially here is that we’re taking some of the best practices from software development, and we are applying it to the operations world. Think reproducible builds, reproducible deployments.

Padmaja Gohil: You are automating processes, you are testing them. And you’re making sure that no matter how many times you run these processes, you’re getting the same result. There are a few things common as a part of observability as code. Firstly, observability as code, it’s literally code. So it does not have a UI. It is declarative. So you are specifying the exact state in which it should exist. For example, if you write a piece of code to create an alert in New Relic, you should be able to take that same code or a template, and then modify it slightly to create a thousand alerts. It’s also reproducible. You are reducing the amount of time you’re spending in managing your observability systems as well. The first thing is it’s declarative. Secondly, it’s versioned and immutable. Ideally, it should not reside in a shared drive.

Padmaja Gohil: Ideally, you should be using a get for it. You should be able to go back and figure out what were the changes made if things were going wrong. It should be versioned and immutable. And lastly, it’s pulled and reconciled automatically. Now, what I mean by this is that if you had created a dashboard in New Relic or in any other observability system, and let’s say one of your colleague comes to you and says that this is a great dashboard. I want to use it for my own needs. They can go ahead, take the dashboard, and maybe they modify it. Then you go into New Relic and you figure out that your dashboard is modified, and you won’t actually revert the changes. You can directly take the code, apply it, and you can get your original dashboard.

Padmaja Gohil: And now you can take the template that you used, or the code that you used, and you can give it your colleague, and they can use it to create their own dashboard. It’s usually pulled and reconcile automatically. There are a lot of solutions available for observability as code. I’ve mentioned some of these here. We also have our own templates for, let’s say, Terraform, in case if you guys are interested. Feel free to look at it in our docs page. But these are just some of the solutions that you can use to implement observability as code. This brings me to an end of my presentation. I know that we covered a lot of cloud. In case if you guys are interested in knowing more, feel free to reach out to me on LinkedIn or my email address. Thank you so much. I very much enjoyed speaking here.

Angie Chang: Thank you, Padmaja. That was really great. And thank you for leaving an email address so people can reach out to you with any questions. moving on to our next speaker. Kate is a lead principal technical account manager at New Relic. She comes with a background in helping customers thrive in their business with the latest software monitoring tools. In her current role, she partners with customers to help them with their full stack observability requirements. So welcome, Kate.

Kate Kordnejad: Hey, Angie. Hi, everyone. Thank you for hosting us. Give me a second to share my screen, and put it in slide mode. All right. I’ll be talking about customer success and value realization through value metrics. I’m just going to jump into a little bit of legal disclaimer, so don’t make any financial decisions based on our discussions today, and or any statements we make, and some proprietary copyright information. All right.

girl geek x new relic kate kordnejad customer success value metrics

New Relic principal technical account manager Kate Kordnejad talks about the evolution of maturity, TAM goals, maturity journey, maturity metrics & more at Girl Geek X New Relic virtual event. (Watch the talk)

Kate Kordnejad: A little bit about me. My name is Kate, and I’m a principal technical account manager here at New Relic. As TAMs, we are an extension to our customers teams. We help them with their full stack observability requirements. We want to make sure they see value, and we basically help them get enabled, follow best practices. We work as a trusted advisor with them. A little data point about me; I love working out. I love yoga, especially Bikram yoga. I love to travel, and I’m a data nerd.

Kate Kordnejad: Okay. Our agenda for today is going to be evolution of maturity, goals for technical account management, our maturity journey, defining maturity metrics, and how can you define maturity in your organization? All right. Starting off with evolution of maturity. In our evolution and journey, we found ourselves improving efficiency from four to five hours to one minute by automating our solution. I’m going to explain how we did this. As things evolved over time, we found our defined metrics to be meaningful. And we did find out more about our customer’s maturity, and how we can help them improve stickiness. For example, are they using custom attributes, or do they have data instrumented for more visibility? With our help, they started getting more mature within the platform. And we were able to identify the gaps, improve upon them. We did soon realize to deliver an observability platform value for our customers.

Kate Kordnejad: We needed to recognize value drivers and use cases, that actually deliver those business outcomes for each and every customer. For example, to improve customer experience, quadrant you see on the left hand side. We had to understand our customer’s business needs. Card abandonment, any association with an operational gap like card crash rates, were stuff that we needed to figure out. We identified the steps to maturity, is basically summarized in alignment. What that means is we need to align customer priorities to the observability value drivers. And agree on prescribed observability use cases, and then enable based on an agreed upon description work streams with the customer, and then finally, value realization. Reflecting on the business and the operational KPIs that we agreed upon during and prior to going through maturity. We actually evolve from just collecting metrics to quantifying metrics into meaningful business values, with a growth mindset, of course. We realize without having a continuous growth mindset, we won’t be able to evolve and improve our solution.

Kate Kordnejad: Our next thing is the goals that are for technical account management. Having an involved automated way to quantify metrics into business values, provides us leverage as TAMs. TAMs, as in technical account managers. We now have data to analyze customer usage, to reduce overall churn, by identifying any sort of gaps we have in utilization, by providing enablement based on usage, and engage platform users and drive valuable engagement by meeting them where they’re at. And directly communicating with our customers and being a liaison internally and a voice for our customers. And essentially, we want to reach value realization with them.

Kate Kordnejad: The next I want to is our maturity journey. Our journey basically started at looking at our platform per customer account, and literally eyeballing metrics we had identified as crucial to understanding and analyzing customer data. It was really hard to assess the pieces of the product they were using by manually assessing their usage and engagement. The normal customer metrics success wasn’t really working for us anymore. For example, if they were building dashboard, this wasn’t showing us the full picture, or the reason behind it that’s looking at their user behavior. It was very one-dimensional, and we didn’t really know if they were getting value out of it. We basically had to look deeper into the metrics, and then identify and associated with value drivers.

Kate Kordnejad: How do we define maturity metrics to get to that point? As a team, we basically start asking ourselves, what results do we want to see from this? Ultimately, what does a good maturity look like? And what does it look like for each product? We needed KPIs to show actual investments. For example, if we looked at our alerting product, we wanted to drive an alerting strategy, or potentially set our customers up with anomaly detection. Next, we had to break each product into maturity metrics. Initially, this was done manually through APIs and us eyeballing accounts, but after we broke down our KPIs by product, we had to describe a desired performance level, and determine how data is interpreted. We had to set up thresholds, place and score for each one, each of the metrics that make upper and lower limits of a desired performance.

Kate Kordnejad: This basically allowed us to understand overall maturity for each customer product using a heat map, and really made maturity pop up the page for us. Now that we had our results defined, maturity metrics chosen by product, we had to basically come up with a way to automate this. Our internal teams were able to automate the process, build out an app using APIs, grab the required data from accounts, and assess maturity. Finally, the last piece of the puzzle was to ensure we documented every single steps, our definitions that are associated with each of the metrics collected for further analysis. Our document includes a breakdown of the products, the metrics associated with it, and each and every single step you need to take to improve your score. From all of this, we want to cover, how can you define maturity in your organization?

Kate Kordnejad: It really comes down to three pillars. Goals and baseline. You have to ask yourself, what does maturity look like for your organization? Describe those intended results. Do you understand the alternate measures for those intended results? Then you move on to data identification. Have you identified any composite indices as needed? And do you collect any of the data right now? Is it accessible to you? And finally, business alignment. Have you thought about targets? Thresholds? Do you have a baseline that you can work with. And then finally, have you tied your maturity metrics to business values that deliver value realization? That concludes my presentation. Thank you for having me.

Angie Chang: Thank you so much for that talk, Kate. Our next speaker is Carolina Rotstein. She is a solutions consultant at New Relic. She is also an economist and political scientist that fell in love with programming and data, and is passionate about untangling holistic customer journeys across complex stacks, which she’ll be speaking about today. So welcome, Carolina.

Carolina Rotstein: Can everybody see my screen?

Angie Chang: Perfect.

Carolina Rotstein: All right. Oh. Today we’re going to talk about browser monitoring, and how it can help us improve UX and UI. Some safe Harbor information, a bit of housekeeping, some proprietary information, and just please don’t use this to make any financial decisions.

girl geek x new relic carolina rotstein browser monitoring ux ui

New Relic solutions consultant Carolina Rotstein talks about improving website UX and UI with real user monitoring at Girl Geek X New Relic virtual event. (Watch the talk)

Carolina Rotstein: A bit about me. I’m a solutions consultant for New Relic in the commercial E-sales team. I’m an economist and a political scientist, but I fell in love with programming and big data. I’m passionate about untangling holistic customer journeys across complex stack, and my most previous role included optimizing UX and UI for the gaming industry. And yes, we did collect a lot of data.

Carolina Rotstein: Today’s agenda, we’re going to focus on improving the website’s UX and UI, and using real user monitoring for this. Also we’ll cover why we should focus on UX and UI optimizations, and some of the metrics that we can use to do this as well as the metrics that come out of the box for New Relic and some other tools. And then an approach towards optimizing customer experience, including UX and UI, the traditional way and the enhanced way using big data. My peers talked a bit about observability maturity. At New Relic, we focus on data driven decisions. We want to have an approach with this framework towards taking data driven decisions.

Carolina Rotstein: Now, in this part, customer experience is closer to product and support. While it does have a lot of positive impact into how customer support, user product, and just impact on their KPIs. It’s mostly geared around design and product and development. Experience optimization, and a big portion of that is user experience. And also user interface optimizations are closer to the revenue. Even though it’s at the bottom of the the funnel, any impact that we might have into optimizing the experience, will have a monetary increase for the companies that we’re in.

Carolina Rotstein: First I’d like to talk about browser versus synthetics. We talk a lot about the jungle versus the lab. The jungle would be empirical data. So just every browser, every device, every location, and what your customers are using. The lab will be how we are tracking the health of the site just as we mapped it. We eliminate all the variables to just understand performance and solve problems quickly. This is done by synthetics. The jungle piece or the real user monitoring is once we deploy that application into the wild. So the users might take pads that we just did not foresee. For that we use browser monitoring. It’s an essential tool for user experience. It has a couple of places that we would focus on. Product usage, front end performance issues, content strategy, and in this case, websites, UX and UI.

Carolina Rotstein: I’d like to talk a bit about the metrics that we can use for this. This are not all the metrics, but I strongly recommend this as a start. Just for front end performance and monitoring, we have core web vitals, the user time on site. And it’s just user centric health metrics, such as throughput chart. For New Relic, we can divide what the time that it takes to load is split by the front end versus the back end. But for product usage, which is getting closer to that UX and UI, we track that funnel.

Carolina Rotstein: These are those conversion funnel related metrics that map to the business. These are unique to every company and every website. Those are success events, which can be form fills, video watch, purchase, and business classifications. These are custom metrics that we would map. Then we have all these audience insights such as device and location and vanity metrics. The vanity metrics normally come out of every tool, but they’re a great place to just look at your application, sort of like the Canary in a coal mine. Then for content strategy, we can see how users are navigating through the site, such as in the metrics that we would use, are link positions, most popular, previous page, the next page. And we also have pages report, such as the most popular pages, the time spent on site, how long it takes to load an assets. But we also can track audience insights. This can come from your BI data.

Carolina Rotstein: New Relic can take just any sort of data, but with other tools, you can certainly integrate it. This can be things that are a bit more robust, such as persona development, even the VIP level of your users, or the user IDs. And then very targeted towards UX and UI, and specific real user monitoring. We have the time spent on task, which will be the time before a user completes a success event. The ease to perform a task, rage clicks, which is just a user frantically clicking. Marketing funnels is a good one. In New Relic, we have something called the apex score, which is just taking into account the error rate and the load time to proxy some of the survey based customer satisfaction, traditional UX and UI metrics.

Carolina Rotstein: Now, very related to UI in just design, we have AB test, popular device sizes, screen size, and size orientation and night mode. Those are a few ones in there. Finally, I would like to show you what comes out of the box of New Relic. This is browser monitoring. We have those core web vitals that user spent on site, initial page load throughput, and some other additional charts. This comes just out of loading a user agent. It’s as simple as adding a marketing tag, and this dashboards just magically appear. But if we go back to talking about UX and UI, why is this important? It’s like 68%. I’m talking here about eCommerce just because it’s the easiest, cleanest use case to see revenue changes when you deploy UX and UI changes.

Carolina Rotstein: We can see that a lot of eCommerce… 68% of them just had performance issues. And that translates into 40% of those issues resulting in revenue lost. For instance, most eCommerce retailers have reported that they would like to have a response time below two seconds. 90% of that website response time, on average occurs due to do the loading of front end resources. This is why it’s so important to start your optimization with your front end as well. Some core customer experience questions that real user monitoring will help you solve is, for instance, if your website is easy and friendly to use, that would be through the balance rate, for instance. Whether it’s easy to navigate or not, and that will be through the number of pages that your users take to get somewhere.

Carolina Rotstein: Ideally, you want to slice all those additional page views, just because you want a seamless interaction with your site. Just think about loading a YouTube video and having to click 20 times before you go to the music that you want. For instance, how easy it is to get in touch with a customer agent for your user. Now, not all sites want you to immediately get those agents, but those are done through custom events. You track that chat click, or that phone call on a mobile browser, as a success event. And just how comfortable your visitors are after landing in your site, can be done through a number of other metrics that we track on browser. Why is this important as well? Just to do it via RAM, is because it’s big data and data driven.

Carolina Rotstein: So if we look at traditional UX and UI optimization, it’s done through user research, such as interviews, focus groups, usability testing. And they would put a couple of people to see how easy it is to finish a task on their site, through surveys, AB testing sometimes, and session recording. Now, the size of this data tends to be, from my experience, to 100 people, to a thousand people. When we’re talking about big data, it’s millions of people. It helps us prioritize and not get narrow focus on the people for which we’re auto… well, for which we’re optimizing too. That is done to AV testing. Some companies that are very developed, they do multi-variate testing. So they have several versions of the same design, such as… Netflix is one of the big guys at the same time. They’re just running algorithms while they’re doing that.

Carolina Rotstein: The way they pass that data into their systems, is via an integration to their either browser or mobile tracking. Same session recording, conversion funnel. This is just big data that allows you to ultimately do persona building. In the particular case of New Relic, this is fully integrated with observability. Now, just because I’m coming from New Relic, I will like to show you how easy it is to implement custom events and attributes, which are those business events and additional metrics that I was talking to. This is just an example of a JavaScript snippet, and this is how you would pass it into New Relic. It gives you that full stack of observability that my peers for talking about. And that’s it.

Angie Chang: Thank you, Carolina. Our next speaker is Solmaira. She’s a technical account manager at New Relic, based out of Atlanta, serving as a technical advisor for enterprise customers in Latin America. She currently serves as chair of the Relics of Color ERG, which she’ll be speaking about today. Welcome, Solmaira.

girl geek x new relic solmaira flores valadez erg community

New Relic technical account manager Solmaira Flores-Valadez talks about finding community with New Relic ERGs at Girl Geek X New Relic virtual event. (Watch the talk)

Solmaira Flores-Valadez: Hi, everyone. My name is Solmaira Flores-Valadez, and I’m a technical account manager at New Relic. I’ve been with New Relic for about over two and a half years. I serve as pretty much like a technical advisor to some of our larger enterprise customers within the Latin America region. I’m like a post sales resource to them, helping them get the most out of New Relic, and also providing trainings, things like that, to make sure that they are utilizing New Relics to the best of their abilities. Today I’m going to talk about diversity, equity and inclusion, and the part that it plays in my life. How I was able to find a community with New Relic ERGs, which are employee resource groups.

Solmaira Flores-Valadez: A little bit about me. My pronouns are she, her, hers. I live in Atlanta. I went to the University of Georgia. I am a first generation Latina. Mexican-American. First person in my family to go to college. I am a woman in tech, and I’m also a dog mom. First I wanted to start off with a few definitions around what diversity, equity, inclusion are. And then I’ll jump in and talk a little bit more about what it means to me, how I got involved, and all of that. Diversity is the presence of differences that may include race, gender, religion, sexual orientation, ethnicity, nationality, socioeconomic status, language, disability, age, religious commitment or political perspective. These populations have been and remain underrepresented within the broader society, and within practitioners in the field as well, within the workplace.

Solmaira Flores-Valadez: Equity is promoting justice, impartiality and fairness within the procedures, processes and distribution of resources by institutions or systems. Equity is really the approach to ensure that everybody has access to the same opportunity. In the context of the workplace, how is it that employees have access to the same levels of attraction, promotion and retention within the company?

Solmaira Flores-Valadez: And then lastly we have inclusion, which is an outcome to ensure that those that are from these diverse backgrounds actually feel and or are welcome. It pretty much boils down to people with different identities, feeling or being valued. All right. So why is it important to me? I actually started doing D and I type of work long before I even knew it was D and I work. As I mentioned, I went to the University of Georgia. UGA is a predominantly white institution. So there was very little people that looked like me. I was always looking for my community, people that looked like me, that share common backgrounds, but then at the same time, got involved with certain organizations such as Students for Latino Empowerment, that not only helped build that community, gave me that social aspect in college, but also we were doing D and I work [inaudible] students into the campus. We have various events throughout the year, where we would pretty much show them the ropes, let them know, if I can do it, you can do it. They’ll be able to tour campus.

Solmaira Flores-Valadez: We would give them workshops around financial aid, how to get started, the college journey and all of that. That’s what sparked I guess that interest in being involved within D and I type of efforts. As I have here, it’s important to me, for me to lift as I climb to, to be that change that I wish to see in the world. And also D and I has been very important in my life, not only because I’m able to in a way give back, but also it’s helped me in my professional, in my personal growth. Being able to develop certain leadership skills.

Solmaira Flores-Valadez: A little bit about how I got involved at New Relic. I was involved in college with those type of organizations. As I left college and I moved on to my professional career, my first job I worked at a big accounting firm. I got there similar to when I joined UGA. Not a lot of people that looked like me. They had an Hispanic network. I joined that. We did a lot of social events, but at the same time, we also did a lot of also lifting as you climb, bringing in students. We also did events for students and things like that. I loved being plugged in, being that person, going to recruiting events and seeing others like me, and then being able to see that they could also do it.

Solmaira Flores-Valadez: And then after that I switched careers. I came over to New Relic. As soon as I joined, I let my manager know that I was interested in still being a part of something like this. I asked if he had a Hispanic network. He told me he wasn’t familiar if there was an Hispanic network, but there were employee resource groups, and got me connected to the person that was the D and I manager at the time. Met with her. I talked about my experiences. She got me connected to the Relics of Color, which is the employee resource groups for our POC at New Relic. I got to meet them. Loved what they were doing, got to participate in some of their events. When I joined, it was right around Hispanic heritage month.

Solmaira Flores-Valadez: I asked if there was anything I could help with. At the moment, the Atlanta office was pretty new. There wasn’t a lot of representation there. I told them that I wanted to host an event. I was brand new. I didn’t know how people were going to take it, but I knew that I wanted to do this. Since it was a little bit later in the Hispanic heritage timeframe, I decided to give a twist. We did a day of the dead event, which I have some pictures here. I put this together. We painted skulls. And then we watched Coco, and we also ordered Tamales and we had a really good time doing this event. That was pretty much my golden ticket into not only being a member of the Relics of Color, but becoming an executive, one of the ROC execs. After I did that, the leader of the Relics of Color reached out and was like, “Oh, I want you to be part of the exec board.” And then that’s how I got plugged in my golden ticket.

Solmaira Flores-Valadez: I loved it ever since. I’ve been able to help with a lot. Currently, I’m one of the co-chairs of the Relics of Color here. Have our exec board of our offsite that we had earlier this year, where we got together to build out our strategy, the events. We host and we celebrate different events throughout the year, black history month, Hispanic heritage month, Asian Pacific Islander month. Putting together content around that. And then on the right hand side, that was us at the sales kickoff. We managed to get people together before 8:00 AM or at breakfast. It was great. It was intimate. We had our relative color sponsor. Tracy Williams, she’s our chief diversity equity and inclusion officer, as well as our chief people officer there. Being part of the Relics of Color, being part of the exec board, has been, like I said, great.

Solmaira Flores-Valadez: I’ve been able to learn a lot, gain exposure to different things. For example, we meet with our C-suite on a quarterly basis. Being able to have visibility into the C-suite. And not only that, but be able to represent the Relics of Color as a whole, be able to communicate some of our challenges, what we’re doing, where we want to get, our goals. And then listening to us and our needs and seeing where and what they can do to help. It’s been great. On the social aspect, I’ve made really good friends, but also helped me grow professionally.

Solmaira Flores-Valadez: I talked about the Relics of Color, but we do have other employee resource groups. We have the women at New Relic. We have the veterans at New Relic, and we have access at New Relic, which encompasses neurodiversity, mental health and disability. Relics of Color, which is the ERG that I’m a part of. And then we also have our Rainbow Relics for LGBTQ plus relics. These are the different ERGs that you can get involved with. At Relic, we are working towards a more perfect New Relic. These are some of the initiatives that we have going on. We definitely believe that inclusion means everyone. We want to make sure that we’re having some progress. We understand that there isn’t always… or there is always more work that needs to be done, but we do value the progress over the perfection.

Solmaira Flores-Valadez: These are some of the initiatives that we have at New Relic. To help us accomplish that we have, for example, the Mikey rule, named in honor of our departed team, VP of engineering, who was the executive sponsor of our first employee resource group, which was the Relics of Color. This Mikey role focuses on sourcing and hiring more relics from underrepresented groups. Whenever we have an opening, this Mikey rule kicks in. We also have leader-led action plans. These were started in 2020 by our founder, Lew Cirne. He challenged the company to level up with D and I leader-led action thoughts, and maximize the recruitment retention and career growth for underrepresented groups. And now it’s one of the top level organizational priorities across the board for every single part of our business.

Solmaira Flores-Valadez: We also have D and I working groups. Our company leaders, like I said, sit down with us, with the ERG executives, to ensure that our commitment to diversity, equity, inclusion, is put into practice around the globe. Just wanted to call out some of our progress that we’ve seen with these initiatives. We’ve definitely increased our BIPOC engagement. We’ve also helped reduce bias. There’s different trainings that our managers have to take, every year or every so often around bias. We’ve also reached pay equity. There was an analysis that was made a couple years ago, that took a look at the pay, and made sure that everyone’s pay was equal. There’s been a lot of progress lately around career mobility, where we’ve built a lot of mentorship groups throughout the different businesses, to be able to help the career mobility of our underrepresented groups. And then as I mentioned, you also have the Mikey role, which focuses on the recruiting efforts. All right. Well, that’s all that I have for today. Thank you all for joining. Have a great day.

Angie Chang: Our next speaker is Nora, who is a solutions engineer at New Relic, where she advises enterprise clients on their observability engineering practices to answer the what, how and why of system performance. Her research focuses on application of blockchain, and she speaks Portuguese, Spanish and French, and resides in Florida. So welcome, Nora.

Nora Shannon Johnson: Hi, everybody. How are you all doing? Well, I’ll assume you’re doing fine because I can’t hear you, but can you see my screen?

Angie Chang: Yes.

Nora Shannon Johnson: Awesome. Okay. Cool. Like everyone else said, I don’t know enough to make any predictions that you guys could invest in, but anyway, welcome.

girl geek x new relic nora shannon johnson observability web

New Relic solutions consultant Nora Shannon Johnson talks about observability in the age of web3 at Girl Geek X New Relic virtual event. (Watch the talk)

Nora Shannon Johnson: Today I’m going to talk about observability in the context of Web3. A little bit about me. Like Angie said, my name is Nora Shannon Johnson. I’m a solutions consultant, which basically means that I help customers answer the what, how and why of system performance. Outside of work, I love languages and linguistics. I love planting things, but everything I’ve ever planted has died, unfortunately. So still working on that. And skateboarding. Today we’re going to talk about applying the principles of observability to Web3, and what the specifics of monitoring blockchain technologies looks like. I took an interest in this because I work with a lot of financial services and eCommerce organizations in Latin America.

Nora Shannon Johnson: The integration of blockchain into their existing business operations is a big question for them right now, for reasons that I’ll get into in a few minutes. This is not New Relic’s main use case, but as a solutions consultant, a lot of times customers come to you with their data, their technology, and their business requirements, and say, “Make it work.” Which is my favorite part of the job, when somebody says, “How do you do this?” And I say, “I don’t know. Let’s figure it out together.” So this is an example of doing that. Over the next nine or 10 minutes, we’re going to talk at a very, very high level about what Web3 is, why we would care about monitoring it, what specifically we would be monitoring what we want to look at. And then a quick example of what that might look like.

Nora Shannon Johnson: To get started, what is Web3? Web3 is the name given to the idea, and idea is a very keyword here, of a new sort of internet that is built using decentralized blockchains. As a disclaimer, throughout this entire presentation, I’m describing the idea, not the reality of what may come to fruition. Again, Web3 is powered by the concept of… or by blockchain technology. Blockchain is a relatively new method of storing data online. It’s built around two core concepts, those being decentralized computing and encryption. The fact that it is decentralized, means that files or data is shared across many computers or servers, rather than centralized in a single server or group of servers. You might hear it referred to as a peer to peer network for that reason. The fact that it’s decentralized also means that it’s immutable. You can’t change data on the blockchain, because in order to do so, you would’ve to corrupt data on every single machine that’s participating in the network, which is just really not feasible when you’re looking at large scale blockchain like Ethereum, which is the example we’re going to use.

Nora Shannon Johnson: And then the fact that it’s encrypted, means that people can’t access it unless they have permission to do so, and you can give and rescind access as you choose. So why would we want to monitor Web3? Frankly, for a lot of the same reasons that we already monitor the existing technology, the web two technology, so to speak, in the same industries. for financial services, that’s eCommerce integrating with blockchain for payments. It’s important to know that this isn’t just like cryptocurrency exchanges. This is brands like Gucci and the Dallas Mavericks and Microsoft, Whole Foods, even Save the Children. They all accept one or more cryptocurrencies for payments. Across a ton of different industries, this is an important aspect of their technology stack. We’ve also got healthcare. One of the driving or the driving use case for applying blockchain technology to healthcare, is to restore the rights of data back to users or patients in this case.

Nora Shannon Johnson: You would be able to give or rescind access to your health records to a healthcare professional, organization at will. Whereas right now your test records or health records are held in a database owned by maybe some company like Quest. And you wouldn’t really necessarily be able to remove it if you wanted to. And then finally supply chain. Supply chain is arguably at the enterprise level, the most interesting use case, the most sought after use case for blockchain. Specifically the validation of providence or origin and authenticity. Using a public ledger like Ethereum, you could actually trace the roots of a product that you purchased, to ensure that it is in fact organic or fair trade, or even from a location that you believe it to be from, which is pretty interesting use case. There’s many more, but in all of these use cases, we’re talking about people’s privacy, their security, their wellbeing. Obviously, their financial assets.

Nora Shannon Johnson: The fact that data on blockchain is immutable and that it’s decentralized, doesn’t mean that it’s immune to failure or to attack. It’s simply is creating this new monitoring paradigm. What might we monitor on the blockchain? I’m not going to explain all these, but like people before may have said, the slide decks will be shared out. But you might monitor something like a decentralized application or dApp. Decentralized autonomous organization or DAO. Decentralized finance exchanges or DeFi. And then non fungible tokens, which I think everybody’s probably familiar with. The infamous apes, or NFTs. Lots of acronyms here, cause it’s a mouthful. And then of course, you can monitor the blockchain itself too. Which is the example that we’re going to look at. We’re going to look at the Ethereum blockchain. If you’re not familiar, Ethereum is, you guess it, a blockchain platform with its own cryptocurrency. Ether, shortened to ETH. It’s also got a programming language called Solidity, which you can use to write smart contracts, and decentralized applications so that you can actually interact with the blockchain.

Nora Shannon Johnson: This is by far, especially for DeFi, the most popular blockchain, but there are a lot of alternatives that are gaining popularity, things like Cardano and Solana, because they’re faster and cheaper than working with Ethereum. Monitoring a blockchain or assets that are deployed to a blockchain, is going to include both the typical metrics and data types that we’d be used to seeing, as well as some that are specific to this realm. There’s three example categories here. We’ve got system performance, security events and business metrics. If you go from left to right, this is like more familiar to less familiar. Something like system performance is something that we’re very seeing.

Nora Shannon Johnson: When Netflix is… well, all the time it’s up and running, they want to know how quickly transactions are executing, the rate of error, as well as resource utilization. The only difference being in a world of Web3, this might be the number of nodes, but very similar to what we see today in terms of the number of idle versus busy workers. For security, this is very important. We’ve all heard about many attacks made to different blockchain or cryptocurrency exchanges. Things like changes to access controls, when there’s a lot of failed login attempts from especially specific IP address or geographic location where you don’t normally have those. And then finally, unusual transaction patterns. So there being a lot of transaction outside of your normal business operating hours.

Nora Shannon Johnson: And then all the way to the right. And this is where we see things that are more specific to the use cases that I described earlier. Things like measuring the gas fee. When you interact with blockchain, you have to pay a transaction fee. And it’s a dynamic transaction fee. It changes throughout the day. We’ll take a look at that in a minute when we get into New Relic. But that’s something you want to pay attention to, because whether you are paying that or receiving that, that affects your bottom line. You’d also want to pay attention to things like, the number of active users or wallet holders, the number of active connections. And then of course, the number of minors that are mining. And then finally something like the rate of currency being paid out. As you probably know, miners mine, because they get paid for it. But there’s a lot of blockchain platforms, especially ones that are oriented towards the arts and culture, where they will actually pay you for posting your content to their platform.

Nora Shannon Johnson: They pay the content creators. You want to know, again, whether you’re paying or receiving or you’re somewhere in the middle. As an integrator, you want to know what the rate of payout is. How might we do this? We know what we want to look at. We know the importance of it, but how might we actually do that? We’re going to go through very quickly, the two pieces that fit together, and then we’re going to look at what that looks like in an observability platform. In any situation, there’s two parts to monitoring. There’s the data and the platform. There’s, how can we get it, it being the data. And then the second part is, how do we make sense of it? Because just having the data is not super helpful to anyone unless you’re a computer.

Nora Shannon Johnson: We might use something like, which is a Python library for interacting with Ethereum. You can do all kinds of cool stuff. You can read data, you can send transaction, you can even set the gas price if you are the owner. Not the owner, but you’re responsible for the operation of the blockchain. And so we see on the right hand side, we can import the Web3 library, confirm that we’re connected to Web3. And then as the last example shows, we can read block data or look at different people’s wallets. Here I’m pulling Snoop Dogg and Paris’s wallet balance. Which again, we’ll take a look at in a minute. This is the part of how we’re getting it. if you’re more of a fan of JavaScript, there’s also a Web3.js library that you can use instead. I’m just a Python loyalist.

Nora Shannon Johnson: And then the second part, which is, how do we make sense of it? Using a wonderful observability platform like New Relic, we can use an API suite to pull in the metrics, events, logs and traces that are important to us. And then you can look at all of the block statistics for the last hour, week or month or whatever the case may be. Let’s take a look at what that might look like if we were actually quoting it over to… oops. [inaudible]. I didn’t need to unshare. Let me reshare. Quoting it over to an observability platform. This is a simple use case, but this is just basically a dashboard that I put together. We’ve got our cool little… I don’t own. This guy, unfortunately, no Ethereum funds for that.

Nora Shannon Johnson: But we can look at changes to the different whales. Here I’m tracking people like Snoop Dogg. He’s buying and selling all over the place. We’ve got Lindsay Lohan, Mark Cuban. We can see the top miners for the period of time. The gas price, that’s what I mentioned earlier. And then again, the black activity, which is interesting to see. Whether you are responsible, you’re part of some exchange that is leveraging the Ethereum on blockchain, or you are a decentralized application developer, or maybe you’re just somebody that is posting their content or their NFTs to a blockchain. This is all going to be relevant for you. You can also make actionable insights based on the data that you poured over. If we take a look at, for example, logs. Logs are just like step by step data coming from either applications or servers or what have you.

Nora Shannon Johnson: And in this case, it’s coming from the Ethereum blockchain. I’m going to filter on a certain block number. Let’s do this guy. We can actually see. I’m going to shut this down. We can actually see the step by step of what it looks like. We can see that transaction was requested, that the block creation was initiated. Blocks were then sent to nodes in the network. It got validated. Transaction is now complete. And then they update that to… or they send that update to the network. The block gets added to it, and the proof of work is dispersed. People get paid on it. But as we know, things do not always work as we anticipate that they will in technology. So if we take a look at this other block, we can see that in this example, the transaction is requested, the block creation is initiated. The block is sent to the nodes in the network, but then the transaction is pending validation several times.

Nora Shannon Johnson: We might do something like create an automated remediation workflow here. Based on maybe the messages, the strings in this message, or repeated data, or examples of the same messages over a long period of time, we could actually set it up such that it automatically triggers external events based on what we see in the log messages. Again, this has been a very quick and very high level example of what you might see if you wanted to monitor something on the blockchain, and how you could make use of that information in a wonderful observability platform like New Relic. I hope you enjoyed it. Thank you very much for your time, and I hope to talk to you all soon.

Angie Chang: Awesome. Thank you, Nora. So Sarah is a solutions consultant at New Relic. She loves working with data, and in a previous life was a math teacher. She uses her skills to help customers use their own data to improve their uptime, performance resilience, reliability, and customer experience. Welcome, Sarah.

Sarah Hudspeth: Okay. Hopefully you can see me and my slide.

Angie Chang: Yes.

Sarah Hudspeth: Are we good? Okay. Hi. All right. Hi, all. I’m excited to talk to you all about APIs, and getting your data when you want it and how you want it. It’s a very common theme here at New Relic. We love data. We’re data nerds. And we have a safe Harbor just for legal purposes.

girl geek x new relic sarah hudspeth apis graphql

New Relic solutions consultant Sarah Hudspeth talks about REST APIs and GraphQL queries at Girl Geek X New Relic virtual event. (Watch the talk)

Sarah Hudspeth: A quick bio about me. Yes, I’ve probably been in tech for three and a half, four years. Before that I was a math teacher. I taught middle school and high school math. I did attend Hackbright Academy, and so I’m a boot camp graduate. If you have any questions about that, please reach out. I’m a mom of two, plus I have a puppy, a lab mix, and then you can see the hamster in the background.

Sarah Hudspeth: I’m a huge reader. And you’ll see one of the projects I walk you through is all about books. The last book I read was Stalingrad, super interesting. The best part of my job is working with customers and helping them solve their problems. And yes, we are all about data and using data. Feel free to put stuff in the chat or follow up with me afterwards. Here are my objectives for my talk. I am still a teacher at heart. I want you guys to understand what REST APIs are, how they’re used, what is GraphQL, and what are some interesting trends in APIs today? I want you to understand the difference between your REST APIs and GraphQL APIs, and possibly articulate use cases for each. Also we’re going to be talking about GraphQLs, query and mutations.

Sarah Hudspeth: I’m just making sure you understand the difference between that. And then I am giving you all homework. After this session, if you haven’t played with API calls, go find some APIs, play with them. Go play with GraphQL, do some queries and mutations. If you need a GraphQL API Explorer, New Relic, you can sign up for a free account and play with our GraphQL API, which we call NerdGraph. Feel free to do that. APIs. API stands for application programming interface. It’s basically a way for clients and servers to talk to each other. It’s a set of protocols and it’s called a REST. I’m going to be talking about REST APIs, because those are usually the ones I would say they’re the most popular. And REST is short for RESTful, meaning stateless.

Sarah Hudspeth: The state of the client doesn’t affect the state of the server. They should be able to talk no matter what’s going on within their own environments. I like to think of API calls as programs, throwing Frisbees back and forth. Even though the Frisbee is actually data. But a client will make a call, throw the Frisbee to a server. The server gets the Frisbees if there are any instructions, and throws the Frisbee back as a response. If all goes well, you get a 200 response. If it doesn’t, you’ll get one of the four hundreds or five hundreds based on whatever the errors are. Let’s take a look at what an API call looks like. This is code from my virtual bookshelf project that I did at Hackbright. I allowed folks to build out this visual bookshelf of the books they were reading.

Sarah Hudspeth: The main API I used was Google’s Books API, where I could get a thumbnail of a picture of the cover of the book, and a lot of information. When I was feeding my database, I had a list of titles and authors, and then I made a call to the Google API, Book API, using my Google API key. I used the Python HTDP request library to get that information. And then I stored the response in a dictionary in JSON form. so that I could fill out my database with all sorts of interesting things. Hold on. I was going to say, there’s a few things. We have a URL. I have some variables in here, parameters that changed that I had to go through in a for loop.

Sarah Hudspeth: And then I also needed permission to have access to APIs. Those are the key components of an API call. This was one book. This was one response I got back from Google’s API. There’s a lot going on here. I would say, there’s a lot of information here, some of which I needed, a lot of which I didn’t need, and I had to sort through it and figure out, what is going to be helpful to me in my project, and then get rid of the rest, which if you notice, is a lot of waste. My code was not optimized. This was the slow part of my program, which if I go back, I would focus on this and try to do this in a better way, just because it ate up so much [inaudible].

Sarah Hudspeth: To summarize, I showed you what the components of the REST API and the results are. You have to have a URL. You have to call to someplace. You can send parameters on variables. I did title and author. You usually need a key to access the APIs, so you have permission to get the information. You need some HTTP requests. I use the Python’s request, but I’ll show you a cURL snippet when I do the GraphQL. The other interesting thing to note is that each API, you can call various APIs, will have their own way of formatting the data. Google Books API – just sent me everything I could possibly need about a book. And it was up to me to go through and figure out the structure of it.

Sarah Hudspeth: I showed you a get REST API call, but there are also posts where you can actually post data to the API. You can update data or you can delete it. I said, “This is kind of ugly data.” There was a nested JSON. I found out the hard way that sometimes some of the things I wanted were empty, and I had to find workarounds. I had to go and I had to clean up and structure the data. There even updates and the data would get restructured and I’d have to go back and figure out how to do that. I’m glad to now transition to a new way to get data, called GraphQL. It is also an API, but it is a very structured way we can access data. This is an example of New Relic’s NerdGraph API Explorer.

Sarah Hudspeth: And if you notice, I have to my left, a query builder with very specific key value pairs that I can build out for a query. Here I’m going to query an account and get the name and ID, and here I’m going to do an entity search. You all have been hearing us talk about observability, and learning about applications and performance and getting metrics and events. This is a way you can go in. I’m just going to get the name of things I want to monitor, the type. I’m going to get a special GUI. And then I’m just actually going to get the tags that I’ve tagged with my entity. It’ll pop up here in a very nice structured JSON. I know exactly how many levels I need to go in to get specific information. And then here’s how you could do it and build in a program.

Sarah Hudspeth: We talk about automation and observability as code. It’s really easy to take these GraphQL calls, and build in structures and processes to get the information that you can then take action on. Again, here’s just the API link. I have some headers with my key. And then here, I’m sending this query that’s going to go to the GraphQL server, and pass back all this information about this application, name, box, that’s in development. All right. Let me quickly summarize what we did or what I just showed you with GraphQL. Instead of posting or getting data, we’re going to query data and mutate data in order to update it. You might see that you can use GraphQL iteratively. I had that GUI ID that I could query for and then use it to change of I needed to update the application, add it to an alert policy, add it to a dashboard.

Sarah Hudspeth: It’s nice that you can just build off each other. I know exactly what data I’m going to get, and I’m only going to get that data. It’s going to be nice and structured. It’s going to be fast. I’ll tell you right now, New Relic is powered by this NerdGraph which you saw. That data that we accessed, we inside our platform also use it to access… or to build out all the dashboards and charts. I should say that GraphQL was developed by Facebook in 2012. Obviously when you’re processing that much amount of data, you want to be specific about the data you get, and get it as quickly as possible. The one downside is it does require a lot of upfront work. You have to build out that data schema so that folks can get the access.

Sarah Hudspeth: But once you have it built, you have a very powerful GraphQL engine. There’s some other cool things. I was going to say, with my API call, I had to call it many times in that for loop, because I could only get one book at a time. In GraphQL, you can make multiple calls even to multiple servers to get multiple data requests. It’s just a lot more robust and flexible. I’m quickly going to go through this slide. I think from the other talks, you’ve seen how we use data and how we want access to data and how we want to build it out programmatically, and automate and really be able to empower our data to… or empower our customers to use their data in a lot of different ways. Some of those are alerts. Getting alerted on any issues, updating with microservices and Kubernetes. You can spin things up, spin things down. You need to add them to alert policies or delete them.

Sarah Hudspeth: I also work with customers a lot about either storing or dropping data they don’t need. Sometimes companies need to store their logs to be in compliance with certain data rules. And so we can export data rules and NerdGraph to AWS buckets so they meet that requirement. We did talk about dashboards and S… or others talked about dashboards and SLOs. You can update dashboards with GraphQL. You can add things, you can subtract things. You can actually have a call to get a PDF. So if you need to email it to your superior and be like, “Hey, look at our application performance for the week.” You’re able to do that with a GraphQL API call, and then synthetics as well. If you want to check on Ping Checks if anything’s failing, or if you need to update, add end points, you can all do that in GraphQL.

Sarah Hudspeth: I think I’m good on time. I was just going to quickly show you how you can build out the query in the query builder. Let’s see. Maybe I’ll get the synthetic monitor. If I just wanted a list of synthetic monitors, I could just click whatever I wanted to see. I could add here. And when I press play, it just comes up to the right. I did add a permalink. So if maybe there was something I noticed, it was a critical learner. When I wanted to go check it out, I could quickly copy and paste or build out a script to go into New Relic and see what was happening. Looks like this check is okay, but I can go in and get that view. If I wanted to mutate, I could just continue to build out.

Sarah Hudspeth: Let’s say I wanted to create a workload. I could build out a workload using whatever data here. You can use the cURL up here. You could use our New Relic command line interface. It’s really flexible and robust. For all the data nerds out there, it’s just really fun to use. That was my talk. Hopefully you picked up a lot or a little about REST APIs and GraphQL and the differences. Just wanted to let you know, my team is hiring, so please reach out. Tap me up if you have questions, but thank you for listening.

Angie Chang: Thank you, Sarah, for the talk and demo on GraphQL. It’s very informative. I’m sure people have lots of questions will like to connect with you. So thank you so much. Our next speaker, we’re going to try Jo Ann again.

Angie Chang: Jo Ann is a senior technical account manager at New Relic. Has been working directly with customers, helping them use and implement the New Relic platform, including best practices. Prior to that, she was a solutions architect at Delta Airlines in Atlanta. So welcome, Jo Ann.

girl geek x new relic jo ann de leon react js

New Relic senior technical account manager Jo Ann de Leon talks about programmability, React, Nerdpacks and much more at Girl Geek X New Relic virtual event. (Watch the talk)

Jo Ann de Leon: Thank you, Angie. All right. Hello, everyone. I am Jo Ann de Leon, and I will be talking about the power of ReactJS and how it transformed the New Relic platform to be an open connected and programmable platform. Before I get started, I’d like to share some tidbits about myself. I am a senior technical account manager. I have been with New Relic for three and a half years, working directly with customers, acting as a technical advisor and solutions architect, to help them implement their observability use cases. I was born and raised in the Philippines. I graduated with a math degree, but never really thought I’d work in the IT industry. But in the past 20 something years, I have worn a lot of different IT hats, including a software developer, a designer, architect and project manager. Outside of work, my wife and I enjoy traveling, playing bocce, and cuddling with our two adorable orange tabbies.

Jo Ann de Leon: For this talk, I will introduce the concept of programmability. Show where you can find some of the open source apps and custom visualizations. And finally do a quick demo of how you can build your own. In a nutshell, programmability is about giving engineers full access to the New Relic database engine, and the building blocks they need to consume data in ways that solve their unique business problems. It also means giving our engineer users and customers the same set of tools our own engineers use to build our platform key rated experiences. What does this look like?

Jo Ann de Leon: At its foundation, is the telemetry data platform, that is able to ingest not just the data from the New Relic agents, but also from integrations that support open standards such as open telemetry. On top of this data platform, is a series of scalable services such as GraphQL APIs, as well as the developer tools, such as the software development kit or SDK for short, and the command line interface, or CLI for short, that allow you to access and interact with the data. Finally, a user interface is built on open source React JavaScript, with a flexibility to support the development of custom applications and visualizations. If you’re like me, I find it really helpful to look at what others have already created before I try to write a piece of code. It’s a good idea to explore what is already available in the open source community, as it may help inspire you to build your own New Relic custom application and visualization.

Jo Ann de Leon: The first place to explore is the New Relic Instant Observability or IO, which you can find via the apps icon in the New Relic toolbar. It contains a catalog of public apps and visualizations that are maintained by New Relic, and can be managed via the UI. The catalog also allows you to manage your own custom apps. You can find a number of other open source apps and visualizations in the New Relic open source website. The great thing about open source is that these apps are extensible, meaning you can customize them to fit your needs, and you can easily install them via the CLI.

Jo Ann de Leon: Here are a couple of examples that I wanted to showcase. The first one is a cloud optimized application, which analyzes your cloud environment, figures out where you’re wasting money on excess cloud capacity. The application compares the size of your instances to their utilization, finds resources that are sized larger than needed, and estimates how much you could save by optimizing the resource size. The browser analyzer app displays an analysis of performance, and forecast how improving the performance of your website can impact your key performance indicators, such as bounce rate or traffic. It also figures out which individual site pages have the worst impact on performance, so you know where to start making fixes and improvements.

Jo Ann de Leon: A popular visualization is the status widget pack, which contains three types of visualizations. One of those three is this status timeline widget, that allows you to display how your services are performing over time using traffic lights as visual indicators. Now it’s time to build our own app.

Jo Ann de Leon: I will show you how to build a Nerdpack, which is the deployable package of an application containing all the source code and resources required to run it. It is basically a collection of React components, including launchers, nerdlets and virtualizations, all structured into a JavaScript app bundle. A launcher is a declarative file. It allows you to configure your application’s name and description, as well as which nerdlet within the nerdpack to run when it is clicked. An application is made up of one or more nerdlets, which are renderable views or windows. So they can link to each other or be launched by launchers. And finally, a visualization is a custom view or widget that can be added to a dashboard. Similar to nerdlets, it can display data whether it’s from New Relic or an external data source. You can find them via the custom visualizations app.

Jo Ann de Leon: All right. In an alternate universe, I have open a number of cat cafes around the country, where I serve coffee and cute cats or lunging around to entertain my customers, who may then fall in love with them, and decide to adopt them. In order to achieve my goal of helping these cats find their forever home, I need to keep track of how many have been adopted, and how many are still up for adoption. I went ahead and sent this data to New Relic, but how do I visualize all my data since I have so many cat cafes around the country. Luckily, I can build an awesome nerdpack. So let’s go ahead and create it.

Jo Ann de Leon: I am in the New Relic homepage. I hope you can still see it. In the New Relic homepage, you can go to the apps and click on build your own app. You can follow these instructions in the quick start. If you haven’t already done so, you can create an API key in your New Relic account, or select an existing API key. This is where you can download and install the NR One CLI, and make sure that it is up and running. And then the last step before you build your nerdpack, is to save your credentials. Let me copy this, and we’ll go ahead and create the package and run it. I am going to name my nerdlet as cat café tracker, and launcher as cat café launcher.

Jo Ann de Leon: Install the dependencies and create all the different components needed for my app. And then I can go to that NerdPack and let me open this in my Visual Studio Code. All right. Let me open the Terminal here, and then I can run my server through the New Relic One CLI, with this command: nr1 nerdpack:serve. All right. You will notice that now you can run with nerdpacks=local. This means that any local development you make can be tested in the New Relic platform. You’re also given a shortcut to the launcher, which will open your Nerdlet directly. So let’s go ahead and copy that. And let’s go back to the browser here, and let’s close this prompt.

Jo Ann de Leon: And now we have our Hello World version for our cat cafe tracker, but it’s not really very exciting. Let’s go back to our code. For the sake of time, I will be copying and pasting my code, including index.js. This code will contain the logic to retrieve the data from the database and display it in two views, a table view and a map view. Let me go ahead and do that.

Jo Ann de Leon: And then I also need to update my styles.css. This will contain styling elements for my custom UI. All right. Third one. I need to update my package.json dependencies, because we will be using the leaflet package to create a map. All right. And then finally, I need the webpack config, which we will need to support the use of map tiling information data from leaflet. This will be copied at the root folder of our package. All right. Let me save all of that. I have to restart my server. Let me clear that. I have to do an npm install first, since I had to update my package.json. And now I’m going to restart my server and relaunch my app. All right. Let’s copy that new link. Go back to our browser.

Jo Ann de Leon: Hopefully this will work. There we go. All right. So now I can view all my cat cafes around the country. I created my visualization such that the size of the circles indicates how many cats are available for adoption in that area. The bigger the circle, the more cats are available. The green color means more cats have been adopted, while those that are yellow or dark orange means we have some work to do to get more cats adopted. Finally, I have also displayed my data in a table view to the side of my map. All right. I hope you have enjoyed this quick demo on how programmability through the use of ReactJS can help you create visualizations that focus on solving business problems. Please feel free to connect with me through my email or LinkedIn. Thank you.

Angie Chang: Thank you, Jo Ann, for that talk and demo, and we’ll be sure to connect with you on LinkedIn. Our leadership panel will talk about New Relic culture, inclusion, career development, and successful interview prep.

Angie Chang: Our moderator today is Ariane Evans. She’s a diversity equity and inclusion manager at New Relic, working with the talent acquisition, hiring managers, employees, and external organizations to recruit, engage, develop underrepresented communities. And she co-leads the Relics of Color ERG. Welcome, Ariane.

new relic girl geek x panel discussion

Ariane Evans moderates New Relic leadership panel with Nada Da Veiga, Erin Dieterich, Kim Camacho, Tracy Ravenscraft, and Stefanie Smith. (Watch on YouTube)

Ariane Evans: Thanks, Angie. Hi, everyone. My name is Ariane Evans. And as Angie mentioned, I’m a [inaudible] manager at New Relic. I love that I get to spend a little time with you and facilitate a conversation with some of our incredible leaders at New Relic. All of them, women. It’s so inspiring to have leaders that are not only passionate about their work, but the communities that they work within. Before I dive into the questions to know more about New Relic and the areas of expertise of each of these leaders, let’s go through a quick lightning round of introductions. Please give me your name, title, and a sweet little fun fact about you. Let’s start with Kim.

Kim Camacho: Hi. Hi, everyone. Happy pride month. My name’s Kim Camacho, and my pronouns are she and her. I’ve been the director of DE&I at New Relic for about a year, and have also about 20 years of DE and I and HR experience. A fun fact about me is, I met Barack Obama right after he announced his candidacy for presidency a long time ago. So that is fun fact

Ariane Evans: Very cool, and also now very jealous. Let’s go ahead and hear from Erin.

Erin Dieterich: Very jealous of that fun fact. Hi, I’m Erin Dietrich. I lead the social impact and environmental, social and governance organizations at New Relic. My pronouns are she and her. I’ve been at New Relic for about four and a half years, and I’m based in Portland. My fun fact is that I have two small children, a one and a half year old little girl and a five and a half year old little boy. And they keep me incredibly busy, and very tired all the time. I don’t think I’ve slept well in five years. Fun fact.

Ariane Evans: Well, you look great even on little sleep Erin. Thanks for joining. Let’s hear from Tracy next.

Tracy Ravenscraft: Hi, my name is Tracy Ravenscraft. I’ve been here at New Relic for about five and a half years. I run a technical account manager team in central. My fun fact is I have two dogs, one Pomeranian, one Pomsky, and they have names like Friends characters, so their names are Phoebe and Ross. Thank you.

Ariane Evans: Love a good Friends joke. Let’s hear from Nada next.

Nada Da Veiga: Hi, everybody. I lead customer adoption organization. America’s customer adoption organization here at New Relic. Been here for five years. If you’re wondering what customer adoption is, basically, all engineers that work closely with our customers, helping them learn how to use our platform to solve their technical and business problem, basically. Fun fact: throughout my life, I have had five different passports. So no, I’m not a female version of James Bond, but that’s what my husband likes to think.

Ariane Evans: I might also think of a reference to Carmen Sandiego. Where in the world is Debeka? Where is she going next? Next let’s have Stephanie.

Stefanie Smith: Hi. Thanks, Ariane. I’m Stephanie Smith. I’m based in Massachusetts, I’ve been with New Relic for six years. Currently senior manager of talent acquisition. My team supports go to market customer adoption. Let’s see. Fun fact about me is I have two teenage daughters, one of which just graduated high school last weekend, which is very hard to believe, and a younger one. She’s a sophomore, she’ll be a junior. Erin, the exhaustion doesn’t stop. It only gets different. It’s bigger problems with bigger kids, but it’s all worth it. Fun ride for sure. Excited to be here.

Ariane Evans: Thanks, Stephanie. And thank you all. We all just listened to quite a few talks learning about why observability is important. What is monitoring? How do we implement these different products and technology? And also this happens inside of a company where the people work together. There’s a culture that allows us to do that work at our best and highest potential. I’d love to hear from each of you on how you are not only living those in practices, but working that out in your teams and your strategies at New Relic. I will start with a bit of our culture and understanding how is New Relic creating a culture where people from all backgrounds feel included.

Kim Camacho: All right. I could take a step at that, Ariane. First and foremost, I think we are very clear about our commitment to diversity, equity and inclusion. We communicate our vision, mission and objectives annually as we build out our short term annual plans and our long term strategy. All new employees and interns hear about our strategy as well as our organization when they onboard. We measure regularly how employees are feeling. The extent to which they feel belonging and respect to the company. So important to do that. I think also for our employees, one of the big things that’s really important is having communities of people that you can bond with, that are recognizable to you and have the same interests and backgrounds that you have. We have employee resource groups at New Relic. They’re fully funded and have leadership organizations as well as executive sponsors.

Kim Camacho: It’s through these organizations that we hope that people are building relationships, bonding, getting to know each other outside of their regular roles. In addition to our ERGs, we have other slack channels based on whatever people want to connect with. Whether it’s dogs, bunnies. There was one that was just started on crime channels, which I’m in love with, so you can bond. The last thing I’ll say, as it relates to really creating a culture where people feel connected is, the importance of managers. I think as our audience will know, your manager makes a big difference. Here at New Relic, it’s really important that we support train, help our managers really understand cultural competency, how to build a diverse and equitable workplace. Everyone I think on this call knows, because they’ve been through some of our trainings and are actively involved in these efforts. It’s just really important that we’re working with our managers so that they understand their role in helping create a nurturing environment for our employees. That’s a little bit about from that perspective.

Ariane Evans: That is all really cool. I know that there are also more things that New Relic is engaging. Erin, maybe you can tell us, what is New Relic focused on, or engaging our employees and social impact.

Erin Dieterich: Yeah, thanks Ariane. is the name of our social impact work. We started it in early 2019, and really committed at that point to this mission of, how do we as a company, continue to push for more equitable access to technology? We really believe that accessing not just physical technology, having the best computer, having the best SaaS tools, but having the access to understanding what technology careers actually look like, what kinds of roles there are within technology. That is such a critical piece to creating this more equitable future for the industry, and to thinking about, how do we help people all along their learning journey? Whether they’re somebody who’s had a couple careers already, and are starting a career in technology, or a student who’s early in life thinking about what they want to be when they grow up. How do we give all of those people access to our incredible employees, so that they can hear the stories about how we all got where we are, and be able to start seeing themselves on this whole rainbow of pathways.

Erin Dieterich: It is not just one clear, point A to point B gets you a tech career. There are so many different ways to get where you’re going, and so many different destinations along the way. And so we’re just really passionate about infusing that into everything we do in social impact, and thinking about how we take the 2000 plus employees around the world with us on that journey. Some of the ways that we do that, we have a bunch of benefits that all of our employees get access to. They get to have 16 hours of paid time off to volunteer a year, plus we now have a set global day of service every winter. That’s three full days of volunteering, and you can slice and dice that however you want throughout the year. We incentivize our employees using that volunteering, by actually giving them dollars that they can push towards their fair charities every time they log their hours of volunteering.

Erin Dieterich: We have a $200 a year matching program. Employees can get up to $200 a year matched to any number of global charities. I think there’s 20,000 charities that they can pick from. And then we do a bunch of special campaigns. And so some of the things I really love that we’ve been building and you’ve actually been a big part of building these with us, Ariane, are some of our partnerships with our employee resource groups. Where we’re really going to our employee resource groups and helping them give us the understanding of where they want to impact in their communities, what organizations they want to work with. And then working together to make sure that that information is accessible to our employees, to incentivize and point them towards making really smart decisions with their wealth of how they can build this more equitable future.

Erin Dieterich: A great example of that is, since it’s June and it’s pride month, we are working with our rainbow relics ERG and just launched a $25,000 additional matching campaigns. In addition to those $200 employees have, they now can also put additional dollars towards this matching campaign, that goes to five different organizations that our rainbow [inaudible] helped us identify and pick in their communities. Organizations that they really care about that are helping the LGBTQ community with all of the different things going on, both in the US and abroad. Being able to be a part of understanding what that ERG community wants employees to support, and then helping employees understand how they can use their dollars to support their fellow relics, and the things they care about, is something that just makes me so excited.

Erin Dieterich: I just love seeing the way our employees are supporting each other through those special campaigns. I think I’m almost out of time, but I’ll tell one other very quick story, which is, since we have so many technical and inspiring folks on this call, I always like to take the opportunity to just pause and remind folks how valuable your skills are. Technology skills are so incredible. There’s a myriad of ways you could apply those to social good. Something we love to do is partnering our employees up with our nonprofit customers who get to use expanded access to New Relic for free. But we know that they need help with enablement. And so we partner them up with employees and the employees take on pro bono volunteering projects, where they’re using their technical skills to really support observability in nonprofits.

Erin Dieterich: And so you don’t have to be a New Relic employee to do something like that. You can really step back and say, “What causes are super important to me? What organizations do I love?” Reach out to them and say, “I want to talk to whoever’s running your technology, and see how I can be of support. I have X, Y, Z skillset that I’m really proud of. Is there a project I could help you on pro bono, and volunteer and support your organization, building your digital environment?” Because that is what every organization needs in order to power their mission. Every person with technology skills has just so much that they can give back. And so we love to do that at New Relic, but I also just love to encourage anyone anytime I can, to think about how you can use your skills out there in your community to power the charities and the causes that you care about.

Ariane Evans: Thanks, Erin. It sounds like New Relic is really building out a culture for people to live a life fully as they’d like, both internally and their communities. The things that they care about, but also themselves wholly. I’d love to hear from you, Tracy. Describing to us, which areas of your life would you like to spend more quality time when you think about work life balance.

Tracy Ravenscraft: That’s a great question. Thank you. When I think about where I like to spend my time outside of work, definitely with family and friends. Everybody wants that more family, friend time. But not only just spending more time with them, being present. Not checking my phone for slack messages, going on vacations and being able to completely disconnect. That’s what New Relic has brought to my life.

Tracy Ravenscraft: I’ve been at New Relic for five and a half years. I did site reliability in the past, network administration, network engineering. I never realized how I wasn’t there. I’m always looking for the next page. When I have time off, I’m bringing my laptop, I’m bringing my phone. I feel like New Relic, with our recharge week, which the summer that we all get off at the same time. FTO, so it’s flex time off. There’s really no limit to my vacation. Just some of the applications we have, like Ginger, that helps with mental health. I really feel connected when I’m using my own personal time and being with my family. So yeah, that’s how I like to recharge, if you will.

Ariane Evans: Yeah. So important. When you are moving on to the next project, or you are trying to get to the deadline of a particular thing, you can’t do that if you’re empty, and you don’t have the energy within you. And so I guess moving on, switching gears a little bit, want to talk with you, Nada, about navigating careers and career challenges. Career journeys can vary person to person. As Tracy just described, she’s been across the board of different kinds of engineers, and now a customer adoption leader, but how might you recommend navigating a career journey, and even a career journey into leadership?

Nada Da Veiga: Yeah. I mean, I think that’s an excellent question really. What I, or what we in general try to encourage folks in my organization, is to own their career, and be really proactive about it. And so a lot of people early in their careers think that they should somehow just wait for their manager to have these types of conversations. I would say quite the opposite. Be proactive about it, ask questions, share, what do you want for yourself? Where do you want to be three years from now, five years from now? Ask your manager, “What do I need to do to get there?” Because if you are informed and you know what this person expects from you, what three, five things they want to see from you in order for you to actually make it there, guess what? You have a lot higher chance of getting there, than if you’re just sitting and waiting for them to tell you, because they may or may not tell you actually.

Nada Da Veiga: They may or may not understand that you want to get from this role to some other role. That is what we see a lot with our teams. At New Relic, we are very much committed to our employee’s career progression. These are proactive conversations that are happening continuously. We encourage our employees to put together their career plans, to share those with their managers. And then some of them just want to go, “Hey, how do I go from this role that I’m in today, maybe to a senior role or a principal role?” Others want to move maybe from one org to a different org, so they want help with that path.

Nada Da Veiga: Third group will say, “Well, I want to get to leadership.” But I think how you approach it really doesn’t matter. New Relic specifically, if you are interested in leadership, we have about 14 different management classes that we recommend to folks that are setting you on that leadership path at New Relic. But whether you’re at New Relic or somewhere else, show your manager, show your leadership what you really are interested in, where your heart is at, and be proactive about it. That’s probably the best advice I can give.

Ariane Evans: Yeah. I love that. I will say that I think my career journey at New Relic is a testament to that. Starting in talent acquisition and getting to be a partner to Stephanie, but then moving into social impact and getting to learn from Erin, and now today, being a part of the DE and I team, and getting to work very closely with Kim, and that has all been championed by New Relic and the leaders within… I just said, “I’m interested in this thing, and I’m not really sure where to go from here.” But it did start with an interview. It started with a conversation with my manager. And so I’d love to kick it over to you, Stephanie, and think about, for a lot of people, getting started in your career, or looking for new opportunities, it starts with that interview process. You’ve interviewed hundreds of people in your career. And now as a recruiting leader, what is the best advice you have for anyone that is preparing to interview or in the process of interviewing currently?

Stefanie Smith: That’s a good question. I do want to just talk about just quickly, Ariane, your career progression. There’s so many people at New Relic that have had career progression, me included. I started off as a recruiter, and promoted along the way to senior manager. So there’s so much opportunity. But yes, there is an interview that’s involved. Interviewing with the company really, it’s your first impression, but it’s also our first impression to you as well. I always tell people that it’s your interview as much as it is ours. Make sure you qualify. Know what the company does. Really know what the company does. Do some research, do your homework. There’s a wealth of information about companies on the internet. It’s incredible. Link in with people on LinkedIn. Understand the roles and responsibilities, what people are.

Stefanie Smith: And then when you are talking to someone, likely it’s going to be a recruiter first, it’s a conversation. Like I said, really, you’re qualifying us, we’re qualifying you. Part of our core values is being authentic. I think that you’ve probably seen a lot of authenticity throughout this entire panel, and previous to the speakers. Be authentic during the interview, be yourself. Find some common ground. Look at it as just a conversation. Working, we spend more time than anywhere else. New Relic encourages everyone to be their best authentic self. When you’re in the interview, just really be yourself and ask good questions, and talk about career pathing and all the things that are important to you.

Stefanie Smith: Realize, if this is the right company, position, and so forth. And also even ask for guidance along the way. Your recruiter’s going to be the first step, and the recruiters are going to send you on for the next interview. Connect with the recruiter as often as possible. Even connect with the people that you’re interviewing with. We have multiple steps of roles when we interview here at New Relic. People are always going to be available to help guide you through the process. Ultimately, like I said, it’s your interview as much as it’s ours.

Ariane Evans: Yeah. I totally agree with that. Since we’ve also wrapped up this time with all of our leaders, I want to thank Girl Geek, thank New Relic for also putting this together, and everybody for listening in. I hope that you’ve gotten to pull out some nuggets of advice that are beneficial to you. If you are interested in learning more about New Relic or careers or opportunities, there are some things that Kim dropped into about our ERGs and our benefits. Please take a look at It will take you to our careers page and the opportunities that are currently live across the world. There are many.

Angie Chang: Thank you, Ariane, for moderating the panel, and to all the panelists for joining us. So now is time for our networking session. If you can click on the link at the bottom of the chat to our Zoom meeting, we can go into a Zoom meeting and have some breakout rooms where we can meet each other in person, and chat a little with our remaining 15 minutes that we have today. So if you can click on that link in chat that Amy has added, I’ll see you over at Zoom meeting and talk to you there. Thanks for coming.

new relic girl geek x speakers live square

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Girl Geek X MosaicML Lightning Talks (Video + Transcript)

mosaicml girl geek dinner outdoor patio networking dinner palo alto playground global

Over 120 girl geeks joined networking and talks at the sold-out MosaicML Girl Geek Dinner from women working in machine learning at MosaicML, Meta AI, Atomwise, Salesforce Research, OpenAI, Amazon, and Hala Systems.

Speakers discuss efficient machine learning training with MosaicML, reinforcement learning, ML-based drug discovery with AtomNet, evaluating recommendation robustness with RGRecSys, turning generative models into products at OpenAI, seeking the bigger picture at AWS, and more.

Table of Contents

  1. Welcome – Julie Choi, VP and Chief of Growth at MosaicMLwatch her talk or read her words

  2. Making ML Training Faster, Algorithmically – Laura Florescu, AI Researcher at MosaicMLwatch her talk or read her words

  3. Reinforcement Learning: A Career Journey – Amy Zhang, Research Scientist at Meta AIwatch her talk or read her words

  4. Addressing Challenges in Drug Discovery – Tiffany Williams, Staff Software Engineer at Atomwisewatch her talk or read her words

  5. Evaluating Recommendation System Robustness – Shelby Heinecke, Senior Research Scientist at Salesforce Researchwatch her talk or read her words

  6. Turning Generative Models From Research Into Products – Angela Jiang, Product Manager at OpenAIwatch her talk or read her words

  7. Seeking the Bigger Picture – Banu Nagasundaram, Machine Learning Product Leader at Amazon Web Serviceswatch her talk or read her words

  8. 10 Lessons Learned from Building High Performance Diverse Teams – Lamya Alaoui, Director of People Ops at Hala Systemswatch her talk or read her words

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Transcript of MosaicML Girl Geek Dinner – Lightning Talks:

Angie Chang: Thank you so much for coming out. I’m so glad you’re here. My name is Angie Chang and I’m the founder of Girl Geek X. We’ve been doing Girl Geek Dinners in the San Francisco Bay Area for, if you can believe it, almost 15 years now. It’s the first Girl Geek Dinner in over two years, because Julie is a rock star and wanted to do a Girl Geek Dinner in person in the pandemic and we’re like, “Yes!” It was postponed and now in May, we’re finally doing this event. I’m so glad that we have a sold out event of amazing women in machine learning that we’re going to be hearing from tonight!

mosaicml girl geek dinner sold out angie chang speaking girl geek x

Girl Geek X founder Angie Chang welcomes the sold-out crowd to our first IRL Girl Geek Dinner in over two years during the pandemic! (Watch on YouTube)

Angie Chang: I don’t want to steal too much of the time, but I wanted to do a quick raffle of a tote bag that I have. I’m going to ask, who has been to the first Girl Geek Dinner and can name a speaker from that event? Sometimes I meet people who have. Are we making this really hard? Okay, the first Girl Geek Dinner was at Google. We had over 400 women show up for a panel of inspiring women. I just wanted to see because that’s a lot of people. Who thinks they’ve been to the most Girl Geek Dinners in the room? Okay.

Audience Member: It’s actually not me, but like, this clutch was designed by somebody who hates women because its super heavy – and I see that that [Girl Geek X tote] has handles.

Angie Chang: Okay, so we have a full agenda of machine learning lightning talks and I’m going to introduce you to our host for tonight. Julie Choi is the Chief Growth Officer of MosaicML and she is an amazing supporter of women and I would like to invite her up.

Julie Choi: Oh, thank you. Hi everyone. Thank you so much for coming out to the MosaicML Girl Geek Dinner. I am Julie Choi and I actually did go to the first Girl Geek Dinner. I want to thank Angie Chang and the Girl Geek Dinner organization. Angie has just been a pioneer and truly, just a very special person, bringing us together ever since, was that 2010 or something, I don’t know. When was that?

mosaicml girl geek dinner julie choi

MosaicML VP and Chief Growth Officer Julie Choi welcomes the audience. She emcees the evening at MosaicML Girl Geek Dinner. (Watch on YouTube)

Angie Chang: The first event was 2008.

Julie Choi: 2008. Yes. And tonight we have amazing speakers to share with us about machine learning, about engineering, about diversity, and how that can really supercharge productivity in high growth organizations and machine learning research just from some of the world’s best AI companies and organizations.

Julie Choi: Our first speaker of the evening is my dear colleague at MosaicML, Laura Florescu. Laura and I met a little over a year ago. You greeted me at the front door. You were the first face I saw at MosaicML. She has just been an inspiration to me as an ML researcher at our company. Prior to joining MosaicML, Laura actually worked at several unicorn AI hardware startups. Then prior to that, she got her PhD from NYU in mathematics and is just a brilliant lady. Laura, can you come up and tell us about this amazing topic?

Laura Florescu: Thank you.

Julie Choi: Let’s give her a hand.

Laura Florescu: Can you hear me? Hi, thank you so much everybody for being here. Thanks to Julie, Sarah, Angie, Playground, for having this event. Very honored to be here.

mosaicml girl geek dinner laura florescu speaking training nlp

MosaicML AI Researcher Laura Florescu talks about making ML training faster, algorithmically with Composer and Compute, MosaicML’s latest offerings for efficient ML. (Watch on YouTube)

Laura Florescu: As Julie said, I’m a researcher at MosaicML. A little bit about myself. I’m originally from Romania. I came to the states, did my undergrad in math, did a PhD at NYU, and then I got the kind of Silicon Valley bug. And now I’m at MosaicML. And so what we do is develop algorithms and infrastructure to train neural networks efficiently.

Laura Florescu: Basically for the people in the audience who are not into ML, training neural networks is at the core of artificial intelligence. It uses a lot of data. It’s applied to many different fields with image, language, speech, and kind of like a takeaway from this is that, for large powerful models, the training costs for one single run can get in the millions of dollars, to train one such model. And in order to get to a really good model, you need to do several such runs. So the cost can get extremely, extremely expensive.

Laura Florescu: Our belief at MosaicML is that state of the art, large, powerful models should not be limited to just the top companies. As we have seen over the last few years, the costs are getting larger and larger due to both the size of the models, also the data that the models ingest is just exploding.

Laura Florescu: A couple of years ago, a state-of-the-art model, Megatron, actually cost $15 million to train. As you can imagine, startups probably cannot really train models like that. And at MosaicML, this is our belief, that this kind of training should be accessible to other partners as well.

Laura Florescu: That’s where we come in. We want the state-of-the-art efficient ML training. Our co-founders are Naveen and Hanlin from Nirvana and Intel AI. Also, founding advisors from MIT and also founding engineers from leading AI companies. All of us have the same kind of goal to train machine learning basically faster and better and cheaper.

Laura Florescu: Our thesis is, the core of Mosaic is that algorithmic and system optimizations are the key to ML efficiency, right? And then the proof to that so far, is that we are working with enterprises to train ML models efficiently. And we want to enable our organizations to train the best ML models, the cheapest and the most efficient.

Laura Florescu: Some results that we already have. As I said, we want to be agnostic about the kind of models, data that we ingest. For some image classification tasks, we have shown 6x speed ups, like 6x cheaper, faster than like regular training. About 3x faster for image segmentation, about 2x for language models, language generation, and 1.5x for language understanding. We have been around as Julie said for about a year, a year and a few months, and these are already some of the results that we have achieved.

Laura Florescu: A use case is to train NLP models, such as BERT, for those of you who know that, and on our specific platform and without algorithms. Use case is for example, to increase sales productivity. If you see there, on our MosaicML 4-node, with our speed ups, which I’m going to discuss in a little bit, we can see up to 2x speed ups of training such models. And also about 60% training costs reduced by training with both our algorithms and on our platform, on our cloud.

Laura Florescu: The MosaicML cloud, we want it to be the first AI optimized cloud designed specifically for AI and directly to reduce training cost at any layer of the stack. For example, in the training flow, we want to reuse data from past runs. In the models that we’re using, so that’s where the kind of optimized model definitions come in, Composer, which is our open source library. We are doing the algorithmic speed ups, the training. And kind of like at the lower level, we want to also be able to choose the best hardware in order to get to the lowest training cost. At each layer, we are optimizing all of these system optimizations and composing all of those basically leads to the best training runs.

Laura Florescu: As I mentioned, Composer is our open source library. We have a QR code there if you want to check it out. We have about 25+ algorithms that we have worked on and given the name, they compose together, and that’s how we achieve basically the best, 2x to 5x speed ups. And as an example, for a BERT model, we have seen 2.5x speed up for pre-training, which as you can see, goes from nine days to about three days. That’s a huge win. Check it out if you would like. As I said, open source, so we’re always looking for feedback and contributions.

Laura Florescu: We’re open to partnering with any kind of corporate users, for anybody who has vision or language tasks, and then also industry, we want to be industry agnostic and global. Again, we want to optimize basically any kind of models. And as I said, the open source Composer speed up, we’re open to feedback and partnerships for that as well. And of course we are hiring. Yeah, it’s a really great team, really fun, really ambitious. And I’m so honored to work there and we’re looking for all kind of builders, engineers, researchers, products. And thank you very much.

Julie Choi: Thank you so much, Laura. Okay, that was wonderful. Let me just go to the next talk. Our next speaker is Amy Zhang, and Amy comes to us from, she’s currently a research scientist at Meta AI research, and a postdoc I think, you’re not a postdoc anymore, are you?

Amy Zhang: It’s kind of like a part-time thing.

Julie Choi: It’s like, never ending, huh? But you’re on your way to your assistant professorship at UT Austin, amazing, in Spring 2023. And Amy’s work focuses on improving generalization and reinforcement learning through bridging theory and practice. And her work, she was on the board most recently of women in machine learning for the past two years. And she got her PhD at McGill University and the Mila Institute and prior to that obtained her M.Eng. in EECS and dual Bachelors of Science degrees in math and EECS at MIT. Let’s welcome Amy Zhang to the stage. Thank you.

Amy Zhang: Thank you Julie for the really kind introduction and for planning this amazing event. It’s so nice to see people in real life. This is my first large in-person talk in over two years so please bear with me.

mosaicml girl geek dinner amy zhang speaking

Meta AI Research Scientist Amy Zhang speaks about her career journey in reinforcement learning, from academia to industry, at MosaicML Girl Geek Dinner. (Watch on YouTube)

Amy Zhang: Today I’ll be talking about my research which is in reinforcement learning, but I first wanted to just give a little bit of introduction of myself. To me, I feel like I’ve had a fairly meandering journey through academia and industry and research in general. I wanted to give a little bit of introduction of what I’ve done so that if any of you feel like you’re going through something similar, please reach out and I’m happy to chat and give more details. Like Julie said, I did my undergrad at MIT, a year after I finished my masters, I started a PhD at UCSD. It did not go well, through no one’s fault, really. I just felt really isolated and so after about a year, I quit my PhD, meandered through a couple of startups, and then eventually found my way to Facebook, which is now Meta.

Amy Zhang: At Facebook, I initially started on the core data science team. I was a data scientist, but I was working on computer vision and deep learning. This was 2015, still fairly early days in terms of deep learning and everyone is really excited about the gains that it had shown for computer vision at the time. I was working on population density, so we were taking satellite images and doing building detection to find houses, to figure out mostly in third world countries, what the population density was so that we could provide internet to people and figure out what was the best way to provide that internet.

Amy Zhang: After about a year of that, I ended up joining the Facebook AI research team FAIR, as a research engineer. And after about a year of that, I happened to meet the person who became my PhD advisor, Joelle Pineau, who is now director of FAIR, and I got to do my PhD with her at McGill University while still staying in FAIR. Fast forward a few more years and I defended my PhD remotely in the middle of the pandemic last year and am now back in the Bay Area as a research scientist at FAIR and like I said, was part-time postdoc-ing at UC Berkeley.

Amy Zhang: After spending all this time in industry, and having a really great time in industry, and getting all of these amazing opportunities to do my PhD while in industry, with all of the nice resources that that provides, I decided to go back into academia. And, last year I was on the faculty market, on the academic faculty market, and I accepted a position at UT Austin. I will be starting there next year as an assistant professor.

Amy Zhang: With that, I’m just going to jump straight into my research. And this is going to be, again like I said, a very whirlwind, high level overview. I’m passionate about reinforcement learning. I love the idea of agents that can interact with the world, with us, and it can grow and learn from that interaction.

Amy Zhang: Okay, thinking a little bit about what reinforcement learning (RL) can do. I’m personally really excited about the idea of applying reinforcement learning to solve real world problems. To me, this is personalized household robots, having a robot that will do your dishes, clean the house, make your bed, learned autonomous driving so you can just drive in a car without having to actually drive and pay attention to the road, and personalize healthcare, so having like a robo-doctor who knows everything about you and can personalize healthcare and give recommendations for you specifically.

Amy Zhang: Unfortunately we are not there yet, as I think maybe all of us can tell. Deep reinforcement learning has had a lot of successes in the last two years. Maybe some of these things are familiar to you. AlphaGo, where we have an RL agent that was able to beat the best human experts. OpenAI with playing video games, which I’m not very familiar with, but like Dota and StarCraft. These are the things that have been hitting the news in terms of what Deep RL is capable of. But there are still a lot of disappointing failures and I think none of these videos are going to show, but imagine this robot trying to kick this ball and then just falling flat on its face. That’s what that video is. And in the other little cheetah looking thing is supposed to be like tripping and falling. Anyway, this is where we are currently with RL.

Amy Zhang: Why do we still see this discrepancy? How are we getting these amazing gains but still seeing these failures? And what we’re really seeing is that Deep RL works really well in these single task settings, in simulation, when you have access to tons and tons of training data, but it works less well in visually complex and natural settings. Basically we’re not seeing the same type of generalization performance that we’ve been getting out of deep learning in computer vision and natural language processing. My research agenda is mainly about how can we achieve RL in the real world? How can we solve these problems? And, to me, it seems that abstraction is one key to generalization. And I use this type of idea to develop algorithms that have theory-backed guarantees.

Amy Zhang: I’m going to not really talk a little bit about this math, but I’m particularly interested in being able to train reinforcement learning agents that can solve tasks from pixels. Imagine that you have this household robot or this autonomous driving car that is receiving information about the world through a camera, through RGB input, and that’s a big part of how we also perceive the world. There are things in this image that are relevant for the autonomous driver here and there are things that are not. And we want to figure out how can we determine from just a reward signal, what things are relevant versus irrelevant.

Amy Zhang: I’m just going to, as part of this project, we developed this representation learning method using this idea by simulation, and showed that in this type of simulation driving task, which is done in this platform called Carla, so we have just this figure eight simulation environment where this car is just driving along this highway, and there’s lots of other cars in the road and basically, it’s designed to try and drive as quickly as possible. Using the break as little as possible and maximizing throttle while also not hitting anything. And we find that our method which can basically ignore these kind of irrelevant details and figure out what things are irrelevant does much better compared to a lot of existing methods.

Amy Zhang: One really cool thing is that when we look at the representation that we actually learn here, and we look at what kind of observations get mapped to be close together in this representation space, so what information is actually getting captured by this representation, we see that… this is the agent’s point of view. We see that, in these three examples, you’re always on this straight road where you have an obstacle on the right hand side. It doesn’t matter what the obstacle is, but the representation just captures that something is there, which means that you can’t turn right. This is just kind of an example of what our algorithm can do. And unfortunately I’m going to skip over this, because these were just some videos showing what our agent can do.

Amy Zhang: I just wanted to end on talking about some open problems that I’m particularly excited about. I’m particularly excited about compositionality. How can we solve combinatorially difficult problems. And these correspond to a lot of real world tasks that we should care about. When we think about really simple versions of problems like this, you can have a block stacking task. You can have any number of blocks or any combination of blocks and so you can always have new environments that you give to your agent to try and solve.

Amy Zhang: Similarly, moving more towards actual real world problems that we care about. Again, going back to the dishwasher example or an agent that is trying to move boxes around in a warehouse. These are all settings where the exact environment, the exact state that the agent has to deal with, is constantly different. The objects that you want to place in your dishwasher on a day to day basis is always going to look different. How can we get agents that can actually generalize to all of these new states?

Amy Zhang: I think one really exciting direction to go when trying to solve this type of problem is to think about factorization. How can we break down a problem into smaller, easier building blocks? So if we understand how one block, the dynamics of one block moves, so creating a stack of two blocks, right? Babies play around with this sort of thing and then as they get older, they automatically can extrapolate to building like gigantic towers and castles. So how can we take that idea and give it to reinforcement learning? So this is something that I’m particularly excited by.

Amy Zhang: I just wanted to end talking about my sort of wider research agenda. Now that I’m starting as faculty, I have to start recruiting a group. If any of you are interested in doing a PhD at UT Austin or know anybody who is, please send them my way. But when I think about what my research group does, I’m particularly interested in these three applications of reinforcement learning.

Amy Zhang: The first is in robotics, trying to solve manipulation tasks. Going back to that block stacking example, trying to solve locomotion and navigation tasks. How can we build an autonomous driving system purely from first principles, like end to end machine learning? Reinforcement learning has to be a part of that.

Amy Zhang: Natural language processing, so using RL for text generation, being able to extract knowledge from text, when you build an interactive agent, how do you give it information about the world? We learn from textbooks, we don’t want an agent just deployed in the real world with no basic information. Healthcare, how do we build RL agents that can help out with diagnosis and treatment and tackle a lot of the problems that we have there? That’s basically it, very grandiose. I probably won’t make much progress on a lot of these fronts, but this is the dream. And thanks for listening.

Julie Choi: Thank you so much. I think this is very grandiose and amazing that you’re working on it. Amazing. Thank you, Amy, okay.

Julie Choi: We’re going to just clean some of this up. Okay. And we’re going to open up our next talk, which I’m extremely excited about. Let me introduce our next speaker. Our next speaker journeyed probably the furthest to join us tonight… all the way from the east coast, North Carolina, to be here today to deliver this talk. I want to thank you. Thank you.

Julie Choi: Tiffany Williams is a Staff Software Engineer at Atomwise working on AI-powered drug discovery. Prior to Atomwise, Tiffany was a Staff Software Engineer at Project Ronin where she was developing cancer intelligence software. Tiffany earned her PhD in cancer biology from Stanford University and her Bachelor’s in biology from the University of Maryland. Let’s give a warm welcome to Tiffany Williams.

Tiffany Williams: All right. Hello, everyone. I’ll admit, I have some notes here so I can stay on track, but I’m really glad to be here this evening. I was searching through my inbox and realized that I attended my first Girl Geek Dinner in April of 2015. I was fresh out of grad school and a coding boot camp. Eager to form connections, acquired some cool swag, and to be honest, eat some free food and have some drinks.

mosaicml girl geek dinner tiffany williams speaking

Atomwise Staff Software Engineer Tiffany Williams discusses the drug discovery process with AtomNet at MosaicML Girl Geek Dinner. (Watch on YouTube)

Tiffany Williams: I went to grad school to study cancer biology. My research was from the perspective of a molecular biologist exploring the role of a protein as a target for therapy, and skin cancer. And my interests have generally been at that sweet spot, leveraging data, and technology to improve human health. Coming from a biology research background, in my current role as a software engineer at Atomwise, I’m able to look further down that drug discovery process. I’m happy to share with you all today, what that looks like.

Tiffany Williams: Now, I’ll start by giving a very brief primer on the current state of the drug discovery process, as well as a very brief primer on biochemistry. From there, I’ll talk about what we’re doing at Atomwise to make a significant impact in human health, and some interesting challenges ahead of us. I do want to give a disclaimer that I am merely scratching the surface of what could be discussed in drug design, drug development, and even applying ML on top of that. But what I hope you all take away from my time with you all this evening is just general excitement about the possibility of improving human health.

Tiffany Williams: Beyond that, I hope you feel empowered to do even more digging into the drug discovery process, and maybe you’ll feel empowered to even… to find opportunities to make an impact in that space. For me personally, I’m coming from the east coast, but I have a personal attachment to improving the drug discovery process. I’m on the east coast serving as a caregiver now – my mom has endometrial cancer and it’s at a point where there’s only one treatment option. If any of you have ever been in, unfortunately, in a predicament like that, it sucks.

Tiffany Williams: There’s a lot of data out there, technology’s improving, and it would be great if we can leverage that to improve health, right? This diagram depicts the drug discovery process from the basic research to identify a potential drug target all the way to FDA approval. On the right, I’ve noted the average number of years of the different stages in the drug discovery process. In the middle, what it shows are like basically in these different stages, there are certain types of experiments that are done that basically kind of knock out the potential candidates in that step.

Tiffany Williams: Initially you might have like a candidate pool of over like 10,000 drugs, but in an early stage, which is known as like kid identification, computer simulation is used to predict potential drugs ability to bind to a target of interest. These subsequent steps will test for other characteristics like potential toxicity, or efficacy in cell cultures, and animal models. Eventually, hopefully, we end up with a few candidates that reach the human clinical trial phase to verify safety, and any other side effects along with a few other things.

Tiffany Williams: What I hope that you take away from this slide really is that the current basic research to FDA approval drug process takes a really long time. It takes on average, it’s estimated to take about 15 years, and it’s also really expensive for each drug that goes to market, it’s estimated that 2.6 billion is spent.It’s the case that not all of these trials have a happy ending. For every drug that makes it to the market, millions may have been screened, and discarded. We have to improve this process, right? But in order to appreciate how this process can be improved, let me first give that very, very, very brief biochem primer. And I’ll focus specifically on protein interactions.

Tiffany Williams: Up to this point, and then even later in this talk, I’m going to be using some words interchangeably, and I just want to make sure I bring some clarity to what I’m actually talking about. When I say like ligand, or ligand, I’m specifically referring to any molecule that combined to a receiving protein, and that receiving protein is also known as a receptor. And in this presentation, when I use the word or phrase drug candidate, I’m actually referring to that ligand and the receptor would be that the drug target.

Tiffany Williams: One model for how proteins interact is this lock and key model. And the gist of this is that the ligand and the receptor have these somewhat complimentary sites. And the ligand combined to this complimentary site or active site, and basically alter the shape, and or activity of the receptor. And one more thing is that this binding is also referred to as docking. If we know our bodies are made up of proteins, and they have a diversity of functions within the body, but in a disease state, a normal process can become dysregulated. This image on the far right, is taken from a classic cancer biology paper called The Hallmarks of Cancer. The premise of this is that there about 10 biological capabilities that cells take on as they like morph into this more cancerous state. And I’m not going to go over all of these, but I want to highlight that two of these biological capabilities would be like cell growth and like cell motility.

Tiffany Williams: Those are normal functions within the body. But I guess within a diseased state you’ll have overgrowth, or you may have cells that are primarily concentrated in one area, develop the capability to invade into other tissues or metastasize. I say all this to say that like in small molecule drug discovery, what we want to do is actually figure out what sort of structure is needed for a ligand to bind to this problematic protein or proteins, and counteract that like not so great behavior. If this is where Atomwise comes in, right? Or it does, if you don’t know. We want to know how can we efficiently explore this space of all potential chemical compounds to better identify small molecule drugs faster.

Tiffany Williams: Atomwise actually developed AtomNet the first deep convolution neural network for structure based drug design, so that we can actually make better predictions for potential drug candidates earlier in the drug discovery process and faster. I’m highlighting this paper, again, just trying to give high level overviews. Feel free to check this paper out, but what I want you to take away from this is that the AtomNet technology is currently being used in real drug research, in cancer, neurological diseases, various spaces, on the right is, it’s actually a GIF, but since it’s a PDF, it’s not showing up as a GIF, but basically this GIF would show the AtomNet model. It would simulate the AtomNet model, predicting candidate treatments for Ebola. And this prediction that AtomNet made is actually has led to candidate molecules that are now being studied in animal models.

Tiffany Williams: One, despite everything I’ve said up till now, you actually don’t need a background in wet lab research, or chemistry to appreciate what’s happening here. The power of convolute or the power of convolution neural networks, or CNN, is that it allows us to take these complex concepts as a combination of smaller bits of information. And I think if you’re familiar with CNN, or even if you don’t like one area that is really popular, has been computer vision.

Tiffany Williams: I’ll briefly go over like an example of image recognition, and then kind of like try to tie it into how AtomNet works. An image is essentially represented as a 2D grid with three channels, you have red, green, and blue. And this network learns images of objects or faces, for example, by first learning a set of basic features in an image like edges. Then, from there, by combining those edges, the model can then learn to identify different parts of that object. In the case of a face that might be ears, eyes, nose, et cetera. With AtomNet that it’s working in a similar fashion, that receptor ligand pair is represented in a 3D grid, and the channels are essentially the elements that make up protein like carbon oxygen, nitrogen, et cetera. In the case of AtomNet, that the learning of edges is actually the learning of the types of, or predicting the types of bonds between those elements.

Tiffany Williams: Then from there, the compliment to the ear eye detection would be actually identifying more complex molecular bonds. You could say, essentially, that this network is like learning organic chemistry 101. This is powerful because we can then train these models to make predictions about different aspects of the drug discovery process. Like what ligands, or what type of structures are most likely to bind to a certain target? At what strength? What are the additional modifications that we can make to a potential drug candidate to strengthen that bond? Beyond that, it’s not enough just for the ligand to bind to the receptor, it also has to be like biologically relevant in that, let’s say, if we’re looking for something that is treating some neurological disease. We need that ligand to be small enough to cross the blood brain barrier. Or we may need to take into account toxicity or other effects that may happen in the body. Metabolism. We don’t want that small molecule to become quickly metabolized in the body before it has the opportunity to have the intended effect.

Tiffany Williams: These are exciting times, and I’m really, really passionate about the work that we’re doing at Atomwise and any work that is being done at the intersection of health and technology. I wanted to briefly go over some of the projects that my team is currently working on. I work on the drug discovery team at Atomwise, it’s within the engineering team. I think we’re working on some pretty interesting issues. One of my, my teammates Shinji, he recently has been working on bringing best engineering practices, and improving performance of some of our ML tools. Adrian, my teammate, is working on optimizing algorithms to explore a three trillion chemical space. He is also been working on, or has been able to create simulating mocular… mocular, I’m combining words… molecular docking on GPUs.

Tiffany Williams: Then another thing that we’re working on that I actually have more of a hand in, is building a research platform to better enable drug discovery. Oh, I didn’t mention the third person Shabbir, who’s our manager, and he has his hands in a bit of everything. What I hope you take away from this is that I think we’re at an exciting time in today to like really leverage data, and technology to make a major impact in human health. I think there are a lot of challenges, interesting challenges in drug discovery. I hope that you may have been convinced that you actually don’t need a background in chemistry to contribute. There are a lot of transferable skills. If you just know software engineering, or you know ML, or if you’re in product or marketing, there is a place for you in this space.

Tiffany Williams: Finally, really important takeaway is that we are hiring. My team alone, please, if you have any front end experience, if you have backend experience, or if you have a background in computational chemistry, those are some of the positions that are open right now on my team, but then outside of my team, we also are hiring. Definitely check out our careers page. If you have any more questions or are interested in chatting, feel free to reach out to me. I have my LinkedIn handle as well as my Twitter handle, posted here. Then finally, I think I mentioned that I had some references to share. Again, I’m only scratching the surface. There’s so much information out there. I wanted to highlight two Medium articles that were written by former Atom, which is what we call people that work at Atomwise, machine learning for drug discovery, in a nutshell, I highly recommend starting there if you want to do a deeper dive. That’s it.

Julie Choi: Thank you so much, Tiffany. Yes. I think drug discovery is an incredible application domain for deep learning. Really appreciate your talk. Okay, let me introduce our next speaker. Our next speaker is Shelby Heinecke. Shelby is a senior research scientist. Again, I did not touch. Okay.

Julie Choi: Shelby is a Senior Research Scientist at Salesforce Research, developing new AI methods for research and products. Her work spans from theory-driven, robust ML algorithms to practical methods, and toolkits for addressing robustness in applied NLP and recommendation systems. She has a PhD in Mathematics from the University of Illinois at Chicago, and a Master’s as well in Math from Northwestern and her bachelor’s is from MIT. Let’s welcome Shelby.

Shelby Heinecke: Thanks so much. Awesome. I have to give a thank you to Julie for including me in this event, inviting me. This is my first Girl Geek, and not my last Girl Geek event. I’m super excited to be here. Yeah, let me get started.

mosaicml girl geek dinner shelby heinecke speaking recommendation systems salesforce research

Salesforce Research Senior Research Scientist Shelby Heinecke speaks about how to evaluate recommendation system robustness with RGRecSys at MosaicML Girl Geek Dinner. (Watch on YouTube)

Shelby Heinecke: Today I’m going to be talking about evaluating recommendation system robustness, but first I feel kind of like the new kid on the block. Let me just give a little bit of background about myself. I moved here to the Bay Area about a year and a half ago in the middle of the pandemic. Super excited to be here in person to meet people.

Shelby Heinecke: Currently I’m a Senior Research Scientist at Salesforce Research. As Julie mentioned, I work on both research and product. It’s pretty awesome to develop prototypes, and see them in production. Before I was here at Salesforce in the Bay Area, I was doing my PhD in math in Chicago. There, I focused on creating new ML algorithms that were robust. I worked on problems in the space of network resilience. Before that I got my master’s in math, kind of focusing on pure math at Northwestern. Originally I hail from MIT Math, focusing on pure math there. That’s my background. Today’s talk recommendation systems and robustness.

Shelby Heinecke: Let’s get started. A crash course and recommendation systems. So, what is a recommendation system? Well, it consists of models that learn to recommend items based on user interaction histories, user attributes, and or item attributes. Let me give you an example, say we want to build a recommender system that recommends movies to users. Well, what kind of data can we use? We’re going to use the users, previous movies. They viewed we’re going to use the ratings that they rated those movies.

Shelby Heinecke: We’re probably going to take a look at the user’s age. We’re going to look at the user’s location. We’re also going to take into account item attributes. The movie attributes, like the movie title, the movie genre, we’re going to take all that. That’s all of our data, and we’re going to train models. The models can build the recommendation system. As you can imagine, recommendation systems influence our daily lives. We’re all exposed to recommendation systems every day. Just think about purchases. Think about movies.

Shelby Heinecke: Think about songs you’re recommended. Think about the ads that you see every day, people, news, information. We are at the mercy of these recommendation systems and a lot of our decisions are highly influenced by what the recommendation systems decide to show us. Let’s think about the models that we see in recommendation systems. Models can range from simple heuristic approaches, like a rules based approach or co-sign similarity approach to complex deep learning approaches, think neural collaborative filtering, or even now we’re seeing transformer based approaches coming to light.

Shelby Heinecke: With the vast range of models available and how greatly they impact our daily lives, understanding the vulnerabilities of these models of each of these types of models is super critical. Let me get started about recommendation model robustness. As we all know, machine learning models are trained on data, and ultimately deployed to production. And in that process, there are some hidden sources of vulnerabilities that I want to bring to light.

Shelby Heinecke: One of the big issues is that training data may not reflect the real world data. In many cases, we’re training on data that’s been highly curated. That’s been cleaned up. And yeah, and so because of that, when we train a model on that very clean, highly curated data, it’s not going to necessarily perform well when it’s exposed to the messy data of the real world. Real world data has noise. Real world data is just can be unpredictable. As a result, sometimes we train model, we train a recommendation model, but we see poor performance and production. That’s definitely one type of vulnerability we need to watch out for.

Audience Member: Woo! So true.

Shelby Heinecke: What? Okay. Another type of vulnerability. I love the enthusiasm, okay? Another type of vulnerability. As you can imagine, recommendation systems are closely tied to revenue for a lot of different parties for companies, for sellers. There’s an issue that participants may intentionally manipulate the model. Think about creating fake accounts, trying to do things, to get your item higher on the list for customers, things like that. That is a reality and that’s something we have to take into account.

Shelby Heinecke: The last thing I want to bring to light is poor performance on subpopulations. This is a well known issue across all of ML, but I just wanted to bring it to light, to recommendation too, that when we train models and we test on the evaluation set, usually the basic evaluation methods think precision recall F1. We’re computing that on the entire test set, so we’re averaging overall users. And in that average, sometimes we’re hiding…

Shelby Heinecke: Sometimes there’s poor performance on subpopulations that are hidden. For example, a subpopulation that you may care about could be new users, or maybe a users of a certain gender. That’s something that we just need to keep an eye on. We don’t want poor performance on key subpopulations I’ve told you about all these different types of vulnerability.

Shelby Heinecke: What is a robust model? Well, a robust model you can think of it intuitively as a model that will retain great performance in light of all of these potential perturbations, or in light of all of these scenarios. The question is how can we assess the robustness of models? That is where one of our contributions comes in.

Shelby Heinecke: I want to introduce one of our open source repos called RGRecSys, and it stands for robustness gym for recommendation systems. Our library kind of automates stress testing for recommendation models. By stress testing, I mean, you can pick a model, you can pick a data set, and you can stress test it in the sense that you can manipulate the data set.

Shelby Heinecke: You can add in attacks, you can add in noise and so on. I’ll actually go into more detail about that. And you can see in a really simple way how your model, the robustness of your model. As I mentioned, RGRecSys, is a software, is a software toolkit. It’s on GitHub. I’ll talk about that in a second and it’s going to help you assess the robustness of your models.

Shelby Heinecke: One thing that we contribute is that RGRecSys provides a holistic evaluation of recommendation models across five dimensions of robustness. When I say I told you about various types of vulnerabilities, there’s various types of robustness. Our library helps you to quickly and easily test all these different types of robustness for your model. Using our API, you just simply select a model for testing, and then you specify the robustness test that you’re interested in trying along with the robustness test parameters. What I’ll do is I’ll go over the types of tests that are in the library.

Shelby Heinecke: Let’s talk about the different tests that we have in our library. First is around subpopulations and this kind of goes back to what I mentioned in the previous slides. This will allow you the test. What is the model performance on subgroup A versus subgroup B? And this is going to be very useful because, as I mentioned, most of the time, we’re just computing precision recall these usual metrics on the test set, but this gives you an easy way to test performance on specific subgroups. This could be useful, for example, if you want to test you want to test performance on gender A versus gender B, or new users versus old versus like users that have been in the system for longer time. Just some examples there. That is one type of test you can run in our library. The second type of test is around sparsity. If you think about recommendation systems and you think about the items available like a movie recommender, or a purchasing recommender there’s millions of items, and each user really only interacts with a handful of those items.

Shelby Heinecke: Each user is only clicking, only purchasing, only viewing a handful. This is a source of data sparsity. Data sparsity is a huge problem in recommendation systems. It will be good to test the degree to which your model is sensitive to sparse data. That’s one thing you can test with our library. The third test is around transformations. There are a lot of ways that data can be perturbed when you’re training a recommendation system model. For example, maybe you’re gathering data about your users and maybe that data is erroneous in some ways. And because of that, you might want to test if a recommendation model will be robust, if user features, for example, are perturbed. The fourth test that you can test is around attacks. As I mentioned, there’s a lot of reasons why people would try to attack a recommendation system it’s tied to revenue ultimately.

Shelby Heinecke: What you can do with our library is implement some attacks and test how your model performance would change under those manipulations. And finally distribution shift. What we see is that the training data that you train on is often different from the data that you’ll see in production. It’s super important to be able to know, get a sense of how was my model going to perform if it’s exposed to data from a slightly different distribution? You can go ahead and test that with our library., I definitely encourage you to check out our library on GitHub, and feel free to check out the paper for way more details about the capabilities. And with that, thank you so much for listening. It was great to share it. Feel free to reach out.

Julie Choi: Thank you, Shelby. That was great. It’s very important to be robust when we’re doing model deployment. Okay. Angela Jiang, thank you so much for joining us tonight.

Julie Choi: Angela is currently on the product team at OpenAI. Previously, she was a product manager at Determined AI building, deep learning training, software and hardware, deep learning… I think Determined AI was recently acquired by HPE. And Angela graduated with a PhD in machine learning systems from the CS department at Carnegie Mellon University. And we actually have built some of our own speed up methods on your research. It’s an honor to have you here today.

Angela Jiang: Thank you so much, Julie, for inviting me, for the introduction, as well as Sarah, and Angie for organizing the event and bringing us all together too. Yeah. Like Julie mentioned, I’m Angela. I work on the product team at OpenAI.

mosaicml girl geek dinner angela jiang openai

OpenAI Product Manager Angela Jiang speaks about turning generative models from research into products at MosaicML Girl Geek Dinner. (Watch on YouTube)

Angela Jiang: I work on our Applied team where we really focus on essentially bringing all of the awesome research coming out of the org and turning them into products that are hopefully useful, safe, and easy to use.

Angela Jiang: Most of my day, I’m thinking about how to turn generative models from research into products. I thought I’d make the talk about that as well. This might as well be a list of things that keep me up at night, think about it that way. But in particular, what I really wanted to highlight is just some of the observations I’ve had about things that make these generative models unique and tend to have large implications on how we actually deploy them into the market.

Angela Jiang: To start, I want to share a little bit about what OpenAI’s products are. OpenAI does a lot of AI research. In particular, we do a lot of generative models. Over the last two years, we’ve really started to work to bring those generative models into real products. Three examples here are GPT-3, that does text generation; Codex, that does code generation; and most recently, DALL-E, which does image generation. These were all made by DALL-E.

Angela Jiang: To get a little bit more concrete, what these products are is that we expose these models like GPT-3 and Codex via our APIs. As a user, you can go to the OpenAI website, sign up, and then hit these endpoints and start using these models. And here’s an example of how you might use the GPT-3 model. Here we have an example of your input in the gray box. It might say something like, “Convert my shorthand into a firsthand account of the meeting and have some meeting notes.” And you submit this to the model, and then the model will do its very best to return you an output that completes this text. And in this case, the response is essentially a summarization of your notes.

Angela Jiang: Codex works very similarly, except it’s also specialized for code. And in this case, our input is not only text, but it’s also some code, some JavaScript code, and some hints that we want to transform it into Python. And the output is some Python code that does that. DALL-E is very, very new. It’s been out for around a month, so it’s not available in the API or anything yet. But it has a very similar interaction mode where you have, again, an input, which is a description of the image that you want. And then as the output, it provides some images that relate to that description, which in this case is a cute tropical fish.

Angela Jiang: I think GPT-3, Codex, and DALL-E are all really good examples of taking research and seeing that there’s a really big market need for the kinds of capabilities that they expose. Then working as an Applied team to actually transform that research into a productionized product by essentially figuring out where that user value is, designing the product so that we can deliver that user value in a way where users are set up for success, making sure it’s deployed responsibly, et cetera, et cetera.

Angela Jiang: To date, there’s hundreds of applications that have been submitted for production review that is built upon this API. Those are applications like writing assistants like CopyAI if you’ve heard of that, coding assistants like GitHub’s Copilot is built on top of Codex, as well as a lot of other applications like chatbots for video games, question answering bots, etc. This is pretty good validation that these models are not only really exciting research, but are also solving problems in the market.

Angela Jiang: Now, when we think about bringing these to market and turning them into real products, I think there are two interesting properties of these generative models that really change who uses them as well as how we actually deploy them. I’ll talk about how these models are stochastic and how they’re very general. With typical products, you might expect that the results are very predictable and deterministic. For instance, if you’re coding up a payments API, every time you use that payments API, you’d expect it to react in a similar way. This is not how our models work at all.

Angela Jiang: Our models are probabilistic. Every time you submit a prompt, you could get a different response back. If you tweak that prompt ever slightly, then you could get a very different response back. That really changes how you interact with a product and what kind of applications can be built on top of it. Another difference is I think that typical software products that I think about are typically focused on solving a need or a problem very directly, whereas, in these generative models, they’re very, very general. They’re capable of doing a lot of things all simultaneously.

Angela Jiang: For instance, you would think that GPT-3 would be able to summarize legal texts and also simultaneously write poems or maybe even code. I think this is super exciting from a science perspective as we get more and more powerful models, closer and closer to something that’s super general. But from a product perspective, this comes with some challenges because we have to now take one model and one product and have it serve many, many use cases simultaneously.

Angela Jiang: For the rest of this talk, I’ll talk about, again, more detail about how these two properties affect the way that we deploy our products, and I’ll give some examples of each. Okay, starting with stochasticity. Right. Our models, again, are probabilistic, which means that every time you use it, you might get a different result. The result might not be what you want or maybe the result is what you want every third time you try the model.

Angela Jiang: It might be surprising that you can actually build real applications on top of this kind of behavior. What we actually find is that for a lot of tasks, especially very simple tasks, we can have very concrete and reproducible behavior for even tasks where the outputs are imperfect or are only correct some of the time, these results are actually still very useful for many applications in the right context.

Angela Jiang: We find that these models work really well, for instance, in creating productivity tools where there’s a human in the loop. A really good example of this is Codex Copilot. Some of you may have used it before, but this is a screenshot of it in action, where it’s a Visual Studio Code extension and it just gives you auto-complete code suggestions that the user can then tab to accept or reject.

Angela Jiang: This is great for a product like Codex because you have actually an expert there that’s telling you, is this completion good or not? And generating more. What we actually see is that most of the suggestions from Copilot are rejected by the user, but still, developers tell us that this is an integral part of their deployment pipeline or their development pipeline I should say. It does not need to be perfect every time we generate something. Those are cases where you don’t need a perfect result every time. What you’re looking for is just inspiration or getting over writer’s block. In some of the applications built on top of these APIs, you actually don’t have a correct answer to shoot for.

Angela Jiang: Applications that are doing art or creative tasks or entertainment, that’s a case where having variety really is helpful actually. Here I have a DALL-E prompt, which is a bright oil painting of a robot painting a flower. And here are different generations from DALL-E. And in this case, it’s really helpful actually to have probabilistic nature because, for one, I want to have different options. And it’s also really cool to have generations that you’ve never seen before or other people haven’t seen before.

Angela Jiang: I think it’s even counterintuitive that, in this case, we’ve actually learned that these AI systems seem to be doing really, really well and surprising us in creative tasks as opposed to rote tasks, which you might have expected the opposite earlier. Those are some examples where the properties of these generative models affect what applications are built on top, but it also affects how we deploy these models. Going into a little bit more detail, one property is that these products are really, really hard to evaluate.

Angela Jiang: This makes being a Product Manager, among other things, quite challenging because we really need to know how our products perform so we can position them in the correct way to the users, know what to deploy, and tell users how to use them correctly. For example, we have these text models, GPT-3, and they have different capabilities, right? One capability is they might be able to complete text, and another capability is they might be able to edit existing text. There’s some overlap there on which one you should use.

Angela Jiang: It’s really important that we understand exactly how this model performs on different tasks so that we can direct users to use the right tool at the right time. We try really hard to figure out creative ways to evaluate these at scale, given that we often need a human in the loop to be telling us if a generation is good or not.

Angela Jiang: We do things like large-scale application-specific A/B testing. Like we see if we use one model or the other, then which model gets more engagement for this writing assistant.

Angela Jiang: Next, I’ll just give a couple of examples of how generality also comes into the picture here. Like I mentioned, these models are capable of a lot of things simultaneously because the way that they’re trained is that they’re trained for a very long time, for months, on a lot of data. And the data spans the internet, books, code, so many different things.

Angela Jiang: By the time we get this model as the Applied team, we’re really still not sure what kind of capabilities it’s picked up at that time. But like I mentioned, it’s really important that we understand that quickly. This is also really exciting because, at any given time, we can always discover a new capability of the model, and it’s often discovered after training.

Angela Jiang: An example of this is that the original GPT-3 model was built with text for text. And it was really a discovery after the fact that it also happened to be kind of good at code as well. And these signs of life and discovery is what ultimately sparked this idea of a Codex series that specialized in code.

Angela Jiang: As a Product Manager, it’s really critical that we keep on discovering and probing these models to really understand what’s the frontier of what they can do. And again, we are still figuring out what is the best way to go about this. But something counterintuitive that I realized after I started is that traditional user research that we’re used to doesn’t actually work really well for this use case because you might expect that you would go to your users and ask, how is this model? What can you do with it?

Angela Jiang: What we find is that our users, by definition, are working on tasks that the model was already really capable at. Their focus is not on the new capabilities of the model and pushing that frontier oftentimes. What’s worked better for us is working with creatives or domain experts to really hack and probe at the model and see what the limits of it are.

Angela Jiang: That’s the exciting part of generality, that there’s always a new capability around the corner to discover. But there’s also some risk to it because there’s not only all these capabilities in the model, but the model can also generate things that are not useful for you, and it can also generate things that you really don’t want to generate.

Angela Jiang: For instance, GPT-3, when prompted in the right way, can generate things like hate speech, spam, violence, things you don’t want your users to be generating. This is also a big part of our goals as a research and applied organization is to figure out how to deploy these models in a way that doesn’t have toxicity in them.

Angela Jiang: There’s a lot of different approaches for doing this, spanning from policy research to product mitigations to research mitigations, but I’ll highlight two things here. One thing that’s worked really well for us is actually fine-tuning these models after the fact with a human in the loop, telling you exactly what kind of content is good content.

Angela Jiang: What we found by doing this is that we have a much better time at having the model follow the user’s intentions. And we have a much less toxic and more truthful model as a result of it. At this point, every time we deploy a model, we also fine-tune the language model in this way. We also do a lot of research and provide free tooling to help the users understand what the models are generating at scale so they can understand if there’s anything that they need to intervene in.

Angela Jiang: Okay. Finally, even though I think it’s very, very cool that these models can do a lot of different things at once, sometimes it really just can’t serve multiple use cases simultaneously well. Different use cases often will just require, for instance, different ways of completion.A chatbot for children is going to want a very different personality than a chatbot for support.

Angela Jiang: You also just have different product requirements in terms of accuracy or latency or even price. Copilot is a really good example of something that needs an interactive latency so that it can continue to be useful in real time. But contrast that with something like SQL query generation, which doesn’t need a fast latency, but actually just wants the most accurate response that you can give it. What we’ve found is that the combination of two things have allowed us to provide this flexibility to serve the use cases that we need to serve.

Angela Jiang: One is that we don’t just expose the best model that we have or the most accurate model we have I should say, we expose models with different capability and latency trade-offs for the users to choose from. And then we also offer fine-tuning as a first-class product so that you can take a model and then fine-tune it so that it has your personalized tone or your personalized data bank.

Angela Jiang: These are, hopefully, examples that give you a flavor of what it’s like to deploy generative models. This is also just the tip of the iceberg. If any of this stuff is interesting, please feel free to come and chat with me. I should also mention that we are hiring. We are. Thank you so much.

Julie Choi: Thank you so much, Angela. After DALL-E was launched, the productivity went a little down. We were so distracted by the DALL-E. I don’t know whether to thank or curse your team for that.

Julie Choi: Thank you, Banu. It is just a joy to introduce our next speaker. She’s a friend of mine and a former colleague, really an all-star. It’s so wonderful to have you here, Banu. Thank you.

Julie Choi: Banu Nagasundaram is a machine learning product leader at Amazon Web Services where she owns the go-to-market strategy and execution for AWS Panorama, an edge computer vision appliance and service. Prior to AWS, Banu has spent over a decade in technology, building AI and high-performance computing products for data centers and low-power processors for mobile computing. Banu holds a Master’s in EECS from the University of Florida and an MBA from UC Berkeley’s Haas School of Business. Let’s all welcome Banu.

Banu Nagasundaram: Thank you, Julie and team for having me here. I’m super excited to be here. One difference from the other speakers is this is not my area of expertise, the title of the talk. It is something I’m trying to do better at that I wanted to share with you.

mosaicml girl geek dinner banu nagasundaram speaking aws

AWS Product Manager Banu Nagasundaram speaks about seeking the bigger picture as a ML product leader at MosaicML Girl Geek Dinner. (Watch on YouTube)

Banu Nagasundaram: I’m trying to seek the bigger picture at work. I’m a Product Manager and I’m trying to see why companies do what they do and learn more in that process. What I want you to take away from this is how you can also seek the big picture in your roles that you do either as engineering or product leaders.

Banu Nagasundaram: With that, I wanted to share a little bit about the companies that I work with on a daily basis. These are concerns who use machine learning and AI and work with AWS to implement the services in production. This is different from the research that we spoke about.

Banu Nagasundaram: These companies are looking at getting value out of these systems that they put in place, of course, based on the research, but taking into production. What I implore you to think about is put on a hat of a CTO or a CIO in each of these companies and think about how and why you would make the investment decisions in machine learning and AI.

Banu Nagasundaram: For example, I work with healthcare and life sciences team. I learned a lot from the drug discovery talk earlier from Tiffany here, but I do work with healthcare and life sciences team to understand how they can take the vast amounts of health data that they have to translate into patient information that they can use to serve patients better.

Banu Nagasundaram: They use multiple services to personalize, to extract value from the text data, a lot of unstructured data that they have. That’s one category of customers.

Banu Nagasundaram: The second type of customers that I work with include industrial and manufacturing. The key component that they’re trying to improve is productivity and also optimizing their manufacturing throughput.

Banu Nagasundaram: The questions that they ask and they seek to improve include automating visual inspection. How can I improve the product quality across my manufacturing sites? I have thousands of sites in the US. I scale globally. How can I implement this process not only in one site but uniformly across those thousands of sites to achieve something like predictive maintenance on the tools, improve uptime of their equipment, etc?

Banu Nagasundaram: Third set of customers we work with include financial services. They are looking at data to improve or reduce the risk in the decisions that they make. They’re trying to target customer segments better so they can understand underserved populations but lower the risk in making those products and offerings that they want to do.

Banu Nagasundaram: They also look into fraud detection and many applications around financial services. Retail, this is one I work closely with because I work in the computer vision team right now. And retail is trying to use the insights from computer vision products to see how they can reduce stockouts, which is basically when you go to the store, is the product available? Can they sell it to you? How can they manage inventory? Can they keep track of the count or the number of people entering the store?

Banu Nagasundaram: You may have heard about Amazon Go, for example, a store with just a walkout experience. A lot of retail companies are working with us to understand how they can use computer vision to build experiences like that.

Banu Nagasundaram: And it doesn’t stop at a retail store. Think about the operations officer, a centralized person who’s sitting and trying to analyze which region should I invest more on? Which region should I improve security? Should it be in North Carolina? Should it be in California? Or should it be in a whole different country? They’re trying to collect and gather insights across their stores regionally, nationally, internationally to make those decisions.

Banu Nagasundaram: Then there’s media and entertainment too, which we touched upon a little bit around recommendation systems and personalization. Here we work with customers who are looking to improve monetization, who are looking to create differentiation in the marketplace, in the very highly competitive media and content marketplace, through those recommendation systems and personalization that they have.

Banu Nagasundaram: Across all of these customers, the core task as a product manager that I work on is understanding their requirements and then translating it into product features so that they are served better. But what I learned in the process is that it’s so much more in decision-making than just understanding about features or product requirements.

Banu Nagasundaram: It’s about what enables these CTOs to make those decisions is value creation. How can they use all of these AI ML systems to realize value from the systems that they put in place? One simple way to think about what value creation is, is that it’s an aggregation of data, analytics, and IT that brings the machine learning together. But there’s a second part to it, which includes people and processes.

Banu Nagasundaram: What I mean by that is all of this analytics and machine learning and data can help them understand something, but they still have to lean on people to analyze the data that they gather, make improvements in the process in order to recognize and create value. And that’s the workflow for decision-making across all of these companies.

Banu Nagasundaram: We can look at this as two buckets, one in data analytics and IT, and the second one as people and processes. For the computer vision product that I own, I wanted to talk to you about the value chain for the first part, which is the data analytics and IT portion.

Banu Nagasundaram: This might look a complex set of boxes. It is. Once I finish the pitch here, if this excites you, I’m hiring too, growing my team. Hopefully, I do a good job in explaining this value chain.

Banu Nagasundaram: I started my career in the bottom left, by the way, in silicon processor design. Pretty much in the bottom row, trying to understand how silicon design is then aggregated into components and how distributors sell those components to OEMs who are equipment manufacturers, and how from those equipment manufacturers, the equipments reach the consumers, which is through those equipment distributors.

Banu Nagasundaram: My product is currently both an appliance and a service. I do start from the silicon side, working with partners, I can give examples like Nvidia on the silicon side, Lenovo on the OEM side. And then once you have these equipment distributors selling these devices or equipment I should say to infrastructure providers, one of the examples of infrastructure providers is cloud service providers, but it can also be on-prem equipment providers.

Banu Nagasundaram: You then go to the infrastructure providers, which is the second row from the bottom, and these infrastructure providers then build the tools and frameworks either themselves or through the partner ecosystem. Those tools and frameworks essentially put in place to make efficient use of that underlying infrastructure. It’s a motivation for these infrastructure providers to offer the best tool and frameworks so that you can gain that value out of that underlying infrastructure.

Banu Nagasundaram: Then comes the ML services. You have the overall MLOps flow is what would fit in this bucket. Once you have the tools and the frameworks, how can all of these pieces get grouped together to build a robust system that can scale in production? This goes from data annotation, labeling, training to predictions, to model monitoring. “How can you maintain this when it is in production” is a key question for these decision-makers.

Banu Nagasundaram: Then comes the AI services that are built on top of these ML services. AI services can either be services offered by CSPs or it can be services offered by startups or companies who are trying to build the microservices or services on a specific use case. This is where the customer use case comes into play, where you have that specific use case. In case of computer vision, you can think of services like Rekognition or Lookout for Vision. Those are two examples that AWS offers for computer vision services.

Banu Nagasundaram: Then comes the final layer. This is the layer that the customers that I refer to, the CTOs, work closely with. The independent software vendors or ISVs are companies that build these software solutions, aggregating everything that I spoke to you about, but building the software components of it. But the software component by itself is not going to function in a customer’s premise. For the customer to realize value, the solution, the software solution for their use case has to integrate with their existing system. That’s where system integrators come into play.

Banu Nagasundaram: For example, Deloitte. You can think of Deloitte, Accenture, et cetera are system integrators who bring this whole puzzle together for customers, build that reference architecture for that solution. And they’re like, okay, so now we have this architecture in place. Then they bring in the value-added resellers, who are companies like Convergent or Stanley, for example, who take this entire system that’s put in place and roll it into individual sites.

Banu Nagasundaram: This is where it reaches the scale of thousands of sites. Once a solution is put together, proven in a pilot or a production pilot or a proof of concept, when you go to production, across globally, across cities, across countries, the value-added resellers roll the system into place in the customer’s site. But it doesn’t end there. There need to be managed service providers who can offer service contracts to maintain the system in place once it is on a customer’s premise.

Banu Nagasundaram: All of these building blocks in the value chain is what makes ML, machine learning, in production, intangible for a customer to realize value. It is a big journey. And this is the team I’m building who will work with individual partners across this value chain. And if this is something that excites you, we can talk after. In that value chain, we started with value creation. What is it that companies are looking for? We saw the value chain, but the value realization, let’s say the companies went through this process, put this whole system in place.

Banu Nagasundaram: What is it that actually helps them realize that value? When we think about data analytics in IT, many of the ML practitioners that I talk to, talk to me about the output, the visualization, the dashboards, the histograms for decision making, but it doesn’t stop there. That is not sufficient for these companies. There has to be people in those companies who take this output and actually work to achieve an outcome. The outcome is increase in productivity, increasing in throughput. It’s like, if I can know my demand better as a retail store, or if I can forecast the demand better, I can do so much better in my business.

Banu Nagasundaram: That is an outcome that I’m looking for from this data output that I’m getting from the machine learning models. But is that outcome sufficient? No, the outcome is like one place, one time, you are able to visualize that outcome, but you have to scale that outcome globally in order to achieve the impact. The impact that businesses look for is that you have to either increase revenue, reduce costs, reduce their risk, improve their sustainability, or create a competitive advantage. This is what the ML journey in production looks like. While I walked you through incrementally from the beginning on what it would take customers to get there, customers don’t work through individual steps, reach there and see what’s the next step.

Banu Nagasundaram: They actually have to make assumptions along the way and then understand what the impact might be and this is an AWS term that I’m going to throw and work backwards, which is understand what is it that you want and then build out what is it that you can, what you need to do towards achieving that end goal. That’s the big picture that I want to leave you with. In this whole ecosystem of having machine learning in production, can you think big on behalf of the customer, can you seek the big picture that the customer is looking for? If you’re working on a feature, what is the end state of that feature? What is the end state of that business? Who is actually your end customer? Your end customer may not be the team that you’re immediately working with and who is the decision maker for that overall flow?

Banu Nagasundaram: One of the simple frameworks, this might sound silly. It’s super simple, is to ask five whys, which is in a customer discovery or any feature that you’re building. Just ask, why is that outcome important? Why is that output important? How is it going to help? Why is it going to be something that helps the customer? Why is it needed tomorrow? All of these questions is just going to help drive a little bit more clarity into the bigger picture and motivations for the customer on how they make investment decisions and choices in your particular products and features that you’re building. And that’s it. I want to leave you with one fair warning. If you try that five why’s with your partner, that’s on your own.

Julie Choi: All right. Thank you. Banu, that was great. Thank you so much. It is my honor to introduce Lamya Alaoui, a dear friend of mine and I’m so thankful to you, Lamya, for agreeing to give the closing talk of the night. Lamya is currently Chief People Officer at Hala Systems, and she has been committed throughout her career to supporting organizations as they shift behaviors to align their talent strategies with their business objectives. Her corporate background includes over 15 years of experience in talent acquisition and management, where she has had the opportunity to build teams for companies such as Bertelsmann, Orange, Groupon, Google and Microsoft. Her work experiences span North America, Europe, Asia, Middle East, and North Africa. Let us welcome Lamya Alaoui.

mosaicml girl geek dinner lamya alaoui speaking hala systems

Hala Systems Director of People Ops Lamya Alaoui talks about 10 lessons learned from building high performance diverse teams at MosaicML Girl Geek Dinner. (Watch on YouTube)

Lamya Alaoui: Thank you so much, Julie. Hi everyone. Thank you. I don’t see Sarah here, but I want to thank her personally because she has been so patient and Angie as well for inviting me here. How is everyone feeling today? This is the non-tech talk. This is the people talk. A quick background before we get into it. At Hala System we develop early warning systems in war zones. Not the type of things that you broadcast usually when you are in this type of settings, but we’re hiring as well, especially for our AI team. And Julie announced my promotion that no one else knows about, even in the company so, thank you. That stays here, please, in this room, until next all hands, on Wednesday. With that being said, a little bit of background about me. I’m Moroccan.

Lamya Alaoui: I moved to the US about 10 years ago. I will ask a lot of grace because my brain is wired in French. Sometimes there will be French words that will come out from my mouth so please be graceful about it if you can. With that being said, this is one of my favorite things because as a Moroccan who’s half Muslim, half Jewish, went to Catholic school, I thought I got it covered, but I was the lady the first time I landed in Germany because I was not aware that you’re not supposed to kiss people to greet them. But this is one of my favorite pictures to show and talks or even in workshops, because I’m pretty sure it happened to all of us one way or another. We show up and we think that we got it right. It’s what we’re familiar with and actually it’s not what the other person is expected.

Lamya Alaoui: Throughout my career, I have built teams in very, very different countries, different cultures. At Hala we have over 17 nationalities. Altogether we speak 25 languages. It makes it interesting for the meetings, believe me, where you have side conversational Slack in a whole different language, but you still need to deliver officially in English. The translation is super weird sometimes, but those are anecdotes for another time. That leads me to one of the lessons that I learned very early on. It seems obvious that when you’re building teams and you want them to perform, one might assume you want to know your mission and your values, but it doesn’t happen as often as one might think. The mission is basically your GPS. You want people to rally behind a common goal.

Lamya Alaoui: You want them to believe in that and it’s also what’s in it for them when they’re working for a mission. That means that it needs to be aligned with their own values or their belief system. SVlues start very early on. We all have seen or worked for companies where the values are stated. How many of you when you joined the company saw values listed on the website or have been talked to? How many of you were explained to what are the associated behaviors and expectations when it comes to those values? Oh, we have less people. This is one of the first lessons that I always recommend to people to kind of follow. I learned the hard way, by the way, is that know your mission, state your values. Values should come from the leadership of course, but also account for it in the hiring process, which we will get a little bit into it, but be very clear about what are your expectations are when it comes to values.

Lamya Alaoui: If I’m thinking about transparency, for me, it means having information that I need to do my job, but some other people, they think that they need transparency. They need to have access to everything. We all have someone like that in our companies. Don’t we? Setting those clear expectations and associated behaviors are very important. And mission is like literally your North Star. This is why it’s really important. The second one is seek first to understand. This is from Stephen Covey. There is a second part to it which is, then to be understood. When you are building diverse teams, we all come from different backgrounds, we have different understandings, often we speak different languages. We can also come from different cultures, high context or low context and if we don’t have mutual understanding we cannot succeed because then everyone is convinced that they’re right, that their approach is again the one that everyone needs to follow, but that kind of hinders teamwork, which is actually essential to high performing teams.

Lamya Alaoui: Listen. We all think that we listen pretty well, but we actually don’t because as human beings we have been trained, for the last few centuries, to listen to answer. We’re almost never listening for the sake of just listening and there is amazing research about that, that I will be more than glad to share. In working in very diverse teams or diverse teams in general, listen, help, create understanding, respect, which is a very important when it comes to having high performance teams as well. How many of you would say that they listen really, really, really well. Okay. We have four people in the whole room. That gives you something to think about. That’s another, and I’m a good listener, but still a lot of work to do with that.

Lamya Alaoui: Acknowledge that you will face cultural differences. When you join a new team, we all want to get along. Again, its human nature. We want to belong somewhere. We’re super excited. It’s a new job. We just went through the hiring process. We landed the job and then you show up in that meeting or you’re meeting your team and sometimes you see people that you have nothing in common with, and you still try to ignore that, which is again, human nature. Beautiful thing. Just as a manager, as a team member, be prepared that yes, we will have some challenges. We’re not seeing things the same way. One of the best examples that I always give here is like, I’m from what would qualify as a high context culture, which is there is no explicit direct messaging.

Lamya Alaoui: Basically I can be in my office and say, hey, I have a lot of work and I’m hungry. My expectation is that my coworker will just get it and go get me some food. Now, imagine if I’m having someone from a low context culture. In their head, they’re like, I mean, you’re going to go get your food when you’re done. That creates a fracture that it’s not even intentional because there will be resentment from my end. Why didn’t he get it? Having those conversations upfront is quite helpful because then people know again, what to expect. Assess the degree of interactions. When teams are being built, most of the time what happens is no one is thinking about how often the team members will have to work together, at what intensity.

Lamya Alaoui: Before starting to build the teams or adding members to your teams, just make sure that, hey, how often do they have to have, I don’t know, sprints. If it’s once a month, eh, you might want to have people that are pretty much aligned having somehow the same thought process and we’ll get to that in a little bit. If they’re not interacting a lot, you have a lot more room to assemble your team. Always, there are three degrees of interactions. We have low interdependency, which is people are doing their things on their own and sometimes it’s like going in a [inaudible]. Medium, which is a hybrid and then high interdependency, which the output from someone becomes the input for someone else.

Lamya Alaoui: Therefore, things need to be structured in a very certain way in making sure that people get along and understand, again, what’s expected from them. Communicate, communicate, and communicate and when you think you are done, communicate some more. One of the things about communication is that we all think that it happened. Again, the question, oh, disclosure, I ask a lot of questions. I mean, I used to be a recruiter. It comes with the territory basically. How many of you thought in a meeting that you were crystal clear and then two days later you discover that you were not.

Lamya Alaoui: When it comes to communication in teams, again, thank you, Slack, everyone in meetings, or usually where everyone is doing a lot of things at the same time. You’re talking, you think that you got your point across. Always ask a question at the end. What do you think? What are your takeaways? Just to make sure that people are on the same page and teams, high performing one’s, one of the best practices is having action items assigned, even if it’s an informal meeting or if a conversation happened on Slack, on, I don’t know, any other platforms. Even sometimes now, oh, we have also Asana now people add us to Asana and they think that, hey, we’re good.

Lamya Alaoui: No, make sure that the person actually went in, is understanding what you’re waiting for or expecting. Diversity and leadership. Clearly icons still have a long way to go in terms of diversity, but again, high performance teams when it comes to diversity need to see that diversity reflected in the leadership and in the management. Research shows that usually entry level management, there are sometimes good numbers, but as soon as we go to director, VP and above underrepresented groups tend to become less visible or they’re non-existent and C-suites and VP, EVP, SVP type of roles.

Lamya Alaoui: It’s also very important that the leadership reflects the diversity that the company is striving for. Again, an obvious one, hire the right people. This one is very dear to my heart because as I said, I’m a recruiter. Again, we’re only humans. We tend to hire people who are like us, who think like us, who share our values. Anyone wants to venture what happens when you hire people who are like you and your team and you’re building a team of clones, basically from a cognitive perspective. Anyone? Yes.

Shika: You wouldn’t get to learn something different from what you already know.

Lamya Alaoui: Yes. Anyone? Thank you so much. What’s your name?

Shika: Shika.

Lamya Alaoui: Shika. Thank you, Shika for volunteering. When you build a team of clones and this is a tech talk after all, how do you think that you can innovate? How do you think you can perform if everyone is thinking the same way and not challenging the ideas that are being discussed? Chances are very, very, very slim.

Lamya Alaoui: When you’re hiring the right people for your teams you have to look at three things. The first one is the values, because that’s, again, what will be the foundation of the team that you’re building. The second one is what’s missing from your team. An ideal team has five type of people. You have a theorist, you have a strategist, you have someone who is an analyst, you have a manager and you have an implementer. When we hire our teams usually we hire in desperation because you know, you got to deliver on that project.

Lamya Alaoui: No one is thinking about the composition of the team in itself. In tech, my experience showed we have a lot of theorists and strategists and a lot of analysts, sometimes zero implementer and zero managers. Things don’t get done somehow. Always look at what’s missing and finally, the third part is people who are willing to learn, who have a curious mindset and are eager to grow, because this is how actually innovation happens. People who are not afraid basically of questioning the status quo.

Lamya Alaoui: The other lesson is constantly scan for misunderstandings and ways to clarify. Again, here I will go to the low context, high context type of thing so, let me give you an example. I think it might be easier to make my point that way and I will ask a question at the end. In high context cultures or in some cultures when you ask questions, no, is not an acceptable answer. You have to say yes and some others you have to say no three times before saying yes.

Lamya Alaoui: Imagine when you have people from different cultures in the same room and someone is asking a close question that requires a yes or a no. What are the chances, based on what I said, that you will get an accurate answer to your question? 50%, 60%, 20%, a hundred percent. For this always ask open ended questions to give people space to answer without having to break their own boundaries and to make sure that you are getting clarity in the answers. How many of you were in meetings where you felt that you were misunderstood when you were speaking? What did you do? Anyone wants to volunteer again? When you felt that you were misunderstood? Yes.

Angela: Have you summarized in a written notification? I gave action items to the people I wanted clarification from that helped with the misunderstandings.

Lamya Alaoui: Thank you, Angela. One of the things that is really helpful and again, best practice for high performing teams, always summarize and put things in writing and this setting English is the working language. We do have a lot of people, I know in my team, I have a lot of non-native speakers. I’m not a native speaker so to make sure that the message gets across and that it’s clear it’s always in writing, always, which we tend not to do because everyone is working long hours or you have back to back meetings. Then at the end of the day you just want to finally get to do your work and you never summarize in writing what happened in those meetings.

Lamya Alaoui: Finally, the structure. Build a structure that is around diverse teams. When we’re usually building teams we have, we’re looking at what now is referred to as visible diversity, which can be ethnicity, religion, gender and many other things, but we’re never looking at the cognitive one that also has a heavy part, especially in tech. Those are recommendation that I’ve always lived by every time that I join a new company. We do a reintroduction and relaunch of the core values. We redefine them. We create, and this is like a company work, it’s not just someone in their corner doing it, but it’s a collaborative work setting what are the core values, what are the expected behaviors and the ones associated with each one of the values, and then it’s communicated throughout the whole company. When it comes to performance review it needs to be designed with underrepresented communities in mind.

Lamya Alaoui: Performance reviews, if you look at the history of how it was designed, it was designed for a very specific community or population. It doesn’t fit other people. Just leave it at that. Work on creating a performance review management system that is more inclusive. Yeah. Targeted networking. Again, this ties to the diversity and leadership. One of the things, again, hiring in despair. When those roles are open, people tend to hire fast from networks that they know because they don’t have networks that are already established. This is something that is quite helpful. I don’t think there are any people in the HR realm tonight, but all of you can be ambassadors for something like this to be established in your companies. And again, clear expectations around the engagement and the roles. What are the pathways to promotions? Again, if you look at internal promotions or internal applications, woman tend for, this is just an example, women tend not to apply to internal openings when it comes to higher positions in the company.

Lamya Alaoui: Sometimes just because they don’t check all the boxes so the work starts before that, where we actually need to make sure that the job descriptions or the job openings are reflecting and are being thoughtful about underrepresented groups. Finally, this is one of my favorite things, Kaizen. There is always room for improvement. Performance and diversity are a very long journey. Each company is at a different stage or different phase. Just ask questions, be kind because not a lot of people think about it when it comes to those things and be as curious as possible because at the end of the day the only way to know other people is to ask questions. Thank you so much everyone.

Julie Choi: I want to thank all of our speakers tonight and for all of you, for being such an incredible audience. We have people who have been watching these talks outside in the overflow and I’m told that, I think it might be better if we just all go out and just mingle. Get some fresh air after all this time is for us to connect and network. Why don’t we continue the conversation outside? Thank you everybody.

mosaicml girl geek dinner ukranian original borsch tshirt

This site reliability engineer discusses Ukranian borscht or machine learning, or both, at MosaicML Girl Geek Dinner.

mosaicml girl geek dinner speakers tiffany williams banu nagasundaram laura florescu julie choi lamya alaoui shelby heinecke angela jiang angie chang amy zhang

MosaicML Girl Geek Dinner speakers after the event: Tiffany Williams, Banu Nagasundaram, Laura Florescu, Julie Choi, Lamya Alaoui, Shelby Heinecke, Angela Jiang, Angie Chang, and Amy Zhang.

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Cadence Girl Geek Dinner – Lightning Talks & Panel! (Video + Transcript)

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Transcript of Cadence Girl Geek Dinner – Lightning Talks & Panel:

Angie Chang: Hi. Welcome everyone to the Cadence Girl Geek X event. My name is Angie Chang, and I am the founder of Girl Geek X. By means of introductions, I mean to say my name’s Angie Chang and I’m the founder of Girl Geek X and Women 2.0. I also spent some time working at a company called Hackbright Academy, which is a women’s coding bootcamp. I also spend a lot of time talking about women starting high-growth, high-tech companies, working in tech, blogging about it. Sometimes I write listicles of women architects, women CTOs, VPs, security chiefs and such.

Angie Chang: Let’s see. Why don’t we do something? Why don’t we pretend that we have our “Hello my name is” name badges, and write in the chat like, “Hello, my name is Angie Chang, Founder, Girl Geek X,” and then put in your LinkedIn URL. I’ll start. I’ll copy and paste this into the chat. That way… I’ve noticed that people at our Zoom events have been sharing their LinkedIn profiles. I want to be the first person to say that, yes, let’s definitely share LinkedIn profiles and share a bit about ourselves more than we can see on these Zoom meetings or Zoom webinars.

Angie Chang: Let’s see. A bit about Girl Geek X. We’ve been doing Girl Geek Dinners since 2008. We started with events at Google and Facebook when they were smaller companies in 2008, and then we went to all these different tech companies. We went to biotech companies. We went to a bunch of companies I’d never heard about before.

Angie Chang: But the thing about that is once I was there, I would learn so much about that company. I’d learn about the industry. I’d learn about the women that worked in it. I would see their job titles and I would be very inspired and educated at that point to recommend that company say to friends. Also, it was really great for networking.

Angie Chang: Hopefully if you have time, you can hang out tonight. Later at seven or so, we’re going to start the networking and Zoom breakout rooms where you can actually chat with each other and connect more in person. But if you can’t, it’s okay, this talk is recorded. All of our events are recorded and put on YouTube later, and that URL is

Angie Chang: What else? If you want to look at all the events that we have hosted in the past, they’re all on our website. It’s at, and you can find all our previous events. For example, we were at Discord a week or two ago.

Angie Chang: We just wrapped our annual Elevate conference, which is something we do every year for International Women’s Day.We have an all-day event celebrating women and having a bunch of exciting women leaders speaking about topics like mental health and leadership, not just very ambiguous things. We literally had a keynote on decision making from a VP of engineering.

Angie Chang: We always have a call for submissions. People can in the fall apply to speak at that conference. About 10 of those people who applied to become speakers became speakers at Elevate. There’s definitely a chance that if you submit something, you can be selected to speak. You can also sponsor a Girl Geek event like the Cadence event, where you have an opportunity to put your women on stage and give new tech talks followed by a panel. Then we have some networking.

Angie Chang: There’s just so many companies out there. That’s I think a great opportunity to get out there in front of a bunch of eyeballs and create some great talks that we then put on YouTube. What else? Oh yes. We have a Q&A. If you have a question throughout the course of the event tonight, feel free to put it in the Q&A or you can ask it in the chat, but there is a Q&A feature, so feel free to use it. Some of our speakers may want to answer questions, but they may not have time to answer them on screen so they can pop into the chat and answer them later if you ask them. I want to do my first introduction.

Angie Chang: Alinka is the chief legal officer and corporate secretary at Cadence. She’s responsible for Cadence worldwide legal operations. She has served semiconductor and software companies and her entire in-house legal career at a lot of companies that you may have heard of. Before moving in house, she was in private practice for a decade litigating chemical product liability matters. Welcome Alinka.

angie chang girl geek x alinka flaminia cadence
Cadence Chief Legal Officer Alinka Flaninia welcomes audience to Cadence Girl Geek Dinner 2022.

Alinka Flaminia: Thanks Angie. Thanks for the introduction. On behalf of Cadence, we are so honored to partner with Girl Geek X to host this conference in celebration of Women’s History Month. Women have played a significant role in Cadence’s 34-year history, and I’m thrilled to share with you some of our efforts to create a more inclusive and equitable workplace for women and underrepresented groups in STEM. But before I describe some of our DEI efforts at Cadence, let me first tell you a little bit about the company, for those of you who are not familiar with us.

Alinka Flaminia: Cadence is a pivotal leader in electronic design, building upon more than 30 of computational software expertise. Manifesting our intelligence system design strategy, Cadence delivers world-class software, hardware, IP across all aspects of the design electronic systems. Our customers include the world’s leading companies, delivering extraordinary products from chips to board to complete systems for the dynamic market applications, including cloud and hyperscale computing, 5G communications, automotive, mobile, aerospace, consumer industrial and healthcare. It’s fantastic to work at a company where the same set of tools enables innovation across such a diverse set of industries.

Alinka Flaminia: Actually for me, it’s kind of mind blowing, and I believe that the true enabler behind Cadence’s success is our high-performance inclusive culture. Our one Cadence, one team spirit is core to who we are, and embracing diversity and fostering inclusion are key tenets of our Cadence culture. Cadence encourages and fosters diversity equity and inclusion on many fronts, internal and external, through recruiting and university partnerships, education, leadership training, pay equity and promotion and building community.

Alinka Flaminia: A few examples include our sponsorship, diversity and technology scholarship programs for women, black students and Latinx student to support these underrepresented groups in their pursuit of STEM education. We celebrate and support our employee-led inclusion groups for black, LatinX, veteran, LGBTQ+ and women employees and their allies to build community at Cadence and beyond. Cadence offers professional development through advanced leadership and mentorship programs specifically geared toward our girl geeks and black and Latinx employees.

Alinka Flaminia: Cadence is investing in the pipeline of a more diverse employee population through partnerships with nonprofits and organizations that serve underrepresented groups in STEM like the National GEM Consortium, Society of Hispanic Professional Engineers, National Society of Black Engineers, Society of Women Engineers, Out In Tech, Girls Who Code, and I could go on.

Alinka Flaminia: Our culture has been recognized globally, earning Great Places To Work awards in 14 countries around the world, including seven years in a row on Fortune’s 100 Best Companies To Work For, rankings on Europe and Asia’s Best Workplaces, listed on Newsweek’s Most Love Workplaces and the Best Place To Work For LGBTQ+ Equality on the Human Rights Campaign’s 2022 Corporate Equality Index. Diversity, equity, and inclusion are top priorities for me and the rest of the executive management team, and our board of directors.

Alinka Flaminia: We are excited to work with Girl Geek and highlight in this conference some of our amazing innovators at Cadence and hear how they are helping us solve technology’s toughest challenges. I am not a technologist, but I am most definitely a geek. Regardless of your specific interests, we girl geeks are united by our passion, our drive, and most especially by our curiosity. The speakers and panelists are some of Cadence’s very best girl geeks, I’m certain they’ll stoke your curiosity about our business and provide great tips for advancing your career. Thank you for joining us today, and I’ll now turn it back over to Angie to introduce our first speaker.

Angie Chang: Thank you, Alinka for that warm welcome from Cadence. Our first speaker tonight is Helen Zhan. She graduated from the University of Tianjin in 2000 with a double major in computer science and economics. She began her career as an IP/SOC engineer at NEC in Japan focusing on both design and verification. Shortly after that, she joined Cadence, where she’s been for the last decade plus. Her passions for debugging failures and finding the root cause of issues has allowed her to grow her career at Cadence. Welcome Helen, who is joining us from Beijing tonight.

helen zhan cadence growth engineering beyond metrics
“Growth Engineering Beyond Metrics” by Helen Zhang, Cadence Design Engineering Group Director.

Helen Zhan: Thank you everyone. I’m Helen from Cadence. Today, I would like to share my story along with the team girls. Okay. I will use them putting in my later talk. I will use the integrated IC, IP, IPG, DDR, LPDDR, PHY and Mbps in my later talk.

Helen Zhan: Firstly, I would like to introduce the function group of my team. Similar to other any centers, we do have the six different function groups along with our journey like the different kind of the design, verification, solution team and the success team. With all this different function team, it provide the complete system solution for the memory system of each customers.

Helen Zhan: Our team did start from the 2011. Then we have the big growth from that year. After three years, we did break into top four of the senior IP core ranking and along with that long journey, we did achieve many milestones of the word first silicon to ensure our IP become more challenging to the market.

Helen Zhan: Today, I would like to share with the story of the team grows. When we start the team, we need to find the proper goal of our team. So we had we will be the market co-developer of the team start. We did have some big investigation of the market. We find there are two type of the current IP company. One is like the flea market, which you could find everything you want, but that may not give you repeatable supply and with good quality. Another is like the mega store. You could find the product with a good quality, but it may not satisfy all your requirement.

Helen Zhan: Where should we go? We leave this question to the marketing investigation. Then we find in cloud market, we do have the different application like the mobile application, consumer application, cloud application and also automotive application. Often shows the market orientation thing. We figure out our target is to build a different style of the IP vendor and supplier to make a customized configurable solution with a good quality to fulfill the different customer request, satisfy all the products.

Helen Zhan: With this clear goal, then we will build a good team and to make the team improvement to satisfy the market requirement. As I started from the beginning, we did have six different function groups. Today, I will check the digital design group as an example for the team roles. When the team grows, we need to divide the team into different separations to have the different function focus. With that kind of, we could have the expert in each field to make our product become more productive and have the leading age technology.

Helen Zhan: With this subdivision, we need to have the clear ownership of each different field, and we do not want to limit any engineer in that field only. We would also like to expand his focus in different area. We want the clear ownership of the each field. In the meantime, we also want the mixed function focus of the different area. With this kind of definition, we could increase the team skill and we could also back up each other and improve together. This makes the team become better and more productive and efficient with a single one.

Helen Zhan: In the meantime, with a complex of the IC product, we have the clear boundary of the each function group, but we want that boundary to have the team have its own focus. We do not want that boundary become any barrier of the cross-function group communication. we would like this boundary become multi care. That will stress our boundary to avoid any backslide and any part we are missing in the development that to avoid any for other issue and surprise in the later production.

Helen Zhan: With all this strategy and the teamwork, we did build a fully verified DDR subsystem. As you can see in this picture, we could support the different configuration product, and it could have the customized features, which could satisfy the different market requirement like the mobile, automotive, cloud and consumer. That also help us that our DDR product in the leading edge to have won more customer today

Helen Zhan: With this 11 years journey, our team is already increased to a big size. How do the junior engineer now become the senior engineer and the senior becomes an expert and with the supervisor like the leadership? But how to help this big team become more stable and more productive. We considers the two area. One is from the technic side. We want the team always stand at the leading edge of the product. That allow us to always to work with a new protocol and to achieve the highest speed in the world.

Helen Zhan: When we start a product, when we define the protocol, that means there are many unclear area which won’t be exist when we start the product that need us to have the flexibility design to accommodate any new requirement coming later. Also, we need to use our experience to doing the predict and analyze for the orientation to avoid we are going to operate position to the market. Also the high speed is everyone is chasing today. We need to build the high speed architecture to satisfy the design requirement.

Helen Zhan: With this technical innovations, that allow our expert to have their own focus that ensure they always have the interest of this product in their career path. In the meantime, to leverage different Cadence, we could always use the latest methodology and advanced technique that help us to use a new design methodology in that every field and everywhere along with our IP development. Also, we adopted advanced flow and tool in our IP quality check to ensure we have the product with good qualities.

Helen Zhan: The silicon proven is another advantage here because with more and more high-speed requirement, the silicon proven as a big fact for the customers who want the IP supplier to provide. In the meantime with a big team, we also want to improve work efficiency. When we start any new product or any new feature development, we will need to avoid any one-off development to make our effort could be reusable or repeatable in the later product. In the meantime, we also need a comprehensive quality system to have the issue being detected earlier, to send a alert to the design team or other development team to avoid any later surprise in the customer products

Helen Zhan: With all this QC and the strategies we development the automation flow that could help us to release the manual resource to focus on the technical side. In the meantime, it’ll also reduce the effort and error with the manual operation. With all this strategy and technical, I believe that communication is the most important to have all this done. First, I would have to introduce these three word, look listen and learn. We need to look what the team’s working like, what the customer require and what’s the marketing requirement.

Helen Zhan: We also need to listen to the voice from everywhere that will have the leader to clear the request and clears the issues in our team and in the market and brand customer. With all these facts, we could learn what we want to do in the next space and in the next step. Also, we could find what we should do to solve the current to and improve the team. Then I believe the different type of the talk is also very important. That is not the leader to talk to the team member. It also want team members to talk his mentor, his mentor, and his leader to share are the different ideas.

Helen Zhan: I believe everyone did have its own thought on his work. Then we need a clear communication between each channel to help the team members understand the requirement and purpose and the goals of the leader and the management team and also help the leader to understand the consent or problems in the team member side to help them to solve that. With all this good communication and consideration, we will make this become execution that is to help us to make our goals and consideration become true. Thank you.

Angie Chang: Thank you, Helen. Our next speaker is Elena. She is currently the Global Public Relations and Social Media Director at Cadence. Previously, she had held communications and marketing roles at AgilOne, Coupa Software, SugarCRM and more. She spent over five years freelancing and consulting to communications and marketing. Welcome to Elena.

elena annuzzi cadence finding your growth career path

“Finding Your Growth Path” by Elena Annuzzi, Cadence Global Public Relations and Social Media Director.

Elena Annuzzi: There we go. All right. Thank you, Angie, for the intro. Hello, everyone. Welcome. As Angie said, my name is Elena, Elena Annuzzi. I’m the Global PR and Social Media Director at Cadence. My presentation tonight is going to be focused on Finding Your Growth Path.

Elena Annuzzi: How many of you have ever felt stuck in your career and you’re trying to figure out how you might be able to move it along? I think a lot of times it’s seen that promotions are an obvious way to move yourself along in your career, but there are also a lot of other things that you can do to propel your career and take it to the next step. I’m going to talk to you tonight about my own personal growth journey and also impart some tips that you can leverage to find your own unique growth path.

Elena Annuzzi: With that, I will start with my own personal growth journey. I’ve been in technical communications positions for the last 22 years. A lot of my career has been spent handling public relations, but I’ve also spent a lot of time doing analyst relations programs, content marketing, social media marketing in customer marketing programs. When I started, my role was strictly doing PR and I worked in house in a corporate environment.

Elena Annuzzi: A lot of times people in my career field do start in the PR agency realm, where they have access to lots of training and resources. I kind of missed out on that a little bit. I did find it a little bit difficult to be in a corporate position and kind of rise through the ranks in there, but I absolutely did best that I could to try to learn different facets of the business. Ultimately, I decided I wanted more growth, which led me to a path to consultancy. When I did that, I worked for a few Bay Area PR firms and also had clients of my own.

Elena Annuzzi: I definitely had my hands full for sure, but what that it is it kind of pushed me out of the comfort zone PR box that I started in. I got to dip my toes into other areas such as the ones that I mentioned, analyst relations, customer programs, things like that. It also imparted a lot of confidence in me as well because I had clients who were in all different industries, big or small. Oftentimes, if they were small, they might have been a one person marketing shop, so they were looking to me for leadership.

Elena Annuzzi: That really gave me the confidence to become the leaders that they needed and also acquire a much broader skillset than I ever anticipated. That was really a great period that I think then after five and a half years, I decided to reenter corporate. As I did that, I came in at that point as a very experienced person, leading teams and working on projects to get visibility for the firms that I worked in a lot of times from the ground up. They had never had PR before and they didn’t know what to do, so I kind of in there to build it back up.

Elena Annuzzi: Now, for the last seven years, I’ve been at Cadence. I started at Cadence as a senior manager in an individual contributor function. Now, I’m the director of the group, and I manage a team. The team is responsible for handling anything that is publicly distributed in the form of news releases, as well as contributed content. All the social media platforms are managed by our group and a variety of other things. We also work very closely with executive management. There’s lots of things that are sensitive or require them to do media interviews and things like that. Definitely something I really enjoy.

Elena Annuzzi: I’m glad that I had the opportunity to try lots of different things. I kind of take my consulting experience and I’ve sort of taken that along with me as I’ve gone along through my career. I try to always look kind of at the company from an outside view and try to establish, “Okay, well, if I was consulting, what would I recommend that this company do?” I’ve kind of had an interesting path and I’m currently very happy at Cadence and I have a great team, all amazing people.

Elena Annuzzi: Let me now continue with some tips. I will share some growth tips with you. The first one I have is make sure that you’re having open in conversations with your managers. If you haven’t discussed a growth plan already, make sure that you do that. If you haven’t really thought about it, maybe write some notes down before you have that conversation so that you then go into that conversation prepared.

Elena Annuzzi: Another thing I would say is to offer to take on new projects that are outside your comfort zone because then you’re sort of pushed to try something that you may not have otherwise done. You may experience a very pleasant surprise and that something worked out so fantastic for you that it would definitely be worthwhile to make the investment to try something different. Another tip would be to find groups who have similar interests to you, and that way you can gain new inspiration from others, as well as make some good connections.

Elena Annuzzi: There’s lots of ways to do that online today, for example, and you may be familiar already with social media groups in your related career field, so feel free to take a look at those. LinkedIn is probably the most obvious place, but other platforms have groups as well that relate to professional fields. The other thing too is if you have local meetups, check some of those out or even leverage your university, if they have alumni groups and more specifically alumni groups within your field of study.

Elena Annuzzi: Lastly, maybe volunteer with an organization that is also passionate about the things that you’re passionate about from a work perspective, like say you’re volunteering with a STEM group and you’re in a STEM field. You may meet some great connections that way and gain some new insights.

Elena Annuzzi: Continuing on, the next tip is be relevant. What I mean by that is making sure that you’re always kind of staying fresh and up to date on what the industry’s current best practices are. How might you go about doing that? You can have conversations with others, whether they’re peer groups in your company or people that you’ve worked with in the past who hold similar job functions and just kind of ask them how they’re approaching their job. Obviously certain things are proprietary, so there’s limits, but you can kind of get a good gauge as to how others are tackling a similar job to you.

Elena Annuzzi: The other thing that I’ll recommend, and this is not meant to sound intimidating to employers in the least… It’s actually for your benefit… is to check out job descriptions. The reason that I say that is you can take a look at job descriptions in a role that’s similar to yours and even look at those that are above your level because then you’ll quickly figure out what companies are demanding of people in those functions today. You can quickly realize, “Okay, I have these skills, but maybe I’m missing a couple,” so you can identify the gaps and then work to figure out how you can get that experience in your current role.

Elena Annuzzi: That would be something to talk to your manager about. If you’ve identified a gap, “Here’s something that I’m interested in trying, let’s do that.” Then in looking at the job of positions above yours, then you also have a gauge of what to shoot for kind of in your next step. Similarly, if you realize that you have some gaps, then you can work to address those. The next thing I would say is acquire new skills, taking new courses or attending conferences where you’ll have access to new information that you may not have otherwise had, or ask your employer, your HR department, or your manager about job sharing.

Elena Annuzzi: If you’re not familiar with that concept, it would be where you essentially do a job swap for a limited amount of time. Let’s say you have a peer organization and you want to take on some function of your peer because you have an interest there and want to explore that, you can maybe switch jobs for five hours a week and both of you are actually gaining a new skillset by doing that. The next thing I would recommend is mentorship. I would say find a mentor if you don’t don’t have one or be a mentor. Both things are absolutely critical.

Elena Annuzzi: I am so glad that over the past five years or so, I’ve seen a lot of mentorship programs kind of budding in the industry. That’s really a great thing to see. I kind of wish that I had those types of things when I was first starting my career. Cadence also does a really great job with this, by the way. We have an internal mentorship program where they match pairs up. It’s really just a phenomenal thing. If you’re not already in the realm of finding a mentor or being a mentor, I highly recommend that. The mentor for you can obviously serve as a sounding board. Whether the person’s in your industry or not, or maybe they’re your manager, maybe they’re someone who’s completely disconnected from your field altogether, it’s great to have somebody who can function as that sounding board for you.

Elena Annuzzi: Also being a mentor. It’s such a rewarding experience to pay it forward. I highly recommend that you try this and there may be some of you who currently aren’t managing a team, let’s say. If that’s the case for you, being a mentor, that will give you leadership experience. I highly recommend that next.

Elena Annuzzi: Next, here’s a few points to keep in mind. No two growth paths will look the same. Try not to compare yourself to others. The next thing I’ll say is always be curious. I always tell my team members the minute you’ve accepted the status quo, you’ve stopped growing in your career. Always keep that explorer hat on and try to figure out what you could be doing that’s different. The next thing I’ll share is ensure that those new areas that you decide to explore align with your organization’s business. If what you want to try aligns with the business, then it’s a much easier sell when trying to get buy in.

Elena Annuzzi: The next thing I’ll say is surround yourself with people who support you, whether it’s people inside your company, outside your company. It could be a mentor or just your team members, your manager, people in peer groups, make sure that you have great support all around you. Then the last thing I’ll say is have fun in the process. We all need to have some fun.

Elena Annuzzi: To conclude, I want to encourage all of you to start taking steps today to grow your career path. Those moves that you take today will start impacting your career now and well into the future. As a key takeaway, remember that it’s you who’s in the driver’s seat. Thank you very much for your time.

Angie Chang: Thank you, Elena. That was excellent. Our next speaker is Didem Turker. She’s a design engineering director in the IP group at Cadence, where she leads development of high-speed, high-performance communications circuits and systems. Before joining Cadence, she was the Senior Design Engineering Manager at Xilinx in the service technology group. She holds 11 US patents and authored numerous technical papers in the field of analog and mixed-signal circuit design. Dr. Turker has a PhD degree in electrical engineering from Texas A&M University. Welcome Didem.

didem turker melek cadence engineering director ip group effective technical presentations a powerful tool for your career success
“Effective Technical Presentations: A Powerful Tool for Your Career Success” by Didem Turker Melek, Cadence Engineering Director, IP Group.

Didem Turker Melek: Hello. Thank you, Angie. Okay, let me share my screen. Okay. All right. Okay. Thank you for this introduction, Angie. Hello, everyone. I’m Didem. Today, I’ll talk about effective technical presentations and how they have a key role in your and your team’s success. Before I begin, throughout my career, I found that being able to will communicate my work to my colleagues clearly had significant impact on the type of feedback that I got, but also on my work being recognized.

Didem Turker Melek: Over the years, this is something I championed in the teams that I worked with and we always saw really positive results. I’m hoping that this discussion will be helpful today for you too. Okay, let’s begin. When we talk about technical presentations, they are different than the general presentations that we may give to a wider audience.

Didem Turker Melek: We also need to share data and talk about more detailed material with certain technical complexity. Now, throughout our career, there’ll be different occasions where technical presentation may be called for. This could be academic conferences, customer presentations or when we are collaborating across different organizations in our company, it could even be within our own team if this would be to our close peers, our colleagues and maybe our management.

Didem Turker Melek: It is this last one that I want to highlight because this is a situation that we encounter really frequently, yet it’s also the one that we overlook the most. I really want to emphasize how important it is to communicate technical information through well prepared, clear presentations and especially around the audiences, people that you work with every day.

Didem Turker Melek: Even though the occasions and audience may be different, there are common goals when we are giving you technical presentation. The first one is effective information sharing. Being prepared with proper organized material will make a big difference over opening live results, showing live data or giving a verbal description. This is true even in a more informal team setting because for a discussion where we have technical complexity to discuss, the audience will have a hard time following if you’re doing it verbally.

Didem Turker Melek: The second goal would be to get feedback. You probably have bright people from different technical backgrounds and experience living in your audience, so use that brain power. The best way to get good feedback from them is by communicating your findings in a clear way. Third goal would be to train others so people can learn from your experience and maybe save some time.

Didem Turker Melek: Finally, it’s documenting our progress. The presentation material that you prepare will serve as good documentation of your work. It’ll help you look back in the future to track where you have been at a certain time. It’ll also help others in the future to look back and understand your work better. Depending on the situation, one of these goals may be more dominant than the others in your talk and you can prepare your material accordingly.

Didem Turker Melek: Okay, let’s talk about some presentation tips. First is know your audience. It’s important to know who the target audience is and their familiarity with the material. But here, what I want to emphasize is that they are not you. What I mean by this is when we spend so much time in the details of our work, we tend to forget that what’s obvious to us is probably not obvious to others. It’s important to keep this perspective in mind when preparing your material.

Didem Turker Melek: I think something that helps with this, and it’s a really good strategy overall, is to have a story. As you plan your slides, remember to build this story so you can bring your audience up to speed and along with you. Start by setting the big picture, why we started. This would be where you talk about the goal, the problem definition and big picture stuff. Next would be how we got here. If there were previous discussion or decisions that were taken, try to recap. Next is where we are now. This was the main discussion that you want to cover. Finally, where we go next. It’s always helpful to finish with next steps and a plan.

Didem Turker Melek: Now, another very important tip is use your voice and your point of view. I can’t emphasize this enough. When you are presenting your work, please remember that you are an expert and this is true even if the audience have people with more experience. You are the expert on your own data. What can we do? Each slide should have at least one key takeaway that you highlight. Please avoid doing a data dump and letting the data speak for itself. It’s really important that you make observations because that’s your contribution. You can use metrics to help people interpret the data, metrics such as target value specification, maybe margin to that spec and so on.

Didem Turker Melek: Finally, don’t be afraid to raise possible issues and don’t be afraid to ask questions. Let’s look at some examples. Here is a slide you may encounter in a technical presentation. Now, this is what I would call a data dump. This is a bunch of numbers and while it may be obvious to you, for someone who just saw this and has only minutes to digest, it’ll not be clear. What’s the takeaway here? Is there a target value and what are the units?

Didem Turker Melek: How can we make this better? First, notice that I removed some columns. When you have a large amount of data, it’s helpful to do a divide and conquer approach and present it in smaller, meaningful pieces. Now that I edit the target specification, this will help set the key numbers here in this table into context. I’m also using getting a visual help by making the most important column, which in this case is bandwidth mode and using color according to mark failures.

Didem Turker Melek: Finally, in the second bullet, I’m including my key takeaway and observation from this data, which is that we fail the spec at certain cases. Now, in addition to the key observation from the data that I just showed, I can also build up on it by adding more information. For example, I can explain why I think this failure happens and propose a mitigation plan. Now, the goal of this is to facilitate the right discussion. This is why you think the problem may be happening and this is how you think you may be able to solve it.

Didem Turker Melek: By sharing it this way, you can get the right feedback about your plan and maybe come up with a better plan as a team. Okay. I want to pause here and add a bonus tip. While I mostly focused on how you can help your audience better understand the data, there is one significant benefit of having slides like this with clear points. Let’s go back to this slide. You may have attended presentations where someone needs to share large volumes of data, maybe 50 to 100 slides. Every now and then, a slide like this will appear and they will go, “What was I going to talk about here?”

Didem Turker Melek: Now, it can happen to any of us? Instead, if you have a slide like this, now, even if you’re tired or anxious or nervous, or if you’ve just lost your train of thought, you have all the help you need in your own slides. You have the key point that you wanted to make, you have the discussion points to help you, and you have the visuals to make that up. By preparing slides like this, not only you’re helping the audience understand you better. You’re also helping yourself present it in a more clear and easy way.

Didem Turker Melek: Okay, Let’s go with another important tip. Drive the discussion. As the presenter, we are in the driver’s seat. It’s our responsibility to guide the attention of the audience to key points. Please remember that just because something is on a slide, doesn’t mean that the audience will notice it. You can use visual aids like the ones that I used in the previous slide, such as bold lettering, colors, boxes and circles. You can also use keywords such as issue, risk, meets, does not meet to grab the audience’s attention.

Didem Turker Melek: Okay, let’s look at another example. Here I am summarizing some results. This is basically a big block of text. There’s too much information packed in this one slide. It’s too busy and it’s not easy to digest. You may also notice that it’s inconsistent in the way it talks about results. I first see a number about some typical corner. Then I talk about something else meeting a spec. I throw in some comments about some simulation set up or environment and then I throw in more numbers and more setup related material.

Didem Turker Melek: Instead, what I can do is divide this into multiple pieces such as first setup and then the results and clear it up. But there is one more problem that I want to show. I don’t know how many of you here even noticed this, but there seems to be a major issue and it’s buried in a small bullet in the text. Something does not work. If we want to talk about an important issue or make sure that our audience knows about an issue that we observe, this is really not the best way. Now let’s try a different way.

Didem Turker Melek: First, notice that I use the keyword in the slide, issues observed. Now, this will definitely get the attention of the audience and I. There is no doubt that now this issue will be noticed. Next up, I state the issue itself. On top of that, I add some explanation and a possible resolution. I also included data in a graphical format. Now, whenever we highlight a key discussion point, it’s very helpful to have the data to back that up especially in a visual form like this.

Didem Turker Melek: I do want to note that when you include graphs, please remember to include axis titles because again, they may be obvious to you, but it may not be obvious to everyone and it makes it much clearer this way. Overall, when I present the issue like this, it’ll help me highlight and make sure that I get the right feedback and it’ll facilitate the right type of discussion

Didem Turker Melek: All right. Let’s recap with some key takeaways. First, well-prepare, technical presentations are powerful tools to help you communicate your work better, and you can utilize them in your weekly or regular technical meetings with your own team too. Two, if you’re presenting data, do it in a clear and organized way, so you’ll be accurately interpreted. A bonus tip here was that well organized slides will actually help you too when you’re presenting.

Didem Turker Melek: Third, for effective communication, use your point of view and guide the audience’s attention to where it needs to be. I’d also like to add that this is a skill like any other and practice will make it better. Start preparing those slides, everyone. Okay. Thank you. Thank you for your time.

Angie Chang: Thank you. That was excellent. Now, I’m going to bring up our panel and introduce to you our moderator for tonight. Jeannette Guinn leads the demand generation marketing organization at Cadence. Her experience includes a 20 plus year career in B2B tech marketing, owning a floral business and performing vocals of various cover bands across the Bay Area. She has volunteered as a Court Appointed Special Advocate, CASA, to foster children and currently serves on the Child Advocates of Silicon Valley board of directors. Welcome Jeannette.

rishu misri jeanette guinn dimitra papazoglou karna nisewaner cadence girl geek dinner
Clockwise from top left: Rishu Misri, Jeanette Guinn, Dimitra Papazoglou, Karna Nisewaner.

Jeannette Guinn: Hello, good to be here. I’m sorry. My audio cut out when you started the introduction. I’m assuming we’re going to kick this off. Hello everyone and welcome to the Cadence Panel on Women Empowerment. My name is Jeanette Zelaya Guinn, and I’m the Group Director for the Demand Gen Marketing Team here at Cadence. It is a true honor to be here today and it gives us a wonderful opportunity to have our voices be heard and valued. I’m joined here on the virtual stage by three amazing Cadence colleagues. To get this, this discussion going, I’d like to take a moment for each of them to do a quick introduction. Karna, let’s start with you.

Karna Nisewaner: Hi, my name is Karna Nisewaner, and I’m a vice president and deputy general counsel in the legal department here at Cadence. I started my career as an engineer, studying engineering at Princeton before moving to Singapore to teach basic electronics and seed programming at one of the polytechnics there before I pivoted my career over to law.

Karna Nisewaner: I’ve been honored really to be able to work for a number of different technical companies and for the last almost 11 years here at Cadence. I feel like my background in technology makes me a better lawyer for the company and allows me to really engage with all of the different teams and people here at Cadence. To me, that’s one of the best things about starting out your career studying technology is you have all these different options available to you, both as somebody that’s designing the IPs to somebody that’s marketing and telling people about stuff to somebody that’s helping on the backend with the legal patent protection, IP protection, or just basic contracts.

Karna Nisewaner: It’s just really so exciting to be part of what I think of as the future of the world, which is technology. For me, it’s great to be at Cadence, a place that’s really helping all these companies out there build the future. I’m just so excited to see where things can go. That’s why I really love my job and my company.

Jeannette Guinn: Awesome. Thank you so much, Karna. Thank you for being here. Rishu, let’s go to you.

Rishu Misri: Thanks Jeanette. Hi, I am Rishu Misri Jaggi. I work with Cadence as a senior principal technical communications engineer, but that’s a very long title. Doesn’t mean that I do the most important job at Cadence, but what it does mean is that I work with an organization that is at the center of technology, that I work with a male-dominated workforce.

Rishu Misri: Being a woman and a mother working at Cadence, what it means is that I get to maintain a very good work-life balance. I get to spend a lot of time with my kids whenever needed. I can attend to the parent-teacher meetings. At the same time, I can also be at the [inaudible] working and supporting on technology advancements with my other male counterparts. I can volunteer for various Cadence-sponsored community outreach programs that are focused towards empowering other women, kids and students.

Jeannette Guinn: Wonderful. Thank you so much, Rishi. To close it out with Dimitra on your introduction.

Dimitra Papazoglou: Okay. Hi everyone. My name is Dimitra Papazoglou, and I’m an application engineer at Cadence. I support the analog and mixed signal front of Cadence tools. My base is in UK, so it’s a bit late for me, almost 2:00 AM. At the same time, I need to watch my daughter. She’s 12 months old. She’s sleeping, so that’s good. That’s good. We can go and continue.

Dimitra Papazoglou: I’ve been working with Cadence nine years. I joined Cadence straight after university. I can say that I built my career at Cadence. I want to share with you my experience so far. When I started, I realized very quickly how challenging it is to work in this male-dominated industry. I still remember my first visit when I visited customer site and there were 10, 15 men, very experienced, and I was on the other hand very young and with no experience.

Dimitra Papazoglou: Since then, I had been trying to find answers to questions like how should I… What is the right position to stand? How should I use my voice? How can I look confident? In the end, I found all these answers to these questions, and then the support that I needed through a women community that was built internally at Cadence. I had the chance to meet and listen to the stories of several women and quickly realized that these are the women that really inspired me, my female colleagues.

Dimitra Papazoglou: Through them and through their stories, I got also inspired how to get promoted to the next level, how to face my return back to work this January when I came back from maternity leave. I’m really happy to have my female colleagues and those are the ones that really have inspired me and motivated to continue and navigate my career.

Jeannette Guinn: Wonderful. Thank you so much, Dimitra. I wanted to kick off the conversation with talking about current advocacy and what each of us do to empower women and underrepresented groups and why you do it. Why is it important to you?

Jeannette Guinn: I’ll kick it off. I recently became involved with a couple programs that were important to me. In my intro, as Angie stated, I am a board member for the Child Advocates of Silicon Valley program. It’s a nonprofit organization that provides court-appointed advocates for neglected and abused children. I was a former CASA volunteer. If you don’t know what that is, either reach out to me or look it up. It’s amazing. I did that for about five years and it changed my life and it made me realize how badly I wanted to become a mother. That’s where I started off my volunteer work.

Jeannette Guinn: I currently lead the Latinx inclusion group here at Cadence. It’s an opportunity to provide education on the Latin community. I’ve learned a lot and we’re interacting and learning a lot from the other DE&I groups at the company, which is just fascinating. Also, a committee member for the women and tech organization here at Cadence.

Jeannette Guinn: Then in my spare time, I just joined my local Little League board. I have two little girls, six and eight years old, Mia and Zoe. I often call it the Mimi’s and Zozo’s show because that’s pretty much my life. They’re both avid softball players. This was the second year that the league decided to do both baseball and softball under one organization. I saw the lack of softball visibility, and the girls were definitely treated differently. Wasn’t going to sit back and watch. I joined the board and with another female board member, we elevated the softball side significantly.

Jeannette Guinn: Yes, I use my very loud voice when I coach Mia and Zoe’s green Yoda’s softball team. Yes, very involved in that organization. Why do I do all of it besides trying to go crazy? I found myself just constantly complaining about things that were happening around me, and I didn’t want to sit back and watch. I wanted to make a difference and I wanted to make a change. I also want to be an example to my girls. I’m proving that we can make an impact in this world. That’s why I do it. What about you, Rishu?

Rishu Misri: Well, yes, I think I started with saying that I do get a lot of opportunity at Cadence to volunteer for various community outreach programs. I’ve been a member of the Make A Child Smile Society. We do anything that can bring a smile to a child, organizing fundraising events to sponsor the education or painting their schools or looking after their healthcare, taking them out for health checkups, even emotional care. We could take kids out for a day trip if needed, whatever that can make them feel a little better.

Rishu Misri: I’ve also been a member of the FMA committee at Cadence, which works towards female welfare. Under this program, we partner with an NGO in India called Goonj. We sponsor and one of the initiatives which focuses on welfare. The initiative is called Not Just A Piece Of Cloth and it focuses on increasing the importance in awareness around menstrual hygiene. There’s a taboo around talk about it, so we’re trying to break that taboo. Also raise funds that can go into providing for safe supplies for women and underprivileged sections.

Rishu Misri: More recently, I’ve also been volunteering for the Cadence scholarship program. Here we interact with military students from underprivileged societies. These are kids who are very bright, very enthusiastic, clear about their vision. A lot of them want to get into STEM careers, and the Cadence scholarship helps fund their academic goals. As mentors, we try to give them support with confidence building, time management, communication skills, and sometimes just act as sounding boards because the kind of issues they face with their academic sites, they may not have anybody at home to give them the ear. We sort of just support them there.

Rishu Misri: Those are all the kind of things. Sometimes also go and volunteer outside at my personal level. That’s really all the kind of things that I’m doing. Talking about why it’s important to help empower somebody, every time I come back from these events or an interaction like this, I may want to say that I have empowered somebody, but I think what I hear is I am empowered. It brings a lot more energy back into me when I come back from an event like this. It is not just the beneficiaries’ win. It is my win as well. It strengthens me a lot. That’s why it’s important.

Jeannette Guinn: Awesome. Dimitra, what about you?

Dimitra Papazoglou: For me, some years ago I’ve been asked and I’ve been honored actually to build and lead an internal women community at Cadence. I had the great chance to travel and meet in person more than 50 women from Cadence in Europe and Middle East. I had a great chance to talk to them and listen to their stories, understand their needs, and also the challenges that they face working in this environment, in this industry.

Dimitra Papazoglou: We as community team, we wanted to listen first to women and then set the objectives and find the best ways to empower them. What we have done is a set of actions, events. I’m going to mention some of them that I think that they can be also beneficial to everyone here, for the audience. Very beneficial is the talks given by women. The woman can be from outside or inside the Cadence organization. It can be from any level, from senior level or from an early career woman.

Dimitra Papazoglou: I do believe that everyone… You can always learn from a woman, no matter the level that she is. I can tell you an example. Karna, she’s also part of the panel. She actually gave an inspiring talk to the women of our community. She talked about her story, her career, the obstacles that she faced and how she overcame these obstacles. As you see that listening to this woman, you actually get the strength and the confidence on how to navigate and achieve your career and achieve your goals.

Dimitra Papazoglou: Another thing is what we do very interesting is regular meetings where we talk about topics like leadership, work-life balance. We talk about the talents that those topics have, and we try to find solutions together. Again, we talk to each other and try to help each other through these regular meetings. Another important thing is the trainings. We have done career trainings, but also body language trainings. I totally recommend this one. It’s one of the best trainings that I have ever done.

Dimitra Papazoglou: It is all about position, the right position to stand in, how to do the best use of your voice. I think many, many people have these issues like how should I talk? How should I present? I totally recommend these kind of trainings. They definitely can help you to strengthen your confidence. Why I think the women community is very important? Because through the networking that offers you and also the set of actions and events that I mentioned some of them, you can find through a community the mentors. You can find the role models. You can find the sponsors.

Dimitra Papazoglou: You can find all the answers about how to navigate your career and how to go to the next level. It can certainly contribute on how to achieve your career goals. I think it’s one of the best way for all the women.

Jeannette Guinn: Awesome. I just have to say a side note, the fact that you’re able to complete sentences at 2:00 AM in the morning is just impressive within itself.

Dimitra Papazoglou: And having a 12 month daughter, right?

Jeannette Guinn: Huge praises to you and onto Karna, your thoughts.

Karna Nisewaner: I feel like one of the things that I get the most joy from and that really helps benefit the community is the mentoring that I do for people, both internal to Cadence and external. You don’t have to be in the same subject matter as someone to help be that person that bounces ideas off of. As Elena mentioned earlier, it’s important to go to your manager with a plan or ideas to be that person that helps people come up with those plans or ideas and helps them review things ahead of time.

Karna Nisewaner: I feel like the internal mentoring I do within Cadence, particularly during the pandemic… I think it’s been important to help people as they’re just dealing with a lot of different issues and to be that sounding board. I feel like the more I progress in my career, the more important it is for me to reach out and be there for people.

Karna Nisewaner: Now, in the past, one of the things I loved doing was traveling. I think three or four years ago for International Women’s Day, I did a talk at one of our India sites. I went to all of our India sites and did talks to the women’s groups there. I loved being able to reach out to Dimitra’s group and do a talk right before she left on maternity leave. I thought that was great.

Karna Nisewaner: For me, it’s that ability to reach out and connect with people internally and externally and help be that sounding board that helps them move forward. To me, that’s how you, as an individual, can help others. You don’t have to be more senior. You don’t have to be in the same area, but you can be that really good sounding board and person who can walk through the ideas with somebody or can brainstorm things to think about. In the greater community, one of the things that I’m passionate about is making sure that women are able to work.

Karna Nisewaner: One of the things that really makes it difficult is effective childcare and during the course of the pandemic was also having school, which is a place where many of us have our kids and that allows us to have time at home to work. I’m on the board of a childcare organization in my community that runs the afterschool program and several infant and preschool programs because if you don’t have a place for your children to go, the people that tend to stay home are the moms, not the dads. I just think it’s important that we don’t cut people out of the workforce because they don’t have the support necessary to be able to go into work.

Karna Nisewaner: Then I think it’s also important to support your local school. I’m on a school psych council and help planning to create those environments where achievement gaps are addressed in kindergarten, where you’re looking at why is one group behind in reading, behind in math and behind in writing. What can we do starting in kindergarten, first grade, second grade to really stop the achievement gap there, build the confidence of everyone there, so that by the time they hit middle school and high school, everyone’s excited to learn? Everyone has that same background and the necessary ground level education in order to be successful. That’s another place where I spend some of my time.

Jeannette Guinn: Awesome. Then I guess I want to take it to… For all of be, what advice do you have for other women based on some of your experiences, your influences? I know that a couple people mentioned the importance, and Elena talked about it too, importance of having a mentor. I agree. Being a mentor and having one, the benefits of that just are endless. Dimitra, you talked about being influenced by Karna. I can say that the same has happened for me, so thank you, Karna, for everything that you’ve done for me. Just working on confidence, how to present in front of executives, how to become politically savvy, all of that is so important to growth. Dimitra, how would you like to expand on that?

Dimitra Papazoglou: Okay. I’ll share advice not really coming from my experience, but again, from women that talk about their stories, their experience through the women community. I’ll tell you three stories and what I have got from them. The first story was about the new role. There was a new role in her team. However, this role was in a different location, very far away from her location. Her manager never thought of her as a candidate because of the location, but then what she managed to do is to persuade that she’s the best for her role. No matter of the location, she actually managed to take the role. They found, together with her manager, a solution about the location issue and she actually got the role.

Dimitra Papazoglou: The advice that I got from that is that don’t wait to be given the opportunity, just believe in yourself and go and just take the opportunity. The second story is mostly advice. I’ll talk about my experience. I thought in the beginning that in order to go to the next level and get promoted, my manager actually will see that I’m doing awesome things and she or he will offer me the role, the promotion.

Dimitra Papazoglou: But then what I got through advice actually from another women was that when you want the role, just go to your manager, make it clear about what you want. Ask what you need to do in order to get the next role and just make sure that you take all the bullets and then just go to your manager and say, “I do all of this, so I can get the role.”

Dimitra Papazoglou: On top of that, she actually told me that even when you take the role, when you take the promotion, even then, go to next day and ask what you need to do for the next promotion. That’s also good advice. The third story that I want to share is about a pay rise. She wanted to get a pay rise in the beginning. She couldn’t really get it. She thought that she should give up, but then one thing that you said about mentorship, she had a great mentor.

Dimitra Papazoglou: In Cadence, we have great mentorship programs. The mentor was very, very supportive. Also through the community and, again, listening to other stories about similar topics and negotiations, she actually decided to keep trying. She got the confidence and then in the end, she got the pay rise. I will say just keep trying and never, never, never give up. That’s all.

Jeannette Guinn: Thank you.

Dimitra Papazoglou: I want to say that this advice… I’m sharing this advice because this advice has also influenced me and also has affected how I navigate my career.

Jeannette Guinn: Yeah. Yeah. Karna, what about you, influences, experiences?

Karna Nisewaner: I think one of the most important things is really just your own internal confidence and knowing that you are the best, knowing that you are capable of doing things and knowing that even if you don’t check all those boxes, you can check all those boxes if you’re just given an opportunity to try. I think back to several of the jobs that I got, where people were like, “Oh, you only got that job because you’re a woman.” I was like, “No, I got it because I’m better than you. I have more potential than you. I’m smarter than you.

Karna Nisewaner: I think feeling that and knowing that… Yeah, we’re all absolutely capable and you just need to internalize how capable and confident you should be because you can do it. You can absolutely do it. One of the pieces of advice I give to people is really just know your worth, know how valuable you are, know how much you can really do and do that.

Karna Nisewaner: I happen to have been raised in a family by a father that just made me feel super confident. I think that’s the best thing everybody can do is work on that however it makes sense to work on it. The other thing I to talk about is really work on building relationships with others. It doesn’t have to be anyone specific, but building the relationships across an organization will really help you grow your career because you’ll hear about things that are going on that you might not otherwise hear about. You’ll be able to make connections and help other people. Then in the future, they’ll know, “Oh hey, maybe I should help Karna.”

Karna Nisewaner: The other thing I would say is ask for things that you want. I wanted different experiences. I was focused in one area and I was like, “I want more. I want something else.” I said, “Hey… to my manager… “I want something more to do.” Then they gave me something more to do, and I did a good job with it, so then they gave me even more to do. I feel like you have to ask for those things because people don’t know what you want until you tell them. They can’t read your mind. They might say no, or it might not be the right time, but at least they’ll have that in their head and you’re no worse off by sharing what you want than you would be. You’re worse off not sharing really.

Karna Nisewaner:I just feel like raising your hand to say what you want, getting yourself out there… Being competent in your capability and ability to do any job that’s out there if just given the time and support to do it is really, to me, what I think is important that everybody kind of take away from this. Then as leaders and as members of the community, how can we help other people do that? How can we be the person that listens to what somebody’s saying in this, “Okay, this is what you can do. Let’s role play. Let’s make it happen.” I feel like that’s how we can really empower others is be that amplifier of other people’s voices. When somebody does something great, remind people, but then also shout out for yourself because you’re valuable.

Jeannette Guinn: I don’t know about the rest of you, but I’m pumped. I’m like, “I’m going to take over the world right now, Karna, that was awesome. Thank you so much and, Rishu, your thoughts.

Rishu Misri: I think pretty much whatever everybody else has already said, but my two cents will be just we need to make our tribe grow. For that, whatever it takes. Depending on where we are in our life and in our career path… If you’re in entry level, you will probably have to be focusing more on building your skills, trying to build the right networks. We’ve talked about mentorship and having that confidence. Like we say, that’s the most important thing, having the belief in yourself that you can do it, being resilient.

Rishu Misri: As you grow and are in a position to even be able to support others, then be compassionate towards the other women. Being a woman and being in a workforce, it’s not going to be easy. There are going to be times when it’s going to be tougher for you than it is going to be for your male counterparts. I mean, no offense there. I know everybody’s competent, but we’re going to be taking so many additional roles and nobody can take it apart from us.

Rishu Misri: I think it’s important that as a community, we stay more connected and we stay more compassionate towards each other and support each other in whatever positions we can and I think we also need to get more focused to bringing those women back who had to apply brakes to their careers. Be compassionate towards them. If there have been a lot of women who’ve applied brakes because they wanted to take care of children or they had had elder care to take care of or whatever other personal requirements…

Rishu Misri: If anybody had a career aspiration, a dream and we can help motivate those people back into the system, the workforce, I think that’s important. Just as everybody said, having belief in yourselves and just continuing to take the risks, I think that’s very important. Being able to try out new things and having the confidence that it’s… Tough times will be there, but I’m going to overcome them with my training, with my mentor support or whatever.

Jeannette Guinn: Yep, absolutely. Thank you Rishu, and as we wrap up this panel, last words of wisdom to women that are in the tech space that are working towards advancing their career… I’ll kick it off because it’s kind of wrapping up some of the things that you’ve all said. I say this to myself, to my team, to my family members. Don’t allow a struggle or a hardship to bring you down. It’s an opportunity or use it as an opportunity to grow stronger.

Jeannette Guinn: I could have a whole other session on my history, but I was financially on my own starting at the age of 17, and suffered years of abuse until I was about 23 years old. It sucked and you take each and every moment as learning opportunities and you make the best out of those crappy situations. Anything that I had to deal with in my 20s, as I was trying to advance my career, there were little nuggets of learning lessons.

Jeannette Guinn: If you want something, you go after it. Take that chance. There are going to be risks involved. There are going to be failures and that’s okay. You just don’t look back. You just keep looking forward. There’s a phrase that I use a lot. I say it a lot, but I was in a 12-month program with Women Unlimited, fabulous program. They taught me that you strive for excellence, not perfection because perfection’s just not possible. every day I just do my best and you strive for excellence. that’s my last words of wisdom. Rishu, any last words of wisdom from you.

Rishu Misri: I think I just continue build on what I said in my previous… I think it’s important that we continue to be resilient. That’s what is important. Just stay there, hang in, and if needed, seek support. There will be a lot of we people willing to help you. A lot of times, we may feel, “Am I doing the right thing being here? Is this where I should be? Maybe I should quit. Maybe this is not for me. Maybe… There’s so many questions that be come in to our mind. It’s not just for you. It’s for everybody.

Rishu Misri: Seek support. If you need to apply the brakes, do that. I’ve done that as well. When I had my daughter, I applied the brakes. Then when I had my son, I sought support. That’s ways I was able to continue doing what I wanted to do. I think that’s the other most important piece of advice that I have. That is whatever you choose to do in that moment. Do not be guilty about your choices.

Jeannette Guinn: Yes, yes, absolutely.

Rishu Misri: It was your decision. Don’t be guilty for whatever the choice you made. That’s important. Be resilient, seek support, don’t be guilty. That’s important. I think that’s all that I would say. Thank you.

Jeannette Guinn: Great. Thank you. Dimitra?

Dimitra Papazoglou: Yeah. For me, I’d like to actually say three things. For me, always have a career plan for the next two to five years and make it clear to your manager. Second thing, find the ways to strengthen your confidence. It can be this conference, it can be this panel. Find the Karna that will help you to have the confidence and say, “Okay, I’ll go for it. Karna said that. I’ll get all this confidence and I’ll go for it and I’ll take it.” The third is seek for opportunities. Don’t wait for them, okay? Don’t wait for others to give you the opportunities. You need to seek for them.

Jeannette Guinn: Thank you so much, Dimitra. And Karna?

Karna Nisewaner: I’ll build on what Dimitra said. It’s not just seeking opportunities. It’s being okay with change, being okay with saying, “This isn’t working out for me. I need to find a different environment, a different set of colleagues,” and having that community, having the people to support you.

Karna Nisewaner: I feel like you need to also be open to new things and maybe it’s a change in your role at a company. Maybe it’s a change of companies, but being flexible with yourself and not feeling like you’re stuck or stagnated into one thing, but that you can really do anything because I do believe that there are so many possible options for everyone. We just need to try and we just need to experience them. Sometimes things will be great. Sometimes they won’t be great. What can you change to make it better? Because you control your environment.

Karna Nisewaner: Yes, there are certain things we need. We need our paychecks, but you do control a lot of your environment and you need to create and find that environment that’s supportive, that’s there for you and that wants you to be successful. I feel like that’s what I found at Cadence is an environment where managers, colleagues, other people I worked with, they wanted me to be successful and they wanted to help me find that next thing.

Karna Nisewaner: You don’t find that in all jobs. If you’re not finding that, find people that will help you. Find a new role. Find others that will really amplify the value that you’re adding and really appreciate the way in which you add that value. I feel like we control our future, but we need to be out there saying what we want, sharing what we can do for others.

Karna Nisewaner: We can all have great careers. I just love how many more women are engaged and how many more of the underrepresented minorities are engaged in the community here at Cadence, are engaged in the Bay Area and are engaged worldwide. It’s great to see that growth. I just really hope it continues and that we continue to really show everyone that we are amazing. We are the best. We’ll rule the world, right?

Jeannette Guinn: Yeah, absolutely. Absolutely. I love it, Karna. Thank you so much, Karna, Dimitra, Rishu. It’s been a pleasure. On behalf of Cadence, thank you all. I hope this was helpful. Angie and Girl Geek, thank you for this opportunity. It was a wonderful experience. With that, go onto networking. Thank you so much.

Karna Nisewaner: Thank you.

Rishu Misri: Thank you.

Dimitra Papazoglou: Thank you.

Angie Chang: Thank you for being a part of that panel. I feel very empowered and ready to dig in. Now, I want to just really quickly plug that Cadence is hiring. They’re hiring for engineering jobs in cities like San Jose, California, Cary, North Carolina, and Austin, Texas. Now, we’re going to move onto our Girl Geek X networking hour. There’s a link that will be put into the chat. If you click on that, it’ll go to Zoom meeting, and we’ll see in a Zoom breakout room very soon.

cadence girl geek x speakers zooms
Cadence Girl Geek Dinner on March 16, 2022.

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

“Economic Justice and Cryptocurrencies / Web3”: Jen-Mei Wu, Community Organizer at PaRTEE4Justice (Video + Transcript)

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Angie Chang: So our final session is Jen-Mei Wu, who will be talking about cryptocurrency and how it is a lever for inclusion and economic justice. Welcome, Jen-Mei.

Jen-Mei Wu: Hi everyone. Welcome to the talk about economic justice and cryptocurrency. My name is Jen-Mei, and I will tell you a little bit about myself. I am based in Huchiun, an unceded Lisjan territory, aka Oakland, California.

Jen-Mei Wu: I’m an activist, an engineer, an artist, and one of the founders of LOL Maker Space in Oakland. Also, Power and Resilience Through Experiential Education, our PaRTEE4Justice, and I have a bunch of experience working both in non-profit tech and also in for-profit techs as well. And I have a particular interest in digital privacy and security. And part of that is what brought me into this space. So this is just an overview of what I’ll be over today.

Jen-Mei Wu: We’ll be talking about an opportunity to address inequity in financial systems and going over some examples of projects that are making a difference in this space, and how small teams can have really big impacts and that there’s a role for everyone. So including you, if you want to come and join this stuff.

Jen-Mei Wu: I’m going to talk about the importance of self-determination and also some collaborations that I’m doing with Black Voices and Gaming and Sistah Scifi, as well as the future, which is the decentralized web or Web3, as you may have heard. And also about how to get started in this space.

Jen-Mei Wu: So this slide here is showing the market cap of cryptocurrencies altogether. As you can see, just in the last two years, we’ve had more than a trillion dollars flood into the space. A trillion. That’s a trillion with a T. That’s a lot of money. And actually, at some point last year actually, it was two trillion dollars, and I’m guessing it will get there again in the future. And so the question is, so all this wealth is being traded, but who’s benefiting? Well unfortunately, right now, cryptocurrency isn’t the most diverse space. It’s not the most inclusive space, and this talk is about how that can change.

Jen-Mei Wu: So moving on, before I get into the opportunity though, I just want to share a quick note doubts. So when I talk to people about cryptocurrencies some of them are really excited. But others are like, “well, but what about the dark web?” Or isn’t it bad for the environments? Or are there like a lot of tech bros running around doing tech bro things? And most of all, “isn’t it all made up? Is just fictional? Is there any value there?” So I’ve done my own research, encourage you to do your own research. I’m not particularly concerned about these. And in some cases I see some of these challenges, which are real and are legit, as being opportunities for us to make some really big and important changes.

Jen-Mei Wu: I just want to share what I found to be an inspiring experience that had to do with another disruptive technology, and that would be 3D printing. Some friends a few years ago, I think it was like eight, nine years ago, were really concerned that 3D printing is a technology that could take jobs away from workers. It’d be an opportunity for large corporations to create 3D printers and the materials that go into those 3D printers, and have people make stuff at home. Therefore, centralizing control and hanging out to the profits, which is ironic because 3D printing was meant to be decentralized and meant to be hyper local. And that’s kind of attention that you see and cryptocurrencies.

Jen-Mei Wu: But the interesting thing about this story is that a few days later, I went to a talk by Grace Lee Boggs, who’s very amazing. I encourage you to look her up if you don’t know who she is. And it wasn’t a talk about 3D printing, but she mentioned 3D printing. And what she mentioned was that she, and she’s 99 years old at this time, this is about a year before she passed away, and she said that she saw 3D printing as being really exciting because it could be a way to end capitalism. So she saw how 3D printers could be used to create other 3D printers and how a lot of initiatives are helping people create materials to feed into their 3D printers locally without having to rely on filament companies. And so, she was able to think big, and that’s what I would like us all to do is to think big. Think about how challenges can be opportunities. And in that spirit, I’m going to talk about what some of those opportunities are.

Jen-Mei Wu: We talk about some projects making a difference, just to give you a sense of the space. This is a screenshot from the Movement for Black Lives webpage. They take cryptocurrency donations, and they do that through this project here called The Giving Block. The Giving Block allows organizations to accept crypto and individuals to donate crypto. And that’s great. Endaoment is another website that’s similar. Except with Endaoment, they don’t already have to have signed up to accept crypto. You can give to Endaoment, and it’ll create a donor-advised fund for you. I’ll let you do some research on your own about what a donor-advised fund is. But the idea is that you can donate to Endaoment, a nonprofit organization, and then later have those funds go over as regular cash to the organizations that you want to support. For example, the Sogorea Te’ Land Trust. Sorry, I have to adjust my image. That’s for straightforward, right? That’s like, “Hey, you can give cryptocurrencies away.”

Jen-Mei Wu: Where it really gets interesting is when you can start doing some things that are pretty unique to decentralized finance, which is built on top of the blockchain. So this is an example of Angel Protocol. And their idea is that instead of donating directly to a nonprofit. You donate to a nonprofit’s endowment and Angel Protocol will reinvest those funds, generate a yield, and that yield can go to fund the nonprofit’s operations, potentially funding that nonprofit for forever. And the way that they’re able to do that is Angel Protocol is on the Terra Blockchain, which is a low energy blockchain FYI, and another protocol on that blockchain is Anchor protocol.

Jen-Mei Wu: And Anchor protocol is a savings account where people can take their U.S. dollar equivalents in UST. It doesn’t go up or down like some volatile cryptocurrencies does, it’s always equal to a dollar, and you can get 20% interest on them. Which is pretty amazing when you compare that to the savings accounts that banks typically offer. And that’s just an example of how one protocol, Angel, is able to leverage another protocol, Anchor, in this case.

Jen-Mei Wu: And another example of that is… I’m going to have to do this again. Is carbon credits. So Moss and Toucan, are two projects that have put carbon credits on the blockchain. Carbon credits are created by things like forestry initiatives, things that are good for the earth, and are purchased by polluters like airlines or manufacturing companies to offset the emissions that they do so that they can claim to be carbon neutral. And that’s interesting, just being able to buy your own carbon credit. So you want to be carbon neutral yourself, just like the airlines, so you can buy your carbon credits on the blockchain real easily. That’s great.

Jen-Mei Wu: But it also created at a building block for KlimaDAO to create a black hole for carbon. Using game theory, KlimaDAO has encouraged people to invest in locking up carbon so it is off the market. And the idea that if it’s off the market, then it becomes harder for companies to get carbon credits. It’s more expensive, which will either generate more money for the forestry initiatives and others, or maybe it’ll encourage companies to think about other ways to reduce emissions, such as by reducing emissions.

Jen-Mei Wu: And I also just want to mention that some of these projects that I mentioned are maintained by small teams. There’s many projects, in fact, that are maintained by small teams. So I’m used to, as many of you might also be used to, being in the tech space… Especially here in the Bay Area, there’s a lot of venture funding and a lot of startups are actually pretty big and require a lot of funding in order to make their work possible. But because projects can be small teams, that means that they have smaller budgets. A smaller number of people can do really interesting things. And protocols can build on top of each other so they can have a very narrow focus. And that also makes things a little easier and helps facilitate the small teams.

Jen-Mei Wu: And there’s a lot of different project types. Sure, there are big projects that have hundreds and hundreds of numbers, and thousands of numbers, maybe even. But there’s many project types that have maybe even one member. There are some artists who have created their own NFT projects. NFTs are non-fungible tokens. They’re basically a way to track art on the blockchain. So you always know the provenance of your art, or whatever, and where everyone’s unique and cannot be duplicated. But there’s individual artists who started these projects and don’t even have other people helping them out. That’s not everybody, but it has happened.

Jen-Mei Wu: And that brings me to my next point, which is just there’s a role for everyone. There are engineers and designers, of course. There’s some very lovely web apps that I showed you that engineers and designers did a big part in, but also game theory, as with KlimaDAO. There is a role for artists in NFT projects and other things as well. But also for community organizers, community is the lifeblood of many of the projects that exist. And people who facilitate community connections, who run events, who help connect other people together. They’re very important, and they come from a variety of backgrounds. Likewise, the space really needs a lot of educators because there’s a lot of unknowns in this space. And it’s important for people to have easy on-ramps. To be able to learn how to participate and about the different projects that are always coming out. There’s always something new as well as analysts and more people. So these are just some examples.

Jen-Mei Wu: And one of the things that I’m really excited about is trying to get away from this slide, which is funding with compromises. Which I feel is what a lot of us are used to. If you work for a VC funded tech startup, you’ll probably have noticed that VCs have the big influence on how tech companies survive. VCs are not after making companies successful. They may disagree with me, but I would say that they’re into helping companies gamble so that their successes are ginormous, but some of them are just going to explode. And in so doing, they pump funds into companies that allow companies to hire above market and at a very high pace. Bringing people from, for example, out of the area, into the area. This has fueled displacement and gentrification that we have all noticed, at least those of us in the Bay Area and other places where this phenomenon has happened. Foundations, similarly, have an influence on nonprofits. Many nonprofits get a significant amount of their funding from one or two or three foundations. And sometimes find that their priorities must be affected by what the foundations want.

Jen-Mei Wu: I see crypto as a way of moving towards funding without compromises, or at least with fewer compromises. Of course, you can do grassroots fund fundraising. You could have people contribute, for example, to an Angel protocol endowment and operate off of that. Or you could create an income generating project, some of which do not require years of development or even months of development, or even weeks of development. You can build projects, but there’s many other opportunities as well, like validation, NFT projects, content creation.

Jen-Mei Wu: And if we can fund our projects without compromises, then we can work towards self-determination. And just another example, like DEI efforts. I have been involved in DEI efforts at tech companies, many of you may have as well, and they’re great. They get people a seat at the table. And they get people in income and that cannot be understated. That’s really important. But having a seat at the table isn’t as good as owning the table, and it’s not as good as being able to design or build something, which might or might not be a table. Right now, if you feed into the existing infrastructure, then you’re playing by their rules, but self-determination means making our own rules.

Jen-Mei Wu: And with that, I’m going to mention a couple of collaborations that I am excited about. Alfonso is one of the folks behind Black Voices in Gaming. He’s also one of the other founders of Liberating Ourselves Locally, the maker space I mentioned earlier. He has this program where he teaches black and brown youth in Oakland how to do DevOps, which is pretty amazing, and we are going to be looking at building on that curriculum, and combining some crypto activities with that as well, to create a program that generates income, for example, by validating blocks on the blockchain.

Jen-Mei Wu: And the idea is that when students come in, and we’ll obviously have to do some investment upfront, and create some of that infrastructure. But when they come in, they can learn while also potentially getting paid to learn. The idea is that… Coding schools, great, but also not everyone can afford $20,000 or whatever in tuition and to take months off work. But if you can get paid to learn, that really changes the game. Now the work that they would be doing would be something that they can do on their own as well. And so they don’t have to join a VC funded startup in order to succeed. They could just keep doing this stuff, self-determination.

Jen-Mei Wu: So the second collaboration is with Isis, Sistah Scifi. She sent me a little video that I’m going to play for you so you can learn about her.

Isis Asare: Hi, I’m Isis Asare. And I just wanted to take a second to introduce you to Sistah Scifi, your new favorite book boutique. We focused on science fiction written by black women. In addition to selling books, we also host a lot of online and in-person events, book clubs, watch parties, you name it. It’s a black, queer own company. It’s owned by myself.

Jen-Mei Wu: What we’re doing is we’re starting an NFT project, which is an opportunity for us to diversify the NFT space. To tell our stories, to fund our initiatives, and increase representation and create opportunities for artists who are not currently in this space. NFTs are like other parts of crypto, not extremely diverse.

Jen-Mei Wu: That brings me to the future. So right now there’s a lot of stuff. A lot of promises in crypto that are not quite realized yet. Like the decentralized web or web3, it’s definitely not here yet. A lot of the websites I showed you are traditional websites. They run on the same hosting that you’re probably used to. If that hosting goes down, the website goes down, and unless people know how to interact with smart contracts on the blockchain directly, it’s definitely not decentralized.

Jen-Mei Wu: But in the future, hopefully there will be a future where corporations don’t control the infrastructure. Where the infrastructure will be distributed and decentralized across the world, and it won’t go down when a single computer goes down. It’s still being defined, and now is a good time to be part of that definition.

Jen-Mei Wu: So here’s some information about how to get started. Social media is a great place to learn about this stuff, but I would consider being anonymous because there are scammers out there. In any extremely non-diverse space, there’s some potentially unpleasant interactions that can happen. I, myself, interact with crypto communities using anonymous accounts, not the ones I’m about to show you. And also suggest some healthy skepticism. I think there’s a lot of opportunity here, but I think it should be balanced with some skepticism as well. And don’t be too quick to trust folks because people will offer to do tech support for you or show you all sorts of things, but they do not have your best interest in mind.

Jen-Mei Wu: I would suggest joining communities that are supportive. And there’s new communities starting all the time. Some of these are communities for women, non-binary folks, LGBTQ+, people of color. And also, we will be having a future workshop at, and you can check out our website to learn more about that.

Jen-Mei Wu: That brings me to the end of the presentation. I hope you go and build stuff. I hope you keep in touch. These are my socials. I’m not really the best at them. Feel free to message me more than once if I happen to miss your message. And a link to the website

Jen-Mei Wu: And that brings me to the end of my presentation. I do see there’s a question, some questions, but I don’t think I have time to answer them. So I’m going to suggest that you follow up with me, and I can answer those questions, and I’ll hand it back to Angie.

Angie Chang: Thank you, Jen-Mei, for that talk on crypto and web3, and definitely see some possibilities. I like your argument, and I am going to look at all those projects that you’ve mentioned later and see how I can get involved. Thank you for sharing.

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

“Why Knowledge is the Future of Data”: Michelle Yi, Senior Director of Applied Artificial Intelligence at RelationalAI (Video + Transcript)

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Angie Chang: So our next session is Michelle Yi from RelationalAI. She is a Senior Director of Applied AI, and she’ll be speaking about harnessing knowledge and data and show us relational knowledge graphs in action.

Michelle Yi: Great. Thank you so much, Angie for the introduction. I’ll just go ahead and screen share, make sure everything is going smoothly. And let me make this big. Here we go. Okay. I think we are good to go. Okay. Yes. So happy International Women’s Day, everyone. And I, plus one, I saw in the chat, a comment about Reggie for president. So plus one to that.

Michelle Yi: So my name is Michelle Yi as Angie said, I’m super excited to share a little bit of perspective on why I believe knowledge is the future of data and how my personal experiences in the data space also align to this common vision that brought me to RelationalAI.

Michelle Yi: And so I thought I could start with sharing a little bit of context and background on myself and the journey that has brought me to RelationalAI. Our vision for what we’re doing really spoke to a lot of the challenges and problems that I saw in the machine learning and data space.

Michelle Yi: And actually to make this a little more fun and interactive, if you guys want to share a little bit about your own journey technology. I’d be curious to see what did you all study, whether it’s undergrad, PhD, masters.

Michelle Yi: What was your last educational focus? Something Heather said in the last talk actually really ties to the demo I have at the end of my talk, which is going to show a little bit of the backgrounds of the women that signed up for this conference.

Michelle Yi: So for me personally, I spent the last 16 years or so in the AI/ML space working with data from both our products R and D side, as well as a consulting perspective. So specifically, I don’t know if anyone will remember this, but in 2012, one of my first projects that I worked on actually aired as IBM Watson and this whole thing with Jeopardy where the computer was playing against the humans.

Michelle Yi: So that’s my one claim to fame. And then after that, I moved more into management consulting because I really wanted to understand the data and data science problems that many customers across many verticals were facing. And so through these experiences over the last couple of decades, I really got a lot of exposure to the impacts of the constantly shifting technical paradigms and how that impacted business.

Michelle Yi: So to give you an example, when I started at IBM ML 16 years ago, was on mainframe. This was before the Cloud. If you can even imagine an era before the Cloud. And then we were after we started getting migrated and pushed to go more Cloud oriented, moving away from on-prem, there was a big, no pun intended, big data movement.

Michelle Yi: Essentially saying, like, “Go collect all the things.” And we didn’t… We collected a lot of data without really always thinking about why did we need that data? And then we were sort of pushed to like, “Okay, well, if you want to use this big data thing and you want to make all those things that you collected useful, you need to go to MapR and Hadoop.”

Michelle Yi: And then what ultimately resulted was this data swamp architecture where we had data everywhere in different silos of many different types. And then that shifted into what’s now more of the modern Cloud data warehousing. So think about BigQuery, Snowflake, Redshift et cetera. And then after we consolidated all of these things, we’re like, “Oh, okay. We finally got it figured out.”

Michelle Yi: But then you kind of see another kind of paradigm around machine learning and for people to take advantage of that you need yet another patchwork tool chain. And we’re going to dig into this a bit more, but the question is why is it that every time we see a paradigm shift, or a new technology, or a new data structure that we kind of go through the same motions over and over again.

Michelle Yi: And so just to speak to a little bit of those problems, I don’t think this is going to be new to anyone in the data space, but basically with each iteration that we’ve gone through, we still see the same needs from the business and the technology side.

Michelle Yi: There’s this desire for kind of more data driven decision making across the board from your executive teams all the way down to the engineering teams. And then there’s this other problem of like, “All right, we went through big data and we collected all the things, but now we don’t really understand everything that we’ve collected.”

Michelle Yi: So we even today, I think many of us would agree that there’s really kind of a lack of understanding of the full extent of the data assets that an enterprise or even a startup has.

Michelle Yi: And then as a result of that, there’s this third bucket of problems where we’ve really seen a rise of just too many point solutions or too many point data applications that sometimes can be repetitive of each other.

Michelle Yi: I don’t know how many times I’ve [inaudible] this and seen to a customer and we’re like, “Hey, you’re interested in a fraud detection, no problem. Oh, by the way, they also built their own fraud detection solution over there in teams D or E.” And so we’re kind of seeing like this common theme across companies and across a long period of time. And again, we need to ask ourselves what’s the root cause of this.

Michelle Yi: And ultimately I think what I saw over and over again is that there’s really something missing from this modern data stack. If we’re really evolving the way that we think about data, why are we seeing the same problems manifest over and over again? And so this is the question I really want us to kind of hone in on and specifically around this concept of knowledge and I’m going to share because you’re like, “All right, knowledge.” That can mean so many different things to basically everybody on this call.

Michelle Yi: And I’d be curious how many data scientists, more on the ML side we have in the room today versus more of the software engineering data app side, I’ve lived in both sides of those worlds. And they’re converging in many ways, right?

Michelle Yi: Because a lot of intelligent data applications today at the core of them, they really are having embedded machine learning whether that’s a machine learning model that you and your teams build or managed service that you receive from a vendor that you buy.

Michelle Yi: And so from my personal experience, I wanted to share an example of a day in the life of a data scientist or a software engineer working on an intelligent application and really hone in on this question using a workflow example of like what happens to the knowledge. And tell me in the chat, let me see.

Michelle Yi: I want to make sure I have it topped up in the screen, but please tell me in the chat if you resonate with this, but one common thing that I think people really have experienced is that we tend to spend like 80% as a data scientist or someone building an intelligent app.

Michelle Yi: We spend like 80% of our time productionalizing things and maybe 20% of our time really modeling, collecting the requirements and the data, et cetera.

Michelle Yi: And if I go into this just like one more level deep and not to get too trapped in the weeds, but just to really hone in on the pain point and why knowledge and embedding knowledge in a workflow is so important is let’s say like all of us are on the same team together.

Michelle Yi: And we want to build this fraud detection application. And at the heart of this application is a machine learning model that gives some predictive score of like, “Yes, that transaction is 50%, 60% likely to be fraudulent.”

Michelle Yi: Well, let’s think about this. So step one, what do we really go do? We let’s say one, we get a sense of our own intuition of what kind of data we need. We probably need something about transactions.

Michelle Yi: And we probably need something about accounts and people related to these transactions and maybe that lives in, I don’t know, BigQuery, let’s say it lives in Teradata, and then it lives in Excel because how many of us store data… Plenty of us store data at Excel. And then let’s also say that we probably need some information from the public web because when people steal things, they need to go sell them and make money.

Michelle Yi: So we get this intuition, we make a list. And then we ultimately, what we end up doing is we go to the business owners or the business experts and saying, “Okay, does it make sense to have this kind of data? What are we missing? Oh, I see, this data has this flag that has a transaction type one. What does that actually mean?”

Michelle Yi: And so we spend a lot of time upfront collecting and gathering data. We work on a subset and that in this 20% bucket of data science work, in that 20% of time, we get a model working that we’re pretty happy with.

Michelle Yi: Let’s say we use Python and a Jupyter Notebook. steps one and two are done. We’re happy. And then we need to scale this up to production. And then what we end up spending 80% of our time on is rewriting everything that we learned in terms of collecting the knowledge from different business stakeholders and our own data science knowledge.

Michelle Yi: And we rewrite that in like Java, Spark and much more heavier imperative programming languages, just so we can productionalize what we already did in steps one and two.

Michelle Yi: So the question is why can we not preserve knowledge across the data, across this entire workflow end to end. And that’s where I really kind of started to think more about this problem, because imagine how many like teams, how much time it would save if I could just preserve all of my learnings that I collected up front from the business about the relationships between transactions and customers, and accounts, and then also like the different constraints.

Michelle Yi: So for example, if I am looking for pictures of cats, I know that cats have two ears. I shouldn’t even think or waste any time processing things with four ears or five ears. I mean, this is a toy example, but I think you get the idea. And then 0.3 is really like, “Okay. If I on team A, I’m building this fraud detection app, why can’t I just easily share this knowledge with somebody in team D so that they don’t have to go do the same requirements gathering?” Because you know, that happens in any organization. And so when we talk about knowledge, it’s how do we preserve these relationships and really save ourselves time and.

Michelle Yi: We preserve these relationships and really save ourselves time and make that accessible to more than just one team. So, there is this concept of a knowledge graph and so you’re like, “Okay, well, yeah. I’ve heard about knowledge graphs.”

Michelle Yi: And there’s sort of like this way of structuring and thinking about data that can somewhat solve this issue, but not exactly and let’s… I want to get into that a little bit really quickly.

Michelle Yi: And so, one of the things is that, here is just an example of a knowledge graph concept, right? And the thing about this picture is even if I don’t give you all the details of like, oh, this lives in inquiry [inaudible], this one lives in another database.

Michelle Yi: Conceptually, you can kind of get that a product has a brand and a product has a category where shoes is an example of a category and a company sells products. It doesn’t matter if you’re an engineer or a business person, you can pretty quickly see what this is.

Michelle Yi: And now imagine if you could actually just query your data as easily as you can read this picture. The thing with knowledge graphs though is that they’re actually not necessarily a new concept.

Michelle Yi: So, it was coined by Google when they created the Google knowledge graph. They wrote this paper that came out in 2012, over 10 years ago now, and it’s been a core competitive advantage to them.

Michelle Yi: So if you ever wonder why search is so powerful at Google, this is one of the secret sauces to that. And when you’re shopping on Amazon, if you’re like, “Wow, my recommendations are amazing.”

Michelle Yi: That’s also another reason why they’re so powerful, is that they’re using this thing called knowledge graphs. And so a lot of other companies have really adopted this thing called the knowledge graph. And you’re like, okay, you can do all these cool things. You can express your business knowledge in the same place as you would do your programming or your data querying, why isn’t everyone else adopting this?

Michelle Yi: Well, the problem is that, and there’s many, many problems, but there’s kind of like three that all high level boil it down to. But one of them is that yes, knowledge graph expertise is kind of rare and not everyone is Google or Facebook or LinkedIn, and they can’t hire hundreds of engineers to go build these things for them, right? There’s not enough people out there to do this.

Michelle Yi: And the second thing is that building and scaling knowledge graphs is really difficult because a lot of the existing solutions are built on really old paradigms. So like the Google knowledge graph paper came out 10 years ago, a lot of the commercially available systems today make it hard to use.

Michelle Yi: Some of these systems are based on theories that came out in the seventies in terms of navigational systems, right? And so it’s really, really hard to use any existing thing to build your own knowledge graph if that’s really what you want to do. And so similarly, operating and maintaining them is really challenging as well.

Michelle Yi: So it’s an amazing concept that just really hasn’t been more commercially viable and accessible to a broader audience. And so, there’s one thing that I want to quickly over is we’re kind of taking a slightly different take and then I’ll show a really fun example to make this more real and in honor of international women’s day here.

Michelle Yi: But one of the things that we’re trying to do is say, let’s build that next generation thing. What does that really look like if we were to take a knowledge graph and make that supercharged and really available to a broader audience.

Michelle Yi: And one of the things that’s key is you see the word knowledge graphs, and then you see this thing called relational and RelationalAI. So I’ll share a bit more before jumping into the demo quickly and then wrapping up.

Michelle Yi: But essentially when it comes down to what we’re trying to do is build this next generation database platform that really gives you that infrastructure layer that’s going to help you consolidate and keep knowledge in the end to end workflow based on a solid shared foundation of a relational knowledge graph.

Michelle Yi: So one of the things that being a relational knowledge graph does is, and this is a bit of an eye chart, but I’ll summarize it in one point, which is that the relational paradigm, when you think about why SQL databases, for example, or you think about why snowflake or BigQuery or Redshift are so popular today is because it separates a lot of the what from the how. So you don’t worry about this huge list of super technical things in the middle, right?

Michelle Yi: A lot of that is actually handled for you. And so that’s something that’s really, really cool about a relational knowledge graph versus other systems. Because again, we share those same technical foundations of what you really expect from that modern data stack and including things like warehousing, et cetera. And so when you think about your favorite SQL system or your favorite database system, I guarantee a large part of that adoption is because your business users, not just your engineering teams, can use it.

Michelle Yi: And so in the future what we’d love to see is like, because we share these same fundamental architectural paradigms, we’d love to see that layer of knowledge that sits across and really pairs with and augments the work that many organizations have already done to consolidate and clean up their warehouse. Basically all the work that everyone’s done going from Hudu to cloud data warehousing, et cetera. This is the thing that we want to say is missing from that modern data stack and that we want to augment and really bring out the power of these things across your organization.

Michelle Yi: All right. So with that said, I’m going to take a look at the chat here and just see at some of the backgrounds. Okay. I love it. Business management, psych. All right. So, in the last three minutes or so I want to wrap up again with just like a simple example where we took some data, thank you to Girl Geek X for providing some of this as well.

Michelle Yi: But basically we took some data on the types of folks we knew would be presenting and attending the conference today. And then we also took some information that’s already… So, for those of you that don’t know about DIFA, we took some information from them. They actually structure all of the information on the public internet in a knowledge graph. And so it’s super easy for us to be able to leverage that in our system. And we took a high level view of kind of the women participating.

Michelle Yi: And basically what you’re seeing here is we put a visualization of what’s called the weakly-connected components graph, right? And so it’s a type of graph algorithm where what you can see quickly is like there’s certain densities and there’s certain areas that are less connected on the edge here.

Michelle Yi: And so we took a survey of sort of what did people study, right? And for women that are in engineering or technology, what did they study as the most recent education? And so what I thought was really fun about this is that when you zoom in, you can kind of see the clusters you might expect.

Michelle Yi: This is a New York if I remember right. And then in New York, there’s lots of people with computer science degrees, et cetera, et cetera. But when you get to the edges a little bit further out, you see a lot of really, really cool majors and folks of women that are in our fields and that have really, really diverse backgrounds.

Michelle Yi: And I love seeing this. So you see like economics, I saw English, English literature. I saw health informatics right here, design and art direction. And so I thought this was like a really fun way using knowledge graphs to quickly show that it doesn’t matter what background you have, but there is a place for you in tech.

Michelle Yi: And the thing is that when you are kind of one of these weakly-connected components, you might sometimes feel like you’re the only one. Right? But actually it’s not true. There’s so many of us that are out here.

Michelle Yi: And so I thought this was a fun way to show that using some real data. So yeah, I thank you so much for all of your time. I think we’re right at the 45 minute mark. And so, really appreciate it. And if you have any questions or you’re interested in graphs or the tech, please don’t hesitate to reach out. Thanks so much.

Angie Chang: Thank you, Michelle. That was very informative. I love the chart and the graph and for explaining everything so clearly. 

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

“Moving Up: How To Fast Forward Your Career by Shifting Out of Auto-Pilot and Rising to the Top”: Raji Subramanian, VP of Engineering at Opendoor, and Heather Natour, Head of Engineering, Seller and Consumer Growth at Opendoor (Video + Transcript)

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Sukrutha Bhadouria: So up next, we have Raji. This is a great segue actually into our next session with Raji who is the VP of Engineering at Opendoor. She’s joined by Heather, Head of Engineering for Seller and Consumer Growth at Opendoor. Welcome to both of you, Raji and Heather.

Raji Subramanian: Thank you.

Heather Natour: Thank you. So first I’d like to introduce myself. I’m Heather, I’m Head of Engineering at Opendoor responsible for the core product experience for home sellers, along with growth initiatives and retail partnerships. And I am very excited to sit down with my colleague Raji Subramanian, Opendoor’s VP of engineering. So Raji do you want to share your role at Opendoor and experience in tech?

Raji Subramanian: Yes. Thank you, Heather. I’m very, very excited to be here. So again, as Heather mentioned, I’m a VP of engineering at Opendoor where I’m responsible for Opendoor’s and to end real estate transaction and operational platform build out.

Raji Subramanian: The goal of this platform is to enable Opendoor to create remarkable consumer experiences and enable the company to scale. Prior to Opendoor I co-founded a company called which was acquired by Opendoor late last year.

Raji Subramanian: And at Pro, I was the COO and the head of product and technology as well before starting Pro I had a long stint at Amazon where I was a pioneering member, as well as led a lot of teams within Opendoor… Within Amazon’s marketplace, as well as AWS and also led teams within Amazon’s Kindle organization, as well as Yahoo Finance.

Raji Subramanian: I care deeply about diversity. I also care deeply about ESG. Both of them are very inter related. I’m a board member and advisor to BoardReady a not for profit. that’s improving the board diversity in public and private company boards.

Heather Natour: It’s an impressive background and really excited to dive in. Before we do that, I wanted to provide all of you just a quick overview of Opendoor. So our mission is to take the complex traditional home buying and selling transaction and make it simple and on demand. And we’ve completed more than 150,000 customer transactions. We operate in 45 markets nationwide, and we continue to scale the company and the team rapidly.

Heather Natour: So as I mentioned, we have a great talk planned. Raji is going to share insights, advice, and experience on setting goals to advance a career in tech.

Heather Natour: Deconstructing, common career roadblocks and delivering measurable impact that helped her get to where she is today. And the goal of the discussion is to help you make the most of your career and serve your organization well by setting meaningful and measurable goals.

Heather Natour: So Raji you co-founded a successful technology platform and helped it grow to one of the nation’s largest general contractors. You were the pioneering member of teams that helped develop Amazon’s online marketplace and AWS.

Heather Natour: How did you set personal and professional goals that empowered you to make the most of your career while impacting the tech industry?

Raji Subramanian: That’s a really good question, Heather. And again, I use a sort of like a self-developed framework, which I call three P framework. The three Ps stands for passion, purpose, and people.

Raji Subramanian: Passion is all about what I like doing, it’s what you really enjoy doing. And you can do day in and day out with the same level of intensity that you started doing it when you started the journey. For me, that’s building businesses at scale, as well as engineering.

Raji Subramanian: The second P is purpose, just about taking your passion and applying it to something that you really care about. In my case, I care about creating transformative customer experiences. An example of it is about how homes impact the lives of so many people. And that’s why I started Pro and that’s what brought me to Opendoor as well.

Raji Subramanian: It’s a purpose that I deeply care about and the last of course is the people. It’s just about loving the work that you do and doing it with people you enjoy working with who are very smart and help you grow.

Raji Subramanian: With that said, with that framework, I actually apply or do a goal setting exercise which is four parts to it. The first thing that I do is I ask myself what’s the value I’m creating. It’s anchored along value creation. Are my goals enabling me personally, as well as where I am working, create value for its customers, employees, and stakeholders, and the business itself. Value creation is one of the most fundamental things from a goal setting perspective. And that’s kind of what, when you create value, you move forward in your career.

Raji Subramanian: The second aspect of goal setting that I look at is a scale of impact. Again, things that you can do can have a small impact and can have large game changing impact. As your revolving in your career it’s important to slide through the scale of moving and transitioning your goal setting from small impact to larger and larger impact. It does not mean you don’t go about making incremental changes, but there is a moving forward in terms of the scale of impact.

Raji Subramanian: The third thing that I look for from a goal setting perspective is, are the goals that I’m setting, helping me grow in a multidimensional way, from a leadership perspective. It’s about bringing in that intersect between engineering, what the customer cares about. It’s about the product, it’s about the P and L of the company.

Raji Subramanian: And being able to operate in that space are the goals allowing you to operate in a multidimensional way is a third thing, or the third prong of goal setting I look for.

Raji Subramanian: And the fourth is very important and close to my heart. It’s about, be purposeful. Diversity is something that I care about very deeply. That’s kind of what took me to BoardReady. Again in the teams that I built, whether at Opendoor or outside, I look for diversity. I seek diversity. I’m an advocate for diversity. And I also promote diversity.

Heather Natour: Yes. Passion, people, purpose. I really love that. And I totally agree. It’s always been important to me to join a company with a mission that I’m passionate about. I really want to be working with people who are smart or challenge me to grow and learn in a positive way.

Heather Natour: And I really want to dive into some of these goal setting parts. I’d love to also hear from the audience, what role does goal setting play in your own career while we move on and post in the chat.

Heather Natour: But before we do that, as one of the first and few women technical leaders and principal engineers at Amazon, what were some of the common career roadblocks you faced and how did you overcome those challenges?

Raji Subramanian: Yes. Again, the challenges that I face and what I’m going to be sharing, you’ll find that there’s a lot of similarities with what all of us have faced. And in fact, as you called out earlier, as we are having this conversation, I’d love to hear from the audience as well, where they can post what the challenges that they faced. Again…

Raji Subramanian: And you’ll find that there’s a lot of commonality. But again, to touch upon a few things, and this is not specific to a certain company, but it’s more specific to the journey itself.

Raji Subramanian: The first, I’m sure all of us as technologists in whatever role that we play in technology, one of the biggest challenges is to become a recognized technical expert. And this is a nuanced topic. The reason I say it’s a nuanced topic is, there is being recognized as a leader, and then there’s being recognized as a technical leader.

Raji Subramanian: There are so many preconceived notions that we as women might be recognized as a leader even amongst our organization but are we recognized as technologists and engineers who can transform that world.

Raji Subramanian: And so how do you break through those preconceived notions?

Raji Subramanian: The next is about being in the know and being in the know is a lot about the ecosystem that you’re working with, the network that you have access to and the network you have deep relationships with.

Raji Subramanian: And as women how do you go ahead and build those deep relationships, whether it’s with peers, with colleagues, with managers and with mentors, and wherever you work is one of the key gateways for you to be in the know and being in the know within any workspace that you are in is what takes you… Is what gives you one aspect or one dimension to what you can… What actually takes you to the next level.

Raji Subramanian: The third is something that I’ve observed. And I’ve personally followed. It’s about leading from the front.

Raji Subramanian: We as women do an incredible job at work, but often we find ourselves that we ourselves sometimes or because of the forcing function of the environment leading from behind. We are silent leaders. It’s important that as we are making the transformation, we not just lead from behind.

Raji Subramanian: We also lead from front. Examples of that include as women leaders, and women technologists, and women engineers, we might vision something. We might be strategizing on something. We might be driving something.

Raji Subramanian: It’s important to also hold the mic and be actually the representative, who gives the voice to it. And that’s super critical, and not let our demons hold us back. I’ll give one example, Heather, both you and I, for example, are leading two of the most critical initiatives, literally the top two initiatives at Opendoor. In many ways as a part of that, it’s important for us to not just lead from behind, but also lead from the front.

Heather Natour: Yes, definitely. And becoming recognized as a technical expert really resonates with me. I personally didn’t study computer science in college. And so I always had this imposter syndrome about my technical aptitude and frankly, it took half my career to realize I was often the tech technical expert in the room.

Heather Natour: Also, as you mentioned, we are each leading from the front driving key engineering organizations at Opendoor. And, I actually think it was a result of very conscious decisions that we’ve respectively made to transform the technical vision in order to make greater impact. So I think these are all really great, helpful points.

Heather Natour: You mentioned Raji that you’re also very passionate about diversity, and environmental, and social governance. How are you working towards bringing change in in the tech industry and at Opendoor?

Raji Subramanian: It’s a combination deal, Heather. And there’s two parts to it. The first is it always has got… It’s always got to start from home.

Raji Subramanian: We’ve got to walk the talk. For example, I’m looking to hire 50 plus people on my team and I’m sure you are as well. And again, it’s about how we walk the talk and make sure that we build the diverse teams and building those diverse teams is what helps companies become durable and generational. And for example, that is one of the core values that we follow at Opendoor.

Raji Subramanian:And so it’s important that it starts at home, we’ve got to live, breathe, and make sure that our hires are the diverse. And we should never compromise on that. It’s got to be a non-negotiable goal, and the second part of it is about what you do beyond just your workspace.

Raji Subramanian: As I called out earlier, I do a lot of work in the ESG space and DEI is the S, the social part of ESG. And it’s one big component of ESG. The other two being environmental and governance, to make sure that we have a holistic approach to how we look at not just DEI but beyond DEI as well.

Raji Subramanian: So the work that I do with BoardReady is about, how do you make sure that management teams and boards are the most diverse? So I do a lot of work in that space. I also publish in that space, the research that I do.

Raji Subramanian: And I also work with a lot of companies, as a part of BoardReady to make sure that we are able to bring in the diversity. The last thing I’ll share is something that my parents actually had conversations with me when I was very, very young and this hit me then, and it still hits me.

Raji Subramanian: And one of the conversations that we had, we’ve had this conversation multiple times is what if women were engineers, innovators, and builders as a part of the Industrial Revolution? Would the companies stream of products may be very different than what they are?

Raji Subramanian: My heart says, “Yes, they would.” Then it’s super important for me that as we go through the digital revolution, women are not just key players, as a part of this revolution, but all the builders, innovators, founders, entrepreneurs, and creators, because that’s when you know, diversity really breaks through all of the ceilings and breaks through all of the biases.

Heather Natour: Yes. I mean, yes. Absolutely, yes. More diversity on boards, we’re getting women in places where decisions are being made, even at the smallest level, your point about it being non-negotiable is I think about being in multiple technical meetings at Opendoor a week where there are multiple women and you don’t maybe notice it when they’re not there, but it makes a huge difference in how you come to work and how you contribute when you do have that. I’m interested as you reflect on your career journey, what is your advice to other female engineers or women in technical roles looking to take their career to the next level?

Raji Subramanian: Yes. And again, and this is something we’ve all asked ourselves and it’s so timely given where we are in the year today, the first thing that I have reflected on my career journey and I’ll also encourage all of us to reflect, visible versus invisible results. Again, two simple words.

Raji Subramanian: So a lot of times you’ll see that as we have done a lot of work and as women, after we put in a ton of effort, we actually feel that the results that we’ve delivered are invisible results.

Raji Subramanian: It’s important to make sure that as we are going through a career, we’re delivering visible results, results that are measurable in their impact. And the measure need not be just a number.

Raji Subramanian: It need not just be a metric quantified, it can be qualitative in terms of the impact that you had on a customer’s life, on a platform that you built, on a pattern that you created, or an innovation that you drove, or an impact to a P and L. It could be any of those.

Raji Subramanian: It’s super important to parcel out. If you look at 2021, it would be great if each of you look at what were the visible results and the invisible results, and where do you spend most of your time? So that’s one, the second we talked about goal setting earlier, I use that framework and I use that framework in most things I do in terms of goal setting.

Raji Subramanian: The three P framework that I called out earlier, even when I try to find a new job or things to do.

Raji Subramanian: The third thing I look for or the third thing that I did personally and I highly recommend is work with mentors who question and challenge you.

Raji Subramanian: Work with mentors who enable you to break through your blind spots, work with mentors who approach you with a growth mindset. I think we, as women need those type of mentors to keep pushing us beyond and enable us to adopt that growth mindset. And the last I would say is be authentic.

Raji Subramanian: This is something that I’ve had to do. And I’ve had to learn how to do, coming in culturally from in the workspace. I think we have to build a leadership style that is unique to us. And that is one thing that I would recommend. And it goes a long way once you reach that point.

Heather Natour: Yes. Those all resonate with me. And I love those challenging the status quo is something I learned from my father. And while it’s sometimes uncomfortable for others, it’s really served me well.

Heather Natour: And I think we’re running out of time, but I encourage people to tell us what advice has made the biggest impact in their career. In the chat, Raji and I will answer a couple questions there and post how you can get in contact with us. Raji, anything else you’d like to share before we say goodbye?

Raji Subramanian: No, again, the only thing I’ll kind of share is that feel free to connect with both Heather and me. Happy to talk about our experiences, happy to talk about the transformation that we’re bringing in with Opendoor, as well as we grow our teams.

Angie Chang: Thank you, Raji and Heather for all your insightful thoughts on technical leadership. That was a fascinating conversation. Thank you.

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

“How To #HumbleBrag Effectively”: Shailvi Wakhlu, Senior Director of Data at Strava (Video + Transcript)

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Angie Chang: Our next session is a Strava coffee session. I think we’ll have to rename it up to, as Sukrutha said, chai, or other teas. We love Strava. So now let’s welcome Shailvi Wakhlu from Strava. She’s here to help us humble brag and advocate for ourselves.

Shailvi Wakhlu: Awesome, thank you so much for the introduction. Angie, let me share my screen. All right, hopefully you can all see that. Hi everyone, my is Shailvi Wakhlu. I go by the she-her pronouns and I’m the Senior Director of Data at Strava. Welcome to the Strava coffee break on the topic “How to humble brag effectively.”

Shailvi Wakhlu: If you want to get better at self-advocacy or know someone else who does, then I hope this talk resonates with you.

Shailvi Wakhlu: So first off I wanted to briefly cover what is self advocacy. It is speaking up for oneself and one’s interest. It’s a simple enough concept, but I found it to be very powerful for us to incorporate for our career success. Understanding, prioritizing and communicating our needs is how we grow ourselves and our careers.

Shailvi Wakhlu: So self advocacy is unfortunately not a skill that comes naturally to most people. Many folks, face difficulties in shining a light on their achievements or talking themselves up.

Shailvi Wakhlu: If that sounds like you, then you should know that you’re not alone. I think culturally, anything that remotely even resembles boasting can be considered a vice. And I also belong to one of those cultures that really encourages people to keep a very low profile as a norm

Shailvi Wakhlu: . Sometimes it’s for safety, sometimes it’s for acceptance and sometimes it’s purely out of habit. Women and marginalized genders also often receive direct and indirect feedback, “Do not be too loud and to focus on being a supporter rather than a promoter.”

Shailvi Wakhlu: Society in general also tends to really discourage self-promotion. Appearing, salesy or flashy often tends to have very negative connotations to it. Our brains have been programmed from society to sometimes just ignore messaging where our people are praising themselves.

Shailvi Wakhlu: And finally, usually at least for me, this has been true. It’s usually our own inner voice with whom we struggle to reconcile the desire that we have to be seen and heard, and to not be seen as bragging. And all of these in many other reasons can make self advocacy very hard.

Shailvi Wakhlu: So while it is hard, it’s still remains extremely important. And I’d like to say that advocating for ourselves is not bragging. In fact, it is a key skill, we all need to practice and learn to make sure that we are doing the right thing by our careers.

Shailvi Wakhlu: Self advocacy is absolutely crucial when it comes time to make more money, negotiate, better salaries. Because after all, how can someone pay you more than they think you are worth when you are the one responsible for telling them your worth.

Shailvi Wakhlu: Promotions also tend to be very heavily dependent on people, clearly articulating how they continue to add value to the business leaders who are in a position to sponsor or promote you, they often tend to need reminders of that.

Shailvi Wakhlu: If you’re a leader of a team or a project, your ability to get resources and head count often needs buy-in from superiors and peers alike who need to comprehend the value of what you and your team are doing.

Shailvi Wakhlu: Your ability to attract talent can really depend on how easy it is for you to convince someone that you’re working on something great. And it’ll be helpful to their career path as well.

Shailvi Wakhlu: Overall, I feel unless you get lucky, success really depends on your ability to make sure that people understand your value. So my pitch is, “Take that time to practice being comfortable, confident, and genuine and advocating for yourself.”

Shailvi Wakhlu: So many of you today in the stock are in the mid to senior part of your career. And I wanted to take this opportunity to plug that self-advocacy, isn’t something that only career professions need.

Shailvi Wakhlu: In fact, you are absolutely never done with self advocacy in your career. Even if right now you are in a stable situation, you’re successful in your role, know that situations can change any time with or without notice.

Shailvi Wakhlu: Maybe you currently have a good boss who’s an advocate for you, but bosses can change. If your current company and or your team supports you remember that can change too. Or maybe you switch something out, maybe you decide to switch companies or you get assigned to a new team.

Shailvi Wakhlu: So in that situation, you will lose access to some known situations that maybe you felt a little bit more prepared for, where you knew how to amplify your message.

Shailvi Wakhlu: Even if absolutely nothing changes, the fact remains that your legacy compounds. You cannot hope to rest on your past successes forever. And if you want your legacy and your impact to grow, you have to continue investing in your success to make sure that people don’t forget about it and make sure that people don’t overlook it.

Shailvi Wakhlu: And finally, this is an example that I often bring up that even CEOs need visibility. How else will you otherwise in that position, get funding, hire great people, support your employees, or get the best outcomes for shareholders? So just separating out that feeling, that you’re at the top of your career and you’re well respected doesn’t mean that you no longer have to invest in self-advocacy.

Shailvi Wakhlu: So with all of that in mind, how do we actually get better at self-advocacy?

Shailvi Wakhlu: I have three things to walk you through. We will first learn how to reframe our own internal narrative mental narrative. And we’ll walk through couple of examples. Then we reframe the external narrative, which is how we choose the right words.

Shailvi Wakhlu: And finally, we will practice, which is why my #humbleragchallenge comes in. So let’s examine some of our mental narratives that tend to hinder our self-advocacy.

Shailvi Wakhlu: The first example is “I am too busy for self-advocacy”.

Shailvi Wakhlu: How many of y’all feel that you just don’t have the time for a lot of things in your career leave alone self-advocacy? However, I feel that if you don’t make for self advocacy, you will hinder the progress you can make in your own growth.

Shailvi Wakhlu: Investing in your growth early is important, and it is essential. You don’t know what tomorrow holds and putting in the habits and processes in place today that might help you get to the next set of opportunities that you desire can really pay dividends when you need it.

Shailvi Wakhlu: Next example is that feeling of my talent to just speak for itself. And a lot of us have actually been told that if you do really good work, you will eventually be rewarded for it. Shine rightly and the world notices, but maybe, maybe people are just not in the same room.

Shailvi Wakhlu: So at the end of the day, if people can’t see something, they can’t acknowledge it. So we tend to assume that our talent will shine and everybody will know that we are great. But maybe everybody around us is really busy and they can’t keep track of the tiny details and examples of how you add value. So acknowledge that you need visibility, and it’ll be much easier to talk about your achievements.

Shailvi Wakhlu: Another mental narrative is that, as you get to more senior roles, the ambiguity increases, and that is true. Often, I think just the previous talk was talking about this, that sometimes there’s no defined part.

Shailvi Wakhlu: There’s no job description, there’s no… Especially for senior roles. And the implication is almost at a lack of defined path means you don’t need to focus on self-advocacy because everything is a little bit of a gamble.

Shailvi Wakhlu: And so focusing on your own visibility may not lead to predictable outcomes and may instead be a distraction. And to that, I say that self-advocacy is indeed taking that time to create your thoughts.

Shailvi Wakhlu: Write your own job description, define what success looks like and connect the dots on how it adds value to the company that you work for. The need for self-advocacy, doesn’t end just because there’s no defined playbook. It still needs intentionality and it still needs attention for you to move forward and grow, even if you’re not sure exactly what the destination is going to be.

Shailvi Wakhlu: And my final example for the mental narrative is that feeling that we have that hey, maybe celebrating wins is just… It’s flashy. It’s unnecessary. And maybe some of us find it inauthentic. However, here again, I say, and I especially say this to leaders that celebrations aren’t just for yourself. They are for absolutely everyone around you. Therefore, that colleague who’s struggling with their work.

Shailvi Wakhlu: Therefore, the people who are looking for their examples of success, or just your teammates, anybody who is involved with the project and anybody who’d like some validation that their work produced something that was good. You are essentially thanking the universe for that opportunity to produce value. And you’re expressing gratitude for that successful outcome. So you’re acknowledging the hard work behind it is going to help really boost everybody’s morale in the process.

Shailvi Wakhlu: So now we’ve covered a few of the mental narratives. We’re going to pivot to the external narratives. And once we feel comfortable in our own mind, that self-advocacy is the right thing to do.

Shailvi Wakhlu: We can focus a little bit more on improving the specific messaging that we use to talk to others. So I’m going to walk through an example of how you might choose to talk about a project that was just finished. That went well.

Shailvi Wakhlu: And there are obviously multiple ways that you could do this. So the default way you maybe… Maybe it’s not default for you, and that’s great. But maybe it’s just downplay it. Project was no big deal. Anybody could have done it. And that’s it. Then there’s the other extreme, which is that you brag about it, which you say, “I did the best job. Nobody else could have done it.” And that’s sort of the other extreme.

Shailvi Wakhlu: So here’s my version of it, which is the team came together to land an incredibly challenging project which also became a fun way to expand our skills. We pushed hard and were able to finish it in half the time that it was expected to take. So this is a mix of sharing a win, but with authenticity that translates into a humble brag.

Shailvi Wakhlu: So notice the keywords that I highlighted here, acknowledging that something was challenging and that you worked hard on it. It shows your ability to accept and shine on tough problems.

Shailvi Wakhlu: Highlighting that you’re capable of bringing together a team to actually collaborate and you are sharing credit for it. It shows important leadership qualities, including a measurable achievement provides something which is a tangible win for people to focus on.

Shailvi Wakhlu: And showing that you actually learned something and grew from it and had fun doing it. That just brings it all together to confirm that this was a natural fit for you. And you’d like some more opportunities like this in the future. So coming up with the right words is helpful, but you don’t need to overthink it.

Shailvi Wakhlu: If you sit down and think about the true feelings that you have about a success story, the words will come to you and your messaging will be authentic. This is just an example. So don’t try to retrofit every story into a success template. Just be genuine and talk about what you truly care about.

Shailvi Wakhlu: I think you’ll be humble bragging effectively in no time. So finally, no commitment is complete unless we figure out a way to go practice it so that it comes naturally. And for that, I have two simple challenges. If you’re new to this, you can start with something small and one is personal, one is public. It really depends on your comfort level.

Shailvi Wakhlu: So on the personal side, this is something you do for yourself. You can just start documenting your wins, make a habit of it. It doesn’t have to be something big, just anything that you’re proud of. Anything that you feel you hit some success on. It can be as simple as, “Hey, I connected two people and they made something else successful.” That’s a win.

Shailvi Wakhlu: So additional guidelines here is be specific so you remember the details and it’s easier to tie back that connection later on and do it with the intention that you will plan to use it for your next promotion. If not the specific example, then at least the themes that you come up with over time, and maybe you don’t end up sharing the list exactly. But it’s really helpful to have that place to reflect back on things that you do well on things that work for you.

Shailvi Wakhlu: And it also makes it easier for you to talk about your work and not blank on it if someone asks you questions about it. So my suggestion here is to do it at least weekly at the bare minimum. And I think it’ll go a long way in building confidence.

Shailvi Wakhlu: Next level is to go public. So whatever your team’s shared mode of communication is find one avenue that you’re comfortable with and post your achievements there. Make it into a team thing. Make it where you share learnings.

Shailvi Wakhlu: Do it at least once a month. And that’s a good way to get everybody involved. We used to definitely do this in one of my teams previously and everybody found it a good way to actually learn from it and just grow from it.

Shailvi Wakhlu: And yes, if you do decide to do this, please… If you do this on Twitter or LinkedIn, please tag me. I would absolutely love to hear about your personal success stories and celebrate with you and amplify you. Thanks all for listening.

Shailvi Wakhlu: I hope this was useful. You can follow me on social media if you’d like to stay in touch. And if you’re interested in the content that I share.

Shailvi Wakhlu: I am also releasing a book later this year on self advocacy and would love feedback on any of the subtopics that might be relevant to you. And finally, a big thank you to Angie and Sukrutha from Girl Geek X for hosting today. Thanks.

Sukrutha Bhadouria: Thank you, Shailvi. 

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

“It’s A Hot Job Market. Do You Stay or Do You Leave?”: Panel (Video + Transcript)

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Angie Chang: So our next session, I want to bring up a panel of three mid-to-senior technologists who changed jobs for a bigger role at another company. And they’re here to talk about how it’s a hot job market and how to know when it’s time to leave. This is very timely with the great resignation, or great reshuffling, you all hear about. And so now let’s welcome Aliza, Rocio and Sharon! And I’ll let you introduce yourselves.

Aliza Carpio: Hello everyone, and welcome to our session. I’d like to take a moment to recognize that we are all in challenging times. And for those of you out there with family, friends, and loved ones who are impacted by the conflict in Ukraine, our hearts and our thoughts are with all of you.

Aliza Carpio: Now, we hope that our story and the tips we’re going to share are helpful for you and we’ll inspire you as well. But before that let’s do some intros. We’ll start with Sharon.

Sharon Hunt: Hey everybody, my name is Sharon Hunt. I am the Head of Product at a company called Clovers. We’re fairly new. I’ve had a pretty, I would say eclectic career journey.

Sharon Hunt: So I’ve done everything from selling cars and bartending to spending the last 10 years in product management and in technology where I had the good fortune of meeting these two ladies, add into it first hand. Rocio, tell us about you.

Rocio Montes: Hi everyone. My name is Rocio Montes. I am a Senior Engineering Manager at GitHub. My team, that compute engine team powers, all the runners that provide the compute power to all of the actions workflows.

Rocio Montes: Like Sharon said, I have also recently switched jobs from Intuit to GitHub. And it was definitely a challenging decision and an emotional rollercoaster to really make that jump. But you grow and you challenge yourself. So here I am very happy with that. So we’re here to tell you all of about it.

Aliza Carpio: Rocio, what’s your fun fact?

Rocio Montes: My fun fact, you always get me on this one Aliza, I can lay on the beat for eight hours straight.

Aliza Carpio: That’s a good one. Hey, everyone, I’m Aliza Carpio, Director, Technology Evangelist at Autodesk. In my role, I get to work with teams and leaders from across the globe to amplify the engineering magic that’s happening at Autodesk and help build our tech story.

Aliza Carpio: Now my fun fact is that, and I think these ladies know, my favorite song to sing in karaoke, if you can get me to sing is Britney Spears “Hit Me Baby One More Time”. So for those of you out there that are Britney fan, I’m definitely with you. So we chose the title. It’s a hot market – do you stay or do you leave?

Aliza Carpio: Because as Rocio mentioned, each of us were faced with this question and the opportunity last year and the year before during the height of the pandemic. Now this pandemic accelerated existing trends in remote work, eCommerce and automation. And many as you all may have heard, even from Angie may have heard the term passive recruitment or job search because for technologists like yourselves, you probably are not having to pour too much energy in searching for that next opportunity.

Aliza Carpio: And I’m sure many of you out there, the opportunity is already knocking at your door. Sharon, let’s start with you.

Sharon Hunt: Yeah, it’s such an interesting time right now. In fact, that’s why we started Clovers because it’s such a candidate’s market. And a lot of companies just don’t know how to create a great candidate experience and that bleeds into creating a great culture, creating a great place where people feel like they can grow. A

Sharon Hunt: And if you really look at what’s going on right now, this whole era of the great resignation, it’s really highest among people between 30 and 45 years old, right in that middle career. And there’s a lot of theories about this. Now that there’s been some really definitive, causal factors there, but there are certainly some indicators. So the fact that it’s, mid-career definitely is a big indicator.

Sharon Hunt: And I think the fact that so much of technology now has opened up to remote work. It really just blew up the landscape of opportunities for people. Because you don’t have to think about, “Hey…” Especially for people with families, “Where can I travel? What’s close by? What’s convenient? How can I grow my career at a place that also lets me live my life?” A lot of that is now way more…

Sharon Hunt: There’s so many more opportunities to do that across the country and even across the world. So one thing I read it in Bloomberg recently actually, is that the number one factor for people, even companies is culture 12 times more likely than compensation? Which is really interesting, especially for us ladies out there.

Sharon Hunt: So I think there’s a lot of… I look around and look at my colleagues. I see the career shifts that people are making and I think there’s a lot of women, other folks as well, but especially women who feel like maybe they’ve reached a little bit of a ceiling at their current roles.

Sharon Hunt: And that’s baked into the culture to a certain degree. And company who are attracting that talent are going to win in the long run. And I really believe that’s a big driver for especially people are leaving. It’s that culture? It’s that feeling you want to grow? And finding that all of a sudden you’ve got way more opportunities than you’ve ever had before because of the nature of remote work. Rocio, what are some things that you’re seeing in the job market?

Rocio Montes: Oh man, Sharon, definitely. These are significant shifts. I also read that as many as 25% of workers may need to switch occupations than before the pandemic. Also saw an article from Hack Reactor where the demand of engineers will continue to increase due to these shifts, right? So it’s definitely a hot market out there. Things are changing.

Rocio Montes: And I have some friends and past colleagues like YouTube who are lending new opportunities, sometimes bigger ones, sometimes different ones. As for me, I recently moved to GitHub from Intuit. And this happened after eight years of being with the Intuit family.

Rocio Montes: And it was definitely a very big decision to make. Some of the things, and I’ll walk you through some of those motivators for me to switch, was that in the last road Intuit, I was reporting to the chief architect with Aliza focusing a lot on development productivity. We founded the open source office and started to kick off the inner source initiative.

Rocio Montes: And GitHub was at the center of both of those things. So for me, it was more of a natural, transition, because I started to get very passionate about all of the features that GitHub was releasing, getting involved with, GitHub as a maintainer, et cetera.

Rocio Montes: And when I saw the opportunity at GitHub as an engineering manager, it just made sense for me as the continuation of my career growth, the continuation of the journey I had been as an engineer.

Rocio Montes: So that was for me, right? Aliza and Sharon, what shifts did you make? And what elements do you think we need to consider when planning your next move?

Aliza Carpio: So you probably all heard that we all kind of came from Intuit. I also made a shift last year. I actually started at Autodesk late August of last year and I actually was seeking growth and learning.

Aliza Carpio: I had been in the company for a long time, started off as an engineer and did a lot of different roles, including product management, dev manager, and even marketing and what I ended up doing, and we’ll talk a little bit about, this is I did write my job description and I’ll tell you all about that.

Aliza Carpio: But my big thing that I’m going to say out there is a principle that I learned from a mutual manager that Rocio and I had, which was the chief architect. Where he always gave people the same advice, “When you were looking for that next role, make sure you are running to somewhere or to something versus away from something.” And it’s a big principle. Hey Sharon, I want to ask you what about you?

Sharon Hunt: Yeah, that’s a great principle, reflecting back on my own career journey for the last few years. When I left into it both of you were there as well. And I was really looking for more ownership as a product manager, those of you and the audience might commiserate with we’re a little bit of control freaks.

Sharon Hunt: We like to be in control of things. And I really wanted to own more strategy, more of the roadmap. And I just realized at a really large company like Intuit, that was going to be much harder to do in the time period that I was looking to get it done at. So I moved to a company called Housecall Pro really exciting, well established startup that was doing field service management work. Spoke to my soul because I come from a family of painters and landscapers and tree climbers and the whole blue collar scene.

Sharon Hunt: So I was building software for people that I love, but really, then was quickly promoted into director at that role and got to fully lead and owning the roadmap and realize what it meant to experiment in the product led space. And I fully felt like I realized what I was looking for in that position. Then I hit another little bit of a, I would say, not a ceiling, but the company shifted.

Sharon Hunt: We went into more operational mode we had product market fit. It was time to scale. And at that point we had been running very lean and we brought in more product management talent and the company wanted to bring in a VP that had gone through a period of massive growth before. Because there are certain lessons that you learn when you go through that if you haven’t been through it before you can’t bring to the table, which was quite fair.

Sharon Hunt: And I actually had a one-on-one with Aliza, because at that same time, my current opportunity fell into my lap as the Head of Product that a new startup. And I said, “Aliza, what should I do? How do I even navigate through this?”

Sharon Hunt: And she really encouraged me to write down all the things that I was looking for in my new role. And why I thought that, I was qualified to do it at Housecall Pro and really be transparent with my manager that time around what I wanted and why I thought I was a good fit.

Sharon Hunt: And he heard me, but I think he had already made his choice. But the exercise of doing that, just writing down what I wanted really codified in my mind what that was. And it helped me make that decision between, “Okay, do I stay here? And I’m going to learn going to continue to grow through a product position. That’s going to go through crazy growth or do I go back over here into a new opportunity to really build something from scratch, to establish processes, to establish leadership principles that I’ve sometimes felt were a gap in previous companies?”

Sharon Hunt: So writing it down really helped and what it made me realize, and this kind of goes to I think what you’re saying Aliza about running to versus away from something is, “What are your hygiene factors? Are you feeling like you’re not appreciated or you’re not paid well enough or you’re being mistreated in some way. If that’s the case, then you might be running away from a position – and that’s probably not a bad thing.

Sharon Hunt: Maybe that’s a toxic position for you and you deserve and belong to something new. But don’t let that make you underestimate or under serve yourself when you go to the next position, because you might not fully bake and understand what truly motivates you, what you really want out of your next role.

Sharon Hunt: And you might just jump on the next thing that at least meets your basic criteria. So it really helped me kind of codify what are my hygiene factors at my last position I felt like I was getting that. But what did, what did I want on top of that was going to help me to grow?

Sharon Hunt: So I would say a principle here is make sure that if you’re leaving your current role, because your basic needs aren’t being met either monetarily or from a respect position, don’t move to your next role just because if they meet those baseline criteria. Still understand, what on top of that you want, and then still aim for getting that full package in your next role.

Rocio Montes: Yeah, I think I would add again, I would go back to what, “What are you passionate about? What excites you in the morning?” To me, it was really clear year about what are some of the things and the features that I wanted to work with and how I was always trying to pull in GitHub features and trying to onboard to all the new features that you know were coming out and it was just that what motivate me…

Rocio Montes: What I realized that I was very passionate about that drove me to really find and to make the leap, to changing roles.

Aliza Carpio: Folks, this is really golden. Let me share a couple of things here. I love what you all are sharing and Sharon and Rocio please do add if I miss anything. But in that question around what elements do you need to think about or consider when planning their next move? Do you want to take that first one, Sharon, and I’ll take the second one?

Sharon Hunt: This is your journey. Take a moment to self reflect. Yeah, I think this kind of pairs with the last one, honestly. What is it that motivates you? It’s all kind of boils down to that. Are you motivated by trying to escape or are you motivated by trying to grow?

Sharon Hunt: Sometimes it’s a combination of the two, but don’t just escape if that’s what you’re trying to do, escape and grow at the same time. Why are you looking? What is it that is driving you out of your current position? Fully understand that.

Sharon Hunt: And then a click deeper really is once you understand what is driving you… Do you understand where that driver comes from? Is this something that you really want? You yourself as a fully formed human, or is it what you were told that you should want by your parents or by society, or by your significant other or your church or your religion?

Sharon Hunt: There’s so much of that influence that is imposed upon us from the outside, that it takes some really deep thinking to truly know what it is that’s driving you so that you can make sure that the next thing that you pick is going to really check those boxes in a deep way, in terms of motivation. So I’d say Aliza, that kind of combines…

Aliza Carpio: The one and three?

Sharon Hunt: Yeah.

Aliza Carpio: That second one I mentioned already around making sure that you don’t do a couple of things. That you avoid running away from something, but shift your mind to running to something thing. And then there’s this second one, which I actually got from Shannon Lietz. She’s VP of Security at Adobe. And I really love it when she asks me, “Are you letting your current job interfere with your career aspirations?”

Aliza Carpio: You might love what it is that you do, but you really need to also think about the fact that you own the destiny of your career. So think about what is it that you are aspiring to do or want to achieve and not let the current job interfere with that. There’s a lot more potentially that you could be doing to be more impactful. So thanks for that. I’m going to stop sharing and get back to our chat. Rocio, I know you were going to maybe have a story about a friend who reached out to you.

Rocio Montes: I wanted to share that I recently had a friend reach out to me because she was contacted by a recruiter and she wanted my advice about what are some of the things that she should consider when switching jobs? And I’m sure that both of you have advise that you could give us, right?

Rocio Montes: What are some thoughtful techniques in searching for the next role or your next organization? What are those things that we should take into account? I can start with one and obviously we all have a great value, a great worth. So look out for compensation. You want to grow, you want to make sure that all of your skills are getting compensated correctly.

Rocio Montes: Think about when you’re asking for your new offer, why are you leaving behind? If you have stocks that you’re leaving behind, really try to think about that monetary aspect, because obviously yes, we need to follow our passion, but we don’t leave out of love. There are bills to pay and things to buy. So it’s really good to sit down, look at numbers. And if you need advice on that, always try and find someone that could give you that advice for those calculations. Sharon, what do you think?

Sharon Hunt: Yeah, I think it got two pieces of advice. One goes back to what we were just talking about around really understanding your motivations. Actually came across a really great article a while back called How to Pick a Career (That Actually Fits You) by a fantastic writer named Tim Urban. He runs a blog called Wait But Why. I highly recommend, its hysterical. But the article is really about deeply understanding what motivates you.

Sharon Hunt: And he has this concept called the yearning octopus, where sometimes you can have conflicting needs. Sometimes we want social status, but we also want to give back to humanity, but we also need to support our parents. So sometimes these things are really in conflict with each other. So really diving deep and articulating all those disparate needs. He breaks this down in a really wonderful framework. The blogs called Wait But Why? And the article is called How to Pick a Career (That Actually Fits You) by Tim Urban.

Sharon Hunt: I really recommend it. It’s a bit of a read, but he actually provides a series of exercises to go through. I actually did it myself and it was one of the things that made me choose to lead my position at Intuit and really helped me understand what it was that drove me, which person was doing work that was more aligned with helping people that I loved, which is why I picked Housecall Pro, which built software, or like I said, for people that I care about.

Sharon Hunt: The other tip that I have is if you are thinking about doing something that is a little bit new, definitely go find someone that does that job and just if you’re able to shadow them, literally sit with them and see what their job is like.

Sharon Hunt: That might be a little tricky, but in lieu of that, at least have a deep conversation. Actually tell them to walk you through what their day was like. “Tell me what you did yesterday.”

Sharon Hunt: Ask them what their favorite parts about their job is and what their least favorite job parts are. And really do that a few times with folks that are in that position to get a deeper understanding of what the role is because your assumptions about the role might not actually be the reality. And so really getting that ethnographic connection with the person who’s doing the role might really have help you understand whether or not it’s going to meet those motivations that we just chatted about. So my [crosstalk].

Aliza Carpio: Sorry, I know we’re running at overload time because we have one more minute, but let me just squeeze this in.

Aliza Carpio: I do recommend that everyone write your own job description. What is it that you want to see in your own reality in what you want to become? And I will tell you right now that I have my role because I wrote my job description and I presented it to a couple of VPs in Autodesk. I also did the same exact thing in my last role at Intuit. And it will actually help you also find your ideal, not only target state, but your ideal job out there.

Aliza Carpio: Whether you’re looking at LinkedIn or just looking around and talking to people. And so we hope that these tips are great. We’d love for you all to connect with us. We’re out of time, but please to connect with us and let us know how we can help you.

Angie Chang: Thank you for sharing job search experience and strategies.

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

“Engineering Leadership”: Engineering Management Panel (Video + Transcript)

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Sukrutha Bhadouria: Right now, it’s time for our next session. It’s the engineering leadership panel with Jenn, who is a Senior Director at Etsy, Kamilah, who is the Head of Financial Products at Gusto, and Willie, who is a VP of Engineering at Salesforce. We’re going to get together and discuss all things engineering leadership. Hi.

Sukrutha Bhadouria: So as we have all the amazing panelists join us via Zoom, I’m going to quickly run through their backgrounds and their experience level as managers so it can set things in context.

Sukrutha Bhadouria: Let’s start with Willie. Willie has been a manager for nine years. Within less than a year of turning into a manager, a role that she sort of begrudgingly got into, she even started to manage a manager.

Sukrutha Bhadouria: So now that I started with you, Willie, and we have everyone here, let’s go into why you had to go into the role of management begrudgingly. Why did you have to be coaxed into it?

Sukrutha Bhadouria: And since you’ve gotten to managing a manager so quickly after that, which one was the harder transition to make? Was it being a new manager or managing a manager?

Willie Hooykaas-Baldwin: Yeah, I call myself a reluctant manager, because the 14 years that I was a software developer, all my managers would always ask me, “Oh, why didn’t you go into management?”

Willie Hooykaas-Baldwin: And I always had this feeling that management managers were, I didn’t have a good [inaudible] them. They played and they had, they manipulated and they weren’t very transparent. That’s what I felt like. And during my time, all of them were men and I didn’t see myself in it.

Willie Hooykaas-Baldwin: So then I joined Salesforce as a developer, and after a while I became scrum master, which I really enjoyed and I loved working with my team. Then there was a reorg and reorganization and our team was moved from one place to the next and our manager stayed behind.

Willie Hooykaas-Baldwin: And at that point, my peers on my team apparently said, “Hey, why didn’t you ask Willie to become the manager?” And that had never happened to me. And so, the guy I was going to be reporting to was a super good manager. And he said, “You know what? Why didn’t you try it for one year? And after a year, your development skills haven’t atrophied. So after a year, you can always go back.”

Willie Hooykaas-Baldwin: And I started and I’m still doing it. So that’s been going well. For me, the actually getting a manager under me was more, I will say, almost dramatic, because I hired a wonderful one and she was super excited, and she would come to me. She’s like, “Hey, why are we doing this? And why are we doing that? And, we should be looking at this.” And every time she said that, I was like, “Oh, I should have done that. And I’m such a failure because I didn’t.”

Willie Hooykaas-Baldwin: And then of course, luckily as she was doing such a great job and I saw what that did to the team, I realized that it was perfect to actually hire her. Because I couldn’t do it anymore. I couldn’t give the team really what it needed. And so because I had other things to work on. So that shift in thinking that it’s not just, I’m a failure, she came in. That was hard. And only when she ended up hiring a manager under her, did I fess up and tell her, and it helped her when she had to go through it. So, yep.

Sukrutha Bhadouria: That’s so interesting. I see a lot of people commenting that they can sort of connect with that. Now, interestingly, you, Jenn, you’ve been for over six years now and you had a little bit of runway before you started to manage managers. And your belief is that your transition into management, first time management was a bigger jump than your org growing and you having to manage a manager. Tell me more.

Jenn Clevenger: Yeah, that’s a great question. I’ve been in management about six and a half years. I managed, a little bit before that in consulting, but it’s the people portion of the management wasn’t there is more about projects.

Jenn Clevenger: And so, why was my transition into management a bigger jump than managing managers? I think it’s because, at the end of the day, I had never really had a good manager before.

Jenn Clevenger: And so the very first job that I accepted was at Etsy and that was my first job into true management as I defined it today. And the job was offered to me by someone named Brave, who was the best manager I had ever had. I’d never had a manager like him before, so I’d never had that experience.

Jenn Clevenger: And so this is a really, really big eye opening experience for me. And a really big jump because honestly, I kind of had to redefine and rewire everything that I thought that I knew about this job, but also while doing the job right. Because I was already hired to be a manager.

Jenn Clevenger: And I figured out pretty quickly that I had no rubric or real understanding of what it meant to be a good manager, had to threw everything away. I had just never seen it. And just like to paint the picture because it’s fun, I wasn’t young at this time. I was like, well into my thirties, I had one and a half kids, I was super pregnant.

Jenn Clevenger: I had already gone through kind of a big long chunk of my career at multiple big companies.

Jenn Clevenger: And I had gotten used to kind of struggling through my career, looking for a mentorship or this elusive thing that people called sponsorship. That’s what you really need is to find an ally and support. And I just never was able to do it successfully. And so I felt really, really alone all the time in navigating my career.

Jenn Clevenger: And so I had to do a lot of reprogramming because it turns out that is actually the manager’s job to help you with those things and to support you. And so that was now my job and I just didn’t even know that.

Jenn Clevenger: So just to make it super tangible, a few things that I remember learning really early on just by watching Brave do his job. And it was kind of this epiphany like, “Holy crap, this is the job. It’s kind of cool.”

Jenn Clevenger: I realize as my manager’s job to advocate for me in places that I’m not, that’s a huge one. And it’s my manager’s job to help me navigate my career and to think about me in opportunities for me, and that is part of their job and not just like a favor or a two minute piece of advice that they’re going to, they have a couple minutes and they’ll drop some advice as they walk past your desk, because that’s like it’s actually a foundational piece of their job right?

Jenn Clevenger: And then, outside of me, me, me, me, and it’s also your manager’s job and responsibility to prioritize things for your org and your team like fairness, inclusion, diversity, building a culture of psychological safety. Like those things don’t happen for free or out of magic, it’s your actual job to build these things into your team. And that was really fascinating to me. I just never seen it before and I’d never considered it as part of the manager’s job.

Jenn Clevenger: And so, I think it was a really hard transition for me because I had to relearn all these things very late in my life. But it was a really good one, and I’m so thankful for having had such a great manager at that really important transition point in my life, to show me what I think is the difference between being a good leader, versus just being a manager. Like, can you imagine the type of manager I might have turned out to be with this whole life of experiences that I was carrying along with me and going in this total other direction. So that was a really big, big change for me.

Sukrutha Bhadouria: That’s interesting Jen, because we have on the other side of the spectrum, our newest entrant into the dark side, Kamilah. You’ve not been a manager for a full year even.

Sukrutha Bhadouria: We’ve known each other a long time and when I first met you, I feel like I remember you saying you’d never move into management. You were, “an IC for life.” like, wanted to grow as a tech lead.

Sukrutha Bhadouria: So I’m curious and I’m sure most people will be, what made you change your mind? We meet a lot of people at Girl League events that are sort of sitting on the fence and not really knowing what their path is. So I’m sure it’s going to be insightful for everyone.

Kamilah Taylor: Yeah, yeah, absolutely. I very much had the, I just wanted to grow on this tech lead path, which I actually did. And so I think there was part of it where I got to that goal of like, “Okay, I can really do this. I can be a very senior tech lead, I’m able to accomplish this and I can be really effective at this as well.”

Kamilah Taylor: And I think that was a really rewarding experience for me. But I think sort of similar to Jenn, I’d also had just a lot of anti examples of what to do as a manager. And I think that was a huge part of my hesitance is that I was like, “I couldn’t see the value in the job because I wasn’t seeing a lot of great examples of it, and it seemed like a fairly thankless role.

Kamilah Taylor: And there were a couple of things that happened. One was that, I took this course some years ago, General Management through Harrison Metal. And that actually started to shift my mind and I was like, “Oh, I understand what the role of management is and in making this a functioning organization and really what you want to see.”

Kamilah Taylor: And I think that was like the beginning of my, maybe not opposed to trying this out at some point. Like, I can see where this, how impactful it can be to have the right people in this role. So I had that in the back of my mind.

Kamilah Taylor: And then when I joined Gusto, I think similar to Jenn, I had finally my first example of like, “Oh, this is what a good manager is.” And I voiced pretty early on when I joined that this is something I was thinking of.

Kamilah Taylor: And again, very similarly, he really advocated for me and had me in the right rooms. And when I wasn’t there, would voice and sponsor me. And I remember having this moment, I think the first time that happened like, “Oh, is this sponsorship amazing?” I read about this for so long, now I see what they mean when they talk about sponsorship.

Kamilah Taylor: And then seeing who would work through coaching people on his team at different levels and really helping to grow engineers. And I think all of that just gave me okay, like, this is something that I think would be really rewarding.

Kamilah Taylor: And then the last part of it is that there’s, I think if you’re really a very, or there are a couple of different archetypes of the senior tech lead and I’d say there’s at least one of them that starts to overlap with an engineering manager, because you are coaching a lot of engineers, right? And you’re helping with that prioritization and the strategy.

Kamilah Taylor: And so as I found myself growing into that archetype of the tech lead, I thought, “Let me try this out and see how this goes. So, I’m still within my one year, I have a friend who has a bet. They believe I will stick through it past the one year. And I’m honestly, I am enjoying it. It’s been very rewarding so far.

Sukrutha Bhadouria: That’s wonderful, glad you’re more inclined to stay and now you’re also managing a manager. So [crosstalk].

Kamilah Taylor: Yeah. Yeah. I mean, I probably owe my friend a hundred dollar dinner, but that’s fine.

Kamilah Taylor: So, Willie you spoke about the early struggles and I know from our conversations you’ve considered not really having that strong network as one of your early missteps.

Willie Hooykaas-Baldwin: Yeah.

Sukrutha Bhadouria: Well, tell us more about that and what is it that people can do and what the benefits are?

Willie Hooykaas-Baldwin: Yeah. I think for me, it was very much, there were very few women around during my whole career, not until I got to Salesforce to become a little bit more common. And so I was not used to confide in people or reaching out to people. Asking for help is actually a skill. And I definitely then wasn’t good at it.

Willie Hooykaas-Baldwin: So what happened was, I became a manager, had some manager under me and then I would feel super, super heavy that everything depended on me, and I had so much responsibility, and it freaked me out from time to time.

Willie Hooykaas-Baldwin: So what I would do is, this was still pre-COVID in the office. So I would get through a very difficult meeting, run to the bathroom, get a cry out and just like, “Ugh…” And then suck it up, and kind of like, “All right, I got this, going there.”

Willie Hooykaas-Baldwin: If I look back on it, there’s such a waste of energy and such a lot of, it takes too much out of yourself. And so the stupid thing of me was that I did tell my ICs that they needed help and they needed somebody to commiserate with. But I hadn’t figured that for myself yet.

Willie Hooykaas-Baldwin: Now I’m a lot better. I have a very solid network of friends and peers, but also people above me and people below me. So people above you is super important because they kind of have the experience and that viewpoint that you’re striving to get to.

Willie Hooykaas-Baldwin: And so one of the benefits there was, early on when I had to add more managers, I had this feeling like, “So, I’m I just going to hire the same person over and over again in a way the same template?”

Willie Hooykaas-Baldwin: And my then mentor, who was, I think even two levels above me, she was just like, “No. Well, if one of the things that have worked for me is that I hire for what I am not good at or what I don’t like to do.” And that was spot on, right? Because you’re forming a team and in a team you need different abilities.

Willie Hooykaas-Baldwin: For myself, I actually really enjoy mentoring. I have quite a few. From time to time, I’ve had to cut back because it does take time and it takes a lot of listening. But it’s a way of giving back.

Willie Hooykaas-Baldwin: My network is where I commiserate, where I vent, where I ask for help, where I start my first ideas. And, “Hey, what do you think?” I’m thinking this, or there’s this situation.”

Willie Hooykaas-Baldwin: Doesn’t mean that they’re all at the same company. Keep people around, I would say, people that I trust and that have nothing to do with my current company. That can be super helpful too. So yeah, having that network is really important. Everyone in my org, I ask, “Do you have it? If not, can I help you get one? Like, can I matchmaker?” That kind of thing. It’s important.

Willie Hooykaas-Baldwin: I feel like, especially remote, right? It’s so valuable to have that network of folks who are doing this job or similar jobs in other companies, helps you like right now you’re not going crazy or everybody’s trying to figure this thing out right now.

Willie Hooykaas-Baldwin: Yeah, and anything in life almost takes a village, right? It’s not just rearing kids, as they always say, but, it’s almost anything. And that village, that can be your network and that can be so much easier. Life doesn’t have to be that hard. Not like I’ve made it.

Jenn Clevenger: I wonder. Can I ask, because I feel like that’s one of the things that I struggled with is building that network. I’m just not great at it, I never really have been. And then the past two years have just made it so much worse.

Jenn Clevenger: I’m not even really good at keeping in touch with my friends, let alone going so far out of my comfort zone and building a network from scratch. Like what kind of tips?

Jenn Clevenger: It sounds like you have had a lot of success in being able to build and maintain that and that’s awesome. I feel super jealous. Like what kind of tips can you give about that?

Willie Hooykaas-Baldwin: I picked up many of them through work, different jobs that I’ve had. Then at classes, very often at certain courses that you take, there’ll be somebody and you’re put together and you have to do some stupid exercise and yet you find and it clicks and then I’m like, “Oh, okay, let me talk some more.” And the ones that it clicks with, I kind of stay in touch with now.

Willie Hooykaas-Baldwin: During COVID, just like I’ve had a really bad time, I have not added on anyone for me. I’ve had mentees reach out and I’ve taken on new mentees, but yeah, I’m really looking forward to the next kind of big gathering, and talking, and meeting people again in the hopes that something happens there.

Willie Hooykaas-Baldwin: But most of them are different projects I’ve done, classes, that kind of stuff. It’s hard. Networking is super hard.

Jenn Clevenger: Yeah. It’s super hard.

Willie Hooykaas-Baldwin: It’s super uncomfortable. Yeah.

Sukrutha Bhadouria: Yeah, and it’s also like, how do you break the ice like you said? Jenn, things have just gotten so much harder with COVID, but I feel like just from the comments we’re seeing in this conference, that there’s a lot of people who are feeling very energized to go outside of their comfort zone, including me.

Sukrutha Bhadouria: So I feel like all of you spoke about having this, not such a great opinion about management and through that I sort of sensed and also directly got that you probably just didn’t have the right examples of good managers.

Sukrutha Bhadouria: So Kamilah, now that you are, you’ve been managing a manager for the last few months, what are the qualities you believe should be in a manager? That’s not just managing an IC, but also managing managers, more junior or more senior, what is it that everyone can learn from?

Kamilah Taylor: Yeah, it’s definitely been really an interesting transition to, and in fast succession, which is part of the life at a growing startup, for sure. Something that I found interesting is that there are a fair amount of similarities to managing or coaching, coaching a manager and then also coaching a more senior engineer.

Kamilah Taylor: I think that was something that I didn’t sort of got on onto immediately, and of course did my thing, I read, read lots of books, read lots of things as I tried to figure out how to get into that mindset. And there was some differences, but also like honestly I think a huge overlap.

Kamilah Taylor: The other thing that’s been helpful for me is to also recognize that everyone has their different way that they prioritize or think of things when they’re making those decisions and leading a team and not trying to, like for me to be an effective of manager, I have to meet people where they are and understand what is the right types of guidance and advice to give people, that’ll resonate with how they work through and how they lead on.

Kamilah Taylor: And yeah, I found that helpful, as I said, yeah, for managing tech leads and also for managing manager is really a large overlap.

Kamilah Taylor: The one other, I would say sort of difference and something like I’m still learning though, is that you do have to, there’s definitely a, when it’s someone immediately an IC on your team, right? You have a little bit more insight and overview around how the project is executing, right? And a ties loop, feedback loop, when understanding when you need to adjust how you’re operating and it’s a little bit more of a delay.

Kamilah Taylor: You get it back in layers. So I would say that’s the other thing. And you see, folks talking about that a lot. I think it’s in some ways, maybe even a little trickier with us being remote and distributed, because it takes a little bit of a while sometimes to get that signal back.

Sukrutha Bhadouria: Yeah. I mean, while we are trying to be that perfect manager, because we all know what a bad manager looks like, but it’s a little bit harder to turn into that perfect manager that we wish we had. There’s going to be missteps along the way. And Jen, from our last conversation, you had a really interesting story about how you needed to adapt, while you all changed, so tell us more about that.

Jenn Clevenger: Yeah. Yeah. I’m glad you thought that story was fun. It’s a learning story for me. And it’s kind of about growth and change. Etsy has changed a lot and grown a lot in the past few years, not just in size, the company overall, but kind of in our commitment.

Jenn Clevenger: To like, being a data driven machine learning first company and I run a data engineering team. So this is like a lot of change and growth, and fast paced movement in my org. So just for context, I’m a person that loves context. When I first started at Etsy, I was hired to manage a small two data engineers within a smaller data end construct.

Jenn Clevenger: And now fast forward six years, I run six teams with 60 people on them and with eight managers, and two directors, like me and my reporting chain. And I’m not giving you this context to be braggy, but because it’s very relevant to my story in the less and that I learned.

Jenn Clevenger: And I guess my point is like, when I first started at Etsy, we got big, but first we were small, right? And we felt really small. And I kind of grew up with this small team of people around me with two to three managers reporting to me. And we worked side by side, making decisions together, blurred reporting lines, it was a very, very flat hierarchy.

Jenn Clevenger: And my reports, even the ICs that report to me, we all felt like colleagues. I always leaned really, really hard because of that into leadership by consensus and that worked really, really well for us. And it feels good, leadership by consensus feels good. Everyone’s agreeing and kind of coming to these conclusions together.

Jenn Clevenger: And then, it felt pretty all of a sudden without any announcement or flagship moment, or I didn’t get a ribbon, or something to indicate that this was going to happen.

Jenn Clevenger: Things just stopped working so well, and I started noticing that the old practices weren’t working anymore and they’re actually like creating a ton of confusion and the gutty within this org that had grown, had kind of grown, grown.

Jenn Clevenger: So what happened? We had hired more managers, people who didn’t know me and via osmosis, people who didn’t me but also who didn’t even report directly to me there, like skip level reports.

Jenn Clevenger: And by osmosis I was kind of hearing that they’re confused by my engagement style. Like in our weekly managers meeting, people were confused by my questions, “What is Jen asking us to make a decision? Or is she like, why is she oversharing? Is she telling us what’s to do? Or all these open ended questions?” Like it was just creating a lot of confusion for people.

Jenn Clevenger: And it was confusing for people because I was still treating them like what I thought was the management style, leadership style that had worked for such a long time. It’s like, familiar old friends sitting around a living room together, troubleshooting problems for the org and then going out and fixing them altogether.

Jenn Clevenger: But what I learned was that my org had grown and changed and I knew that and that was obvious. Because you can see that change, but there were less obvious changes that had happened that I did not detect. Like more subtle and that I had to change my leadership style in order to accommodate for that, and the leadership style that I was using, that I was leaning into, it just wasn’t working anymore.

Jenn Clevenger: And it was actually causing like a negative impact for my people. So it’s not just that it stopped working, but it was creating a bad experience for people. We had outgrown this. And this was a really big change for me. And it was a change I didn’t like it.

Jenn Clevenger: I didn’t like to feel like I wasn’t part of this group of people anymore. And so what worked and what I ended up doing is I started to kind of distance myself from this. Like now large group of people, these managers, who I had thought of as my peers, my collaborators, and I had to play a different role for them and find a totally different way to lead for all of them. Not just the ones I knew and the ones I didn’t know, but for all of them in aggregate, there wasn’t any picking and choosing.

Jenn Clevenger: It was a little bit lonely, I had to find, kind of back away from these people that I felt so familiar with and find my own peers elsewhere, other directors in other orgs to seek advice from, this goes to your networking. I’m not awesome at networking.

Jenn Clevenger: And it’s an interesting lesson to learn because if I look back, it feels super obvious, like I’ve read a million articles that say that this happens. It’s not an amazing epiphany. Everyone’s like, “Wow, I can’t believe you learned that amazing thing.”

Jenn Clevenger: But it surprised me how subtle the change was. I didn’t get a memo or anything and I didn’t recognize it until it was right on top of me, kicking me in the face. So, I thought that was a really, really interesting lesson to learn that I learned on the path.

Sukrutha Bhadouria: Yeah. It’s a little unfortunate sometimes because by the time you get feedback as a manager, it feels like it’s coming in when a lot later than you would like, because it’s almost like everyone expects you to have your crap together [crosstalk]

Jenn Clevenger: Yeah, that feedback loop, right? The feedback loop is so slow. That was one of the things I noticed in transitioning from being an IC to management is there’s no satisfying feedback loop where you can finish something, run the test and be like, this is good. I did a thing today. Like the feedback loop in management would be like a year, like months, maybe even multiple years. And it’s very, very rewarding work, but that you got to have some patience for it.

Kamilah Taylor: Yeah, absolutely. Yeah, I think that my managers had said was that, like adrenaline that you get when you’re able to build something, you’re like, “Yes.” Compiles or like, “Yes, it went through.” The way you don’t get that hit, but you do get it but isn’t a different thing.

Kamilah Taylor: It’s like, you’re able to get your intern an offer, like you’re able to get someone promoted or a thing that someone was working towards, they did that and you were able to see them do that.

Kamilah Taylor: But it’s a much longer investment. And so it takes longer to get that like, “Oh yes.” Or conversely, if you were trying to coach someone towards something, also takes longer to be like, “Oh no, this is not working.” Like you got to change tactics.

Jenn Clevenger: Yes. I feel like the highs are higher, but the lows are lower.

Willie Hooykaas-Baldwin: The lonelier, that’s for sure.

Kamilah Taylor: Yeah. Yeah. Yeah. You also, you don’t have to like the same, like as I see can sort of rant to anyone and it’s just not, it’s not true anymore.

Jenn Clevenger: Yes.

Kamilah Taylor: Yeah.

Sukrutha Bhadouria: The more senior you are as a manager, you get more and more feedback about how you show up. Which sometimes feels like, “What has this got to do with anything, how much I smile?”

Willie Hooykaas-Baldwin: And how other people see you show up can be completely different than from inside, right? Like I’ve had moments where I thought I was being, because I was so angry that I was really being raging.

Willie Hooykaas-Baldwin: And I asked somebody and they were like, “Oh no, no, you were just very strict. And you let it be known that you did not agree with that.” And I’m like, “What? Really?” Or they think I’m authoritarian, and I’m like, “That’s the last thing I want to be.” That to me, it’s almost like bad thing to tell me. So yeah, it’s almost a bad thing to tell me. So yeah, it’s difficult.

Sukrutha Bhadouria: Yeah, when it feels a little personal that’s when it’s a bit heartbreaking. But you know what, through all the missteps, like you said, “There are successes.”

Sukrutha Bhadouria: So let’s switch on to the more positive side of things we, because you know, why you learn a lot from your missteps. You also learn from your successes because it’s this loop, right? Sometimes things are going great and sometimes they’re not. And how you deal with it is…

Sukrutha Bhadouria:How you emerge from it is the mark of how you’re going to end up. So over you Willie. And I want to hear from all of you, but starting with you, Willie, what was that project deliverable? What is that one thing that you achieved that made you finally feel like, “I’ve got this. I’m actually good at this?”

Willie Hooykaas-Baldwin: I realized after last time we spoke. I’ve kind of had that every single step when I have grown. And there’s these milestones that you have where suddenly, boom, you do something new. But one of the first ones was where I was asked to help on a project that was behind on the time deliverable. It had people that didn’t report into me at all. And it was very high profile. So it was around a mobile app for our user conference. And that was not easy. And I had never had it where people don’t report into me yet I am going to have to get like, “Come on, guys, let’s go! Or let do this or…” And so in the beginning I was… And then I was just like, “Okay.”

Willie Hooykaas-Baldwin: I said yes to this. So I might as well just do my thing. And my thing is just get to know the people, talk to them, make sure that they have a voice and get them to trust me. And those things kind of go in hand. You start small with like, “Hey, shall we do this?” You promise something or you show something, you make it happen. And then you just repeat and repeat. And we did it ended up being super rewarding because people that didn’t report to me suddenly gave me feedback of like, “Oh, this was super cool. And that was great. And thank you for doing so and so.”

Willie Hooykaas-Baldwin: And they didn’t have to, because it’s not like I control their salaries, so that was super meaningful to me. And yeah, we became a nice type tight group and we delivered what we had to deliver. It was hard. It was a bit of a death March, which I am not a fan of. But we did it and that was the first time once it was delivered and I was at the user conference, I was just like, “Yes, did it. Now I can take a break.”

Sukrutha Bhadouria: That’s amazing. We almost at time, Jenn or Kamilah, did you want to share a quick story?

Kamilah Taylor: I mean, at least as a manager, I think the first time where I felt like, “I got this,” was probably going through my first performance review cycle and then coming at end feeling like, “Yeah, as able advocate for the folks on my team, this went well.” So I think that was my big break like, “Okay, I think I can do this. This is a thing.” There were no surprises, I was able to argue for folks and I felt really good about that.

Sukrutha Bhadouria: Nice, how about you, Jenn?

Jenn Clevenger: I’ll say that I never really feel like I got this. But when I had to pick one thing that I felt like that was a really useful learning, that made me feel more confident if you will, is at some point in the past few years, I’ve had a huge past few years, I guess I realize as a manager or person who leads a team and then an org and then multiple teams, no one is going to do the work if you don’t do it.

Jenn Clevenger: And I don’t know, as an IC, a lot of people do things for you and they put things in place and then you follow along. And even that’s true for entry level managers. And there’s always someone above you who is setting the framework for you, to insert yourself on it gets a little bit more and more amorphous over time.

Jenn Clevenger: But at some point I realized these teams are not going to grow unless I advocate for them. No one is going to tell me like, “Oh, your teams look small. You should probably ask for more head count.” You independently have to come up… And it’s a level of creativity that I didn’t think that you could exercise, not being an IC anymore. Because engineering’s … you’re creating things and it’s so fun.

Jenn Clevenger: And as a manager, I just tell people what to do [inaudible ]. But there’s actually an element to it which is really fun and creative. Once you embrace all of the different pieces that you have to use to paint your picture, if you will.

Sukrutha Bhadouria: Yeah, oh my gosh! Absolutely. But I’m with you. I’m like that too sometimes I’m like, “Have I really got this?” And I go despite this, but with that, I’m going to wrap.

Sukrutha Bhadouria: Thank you so much, ladies, for taking time out of your absolutely busy day to, educate in smile and lead us through this. Thank you so much.

Jenn Clevenger: Thank you for having us.

Willie Hooykaas-Baldwin: Thank you, and now I have three extra people in my network.

Jenn Clevenger: I know. Winning!

Angie Chang: Thank you all.