Posted on

“Scaling Women in Tech as an AWS Educator”: Tasha Penwell, Founder and Educator at Bytes and Bits (Video + Transcript)

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Angie Chang: Welcoming Tasha Penwell, who is an educator and founder at Bytes and Bits. Did I get that right this time? Yes.

Tasha Penwell: <Laugh>

Angie Chang: She is passionate about scaling women in tech as an AWS educator, and will be sharing her insights for teaching training and certification for AWS Academy and at AWS re/Start. So, welcome, Tasha.

Tasha Penwell: Thank you. Thank you everybody for joining me, and thank you for having me. We are obviously on a tight schedule, so I’m just gonna go ahead and get started. My name is Tasha Penwell and the founder of Bites and Bits located out here in southeastern Ohio. A little bit about me. I’m an AWS educator, certified AWS solutions architect. I’m also an AWS community builder specializing in security cloud security. Google has a program called Women Tech Maker. I’m part of that and all of the thing, all the resources that you’re gonna be seeing today is available through this link. AWS is awesome because it is, and there’s also that QR code that you’re more than welcome to grab.

Tasha Penwell: And I will be taking any kind of questions that you have at the last few minutes of our time together. Please feel free to put ’em in the chat and I’ll be reviewing them as we go along or after we finish. Why AWS? AWS is, first of all, AWS stands for Amazon Web Services, and it is one of the most in-demand skills that’s needed in technology today. There’s a variety of different certifications, no matter the path or interests of computing that you may have.

Tasha Penwell: Maybe you’re interested in security like I am, or you have an interest in data analytics, gaming, IOT, high performance computing, such as AI, machine learning, and there’s a variety of different industries. And I always tell my learners that to find something that you’re the most interested in for me it’s security and just see how you can utilize that with AWS and you’re gonna have a, it is gonna help you build a really awesome career that is something you enjoy and would be help you live the life that you want to live.

Tasha Penwell: Whenever I have a new class, one of the first things I ask my learners, I is like, who’s a lifelong learner? And I told them, if they don’t raise their hands, they’re in the wrong class. Because working in technology or specifically in AWS, you need to have that mentality of being a lifelong learner. And that means trials and errors. That means some may be some failures along the way. And you have to have that mentality that every failure is going to lead you to a learning opportunity. And those, you know, those will stack up on top of each other and will give you a wonderful, wonderful experience being you know, continuing to inspire some curiosity in your growth in a really fun and challenging field.

Tasha Penwell: The three takeaways that I would like everybody to have from our time together today is to be able to identify two training and library sources that are available to anyone. And I’m going to share my personal methods for guided notes and note taking. These are things that I do whenever I’m teaching a class or whenever I’m, you know, studying for my own certifications or just my own, you know, continued knowledge and also explore some different paths and resources for learning, working towards certification. Certification is obviously one goal that you have, you know, earning their cloud practitioner certification, your solutions architect or maybe even a specialty certification. That’s one goal that you may have, but there’s other incremental goals that you can have that can continue to propel you and inspire you. And I’ll give you the, the proper motivation to kind of get over the hard, because it does get hard sometimes.

training and lab resources aws skillbuilder aws educate

Tasha Penwell: The first thing we’re gonna talk about is training and lab resources. AWS Skill Builder and AWS Educate are two wonderful resources that are available free. At Skill Builder, there’s a free platform in the late last year, they also launched another kinda like a premium version of Skill Builders like $29 a month. But the free is still awesome, and that’s actually what I still use, and this is also what I encourage my students to use as well to supplement. There’s a variety of subjects and you don’t need to sign up for an AWS account. All you need is your amazon.com credentials. This is, there’s an assumption being made that everybody has purchased something from Amazon before. And whatever your login credentials were for that, that is the credentials that you need to use to log in to Skill Builder to create an account with Skill Builder.

Tasha Penwell: Now, because of the time that we have in our class today, I offer you, whenever you see this little link with Highlight, it says video that is actually going to take you to a platform I like to use called Loom, and it, you can play the video. These are short videos that will go into more details of how to use these platforms that just don’t have time to cover in our 20 minutes together today, and that’s available for anybody to access. One of the things that I really like about this platform for anybody who’s not familiar with it, is like you can hit play mm-hmm. <Affirmative>, and I’m just gonna go ahead and mute that. And if you get to a certain point where you have a question or you’re not understanding something in the instructions or maybe you don’t see something, there’s a little button here where you can comment and you can say, I don’t see that. Something along those lines. And I can, I will actually get a notification in the timestamp of when that, whenever you have that question, and I can respond to you very quickly. So even after the day’s over, or, you know, later on this weekend, maybe next week, next month, next year, if you have troubles on anything, feel free to just comment on there and I will get that notification and I’ll be able to respond to you directly. So with throughout these slides you’ll see those type of videos embedded in there.

Tasha Penwell: AWS Educate is also another great resource. And let’s see. So AWS Educate, it actually started for it was only intended for high school and college students. I was originally, I was one of the original AWS Educate Cloud ambassadors back in 2019, I think it was whenever it was coming around. And it has a lot of great training and labs that you can do on a variety of subjects. Again, it’s originally targeted at high school and college students but it’s now available to anybody. And the only, there are some kind of hiccups with the registration process, and I kind of go over that in the video link here. It just kind of tells you how to get over the hiccups because it used to be re pretty strict on who could access it. Now it’s open for everybody, but some of the signed up processes are kind of still a little, there’s still some hiccups in there.

Tasha Penwell: Okay. So there’s, you know, again, some videos, if we have time at the end of our class, we’ll go over one of these. But like I said, class at the time of our session, we’ll go over them, but I just wanna be mindful of our time. Te next thing that we’re gonna talk about is guided notes and note taking. Guided notes is something that I learned whenever I was working in higher ed. I worked in higher ed for about eight years. And guided notes are basically, they are prepared handouts that I provide to my learners. And the purpose of the guided notice is to give them some structure to their note-taking. I don’t know if anybody’s heard of the of the phrase, you know, trying to drink from a fire hose and, but basically if you’re brand new to technology, if you’re brand new to computing, if you’re brand new to AWS, there’s a lot of content just being, you know, coming at you.

Tasha Penwell: There’s a lot of variables, there’s a lot of things you gotta take in considerations and trying to make the right choice, trying to be cost optimized, think, you know, thinking about high availability, all these things that come into play whenever you’re trying to make a decision, and it can be overwhelming. The purpose of the guided notes is to try to give the, the the learners an opportunity to focus on terminology and scaffolding, scaffolding what they’re learning as an iterative process. And in the future, slides you’ll see that there’s a link that gives you an idea of what that looks like, but the feedback from the students ultimately has been very positive. They have admitted that they have poor note-taking skills and poor note-taking habits, and they said that this has helped them give them some structure to the note-taking as opposed to just trying to do it on their own.

Tasha Penwell: I’ll go back to the note-taking section here in a little bit, but I just wanna kind of follow up with the unguided notes. The guided notes, the learner, like I said, the, they have the option to use this. It’s not a requirement, it’s just meant to be a tool to help them with their studies. If they know they’re, they already have a learning style that works well for them, they’re more than welcome to use it. It’s just, that’s another tool for their toolbox if they need some assistance. After the guided notes, we review and discuss, one of the things that I tell my students, first thing is, I hate reading from slides. I’m not gonna read from slides. You guys know how to read, I am not gonna read to you. You guys are expected to review the content.

Tasha Penwell: And then we come in for discussion. We have a review and a discussion, and we also do some knowledge checks. It can be in a game format such as using Cahoot or just do the knowledge check that’s provided by AWS as part of their curriculum. In here, there’s another video link that explains it in a little bit more detail, but in here is just like a really simple sample copy of what guided notes look like. There says there’s a link to the skill builder, which again is available to anybody with an amazon.com account. And there’s three sections to it. The first section is focusing on terminology. All they need to worry about is just focusing on understanding the terms. And then their second section is a short answer. And again, they’re going through the content and going through and, you know, filling in the blanks, kind of like a scavenger hunt. And then the third section is the loan answer.

Tasha Penwell: The expectation is that the students, by the time they’ve gone through the first and second section, that for the loan answer that they, you know, or should be able to answer these without some students who actually take this as a way to quiz themselves. If they’re able to answer the questions without looking it up, then they know they, they have a good grasp on it. If they’re struggling with it, then that gives them a, a barometer way to check to see that that gives them something to look into so that they know that this is something they need to focus on to improve their skills or to improve their studies.

Tasha Penwell: Let’s see, and I’m gonna go back to this. Note-taking, I talk about learning styles a lot. Note-taking is more for like the self-learner. I was, I’m completely self-taught on AWS and I struggled actually to find my own learning style with AWS when learning AWS. Whenever I was in school, whenever I was in high school and college I didn’t really develop good study habits. I’m one of those people that I was talking about have not developed good study habits, and I had to, you know, improve on my study habits because a w s was just, you know, it was challenging to me and that’s why I still enjoy it. It still challenges me, but I, I had to find something that worked well for me and for what I’ve learned is note taking, like improved a lot more on the note taking.

Tasha Penwell: Sometimes I’ll do something more of creative, like using Canva or Figma to create some sort of infographic or even look a little book with characters. It just, it has a creative outlet when I feel like I need to be creative, like I cannot do anymore. Analytical. and the also listening to podcast I’ll give and the, and the slides. There’s some podcast links with aAWS and some other resources. I also encourage my students to make your own flashcards. I encourage you to make them, instead of buying them from Amazon, you can purchase them from Amazon, but with the act of writing out and creating your own flashcards, using the index cards you can get from a dollar store, that is one way that you can, you know, that’s a study that you are studying whenever you’re having to read and write and you put it on the flashcard.

Tasha Penwell: Then whenever you’re reviewing them on your own homemade flashcards, you’re again, you’re you’re improving your knowledge, you’re, you’re re improving your understanding in retention of the information you’re trying to learn. And this again, a video. Well this is one of the feedback that I’ve gotten from my learners about guided notes. And as you can see, she talks about, you know, guided notes have really helped her a lot. And she says that she would quickly lose patience reading the text and will lose track of what she was reading. And I’m the same way, like if I’m trying to read something, I have to make a conscientious effort to ignore, ignore any squirrels or shiny objects that’s coming in my way because I will get distracted. And she likes to fill in the blanks and it shows that the important point to note down and remember.

Tasha Penwell: It gives them, you know, some guidance, something very pointed to this is what they need to focus on, ignore and remove all the distractors and then showcasing the accomplishments. So AWS Educate again was one of those resources that I introduced earlier and one year you are I have my students actually doing this right now with their AWS Academy class. They have courses that you can take and you can earn a badge. So those badges is after, you know, storage or this, these two right here is compute and storage and you can, you know, finish a course in there and you’ll earn that digital badge that is a digital badge that you can share it on LinkedIn and other platforms. It’s a small accomplishment towards your larger goal, which is the certification. So if you go and AWS Educate, these are the different badges that you can do.

Tasha Penwell: And again, there’s a video on it with Skill Builder. You also get certificates of completion after you complete a course you’ll get a certificate of completion again. And that’s something that you can share on LinkedIn. When I worked in higher ed, whenever I completed it, I would actually send it to HR and my dean, who is my supervisor’s, like, Hey, I am doing this. Put it in my portfolio. So whenever there’s an opportunity, remember that I’m doing this. I go, I’m trying to, you know, make, be mindful for time. There’s also Skill Builder learning badges and these are just, just similar to the Educate badges. They’re just a little bit different and they are a lot more on the time. You can see like 62 hours, 11 hours, nine hours, et cetera. But these are also badges that you can learn again and shared on platforms such as LinkedIn.

Tasha Penwell: And there’s a video on that. Building a digital presence using LinkedIn this gives you, so if you’re struggling on how to fill, you know, build your LinkedIn profile, there is some tips on how you can do that if you are an AWS Academy, you know, learner or if you’re in some sort of similar program finding your path. So AWS has some resources, you know, if you’re looking, if you know what kind of role you’re wanting to be in cloud practitioner, developer DevOps, or if you have a particular type of specialty, like I said earlier, minus security, and you can go through here and and I know I’m going fast, but I’m just trying to let and be mindful of the time. They have a ramp up guide. And these ramp up guides can give you some specific pathway of specific resources with the links and the types in the amount of time.

Tasha Penwell: And it gives you some, because it’s a lot of different things if you just try to go into AWS and try to, you know, figure out what you need to know. This gives you a specific pathway based off whatever interest you want to have to your career in and again, minus security. This could give you some guidance on specific resources that you know to help build that career for you to meet your career goals. And let’s see a video on that. White papers, you know, saying awesome with AW s white papers, blogs, podcast. Victoria Seaman, she is she’s an advocate. She’s awesome. If you don’t follow her on LinkedIn yeah, she is just constantly sharing great resources, so I highly recommend checking her out and following her on LinkedIn. And then these are my how you can find me, <laugh>. Thank you.

Angie Chang: We think we’re at time, but thank you so much. I think this was very, very helpful and educational and a great resource. Thank you again for your time and we’re gonna hop to our next and last session of Elevate. Thank you.

Tasha Penwell: Thank you.

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Posted on

“Building and Managing a UX Practice”: Marilyn Hollinger, Senior Director, User Experience at Betterworks (Video + Transcript)

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Sukrutha Bhadouria: Marilyn is the Senior Director of User Experience at BetterWorks. Her passion is designing and building experiences that delight users and maximizes productivity. She has worked at Mark Logic, Guidewire, Genentech, Intuit, Oracle, and Xerox. We are excited to hear her UX insights. Welcome.

Marilyn Hollinger: Thank you. Thank you so much. I’m very glad to see there are still people trickling in. So I’ll just but I have 20 minutes and those of you have been attending sessions. You know, that 20 minutes is <laugh> very short. I’m gonna go through quite a number of slides. I will talk fast. I’m from New York. I can talk very fast. And at the end there will hopefully be time for questions and if not, my email address will be on the last slide and you’ll be, I’m happy to take questions by email. Feel free to do that.

Marilyn Hollinger: I’m just gonna jump right into this background for me, I started as a developer and doing front end work, working very closely with UX people. Ended up getting doing transition into doing UX full-time many years ago and have been leading UX teams for long, long, long time.

Marilyn Hollinger: I’ve done this many times at different companies and I’ve kind of worked out what I think works best. Let’s talk about how UX teams evolve in a company. Generally the first thing that needs to happen is there needs to be an appetite for, for a team to exist. There are many, many companies out there, including the one I’m at right now, which start, start off with nobody do having the role of, of UX that usually product managers and developers kind of partner on this. If there is a product manager, sometimes there’s just a developer who kind of knows the front end, so really it’s about having people understand what it means to have a UX team. And we’re gonna go into detail on this in a minute. And then showing the value of such a team. Okay, now I understand it, but why do I, why is it important for me to spend headcount and time?

Marilyn Hollinger: Why is it important for me to invest? Cuz value is about your investment and what you get out of your investment. And then working on how a u so now you get to a point where a team can exist. How does that team collaborate with the rest of the organization? And then finally, how can the UX team become strategic partners in the organization? And my background is in companies that deliver software. And so I’m gonna really be talking about this in a software development environment, not so much in a marketing or anything like that, but very much in a software development environment. Although a lot of what I’m gonna talk about is very applicable to different environments. Let’s talk about this.

Marilyn Hollinger: The Nielsen Norman Group highly recommend this website if you’re doing anything in UX has this concept of stages of UX maturity. If we look at the evolution of how you build a UX, it really tracks to this, to these stages of maturity. You have to build the appetite. You generally, you’re building the appetite when you’re in these very early stages where there isn’t a UX team where it’s just people have no idea what UX is. And if you’re very lucky at this point, you might be able to get one person who would be a generalist. And I’ll talk a little bit more about that in just a minute. And really this is about the appetite can come from uncovering negative user experiences, so lemme talk about that. A lot of the appetite comes from people saying, well, they don’t even understand what it means to do. Ux, UX is often seen and I’ll talk about this in a minute as pretty pictures.

what is ux

Marilyn Hollinger: You need to figure, make sure they know what UX is, understand the benefits of it and overcoming some fear. I wanna get into this a little bit. UX is usually seen as the, the pictures on the screens. Can you design me a dialogue box? Can you make a pretty image for this? Those are really the two things that most people think UX is, and I use UX as user experience as opposed to user interface design or anything like that, because I consider UX is a very all-encompassing profession where you have interaction design and that’s like what controls do you put on the screen? In what order do people click on things? And that’s interaction design. And then you have to make it look nice and that’s the visual design. And behind all that, you really have to do research to build software that is to build designs that match user expectations that match the environments they work in, that answer the questions they need, answers that get the jobs done, they need done, not just, this sounds like a good idea.

Marilyn Hollinger: Let me give you an example. I was doing a, a UX design for people who determine insurance rates. I know that sounds really dry, but it was great, great gig at Guidewire. And in fact, I got a a patent for this. And what we did was we talked to people who do that which are actuaries and actuaries. It turns out live and breathe Excel. So we and had we not talked to them and done the research, we wouldn’t have known that. So the interface that we designed, we made it as close to excel as we could, and we allowed important export to and from Excel so that actuaries could work in their native environment and then bring that into the tool at Guidewire, so really important to understand what people, where people live, and then what words do they use when you talk, when you talk to actuaries, they use various terms and we had to make sure that we knew what terms to use so that our design could, could parallel that how they think.

Marilyn Hollinger: The information architecture, so how we lay things out and what hierarchy could match what they were doing. And this is super important to understand all of these different aspects of UX and be able to explain that to people if you’re evangelizing. So the other thing is what is why is why should we do this? What’s the value? Okay, so there’s a couple of values. One is if you design things better, the company does better. And this is super important. This is you know, these statistics are a little old at this point, but it really shows that design centric companies do better because their products do better. And there’s, there’s a lot of literature out there about this and there’s a cost involved. So there’s be cost and benefits and that’s what value’s about. And there’s many, many iterations of this particular picture out there in the, in the universe.

Marilyn Hollinger: This is kind of a short one where, you know, customers can explain what they want and it’s often not what they end up with because we don’t actually do the research to watch them work. Well, I remember going into, again, at Guidewire going into insurance offices and listening in on people’s phone calls and watching how they worked to see what it was that they really needed. And or else you’ll end up with who knows what, because, you know, marketing people and salespeople will come up with a thousand different features and really the customer needs three of them. There’s cost of, if you don’t do that, user analysis and design. And this is where we really want to explain that, to help build that appetite to show, you know, if you don’t build what users want, your your first version is not gonna sell and you’re, and you may be dead at that point, your company may be dead.

Marilyn Hollinger: Here we’ve got a quote because there’s a fear involved. There’s a fear that, oh, if we, if we put this extra thing in our process, we won’t get our software out as in as much time. Well, if you think about the software engineering process in good software engineering, you do, you do architectural design upfront, you do your implementation, and then you do code reviews, right? And you wouldn’t think of skipping and then you do QA and you wouldn’t think of skipping any of those parts. It doesn’t make sense to skip any of that because your quality will go down. And it’s exactly the same argument with user experience. If you don’t take the time to design it right, then your development will have to be redone and you have to redo that whole cycle. When you talk about in the software engineering process doing the architecture right upfront saves you in the long run go back one step, even doing the user experience, right?

Marilyn Hollinger: Saves you in the architecture, which saves you in the development cost. I’s understanding that that’s doing making sure that you do it right or else you’ll, you know, it doesn’t matter how fast your product gets out there, it won’t succeed. Once you’ve got some kind of appetite we’ve also, we’ve talked a little bit about the value, but what, what usually I’ve seen is at the very beginning when you’re building the appetite, you’re, you’re pointing out bad user experiences and say, you know, we could have done this better. And then you say, and this is how it is better. Now you start looking at touching those user experiences to improve them and say, this is what we have. We’ve gotten a lot of complaints about it. What here is a redesign? And you see how much better it could be, and the little light bulbs start going on for people.

Marilyn Hollinger: Okay? And, and this is where you point out, look, we have one function that works away this way on this screen. And look, that same function works differently on another screen. It’s not consistent. Look, we can have one design and users will be able to use them better because they’ll, they’ll understand how to use that same function in both places. And this is where your team might grow to just be kind of generalist still, but having strengths in those three areas. Like one might be a slightly better visual designer, one might be an interaction designer and one might be a researcher, for example. But they all have to be able to cross Polly. When I joined Guidewire, I was the only UX person there and I, and I am in those three areas. My weakness is in visual design. I can do it, but I’m kind of the B team.

Marilyn Hollinger: When I got hired on there, I said, you know, I’m gonna have to spend some money to hire a visual designer to assist me when it comes to visual design issues. And they were fine with that so they understood what they were getting there. And then my next hire was very strong in visual design, although she could do interaction design and research. You build up the team with these different strengths and also you show over and over again, look how we can improve existing user experiences or you build new experiences that are people look at and go, oh, that’s great. And that it, it’s it feeds itself. As you design better experiences, you get more support, and you can build better experiences when you’ve got a team in place. That’s the time where you really have to think about structure.

Marilyn Hollinger: I’m gonna talk about that in just a minute. How you actually structure your team. This is where you really need to have processes where in place to make sure that you are part of the team collaboration. What I mean by that is that there is and we’re gonna talk about this in just a minute. There’s early on ideation, then there should be design and then architecture, and then development, and then QA. So there, it’s a linear process. I mean, it repeats itself in cycles and there’s, there’s agile work and things like that. But it’s basically that same, that same process and figuring out where UX fits in with each of those processes is very important. And that’s what allows you to design truly outstanding user interfaces. And at this point in a team, you start to have more specialists, start to have people who are like the UX research lead, okay?

Marilyn Hollinger: Or the visual design lead who owns your style guide, for example, okay? And then when the, the organization truly has maturity in terms of UX, you start to be a strategic partner, you start actually saying, here’s, here’s a whole initiative that’s going to drive the product. Here’s a whole area of the product that we’re not doing, but it will make the overall experience better. And this is where people wouldn’t even dream. Actually, hopefully in the, in the step four people wouldn’t even dream of not having the UX team. It’s very, very important to to sort of strive for this, where you end up having an architect who oversees everything. You have specialists and you’re really a strategic partner in the organization. And in organizations that are mature that I’ve worked in, I I sit, I help drive product content because that’s part of the strategy.

Marilyn Hollinger: User experience needs to be a core part of the strategy. Let’s dive into this a little bit more in terms of process. I mentioned this a little while ago. There’s, there’s the product visioning. What are we gonna build? What should we be building? Not just, I mean, it could be very broad. What product are we building? Or it could be what features do we need to add or what new areas or new ways of thinking? And then there’s the design before build process, which is before anybody starts coding anything except for proof of concept or anything, you do design work, you don’t finish it. But I always think about it as, as building a house, you would never bring a carpenter in until the architect is done. And it doesn’t need to be quite as baked as that. Because there is, there is agile processing that you can do.

Marilyn Hollinger: You can, but you have to have a pretty good idea of what the house is gonna look like. <Laugh> before you start, you know, hammering and nailing. And then during the build process, you need to have design come in because there are gonna be changes, there are gonna be misinterpretations of the design, there are gonna be use cases we didn’t think of. So there’s a partnership that happens there. And then there’s an acceptance time when you say, yes, this is the user experience I designed, you have built it the way I, I spec it, or we discussed it and let’s ship it. Let’s go through each of these. So in product visioning, this is really, it’s generally, now I’m gonna not say how it should be necessarily, but I’m saying how it generally works. Well, and this can be different at your company, there can be other people involved in this, but ascent, but how I’ve seen it be really successful is you have an owner for the product vision, generally product management and ux.

Marilyn Hollinger: And you do this together. And there are other people involved in this. Usually you have customer people, you have sales, you have marketing bring giving input, but the key people who decide where are we going with the product, 10, if you have a partnership between product management and UX, that works really well. And this is where you say, what are these cases? What problems are we trying to solve? What are the requirements there? And really develop that and, and spec it. Write it down because we all speak different languages, not just, not just actual com human communication languages, but I look at things from a user perspective and an artistic perspective, and a product manager looks at it from the functionality perspective and a customer perspective. And I do too look at it from a customer perspective, but we don’t often speak the same language.

Marilyn Hollinger: It’s important to write it down. You can’t, and and this is where like taking minutes and meetings where you have agreements, this good meeting management underlying all of this. And there’s tools that can help you. Often you’ll have there’s tools like aha, I’ve got some tools listed along the bottom of some of these slides. Putting together road maps to say what we’re gonna do over different quarters. For example, writing requirement specs. And by the way, good requirement specs don’t say we need a button to blah, blah, blah, blah, blah. It should say we need to do blah, blah, blah, blah, blah. So requirements are not about design. They’re not about what they’re about, the what we need, not the how we’re gonna do it. And there’s, there’s whole classes on doing requirement specs. I’m not gonna go into that. This is this type stage that you have, you haven’t already.

Marilyn Hollinger: You do the research, you go in and you talk to people and you do the things we talked about before. How do they speak? What do they need? What’s missing in our current product? And it’s really, really important for the UX team to get firsthand exposure to those customers, to be able to have a conversation with them, not have it filtered through a lens of, of customer people or product people or whatever. But to actually hear those people talk and be able to ask them questions, very, very important at the product visioning stage. And also that comes in later at the design stage. I can’t design for someone that I haven’t actually had a conversation with. It just doesn’t work very well. That’s product vision. In this decision making, often you when you decide about what you’re gonna put in the product, it comes from a lot of sources, it comes from the competition, it comes from, you know, there’s sort of just boxes I need to check off to make sure I can even get in the, in the game.

Marilyn Hollinger: We need to fix bugs. We, and then there’s this concept of, oh, this would be cool if we did blah. And all the use cases, all the things that people could think of about doing with the product tend to be created equal when you don’t have that, that UX perspective. Oka? Especially when it comes to engineers. Engineers can think of the 35 different use cases and, and they wanna build all of them because they tend to, and I’m, I’m doing a vast generalization here. I know, no offense to anybody, I used to be an engineer, but really it’s like, well, here’s all these different options. I, I need to code all of them. But what we really wanna get to is keeping track of that competitive advances, advanced, excuse me, advantages, making sure we are checking off the boxes, making sure we are fixing the bugs.

Marilyn Hollinger: But what UX brings to the table is what problems are we solving? What, how can we streamline the, the most important use cases to make those easy, the happy path. How can we keep different parts of the product consistent? And making sure that UX bugs are fixed too, not just functional bugs. Okay, so this is where we wanna make decisions about what goes in each release. Bringing both lenses to the table, take a breath, moving on. Design before build, this is about, then now it’s the UX team. The UX team. In the earlier stage, the product management team generally drives, in this stage, UX generally drives and it’s a partnership. Okay? It’s really important to have a design library to say, okay, all of our buttons are gonna look like this and all of our searches are gonna work like this.

Marilyn Hollinger: And all of our, our wizard interactions are gonna be like this so that you get consistency and you align out with development. You say, okay, you guys build the button once and everybody uses that same button, and we’re gonna talk about that in just a minute. It’s important to track this work to make sure it’s really, really common for UX person to be working on something and have a product manager go, I need you to add this other feature to this other thing and mock that up. <Laugh>, okay? And so they get sidetracked. UX tends to be very responsive in general. And so of course we say yes, right? <Laugh>, but it’s really important to track that. So when you get dis behind in your, in your design on this thing, because this other thing came on, it’s really important to track that and be able to say, Hey, this, and, and to ask for prioritization, okay, you want me to add this new feature?

Marilyn Hollinger: I can add that new feature, but this other feature that I’m working on is gonna slip. And what I often see happening with UX teams is that things just get, keep getting piled on. And so it’s really important to be able to say, you know, Susie is already working on these four UX tickets, which one of these is lower priority than this new thing you’re asking about. Is it more important and to get actual sign off to say we are done? We do that in, I’ve done this in several organizations where we do this by having UX tickets for all the work. We often using Jira all the work with links to the actual designs and something like Figma Sketch or InVision, okay? And having phases of it’s in progress, it’s being reviewed, I’m awaiting input. Because often you need like a developer to say, is this even implementable, for example, or is this a key feature?

Marilyn Hollinger: Or you need something about terminology from a customer. There’s a waiting input is another is another status. And then design complete. And that’s when everybody’s given a thumbs up. And so there’s a very visible way to say, I’m done with this, I’m putting this aside and moving on to something else. And then anyone who can look at the, who needs to look at the design can go to the design complete ticket look, which is now linked to the actual design and, and see everything. Very, very important that anybody outside the UX team needs a way to find all of your designs in a very structured manner and be able to see what state they’re in. I highly recommend Jira or a similar type of tool for this. The designs can be in whatever you want to use. I’ve, I’ve named Figma and Sketch and InVision.

Marilyn Hollinger: You can use whatever tool you want as long as there’s a way for people to get into the design, see them and comment on them. That’s really important. There needs to be ongoing commentary about your designs, preferably in the designs themselves. Figma, Sketch and InVision, they all allow that. And if you’re using a different tool, having that functionality is important. Making sure that they’re that you look at competition make. And there’s this concept of sprint zero.

Marilyn Hollinger: There’s a lot of folks who do agile who don’t wanna do waterfall, don’t do this in any particular order, but I’m sorry, you need to do design before you do development. And maybe that’s three sprint zeros depending on the side length of your sprints and the amount of design work to do. You have to be planning. I like to tell people you have to plan at least a quarter in advance to do your product and UX work before you start your development sprints, at least, if not more, depending on the size of what you’re working on.

Marilyn Hollinger: Okay, moving on. During the build process, again, now we went from product management, sort of owning the, the planning process to UX owning the design process. Dev owns the, the build process. They’re actually putting their fingers on the keyboard. And this is where we really highly recommend to get good UX, you have common base set of components that the development team built uses. Everyone’s using the same buttons, everyone’s using the same search search functionality, so UX is not designing custom buttons or custom dialogues or custom anything for standard components. So I really like to track the componentry with the design, with the development component library. And now you have to have dev tickets, right? And we’re, a lot of us are familiar with this. Often you use Jira for this, but what you need is a designer for every dev ticket that has UX involved.

Marilyn Hollinger: Obviously backend tickets, you don’t have to, but you need a designer who’s watching that ticket and tracking it. And it, there’s now a conversation, there’s now this teamwork where the designers attend the standups. There’s active conversations between the developer and the designer. If there’s any questions on the design that you, I’ve, I’ve seen organizations where the developer goes to the product manager who goes to the designer, who goes to the product manager, goes to the developer, and you end up with this great game of telephone. And, and that’s just not okay. You need to have this be a team environment. And often you’re working on agile sprints, and that’s perfectly fine. There’s nothing that that gets in the way in terms of UX around that, but UX needs to be a part of that conversation during the build process. Okay? And finally, there’s an acceptance criteria.

Marilyn Hollinger: And what I mean by this is a built-in process that tickets get stories, get built by developers, and then when they say they’re done, it’s an automated process for UX to, to jump in and do a review. What I’ve got on the bottom here is this is a suggested flow for Jira tickets where you create the ticket, you set UX required, yes or no, okay? And then the build happen after design <laugh>, the build happens again, conversation’s going on. And when the build, when the developer says, yep, I think I’m finished, then it, if there’s no UX required, it just goes into QA or whatever your process is. If there’s a UX required, then there’s a UX review. And I’ll talk about the SLA in just a minute. The UX review, the UX person has to have, so the, by the way, the classification, that’s the UX required, the UX person needs to have the authority to say no, to say no, it’s not done, and be able to send that back to build.

Marilyn Hollinger: In the current organization I’m in, the UX member gets a notification when it goes into, we have a, a design acceptance phase for the ticket, and they get to do one of two things with it. Well, first of all, they can have a conversation, ask questions, they have to be able to review the, the design, and then they have the authority to send it back to impart if they know you’re not done and they add a comment about why, and they, then they have the authority to send it on to QA.

Marilyn Hollinger: It’s really important that the UX person has that authority and that they have an environment that they can do a real time review of, of the, the whatever it is that they’re reviewing. And that can happen in multiple ways. That can be very, very fast. Sometimes we just jump on a quick zoom call, the developer shows, the, the designer designer asks questions. They’re done in five minutes. Sometimes it’s a, we need a whole test environment. And so we’ll do that and have a whole test environment. So, but you need to have some way to do that. I’m sorry, we’re at five minutes. Yeah,

Sukrutha Bhadouria: We’re, yeah, we’re five minutes fast.

Marilyn Hollinger: I’m good.

Sukrutha Bhadouria: Shall we wrap?

Marilyn Hollinger: I need, I think we have, I have four more minutes, don’t I? I’m almost done.

Sukrutha Bhadouria: Okay. Right.

Marilyn Hollinger: I’ll be done shortly.

Sukrutha Bhadouria: Okay.

Marilyn Hollinger: I will be done shortly. I thought I had to 1:50.

Sukrutha Bhadouria: We’re we’re past. Yeah, we’re past time. We’re overtime right now.

Marilyn Hollinger: I thought it was 50. It was 20 to 50. No, it’s 20 minutes. Okay. I’m gonna, there’s my summary, there’s my email address. We’re done. I’m sorry, I, I got my timing off. You’re right, it’s 20.

Sukrutha Bhadouria: Minutes. No worries at all.

Marilyn Hollinger: Very fast. No questions. I apologize. Please send me email if you have any questions. And, and thank you so much for, for your attention and your attendance. Sorry.

Sukrutha Bhadouria: About that. No, thank you, Marilyn. Please do reach out to Marilyn via her emails, take a quick screenshot and we’re going to move on to the next session. Thank you, Marilyn.

Marilyn Hollinger: Thank you. Apologies.

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Posted on

“Consider a Career in Accessibility”: Sheri Byrne-Haber, Senior Staff, Accessibility Architect at VMware (Video + Transcript)

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Angie Chang: I have with us today Sheri Byrne-Haber, who’s a Senior Staff, Accessibility Architect at VMware. She’s a prominent global subject matter expert in the field of disability and accessibility, and known for launching digital accessibility programs at McDonald’s, Albertsons and VMware. And she writes a popular blog called “This Week in Accessibility”. Welcome, Sheri.

Sheri Byrne-Haber: Well, thank you so much, Angie. I’m really excited to be here. I always like to drop the Sheri’s secret fun fact before I start events like this, which I was the first Girl Scout in the US to get a badge in Computer Science coming up on my 45th anniversary of that event this August. I’ve been doing tech for a long time.

Sheri Byrne-Haber: I did my first degree at Cal in computer science back when it was, you know, 90% guys, and I was basically the diversity in the room. Been doing this for a long time. Went and got a law degree 10 years after my computer science degree, then did an MBA 10 years after that. I’m here today to talk to you about why access. Yeah, Go Bears, Angie <laugh>. Why accessibility, what it is.. Okay, so the the brief 50,000 foot version and why it is a great career, especially for women.

Sheri Byrne-Haber: When most people think about accessibility, if they’ve heard that word, accessibility means making stuff work for people with disabilities, that’s kind of the TLDR version. They think about visible disability. You might think about somebody with a prosthetic arm.

Screenshot at .. PM

Sheri Byrne-Haber: This is actually me practicing in my wheelchair on my Olympic range at home. I’m trying to qualify for the 2024 Paralympic games. People with service animals. People with hearing aids. Something that you can see. Accessibility has to take care of a lot more things than that.

Sheri Byrne-Haber: First of all, we have to deal with hidden disabilities, disabilities that aren’t obvious, that can’t be seen. That might be, I tell people all the time, you see me in a wheelchair, you assume, you know what my disability is, right?

Sheri Byrne-Haber: My real disability is type one diabetes kicks my ass on a daily basis. It interferes with everything I do. My wheelchair is just a way to get around. And I’ve been doing it for a very long time.

Sheri Byrne-Haber: You need to think about hidden disabilities. And some examples of hidden disabilities include Millie Bobby Brown, who’s deaf in one ear. Bono wears tinted sunglasses because he has a glaucoma. It’s not a rockstar affectation.

Sheri Byrne-Haber: Neurodiverse statuses. Mental health issues. The reason why all of the colors in Facebook are blue is that it’s the only color that Mark Zuckerberg sees. When you’re thinking about disability for starters, you really have to broaden the definition to make sure that you’re including both visible disabilities and invisible disabilities.

Sheri Byrne-Haber: Then you need to add two different types of disability. A permanent disability might be a limb difference, but if somebody tears their rotator cuff temporarily, they’re gonna have the same disability as somebody with a limb difference. They’re not gonna be able to use their arm or situationally you might be holding something that prevents you from using an arm.

Sheri Byrne-Haber: When you take permanent plus, temporary plus situational disabilities and, and look at it from both the visible and the invisible perspective, you’re talking about 30% of your potential users. And accessibility is about making technology work for that 30% of users.

Sheri Byrne-Haber: Okay, so what do accessibility testers need to learn? First of all accessibility testing is a lot about interacting with assistive technology. You may have heard from other people talking about software testing as a field that automating is the greatest thing ever because then you can just push a button and repeat all those tests and not have to do anything that requires manual intensive interaction. It’s not so easy to do with accessibility because only about 30% of the tests can be executed in an automated manner by inspecting the code.

Sheri Byrne-Haber: 70% actually require being able to interact with the assistive technology. And so that includes things like screen readers which is what the woman in the middle graphic is using. She’s listening to her iPhone, tell her what’s on the screen in front of her that she can’t see. Some other forms of assistive technology are not using a mouse. Using alternative input devices like keyboards touch pads you know, those graphics pens things of that nature captions, magnification.

Sheri Byrne-Haber: Then we get into a little bit more obscure, slightly less used assistive technology that would include things like sip and puff devices, which is how people who are quadriplegic interact with the internet. Obviously speech recognition is becoming more and more popular and, and actually better and cheaper than it used to be in the past. Once you know how to use assistive technology, you have to learn about the accessibility guidelines.

Sheri Byrne-Haber: There’s something called WCAG, which stands for Web Content Accessibility Guidelines. The version that’s just about to come out is version 2.2. And that is a standard WCAG that has been adopted pretty much globally. Anywhere that you have a law that requires inclusion of people with disabilities, usually it references one of the WCAG versions, not always the same version. That’s would make things too easy, right?

Sheri Byrne-Haber: The EU, Canada, Australia, the us India, some countries in Africa, they all use WCAG as the standard to determine whether or not you’ve made something accessible enough. That is, that, you know, the majority of people with disabilities would be able to use it just as if they didn’t have a disability. These are the two basic things that entry level accessibility testers focus on.

design build test all thru lens of accessibility

Sheri Byrne-Haber: What do they do once they know how to do all that stuff? Well, they participate in designing, building, and testing software, but a hundred percent through the lens of accessibility, not whether or not does it work which is the functional side of the fence, but does it work with assistive technology that people with disabilities are likely to use? And do those people, are they having an equal experience? Okay.

Sheri Byrne-Haber: Those are the two things that the lens of accessibility provides for an accessibility tester. Other than that you’re participating throughout the entire life cycle, just as if you were a, a designer, a builder, or a tester. You’re just looking at it with a very particular point of view, okay? Women are really well represented in the accessibility space. There’s five times as many women in accessibility as there are women in non-accessible roles, just traditional straight up software testing you know, analytics coding, program management, things like that.

Sheri Byrne-Haber: It’s actually a good place to be because there are other women that can help you support your careers who have been there and done that. And you, you may get a better level of, of understanding from getting mentored by other women than you might be by getting mentored by somebody who doesn’t have the lived experience that you do trying to survive in your career. Okay?

Sheri Byrne-Haber: There is a significant demand for accessibility testers. Unless you work for Elon Musk, chances are you are not gonna get laid off, and that’s because the demand for accessibility testing is being driven by regulations and litigation, especially in the us. So the Americans with Disabilities Act require it, it, the language of the law itself doesn’t require accessibility, but it requires equal access. And the litigation, and we have about 4,000 plus or minus cases per calendar year in the US is focusing on WCAG as that standard to determine whether or not something’s accessible enough.

Sheri Byrne-Haber: As long as there’s laws and there’s long as there’s litigation, there is going to be a demand for accessibility testers. And right now we’re in a place where colleges are not turning out a lot of people skilled in accessibility testing because it’s not required as part of the computer science program. You barely even touch on it if you’re in a graduate HCI program. This is something that’s very much self-taught, and to be honest with you, it’s also very much passion driven.

Sheri Byrne-Haber: A lot of people get involved in accessibility because they have a personal experience with the disability. Again, don’t make assumptions. People see the wheelchair and they’re like, ah, I know why Sheri got into accessibility. Now, I actually got into accessibility because I have a deaf daughter and my deaf daughter you know, experienced a lot of issues when captions weren’t made available to her.

Sheri Byrne-Haber: There a lot of times there’s this, like I said, personal connection that makes people passionate about being in this space. Keep in mind disability is the only dimension of DEI – diversity, equity, and inclusion – that everybody is guaranteed to experience at one point in time or another in their lives. Unless you die getting struck by lightning, never having broken a bone in your life, chances are at some point in time, if you’re not disabled right now, you are going to be disabled.

Sheri Byrne-Haber: When you’re working inaccessibility, you’re working to make the make the place better for your future self. That’s a, that’s another way to look at it if you don’t have a personal connection to disability currently. Okay. being disabled is actually a bonus when you’re working in the field of accessibility, because not only are you bringing the things that you learned about screen readers and, and other assistive technology and the things that you learned about WCAG, you’re bringing lived experience.

Sheri Byrne-Haber: And that’s something that’s very valuable for this type of work. The other thing is work from home has been a thing for people in the field of accessibility. Long before the pandemic 30% of people with disabilities can’t drive. And so work from home is critical, especially if their disabilities prevent them from being able to commute or make it harder or more expensive for them to commute.

Sheri Byrne-Haber: Other than the usual, you get paid well, it’s a fun job. You get to make the life lives of other people better. But this is, this is somewhere where we’re having a disability and being willing to talk about that disability actually helps. And if you need to work from home or if you would benefit from to work from home it’s something that the accessibility managers in the world are very accustomed to.

Sheri Byrne-Haber: There are a broad range of employment opportunities government and education anything attached to federal money, okay? Including money that passes through states and cities and counties has to be accessible. There are strings attached, and those strings are called Section 508. Universities have to make things accessible. Hospitals have to make things accessible. Courts, anything municipal, anything federal, all has to be accessible. The nonprofit space also wants to be accessible because they don’t wanna say, oh, we’re here to help out this group of people, but hey, you people with disabilities, you get in the back of the line. There is typically you know, NPR has somebody dedicated to accessibility. Washington Post, New York Times, they all have accessibility specialists. Those aren’t exactly nonprofits, but it’s places that you see accessibility thought about where you might otherwise think that it wouldn’t be addressed.

Sheri Byrne-Haber: There’s lots of accessibility consulting companies all the retail operations on the internet. If you’re selling in the us it has to be accessible or, or you’re probably going to get sued at one at some point in time. And then, as I mentioned, healthcare is another big field. For each one of these areas, you’re still taking the same domain knowledge that you have on assistive technology and the WCAG guidelines, you’re just applying it to one of these vertical markets. It does not take a whole lot to get started. It doesn’t, being in the field of accessibility does not require a college degree. There are apprenticeship programs for people who wanna get started in accessibility. There’s quite a few resources that are available for free or for low cost online.

Sheri Byrne-Haber: You don’t have to go out and get a college degree in accessibility. In fact, such a thing does not exist. What you have to do is you have to care enough to go learn about all this stuff yourself, invest time, go to meetups, talk to people who are already in the field. I think of accessibility today as where Quality Assurance (QA) was, you know 35 years ago when I had just graduated from Cal 35 years ago for QA, there were no degrees in QA. There was no Six Sigma. These things didn’t exist. You had to apprentice yourself basically to somebody who was really, really good and, and learned from them. And now you can get a degree in QA. You can get all kinds of certifications in QA.

Sheri Byrne-Haber: Accessibility today is where QA was 35 years ago in terms of how to, to, you know, get your foot in the door for the career, so to speak. You can easily evolve from accessibility into more senior careers. A lot of people who spend three to five years in accessibility will then move on to design or UX or UI and front end development, because you will learn a lot about these three things as you’re doing your accessibility testing work. And so if, if this is, if you’re interested in these three areas but don’t have the time to go back to get a degree or go to a boot camp or something else, you can use accessibility as a way to get into the door for some of these other careers, there are I’ve got here a list of some starting points if you’re interested in accessibility.

Sheri Byrne-Haber: Siri was actually invented for people with disabilities. And the iOS voiceover, which is kind of twined with Siri is the screen reader. If you don’t have an Apple platform, then NVDA is a free screen reader that you can use on Windows. Spend an hour not using your mouse. Lots of people can’t use mice. I can’t use a mouse because I’ve got pretty bad arthritis in my hands. That will give you a pretty quick perspective on what it’s like to be a keyboard only user.

Sheri Byrne-Haber: There’s a couple of places that you can register to be a crowdsourced accessibility tester that will help you learn more about how to find bugs, how to report bugs what is it that people are looking for. Most major city centers have a Lighthouse for the Blind, or Center for Independent Living. They usually have ways that they can point people to learn more things about accessibility. And we’ve got meetups all over the place.

Sheri Byrne-Haber: Thanks to the pandemic, most of the meetups are actually now hybrid. If the accessibility group in Orlando is meeting up, you know, it doesn’t matter you live in Portland, you can still go because they’re, like I said, largely hybrid these days. I wanted to give people my contact information.

Sheri Byrne-Haber: I have a website, which is sheribyrnehaber.com. It’s got a free archive of all the blogs that I’ve written over the years about accessibility. There’s probably 200 or 250 articles on there right now. If you don’t see something, ask me because I’m still writing, I’m not writing quite as much as I used to but I get a lot of my ideas from people pinging me and saying, well, what about, you know, how do you make a toast message accessible?

Sheri Byrne-Haber: One of my most popular articles I ever wrote came out of a question that somebody gave me on LinkedIn. And if you use that QR code, it should take you to my LinkedIn profile. I don’t use Twitter. LinkedIn is my only form of social media but I love to connect with people who are interested in accessibility, and you can always ask me questions.

Angie Chang: Thank you, Sheri. That was a really informative talk and accessibility. I love all the resources and the, the the knowledge you dropped on us today. This is the last talk of this career track. Thank you so much for being a part of ELEVATE and for everyone who’s still here with us after two days of nonstop talks, developer workshops, networking, meeting recruiters.

Angie Chang: Thank you again. Networking is gonna start. We’re gonna have some fluid networking, so if you’ve seen Everything Everywhere All At once, it’s gonna make some sense to you, or I think it makes sense to someone who’s in that movie. It might be something else to you. I’ll see you in networking. Thank you Sheri, for being so open and willing to connect on LinkedIn. And yeah, I’ll see you on the other side. Thank you.

Sheri Byrne-Haber: Much. Okay. And I just answered the one question that came in about IAAP certifications. Disclaimer, I’m on the certification committee. I actually help write the test. So yes, I believe that they’re worth worthwhile. They are standardized. They’ve been around for going on seven years now. But they’re not cheap, right? It’s $375 to take each of the tests. Plus, you know, if you wanna sign up for a membership, that’s another a hundred bucks.

Sheri Byrne-Haber: If you can’t afford the IAAP memberships, another path you can go is with the US Federal Government. It’s called Trusted Tester. It’s free, it requires a significant investment in time. Took me about 120 hours to complete it. Most people actually go faster. I had to struggle to unlearn everything that I knew and only respond in the way the government wanted me to respond. That was really hard to do. But if you’re new to accessibility, you should actually be able to get your Trusted Tester certification faster.

Angie Chang: Thank you.

Sheri Byrne-Haber: Thanks everybody.

Angie Chang: Thank you.

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Posted on

“Dr. Robot Will See You Now”: Rashmy Parimi, Manufacturing Test Engineering Manager at Johnson & Johnson (Video + Transcript)

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Sukrutha Bhadouria: I hope you’ve been having a good session so far. Good time so far in the conference and we are ready for the next session. Thank you all for waiting for us. Rashmy is here to give us our next talk.

Sukrutha Bhadouria: Rashmy is a manufacturing test engineering manager working at robotics and digital solutions at Johnson and Johnson. In her career over the last 13 years, she has worked on consumer products, trained signaling, and more recently robotic applications and medical devices. She’s passionate about making an impact on our society with technology and helping fellow women in tech in their journey. Welcome, Rashmy.

Rashmy Parimi: Thank you for the kind introduction. Hi everyone. I’m Rashmiy I am part of the robotics group in Johnson and Johnson currently working on the manufacturing side of one of our new robotics products soon to be released to the market. Through this talk “Dr Robot Will Now See You” I’d like to transport you to this future vision where this will be a more accessible reality for a lot of people. <Laugh>.

Rashmy Parimi: I want to go back a little in the history before I transport you to where we are today and what the future looks like. A lot of you must have seen this picture on the left of an early operating room where surgery was more of a spectator show. Antiseptics and anesthetics were not something of commonplace. There was no concept of sterilization and for a lot of, I would say decades back then, laughing gas was a commonly used anesthetic.

Rashmy Parimi: Even that was not highly recommended because there was mixed feelings either by the patients or the doctors to use it. A dentist came across ether being an effective anesthetic and he compelled the rest of the medical community to conduct a clinical trial to give more substantial data. And that was one of the starting milestones of making anesthesia a regular process of surgery.

Rashmy Parimi: I think the data convince people that one anesthetics are good. They’re not necessarily something that take you out of control. And also convince surgeons that they didn’t have to resort to methods like strapping down the patients, to help them go through the surgery, because without an anesthetic, the pain will make them move and that’s not something ideal. And they also felt that having a PA stable patient would give them more dexterity and stability to operate.

Rashmy Parimi: That was a very fast history of surgery back then. But from then to now, like there’s so much, you know, medicine has gone grown from deeps and bounds increasing human lifespan by at least 30 years.

Rashmy Parimi: And even today, I think the whole fascination with watching surgery has not gone away, but it’s a little more, I’d say refined from how it was in the photo depicted on the right towards, sorry, on the left to where it is on the right where there is more advanced rendering of the surgical procedure, either during to help other specialists participate in it or to a surgeon or a medical team in a far away location to help add more perspective to a complicated situation.

Rashmy Parimi: From a very low out like low outcome pain causing and a long recovery method to introduction of laparoscopy and endo, which has improved patient outcomes and reduced the recovery time and also improved the accessibility to a lot of people for complicated procedures. This is where I think with this is what most people are familiar with and laparoscopic was what sewed the seeds for the first ever use of robotic surgery.

Rashmy Parimi: This particular arm is maybe familiar to a lot of people as something used in, you know, large industrial assembly houses for large scale manufacturing, more like you know, car assembly facilities or other large equipment facilities.

Rashmy Parimi: But you’ll be surprised to learn was this was one of the first experimentations of whether robotic surgery can be used or not. And you will be even more surprised to learn that the area in which this was used was brain surgery. <Laugh>.

Rashmy Parimi: This was used to guide a percutaneous needle to do brain biopsy back about more than 25 years ago. And then this concept was further expanded to a colostomy and TransU urethral resection to further peak people’s re and research group’s interest to develop the concept of robotic surgery even more and work towards bringing it from a lab prototype to more of a reality.

Rashmy Parimi: In 2000, one of the pioneer companies of robotic surgery, Intuitive Surgical, they broke the ground finally when their system, the first ever Da Vinci system got FDA approval for general laparoscopic surgery.

Rashmy Parimi: It was this innovative device with robotic arms with visual systems and also they had help from nonprofit scientific research organization, SRI, to help them advance a lot of these initial prototypes. And that’s was how most people today, if they are familiar with robotic surgery, I think this is the one name they recognize instantly.

Rashmy Parimi: Let’s talk about what are the advantages of robotic surgery that makes it so attractive to use when evryone would admit that laparoscopy already takes us through a good bit of path onto, you know, smaller incisions and all of that.

Rashmy Parimi: We still get the same advantage as laparoscopy that is a smaller incision, which means quicker healing, lesser hospitalized time, which I’m sure all of you will, you know, relate to the expensive insurance bills and not having to deal with that. And also, it is cost saving and the body will recover faster through a smaller incision, since the amount of trauma is less.

Rashmy Parimi: The other advantage is the precision the instruments can reach into hard to reach places of the body without having a wide incision with accurate precision and stability, which makes a big difference in terms of your outcome of the surgery. And also with this precision al the comes with it, it adds an extra, I’d say boost to the surgeon’s abilities and gives them the confidence to tackle some really tricky procedures.

Rashmy Parimi: One of the important things of having a successful surgical outcome is good visualization. When you know you cut a part of the body, there is obviously going to be blood involved, and in typical surgery it could a lot of times block the view of what is going on there, but with the time your incision smaller cuts, that disadvantage can be overcome and it leads to a better outcome.

Rashmy Parimi: There’s a good example that I would like to use for how pressure virtualization improves the surgery. Having robotic vision is like if you want open surgery is like using a flashlight to look through a window into your house, while robotic surgery is like opening the door, turning on the lights, and then trying to look at your house. You can see it’s evident, which is a better way to look at your house.

Rashmy Parimi: And that advantage is offered to by the advanced imaging that comes with robotics surgery and with, in addition to all of these, the other advantage is exceptional dexterity. Everyone is familiar with how surgeons have these long schedules and if things do not go as planned, there is a lot of fatigue on them with the long hours and that can lead to that showing up on the surgery itself.

Rashmy Parimi: With robotic surgery, one of the things that can be controlled is to remove the tremor and other fatigue related impacts so we can reduce these inadvertent punctures or nicks which can cause unwanted bleeding into the body. Let’s look at few of the areas where today robotic surgery is used in one way of the other heart surgery where these very precise repairs that are needed is done using robotics stomach, though it looks like a big area, there is a lot of fine precise procedures that can be done in a better fashion using robotics.

Rashmy Parimi: General surgery of course, is another area where with a smaller incision and the precision offered, you can do a lot more compared to non robotic surgery. And same goes with the area of GY gynecological surgery where there is, you know, access issues and you want to make sure you don’t impact the healthy tissue or healthy organ parts.

Rashmy Parimi: Same thing goes to lungs where the access is extremely difficult and with kidneys where the, the areas so delicate important that you want to make sure you do not cause unwanted damage to the existing parts. In the area of orthopedic surgery, robotics have given an added advantage of very precise cuts and placement for implants and you know, it’s popularly used I think in hip replacement and knee replacements, which has become very common place in the society today. In the area of dental surgery, there is a product in the market today which help with dental implants and there’s, I’m sure there’s a lot more research going on.

Rashmy Parimi: And as I explained in my first example brain surgery, it started off <laugh>. The whole idea for this was sewn with brain surgery and it is still an area of widely researched today and they are trying to develop products in that area. So here I have some examples of some popular players in the market today. Roughly going over that, the first one is Johnson and Johnson’s robot Monarch, which is, which has FDA approval in the lung cancer and kidney stone management space.

Rashmy Parimi: Below that you have Medtronic’s robot Hugo, which has approvals in the general surgery space. And the picture below is Intuitives’ DaVinci. It’s a newer generation of it, which also has approvals in general surgery and a lot more areas on the right hand side. The first one is the Yumi robot, which is used in the dental surgery field. Their application right now is in the area of implants. The one below from Striker is the maker robot used for the orthopedic area. I don’t want to guess the wrong thing, but I think in the, a place of hip replacement probably. And the one below is from Siemens and this is a robot used in the cardiovascular area.

Rashmy Parimi: Now that I’ve peaked your interest on how, what are the advantages that come with this novel application? I’m sure all of you must be curious, how do you break it into this field? What are your pathways? Is it something very niche? Is it very small exclusive circle?

Rashmy Parimi: Well, I’d like to walk you through my own career path to kind of show you it’s really not all that difficult. And in the next slide, I will also kind of walk you through during the various stages in the life cycle of a product development, what are the different functions that interact and how different disciplines come together to successfully build a robotic surgical product. I started off by education as an electrical engineer, but using that as my foundation, I have worked on firmware for different products, electricity meters, crane systems, small devices which include wearables, thermostats.

Screenshot at .. AM

Rashmy Parimi: I went into this not through either medicine or robotics. I started from a very normal field, which I’m sure most of you feel <laugh> a little easy to relate to. I did have a small ex in brush with medical devices early in my career where I was working as a part of a team on a prototype of a USB based ECG monitor.

Rashmy Parimi: If any of you have noticed the ECG monitor today used in the hospitals, it’s a big piece of equipment and it’s not portable. It’s used in a remote location and they want to share the data around for more opinions. It’s not easily done. There is that accessibility issue. But if it were in a USB form and the data can be collected wirelessly and shared across seamlessly without the boundary of a physical location, it it would be a great blessing to bringing healthcare to rural areas where accessibility is a big issue.

Rashmy Parimi: The proposition of that product was very interesting. And back then, I wanted to continue in that but then again it was just one research project. As I grew in my career, one of the chances I encountered was to be part of the startup Verb Surgical, which was working on a soft tissue surgical platform.

Rashmy Parimi: Verb Surgical has been acquired by Johnson and Johnson and that team is continuing the work on that platform. Hopefully soon that will be in the market helping people improve their quality of lives. And even if you notice through my career, the job duties I’ve done has varied from pure research projects to some integration to what I do today, which is manufacturing test. All of this is more about applying your skills, existing skills across different areas. I have not taken any new courses.

Rashmy Parimi: I have always maintained this curiosity to upskill myself on the job and try to read more on things I don’t much, that was how I was able to work through different domains within the same company.

Rashmy Parimi: Next, I want to talk about what are the various disciplines and roles that participate together during the development of a product. Initially when you want to establish the user needs and make sure a certain product is feasible from a regulatory perspective, the team that typically does the groundwork, the product managers who talk to the customers such as the physicians to make sure they understand what will help them. Then you have the systems engineers, who translate those customer needs into some kind of actionable product requirements. And then the clinical engineers, who also bridge the gap from a clinical perspective.

Rashmy Parimi: The regulatory affairs team helps trying to understand what, how the impact of that, you know, what is the burden of this product to make sure we are safe. And also how, how do we prove that this product is safe to use on human beings once the use case has been established And there is this clear requirements for the product.

Rashmy Parimi: Then comes a design phase where you have design engineers and various arenas. You have electrical design engineers, mechanical design engineers, UI engineers, UX engineers, all coming together to build different pieces of the system and of course test engineers to test all that has been built.

Rashmy Parimi: And for most large scale products, one of the things that has made big difference if the product moves forward in a given timeline or it does not launch off is the integration piece of it.

Rashmy Parimi: There is a lot of complex software and hardware coming together and integration plays a big role. We have the systems integration engineers trying to piece those puzzles, making sure two independent modules operate together as one big unit, and also clinical engineers from time to time to make sure what physically was decided in the beginning is still what the goal of it is towards the end.

Rashmy Parimi: And as the product goes into its future stages, the burden is to val validate and verify it so that we have the essential documentation for FDA approval. But before that, the manufacturing team and the supplier make sure they work with various vendors and internally and to build up these units that will provide the data for FDA to review and approve the device.

Rashmy Parimi: Once that is done during the commercialization phase, you have marketing team, the sales team, the service team to make sure the product is supported within the customers who are using it and also provide the feedback to support the next level of iteration of design and all of these resulting in a complete cycle.

Rashmy Parimi: As you can see, quality is something which is critically important through the whole process and weigh in in all of the design phases and the later validation and commercialization phases.

elevate rashmy parimi johnson and johnson dr robot will see you now video transcript

Rashmy Parimi: What is the future outlook for this field? This is an illustration from before the pandemic. Just few years ago, there’s been 77 companies and these are only the companies that are have gone public. There are a lot more stealth companies, who maybe close to finishing their product.

Rashmy Parimi: The number of companies have increased from a few million in the beginning of last decade to a lot more billions now. It’s a fast growing industry and there has been a lot of acceptance to make sure this field is supported.

Rashmy Parimi: And in general you’ll see these are the two areas where there has been a lot more progress in terms of adding new procedures and support in terms of surgeon’s interest and also success rates in the field.

Sukrutha Bhadouria: Rashmy, we can wrap up. It’ll be great.

Rashmy Parimi: Yeah, so I think this is my last slide, <laugh>. With this, I hope a lot of people have a lot of questions. I’m happy to answer that later. Please feel free to connect with me on LinkedIn. Thank you everyone for your time and thanks for having me here, <laugh>.

Sukrutha Bhadouria: Thank you so much Rashmy and thank you to everyone for attending and you know, posting all your comments and sharing your insights. Thank you.

Rashmy Parimi: Thank you.

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Posted on

Girl Geek X OpenAI Lightning Talks (Video + Transcript)

openai girl geek dinner san francisco mission district residency

Over 120 girl geeks joined networking and talks at the sold-out OpenAI Girl Geek Dinner on September 14, 2022 in San Francisco’s Mission district.

Hear lightning talks from OpenAI women working in AI with music and deep learning, sharing the power of trying and trying again, how to make language models useful, and much more at the OpenAI Girl Geek Dinner video on YouTube!

OpenAI Residency applications are open! OpenAI is looking for engineers and researchers who are interested in applying their skills to AI and machine learning. Please apply for OpenAI jobs here!

If you have an unconventional educational background, we encourage you to apply to OpenAI Residency (applications are open through September 30, 2022).

Table of Contents

  1. Welcome – Elena Chatziathanasiadou, Talent Programs Lead at OpenAI, Recruiting & People – watch her talk or read her words

  2. Multimodal Research: MuseNet & Jukebox – Christine McLeavey, Member of Technical Staff at OpenAI, Multimodal – watch her talk or read her words

  3. If At First You Don’t Succeed, Try Try Again – Alethea Power, Member of Technical Staff at OpenAI – watch them talk or read their words

  4. Making Language Models Useful – Tyna Eloundou, Member of Policy Staff at OpenAI, Policy Research – watch her talk or read her words

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Transcript of OpenAI Girl Geek Dinner – Lightning Talks:

Angie Chang: Hello. Thank you everyone for coming tonight. My name’s Angie Chang and I’m one of the founders of Girl Geek X. We started over a decade ago as, Bay Area Girl Geek Dinners, and we’re still going strong. Thank you to OpenAI for hosting us for a second time. We’re really excited to see the new office and invite a bunch of Girl Geeks over to hear these lightning talks on AI and policy and all these things that we’re so excited to learn about tonight!

Sukrutha Bhadouria: Hi. I know you all were still chatting when Angie introduced herself, but she’s Angie and Girl Geek X is basically her brainchild. It started off with Angie looking to bring women together, I’m doing your pitch, Angie for you because I have a louder voice. Some people, they ask me if I swallowed a mic as a child because I’m so loud and I don’t need a mic.

OpenAI Girl Geek Dinner Facebook Cover

Sukrutha Bhadouria: Anyway, I’m Sukrutha, so Angie started Girl Geek and it was back then called Bay Area Girl Geek Dinners, this was over 10 years ago. And when I had just moved to the Bay Area, looking for ways to meet new people and I found out about Bay Area Girl Geek Dinners dot com at that time, and I tried really hard to meet with Angie, but she was a busy bee doing all sorts of cool things, trying to change the world. And this was way before ERGs existed, right? So people didn’t have a way to connect with the community until they went to meetups.

Sukrutha Bhadouria: And Girl Geek Dinners, at that time, was the one way you could also get an insight into what these sponsoring companies worked on, what life was like. And so it also allowed people to get an opportunity to speak and a lot of the speakers at Girl Geek Dinners were first time speakers. They were too afraid to sign up for conferences. If you go to our website (girlgeek.io), you’ll see all these amazing stats on how since Angie started, there’s been a real shift in the environment in how people are more willing to speak at conferences, due to some of the chances they’ve gotten as a result of speaking at an event sponsored by their company. This organization exists.

Sukrutha Bhadouria: I joined Angie and we tried to change the world together. I’m happy to report that I think we actually did. We rebranded to Girl Geek X, and that’s when the organization hit 10 years. It was a sizable number of people working on it, it was Angie and me and it was just the two of us. And then Angie had this idea to really evolving into a company and so that’s when she started to bring on contractors, more people such as somebody who could take video of our events to make us look a little bit more professional and somebody else to do our website besides me. And we started to do podcasts.

Sukrutha Bhadouria: We started to do virtual annual conferences and we really, really, really were always consistently sold out for our in-person events that would happen at various companies that we partnered with through the Bay Area. Then COVID hit and the good thing is that we had already started to have a global presence through the virtual conferences that we had and we’ve now had four? Five, yeah.

Sukrutha Bhadouria: We used to be carpooling all around the Bay Area together to these events after work and now we are moms. So it’s amazing. We would look up and see amazing people working at these sponsoring companies speak and we’d be like, “Wow, look at them managing their mom life and parent life and coming to these events.” But I just think that it’s now become such a common thing that it’s not as isolated anymore. And I’m hopeful that, you all can come back again and again, because this in person event has really made me really happy.

Sukrutha Bhadouria: I’ve been holed up in my home office today, which is basically a room which also has my… What’s it called? A bike that stays in one place, stationary bike, so it has too many things going on in the room, but I wanted to give a big thanks to OpenAI for hosting us for the second time, for sponsoring for the second time. And I hope that we can keep doing this. So please do get your companies to sponsor and encourage them to do it in person. That’s all I will say. I know I said a lot more than I had planned, but thank you again, and Angie.

Angie Chang: Thank you Sukrutha, for the intro. I guess I should talk up Sukrutha a little more. When I first met her, she was a software engineer in test, and now she is at Salesforce as a Senior Director of Engineering there, so I’m very proud of her. And over the years we… She mentioned we have a podcast, we have annual virtual conferences!

Angie Chang: We’ll be launching a career fair virtually as well, to be announced. And I don’t want to say too much. We have an amazing line up of speakers tonight and we’re going to invite up first, Elena, who is our host for the night from OpenAI.

Elena Chatziathanasiadou: Hi everyone, I’m Elena. I work here and I’m on the recruiting team, I’m leading the Residency program right now. I’m very excited that you’re all here and have joined us together. Really want to thank Angie and Girl Geek X. We’re very excited to deepen our partnership together and to be back in the office here all together, in the new space and to experience this tonight.

openai girl geek dinner Elena Chatziathanasiadou

Elena Chatziathanasiadou: We’re very excited about having you here and in terms of what we’ll see tonight, we’ll have a series of lightning talks and then that will be followed by Q&A and then we’ll get some dessert in the area that we were before and then we’ll wrap up at 8:30. But before we get started, I did want to take a moment to make a quick plug and share that…

Elena Chatziathanasiadou: We’re actively hiring for our Residency program and that includes both research and engineering roles and the goal of it is really to help develop AI talent. The program, it offers a pathway to a full-time role at OpenAI for folks that are currently not focusing on AI and are already researchers or engineers in a different field.

Elena Chatziathanasiadou: We’re really excited to hear from you. If you do have an interest in making this career switch, come talk to me after. And we’ll also have full time recruiting team members and positions that we’re hiring for across research product and engineering that we can tell you more about. Please come find us and learn more about the interview process, but also what the program offers.

Elena Chatziathanasiadou: With that I wanted to introduce our first speaker, Christine, who’s currently managing our multimodal team and previously worked on music generation research, created MuseNet and was collaborating on Jukebox. And before that was a classical pianist who transitioned into a researcher as well. I’ll hand it over to Christine. Thank you so much.

Christine McLeavey: Thank you. So yes, it’s really an honor to be here tonight. Thank you all for being here. And this Residency program is near and dear to my own heart, because I first joined OpenAI through, what was then the Scholars Program and the Fellows Program and those are the programs which have since evolved into this Residency program. I’ll put a plug in for anyone who’s considering it.

openai girl geek dinner Christine McLeavey

Christine McLeavey: I want to talk this evening about my own path through OpenAI, but especially about the two music models that I worked on during the time here. I thought I’d start by just going ahead and playing an example of each of the models. The first one, this is the one I worked on when I was doing the Scholars and Fellows program. This is MuseNet, which works in the MIDI domain, so this is the model trying to generate in the style of jazz. Okay, I’ll cut that off and then after I joined full time, I was lucky enough to collaborate with some amazing researchers here to work on a model that was instead working in the raw audio domain. The fun of that is you get to imitate human voices. This is trying to do the style of Elvis with lyrics by Heewoo. Okay.

Christine McLeavey: Elena mentioned before being at OpenAI, I was actually working as a pianist, I had done some math and physics in college, but obviously it had been a long time and so I think I took a good year of self studying before I applied to anything. And I thought I would just give a shout out to three of the online programs that I particularly liked at that point. They’re all amazing. But then I was lucky enough to join the first cohort of scholars that we had here. And at that point I was just trying to do this process of learning about all these different models. And I had this feeling that instead of just copying a model or copying what someone else has done, let me just try to translate it into a field that I know well, which was music. And so what became MuseNet was really my attempt to take all of the stuff I was learning and then apply it to the music domain instead.

Christine McLeavey: MIDI format is this really nice representation of music. I think of it as the way that a composer thinks of music, so it’ll do things like it tells you what notes it plays when, the timing of it, the volume of it, things like that, which instrument is supposed to play. But it loses all the actual detail of when a human takes it and performs it. You don’t get a person’s voice, you don’t get the sound of a great cellist, anything like that.

Christine McLeavey: The nice thing is it’s what you trade in expressivity, you get in this nice really meaningful representation. It does sound pretty terrible when you try to render materials. As a musician, just thinking about the structure of music, this was a nice simplification for a scholars project. What I did is I took a bunch of MIDI files and I tried to pull them out and turned them into a sort of language to make them look as much the sort of thing that you could get in your own net to predict as possible.

Christine McLeavey: I did things like I would always tell the model which composer or which band was going to be first and then things like what tempo was going to be when notes would turn on and off, and a wait token, which would tell the model how long to wait, things like that. And then what you end up doing is you translate that tokenization into just a dictionary of numbers and the model sees something like this. Which I think that this is the first page of a Chopin bellade or something.

Christine McLeavey: What the model is faced with is this task of given the very first number, what number do you think is going to come next? And then given the first two numbers, what number is going to come next? And when you first look at the first thing and when the model first sees it’s like how do you do this? What does that even mean? It feels like an impossible task. But what happens is the model sees many, many, many examples of this.

Christine McLeavey: And over time it starts to pick up on, ah, if I see 4,006 somehow I tend to see 586 more often after that or something. It starts to pick up on these patterns, which we know because we know the tokenization was like, oh, if a piano plays the note G, then probably soon after it’s going to turn off the note G or something. It has real musical meaning to us. But the model is just seeing these numbers like that. The nice thing is the model gets really good at this job and then you can turn it into a generator just by sampling based on, I thinks there’s like a 20% chance this token’s going to come next, so 20% of the time take that.

Christine McLeavey: The other really fun thing you can do is you can then study the sort of mathematical representation you’ve gotten for these tokens. So I was always giving it the composer or band token in the beginning and now you can look at the vectors or the sort of embedding that it learns through these composers.

Christine McLeavey: And as a musician it’s really fun because I would clearly think that Da Vinci and Ravel, for all these French guys are related and the model just picked up on the same thing, which is cool. But the other really fun thing is that you can mix and match those [inaudible]. So here is the start of one of my very favorite Chopin, Nocturnes. So I actually just gave the model the first six notes of that and this is what the model thought, if instead it was being written by [inaudible] It was a bunch of VPs. It goes on for a while, but I’ll cut it off there. And that was MuseNet.

Christine McLeavey: And then I ended up joining full time after that and I was lucky enough to collaborate with Prafulla and Heewoo on taking music generation over to the raw audio domain. And so in a way this is a much harder problem because now whereas in MIDI world you have just nice tokens which are meaningful in a musical way, raw audio is just literally 22,000 or 44,000 times per second.

Christine McLeavey: You’re recording how loud the sound is at that moment in time and the nice thing about it is it gives you all this expressive freedom, right? Literally any sound you can imagine you can represent as a sound wave, just audio recording to that. The trouble is there are just so many ways for those waves to go wrong or those patterns to go wrong. If you mess up on the short scale, it’s just like crazy hissing noise. If you mess up on long scale, your piece sadly starts getting out of tune or the rhythm drifts or so many ways it can go wrong, it’s really an unforgiving sort of medium. And the problem is now in order to get a minute of music, it’s no longer maybe 3000 tokens you have to do, it’s maybe a million numbers that you have to get correct.

Christine McLeavey: We approached this by looking at ways that we could compress the music to make it more tractable because at that point a transformer could maybe deal well with the context of 4,000 tokens or something. We used an auto encoder to do three different layers or levels of compression and the sort of least compressed on the bottom. The nice thing about that is it’s very easy to translate it back to the regular raw audio. If you put some original song in and then back out, you don’t notice any loss at all. Whereas if you put it through the most compressed version, the nice thing is now it’s super compressed, like 3000 tokens might get you half a minute of music or something. But if you go through this simple just trying to reconstruct the raw audio, it sounds really bad. You can sort of tell that someone’s singing but you’ve lost most of the detail.

Christine McLeavey: The nice thing about it is when you work in that top layer of tokens, now this looks a lot like the MuseNet problem or even just a lot language problem where you’re just predicting tokens. So we train a transformer on that. We sort of added in the same which person was singing, which band was playing, and then we also added in where you can write the lyrics in, so the model conditions on the lyrics and then generates these tokens. And then I won’t get into the details, but we had to train extra transformers to do this upsampling process so that you could get back to raw audio without totally losing all the detail.

Christine McLeavey: The fun thing is you can do things like ask it to generate in the style of Sinatra singing Hot Tub Christmas and I have to put in a book, these were lyrics by at, that point, GPT-2. All right. It’s a Christmas classic now. And then last I wanted to wrap up by talking a little bit about the multimodal team, which is the team that I’m really excited to be managing these days. It’s this really, really great group of people. Unfortunately, our current projects are all internal and I can’t talk about them, although stay tuned, we’ll be publishing them to the blog when we can. You might recognize Clip, which was work done by Alec and Jong Wook both on our team. This is, I guess, nearly two years ago already, but made a really big impact on the image work at that point. And then just to put in a plug for the team, we’re about a group of 10 at this point and we will be hosting a resident in 2023.

Christine McLeavey: Please reach out if anyone’s interested to talk more. And then we’re doing all sorts of projects in the sort of image, audio and video domains both on the sort of understanding side and generation side. And we end up working really closely with algorithms, which is the other team that tends to do a lot of awesome multimodal projects. But then also anytime we get close to things that we’re looking at putting out tech customers, we end up working with applied through that and then also obviously scaling because at OpenAI we believe deeply in this, get a good pattern and then scale it up and it becomes awesome. So thank you so much for your attention.

Elena Chatziathanasiadou: Thank you so much, Christine. That was awesome. So now next we’ll have Alethea. Alethea has spent the last couple of years at OpenAI working on getting neural networks to do math. Before that, they built large infrastructure health system, studied math and philosophy and spent lots of time singing karaoke. Welcome, Alethea.

Alethea Power: Thank you. So this talk is called If At First You Don’t Succeed, Try Try Again. It’s been a wild few years. I decided I wanted to give an uplifting and encouraging talk. It’s a short talk so it doesn’t get too deep into technical details, but if you’re interested in it, please find me afterwards. I will talk your ear off about it.

openai girl geek dinner Alethea Power

Alethea Power: Okay, my name is Alethea Power and yes, Patience is actually my middle name, which will be very relevant for this talk. Okay, so about 10 years ago I was a software engineer and site reliability engineer and my dream was to get into artificial intelligence, but I didn’t know how to do it. I didn’t have a degree in AI, I didn’t have any background in AI, I didn’t have any idea how to break in. So I thought, ah, I probably need to take some time off to study this before I can get into the field.

Alethea Power: I started saving up some money so that I could take time off to study. But by the time I had enough money saved up, I realized I needed to handle my gender issues. So I took that time off to go through a gender transition instead of studying AI. Eventually though I was finally ready to try and break into AI in some form or fashion and that was about the time that OpenAI hosted their last Girl Geek Dinner, that was in 2019. And I came to that talk and I met one of the recruiters who stunned me by telling me I didn’t need to have a degree in AI and I didn’t need to have a background in AI to be able to work here.

Alethea Power: She introduced me to the Scholars Program, the same program that Christine went through, which today is called the Residency Program. And I applied to that and I got in and I had the best mentor in the entire program, Christine. I’m second generation scholar up here. But there were in addition to the obstacles before, there were obstacles after joining the program as well, about three weeks after I joined, there was a pandemic, you may have heard about it. But despite spending a lot of time fearing that I might die or people I love might die for some reason or another, health or political, Christine was very kind and understanding and supportive and she helped me get to the point where I had learned a ton about artificial intelligence and managed to do a great project and I ended up applying full-time and I got three offers here. Thank you. I wasn’t trying to brag, but thank you. This is more to encourage you.

Alethea Power: I ended up taking a job on a team that was trying to teach neural networks to reason and do math. And what I want to talk about here is about a year after I joined that team, I released my first research paper called Grokking: Generalization Beyond Overfitting on Small Datasets. I’m going to give you a very basic introduction to what all that jargon means. And like I said, if you want more technical details, come talk to me afterwards. So first I need to explain how training neural networks works. If you have a background in ML, this is going to be very basic 101. If you don’t, it’s going to be exciting.

Alethea Power: Okay, so usually when we’re trying to train a neural network, we’ve got some amount of data that captures a pattern that we want that neural network to recreate in the future. And often if we’re doing what’s called supervised training, we’ll break that data up into training data and evaluation data. And you can think of this, the training data is sort of what we actually teach the neural network, what it learns from. This is like classroom education and evaluation data is basically like pop quizzes to see how much the neural network learned. And neural networks have this nice property where you can pop quiz them. They don’t learn anything from the pop quiz, they just tell you how they did and then five minutes later you can pop quiz them again and the questions are all new again, they have no memory of them. Throughout the course of training, we measure the performance of the neural network on both the training data, the classroom instruction and the evaluation data, the pop quizzes.

Alethea Power: And there’s two main ways we measure this. One is called loss. I won’t go into details right now about what loss is, but the short version is it’s a differentiable function calculus derivatives that we use to actually figure out how to modify the network, so it learns, when loss goes down. The network is learning. Accuracy is exactly what you would think of being like a test score, so 0% accuracy means you got every question wrong. A hundred percent accuracy means you got every question right. This is what a very successful neural network training looks like. You can see, oh, the x axis here on both of these graphs is steps of training. You can see that as we train this neural network along the loss on both the training and evaluation go down. It’s learning what it’s supposed to learn from and it’s able to generalize that to the pop quizzes.

Alethea Power: It’s doing well on the tests as well and then this is what it’s actually scoring. So by the end of this training it gets up to 90% accuracy, so it’s got an A. Sometimes though, if you train a neural network for too long, it starts to do what’s called overfitting. You might remember the word overfitting from the title of the paper. In this case, the neural network learns too much detail from the training set that doesn’t really generalize to the rest of the world. And so its performance on the quizzes starts to get worse. So an example of this in this paper, I was training neural networks to do math, basic mathematical equations. For instance, if it happened to be the case that the training data had more even numbers than odd numbers, and if it was trying to learn addition, then it might learn that usually the answer is going to be even. Well, in reality that’s not true in addition.

Alethea Power: In reality, you want to actually know how to add and the number’s going to be whatever it is. So that would be an example where it learned some sort of incorrect, non-generalizable information from the training set and that made it start performing worse on the evaluation set. And you can see here in this situation, the accuracy on evaluation would go back down. Sometimes, and this is very common when you’re trying to get a neural network to do math, you have an even worse situation where the same thing happens with your loss, but it consistently fails the pop quiz every time. Gets to a 100% percent accuracy on the training data and fails the pop quiz. This means the network and we were using similar kinds of networks to the ones Christine was talking about, just math instead of music, this means the network never really understood what it was learning, it just memorized it.

Alethea Power: This is like the kid who knows that when you say six plus four, you’re supposed to respond with 10 but has no idea how to actually add. So this was a common scenario when training neural networks to do math. They’re really good at pattern recognition, but they’re not always good at understanding a deep analytical precise truth underneath the pattern. Well then one day we got lucky and by lucky I mean forgetful. So one of my coworkers was running an experiment like this and he went on vacation and forgot to stop it. And so a week later he came back and it had just kept studying and studying and studying and studying and studying and studying and studying and studying and studying. And it learned. So what happened here was, it went into this overfitting regime where usually we’d say, ah, it’s learned all it can learn from this training data.

Alethea Power: There’s no more to learn and see, it still had zero accuracy and it just kept getting worse and worse and worse. And then suddenly long after it memorized all of the training data, it had an ‘aha’ moment and it was like, oh, all this stuff that I memorized actually makes a pattern and the pattern is addition or division or S5 composition or whichever task we had it working on. And then the loss started coming back down on the pop quizzes and it went up and it got a 100%. This is weird, this never happens in neural networks. We dug in and recreated this many times, implemented it twice, saw the same behavior with two completely independent implementations on a wide variety of tasks and there’s all sorts of other interesting stuff about when this happens and when it doesn’t, ask me in the questions afterwards.

Alethea Power: The point here is at first the network didn’t succeed, but it just kept trying the same way I did when at first I couldn’t get into AI, but I just kept trying. We named this phenomenon where it finally figures it out Grokking, and we named this after Robert Heinlein’s novel Stranger in a Strange Land. It’s a science fiction book and Grok is a Martian word in that book, which means, “To understand so thoroughly that the observer becomes a part of the observed to merge, blend, intermarry, lose identity in group experience.” And it turns out this is exactly what these neural networks do. I’m going to let you take pictures before I change the slide.

Alethea Power: This network was trying to learn modular addition and modular addition you can think of is adding hours on a clock. Also, thank you to Christine for that analogy. If you have 11 and you add 3 to it, you don’t end up with 14, you end up with 2 because that’s what happens on the clock. The clock is modular 12, we were having it learn modular 97, and then we tore open the network that had grokked afterwards to see what was going on inside of it and it had actually built internally this circular structure of the numbers. It had created the mathematical structure we were trying to get it to learn that allowed it to actually solve the problem. Did this with all different kinds of problems, so we had one network learning to compose permutations and it found what are called subgroups and co-sets out of that, details later. But the point is, it worked so hard for so long through so much failure that it became the knowledge it was trying to get.

Alethea Power: The point here is, that if your dream is to get into AI, even if you have no background in AI or whatever your dream is, it doesn’t matter. Keep trying and keep trying and keep trying and keep trying and maybe you can get there eventually. And in particular, if your dream is to work at OpenAI, which I highly recommend because this place is fabulous, then try, even if it’s not the background you have already, even if you feel like you have a weird background or you’re not like the people here or like the people in this field.

Alethea Power: We’re a humanitarian organization. Our core mission embodied in our legal structure and our financial structure is to make sure that artificial intelligence benefits all of humanity instead of just a small number of rich people in Silicon Valley. And to be a humanitarian organization with a humanitarian mission, we need a wide diversity of perspectives here. If you have a different life story, a different path, different perspectives than we’ve seen before, that makes you more valuable here, not less, so please consider applying.

Elena Chatziathanasiadou: Thank you so much, Alethea, That was awesome. And now next we’ll have Tyna, who’s on the policy research team currently doing our rotation on applied research and she participated in the OpenAI Scholars Program, has spent some time researching economic impacts of our models, building safety evaluations, and collaborated on web GPT and moderation API. Let’s hear from Tyna.

Tyna Eloundou: Wow, so many of you. Let’s see. Okay, this works. Hi, everyone, thank you so much for coming. I’m Tyna Eloundou, I’ll be speaking to you today about making language models useful. A bit about myself, let’s see, wow, I’m also a former scholar. I can’t make the claim to third generation because Alethea was not my mentor, but they were super helpful in making my experience here amazing. And part of that culture and that welcoming environment was a reason I chose to stay on after the scholars program [now the Residency program].

openai girl geek dinner Tyna Eloundou

Tyna Eloundou: Today we’re going to be talking about language models and by language model, I mean any model that has language as input and output. So that could mean GPT-3, CODE-X, or BigScience’s Bloom, what have you. Okay, this is going to be the only equation you see throughout this talk and it’s really not that important, but I think it gives us some context as to where we’re going.

Tyna Eloundou: Looking back at this, this is the training objective for GPT-3 and for all GPT like models. Given a corpus of tokens, right? We define the objective to maximize this likelihood, L, which is defined as a conditional log probability over a sequence of tokens that is modeled by a neural network with parameters data that is trained by gradient descent. Now you can forget everything I just said.

Tyna Eloundou: Essentially this optimization produces these models that are trained to predict tokens, but that in itself may not be that useful on its own. I don’t think I’m giving away any secret sauce by revealing this equation to you, but it is remarkable that somehow we go from this to models that can produce, oh sorry, that can do that, right? Write prose, write code or parse data and so on.

Tyna Eloundou: I’d like to talk a bit about the notion of usefulness itself. One way to think about whether language models are useful in the first place is in the pragmatic sense. In the ideal scenario, we would be able to succinctly communicate our goals and preferences to a language agent without having to laboriously explain and list what to do and what not to do.

Tyna Eloundou: How did we initially get usefulness out of language models? When these models were first being developed in research labs, some researchers came with some ideas about how to really get them to do what it is that you want them to do. And these are two of the most prominent ones. One was few shot prompting, which is a method by which you really tell the model what the task is and before putting it on the spot, so to speak, you give it some examples of what you like to do, some demonstrations, right? For translate English to French, you could have a pen to [foreign language], I’m hungry to [foreign language], et cetera. And the translation that you actually want, you say, I would like to eat ice cream and hopefully with that same formatting you get the model to translate to French.

Tyna Eloundou: The other method is supervised fine tuning, which involves essentially just having examples for the model and then kicking off another round of training so the model can become hyper focused on your task and hopefully improve its performance on that task. So as many of you probably know, OpenAI has since then adapted this iterative deployment approach, which helps us put models in the hands of real people and understand how they interact with them. At the time of GPT-3 release, it was doing great by research standards, right? And unfortunately a lot of these research metrics are designed around these methods that we’d spoke about before, which are to prompt with few shot prompting or perhaps to do supervised fine tuning. Once we deployed, we really quickly learned that people don’t like prompt engineering. In fact, they don’t really like to do a lot to communicate their goals to the model, which is fine. It’s a feature, not a bug.

Tyna Eloundou: At its most helpful, a language agent can infer what we want without lots of specification and carry out those inferred goals effectively and efficiently. Unlike researchers, people were using natural language instructions to ask GPT-3 for what they wanted. But because of the training objective that we saw previously, the model was really tempted to just pattern match, right? If you gave it a prompt of write a short poem about a wise frog, it would very helpfully give you similar types of prompts instead of following your intent. This spurred a research effort within our alignment team to teach the models how to follow direct instructions. They did this using two insights. The first is borrowing from the supervised fine tuning or supervised learning literature where you can train the model based on examples or demonstrations, right?

Tyna Eloundou: You have a prompt and you tell them what you would ideally like it to do. And the second insight came from the reinforcement learning literature where you have some humans compare outputs. And so this model learns to generate, that model learns to compare, right? That model learns to tell this is good, this is bad. And so now with these two things, you can kick off this joint training process where you have a model that’s generating and then a model that’s critiquing, and this is good, this is not so good.

Tyna Eloundou: Over the course of training, the model learns to get better at pursuing this objective, which is no longer the pure language model laying objective and now it’s the instruction following objective. So the resulting model was InstructGPT, which is presented here. Well, yeah, you can see the output. It’s a poem, it’s about a frog, mentions wisdom, and it’s pretty short. I feel like all the requirements were met for following instructions there.

Tyna Eloundou: This was a plot that was quite striking to me. This is one of the main results from the InstructGPT paper. When I first saw this, it didn’t make a ton of sense until I really understood the research behind it. But I think that you can think of the Y axis as a proxy for usefulness and the X axis. We have model size and conventional wisdom has it that… We’re at OpenAI as you scale things, things get in general better. But you can see that even at its smaller size, right here, if you can’t see it’s 1.5 billion parameters, even at its smallest size InstructGPT was deemed to be more useful than any permutation of the base GPT model. So I started this discussion by talking about how research based approaches were not pushing far enough in terms of getting us usefulness out of these models. There’s now this emerging literature focused on helping models be more effective in tasks.

Tyna Eloundou: Broadly speaking, this literature involves having models break big problems up into smaller problems or things step by step before coming up with a final answer. And this does not need to be at odds with our human alignment driven research. In fact, right here you see a result by Kojima et al. and although their results are great overall across the board, we do see that they make the Instruct models even greater. There’s such a huge gap, a huge gain that we see with the Instruct series of models.

Tyna Eloundou: I would like to conclude by thinking about the next steps in this line of research. We know that there can be some instructions that can be malicious or exploitative or deceptive. If language models were to pursue usefulness at all costs, they might become dangerous in the pursuit of dangerous instructions or dangerous intent. Could there be other objectives we pursue along with usefulness that get us helpful but not dangerous models, perhaps kindness or hopefulness?

Tyna Eloundou: And lastly, with instructions, we’re mainly in the driver’s seat and we initiate interactions. As language models become smarter, perhaps kinder, more capable, it may be appropriate to think of them as collaborators and they may be capable of initiating ideation, creation among other things. What are the different modes of interaction we would like to have with these models? Would we want them to advise us? Would we want them to inspire us? Perhaps at Girl Geek X 2042, it’ll be a language model presenting about something new. Thank you.

Elena Chatziathanasiadou: Thank you so much all for joining. I guess with that note, I did want to mention that we’ll kick off mingling time and dessert in the area that we were before and our speakers will be available for you to ask them questions. We also have some of our recruiting team members here tonight. If you all want to come up to the front to just quickly introduce yourself or just say hi so that people can see you and then you can all come find us.

Elena Chatziathanasiadou: As I mentioned in the beginning, I’m Elena, I’m also hiring for the Residency program, so come talk to me, come find me. And then we also have some demo stands of our Dolly product and also our GPT-3, if you want to check them out. Jessica and Natalie will be doing those demos. So yeah, go find them as well.

Elena Chatziathanasiadou: Thank you all for being here. I hope you enjoyed it. Thank you to our lovely speakers and to Girl Geek X, to Cory and to all of our ops team and everyone who helped put this together and let’s go enjoy some dessert!

openai girl geek dinner networking
openai girl geek dinner organic straus soft serve dessert
openai girl geek dinner networking after talks
OpenAI Girl Geek Dinner

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Posted on

Girl Geek X New Relic Lightning Talks + Panel (Video + Transcript)

Women at New Relic discuss observability, metrics, monitoring, community, APIs, React, and leadership at the New Relic Girl Geek X event with over 190 girl geeks joining the lightning talks and leadership panel discussion online.

Table of Contents

  1. What is Observability? – Padmaja Gohil, Senior Solutions Consultant at New Relic – watch her talk or read her words

  2. Customer Success and Value Realization Through Value Metrics – Kate Kordnejad, Lead Principal Technical Account Manager at New Relic – watch her talk or read her words

  3. How Browser Monitoring Can Be Used To Improve Website UX and UI? – Carolina Rotstein, Solutions Consultant at New Relic – watch her talk or read her words

  4. DE&I – Finding a Community with New Relic ERGs – Solmaira Flores-Valadez, Senior Technical Account Manager at New Relic – watch her talk or read her words

  5. Observability in the Age of Web3 – Nora Shannon Johnson, Solutions Consultant II – LATAM at New Relic – watch her talk or read her words

  6. APIs: Get Your Data When You Want It and How You Want It – Sarah Hudspeth, Solutions Consultant at New Relic – watch her talk or read her words

  7. The Power of React.js – Jo Ann de Leon, Senior Technical Account Manager at New Relic – watch her talk or read her words

  8. Leadership Panel – Ariane Evans, DEI Manager at New Relic, Nada Da Veiga, GVP, Technical Solutions Sales at New Relic, Erin Dieterich, Senior Director, Social Impact & ESG at New Relic, Kim Camacho, Director, DE&I at New Relic, Tracy Ravenscraft, Director, Technical Account Management at New Relic, Stefanie Smith, Senior Manager, Talent Acquisition at New Relic – watch the panel or read their words

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Transcript of Girl Geek X New Relic – Lightning Talks:

Angie Chang: We’re going to give people a chance to join us, but in the meantime, I guess I’ll start with some introductions. Hi. My name is Angie Chang. My pronouns are she, her, hers. I wanted to say, thank you so much for joining us for our Girl Geek X New Relic event. I want to encourage us to connect with each other. If you can, I would invite you to put in the chat, your name, your location, your job title company, and your LinkedIn URL, so we can all get connected. Feel free you to connect with me. I wanted to introduce myself and give you some background as to what Girl Geek Dinners is about.

Angie Chang: I started Girl Geek Dinners in San Francisco when I started working in engineering, and I felt a bit lonely on the team as the only female engineer. And I go to all these tech events, but I wanted to go to tech events where the gender ratio was flipped. These didn’t exist in 2008. I decided to start my own series of Girl Geek Dinners. It turns out, after five days of posting about online, we had over 400 girl geeks that were interested in joining us for our first Girl Geek Dinner. And then the next one was sponsored by Facebook. And then we just snowball from there.

Angie Chang: And now today we have over 200… I think we’re at 300 Girl Geek events. We’ve also started things like a virtual conference every year, celebrating international women’s day. We really have also filled out our product portfolio of this podcast. You can go on YouTube. All the talks that you will hear today will also be on our YouTube channel. I invite you to subscribe to that. It’s at youtube.com/girlgeekx. And you can find all the videos from our previous events, and today’s event on there as well. [inaudible] chatting.

Angie Chang: I wanted to share how much I love learning from going to all these events over the years, because from listening to the women working in the diverse corners of male dominated industries, from engineering to sales, we have heard from people share their expertise. And we also learned things like, that job titles are constantly evolving. I remember thinking that this was a really interesting part of engineering and tech that we often don’t think about, of the first thought of big tech or tech companies.

Angie Chang: When I used to work at Hackbright Academy, a coding bootcamp for women, there was some women that I met at New Relic who were sales engineering leaders. And I thought they were so cool, because they not only knew engineering, but they were also very savvy on the business side. It’s because of the sales stuff. I remember thinking that this was a really interesting part of engineering and tech that we often don’t think about, of the first thought of big tech or tech companies.

Angie Chang: The sales engineering side is overlooked. I’m glad that we have heard from people like Tracy, and all the solutions consultants and technical account managers, who are interested in sharing the projects they’ve been working on and their passion for technology, today with us. We are excited to partner with New Relic, a company leading in full stack observability. We’ll hear from the solutions consultants. And they’re formerly called solutions engineers, sales engineers, and technical account managers. I think what I’ve learned is that solutions consultants are pre-sales, and technical account managers are post-sales, but that’s something that you can have a conversation with people about afterwards in networking.

Angie Chang: These lightning talks will be discussing observability metrics, ReactGraphQL for APIs and more.

Angie Chang: Now our first speaker on customer facing technical roles at New Relic is Padmaja Gohil. Padmaja is a senior solutions consultant at New Relic, and loves being a sales engineer, because it not only helps her stay at the cutting edge of technology, and she gets to work with a multitude of customers using these technologies. In her free time, she loves listening to music and adventure parks. Welcome, Padmaja.

Padmaja Gohil: Thank you, Angie. Hey, everyone. Very nice to you. Is everyone able to see my screen? Angie, can you just give me a thumbs up?

Angie Chang: Yay.

girl geek x new relic padmaja gohil observability code phrases

New Relic solutions engineer Padmaja Gohil talks about observability in software development, the phases of observability, and observability as code at Girl Geek X New Relic virtual event. (Watch the talk)

Padmaja Gohil: Okay. Awesome. I’m Padmaja Gohil. I’m currently a senior solutions engineer with New relic. Today we’re going to be talking about all things observability. Quick disclaimer, please not hold me accountable to any sort of overlooking statements. Before we dive into the presentation itself, I would like to give you guys a quick glimpse into my journey so far. Growing up, I’ve always wanted to be an engineer, but once I started my engineering degree, I realized that my interest lay somewhere at that nexus of tech and business, which led me to do my masters in engineering management, where I studied business concepts, but focused in high tech industry. I’ve also previously dabbled in consulting, venture capital and data privacy.

Padmaja Gohil: I’ve been a solutions engineer with New Relic for the last three years. I absolutely love what I do. New Relic is an observability platform, and because of which I’m going to be talking about observability today. But at the same time, in my day to day, I get to work with a lot of different customers. Understand how they’re using technology, and I help them achieve their goals using New Relic. If you guys have any questions about what solutions consulting, solutions engineering, sales engineering is all about, feel free to reach out to me on LinkedIn or my email address, and I would love to have a chat. The way I’ve structured the presentation today is, we’re going to talk about what were the changes that we saw in the software development space, that led to observability. Why do we really need it? What it is, and the different phases in which you can implement it.

Padmaja Gohil: And finally, we’re going to touch very quickly on observability as code. We’re going to be covering a lot of ground. Again, feel free to get in touch with me if you have more questions, or if you would like to learn more. Now let’s take a look at how has the model software industry evolved. If you look at the left, on the left side of this screen, you’re looking at our past. Our past was primarily Monoliths. They were stood up on on-premise servers. Usually scaled vertically, very static operations based scenario. We would release once or twice a year. I still remember the days when we would have to manually update our softwares. Now, fast forward to today. Today’s architectures are more microservice based. They’re open sourced. They’re more complicated. They’re usually hosted on Kubernetes cluster.

Padmaja Gohil: We went from releasing once or twice a year, to releasing maybe multiple times a day. This has been great in terms of the business. We’re able to push out new code, push out your releases and update our software faster, but it on-boards with it a level of complexity when it comes to troubleshooting, detecting issues and finding resolutions for it. This alongside other reasons is why we need observability. In the days of mainframes and static operations, when things went wrong, what would happen is, we would have maybe a couple of dashboards, that we would get alerted on. Usually these dashboards were static. We had run books for all of them, to figure out what’s going wrong and to fix issues. Now, typically, these systems would fail in the same manner over and over and again and again.

Padmaja Gohil: It was a little more simplistic than maybe today. Now, today if things were to go wrong, I’d be staring at my screen, wondering what’s going wrong. Is my cloud provider seeing an outage? Is someone deploying code? Is that the reason why I’m seeing some sort of an issue. Or I could be staring at the symptoms and not the root cause. There is so many ways in which things could break, that it’s really hard and complicated in how we do troubleshooting today. Also there has been an increased frequency of CodeDeploys. We went from once or twice a year, to multiple times a day, which can increase the chances of things going wrong. We no longer have discrete application owners.

Padmaja Gohil: We have distributed systems, but at the same time, we also have distributed teams working on things. There is a need for contextualized data in case of… if a person were to just come in blind, not knowing the history of the systems, they can quickly take a look at things and start fixing. These are just some of the reasons why we need observability today. But let’s take a look at what the definition is. There are a lot of definitions out there. The way I like to think about it is, how well do you understand your system from the work it does? It enables you to do a lot of things. For example, it enables you to collect and alert on the telemetry data types. There’s four telemetry data types, and these are the pillars of observability.

Padmaja Gohil: I’ll speak to those further in the presentation as well, but it’s metrics, events, logs, and traces. These are the four pillars of observability. Observability allows you to focus on your day to day. As software engineers, your job is to, let’s say, deploy code faster, come out with newer features. Your job is not to spend a lot of time in fixing issues. Observability also allows you to focus on that. It enables you to troubleshoot faster. It makes sure that you are ensuring up time and performance while you push out this newer code. It also gives you the confidence to push out new code, because let’s say if things were to go wrong when you were deploying, you have the confidence that yes, I have the system in place to fix those things. It builds that culture of innovation as well.

Padmaja Gohil: In real life scenario, there are so many different ways in which you can implement observability, but there are three phases, three broad phases in how we implement it. I would like to talk to you about it. The first phase is the reactive phase. All of us might have heard the saying that you cannot improve what you cannot measure. The first phase is where you start instrumenting your entire tech stack to collect data. You’re collecting metrics, events, logs and traces from all of the tech stack. You are then understanding how your applications are behaving. A lot of times you might not know what normal looks like for your applications. What does your normal response time look like? What does the normal error rate look like? The first phase is when you are establishing the normals and the baselines, and then you’re setting up foundational alerts on it.

Padmaja Gohil: That’s what the first phase is about. The second phase is now codifying your team’s work. Now, when I say that, what I mean is, you are setting up service level objectives for your application, because what happens is you’re seeing plethora signals coming at you. And you now need to understand how do you measure the success of your application? One of the ways to do that is by setting up service level objectives, and service level indicators, which are SLIs. Let me give you an example of what an SLO can look like. For a web application, an SLO could be that the videos should start playing within the two seconds, and 499% of the time during that one week period. That is your SLO. Now, the service level indicator, which is the SLI, measures the proportion of videos on the website that start playing in less than two seconds.

Padmaja Gohil: You start setting up these kinds of SLOs, SLIs. You measure them over time in the second phase. Now, lastly, the data driven phase. The ultimate aim of observability is to help teams within a company make data driven decisions. You are doing a lot of trend analysis of the SLOs and the SLIs that you set up. But at the same time, you’re evangelizing this to the teams beyond, let’s say, site reliability, DevOps, or application engineers. You’re pulling in folks from, let’s say, customer support, product. Everyone’s looking at the same data, and you’re making decisions. Eventually, you want to get to a stage where you can figure out, how is it that your digital operations are impacting business KPI. For example, if you were an eCommerce website, if the page load of that eCommerce website increases by, let’s say, 10%, are you seeing a drop in the number of users on the website?

Padmaja Gohil: Are you seeing lesser number of things in your card? These are the kinds of relationships you want to start visualizing and measuring. That’s the last phase of observability. One of the things of last phase, is also being able to automate processes. That’s where observability as code comes into the picture. Now, observability as code can again, mean a lot of things. It could mean that the way you are interacting with your observability platform, you’re automating it, but it can also mean Gitops, config as code, infrastructure as code, CICD. Whenever you hear these things, know that these are observability as code. Now, what we’re doing essentially here is that we’re taking some of the best practices from software development, and we are applying it to the operations world. Think reproducible builds, reproducible deployments.

Padmaja Gohil: You are automating processes, you are testing them. And you’re making sure that no matter how many times you run these processes, you’re getting the same result. There are a few things common as a part of observability as code. Firstly, observability as code, it’s literally code. So it does not have a UI. It is declarative. So you are specifying the exact state in which it should exist. For example, if you write a piece of code to create an alert in New Relic, you should be able to take that same code or a template, and then modify it slightly to create a thousand alerts. It’s also reproducible. You are reducing the amount of time you’re spending in managing your observability systems as well. The first thing is it’s declarative. Secondly, it’s versioned and immutable. Ideally, it should not reside in a shared drive.

Padmaja Gohil: Ideally, you should be using a get for it. You should be able to go back and figure out what were the changes made if things were going wrong. It should be versioned and immutable. And lastly, it’s pulled and reconciled automatically. Now, what I mean by this is that if you had created a dashboard in New Relic or in any other observability system, and let’s say one of your colleague comes to you and says that this is a great dashboard. I want to use it for my own needs. They can go ahead, take the dashboard, and maybe they modify it. Then you go into New Relic and you figure out that your dashboard is modified, and you won’t actually revert the changes. You can directly take the code, apply it, and you can get your original dashboard.

Padmaja Gohil: And now you can take the template that you used, or the code that you used, and you can give it your colleague, and they can use it to create their own dashboard. It’s usually pulled and reconcile automatically. There are a lot of solutions available for observability as code. I’ve mentioned some of these here. We also have our own templates for, let’s say, Terraform, in case if you guys are interested. Feel free to look at it in our docs page. But these are just some of the solutions that you can use to implement observability as code. This brings me to an end of my presentation. I know that we covered a lot of cloud. In case if you guys are interested in knowing more, feel free to reach out to me on LinkedIn or my email address. Thank you so much. I very much enjoyed speaking here.

Angie Chang: Thank you, Padmaja. That was really great. And thank you for leaving an email address so people can reach out to you with any questions. moving on to our next speaker. Kate is a lead principal technical account manager at New Relic. She comes with a background in helping customers thrive in their business with the latest software monitoring tools. In her current role, she partners with customers to help them with their full stack observability requirements. So welcome, Kate.

Kate Kordnejad: Hey, Angie. Hi, everyone. Thank you for hosting us. Give me a second to share my screen, and put it in slide mode. All right. I’ll be talking about customer success and value realization through value metrics. I’m just going to jump into a little bit of legal disclaimer, so don’t make any financial decisions based on our discussions today, and or any statements we make, and some proprietary copyright information. All right.

girl geek x new relic kate kordnejad customer success value metrics

New Relic principal technical account manager Kate Kordnejad talks about the evolution of maturity, TAM goals, maturity journey, maturity metrics & more at Girl Geek X New Relic virtual event. (Watch the talk)

Kate Kordnejad: A little bit about me. My name is Kate, and I’m a principal technical account manager here at New Relic. As TAMs, we are an extension to our customers teams. We help them with their full stack observability requirements. We want to make sure they see value, and we basically help them get enabled, follow best practices. We work as a trusted advisor with them. A little data point about me; I love working out. I love yoga, especially Bikram yoga. I love to travel, and I’m a data nerd.

Kate Kordnejad: Okay. Our agenda for today is going to be evolution of maturity, goals for technical account management, our maturity journey, defining maturity metrics, and how can you define maturity in your organization? All right. Starting off with evolution of maturity. In our evolution and journey, we found ourselves improving efficiency from four to five hours to one minute by automating our solution. I’m going to explain how we did this. As things evolved over time, we found our defined metrics to be meaningful. And we did find out more about our customer’s maturity, and how we can help them improve stickiness. For example, are they using custom attributes, or do they have data instrumented for more visibility? With our help, they started getting more mature within the platform. And we were able to identify the gaps, improve upon them. We did soon realize to deliver an observability platform value for our customers.

Kate Kordnejad: We needed to recognize value drivers and use cases, that actually deliver those business outcomes for each and every customer. For example, to improve customer experience, quadrant you see on the left hand side. We had to understand our customer’s business needs. Card abandonment, any association with an operational gap like card crash rates, were stuff that we needed to figure out. We identified the steps to maturity, is basically summarized in alignment. What that means is we need to align customer priorities to the observability value drivers. And agree on prescribed observability use cases, and then enable based on an agreed upon description work streams with the customer, and then finally, value realization. Reflecting on the business and the operational KPIs that we agreed upon during and prior to going through maturity. We actually evolve from just collecting metrics to quantifying metrics into meaningful business values, with a growth mindset, of course. We realize without having a continuous growth mindset, we won’t be able to evolve and improve our solution.

Kate Kordnejad: Our next thing is the goals that are for technical account management. Having an involved automated way to quantify metrics into business values, provides us leverage as TAMs. TAMs, as in technical account managers. We now have data to analyze customer usage, to reduce overall churn, by identifying any sort of gaps we have in utilization, by providing enablement based on usage, and engage platform users and drive valuable engagement by meeting them where they’re at. And directly communicating with our customers and being a liaison internally and a voice for our customers. And essentially, we want to reach value realization with them.

Kate Kordnejad: The next I want to is our maturity journey. Our journey basically started at looking at our platform per customer account, and literally eyeballing metrics we had identified as crucial to understanding and analyzing customer data. It was really hard to assess the pieces of the product they were using by manually assessing their usage and engagement. The normal customer metrics success wasn’t really working for us anymore. For example, if they were building dashboard, this wasn’t showing us the full picture, or the reason behind it that’s looking at their user behavior. It was very one-dimensional, and we didn’t really know if they were getting value out of it. We basically had to look deeper into the metrics, and then identify and associated with value drivers.

Kate Kordnejad: How do we define maturity metrics to get to that point? As a team, we basically start asking ourselves, what results do we want to see from this? Ultimately, what does a good maturity look like? And what does it look like for each product? We needed KPIs to show actual investments. For example, if we looked at our alerting product, we wanted to drive an alerting strategy, or potentially set our customers up with anomaly detection. Next, we had to break each product into maturity metrics. Initially, this was done manually through APIs and us eyeballing accounts, but after we broke down our KPIs by product, we had to describe a desired performance level, and determine how data is interpreted. We had to set up thresholds, place and score for each one, each of the metrics that make upper and lower limits of a desired performance.

Kate Kordnejad: This basically allowed us to understand overall maturity for each customer product using a heat map, and really made maturity pop up the page for us. Now that we had our results defined, maturity metrics chosen by product, we had to basically come up with a way to automate this. Our internal teams were able to automate the process, build out an app using APIs, grab the required data from accounts, and assess maturity. Finally, the last piece of the puzzle was to ensure we documented every single steps, our definitions that are associated with each of the metrics collected for further analysis. Our document includes a breakdown of the products, the metrics associated with it, and each and every single step you need to take to improve your score. From all of this, we want to cover, how can you define maturity in your organization?

Kate Kordnejad: It really comes down to three pillars. Goals and baseline. You have to ask yourself, what does maturity look like for your organization? Describe those intended results. Do you understand the alternate measures for those intended results? Then you move on to data identification. Have you identified any composite indices as needed? And do you collect any of the data right now? Is it accessible to you? And finally, business alignment. Have you thought about targets? Thresholds? Do you have a baseline that you can work with. And then finally, have you tied your maturity metrics to business values that deliver value realization? That concludes my presentation. Thank you for having me.

Angie Chang: Thank you so much for that talk, Kate. Our next speaker is Carolina Rotstein. She is a solutions consultant at New Relic. She is also an economist and political scientist that fell in love with programming and data, and is passionate about untangling holistic customer journeys across complex stacks, which she’ll be speaking about today. So welcome, Carolina.

Carolina Rotstein: Can everybody see my screen?

Angie Chang: Perfect.

Carolina Rotstein: All right. Oh. Today we’re going to talk about browser monitoring, and how it can help us improve UX and UI. Some safe Harbor information, a bit of housekeeping, some proprietary information, and just please don’t use this to make any financial decisions.

girl geek x new relic carolina rotstein browser monitoring ux ui

New Relic solutions consultant Carolina Rotstein talks about improving website UX and UI with real user monitoring at Girl Geek X New Relic virtual event. (Watch the talk)

Carolina Rotstein: A bit about me. I’m a solutions consultant for New Relic in the commercial E-sales team. I’m an economist and a political scientist, but I fell in love with programming and big data. I’m passionate about untangling holistic customer journeys across complex stack, and my most previous role included optimizing UX and UI for the gaming industry. And yes, we did collect a lot of data.

Carolina Rotstein: Today’s agenda, we’re going to focus on improving the website’s UX and UI, and using real user monitoring for this. Also we’ll cover why we should focus on UX and UI optimizations, and some of the metrics that we can use to do this as well as the metrics that come out of the box for New Relic and some other tools. And then an approach towards optimizing customer experience, including UX and UI, the traditional way and the enhanced way using big data. My peers talked a bit about observability maturity. At New Relic, we focus on data driven decisions. We want to have an approach with this framework towards taking data driven decisions.

Carolina Rotstein: Now, in this part, customer experience is closer to product and support. While it does have a lot of positive impact into how customer support, user product, and just impact on their KPIs. It’s mostly geared around design and product and development. Experience optimization, and a big portion of that is user experience. And also user interface optimizations are closer to the revenue. Even though it’s at the bottom of the the funnel, any impact that we might have into optimizing the experience, will have a monetary increase for the companies that we’re in.

Carolina Rotstein: First I’d like to talk about browser versus synthetics. We talk a lot about the jungle versus the lab. The jungle would be empirical data. So just every browser, every device, every location, and what your customers are using. The lab will be how we are tracking the health of the site just as we mapped it. We eliminate all the variables to just understand performance and solve problems quickly. This is done by synthetics. The jungle piece or the real user monitoring is once we deploy that application into the wild. So the users might take pads that we just did not foresee. For that we use browser monitoring. It’s an essential tool for user experience. It has a couple of places that we would focus on. Product usage, front end performance issues, content strategy, and in this case, websites, UX and UI.

Carolina Rotstein: I’d like to talk a bit about the metrics that we can use for this. This are not all the metrics, but I strongly recommend this as a start. Just for front end performance and monitoring, we have core web vitals, the user time on site. And it’s just user centric health metrics, such as throughput chart. For New Relic, we can divide what the time that it takes to load is split by the front end versus the back end. But for product usage, which is getting closer to that UX and UI, we track that funnel.

Carolina Rotstein: These are those conversion funnel related metrics that map to the business. These are unique to every company and every website. Those are success events, which can be form fills, video watch, purchase, and business classifications. These are custom metrics that we would map. Then we have all these audience insights such as device and location and vanity metrics. The vanity metrics normally come out of every tool, but they’re a great place to just look at your application, sort of like the Canary in a coal mine. Then for content strategy, we can see how users are navigating through the site, such as in the metrics that we would use, are link positions, most popular, previous page, the next page. And we also have pages report, such as the most popular pages, the time spent on site, how long it takes to load an assets. But we also can track audience insights. This can come from your BI data.

Carolina Rotstein: New Relic can take just any sort of data, but with other tools, you can certainly integrate it. This can be things that are a bit more robust, such as persona development, even the VIP level of your users, or the user IDs. And then very targeted towards UX and UI, and specific real user monitoring. We have the time spent on task, which will be the time before a user completes a success event. The ease to perform a task, rage clicks, which is just a user frantically clicking. Marketing funnels is a good one. In New Relic, we have something called the apex score, which is just taking into account the error rate and the load time to proxy some of the survey based customer satisfaction, traditional UX and UI metrics.

Carolina Rotstein: Now, very related to UI in just design, we have AB test, popular device sizes, screen size, and size orientation and night mode. Those are a few ones in there. Finally, I would like to show you what comes out of the box of New Relic. This is browser monitoring. We have those core web vitals that user spent on site, initial page load throughput, and some other additional charts. This comes just out of loading a user agent. It’s as simple as adding a marketing tag, and this dashboards just magically appear. But if we go back to talking about UX and UI, why is this important? It’s like 68%. I’m talking here about eCommerce just because it’s the easiest, cleanest use case to see revenue changes when you deploy UX and UI changes.

Carolina Rotstein: We can see that a lot of eCommerce… 68% of them just had performance issues. And that translates into 40% of those issues resulting in revenue lost. For instance, most eCommerce retailers have reported that they would like to have a response time below two seconds. 90% of that website response time, on average occurs due to do the loading of front end resources. This is why it’s so important to start your optimization with your front end as well. Some core customer experience questions that real user monitoring will help you solve is, for instance, if your website is easy and friendly to use, that would be through the balance rate, for instance. Whether it’s easy to navigate or not, and that will be through the number of pages that your users take to get somewhere.

Carolina Rotstein: Ideally, you want to slice all those additional page views, just because you want a seamless interaction with your site. Just think about loading a YouTube video and having to click 20 times before you go to the music that you want. For instance, how easy it is to get in touch with a customer agent for your user. Now, not all sites want you to immediately get those agents, but those are done through custom events. You track that chat click, or that phone call on a mobile browser, as a success event. And just how comfortable your visitors are after landing in your site, can be done through a number of other metrics that we track on browser. Why is this important as well? Just to do it via RAM, is because it’s big data and data driven.

Carolina Rotstein: So if we look at traditional UX and UI optimization, it’s done through user research, such as interviews, focus groups, usability testing. And they would put a couple of people to see how easy it is to finish a task on their site, through surveys, AB testing sometimes, and session recording. Now, the size of this data tends to be, from my experience, to 100 people, to a thousand people. When we’re talking about big data, it’s millions of people. It helps us prioritize and not get narrow focus on the people for which we’re auto… well, for which we’re optimizing too. That is done to AV testing. Some companies that are very developed, they do multi-variate testing. So they have several versions of the same design, such as… Netflix is one of the big guys at the same time. They’re just running algorithms while they’re doing that.

Carolina Rotstein: The way they pass that data into their systems, is via an integration to their either browser or mobile tracking. Same session recording, conversion funnel. This is just big data that allows you to ultimately do persona building. In the particular case of New Relic, this is fully integrated with observability. Now, just because I’m coming from New Relic, I will like to show you how easy it is to implement custom events and attributes, which are those business events and additional metrics that I was talking to. This is just an example of a JavaScript snippet, and this is how you would pass it into New Relic. It gives you that full stack of observability that my peers for talking about. And that’s it.

Angie Chang: Thank you, Carolina. Our next speaker is Solmaira. She’s a technical account manager at New Relic, based out of Atlanta, serving as a technical advisor for enterprise customers in Latin America. She currently serves as chair of the Relics of Color ERG, which she’ll be speaking about today. Welcome, Solmaira.

girl geek x new relic solmaira flores valadez erg community

New Relic technical account manager Solmaira Flores-Valadez talks about finding community with New Relic ERGs at Girl Geek X New Relic virtual event. (Watch the talk)

Solmaira Flores-Valadez: Hi, everyone. My name is Solmaira Flores-Valadez, and I’m a technical account manager at New Relic. I’ve been with New Relic for about over two and a half years. I serve as pretty much like a technical advisor to some of our larger enterprise customers within the Latin America region. I’m like a post sales resource to them, helping them get the most out of New Relic, and also providing trainings, things like that, to make sure that they are utilizing New Relics to the best of their abilities. Today I’m going to talk about diversity, equity and inclusion, and the part that it plays in my life. How I was able to find a community with New Relic ERGs, which are employee resource groups.

Solmaira Flores-Valadez: A little bit about me. My pronouns are she, her, hers. I live in Atlanta. I went to the University of Georgia. I am a first generation Latina. Mexican-American. First person in my family to go to college. I am a woman in tech, and I’m also a dog mom. First I wanted to start off with a few definitions around what diversity, equity, inclusion are. And then I’ll jump in and talk a little bit more about what it means to me, how I got involved, and all of that. Diversity is the presence of differences that may include race, gender, religion, sexual orientation, ethnicity, nationality, socioeconomic status, language, disability, age, religious commitment or political perspective. These populations have been and remain underrepresented within the broader society, and within practitioners in the field as well, within the workplace.

Solmaira Flores-Valadez: Equity is promoting justice, impartiality and fairness within the procedures, processes and distribution of resources by institutions or systems. Equity is really the approach to ensure that everybody has access to the same opportunity. In the context of the workplace, how is it that employees have access to the same levels of attraction, promotion and retention within the company?

Solmaira Flores-Valadez: And then lastly we have inclusion, which is an outcome to ensure that those that are from these diverse backgrounds actually feel and or are welcome. It pretty much boils down to people with different identities, feeling or being valued. All right. So why is it important to me? I actually started doing D and I type of work long before I even knew it was D and I work. As I mentioned, I went to the University of Georgia. UGA is a predominantly white institution. So there was very little people that looked like me. I was always looking for my community, people that looked like me, that share common backgrounds, but then at the same time, got involved with certain organizations such as Students for Latino Empowerment, that not only helped build that community, gave me that social aspect in college, but also we were doing D and I work [inaudible] students into the campus. We have various events throughout the year, where we would pretty much show them the ropes, let them know, if I can do it, you can do it. They’ll be able to tour campus.

Solmaira Flores-Valadez: We would give them workshops around financial aid, how to get started, the college journey and all of that. That’s what sparked I guess that interest in being involved within D and I type of efforts. As I have here, it’s important to me, for me to lift as I climb to, to be that change that I wish to see in the world. And also D and I has been very important in my life, not only because I’m able to in a way give back, but also it’s helped me in my professional, in my personal growth. Being able to develop certain leadership skills.

Solmaira Flores-Valadez: A little bit about how I got involved at New Relic. I was involved in college with those type of organizations. As I left college and I moved on to my professional career, my first job I worked at a big accounting firm. I got there similar to when I joined UGA. Not a lot of people that looked like me. They had an Hispanic network. I joined that. We did a lot of social events, but at the same time, we also did a lot of also lifting as you climb, bringing in students. We also did events for students and things like that. I loved being plugged in, being that person, going to recruiting events and seeing others like me, and then being able to see that they could also do it.

Solmaira Flores-Valadez: And then after that I switched careers. I came over to New Relic. As soon as I joined, I let my manager know that I was interested in still being a part of something like this. I asked if he had a Hispanic network. He told me he wasn’t familiar if there was an Hispanic network, but there were employee resource groups, and got me connected to the person that was the D and I manager at the time. Met with her. I talked about my experiences. She got me connected to the Relics of Color, which is the employee resource groups for our POC at New Relic. I got to meet them. Loved what they were doing, got to participate in some of their events. When I joined, it was right around Hispanic heritage month.

Solmaira Flores-Valadez: I asked if there was anything I could help with. At the moment, the Atlanta office was pretty new. There wasn’t a lot of representation there. I told them that I wanted to host an event. I was brand new. I didn’t know how people were going to take it, but I knew that I wanted to do this. Since it was a little bit later in the Hispanic heritage timeframe, I decided to give a twist. We did a day of the dead event, which I have some pictures here. I put this together. We painted skulls. And then we watched Coco, and we also ordered Tamales and we had a really good time doing this event. That was pretty much my golden ticket into not only being a member of the Relics of Color, but becoming an executive, one of the ROC execs. After I did that, the leader of the Relics of Color reached out and was like, “Oh, I want you to be part of the exec board.” And then that’s how I got plugged in my golden ticket.

Solmaira Flores-Valadez: I loved it ever since. I’ve been able to help with a lot. Currently, I’m one of the co-chairs of the Relics of Color here. Have our exec board of our offsite that we had earlier this year, where we got together to build out our strategy, the events. We host and we celebrate different events throughout the year, black history month, Hispanic heritage month, Asian Pacific Islander month. Putting together content around that. And then on the right hand side, that was us at the sales kickoff. We managed to get people together before 8:00 AM or at breakfast. It was great. It was intimate. We had our relative color sponsor. Tracy Williams, she’s our chief diversity equity and inclusion officer, as well as our chief people officer there. Being part of the Relics of Color, being part of the exec board, has been, like I said, great.

Solmaira Flores-Valadez: I’ve been able to learn a lot, gain exposure to different things. For example, we meet with our C-suite on a quarterly basis. Being able to have visibility into the C-suite. And not only that, but be able to represent the Relics of Color as a whole, be able to communicate some of our challenges, what we’re doing, where we want to get, our goals. And then listening to us and our needs and seeing where and what they can do to help. It’s been great. On the social aspect, I’ve made really good friends, but also helped me grow professionally.

Solmaira Flores-Valadez: I talked about the Relics of Color, but we do have other employee resource groups. We have the women at New Relic. We have the veterans at New Relic, and we have access at New Relic, which encompasses neurodiversity, mental health and disability. Relics of Color, which is the ERG that I’m a part of. And then we also have our Rainbow Relics for LGBTQ plus relics. These are the different ERGs that you can get involved with. At Relic, we are working towards a more perfect New Relic. These are some of the initiatives that we have going on. We definitely believe that inclusion means everyone. We want to make sure that we’re having some progress. We understand that there isn’t always… or there is always more work that needs to be done, but we do value the progress over the perfection.

Solmaira Flores-Valadez: These are some of the initiatives that we have at New Relic. To help us accomplish that we have, for example, the Mikey rule, named in honor of our departed team, VP of engineering, who was the executive sponsor of our first employee resource group, which was the Relics of Color. This Mikey role focuses on sourcing and hiring more relics from underrepresented groups. Whenever we have an opening, this Mikey rule kicks in. We also have leader-led action plans. These were started in 2020 by our founder, Lew Cirne. He challenged the company to level up with D and I leader-led action thoughts, and maximize the recruitment retention and career growth for underrepresented groups. And now it’s one of the top level organizational priorities across the board for every single part of our business.

Solmaira Flores-Valadez: We also have D and I working groups. Our company leaders, like I said, sit down with us, with the ERG executives, to ensure that our commitment to diversity, equity, inclusion, is put into practice around the globe. Just wanted to call out some of our progress that we’ve seen with these initiatives. We’ve definitely increased our BIPOC engagement. We’ve also helped reduce bias. There’s different trainings that our managers have to take, every year or every so often around bias. We’ve also reached pay equity. There was an analysis that was made a couple years ago, that took a look at the pay, and made sure that everyone’s pay was equal. There’s been a lot of progress lately around career mobility, where we’ve built a lot of mentorship groups throughout the different businesses, to be able to help the career mobility of our underrepresented groups. And then as I mentioned, you also have the Mikey role, which focuses on the recruiting efforts. All right. Well, that’s all that I have for today. Thank you all for joining. Have a great day.

Angie Chang: Our next speaker is Nora, who is a solutions engineer at New Relic, where she advises enterprise clients on their observability engineering practices to answer the what, how and why of system performance. Her research focuses on application of blockchain, and she speaks Portuguese, Spanish and French, and resides in Florida. So welcome, Nora.

Nora Shannon Johnson: Hi, everybody. How are you all doing? Well, I’ll assume you’re doing fine because I can’t hear you, but can you see my screen?

Angie Chang: Yes.

Nora Shannon Johnson: Awesome. Okay. Cool. Like everyone else said, I don’t know enough to make any predictions that you guys could invest in, but anyway, welcome.

girl geek x new relic nora shannon johnson observability web

New Relic solutions consultant Nora Shannon Johnson talks about observability in the age of web3 at Girl Geek X New Relic virtual event. (Watch the talk)

Nora Shannon Johnson: Today I’m going to talk about observability in the context of Web3. A little bit about me. Like Angie said, my name is Nora Shannon Johnson. I’m a solutions consultant, which basically means that I help customers answer the what, how and why of system performance. Outside of work, I love languages and linguistics. I love planting things, but everything I’ve ever planted has died, unfortunately. So still working on that. And skateboarding. Today we’re going to talk about applying the principles of observability to Web3, and what the specifics of monitoring blockchain technologies looks like. I took an interest in this because I work with a lot of financial services and eCommerce organizations in Latin America.

Nora Shannon Johnson: The integration of blockchain into their existing business operations is a big question for them right now, for reasons that I’ll get into in a few minutes. This is not New Relic’s main use case, but as a solutions consultant, a lot of times customers come to you with their data, their technology, and their business requirements, and say, “Make it work.” Which is my favorite part of the job, when somebody says, “How do you do this?” And I say, “I don’t know. Let’s figure it out together.” So this is an example of doing that. Over the next nine or 10 minutes, we’re going to talk at a very, very high level about what Web3 is, why we would care about monitoring it, what specifically we would be monitoring what we want to look at. And then a quick example of what that might look like.

Nora Shannon Johnson: To get started, what is Web3? Web3 is the name given to the idea, and idea is a very keyword here, of a new sort of internet that is built using decentralized blockchains. As a disclaimer, throughout this entire presentation, I’m describing the idea, not the reality of what may come to fruition. Again, Web3 is powered by the concept of… or by blockchain technology. Blockchain is a relatively new method of storing data online. It’s built around two core concepts, those being decentralized computing and encryption. The fact that it is decentralized, means that files or data is shared across many computers or servers, rather than centralized in a single server or group of servers. You might hear it referred to as a peer to peer network for that reason. The fact that it’s decentralized also means that it’s immutable. You can’t change data on the blockchain, because in order to do so, you would’ve to corrupt data on every single machine that’s participating in the network, which is just really not feasible when you’re looking at large scale blockchain like Ethereum, which is the example we’re going to use.

Nora Shannon Johnson: And then the fact that it’s encrypted, means that people can’t access it unless they have permission to do so, and you can give and rescind access as you choose. So why would we want to monitor Web3? Frankly, for a lot of the same reasons that we already monitor the existing technology, the web two technology, so to speak, in the same industries. for financial services, that’s eCommerce integrating with blockchain for payments. It’s important to know that this isn’t just like cryptocurrency exchanges. This is brands like Gucci and the Dallas Mavericks and Microsoft, Whole Foods, even Save the Children. They all accept one or more cryptocurrencies for payments. Across a ton of different industries, this is an important aspect of their technology stack. We’ve also got healthcare. One of the driving or the driving use case for applying blockchain technology to healthcare, is to restore the rights of data back to users or patients in this case.

Nora Shannon Johnson: You would be able to give or rescind access to your health records to a healthcare professional, organization at will. Whereas right now your test records or health records are held in a database owned by maybe some company like Quest. And you wouldn’t really necessarily be able to remove it if you wanted to. And then finally supply chain. Supply chain is arguably at the enterprise level, the most interesting use case, the most sought after use case for blockchain. Specifically the validation of providence or origin and authenticity. Using a public ledger like Ethereum, you could actually trace the roots of a product that you purchased, to ensure that it is in fact organic or fair trade, or even from a location that you believe it to be from, which is pretty interesting use case. There’s many more, but in all of these use cases, we’re talking about people’s privacy, their security, their wellbeing. Obviously, their financial assets.

Nora Shannon Johnson: The fact that data on blockchain is immutable and that it’s decentralized, doesn’t mean that it’s immune to failure or to attack. It’s simply is creating this new monitoring paradigm. What might we monitor on the blockchain? I’m not going to explain all these, but like people before may have said, the slide decks will be shared out. But you might monitor something like a decentralized application or dApp. Decentralized autonomous organization or DAO. Decentralized finance exchanges or DeFi. And then non fungible tokens, which I think everybody’s probably familiar with. The infamous apes, or NFTs. Lots of acronyms here, cause it’s a mouthful. And then of course, you can monitor the blockchain itself too. Which is the example that we’re going to look at. We’re going to look at the Ethereum blockchain. If you’re not familiar, Ethereum is, you guess it, a blockchain platform with its own cryptocurrency. Ether, shortened to ETH. It’s also got a programming language called Solidity, which you can use to write smart contracts, and decentralized applications so that you can actually interact with the blockchain.

Nora Shannon Johnson: This is by far, especially for DeFi, the most popular blockchain, but there are a lot of alternatives that are gaining popularity, things like Cardano and Solana, because they’re faster and cheaper than working with Ethereum. Monitoring a blockchain or assets that are deployed to a blockchain, is going to include both the typical metrics and data types that we’d be used to seeing, as well as some that are specific to this realm. There’s three example categories here. We’ve got system performance, security events and business metrics. If you go from left to right, this is like more familiar to less familiar. Something like system performance is something that we’re very seeing.

Nora Shannon Johnson: When Netflix is… well, all the time it’s up and running, they want to know how quickly transactions are executing, the rate of error, as well as resource utilization. The only difference being in a world of Web3, this might be the number of nodes, but very similar to what we see today in terms of the number of idle versus busy workers. For security, this is very important. We’ve all heard about many attacks made to different blockchain or cryptocurrency exchanges. Things like changes to access controls, when there’s a lot of failed login attempts from especially specific IP address or geographic location where you don’t normally have those. And then finally, unusual transaction patterns. So there being a lot of transaction outside of your normal business operating hours.

Nora Shannon Johnson: And then all the way to the right. And this is where we see things that are more specific to the use cases that I described earlier. Things like measuring the gas fee. When you interact with blockchain, you have to pay a transaction fee. And it’s a dynamic transaction fee. It changes throughout the day. We’ll take a look at that in a minute when we get into New Relic. But that’s something you want to pay attention to, because whether you are paying that or receiving that, that affects your bottom line. You’d also want to pay attention to things like, the number of active users or wallet holders, the number of active connections. And then of course, the number of minors that are mining. And then finally something like the rate of currency being paid out. As you probably know, miners mine, because they get paid for it. But there’s a lot of blockchain platforms, especially ones that are oriented towards the arts and culture, where they will actually pay you for posting your content to their platform.

Nora Shannon Johnson: They pay the content creators. You want to know, again, whether you’re paying or receiving or you’re somewhere in the middle. As an integrator, you want to know what the rate of payout is. How might we do this? We know what we want to look at. We know the importance of it, but how might we actually do that? We’re going to go through very quickly, the two pieces that fit together, and then we’re going to look at what that looks like in an observability platform. In any situation, there’s two parts to monitoring. There’s the data and the platform. There’s, how can we get it, it being the data. And then the second part is, how do we make sense of it? Because just having the data is not super helpful to anyone unless you’re a computer.

Nora Shannon Johnson: We might use something like Web3.py, which is a Python library for interacting with Ethereum. You can do all kinds of cool stuff. You can read data, you can send transaction, you can even set the gas price if you are the owner. Not the owner, but you’re responsible for the operation of the blockchain. And so we see on the right hand side, we can import the Web3 library, confirm that we’re connected to Web3. And then as the last example shows, we can read block data or look at different people’s wallets. Here I’m pulling Snoop Dogg and Paris’s wallet balance. Which again, we’ll take a look at in a minute. This is the part of how we’re getting it. if you’re more of a fan of JavaScript, there’s also a Web3.js library that you can use instead. I’m just a Python loyalist.

Nora Shannon Johnson: And then the second part, which is, how do we make sense of it? Using a wonderful observability platform like New Relic, we can use an API suite to pull in the metrics, events, logs and traces that are important to us. And then you can look at all of the block statistics for the last hour, week or month or whatever the case may be. Let’s take a look at what that might look like if we were actually quoting it over to… oops. [inaudible]. I didn’t need to unshare. Let me reshare. Quoting it over to an observability platform. This is a simple use case, but this is just basically a dashboard that I put together. We’ve got our cool little… I don’t own. This guy, unfortunately, no Ethereum funds for that.

Nora Shannon Johnson: But we can look at changes to the different whales. Here I’m tracking people like Snoop Dogg. He’s buying and selling all over the place. We’ve got Lindsay Lohan, Mark Cuban. We can see the top miners for the period of time. The gas price, that’s what I mentioned earlier. And then again, the black activity, which is interesting to see. Whether you are responsible, you’re part of some exchange that is leveraging the Ethereum on blockchain, or you are a decentralized application developer, or maybe you’re just somebody that is posting their content or their NFTs to a blockchain. This is all going to be relevant for you. You can also make actionable insights based on the data that you poured over. If we take a look at, for example, logs. Logs are just like step by step data coming from either applications or servers or what have you.

Nora Shannon Johnson: And in this case, it’s coming from the Ethereum blockchain. I’m going to filter on a certain block number. Let’s do this guy. We can actually see. I’m going to shut this down. We can actually see the step by step of what it looks like. We can see that transaction was requested, that the block creation was initiated. Blocks were then sent to nodes in the network. It got validated. Transaction is now complete. And then they update that to… or they send that update to the network. The block gets added to it, and the proof of work is dispersed. People get paid on it. But as we know, things do not always work as we anticipate that they will in technology. So if we take a look at this other block, we can see that in this example, the transaction is requested, the block creation is initiated. The block is sent to the nodes in the network, but then the transaction is pending validation several times.

Nora Shannon Johnson: We might do something like create an automated remediation workflow here. Based on maybe the messages, the strings in this message, or repeated data, or examples of the same messages over a long period of time, we could actually set it up such that it automatically triggers external events based on what we see in the log messages. Again, this has been a very quick and very high level example of what you might see if you wanted to monitor something on the blockchain, and how you could make use of that information in a wonderful observability platform like New Relic. I hope you enjoyed it. Thank you very much for your time, and I hope to talk to you all soon.

Angie Chang: Awesome. Thank you, Nora. So Sarah is a solutions consultant at New Relic. She loves working with data, and in a previous life was a math teacher. She uses her skills to help customers use their own data to improve their uptime, performance resilience, reliability, and customer experience. Welcome, Sarah.

Sarah Hudspeth: Okay. Hopefully you can see me and my slide.

Angie Chang: Yes.

Sarah Hudspeth: Are we good? Okay. Hi. All right. Hi, all. I’m excited to talk to you all about APIs, and getting your data when you want it and how you want it. It’s a very common theme here at New Relic. We love data. We’re data nerds. And we have a safe Harbor just for legal purposes.

girl geek x new relic sarah hudspeth apis graphql

New Relic solutions consultant Sarah Hudspeth talks about REST APIs and GraphQL queries at Girl Geek X New Relic virtual event. (Watch the talk)

Sarah Hudspeth: A quick bio about me. Yes, I’ve probably been in tech for three and a half, four years. Before that I was a math teacher. I taught middle school and high school math. I did attend Hackbright Academy, and so I’m a boot camp graduate. If you have any questions about that, please reach out. I’m a mom of two, plus I have a puppy, a lab mix, and then you can see the hamster in the background.

Sarah Hudspeth: I’m a huge reader. And you’ll see one of the projects I walk you through is all about books. The last book I read was Stalingrad, super interesting. The best part of my job is working with customers and helping them solve their problems. And yes, we are all about data and using data. Feel free to put stuff in the chat or follow up with me afterwards. Here are my objectives for my talk. I am still a teacher at heart. I want you guys to understand what REST APIs are, how they’re used, what is GraphQL, and what are some interesting trends in APIs today? I want you to understand the difference between your REST APIs and GraphQL APIs, and possibly articulate use cases for each. Also we’re going to be talking about GraphQLs, query and mutations.

Sarah Hudspeth: I’m just making sure you understand the difference between that. And then I am giving you all homework. After this session, if you haven’t played with API calls, go find some APIs, play with them. Go play with GraphQL, do some queries and mutations. If you need a GraphQL API Explorer, New Relic, you can sign up for a free account and play with our GraphQL API, which we call NerdGraph. Feel free to do that. APIs. API stands for application programming interface. It’s basically a way for clients and servers to talk to each other. It’s a set of protocols and it’s called a REST. I’m going to be talking about REST APIs, because those are usually the ones I would say they’re the most popular. And REST is short for RESTful, meaning stateless.

Sarah Hudspeth: The state of the client doesn’t affect the state of the server. They should be able to talk no matter what’s going on within their own environments. I like to think of API calls as programs, throwing Frisbees back and forth. Even though the Frisbee is actually data. But a client will make a call, throw the Frisbee to a server. The server gets the Frisbees if there are any instructions, and throws the Frisbee back as a response. If all goes well, you get a 200 response. If it doesn’t, you’ll get one of the four hundreds or five hundreds based on whatever the errors are. Let’s take a look at what an API call looks like. This is code from my virtual bookshelf project that I did at Hackbright. I allowed folks to build out this visual bookshelf of the books they were reading.

Sarah Hudspeth: The main API I used was Google’s Books API, where I could get a thumbnail of a picture of the cover of the book, and a lot of information. When I was feeding my database, I had a list of titles and authors, and then I made a call to the Google API, Book API, using my Google API key. I used the Python HTDP request library to get that information. And then I stored the response in a dictionary in JSON form. so that I could fill out my database with all sorts of interesting things. Hold on. I was going to say, there’s a few things. We have a URL. I have some variables in here, parameters that changed that I had to go through in a for loop.

Sarah Hudspeth: And then I also needed permission to have access to APIs. Those are the key components of an API call. This was one book. This was one response I got back from Google’s API. There’s a lot going on here. I would say, there’s a lot of information here, some of which I needed, a lot of which I didn’t need, and I had to sort through it and figure out, what is going to be helpful to me in my project, and then get rid of the rest, which if you notice, is a lot of waste. My code was not optimized. This was the slow part of my program, which if I go back, I would focus on this and try to do this in a better way, just because it ate up so much [inaudible].

Sarah Hudspeth: To summarize, I showed you what the components of the REST API and the results are. You have to have a URL. You have to call to someplace. You can send parameters on variables. I did title and author. You usually need a key to access the APIs, so you have permission to get the information. You need some HTTP requests. I use the Python’s request, but I’ll show you a cURL snippet when I do the GraphQL. The other interesting thing to note is that each API, you can call various APIs, will have their own way of formatting the data. Google Books API – just sent me everything I could possibly need about a book. And it was up to me to go through and figure out the structure of it.

Sarah Hudspeth: I showed you a get REST API call, but there are also posts where you can actually post data to the API. You can update data or you can delete it. I said, “This is kind of ugly data.” There was a nested JSON. I found out the hard way that sometimes some of the things I wanted were empty, and I had to find workarounds. I had to go and I had to clean up and structure the data. There even updates and the data would get restructured and I’d have to go back and figure out how to do that. I’m glad to now transition to a new way to get data, called GraphQL. It is also an API, but it is a very structured way we can access data. This is an example of New Relic’s NerdGraph API Explorer.

Sarah Hudspeth: And if you notice, I have to my left, a query builder with very specific key value pairs that I can build out for a query. Here I’m going to query an account and get the name and ID, and here I’m going to do an entity search. You all have been hearing us talk about observability, and learning about applications and performance and getting metrics and events. This is a way you can go in. I’m just going to get the name of things I want to monitor, the type. I’m going to get a special GUI. And then I’m just actually going to get the tags that I’ve tagged with my entity. It’ll pop up here in a very nice structured JSON. I know exactly how many levels I need to go in to get specific information. And then here’s how you could do it and build in a program.

Sarah Hudspeth: We talk about automation and observability as code. It’s really easy to take these GraphQL calls, and build in structures and processes to get the information that you can then take action on. Again, here’s just the API link. I have some headers with my key. And then here, I’m sending this query that’s going to go to the GraphQL server, and pass back all this information about this application, name, box, that’s in development. All right. Let me quickly summarize what we did or what I just showed you with GraphQL. Instead of posting or getting data, we’re going to query data and mutate data in order to update it. You might see that you can use GraphQL iteratively. I had that GUI ID that I could query for and then use it to change of I needed to update the application, add it to an alert policy, add it to a dashboard.

Sarah Hudspeth: It’s nice that you can just build off each other. I know exactly what data I’m going to get, and I’m only going to get that data. It’s going to be nice and structured. It’s going to be fast. I’ll tell you right now, New Relic is powered by this NerdGraph which you saw. That data that we accessed, we inside our platform also use it to access… or to build out all the dashboards and charts. I should say that GraphQL was developed by Facebook in 2012. Obviously when you’re processing that much amount of data, you want to be specific about the data you get, and get it as quickly as possible. The one downside is it does require a lot of upfront work. You have to build out that data schema so that folks can get the access.

Sarah Hudspeth: But once you have it built, you have a very powerful GraphQL engine. There’s some other cool things. I was going to say, with my API call, I had to call it many times in that for loop, because I could only get one book at a time. In GraphQL, you can make multiple calls even to multiple servers to get multiple data requests. It’s just a lot more robust and flexible. I’m quickly going to go through this slide. I think from the other talks, you’ve seen how we use data and how we want access to data and how we want to build it out programmatically, and automate and really be able to empower our data to… or empower our customers to use their data in a lot of different ways. Some of those are alerts. Getting alerted on any issues, updating with microservices and Kubernetes. You can spin things up, spin things down. You need to add them to alert policies or delete them.

Sarah Hudspeth: I also work with customers a lot about either storing or dropping data they don’t need. Sometimes companies need to store their logs to be in compliance with certain data rules. And so we can export data rules and NerdGraph to AWS buckets so they meet that requirement. We did talk about dashboards and S… or others talked about dashboards and SLOs. You can update dashboards with GraphQL. You can add things, you can subtract things. You can actually have a call to get a PDF. So if you need to email it to your superior and be like, “Hey, look at our application performance for the week.” You’re able to do that with a GraphQL API call, and then synthetics as well. If you want to check on Ping Checks if anything’s failing, or if you need to update, add end points, you can all do that in GraphQL.

Sarah Hudspeth: I think I’m good on time. I was just going to quickly show you how you can build out the query in the query builder. Let’s see. Maybe I’ll get the synthetic monitor. If I just wanted a list of synthetic monitors, I could just click whatever I wanted to see. I could add here. And when I press play, it just comes up to the right. I did add a permalink. So if maybe there was something I noticed, it was a critical learner. When I wanted to go check it out, I could quickly copy and paste or build out a script to go into New Relic and see what was happening. Looks like this check is okay, but I can go in and get that view. If I wanted to mutate, I could just continue to build out.

Sarah Hudspeth: Let’s say I wanted to create a workload. I could build out a workload using whatever data here. You can use the cURL up here. You could use our New Relic command line interface. It’s really flexible and robust. For all the data nerds out there, it’s just really fun to use. That was my talk. Hopefully you picked up a lot or a little about REST APIs and GraphQL and the differences. Just wanted to let you know, my team is hiring, so please reach out. Tap me up if you have questions, but thank you for listening.

Angie Chang: Thank you, Sarah, for the talk and demo on GraphQL. It’s very informative. I’m sure people have lots of questions will like to connect with you. So thank you so much. Our next speaker, we’re going to try Jo Ann again.

Angie Chang: Jo Ann is a senior technical account manager at New Relic. Has been working directly with customers, helping them use and implement the New Relic platform, including best practices. Prior to that, she was a solutions architect at Delta Airlines in Atlanta. So welcome, Jo Ann.

girl geek x new relic jo ann de leon react js

New Relic senior technical account manager Jo Ann de Leon talks about programmability, React, Nerdpacks and much more at Girl Geek X New Relic virtual event. (Watch the talk)

Jo Ann de Leon: Thank you, Angie. All right. Hello, everyone. I am Jo Ann de Leon, and I will be talking about the power of ReactJS and how it transformed the New Relic platform to be an open connected and programmable platform. Before I get started, I’d like to share some tidbits about myself. I am a senior technical account manager. I have been with New Relic for three and a half years, working directly with customers, acting as a technical advisor and solutions architect, to help them implement their observability use cases. I was born and raised in the Philippines. I graduated with a math degree, but never really thought I’d work in the IT industry. But in the past 20 something years, I have worn a lot of different IT hats, including a software developer, a designer, architect and project manager. Outside of work, my wife and I enjoy traveling, playing bocce, and cuddling with our two adorable orange tabbies.

Jo Ann de Leon: For this talk, I will introduce the concept of programmability. Show where you can find some of the open source apps and custom visualizations. And finally do a quick demo of how you can build your own. In a nutshell, programmability is about giving engineers full access to the New Relic database engine, and the building blocks they need to consume data in ways that solve their unique business problems. It also means giving our engineer users and customers the same set of tools our own engineers use to build our platform key rated experiences. What does this look like?

Jo Ann de Leon: At its foundation, is the telemetry data platform, that is able to ingest not just the data from the New Relic agents, but also from integrations that support open standards such as open telemetry. On top of this data platform, is a series of scalable services such as GraphQL APIs, as well as the developer tools, such as the software development kit or SDK for short, and the command line interface, or CLI for short, that allow you to access and interact with the data. Finally, a user interface is built on open source React JavaScript, with a flexibility to support the development of custom applications and visualizations. If you’re like me, I find it really helpful to look at what others have already created before I try to write a piece of code. It’s a good idea to explore what is already available in the open source community, as it may help inspire you to build your own New Relic custom application and visualization.

Jo Ann de Leon: The first place to explore is the New Relic Instant Observability or IO, which you can find via the apps icon in the New Relic toolbar. It contains a catalog of public apps and visualizations that are maintained by New Relic, and can be managed via the UI. The catalog also allows you to manage your own custom apps. You can find a number of other open source apps and visualizations in the New Relic open source website. The great thing about open source is that these apps are extensible, meaning you can customize them to fit your needs, and you can easily install them via the CLI.

Jo Ann de Leon: Here are a couple of examples that I wanted to showcase. The first one is a cloud optimized application, which analyzes your cloud environment, figures out where you’re wasting money on excess cloud capacity. The application compares the size of your instances to their utilization, finds resources that are sized larger than needed, and estimates how much you could save by optimizing the resource size. The browser analyzer app displays an analysis of performance, and forecast how improving the performance of your website can impact your key performance indicators, such as bounce rate or traffic. It also figures out which individual site pages have the worst impact on performance, so you know where to start making fixes and improvements.

Jo Ann de Leon: A popular visualization is the status widget pack, which contains three types of visualizations. One of those three is this status timeline widget, that allows you to display how your services are performing over time using traffic lights as visual indicators. Now it’s time to build our own app.

Jo Ann de Leon: I will show you how to build a Nerdpack, which is the deployable package of an application containing all the source code and resources required to run it. It is basically a collection of React components, including launchers, nerdlets and virtualizations, all structured into a JavaScript app bundle. A launcher is a declarative file. It allows you to configure your application’s name and description, as well as which nerdlet within the nerdpack to run when it is clicked. An application is made up of one or more nerdlets, which are renderable views or windows. So they can link to each other or be launched by launchers. And finally, a visualization is a custom view or widget that can be added to a dashboard. Similar to nerdlets, it can display data whether it’s from New Relic or an external data source. You can find them via the custom visualizations app.

Jo Ann de Leon: All right. In an alternate universe, I have open a number of cat cafes around the country, where I serve coffee and cute cats or lunging around to entertain my customers, who may then fall in love with them, and decide to adopt them. In order to achieve my goal of helping these cats find their forever home, I need to keep track of how many have been adopted, and how many are still up for adoption. I went ahead and sent this data to New Relic, but how do I visualize all my data since I have so many cat cafes around the country. Luckily, I can build an awesome nerdpack. So let’s go ahead and create it.

Jo Ann de Leon: I am in the New Relic homepage. I hope you can still see it. In the New Relic homepage, you can go to the apps and click on build your own app. You can follow these instructions in the quick start. If you haven’t already done so, you can create an API key in your New Relic account, or select an existing API key. This is where you can download and install the NR One CLI, and make sure that it is up and running. And then the last step before you build your nerdpack, is to save your credentials. Let me copy this, and we’ll go ahead and create the package and run it. I am going to name my nerdlet as cat café tracker, and launcher as cat café launcher.

Jo Ann de Leon: Install the dependencies and create all the different components needed for my app. And then I can go to that NerdPack and let me open this in my Visual Studio Code. All right. Let me open the Terminal here, and then I can run my server through the New Relic One CLI, with this command: nr1 nerdpack:serve. All right. You will notice that now you can run one.newrelic.com with nerdpacks=local. This means that any local development you make can be tested in the New Relic platform. You’re also given a shortcut to the launcher, which will open your Nerdlet directly. So let’s go ahead and copy that. And let’s go back to the browser here, and let’s close this prompt.

Jo Ann de Leon: And now we have our Hello World version for our cat cafe tracker, but it’s not really very exciting. Let’s go back to our code. For the sake of time, I will be copying and pasting my code, including index.js. This code will contain the logic to retrieve the data from the database and display it in two views, a table view and a map view. Let me go ahead and do that.

Jo Ann de Leon: And then I also need to update my styles.css. This will contain styling elements for my custom UI. All right. Third one. I need to update my package.json dependencies, because we will be using the leaflet package to create a map. All right. And then finally, I need the webpack config, which we will need to support the use of map tiling information data from leaflet. This will be copied at the root folder of our package. All right. Let me save all of that. I have to restart my server. Let me clear that. I have to do an npm install first, since I had to update my package.json. And now I’m going to restart my server and relaunch my app. All right. Let’s copy that new link. Go back to our browser.

Jo Ann de Leon: Hopefully this will work. There we go. All right. So now I can view all my cat cafes around the country. I created my visualization such that the size of the circles indicates how many cats are available for adoption in that area. The bigger the circle, the more cats are available. The green color means more cats have been adopted, while those that are yellow or dark orange means we have some work to do to get more cats adopted. Finally, I have also displayed my data in a table view to the side of my map. All right. I hope you have enjoyed this quick demo on how programmability through the use of ReactJS can help you create visualizations that focus on solving business problems. Please feel free to connect with me through my email or LinkedIn. Thank you.

Angie Chang: Thank you, Jo Ann, for that talk and demo, and we’ll be sure to connect with you on LinkedIn. Our leadership panel will talk about New Relic culture, inclusion, career development, and successful interview prep.

Angie Chang: Our moderator today is Ariane Evans. She’s a diversity equity and inclusion manager at New Relic, working with the talent acquisition, hiring managers, employees, and external organizations to recruit, engage, develop underrepresented communities. And she co-leads the Relics of Color ERG. Welcome, Ariane.

new relic girl geek x panel discussion

Ariane Evans moderates New Relic leadership panel with Nada Da Veiga, Erin Dieterich, Kim Camacho, Tracy Ravenscraft, and Stefanie Smith. (Watch on YouTube)

Ariane Evans: Thanks, Angie. Hi, everyone. My name is Ariane Evans. And as Angie mentioned, I’m a [inaudible] manager at New Relic. I love that I get to spend a little time with you and facilitate a conversation with some of our incredible leaders at New Relic. All of them, women. It’s so inspiring to have leaders that are not only passionate about their work, but the communities that they work within. Before I dive into the questions to know more about New Relic and the areas of expertise of each of these leaders, let’s go through a quick lightning round of introductions. Please give me your name, title, and a sweet little fun fact about you. Let’s start with Kim.

Kim Camacho: Hi. Hi, everyone. Happy pride month. My name’s Kim Camacho, and my pronouns are she and her. I’ve been the director of DE&I at New Relic for about a year, and have also about 20 years of DE and I and HR experience. A fun fact about me is, I met Barack Obama right after he announced his candidacy for presidency a long time ago. So that is fun fact

Ariane Evans: Very cool, and also now very jealous. Let’s go ahead and hear from Erin.

Erin Dieterich: Very jealous of that fun fact. Hi, I’m Erin Dietrich. I lead the social impact and environmental, social and governance organizations at New Relic. My pronouns are she and her. I’ve been at New Relic for about four and a half years, and I’m based in Portland. My fun fact is that I have two small children, a one and a half year old little girl and a five and a half year old little boy. And they keep me incredibly busy, and very tired all the time. I don’t think I’ve slept well in five years. Fun fact.

Ariane Evans: Well, you look great even on little sleep Erin. Thanks for joining. Let’s hear from Tracy next.

Tracy Ravenscraft: Hi, my name is Tracy Ravenscraft. I’ve been here at New Relic for about five and a half years. I run a technical account manager team in central. My fun fact is I have two dogs, one Pomeranian, one Pomsky, and they have names like Friends characters, so their names are Phoebe and Ross. Thank you.

Ariane Evans: Love a good Friends joke. Let’s hear from Nada next.

Nada Da Veiga: Hi, everybody. I lead customer adoption organization. America’s customer adoption organization here at New Relic. Been here for five years. If you’re wondering what customer adoption is, basically, all engineers that work closely with our customers, helping them learn how to use our platform to solve their technical and business problem, basically. Fun fact: throughout my life, I have had five different passports. So no, I’m not a female version of James Bond, but that’s what my husband likes to think.

Ariane Evans: I might also think of a reference to Carmen Sandiego. Where in the world is Debeka? Where is she going next? Next let’s have Stephanie.

Stefanie Smith: Hi. Thanks, Ariane. I’m Stephanie Smith. I’m based in Massachusetts, I’ve been with New Relic for six years. Currently senior manager of talent acquisition. My team supports go to market customer adoption. Let’s see. Fun fact about me is I have two teenage daughters, one of which just graduated high school last weekend, which is very hard to believe, and a younger one. She’s a sophomore, she’ll be a junior. Erin, the exhaustion doesn’t stop. It only gets different. It’s bigger problems with bigger kids, but it’s all worth it. Fun ride for sure. Excited to be here.

Ariane Evans: Thanks, Stephanie. And thank you all. We all just listened to quite a few talks learning about why observability is important. What is monitoring? How do we implement these different products and technology? And also this happens inside of a company where the people work together. There’s a culture that allows us to do that work at our best and highest potential. I’d love to hear from each of you on how you are not only living those in practices, but working that out in your teams and your strategies at New Relic. I will start with a bit of our culture and understanding how is New Relic creating a culture where people from all backgrounds feel included.

Kim Camacho: All right. I could take a step at that, Ariane. First and foremost, I think we are very clear about our commitment to diversity, equity and inclusion. We communicate our vision, mission and objectives annually as we build out our short term annual plans and our long term strategy. All new employees and interns hear about our strategy as well as our organization when they onboard. We measure regularly how employees are feeling. The extent to which they feel belonging and respect to the company. So important to do that. I think also for our employees, one of the big things that’s really important is having communities of people that you can bond with, that are recognizable to you and have the same interests and backgrounds that you have. We have employee resource groups at New Relic. They’re fully funded and have leadership organizations as well as executive sponsors.

Kim Camacho: It’s through these organizations that we hope that people are building relationships, bonding, getting to know each other outside of their regular roles. In addition to our ERGs, we have other slack channels based on whatever people want to connect with. Whether it’s dogs, bunnies. There was one that was just started on crime channels, which I’m in love with, so you can bond. The last thing I’ll say, as it relates to really creating a culture where people feel connected is, the importance of managers. I think as our audience will know, your manager makes a big difference. Here at New Relic, it’s really important that we support train, help our managers really understand cultural competency, how to build a diverse and equitable workplace. Everyone I think on this call knows, because they’ve been through some of our trainings and are actively involved in these efforts. It’s just really important that we’re working with our managers so that they understand their role in helping create a nurturing environment for our employees. That’s a little bit about from that perspective.

Ariane Evans: That is all really cool. I know that there are also more things that New Relic is engaging. Erin, maybe you can tell us, what is New Relic focused on, or engaging our employees and social impact.

Erin Dieterich: Yeah, thanks Ariane. Newrelic.org is the name of our social impact work. We started it in early 2019, and really committed at that point to this mission of, how do we as a company, continue to push for more equitable access to technology? We really believe that accessing not just physical technology, having the best computer, having the best SaaS tools, but having the access to understanding what technology careers actually look like, what kinds of roles there are within technology. That is such a critical piece to creating this more equitable future for the industry, and to thinking about, how do we help people all along their learning journey? Whether they’re somebody who’s had a couple careers already, and are starting a career in technology, or a student who’s early in life thinking about what they want to be when they grow up. How do we give all of those people access to our incredible employees, so that they can hear the stories about how we all got where we are, and be able to start seeing themselves on this whole rainbow of pathways.

Erin Dieterich: It is not just one clear, point A to point B gets you a tech career. There are so many different ways to get where you’re going, and so many different destinations along the way. And so we’re just really passionate about infusing that into everything we do in social impact, and thinking about how we take the 2000 plus employees around the world with us on that journey. Some of the ways that we do that, we have a bunch of benefits that all of our employees get access to. They get to have 16 hours of paid time off to volunteer a year, plus we now have a set global day of service every winter. That’s three full days of volunteering, and you can slice and dice that however you want throughout the year. We incentivize our employees using that volunteering, by actually giving them dollars that they can push towards their fair charities every time they log their hours of volunteering.

Erin Dieterich: We have a $200 a year matching program. Employees can get up to $200 a year matched to any number of global charities. I think there’s 20,000 charities that they can pick from. And then we do a bunch of special campaigns. And so some of the things I really love that we’ve been building and you’ve actually been a big part of building these with us, Ariane, are some of our partnerships with our employee resource groups. Where we’re really going to our employee resource groups and helping them give us the understanding of where they want to impact in their communities, what organizations they want to work with. And then working together to make sure that that information is accessible to our employees, to incentivize and point them towards making really smart decisions with their wealth of how they can build this more equitable future.

Erin Dieterich: A great example of that is, since it’s June and it’s pride month, we are working with our rainbow relics ERG and just launched a $25,000 additional matching campaigns. In addition to those $200 employees have, they now can also put additional dollars towards this matching campaign, that goes to five different organizations that our rainbow [inaudible] helped us identify and pick in their communities. Organizations that they really care about that are helping the LGBTQ community with all of the different things going on, both in the US and abroad. Being able to be a part of understanding what that ERG community wants employees to support, and then helping employees understand how they can use their dollars to support their fellow relics, and the things they care about, is something that just makes me so excited.

Erin Dieterich: I just love seeing the way our employees are supporting each other through those special campaigns. I think I’m almost out of time, but I’ll tell one other very quick story, which is, since we have so many technical and inspiring folks on this call, I always like to take the opportunity to just pause and remind folks how valuable your skills are. Technology skills are so incredible. There’s a myriad of ways you could apply those to social good. Something we love to do is partnering our employees up with our nonprofit customers who get to use expanded access to New Relic for free. But we know that they need help with enablement. And so we partner them up with employees and the employees take on pro bono volunteering projects, where they’re using their technical skills to really support observability in nonprofits.

Erin Dieterich: And so you don’t have to be a New Relic employee to do something like that. You can really step back and say, “What causes are super important to me? What organizations do I love?” Reach out to them and say, “I want to talk to whoever’s running your technology, and see how I can be of support. I have X, Y, Z skillset that I’m really proud of. Is there a project I could help you on pro bono, and volunteer and support your organization, building your digital environment?” Because that is what every organization needs in order to power their mission. Every person with technology skills has just so much that they can give back. And so we love to do that at New Relic, but I also just love to encourage anyone anytime I can, to think about how you can use your skills out there in your community to power the charities and the causes that you care about.

Ariane Evans: Thanks, Erin. It sounds like New Relic is really building out a culture for people to live a life fully as they’d like, both internally and their communities. The things that they care about, but also themselves wholly. I’d love to hear from you, Tracy. Describing to us, which areas of your life would you like to spend more quality time when you think about work life balance.

Tracy Ravenscraft: That’s a great question. Thank you. When I think about where I like to spend my time outside of work, definitely with family and friends. Everybody wants that more family, friend time. But not only just spending more time with them, being present. Not checking my phone for slack messages, going on vacations and being able to completely disconnect. That’s what New Relic has brought to my life.

Tracy Ravenscraft: I’ve been at New Relic for five and a half years. I did site reliability in the past, network administration, network engineering. I never realized how I wasn’t there. I’m always looking for the next page. When I have time off, I’m bringing my laptop, I’m bringing my phone. I feel like New Relic, with our recharge week, which the summer that we all get off at the same time. FTO, so it’s flex time off. There’s really no limit to my vacation. Just some of the applications we have, like Ginger, that helps with mental health. I really feel connected when I’m using my own personal time and being with my family. So yeah, that’s how I like to recharge, if you will.

Ariane Evans: Yeah. So important. When you are moving on to the next project, or you are trying to get to the deadline of a particular thing, you can’t do that if you’re empty, and you don’t have the energy within you. And so I guess moving on, switching gears a little bit, want to talk with you, Nada, about navigating careers and career challenges. Career journeys can vary person to person. As Tracy just described, she’s been across the board of different kinds of engineers, and now a customer adoption leader, but how might you recommend navigating a career journey, and even a career journey into leadership?

Nada Da Veiga: Yeah. I mean, I think that’s an excellent question really. What I, or what we in general try to encourage folks in my organization, is to own their career, and be really proactive about it. And so a lot of people early in their careers think that they should somehow just wait for their manager to have these types of conversations. I would say quite the opposite. Be proactive about it, ask questions, share, what do you want for yourself? Where do you want to be three years from now, five years from now? Ask your manager, “What do I need to do to get there?” Because if you are informed and you know what this person expects from you, what three, five things they want to see from you in order for you to actually make it there, guess what? You have a lot higher chance of getting there, than if you’re just sitting and waiting for them to tell you, because they may or may not tell you actually.

Nada Da Veiga: They may or may not understand that you want to get from this role to some other role. That is what we see a lot with our teams. At New Relic, we are very much committed to our employee’s career progression. These are proactive conversations that are happening continuously. We encourage our employees to put together their career plans, to share those with their managers. And then some of them just want to go, “Hey, how do I go from this role that I’m in today, maybe to a senior role or a principal role?” Others want to move maybe from one org to a different org, so they want help with that path.

Nada Da Veiga: Third group will say, “Well, I want to get to leadership.” But I think how you approach it really doesn’t matter. New Relic specifically, if you are interested in leadership, we have about 14 different management classes that we recommend to folks that are setting you on that leadership path at New Relic. But whether you’re at New Relic or somewhere else, show your manager, show your leadership what you really are interested in, where your heart is at, and be proactive about it. That’s probably the best advice I can give.

Ariane Evans: Yeah. I love that. I will say that I think my career journey at New Relic is a testament to that. Starting in talent acquisition and getting to be a partner to Stephanie, but then moving into social impact and getting to learn from Erin, and now today, being a part of the DE and I team, and getting to work very closely with Kim, and that has all been championed by New Relic and the leaders within… I just said, “I’m interested in this thing, and I’m not really sure where to go from here.” But it did start with an interview. It started with a conversation with my manager. And so I’d love to kick it over to you, Stephanie, and think about, for a lot of people, getting started in your career, or looking for new opportunities, it starts with that interview process. You’ve interviewed hundreds of people in your career. And now as a recruiting leader, what is the best advice you have for anyone that is preparing to interview or in the process of interviewing currently?

Stefanie Smith: That’s a good question. I do want to just talk about just quickly, Ariane, your career progression. There’s so many people at New Relic that have had career progression, me included. I started off as a recruiter, and promoted along the way to senior manager. So there’s so much opportunity. But yes, there is an interview that’s involved. Interviewing with the company really, it’s your first impression, but it’s also our first impression to you as well. I always tell people that it’s your interview as much as it is ours. Make sure you qualify. Know what the company does. Really know what the company does. Do some research, do your homework. There’s a wealth of information about companies on the internet. It’s incredible. Link in with people on LinkedIn. Understand the roles and responsibilities, what people are.

Stefanie Smith: And then when you are talking to someone, likely it’s going to be a recruiter first, it’s a conversation. Like I said, really, you’re qualifying us, we’re qualifying you. Part of our core values is being authentic. I think that you’ve probably seen a lot of authenticity throughout this entire panel, and previous to the speakers. Be authentic during the interview, be yourself. Find some common ground. Look at it as just a conversation. Working, we spend more time than anywhere else. New Relic encourages everyone to be their best authentic self. When you’re in the interview, just really be yourself and ask good questions, and talk about career pathing and all the things that are important to you.

Stefanie Smith: Realize, if this is the right company, position, and so forth. And also even ask for guidance along the way. Your recruiter’s going to be the first step, and the recruiters are going to send you on for the next interview. Connect with the recruiter as often as possible. Even connect with the people that you’re interviewing with. We have multiple steps of roles when we interview here at New Relic. People are always going to be available to help guide you through the process. Ultimately, like I said, it’s your interview as much as it’s ours.

Ariane Evans: Yeah. I totally agree with that. Since we’ve also wrapped up this time with all of our leaders, I want to thank Girl Geek, thank New Relic for also putting this together, and everybody for listening in. I hope that you’ve gotten to pull out some nuggets of advice that are beneficial to you. If you are interested in learning more about New Relic or careers or opportunities, there are some things that Kim dropped into about our ERGs and our benefits. Please take a look at newrelic.com/culture. It will take you to our careers page and the opportunities that are currently live across the world. There are many.

Angie Chang: Thank you, Ariane, for moderating the panel, and to all the panelists for joining us. So now is time for our networking session. If you can click on the link at the bottom of the chat to our Zoom meeting, we can go into a Zoom meeting and have some breakout rooms where we can meet each other in person, and chat a little with our remaining 15 minutes that we have today. So if you can click on that link in chat that Amy has added, I’ll see you over at Zoom meeting and talk to you there. Thanks for coming.

new relic girl geek x speakers live square

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Posted on

Girl Geek X MosaicML Lightning Talks with OpenAI, Meta AI, Salesforce Research, Atomwise, Amazon Web Services, Hala Systems (Video + Transcript)

mosaicml girl geek dinner outdoor patio networking dinner palo alto playground global

Over 120 girl geeks joined networking and talks at the sold-out MosaicML Girl Geek Dinner from women working in machine learning at MosaicML, Meta AI, Atomwise, Salesforce Research, OpenAI, Amazon, and Hala Systems.

Speakers discuss efficient machine learning training with MosaicML, reinforcement learning, ML-based drug discovery with AtomNet, evaluating recommendation robustness with RGRecSys, turning generative models into products at OpenAI, seeking the bigger picture at AWS, and more.

Table of Contents

  1. Welcome – Julie Choi, VP and Chief of Growth at MosaicML – watch her talk or read her words

  2. Making ML Training Faster, Algorithmically – Laura Florescu, AI Researcher at MosaicML – watch her talk or read her words

  3. Reinforcement Learning: A Career Journey – Amy Zhang, Research Scientist at Meta AI – watch her talk or read her words

  4. Addressing Challenges in Drug Discovery – Tiffany Williams, Staff Software Engineer at Atomwise – watch her talk or read her words

  5. Evaluating Recommendation System Robustness – Shelby Heinecke, Senior Research Scientist at Salesforce Research – watch her talk or read her words

  6. Turning Generative Models From Research Into Products – Angela Jiang, Product Manager at OpenAI – watch her talk or read her words

  7. Seeking the Bigger Picture – Banu Nagasundaram, Machine Learning Product Leader at Amazon Web Services – watch her talk or read her words

  8. 10 Lessons Learned from Building High Performance Diverse Teams – Lamya Alaoui, Director of People Ops at Hala Systems – watch her talk or read her words

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Transcript of MosaicML Girl Geek Dinner – Lightning Talks:

Angie Chang: Thank you so much for coming out. I’m so glad you’re here. My name is Angie Chang and I’m the founder of Girl Geek X. We’ve been doing Girl Geek Dinners in the San Francisco Bay Area for, if you can believe it, almost 15 years now. It’s the first Girl Geek Dinner in over two years, because Julie is a rock star and wanted to do a Girl Geek Dinner in person in the pandemic and we’re like, “Yes!” It was postponed and now in May, we’re finally doing this event. I’m so glad that we have a sold out event of amazing women in machine learning that we’re going to be hearing from tonight!

mosaicml girl geek dinner sold out angie chang speaking girl geek x

Girl Geek X founder Angie Chang welcomes the sold-out crowd to our first IRL Girl Geek Dinner in over two years during the pandemic! (Watch on YouTube)

Angie Chang: I don’t want to steal too much of the time, but I wanted to do a quick raffle of a tote bag that I have. I’m going to ask, who has been to the first Girl Geek Dinner and can name a speaker from that event? Sometimes I meet people who have. Are we making this really hard? Okay, the first Girl Geek Dinner was at Google. We had over 400 women show up for a panel of inspiring women. I just wanted to see because that’s a lot of people. Who thinks they’ve been to the most Girl Geek Dinners in the room? Okay.

Audience Member: It’s actually not me, but like, this clutch was designed by somebody who hates women because its super heavy – and I see that that [Girl Geek X tote] has handles.

Angie Chang: Okay, so we have a full agenda of machine learning lightning talks and I’m going to introduce you to our host for tonight. Julie Choi is the Chief Growth Officer of MosaicML and she is an amazing supporter of women and I would like to invite her up.

Julie Choi: Oh, thank you. Hi everyone. Thank you so much for coming out to the MosaicML Girl Geek Dinner. I am Julie Choi and I actually did go to the first Girl Geek Dinner. I want to thank Angie Chang and the Girl Geek Dinner organization. Angie has just been a pioneer and truly, just a very special person, bringing us together ever since, was that 2010 or something, I don’t know. When was that?

mosaicml girl geek dinner julie choi

MosaicML VP and Chief Growth Officer Julie Choi welcomes the audience. She emcees the evening at MosaicML Girl Geek Dinner. (Watch on YouTube)

Angie Chang: The first event was 2008.

Julie Choi: 2008. Yes. And tonight we have amazing speakers to share with us about machine learning, about engineering, about diversity, and how that can really supercharge productivity in high growth organizations and machine learning research just from some of the world’s best AI companies and organizations.

Julie Choi: Our first speaker of the evening is my dear colleague at MosaicML, Laura Florescu. Laura and I met a little over a year ago. You greeted me at the front door. You were the first face I saw at MosaicML. She has just been an inspiration to me as an ML researcher at our company. Prior to joining MosaicML, Laura actually worked at several unicorn AI hardware startups. Then prior to that, she got her PhD from NYU in mathematics and is just a brilliant lady. Laura, can you come up and tell us about this amazing topic?

Laura Florescu: Thank you.

Julie Choi: Let’s give her a hand.

Laura Florescu: Can you hear me? Hi, thank you so much everybody for being here. Thanks to Julie, Sarah, Angie, Playground, for having this event. Very honored to be here.

mosaicml girl geek dinner laura florescu speaking training nlp

MosaicML AI Researcher Laura Florescu talks about making ML training faster, algorithmically with Composer and Compute, MosaicML’s latest offerings for efficient ML. (Watch on YouTube)

Laura Florescu: As Julie said, I’m a researcher at MosaicML. A little bit about myself. I’m originally from Romania. I came to the states, did my undergrad in math, did a PhD at NYU, and then I got the kind of Silicon Valley bug. And now I’m at MosaicML. And so what we do is develop algorithms and infrastructure to train neural networks efficiently.

Laura Florescu: Basically for the people in the audience who are not into ML, training neural networks is at the core of artificial intelligence. It uses a lot of data. It’s applied to many different fields with image, language, speech, and kind of like a takeaway from this is that, for large powerful models, the training costs for one single run can get in the millions of dollars, to train one such model. And in order to get to a really good model, you need to do several such runs. So the cost can get extremely, extremely expensive.

Laura Florescu: Our belief at MosaicML is that state of the art, large, powerful models should not be limited to just the top companies. As we have seen over the last few years, the costs are getting larger and larger due to both the size of the models, also the data that the models ingest is just exploding.

Laura Florescu: A couple of years ago, a state-of-the-art model, Megatron, actually cost $15 million to train. As you can imagine, startups probably cannot really train models like that. And at MosaicML, this is our belief, that this kind of training should be accessible to other partners as well.

Laura Florescu: That’s where we come in. We want the state-of-the-art efficient ML training. Our co-founders are Naveen and Hanlin from Nirvana and Intel AI. Also, founding advisors from MIT and also founding engineers from leading AI companies. All of us have the same kind of goal to train machine learning basically faster and better and cheaper.

Laura Florescu: Our thesis is, the core of Mosaic is that algorithmic and system optimizations are the key to ML efficiency, right? And then the proof to that so far, is that we are working with enterprises to train ML models efficiently. And we want to enable our organizations to train the best ML models, the cheapest and the most efficient.

Laura Florescu: Some results that we already have. As I said, we want to be agnostic about the kind of models, data that we ingest. For some image classification tasks, we have shown 6x speed ups, like 6x cheaper, faster than like regular training. About 3x faster for image segmentation, about 2x for language models, language generation, and 1.5x for language understanding. We have been around as Julie said for about a year, a year and a few months, and these are already some of the results that we have achieved.

Laura Florescu: A use case is to train NLP models, such as BERT, for those of you who know that, and on our specific platform and without algorithms. Use case is for example, to increase sales productivity. If you see there, on our MosaicML 4-node, with our speed ups, which I’m going to discuss in a little bit, we can see up to 2x speed ups of training such models. And also about 60% training costs reduced by training with both our algorithms and on our platform, on our cloud.

Laura Florescu: The MosaicML cloud, we want it to be the first AI optimized cloud designed specifically for AI and directly to reduce training cost at any layer of the stack. For example, in the training flow, we want to reuse data from past runs. In the models that we’re using, so that’s where the kind of optimized model definitions come in, Composer, which is our open source library. We are doing the algorithmic speed ups, the training. And kind of like at the lower level, we want to also be able to choose the best hardware in order to get to the lowest training cost. At each layer, we are optimizing all of these system optimizations and composing all of those basically leads to the best training runs.

Laura Florescu: As I mentioned, Composer is our open source library. We have a QR code there if you want to check it out. We have about 25+ algorithms that we have worked on and given the name, they compose together, and that’s how we achieve basically the best, 2x to 5x speed ups. And as an example, for a BERT model, we have seen 2.5x speed up for pre-training, which as you can see, goes from nine days to about three days. That’s a huge win. Check it out if you would like. As I said, open source, so we’re always looking for feedback and contributions.

Laura Florescu: We’re open to partnering with any kind of corporate users, for anybody who has vision or language tasks, and then also industry, we want to be industry agnostic and global. Again, we want to optimize basically any kind of models. And as I said, the open source Composer speed up, we’re open to feedback and partnerships for that as well. And of course we are hiring. Yeah, it’s a really great team, really fun, really ambitious. And I’m so honored to work there and we’re looking for all kind of builders, engineers, researchers, products. And thank you very much.

Julie Choi: Thank you so much, Laura. Okay, that was wonderful. Let me just go to the next talk. Our next speaker is Amy Zhang, and Amy comes to us from, she’s currently a research scientist at Meta AI research, and a postdoc I think, you’re not a postdoc anymore, are you?

Amy Zhang: It’s kind of like a part-time thing.

Julie Choi: It’s like, never ending, huh? But you’re on your way to your assistant professorship at UT Austin, amazing, in Spring 2023. And Amy’s work focuses on improving generalization and reinforcement learning through bridging theory and practice. And her work, she was on the board most recently of women in machine learning for the past two years. And she got her PhD at McGill University and the Mila Institute and prior to that obtained her M.Eng. in EECS and dual Bachelors of Science degrees in math and EECS at MIT. Let’s welcome Amy Zhang to the stage. Thank you.

Amy Zhang: Thank you Julie for the really kind introduction and for planning this amazing event. It’s so nice to see people in real life. This is my first large in-person talk in over two years so please bear with me.

mosaicml girl geek dinner amy zhang speaking

Meta AI Research Scientist Amy Zhang speaks about her career journey in reinforcement learning, from academia to industry, at MosaicML Girl Geek Dinner. (Watch on YouTube)

Amy Zhang: Today I’ll be talking about my research which is in reinforcement learning, but I first wanted to just give a little bit of introduction of myself. To me, I feel like I’ve had a fairly meandering journey through academia and industry and research in general. I wanted to give a little bit of introduction of what I’ve done so that if any of you feel like you’re going through something similar, please reach out and I’m happy to chat and give more details. Like Julie said, I did my undergrad at MIT, a year after I finished my masters, I started a PhD at UCSD. It did not go well, through no one’s fault, really. I just felt really isolated and so after about a year, I quit my PhD, meandered through a couple of startups, and then eventually found my way to Facebook, which is now Meta.

Amy Zhang: At Facebook, I initially started on the core data science team. I was a data scientist, but I was working on computer vision and deep learning. This was 2015, still fairly early days in terms of deep learning and everyone is really excited about the gains that it had shown for computer vision at the time. I was working on population density, so we were taking satellite images and doing building detection to find houses, to figure out mostly in third world countries, what the population density was so that we could provide internet to people and figure out what was the best way to provide that internet.

Amy Zhang: After about a year of that, I ended up joining the Facebook AI research team FAIR, as a research engineer. And after about a year of that, I happened to meet the person who became my PhD advisor, Joelle Pineau, who is now director of FAIR, and I got to do my PhD with her at McGill University while still staying in FAIR. Fast forward a few more years and I defended my PhD remotely in the middle of the pandemic last year and am now back in the Bay Area as a research scientist at FAIR and like I said, was part-time postdoc-ing at UC Berkeley.

Amy Zhang: After spending all this time in industry, and having a really great time in industry, and getting all of these amazing opportunities to do my PhD while in industry, with all of the nice resources that that provides, I decided to go back into academia. And, last year I was on the faculty market, on the academic faculty market, and I accepted a position at UT Austin. I will be starting there next year as an assistant professor.

Amy Zhang: With that, I’m just going to jump straight into my research. And this is going to be, again like I said, a very whirlwind, high level overview. I’m passionate about reinforcement learning. I love the idea of agents that can interact with the world, with us, and it can grow and learn from that interaction.

Amy Zhang: Okay, thinking a little bit about what reinforcement learning (RL) can do. I’m personally really excited about the idea of applying reinforcement learning to solve real world problems. To me, this is personalized household robots, having a robot that will do your dishes, clean the house, make your bed, learned autonomous driving so you can just drive in a car without having to actually drive and pay attention to the road, and personalize healthcare, so having like a robo-doctor who knows everything about you and can personalize healthcare and give recommendations for you specifically.

Amy Zhang: Unfortunately we are not there yet, as I think maybe all of us can tell. Deep reinforcement learning has had a lot of successes in the last two years. Maybe some of these things are familiar to you. AlphaGo, where we have an RL agent that was able to beat the best human experts. OpenAI with playing video games, which I’m not very familiar with, but like Dota and StarCraft. These are the things that have been hitting the news in terms of what Deep RL is capable of. But there are still a lot of disappointing failures and I think none of these videos are going to show, but imagine this robot trying to kick this ball and then just falling flat on its face. That’s what that video is. And in the other little cheetah looking thing is supposed to be like tripping and falling. Anyway, this is where we are currently with RL.

Amy Zhang: Why do we still see this discrepancy? How are we getting these amazing gains but still seeing these failures? And what we’re really seeing is that Deep RL works really well in these single task settings, in simulation, when you have access to tons and tons of training data, but it works less well in visually complex and natural settings. Basically we’re not seeing the same type of generalization performance that we’ve been getting out of deep learning in computer vision and natural language processing. My research agenda is mainly about how can we achieve RL in the real world? How can we solve these problems? And, to me, it seems that abstraction is one key to generalization. And I use this type of idea to develop algorithms that have theory-backed guarantees.

Amy Zhang: I’m going to not really talk a little bit about this math, but I’m particularly interested in being able to train reinforcement learning agents that can solve tasks from pixels. Imagine that you have this household robot or this autonomous driving car that is receiving information about the world through a camera, through RGB input, and that’s a big part of how we also perceive the world. There are things in this image that are relevant for the autonomous driver here and there are things that are not. And we want to figure out how can we determine from just a reward signal, what things are relevant versus irrelevant.

Amy Zhang: I’m just going to, as part of this project, we developed this representation learning method using this idea by simulation, and showed that in this type of simulation driving task, which is done in this platform called Carla, so we have just this figure eight simulation environment where this car is just driving along this highway, and there’s lots of other cars in the road and basically, it’s designed to try and drive as quickly as possible. Using the break as little as possible and maximizing throttle while also not hitting anything. And we find that our method which can basically ignore these kind of irrelevant details and figure out what things are irrelevant does much better compared to a lot of existing methods.

Amy Zhang: One really cool thing is that when we look at the representation that we actually learn here, and we look at what kind of observations get mapped to be close together in this representation space, so what information is actually getting captured by this representation, we see that… this is the agent’s point of view. We see that, in these three examples, you’re always on this straight road where you have an obstacle on the right hand side. It doesn’t matter what the obstacle is, but the representation just captures that something is there, which means that you can’t turn right. This is just kind of an example of what our algorithm can do. And unfortunately I’m going to skip over this, because these were just some videos showing what our agent can do.

Amy Zhang: I just wanted to end on talking about some open problems that I’m particularly excited about. I’m particularly excited about compositionality. How can we solve combinatorially difficult problems. And these correspond to a lot of real world tasks that we should care about. When we think about really simple versions of problems like this, you can have a block stacking task. You can have any number of blocks or any combination of blocks and so you can always have new environments that you give to your agent to try and solve.

Amy Zhang: Similarly, moving more towards actual real world problems that we care about. Again, going back to the dishwasher example or an agent that is trying to move boxes around in a warehouse. These are all settings where the exact environment, the exact state that the agent has to deal with, is constantly different. The objects that you want to place in your dishwasher on a day to day basis is always going to look different. How can we get agents that can actually generalize to all of these new states?

Amy Zhang: I think one really exciting direction to go when trying to solve this type of problem is to think about factorization. How can we break down a problem into smaller, easier building blocks? So if we understand how one block, the dynamics of one block moves, so creating a stack of two blocks, right? Babies play around with this sort of thing and then as they get older, they automatically can extrapolate to building like gigantic towers and castles. So how can we take that idea and give it to reinforcement learning? So this is something that I’m particularly excited by.

Amy Zhang: I just wanted to end talking about my sort of wider research agenda. Now that I’m starting as faculty, I have to start recruiting a group. If any of you are interested in doing a PhD at UT Austin or know anybody who is, please send them my way. But when I think about what my research group does, I’m particularly interested in these three applications of reinforcement learning.

Amy Zhang: The first is in robotics, trying to solve manipulation tasks. Going back to that block stacking example, trying to solve locomotion and navigation tasks. How can we build an autonomous driving system purely from first principles, like end to end machine learning? Reinforcement learning has to be a part of that.

Amy Zhang: Natural language processing, so using RL for text generation, being able to extract knowledge from text, when you build an interactive agent, how do you give it information about the world? We learn from textbooks, we don’t want an agent just deployed in the real world with no basic information. Healthcare, how do we build RL agents that can help out with diagnosis and treatment and tackle a lot of the problems that we have there? That’s basically it, very grandiose. I probably won’t make much progress on a lot of these fronts, but this is the dream. And thanks for listening.

Julie Choi: Thank you so much. I think this is very grandiose and amazing that you’re working on it. Amazing. Thank you, Amy, okay.

Julie Choi: We’re going to just clean some of this up. Okay. And we’re going to open up our next talk, which I’m extremely excited about. Let me introduce our next speaker. Our next speaker journeyed probably the furthest to join us tonight… all the way from the east coast, North Carolina, to be here today to deliver this talk. I want to thank you. Thank you.

Julie Choi: Tiffany Williams is a Staff Software Engineer at Atomwise working on AI-powered drug discovery. Prior to Atomwise, Tiffany was a Staff Software Engineer at Project Ronin where she was developing cancer intelligence software. Tiffany earned her PhD in cancer biology from Stanford University and her Bachelor’s in biology from the University of Maryland. Let’s give a warm welcome to Tiffany Williams.

Tiffany Williams: All right. Hello, everyone. I’ll admit, I have some notes here so I can stay on track, but I’m really glad to be here this evening. I was searching through my inbox and realized that I attended my first Girl Geek Dinner in April of 2015. I was fresh out of grad school and a coding boot camp. Eager to form connections, acquired some cool swag, and to be honest, eat some free food and have some drinks.

mosaicml girl geek dinner tiffany williams speaking

Atomwise Staff Software Engineer Tiffany Williams discusses the drug discovery process with AtomNet at MosaicML Girl Geek Dinner. (Watch on YouTube)

Tiffany Williams: I went to grad school to study cancer biology. My research was from the perspective of a molecular biologist exploring the role of a protein as a target for therapy, and skin cancer. And my interests have generally been at that sweet spot, leveraging data, and technology to improve human health. Coming from a biology research background, in my current role as a software engineer at Atomwise, I’m able to look further down that drug discovery process. I’m happy to share with you all today, what that looks like.

Tiffany Williams: Now, I’ll start by giving a very brief primer on the current state of the drug discovery process, as well as a very brief primer on biochemistry. From there, I’ll talk about what we’re doing at Atomwise to make a significant impact in human health, and some interesting challenges ahead of us. I do want to give a disclaimer that I am merely scratching the surface of what could be discussed in drug design, drug development, and even applying ML on top of that. But what I hope you all take away from my time with you all this evening is just general excitement about the possibility of improving human health.

Tiffany Williams: Beyond that, I hope you feel empowered to do even more digging into the drug discovery process, and maybe you’ll feel empowered to even… to find opportunities to make an impact in that space. For me personally, I’m coming from the east coast, but I have a personal attachment to improving the drug discovery process. I’m on the east coast serving as a caregiver now – my mom has endometrial cancer and it’s at a point where there’s only one treatment option. If any of you have ever been in, unfortunately, in a predicament like that, it sucks.

Tiffany Williams: There’s a lot of data out there, technology’s improving, and it would be great if we can leverage that to improve health, right? This diagram depicts the drug discovery process from the basic research to identify a potential drug target all the way to FDA approval. On the right, I’ve noted the average number of years of the different stages in the drug discovery process. In the middle, what it shows are like basically in these different stages, there are certain types of experiments that are done that basically kind of knock out the potential candidates in that step.

Tiffany Williams: Initially you might have like a candidate pool of over like 10,000 drugs, but in an early stage, which is known as like kid identification, computer simulation is used to predict potential drugs ability to bind to a target of interest. These subsequent steps will test for other characteristics like potential toxicity, or efficacy in cell cultures, and animal models. Eventually, hopefully, we end up with a few candidates that reach the human clinical trial phase to verify safety, and any other side effects along with a few other things.

Tiffany Williams: What I hope that you take away from this slide really is that the current basic research to FDA approval drug process takes a really long time. It takes on average, it’s estimated to take about 15 years, and it’s also really expensive for each drug that goes to market, it’s estimated that 2.6 billion is spent.It’s the case that not all of these trials have a happy ending. For every drug that makes it to the market, millions may have been screened, and discarded. We have to improve this process, right? But in order to appreciate how this process can be improved, let me first give that very, very, very brief biochem primer. And I’ll focus specifically on protein interactions.

Tiffany Williams: Up to this point, and then even later in this talk, I’m going to be using some words interchangeably, and I just want to make sure I bring some clarity to what I’m actually talking about. When I say like ligand, or ligand, I’m specifically referring to any molecule that combined to a receiving protein, and that receiving protein is also known as a receptor. And in this presentation, when I use the word or phrase drug candidate, I’m actually referring to that ligand and the receptor would be that the drug target.

Tiffany Williams: One model for how proteins interact is this lock and key model. And the gist of this is that the ligand and the receptor have these somewhat complimentary sites. And the ligand combined to this complimentary site or active site, and basically alter the shape, and or activity of the receptor. And one more thing is that this binding is also referred to as docking. If we know our bodies are made up of proteins, and they have a diversity of functions within the body, but in a disease state, a normal process can become dysregulated. This image on the far right, is taken from a classic cancer biology paper called The Hallmarks of Cancer. The premise of this is that there about 10 biological capabilities that cells take on as they like morph into this more cancerous state. And I’m not going to go over all of these, but I want to highlight that two of these biological capabilities would be like cell growth and like cell motility.

Tiffany Williams: Those are normal functions within the body. But I guess within a diseased state you’ll have overgrowth, or you may have cells that are primarily concentrated in one area, develop the capability to invade into other tissues or metastasize. I say all this to say that like in small molecule drug discovery, what we want to do is actually figure out what sort of structure is needed for a ligand to bind to this problematic protein or proteins, and counteract that like not so great behavior. If this is where Atomwise comes in, right? Or it does, if you don’t know. We want to know how can we efficiently explore this space of all potential chemical compounds to better identify small molecule drugs faster.

Tiffany Williams: Atomwise actually developed AtomNet the first deep convolution neural network for structure based drug design, so that we can actually make better predictions for potential drug candidates earlier in the drug discovery process and faster. I’m highlighting this paper, again, just trying to give high level overviews. Feel free to check this paper out, but what I want you to take away from this is that the AtomNet technology is currently being used in real drug research, in cancer, neurological diseases, various spaces, on the right is, it’s actually a GIF, but since it’s a PDF, it’s not showing up as a GIF, but basically this GIF would show the AtomNet model. It would simulate the AtomNet model, predicting candidate treatments for Ebola. And this prediction that AtomNet made is actually has led to candidate molecules that are now being studied in animal models.

Tiffany Williams: One, despite everything I’ve said up till now, you actually don’t need a background in wet lab research, or chemistry to appreciate what’s happening here. The power of convolute or the power of convolution neural networks, or CNN, is that it allows us to take these complex concepts as a combination of smaller bits of information. And I think if you’re familiar with CNN, or even if you don’t like one area that is really popular, has been computer vision.

Tiffany Williams: I’ll briefly go over like an example of image recognition, and then kind of like try to tie it into how AtomNet works. An image is essentially represented as a 2D grid with three channels, you have red, green, and blue. And this network learns images of objects or faces, for example, by first learning a set of basic features in an image like edges. Then, from there, by combining those edges, the model can then learn to identify different parts of that object. In the case of a face that might be ears, eyes, nose, et cetera. With AtomNet that it’s working in a similar fashion, that receptor ligand pair is represented in a 3D grid, and the channels are essentially the elements that make up protein like carbon oxygen, nitrogen, et cetera. In the case of AtomNet, that the learning of edges is actually the learning of the types of, or predicting the types of bonds between those elements.

Tiffany Williams: Then from there, the compliment to the ear eye detection would be actually identifying more complex molecular bonds. You could say, essentially, that this network is like learning organic chemistry 101. This is powerful because we can then train these models to make predictions about different aspects of the drug discovery process. Like what ligands, or what type of structures are most likely to bind to a certain target? At what strength? What are the additional modifications that we can make to a potential drug candidate to strengthen that bond? Beyond that, it’s not enough just for the ligand to bind to the receptor, it also has to be like biologically relevant in that, let’s say, if we’re looking for something that is treating some neurological disease. We need that ligand to be small enough to cross the blood brain barrier. Or we may need to take into account toxicity or other effects that may happen in the body. Metabolism. We don’t want that small molecule to become quickly metabolized in the body before it has the opportunity to have the intended effect.

Tiffany Williams: These are exciting times, and I’m really, really passionate about the work that we’re doing at Atomwise and any work that is being done at the intersection of health and technology. I wanted to briefly go over some of the projects that my team is currently working on. I work on the drug discovery team at Atomwise, it’s within the engineering team. I think we’re working on some pretty interesting issues. One of my, my teammates Shinji, he recently has been working on bringing best engineering practices, and improving performance of some of our ML tools. Adrian, my teammate, is working on optimizing algorithms to explore a three trillion chemical space. He is also been working on, or has been able to create simulating mocular… mocular, I’m combining words… molecular docking on GPUs.

Tiffany Williams: Then another thing that we’re working on that I actually have more of a hand in, is building a research platform to better enable drug discovery. Oh, I didn’t mention the third person Shabbir, who’s our manager, and he has his hands in a bit of everything. What I hope you take away from this is that I think we’re at an exciting time in today to like really leverage data, and technology to make a major impact in human health. I think there are a lot of challenges, interesting challenges in drug discovery. I hope that you may have been convinced that you actually don’t need a background in chemistry to contribute. There are a lot of transferable skills. If you just know software engineering, or you know ML, or if you’re in product or marketing, there is a place for you in this space.

Tiffany Williams: Finally, really important takeaway is that we are hiring. My team alone, please, if you have any front end experience, if you have backend experience, or if you have a background in computational chemistry, those are some of the positions that are open right now on my team, but then outside of my team, we also are hiring. Definitely check out our careers page. If you have any more questions or are interested in chatting, feel free to reach out to me. I have my LinkedIn handle as well as my Twitter handle, posted here. Then finally, I think I mentioned that I had some references to share. Again, I’m only scratching the surface. There’s so much information out there. I wanted to highlight two Medium articles that were written by former Atom, which is what we call people that work at Atomwise, machine learning for drug discovery, in a nutshell, I highly recommend starting there if you want to do a deeper dive. That’s it.

Julie Choi: Thank you so much, Tiffany. Yes. I think drug discovery is an incredible application domain for deep learning. Really appreciate your talk. Okay, let me introduce our next speaker. Our next speaker is Shelby Heinecke. Shelby is a senior research scientist. Again, I did not touch. Okay.

Julie Choi: Shelby is a Senior Research Scientist at Salesforce Research, developing new AI methods for research and products. Her work spans from theory-driven, robust ML algorithms to practical methods, and toolkits for addressing robustness in applied NLP and recommendation systems. She has a PhD in Mathematics from the University of Illinois at Chicago, and a Master’s as well in Math from Northwestern and her bachelor’s is from MIT. Let’s welcome Shelby.

Shelby Heinecke: Thanks so much. Awesome. I have to give a thank you to Julie for including me in this event, inviting me. This is my first Girl Geek, and not my last Girl Geek event. I’m super excited to be here. Yeah, let me get started.

mosaicml girl geek dinner shelby heinecke speaking recommendation systems salesforce research

Salesforce Research Senior Research Scientist Shelby Heinecke speaks about how to evaluate recommendation system robustness with RGRecSys at MosaicML Girl Geek Dinner. (Watch on YouTube)

Shelby Heinecke: Today I’m going to be talking about evaluating recommendation system robustness, but first I feel kind of like the new kid on the block. Let me just give a little bit of background about myself. I moved here to the Bay Area about a year and a half ago in the middle of the pandemic. Super excited to be here in person to meet people.

Shelby Heinecke: Currently I’m a Senior Research Scientist at Salesforce Research. As Julie mentioned, I work on both research and product. It’s pretty awesome to develop prototypes, and see them in production. Before I was here at Salesforce in the Bay Area, I was doing my PhD in math in Chicago. There, I focused on creating new ML algorithms that were robust. I worked on problems in the space of network resilience. Before that I got my master’s in math, kind of focusing on pure math at Northwestern. Originally I hail from MIT Math, focusing on pure math there. That’s my background. Today’s talk recommendation systems and robustness.

Shelby Heinecke: Let’s get started. A crash course and recommendation systems. So, what is a recommendation system? Well, it consists of models that learn to recommend items based on user interaction histories, user attributes, and or item attributes. Let me give you an example, say we want to build a recommender system that recommends movies to users. Well, what kind of data can we use? We’re going to use the users, previous movies. They viewed we’re going to use the ratings that they rated those movies.

Shelby Heinecke: We’re probably going to take a look at the user’s age. We’re going to look at the user’s location. We’re also going to take into account item attributes. The movie attributes, like the movie title, the movie genre, we’re going to take all that. That’s all of our data, and we’re going to train models. The models can build the recommendation system. As you can imagine, recommendation systems influence our daily lives. We’re all exposed to recommendation systems every day. Just think about purchases. Think about movies.

Shelby Heinecke: Think about songs you’re recommended. Think about the ads that you see every day, people, news, information. We are at the mercy of these recommendation systems and a lot of our decisions are highly influenced by what the recommendation systems decide to show us. Let’s think about the models that we see in recommendation systems. Models can range from simple heuristic approaches, like a rules based approach or co-sign similarity approach to complex deep learning approaches, think neural collaborative filtering, or even now we’re seeing transformer based approaches coming to light.

Shelby Heinecke: With the vast range of models available and how greatly they impact our daily lives, understanding the vulnerabilities of these models of each of these types of models is super critical. Let me get started about recommendation model robustness. As we all know, machine learning models are trained on data, and ultimately deployed to production. And in that process, there are some hidden sources of vulnerabilities that I want to bring to light.

Shelby Heinecke: One of the big issues is that training data may not reflect the real world data. In many cases, we’re training on data that’s been highly curated. That’s been cleaned up. And yeah, and so because of that, when we train a model on that very clean, highly curated data, it’s not going to necessarily perform well when it’s exposed to the messy data of the real world. Real world data has noise. Real world data is just can be unpredictable. As a result, sometimes we train model, we train a recommendation model, but we see poor performance and production. That’s definitely one type of vulnerability we need to watch out for.

Audience Member: Woo! So true.

Shelby Heinecke: What? Okay. Another type of vulnerability. I love the enthusiasm, okay? Another type of vulnerability. As you can imagine, recommendation systems are closely tied to revenue for a lot of different parties for companies, for sellers. There’s an issue that participants may intentionally manipulate the model. Think about creating fake accounts, trying to do things, to get your item higher on the list for customers, things like that. That is a reality and that’s something we have to take into account.

Shelby Heinecke: The last thing I want to bring to light is poor performance on subpopulations. This is a well known issue across all of ML, but I just wanted to bring it to light, to recommendation too, that when we train models and we test on the evaluation set, usually the basic evaluation methods think precision recall F1. We’re computing that on the entire test set, so we’re averaging overall users. And in that average, sometimes we’re hiding…

Shelby Heinecke: Sometimes there’s poor performance on subpopulations that are hidden. For example, a subpopulation that you may care about could be new users, or maybe a users of a certain gender. That’s something that we just need to keep an eye on. We don’t want poor performance on key subpopulations I’ve told you about all these different types of vulnerability.

Shelby Heinecke: What is a robust model? Well, a robust model you can think of it intuitively as a model that will retain great performance in light of all of these potential perturbations, or in light of all of these scenarios. The question is how can we assess the robustness of models? That is where one of our contributions comes in.

Shelby Heinecke: I want to introduce one of our open source repos called RGRecSys, and it stands for robustness gym for recommendation systems. Our library kind of automates stress testing for recommendation models. By stress testing, I mean, you can pick a model, you can pick a data set, and you can stress test it in the sense that you can manipulate the data set.

Shelby Heinecke: You can add in attacks, you can add in noise and so on. I’ll actually go into more detail about that. And you can see in a really simple way how your model, the robustness of your model. As I mentioned, RGRecSys, is a software, is a software toolkit. It’s on GitHub. I’ll talk about that in a second and it’s going to help you assess the robustness of your models.

Shelby Heinecke: One thing that we contribute is that RGRecSys provides a holistic evaluation of recommendation models across five dimensions of robustness. When I say I told you about various types of vulnerabilities, there’s various types of robustness. Our library helps you to quickly and easily test all these different types of robustness for your model. Using our API, you just simply select a model for testing, and then you specify the robustness test that you’re interested in trying along with the robustness test parameters. What I’ll do is I’ll go over the types of tests that are in the library.

Shelby Heinecke: Let’s talk about the different tests that we have in our library. First is around subpopulations and this kind of goes back to what I mentioned in the previous slides. This will allow you the test. What is the model performance on subgroup A versus subgroup B? And this is going to be very useful because, as I mentioned, most of the time, we’re just computing precision recall these usual metrics on the test set, but this gives you an easy way to test performance on specific subgroups. This could be useful, for example, if you want to test you want to test performance on gender A versus gender B, or new users versus old versus like users that have been in the system for longer time. Just some examples there. That is one type of test you can run in our library. The second type of test is around sparsity. If you think about recommendation systems and you think about the items available like a movie recommender, or a purchasing recommender there’s millions of items, and each user really only interacts with a handful of those items.

Shelby Heinecke: Each user is only clicking, only purchasing, only viewing a handful. This is a source of data sparsity. Data sparsity is a huge problem in recommendation systems. It will be good to test the degree to which your model is sensitive to sparse data. That’s one thing you can test with our library. The third test is around transformations. There are a lot of ways that data can be perturbed when you’re training a recommendation system model. For example, maybe you’re gathering data about your users and maybe that data is erroneous in some ways. And because of that, you might want to test if a recommendation model will be robust, if user features, for example, are perturbed. The fourth test that you can test is around attacks. As I mentioned, there’s a lot of reasons why people would try to attack a recommendation system it’s tied to revenue ultimately.

Shelby Heinecke: What you can do with our library is implement some attacks and test how your model performance would change under those manipulations. And finally distribution shift. What we see is that the training data that you train on is often different from the data that you’ll see in production. It’s super important to be able to know, get a sense of how was my model going to perform if it’s exposed to data from a slightly different distribution? You can go ahead and test that with our library., I definitely encourage you to check out our library on GitHub, and feel free to check out the paper for way more details about the capabilities. And with that, thank you so much for listening. It was great to share it. Feel free to reach out.

Julie Choi: Thank you, Shelby. That was great. It’s very important to be robust when we’re doing model deployment. Okay. Angela Jiang, thank you so much for joining us tonight.

Julie Choi: Angela is currently on the product team at OpenAI. Previously, she was a product manager at Determined AI building, deep learning training, software and hardware, deep learning… I think Determined AI was recently acquired by HPE. And Angela graduated with a PhD in machine learning systems from the CS department at Carnegie Mellon University. And we actually have built some of our own speed up methods on your research. It’s an honor to have you here today.

Angela Jiang: Thank you so much, Julie, for inviting me, for the introduction, as well as Sarah, and Angie for organizing the event and bringing us all together too. Yeah. Like Julie mentioned, I’m Angela. I work on the product team at OpenAI.

mosaicml girl geek dinner angela jiang openai

OpenAI Product Manager Angela Jiang speaks about turning generative models from research into products at MosaicML Girl Geek Dinner. (Watch on YouTube)

Angela Jiang: I work on our Applied team where we really focus on essentially bringing all of the awesome research coming out of the org and turning them into products that are hopefully useful, safe, and easy to use.

Angela Jiang: Most of my day, I’m thinking about how to turn generative models from research into products. I thought I’d make the talk about that as well. This might as well be a list of things that keep me up at night, think about it that way. But in particular, what I really wanted to highlight is just some of the observations I’ve had about things that make these generative models unique and tend to have large implications on how we actually deploy them into the market.

Angela Jiang: To start, I want to share a little bit about what OpenAI’s products are. OpenAI does a lot of AI research. In particular, we do a lot of generative models. Over the last two years, we’ve really started to work to bring those generative models into real products. Three examples here are GPT-3, that does text generation; Codex, that does code generation; and most recently, DALL-E, which does image generation. These were all made by DALL-E.

Angela Jiang: To get a little bit more concrete, what these products are is that we expose these models like GPT-3 and Codex via our APIs. As a user, you can go to the OpenAI website, sign up, and then hit these endpoints and start using these models. And here’s an example of how you might use the GPT-3 model. Here we have an example of your input in the gray box. It might say something like, “Convert my shorthand into a firsthand account of the meeting and have some meeting notes.” And you submit this to the model, and then the model will do its very best to return you an output that completes this text. And in this case, the response is essentially a summarization of your notes.

Angela Jiang: Codex works very similarly, except it’s also specialized for code. And in this case, our input is not only text, but it’s also some code, some JavaScript code, and some hints that we want to transform it into Python. And the output is some Python code that does that. DALL-E is very, very new. It’s been out for around a month, so it’s not available in the API or anything yet. But it has a very similar interaction mode where you have, again, an input, which is a description of the image that you want. And then as the output, it provides some images that relate to that description, which in this case is a cute tropical fish.

Angela Jiang: I think GPT-3, Codex, and DALL-E are all really good examples of taking research and seeing that there’s a really big market need for the kinds of capabilities that they expose. Then working as an Applied team to actually transform that research into a productionized product by essentially figuring out where that user value is, designing the product so that we can deliver that user value in a way where users are set up for success, making sure it’s deployed responsibly, et cetera, et cetera.

Angela Jiang: To date, there’s hundreds of applications that have been submitted for production review that is built upon this API. Those are applications like writing assistants like CopyAI if you’ve heard of that, coding assistants like GitHub’s Copilot is built on top of Codex, as well as a lot of other applications like chatbots for video games, question answering bots, etc. This is pretty good validation that these models are not only really exciting research, but are also solving problems in the market.

Angela Jiang: Now, when we think about bringing these to market and turning them into real products, I think there are two interesting properties of these generative models that really change who uses them as well as how we actually deploy them. I’ll talk about how these models are stochastic and how they’re very general. With typical products, you might expect that the results are very predictable and deterministic. For instance, if you’re coding up a payments API, every time you use that payments API, you’d expect it to react in a similar way. This is not how our models work at all.

Angela Jiang: Our models are probabilistic. Every time you submit a prompt, you could get a different response back. If you tweak that prompt ever slightly, then you could get a very different response back. That really changes how you interact with a product and what kind of applications can be built on top of it. Another difference is I think that typical software products that I think about are typically focused on solving a need or a problem very directly, whereas, in these generative models, they’re very, very general. They’re capable of doing a lot of things all simultaneously.

Angela Jiang: For instance, you would think that GPT-3 would be able to summarize legal texts and also simultaneously write poems or maybe even code. I think this is super exciting from a science perspective as we get more and more powerful models, closer and closer to something that’s super general. But from a product perspective, this comes with some challenges because we have to now take one model and one product and have it serve many, many use cases simultaneously.

Angela Jiang: For the rest of this talk, I’ll talk about, again, more detail about how these two properties affect the way that we deploy our products, and I’ll give some examples of each. Okay, starting with stochasticity. Right. Our models, again, are probabilistic, which means that every time you use it, you might get a different result. The result might not be what you want or maybe the result is what you want every third time you try the model.

Angela Jiang: It might be surprising that you can actually build real applications on top of this kind of behavior. What we actually find is that for a lot of tasks, especially very simple tasks, we can have very concrete and reproducible behavior for even tasks where the outputs are imperfect or are only correct some of the time, these results are actually still very useful for many applications in the right context.

Angela Jiang: We find that these models work really well, for instance, in creating productivity tools where there’s a human in the loop. A really good example of this is Codex Copilot. Some of you may have used it before, but this is a screenshot of it in action, where it’s a Visual Studio Code extension and it just gives you auto-complete code suggestions that the user can then tab to accept or reject.

Angela Jiang: This is great for a product like Codex because you have actually an expert there that’s telling you, is this completion good or not? And generating more. What we actually see is that most of the suggestions from Copilot are rejected by the user, but still, developers tell us that this is an integral part of their deployment pipeline or their development pipeline I should say. It does not need to be perfect every time we generate something. Those are cases where you don’t need a perfect result every time. What you’re looking for is just inspiration or getting over writer’s block. In some of the applications built on top of these APIs, you actually don’t have a correct answer to shoot for.

Angela Jiang: Applications that are doing art or creative tasks or entertainment, that’s a case where having variety really is helpful actually. Here I have a DALL-E prompt, which is a bright oil painting of a robot painting a flower. And here are different generations from DALL-E. And in this case, it’s really helpful actually to have probabilistic nature because, for one, I want to have different options. And it’s also really cool to have generations that you’ve never seen before or other people haven’t seen before.

Angela Jiang: I think it’s even counterintuitive that, in this case, we’ve actually learned that these AI systems seem to be doing really, really well and surprising us in creative tasks as opposed to rote tasks, which you might have expected the opposite earlier. Those are some examples where the properties of these generative models affect what applications are built on top, but it also affects how we deploy these models. Going into a little bit more detail, one property is that these products are really, really hard to evaluate.

Angela Jiang: This makes being a Product Manager, among other things, quite challenging because we really need to know how our products perform so we can position them in the correct way to the users, know what to deploy, and tell users how to use them correctly. For example, we have these text models, GPT-3, and they have different capabilities, right? One capability is they might be able to complete text, and another capability is they might be able to edit existing text. There’s some overlap there on which one you should use.

Angela Jiang: It’s really important that we understand exactly how this model performs on different tasks so that we can direct users to use the right tool at the right time. We try really hard to figure out creative ways to evaluate these at scale, given that we often need a human in the loop to be telling us if a generation is good or not.

Angela Jiang: We do things like large-scale application-specific A/B testing. Like we see if we use one model or the other, then which model gets more engagement for this writing assistant.

Angela Jiang: Next, I’ll just give a couple of examples of how generality also comes into the picture here. Like I mentioned, these models are capable of a lot of things simultaneously because the way that they’re trained is that they’re trained for a very long time, for months, on a lot of data. And the data spans the internet, books, code, so many different things.

Angela Jiang: By the time we get this model as the Applied team, we’re really still not sure what kind of capabilities it’s picked up at that time. But like I mentioned, it’s really important that we understand that quickly. This is also really exciting because, at any given time, we can always discover a new capability of the model, and it’s often discovered after training.

Angela Jiang: An example of this is that the original GPT-3 model was built with text for text. And it was really a discovery after the fact that it also happened to be kind of good at code as well. And these signs of life and discovery is what ultimately sparked this idea of a Codex series that specialized in code.

Angela Jiang: As a Product Manager, it’s really critical that we keep on discovering and probing these models to really understand what’s the frontier of what they can do. And again, we are still figuring out what is the best way to go about this. But something counterintuitive that I realized after I started is that traditional user research that we’re used to doesn’t actually work really well for this use case because you might expect that you would go to your users and ask, how is this model? What can you do with it?

Angela Jiang: What we find is that our users, by definition, are working on tasks that the model was already really capable at. Their focus is not on the new capabilities of the model and pushing that frontier oftentimes. What’s worked better for us is working with creatives or domain experts to really hack and probe at the model and see what the limits of it are.

Angela Jiang: That’s the exciting part of generality, that there’s always a new capability around the corner to discover. But there’s also some risk to it because there’s not only all these capabilities in the model, but the model can also generate things that are not useful for you, and it can also generate things that you really don’t want to generate.

Angela Jiang: For instance, GPT-3, when prompted in the right way, can generate things like hate speech, spam, violence, things you don’t want your users to be generating. This is also a big part of our goals as a research and applied organization is to figure out how to deploy these models in a way that doesn’t have toxicity in them.

Angela Jiang: There’s a lot of different approaches for doing this, spanning from policy research to product mitigations to research mitigations, but I’ll highlight two things here. One thing that’s worked really well for us is actually fine-tuning these models after the fact with a human in the loop, telling you exactly what kind of content is good content.

Angela Jiang: What we found by doing this is that we have a much better time at having the model follow the user’s intentions. And we have a much less toxic and more truthful model as a result of it. At this point, every time we deploy a model, we also fine-tune the language model in this way. We also do a lot of research and provide free tooling to help the users understand what the models are generating at scale so they can understand if there’s anything that they need to intervene in.

Angela Jiang: Okay. Finally, even though I think it’s very, very cool that these models can do a lot of different things at once, sometimes it really just can’t serve multiple use cases simultaneously well. Different use cases often will just require, for instance, different ways of completion.A chatbot for children is going to want a very different personality than a chatbot for support.

Angela Jiang: You also just have different product requirements in terms of accuracy or latency or even price. Copilot is a really good example of something that needs an interactive latency so that it can continue to be useful in real time. But contrast that with something like SQL query generation, which doesn’t need a fast latency, but actually just wants the most accurate response that you can give it. What we’ve found is that the combination of two things have allowed us to provide this flexibility to serve the use cases that we need to serve.

Angela Jiang: One is that we don’t just expose the best model that we have or the most accurate model we have I should say, we expose models with different capability and latency trade-offs for the users to choose from. And then we also offer fine-tuning as a first-class product so that you can take a model and then fine-tune it so that it has your personalized tone or your personalized data bank.

Angela Jiang: These are, hopefully, examples that give you a flavor of what it’s like to deploy generative models. This is also just the tip of the iceberg. If any of this stuff is interesting, please feel free to come and chat with me. I should also mention that we are hiring. We are. Thank you so much.

Julie Choi: Thank you so much, Angela. After DALL-E was launched, the productivity went a little down. We were so distracted by the DALL-E. I don’t know whether to thank or curse your team for that.

Julie Choi: Thank you, Banu. It is just a joy to introduce our next speaker. She’s a friend of mine and a former colleague, really an all-star. It’s so wonderful to have you here, Banu. Thank you.

Julie Choi: Banu Nagasundaram is a machine learning product leader at Amazon Web Services where she owns the go-to-market strategy and execution for AWS Panorama, an edge computer vision appliance and service. Prior to AWS, Banu has spent over a decade in technology, building AI and high-performance computing products for data centers and low-power processors for mobile computing. Banu holds a Master’s in EECS from the University of Florida and an MBA from UC Berkeley’s Haas School of Business. Let’s all welcome Banu.

Banu Nagasundaram: Thank you, Julie and team for having me here. I’m super excited to be here. One difference from the other speakers is this is not my area of expertise, the title of the talk. It is something I’m trying to do better at that I wanted to share with you.

mosaicml girl geek dinner banu nagasundaram speaking aws

AWS Product Manager Banu Nagasundaram speaks about seeking the bigger picture as a ML product leader at MosaicML Girl Geek Dinner. (Watch on YouTube)

Banu Nagasundaram: I’m trying to seek the bigger picture at work. I’m a Product Manager and I’m trying to see why companies do what they do and learn more in that process. What I want you to take away from this is how you can also seek the big picture in your roles that you do either as engineering or product leaders.

Banu Nagasundaram: With that, I wanted to share a little bit about the companies that I work with on a daily basis. These are concerns who use machine learning and AI and work with AWS to implement the services in production. This is different from the research that we spoke about.

Banu Nagasundaram: These companies are looking at getting value out of these systems that they put in place, of course, based on the research, but taking into production. What I implore you to think about is put on a hat of a CTO or a CIO in each of these companies and think about how and why you would make the investment decisions in machine learning and AI.

Banu Nagasundaram: For example, I work with healthcare and life sciences team. I learned a lot from the drug discovery talk earlier from Tiffany here, but I do work with healthcare and life sciences team to understand how they can take the vast amounts of health data that they have to translate into patient information that they can use to serve patients better.

Banu Nagasundaram: They use multiple services to personalize, to extract value from the text data, a lot of unstructured data that they have. That’s one category of customers.

Banu Nagasundaram: The second type of customers that I work with include industrial and manufacturing. The key component that they’re trying to improve is productivity and also optimizing their manufacturing throughput.

Banu Nagasundaram: The questions that they ask and they seek to improve include automating visual inspection. How can I improve the product quality across my manufacturing sites? I have thousands of sites in the US. I scale globally. How can I implement this process not only in one site but uniformly across those thousands of sites to achieve something like predictive maintenance on the tools, improve uptime of their equipment, etc?

Banu Nagasundaram: Third set of customers we work with include financial services. They are looking at data to improve or reduce the risk in the decisions that they make. They’re trying to target customer segments better so they can understand underserved populations but lower the risk in making those products and offerings that they want to do.

Banu Nagasundaram: They also look into fraud detection and many applications around financial services. Retail, this is one I work closely with because I work in the computer vision team right now. And retail is trying to use the insights from computer vision products to see how they can reduce stockouts, which is basically when you go to the store, is the product available? Can they sell it to you? How can they manage inventory? Can they keep track of the count or the number of people entering the store?

Banu Nagasundaram: You may have heard about Amazon Go, for example, a store with just a walkout experience. A lot of retail companies are working with us to understand how they can use computer vision to build experiences like that.

Banu Nagasundaram: And it doesn’t stop at a retail store. Think about the operations officer, a centralized person who’s sitting and trying to analyze which region should I invest more on? Which region should I improve security? Should it be in North Carolina? Should it be in California? Or should it be in a whole different country? They’re trying to collect and gather insights across their stores regionally, nationally, internationally to make those decisions.

Banu Nagasundaram: Then there’s media and entertainment too, which we touched upon a little bit around recommendation systems and personalization. Here we work with customers who are looking to improve monetization, who are looking to create differentiation in the marketplace, in the very highly competitive media and content marketplace, through those recommendation systems and personalization that they have.

Banu Nagasundaram: Across all of these customers, the core task as a product manager that I work on is understanding their requirements and then translating it into product features so that they are served better. But what I learned in the process is that it’s so much more in decision-making than just understanding about features or product requirements.

Banu Nagasundaram: It’s about what enables these CTOs to make those decisions is value creation. How can they use all of these AI ML systems to realize value from the systems that they put in place? One simple way to think about what value creation is, is that it’s an aggregation of data, analytics, and IT that brings the machine learning together. But there’s a second part to it, which includes people and processes.

Banu Nagasundaram: What I mean by that is all of this analytics and machine learning and data can help them understand something, but they still have to lean on people to analyze the data that they gather, make improvements in the process in order to recognize and create value. And that’s the workflow for decision-making across all of these companies.

Banu Nagasundaram: We can look at this as two buckets, one in data analytics and IT, and the second one as people and processes. For the computer vision product that I own, I wanted to talk to you about the value chain for the first part, which is the data analytics and IT portion.

Banu Nagasundaram: This might look a complex set of boxes. It is. Once I finish the pitch here, if this excites you, I’m hiring too, growing my team. Hopefully, I do a good job in explaining this value chain.

Banu Nagasundaram: I started my career in the bottom left, by the way, in silicon processor design. Pretty much in the bottom row, trying to understand how silicon design is then aggregated into components and how distributors sell those components to OEMs who are equipment manufacturers, and how from those equipment manufacturers, the equipments reach the consumers, which is through those equipment distributors.

Banu Nagasundaram: My product is currently both an appliance and a service. I do start from the silicon side, working with partners, I can give examples like Nvidia on the silicon side, Lenovo on the OEM side. And then once you have these equipment distributors selling these devices or equipment I should say to infrastructure providers, one of the examples of infrastructure providers is cloud service providers, but it can also be on-prem equipment providers.

Banu Nagasundaram: You then go to the infrastructure providers, which is the second row from the bottom, and these infrastructure providers then build the tools and frameworks either themselves or through the partner ecosystem. Those tools and frameworks essentially put in place to make efficient use of that underlying infrastructure. It’s a motivation for these infrastructure providers to offer the best tool and frameworks so that you can gain that value out of that underlying infrastructure.

Banu Nagasundaram: Then comes the ML services. You have the overall MLOps flow is what would fit in this bucket. Once you have the tools and the frameworks, how can all of these pieces get grouped together to build a robust system that can scale in production? This goes from data annotation, labeling, training to predictions, to model monitoring. “How can you maintain this when it is in production” is a key question for these decision-makers.

Banu Nagasundaram: Then comes the AI services that are built on top of these ML services. AI services can either be services offered by CSPs or it can be services offered by startups or companies who are trying to build the microservices or services on a specific use case. This is where the customer use case comes into play, where you have that specific use case. In case of computer vision, you can think of services like Rekognition or Lookout for Vision. Those are two examples that AWS offers for computer vision services.

Banu Nagasundaram: Then comes the final layer. This is the layer that the customers that I refer to, the CTOs, work closely with. The independent software vendors or ISVs are companies that build these software solutions, aggregating everything that I spoke to you about, but building the software components of it. But the software component by itself is not going to function in a customer’s premise. For the customer to realize value, the solution, the software solution for their use case has to integrate with their existing system. That’s where system integrators come into play.

Banu Nagasundaram: For example, Deloitte. You can think of Deloitte, Accenture, et cetera are system integrators who bring this whole puzzle together for customers, build that reference architecture for that solution. And they’re like, okay, so now we have this architecture in place. Then they bring in the value-added resellers, who are companies like Convergent or Stanley, for example, who take this entire system that’s put in place and roll it into individual sites.

Banu Nagasundaram: This is where it reaches the scale of thousands of sites. Once a solution is put together, proven in a pilot or a production pilot or a proof of concept, when you go to production, across globally, across cities, across countries, the value-added resellers roll the system into place in the customer’s site. But it doesn’t end there. There need to be managed service providers who can offer service contracts to maintain the system in place once it is on a customer’s premise.

Banu Nagasundaram: All of these building blocks in the value chain is what makes ML, machine learning, in production, intangible for a customer to realize value. It is a big journey. And this is the team I’m building who will work with individual partners across this value chain. And if this is something that excites you, we can talk after. In that value chain, we started with value creation. What is it that companies are looking for? We saw the value chain, but the value realization, let’s say the companies went through this process, put this whole system in place.

Banu Nagasundaram: What is it that actually helps them realize that value? When we think about data analytics in IT, many of the ML practitioners that I talk to, talk to me about the output, the visualization, the dashboards, the histograms for decision making, but it doesn’t stop there. That is not sufficient for these companies. There has to be people in those companies who take this output and actually work to achieve an outcome. The outcome is increase in productivity, increasing in throughput. It’s like, if I can know my demand better as a retail store, or if I can forecast the demand better, I can do so much better in my business.

Banu Nagasundaram: That is an outcome that I’m looking for from this data output that I’m getting from the machine learning models. But is that outcome sufficient? No, the outcome is like one place, one time, you are able to visualize that outcome, but you have to scale that outcome globally in order to achieve the impact. The impact that businesses look for is that you have to either increase revenue, reduce costs, reduce their risk, improve their sustainability, or create a competitive advantage. This is what the ML journey in production looks like. While I walked you through incrementally from the beginning on what it would take customers to get there, customers don’t work through individual steps, reach there and see what’s the next step.

Banu Nagasundaram: They actually have to make assumptions along the way and then understand what the impact might be and this is an AWS term that I’m going to throw and work backwards, which is understand what is it that you want and then build out what is it that you can, what you need to do towards achieving that end goal. That’s the big picture that I want to leave you with. In this whole ecosystem of having machine learning in production, can you think big on behalf of the customer, can you seek the big picture that the customer is looking for? If you’re working on a feature, what is the end state of that feature? What is the end state of that business? Who is actually your end customer? Your end customer may not be the team that you’re immediately working with and who is the decision maker for that overall flow?

Banu Nagasundaram: One of the simple frameworks, this might sound silly. It’s super simple, is to ask five whys, which is in a customer discovery or any feature that you’re building. Just ask, why is that outcome important? Why is that output important? How is it going to help? Why is it going to be something that helps the customer? Why is it needed tomorrow? All of these questions is just going to help drive a little bit more clarity into the bigger picture and motivations for the customer on how they make investment decisions and choices in your particular products and features that you’re building. And that’s it. I want to leave you with one fair warning. If you try that five why’s with your partner, that’s on your own.

Julie Choi: All right. Thank you. Banu, that was great. Thank you so much. It is my honor to introduce Lamya Alaoui, a dear friend of mine and I’m so thankful to you, Lamya, for agreeing to give the closing talk of the night. Lamya is currently Chief People Officer at Hala Systems, and she has been committed throughout her career to supporting organizations as they shift behaviors to align their talent strategies with their business objectives. Her corporate background includes over 15 years of experience in talent acquisition and management, where she has had the opportunity to build teams for companies such as Bertelsmann, Orange, Groupon, Google and Microsoft. Her work experiences span North America, Europe, Asia, Middle East, and North Africa. Let us welcome Lamya Alaoui.

mosaicml girl geek dinner lamya alaoui speaking hala systems

Hala Systems Director of People Ops Lamya Alaoui talks about 10 lessons learned from building high performance diverse teams at MosaicML Girl Geek Dinner. (Watch on YouTube)

Lamya Alaoui: Thank you so much, Julie. Hi everyone. Thank you. I don’t see Sarah here, but I want to thank her personally because she has been so patient and Angie as well for inviting me here. How is everyone feeling today? This is the non-tech talk. This is the people talk. A quick background before we get into it. At Hala System we develop early warning systems in war zones. Not the type of things that you broadcast usually when you are in this type of settings, but we’re hiring as well, especially for our AI team. And Julie announced my promotion that no one else knows about, even in the company so, thank you. That stays here, please, in this room, until next all hands, on Wednesday. With that being said, a little bit of background about me. I’m Moroccan.

Lamya Alaoui: I moved to the US about 10 years ago. I will ask a lot of grace because my brain is wired in French. Sometimes there will be French words that will come out from my mouth so please be graceful about it if you can. With that being said, this is one of my favorite things because as a Moroccan who’s half Muslim, half Jewish, went to Catholic school, I thought I got it covered, but I was the lady the first time I landed in Germany because I was not aware that you’re not supposed to kiss people to greet them. But this is one of my favorite pictures to show and talks or even in workshops, because I’m pretty sure it happened to all of us one way or another. We show up and we think that we got it right. It’s what we’re familiar with and actually it’s not what the other person is expected.

Lamya Alaoui: Throughout my career, I have built teams in very, very different countries, different cultures. At Hala we have over 17 nationalities. Altogether we speak 25 languages. It makes it interesting for the meetings, believe me, where you have side conversational Slack in a whole different language, but you still need to deliver officially in English. The translation is super weird sometimes, but those are anecdotes for another time. That leads me to one of the lessons that I learned very early on. It seems obvious that when you’re building teams and you want them to perform, one might assume you want to know your mission and your values, but it doesn’t happen as often as one might think. The mission is basically your GPS. You want people to rally behind a common goal.

Lamya Alaoui: You want them to believe in that and it’s also what’s in it for them when they’re working for a mission. That means that it needs to be aligned with their own values or their belief system. SVlues start very early on. We all have seen or worked for companies where the values are stated. How many of you when you joined the company saw values listed on the website or have been talked to? How many of you were explained to what are the associated behaviors and expectations when it comes to those values? Oh, we have less people. This is one of the first lessons that I always recommend to people to kind of follow. I learned the hard way, by the way, is that know your mission, state your values. Values should come from the leadership of course, but also account for it in the hiring process, which we will get a little bit into it, but be very clear about what are your expectations are when it comes to values.

Lamya Alaoui: If I’m thinking about transparency, for me, it means having information that I need to do my job, but some other people, they think that they need transparency. They need to have access to everything. We all have someone like that in our companies. Don’t we? Setting those clear expectations and associated behaviors are very important. And mission is like literally your North Star. This is why it’s really important. The second one is seek first to understand. This is from Stephen Covey. There is a second part to it which is, then to be understood. When you are building diverse teams, we all come from different backgrounds, we have different understandings, often we speak different languages. We can also come from different cultures, high context or low context and if we don’t have mutual understanding we cannot succeed because then everyone is convinced that they’re right, that their approach is again the one that everyone needs to follow, but that kind of hinders teamwork, which is actually essential to high performing teams.

Lamya Alaoui: Listen. We all think that we listen pretty well, but we actually don’t because as human beings we have been trained, for the last few centuries, to listen to answer. We’re almost never listening for the sake of just listening and there is amazing research about that, that I will be more than glad to share. In working in very diverse teams or diverse teams in general, listen, help, create understanding, respect, which is a very important when it comes to having high performance teams as well. How many of you would say that they listen really, really, really well. Okay. We have four people in the whole room. That gives you something to think about. That’s another, and I’m a good listener, but still a lot of work to do with that.

Lamya Alaoui: Acknowledge that you will face cultural differences. When you join a new team, we all want to get along. Again, its human nature. We want to belong somewhere. We’re super excited. It’s a new job. We just went through the hiring process. We landed the job and then you show up in that meeting or you’re meeting your team and sometimes you see people that you have nothing in common with, and you still try to ignore that, which is again, human nature. Beautiful thing. Just as a manager, as a team member, be prepared that yes, we will have some challenges. We’re not seeing things the same way. One of the best examples that I always give here is like, I’m from what would qualify as a high context culture, which is there is no explicit direct messaging.

Lamya Alaoui: Basically I can be in my office and say, hey, I have a lot of work and I’m hungry. My expectation is that my coworker will just get it and go get me some food. Now, imagine if I’m having someone from a low context culture. In their head, they’re like, I mean, you’re going to go get your food when you’re done. That creates a fracture that it’s not even intentional because there will be resentment from my end. Why didn’t he get it? Having those conversations upfront is quite helpful because then people know again, what to expect. Assess the degree of interactions. When teams are being built, most of the time what happens is no one is thinking about how often the team members will have to work together, at what intensity.

Lamya Alaoui: Before starting to build the teams or adding members to your teams, just make sure that, hey, how often do they have to have, I don’t know, sprints. If it’s once a month, eh, you might want to have people that are pretty much aligned having somehow the same thought process and we’ll get to that in a little bit. If they’re not interacting a lot, you have a lot more room to assemble your team. Always, there are three degrees of interactions. We have low interdependency, which is people are doing their things on their own and sometimes it’s like going in a [inaudible]. Medium, which is a hybrid and then high interdependency, which the output from someone becomes the input for someone else.

Lamya Alaoui: Therefore, things need to be structured in a very certain way in making sure that people get along and understand, again, what’s expected from them. Communicate, communicate, and communicate and when you think you are done, communicate some more. One of the things about communication is that we all think that it happened. Again, the question, oh, disclosure, I ask a lot of questions. I mean, I used to be a recruiter. It comes with the territory basically. How many of you thought in a meeting that you were crystal clear and then two days later you discover that you were not.

Lamya Alaoui: When it comes to communication in teams, again, thank you, Slack, everyone in meetings, or usually where everyone is doing a lot of things at the same time. You’re talking, you think that you got your point across. Always ask a question at the end. What do you think? What are your takeaways? Just to make sure that people are on the same page and teams, high performing one’s, one of the best practices is having action items assigned, even if it’s an informal meeting or if a conversation happened on Slack, on, I don’t know, any other platforms. Even sometimes now, oh, we have also Asana now people add us to Asana and they think that, hey, we’re good.

Lamya Alaoui: No, make sure that the person actually went in, is understanding what you’re waiting for or expecting. Diversity and leadership. Clearly icons still have a long way to go in terms of diversity, but again, high performance teams when it comes to diversity need to see that diversity reflected in the leadership and in the management. Research shows that usually entry level management, there are sometimes good numbers, but as soon as we go to director, VP and above underrepresented groups tend to become less visible or they’re non-existent and C-suites and VP, EVP, SVP type of roles.

Lamya Alaoui: It’s also very important that the leadership reflects the diversity that the company is striving for. Again, an obvious one, hire the right people. This one is very dear to my heart because as I said, I’m a recruiter. Again, we’re only humans. We tend to hire people who are like us, who think like us, who share our values. Anyone wants to venture what happens when you hire people who are like you and your team and you’re building a team of clones, basically from a cognitive perspective. Anyone? Yes.

Shika: You wouldn’t get to learn something different from what you already know.

Lamya Alaoui: Yes. Anyone? Thank you so much. What’s your name?

Shika: Shika.

Lamya Alaoui: Shika. Thank you, Shika for volunteering. When you build a team of clones and this is a tech talk after all, how do you think that you can innovate? How do you think you can perform if everyone is thinking the same way and not challenging the ideas that are being discussed? Chances are very, very, very slim.

Lamya Alaoui: When you’re hiring the right people for your teams you have to look at three things. The first one is the values, because that’s, again, what will be the foundation of the team that you’re building. The second one is what’s missing from your team. An ideal team has five type of people. You have a theorist, you have a strategist, you have someone who is an analyst, you have a manager and you have an implementer. When we hire our teams usually we hire in desperation because you know, you got to deliver on that project.

Lamya Alaoui: No one is thinking about the composition of the team in itself. In tech, my experience showed we have a lot of theorists and strategists and a lot of analysts, sometimes zero implementer and zero managers. Things don’t get done somehow. Always look at what’s missing and finally, the third part is people who are willing to learn, who have a curious mindset and are eager to grow, because this is how actually innovation happens. People who are not afraid basically of questioning the status quo.

Lamya Alaoui: The other lesson is constantly scan for misunderstandings and ways to clarify. Again, here I will go to the low context, high context type of thing so, let me give you an example. I think it might be easier to make my point that way and I will ask a question at the end. In high context cultures or in some cultures when you ask questions, no, is not an acceptable answer. You have to say yes and some others you have to say no three times before saying yes.

Lamya Alaoui: Imagine when you have people from different cultures in the same room and someone is asking a close question that requires a yes or a no. What are the chances, based on what I said, that you will get an accurate answer to your question? 50%, 60%, 20%, a hundred percent. For this always ask open ended questions to give people space to answer without having to break their own boundaries and to make sure that you are getting clarity in the answers. How many of you were in meetings where you felt that you were misunderstood when you were speaking? What did you do? Anyone wants to volunteer again? When you felt that you were misunderstood? Yes.

Angela: Have you summarized in a written notification? I gave action items to the people I wanted clarification from that helped with the misunderstandings.

Lamya Alaoui: Thank you, Angela. One of the things that is really helpful and again, best practice for high performing teams, always summarize and put things in writing and this setting English is the working language. We do have a lot of people, I know in my team, I have a lot of non-native speakers. I’m not a native speaker so to make sure that the message gets across and that it’s clear it’s always in writing, always, which we tend not to do because everyone is working long hours or you have back to back meetings. Then at the end of the day you just want to finally get to do your work and you never summarize in writing what happened in those meetings.

Lamya Alaoui: Finally, the structure. Build a structure that is around diverse teams. When we’re usually building teams we have, we’re looking at what now is referred to as visible diversity, which can be ethnicity, religion, gender and many other things, but we’re never looking at the cognitive one that also has a heavy part, especially in tech. Those are recommendation that I’ve always lived by every time that I join a new company. We do a reintroduction and relaunch of the core values. We redefine them. We create, and this is like a company work, it’s not just someone in their corner doing it, but it’s a collaborative work setting what are the core values, what are the expected behaviors and the ones associated with each one of the values, and then it’s communicated throughout the whole company. When it comes to performance review it needs to be designed with underrepresented communities in mind.

Lamya Alaoui: Performance reviews, if you look at the history of how it was designed, it was designed for a very specific community or population. It doesn’t fit other people. Just leave it at that. Work on creating a performance review management system that is more inclusive. Yeah. Targeted networking. Again, this ties to the diversity and leadership. One of the things, again, hiring in despair. When those roles are open, people tend to hire fast from networks that they know because they don’t have networks that are already established. This is something that is quite helpful. I don’t think there are any people in the HR realm tonight, but all of you can be ambassadors for something like this to be established in your companies. And again, clear expectations around the engagement and the roles. What are the pathways to promotions? Again, if you look at internal promotions or internal applications, woman tend for, this is just an example, women tend not to apply to internal openings when it comes to higher positions in the company.

Lamya Alaoui: Sometimes just because they don’t check all the boxes so the work starts before that, where we actually need to make sure that the job descriptions or the job openings are reflecting and are being thoughtful about underrepresented groups. Finally, this is one of my favorite things, Kaizen. There is always room for improvement. Performance and diversity are a very long journey. Each company is at a different stage or different phase. Just ask questions, be kind because not a lot of people think about it when it comes to those things and be as curious as possible because at the end of the day the only way to know other people is to ask questions. Thank you so much everyone.

Julie Choi: I want to thank all of our speakers tonight and for all of you, for being such an incredible audience. We have people who have been watching these talks outside in the overflow and I’m told that, I think it might be better if we just all go out and just mingle. Get some fresh air after all this time is for us to connect and network. Why don’t we continue the conversation outside? Thank you everybody.

mosaicml girl geek dinner ukranian original borsch tshirt

This site reliability engineer discusses Ukranian borscht or machine learning, or both, at MosaicML Girl Geek Dinner.

mosaicml girl geek dinner speakers tiffany williams banu nagasundaram laura florescu julie choi lamya alaoui shelby heinecke angela jiang angie chang amy zhang

MosaicML Girl Geek Dinner speakers after the event: Tiffany Williams, Banu Nagasundaram, Laura Florescu, Julie Choi, Lamya Alaoui, Shelby Heinecke, Angela Jiang, Angie Chang, and Amy Zhang.

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Posted on

Cadence Girl Geek Dinner – Lightning Talks & Panel! (Video + Transcript)

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Transcript of Cadence Girl Geek Dinner – Lightning Talks & Panel:

Angie Chang: Hi. Welcome everyone to the Cadence Girl Geek X event. My name is Angie Chang, and I am the founder of Girl Geek X. By means of introductions, I mean to say my name’s Angie Chang and I’m the founder of Girl Geek X and Women 2.0. I also spent some time working at a company called Hackbright Academy, which is a women’s coding bootcamp. I also spend a lot of time talking about women starting high-growth, high-tech companies, working in tech, blogging about it. Sometimes I write listicles of women architects, women CTOs, VPs, security chiefs and such.

Angie Chang: Let’s see. Why don’t we do something? Why don’t we pretend that we have our “Hello my name is” name badges, and write in the chat like, “Hello, my name is Angie Chang, Founder, Girl Geek X,” and then put in your LinkedIn URL. I’ll start. I’ll copy and paste this into the chat. That way… I’ve noticed that people at our Zoom events have been sharing their LinkedIn profiles. I want to be the first person to say that, yes, let’s definitely share LinkedIn profiles and share a bit about ourselves more than we can see on these Zoom meetings or Zoom webinars.

Angie Chang: Let’s see. A bit about Girl Geek X. We’ve been doing Girl Geek Dinners since 2008. We started with events at Google and Facebook when they were smaller companies in 2008, and then we went to all these different tech companies. We went to biotech companies. We went to a bunch of companies I’d never heard about before.

Angie Chang: But the thing about that is once I was there, I would learn so much about that company. I’d learn about the industry. I’d learn about the women that worked in it. I would see their job titles and I would be very inspired and educated at that point to recommend that company say to friends. Also, it was really great for networking.

Angie Chang: Hopefully if you have time, you can hang out tonight. Later at seven or so, we’re going to start the networking and Zoom breakout rooms where you can actually chat with each other and connect more in person. But if you can’t, it’s okay, this talk is recorded. All of our events are recorded and put on YouTube later, and that URL is youtube.com/girlgeekx.

Angie Chang: What else? If you want to look at all the events that we have hosted in the past, they’re all on our website. It’s at girlgeek.io/attend, and you can find all our previous events. For example, we were at Discord a week or two ago.

Angie Chang: We just wrapped our annual Elevate conference, which is something we do every year for International Women’s Day.We have an all-day event celebrating women and having a bunch of exciting women leaders speaking about topics like mental health and leadership, not just very ambiguous things. We literally had a keynote on decision making from a VP of engineering.

Angie Chang: We always have a call for submissions. People can in the fall apply to speak at that conference. About 10 of those people who applied to become speakers became speakers at Elevate. There’s definitely a chance that if you submit something, you can be selected to speak. You can also sponsor a Girl Geek event like the Cadence event, where you have an opportunity to put your women on stage and give new tech talks followed by a panel. Then we have some networking.

Angie Chang: There’s just so many companies out there. That’s I think a great opportunity to get out there in front of a bunch of eyeballs and create some great talks that we then put on YouTube. What else? Oh yes. We have a Q&A. If you have a question throughout the course of the event tonight, feel free to put it in the Q&A or you can ask it in the chat, but there is a Q&A feature, so feel free to use it. Some of our speakers may want to answer questions, but they may not have time to answer them on screen so they can pop into the chat and answer them later if you ask them. I want to do my first introduction.

Angie Chang: Alinka is the chief legal officer and corporate secretary at Cadence. She’s responsible for Cadence worldwide legal operations. She has served semiconductor and software companies and her entire in-house legal career at a lot of companies that you may have heard of. Before moving in house, she was in private practice for a decade litigating chemical product liability matters. Welcome Alinka.

angie chang girl geek x alinka flaminia cadence
Cadence Chief Legal Officer Alinka Flaninia welcomes audience to Cadence Girl Geek Dinner 2022.

Alinka Flaminia: Thanks Angie. Thanks for the introduction. On behalf of Cadence, we are so honored to partner with Girl Geek X to host this conference in celebration of Women’s History Month. Women have played a significant role in Cadence’s 34-year history, and I’m thrilled to share with you some of our efforts to create a more inclusive and equitable workplace for women and underrepresented groups in STEM. But before I describe some of our DEI efforts at Cadence, let me first tell you a little bit about the company, for those of you who are not familiar with us.

Alinka Flaminia: Cadence is a pivotal leader in electronic design, building upon more than 30 of computational software expertise. Manifesting our intelligence system design strategy, Cadence delivers world-class software, hardware, IP across all aspects of the design electronic systems. Our customers include the world’s leading companies, delivering extraordinary products from chips to board to complete systems for the dynamic market applications, including cloud and hyperscale computing, 5G communications, automotive, mobile, aerospace, consumer industrial and healthcare. It’s fantastic to work at a company where the same set of tools enables innovation across such a diverse set of industries.

Alinka Flaminia: Actually for me, it’s kind of mind blowing, and I believe that the true enabler behind Cadence’s success is our high-performance inclusive culture. Our one Cadence, one team spirit is core to who we are, and embracing diversity and fostering inclusion are key tenets of our Cadence culture. Cadence encourages and fosters diversity equity and inclusion on many fronts, internal and external, through recruiting and university partnerships, education, leadership training, pay equity and promotion and building community.

Alinka Flaminia: A few examples include our sponsorship, diversity and technology scholarship programs for women, black students and Latinx student to support these underrepresented groups in their pursuit of STEM education. We celebrate and support our employee-led inclusion groups for black, LatinX, veteran, LGBTQ+ and women employees and their allies to build community at Cadence and beyond. Cadence offers professional development through advanced leadership and mentorship programs specifically geared toward our girl geeks and black and Latinx employees.

Alinka Flaminia: Cadence is investing in the pipeline of a more diverse employee population through partnerships with nonprofits and organizations that serve underrepresented groups in STEM like the National GEM Consortium, Society of Hispanic Professional Engineers, National Society of Black Engineers, Society of Women Engineers, Out In Tech, Girls Who Code, and I could go on.

Alinka Flaminia: Our culture has been recognized globally, earning Great Places To Work awards in 14 countries around the world, including seven years in a row on Fortune’s 100 Best Companies To Work For, rankings on Europe and Asia’s Best Workplaces, listed on Newsweek’s Most Love Workplaces and the Best Place To Work For LGBTQ+ Equality on the Human Rights Campaign’s 2022 Corporate Equality Index. Diversity, equity, and inclusion are top priorities for me and the rest of the executive management team, and our board of directors.

Alinka Flaminia: We are excited to work with Girl Geek and highlight in this conference some of our amazing innovators at Cadence and hear how they are helping us solve technology’s toughest challenges. I am not a technologist, but I am most definitely a geek. Regardless of your specific interests, we girl geeks are united by our passion, our drive, and most especially by our curiosity. The speakers and panelists are some of Cadence’s very best girl geeks, I’m certain they’ll stoke your curiosity about our business and provide great tips for advancing your career. Thank you for joining us today, and I’ll now turn it back over to Angie to introduce our first speaker.

Angie Chang: Thank you, Alinka for that warm welcome from Cadence. Our first speaker tonight is Helen Zhan. She graduated from the University of Tianjin in 2000 with a double major in computer science and economics. She began her career as an IP/SOC engineer at NEC in Japan focusing on both design and verification. Shortly after that, she joined Cadence, where she’s been for the last decade plus. Her passions for debugging failures and finding the root cause of issues has allowed her to grow her career at Cadence. Welcome Helen, who is joining us from Beijing tonight.

helen zhan cadence growth engineering beyond metrics
“Growth Engineering Beyond Metrics” by Helen Zhang, Cadence Design Engineering Group Director.

Helen Zhan: Thank you everyone. I’m Helen from Cadence. Today, I would like to share my story along with the team girls. Okay. I will use them putting in my later talk. I will use the integrated IC, IP, IPG, DDR, LPDDR, PHY and Mbps in my later talk.

Helen Zhan: Firstly, I would like to introduce the function group of my team. Similar to other any centers, we do have the six different function groups along with our journey like the different kind of the design, verification, solution team and the success team. With all this different function team, it provide the complete system solution for the memory system of each customers.

Helen Zhan: Our team did start from the 2011. Then we have the big growth from that year. After three years, we did break into top four of the senior IP core ranking and along with that long journey, we did achieve many milestones of the word first silicon to ensure our IP become more challenging to the market.

Helen Zhan: Today, I would like to share with the story of the team grows. When we start the team, we need to find the proper goal of our team. So we had we will be the market co-developer of the team start. We did have some big investigation of the market. We find there are two type of the current IP company. One is like the flea market, which you could find everything you want, but that may not give you repeatable supply and with good quality. Another is like the mega store. You could find the product with a good quality, but it may not satisfy all your requirement.

Helen Zhan: Where should we go? We leave this question to the marketing investigation. Then we find in cloud market, we do have the different application like the mobile application, consumer application, cloud application and also automotive application. Often shows the market orientation thing. We figure out our target is to build a different style of the IP vendor and supplier to make a customized configurable solution with a good quality to fulfill the different customer request, satisfy all the products.

Helen Zhan: With this clear goal, then we will build a good team and to make the team improvement to satisfy the market requirement. As I started from the beginning, we did have six different function groups. Today, I will check the digital design group as an example for the team roles. When the team grows, we need to divide the team into different separations to have the different function focus. With that kind of, we could have the expert in each field to make our product become more productive and have the leading age technology.

Helen Zhan: With this subdivision, we need to have the clear ownership of each different field, and we do not want to limit any engineer in that field only. We would also like to expand his focus in different area. We want the clear ownership of the each field. In the meantime, we also want the mixed function focus of the different area. With this kind of definition, we could increase the team skill and we could also back up each other and improve together. This makes the team become better and more productive and efficient with a single one.

Helen Zhan: In the meantime, with a complex of the IC product, we have the clear boundary of the each function group, but we want that boundary to have the team have its own focus. We do not want that boundary become any barrier of the cross-function group communication. we would like this boundary become multi care. That will stress our boundary to avoid any backslide and any part we are missing in the development that to avoid any for other issue and surprise in the later production.

Helen Zhan: With all this strategy and the teamwork, we did build a fully verified DDR subsystem. As you can see in this picture, we could support the different configuration product, and it could have the customized features, which could satisfy the different market requirement like the mobile, automotive, cloud and consumer. That also help us that our DDR product in the leading edge to have won more customer today

Helen Zhan: With this 11 years journey, our team is already increased to a big size. How do the junior engineer now become the senior engineer and the senior becomes an expert and with the supervisor like the leadership? But how to help this big team become more stable and more productive. We considers the two area. One is from the technic side. We want the team always stand at the leading edge of the product. That allow us to always to work with a new protocol and to achieve the highest speed in the world.

Helen Zhan: When we start a product, when we define the protocol, that means there are many unclear area which won’t be exist when we start the product that need us to have the flexibility design to accommodate any new requirement coming later. Also, we need to use our experience to doing the predict and analyze for the orientation to avoid we are going to operate position to the market. Also the high speed is everyone is chasing today. We need to build the high speed architecture to satisfy the design requirement.

Helen Zhan: With this technical innovations, that allow our expert to have their own focus that ensure they always have the interest of this product in their career path. In the meantime, to leverage different Cadence, we could always use the latest methodology and advanced technique that help us to use a new design methodology in that every field and everywhere along with our IP development. Also, we adopted advanced flow and tool in our IP quality check to ensure we have the product with good qualities.

Helen Zhan: The silicon proven is another advantage here because with more and more high-speed requirement, the silicon proven as a big fact for the customers who want the IP supplier to provide. In the meantime with a big team, we also want to improve work efficiency. When we start any new product or any new feature development, we will need to avoid any one-off development to make our effort could be reusable or repeatable in the later product. In the meantime, we also need a comprehensive quality system to have the issue being detected earlier, to send a alert to the design team or other development team to avoid any later surprise in the customer products

Helen Zhan: With all this QC and the strategies we development the automation flow that could help us to release the manual resource to focus on the technical side. In the meantime, it’ll also reduce the effort and error with the manual operation. With all this strategy and technical, I believe that communication is the most important to have all this done. First, I would have to introduce these three word, look listen and learn. We need to look what the team’s working like, what the customer require and what’s the marketing requirement.

Helen Zhan: We also need to listen to the voice from everywhere that will have the leader to clear the request and clears the issues in our team and in the market and brand customer. With all these facts, we could learn what we want to do in the next space and in the next step. Also, we could find what we should do to solve the current to and improve the team. Then I believe the different type of the talk is also very important. That is not the leader to talk to the team member. It also want team members to talk his mentor, his mentor, and his leader to share are the different ideas.

Helen Zhan: I believe everyone did have its own thought on his work. Then we need a clear communication between each channel to help the team members understand the requirement and purpose and the goals of the leader and the management team and also help the leader to understand the consent or problems in the team member side to help them to solve that. With all this good communication and consideration, we will make this become execution that is to help us to make our goals and consideration become true. Thank you.

Angie Chang: Thank you, Helen. Our next speaker is Elena. She is currently the Global Public Relations and Social Media Director at Cadence. Previously, she had held communications and marketing roles at AgilOne, Coupa Software, SugarCRM and more. She spent over five years freelancing and consulting to communications and marketing. Welcome to Elena.

elena annuzzi cadence finding your growth career path

“Finding Your Growth Path” by Elena Annuzzi, Cadence Global Public Relations and Social Media Director.

Elena Annuzzi: There we go. All right. Thank you, Angie, for the intro. Hello, everyone. Welcome. As Angie said, my name is Elena, Elena Annuzzi. I’m the Global PR and Social Media Director at Cadence. My presentation tonight is going to be focused on Finding Your Growth Path.

Elena Annuzzi: How many of you have ever felt stuck in your career and you’re trying to figure out how you might be able to move it along? I think a lot of times it’s seen that promotions are an obvious way to move yourself along in your career, but there are also a lot of other things that you can do to propel your career and take it to the next step. I’m going to talk to you tonight about my own personal growth journey and also impart some tips that you can leverage to find your own unique growth path.

Elena Annuzzi: With that, I will start with my own personal growth journey. I’ve been in technical communications positions for the last 22 years. A lot of my career has been spent handling public relations, but I’ve also spent a lot of time doing analyst relations programs, content marketing, social media marketing in customer marketing programs. When I started, my role was strictly doing PR and I worked in house in a corporate environment.

Elena Annuzzi: A lot of times people in my career field do start in the PR agency realm, where they have access to lots of training and resources. I kind of missed out on that a little bit. I did find it a little bit difficult to be in a corporate position and kind of rise through the ranks in there, but I absolutely did best that I could to try to learn different facets of the business. Ultimately, I decided I wanted more growth, which led me to a path to consultancy. When I did that, I worked for a few Bay Area PR firms and also had clients of my own.

Elena Annuzzi: I definitely had my hands full for sure, but what that it is it kind of pushed me out of the comfort zone PR box that I started in. I got to dip my toes into other areas such as the ones that I mentioned, analyst relations, customer programs, things like that. It also imparted a lot of confidence in me as well because I had clients who were in all different industries, big or small. Oftentimes, if they were small, they might have been a one person marketing shop, so they were looking to me for leadership.

Elena Annuzzi: That really gave me the confidence to become the leaders that they needed and also acquire a much broader skillset than I ever anticipated. That was really a great period that I think then after five and a half years, I decided to reenter corporate. As I did that, I came in at that point as a very experienced person, leading teams and working on projects to get visibility for the firms that I worked in a lot of times from the ground up. They had never had PR before and they didn’t know what to do, so I kind of in there to build it back up.

Elena Annuzzi: Now, for the last seven years, I’ve been at Cadence. I started at Cadence as a senior manager in an individual contributor function. Now, I’m the director of the group, and I manage a team. The team is responsible for handling anything that is publicly distributed in the form of news releases, as well as contributed content. All the social media platforms are managed by our group and a variety of other things. We also work very closely with executive management. There’s lots of things that are sensitive or require them to do media interviews and things like that. Definitely something I really enjoy.

Elena Annuzzi: I’m glad that I had the opportunity to try lots of different things. I kind of take my consulting experience and I’ve sort of taken that along with me as I’ve gone along through my career. I try to always look kind of at the company from an outside view and try to establish, “Okay, well, if I was consulting, what would I recommend that this company do?” I’ve kind of had an interesting path and I’m currently very happy at Cadence and I have a great team, all amazing people.

Elena Annuzzi: Let me now continue with some tips. I will share some growth tips with you. The first one I have is make sure that you’re having open in conversations with your managers. If you haven’t discussed a growth plan already, make sure that you do that. If you haven’t really thought about it, maybe write some notes down before you have that conversation so that you then go into that conversation prepared.

Elena Annuzzi: Another thing I would say is to offer to take on new projects that are outside your comfort zone because then you’re sort of pushed to try something that you may not have otherwise done. You may experience a very pleasant surprise and that something worked out so fantastic for you that it would definitely be worthwhile to make the investment to try something different. Another tip would be to find groups who have similar interests to you, and that way you can gain new inspiration from others, as well as make some good connections.

Elena Annuzzi: There’s lots of ways to do that online today, for example, and you may be familiar already with social media groups in your related career field, so feel free to take a look at those. LinkedIn is probably the most obvious place, but other platforms have groups as well that relate to professional fields. The other thing too is if you have local meetups, check some of those out or even leverage your university, if they have alumni groups and more specifically alumni groups within your field of study.

Elena Annuzzi: Lastly, maybe volunteer with an organization that is also passionate about the things that you’re passionate about from a work perspective, like say you’re volunteering with a STEM group and you’re in a STEM field. You may meet some great connections that way and gain some new insights.

Elena Annuzzi: Continuing on, the next tip is be relevant. What I mean by that is making sure that you’re always kind of staying fresh and up to date on what the industry’s current best practices are. How might you go about doing that? You can have conversations with others, whether they’re peer groups in your company or people that you’ve worked with in the past who hold similar job functions and just kind of ask them how they’re approaching their job. Obviously certain things are proprietary, so there’s limits, but you can kind of get a good gauge as to how others are tackling a similar job to you.

Elena Annuzzi: The other thing that I’ll recommend, and this is not meant to sound intimidating to employers in the least… It’s actually for your benefit… is to check out job descriptions. The reason that I say that is you can take a look at job descriptions in a role that’s similar to yours and even look at those that are above your level because then you’ll quickly figure out what companies are demanding of people in those functions today. You can quickly realize, “Okay, I have these skills, but maybe I’m missing a couple,” so you can identify the gaps and then work to figure out how you can get that experience in your current role.

Elena Annuzzi: That would be something to talk to your manager about. If you’ve identified a gap, “Here’s something that I’m interested in trying, let’s do that.” Then in looking at the job of positions above yours, then you also have a gauge of what to shoot for kind of in your next step. Similarly, if you realize that you have some gaps, then you can work to address those. The next thing I would say is acquire new skills, taking new courses or attending conferences where you’ll have access to new information that you may not have otherwise had, or ask your employer, your HR department, or your manager about job sharing.

Elena Annuzzi: If you’re not familiar with that concept, it would be where you essentially do a job swap for a limited amount of time. Let’s say you have a peer organization and you want to take on some function of your peer because you have an interest there and want to explore that, you can maybe switch jobs for five hours a week and both of you are actually gaining a new skillset by doing that. The next thing I would recommend is mentorship. I would say find a mentor if you don’t don’t have one or be a mentor. Both things are absolutely critical.

Elena Annuzzi: I am so glad that over the past five years or so, I’ve seen a lot of mentorship programs kind of budding in the industry. That’s really a great thing to see. I kind of wish that I had those types of things when I was first starting my career. Cadence also does a really great job with this, by the way. We have an internal mentorship program where they match pairs up. It’s really just a phenomenal thing. If you’re not already in the realm of finding a mentor or being a mentor, I highly recommend that. The mentor for you can obviously serve as a sounding board. Whether the person’s in your industry or not, or maybe they’re your manager, maybe they’re someone who’s completely disconnected from your field altogether, it’s great to have somebody who can function as that sounding board for you.

Elena Annuzzi: Also being a mentor. It’s such a rewarding experience to pay it forward. I highly recommend that you try this and there may be some of you who currently aren’t managing a team, let’s say. If that’s the case for you, being a mentor, that will give you leadership experience. I highly recommend that next.

Elena Annuzzi: Next, here’s a few points to keep in mind. No two growth paths will look the same. Try not to compare yourself to others. The next thing I’ll say is always be curious. I always tell my team members the minute you’ve accepted the status quo, you’ve stopped growing in your career. Always keep that explorer hat on and try to figure out what you could be doing that’s different. The next thing I’ll share is ensure that those new areas that you decide to explore align with your organization’s business. If what you want to try aligns with the business, then it’s a much easier sell when trying to get buy in.

Elena Annuzzi: The next thing I’ll say is surround yourself with people who support you, whether it’s people inside your company, outside your company. It could be a mentor or just your team members, your manager, people in peer groups, make sure that you have great support all around you. Then the last thing I’ll say is have fun in the process. We all need to have some fun.

Elena Annuzzi: To conclude, I want to encourage all of you to start taking steps today to grow your career path. Those moves that you take today will start impacting your career now and well into the future. As a key takeaway, remember that it’s you who’s in the driver’s seat. Thank you very much for your time.

Angie Chang: Thank you, Elena. That was excellent. Our next speaker is Didem Turker. She’s a design engineering director in the IP group at Cadence, where she leads development of high-speed, high-performance communications circuits and systems. Before joining Cadence, she was the Senior Design Engineering Manager at Xilinx in the service technology group. She holds 11 US patents and authored numerous technical papers in the field of analog and mixed-signal circuit design. Dr. Turker has a PhD degree in electrical engineering from Texas A&M University. Welcome Didem.

didem turker melek cadence engineering director ip group effective technical presentations a powerful tool for your career success
“Effective Technical Presentations: A Powerful Tool for Your Career Success” by Didem Turker Melek, Cadence Engineering Director, IP Group.

Didem Turker Melek: Hello. Thank you, Angie. Okay, let me share my screen. Okay. All right. Okay. Thank you for this introduction, Angie. Hello, everyone. I’m Didem. Today, I’ll talk about effective technical presentations and how they have a key role in your and your team’s success. Before I begin, throughout my career, I found that being able to will communicate my work to my colleagues clearly had significant impact on the type of feedback that I got, but also on my work being recognized.

Didem Turker Melek: Over the years, this is something I championed in the teams that I worked with and we always saw really positive results. I’m hoping that this discussion will be helpful today for you too. Okay, let’s begin. When we talk about technical presentations, they are different than the general presentations that we may give to a wider audience.

Didem Turker Melek: We also need to share data and talk about more detailed material with certain technical complexity. Now, throughout our career, there’ll be different occasions where technical presentation may be called for. This could be academic conferences, customer presentations or when we are collaborating across different organizations in our company, it could even be within our own team if this would be to our close peers, our colleagues and maybe our management.

Didem Turker Melek: It is this last one that I want to highlight because this is a situation that we encounter really frequently, yet it’s also the one that we overlook the most. I really want to emphasize how important it is to communicate technical information through well prepared, clear presentations and especially around the audiences, people that you work with every day.

Didem Turker Melek: Even though the occasions and audience may be different, there are common goals when we are giving you technical presentation. The first one is effective information sharing. Being prepared with proper organized material will make a big difference over opening live results, showing live data or giving a verbal description. This is true even in a more informal team setting because for a discussion where we have technical complexity to discuss, the audience will have a hard time following if you’re doing it verbally.

Didem Turker Melek: The second goal would be to get feedback. You probably have bright people from different technical backgrounds and experience living in your audience, so use that brain power. The best way to get good feedback from them is by communicating your findings in a clear way. Third goal would be to train others so people can learn from your experience and maybe save some time.

Didem Turker Melek: Finally, it’s documenting our progress. The presentation material that you prepare will serve as good documentation of your work. It’ll help you look back in the future to track where you have been at a certain time. It’ll also help others in the future to look back and understand your work better. Depending on the situation, one of these goals may be more dominant than the others in your talk and you can prepare your material accordingly.

Didem Turker Melek: Okay, let’s talk about some presentation tips. First is know your audience. It’s important to know who the target audience is and their familiarity with the material. But here, what I want to emphasize is that they are not you. What I mean by this is when we spend so much time in the details of our work, we tend to forget that what’s obvious to us is probably not obvious to others. It’s important to keep this perspective in mind when preparing your material.

Didem Turker Melek: I think something that helps with this, and it’s a really good strategy overall, is to have a story. As you plan your slides, remember to build this story so you can bring your audience up to speed and along with you. Start by setting the big picture, why we started. This would be where you talk about the goal, the problem definition and big picture stuff. Next would be how we got here. If there were previous discussion or decisions that were taken, try to recap. Next is where we are now. This was the main discussion that you want to cover. Finally, where we go next. It’s always helpful to finish with next steps and a plan.

Didem Turker Melek: Now, another very important tip is use your voice and your point of view. I can’t emphasize this enough. When you are presenting your work, please remember that you are an expert and this is true even if the audience have people with more experience. You are the expert on your own data. What can we do? Each slide should have at least one key takeaway that you highlight. Please avoid doing a data dump and letting the data speak for itself. It’s really important that you make observations because that’s your contribution. You can use metrics to help people interpret the data, metrics such as target value specification, maybe margin to that spec and so on.

Didem Turker Melek: Finally, don’t be afraid to raise possible issues and don’t be afraid to ask questions. Let’s look at some examples. Here is a slide you may encounter in a technical presentation. Now, this is what I would call a data dump. This is a bunch of numbers and while it may be obvious to you, for someone who just saw this and has only minutes to digest, it’ll not be clear. What’s the takeaway here? Is there a target value and what are the units?

Didem Turker Melek: How can we make this better? First, notice that I removed some columns. When you have a large amount of data, it’s helpful to do a divide and conquer approach and present it in smaller, meaningful pieces. Now that I edit the target specification, this will help set the key numbers here in this table into context. I’m also using getting a visual help by making the most important column, which in this case is bandwidth mode and using color according to mark failures.

Didem Turker Melek: Finally, in the second bullet, I’m including my key takeaway and observation from this data, which is that we fail the spec at certain cases. Now, in addition to the key observation from the data that I just showed, I can also build up on it by adding more information. For example, I can explain why I think this failure happens and propose a mitigation plan. Now, the goal of this is to facilitate the right discussion. This is why you think the problem may be happening and this is how you think you may be able to solve it.

Didem Turker Melek: By sharing it this way, you can get the right feedback about your plan and maybe come up with a better plan as a team. Okay. I want to pause here and add a bonus tip. While I mostly focused on how you can help your audience better understand the data, there is one significant benefit of having slides like this with clear points. Let’s go back to this slide. You may have attended presentations where someone needs to share large volumes of data, maybe 50 to 100 slides. Every now and then, a slide like this will appear and they will go, “What was I going to talk about here?”

Didem Turker Melek: Now, it can happen to any of us? Instead, if you have a slide like this, now, even if you’re tired or anxious or nervous, or if you’ve just lost your train of thought, you have all the help you need in your own slides. You have the key point that you wanted to make, you have the discussion points to help you, and you have the visuals to make that up. By preparing slides like this, not only you’re helping the audience understand you better. You’re also helping yourself present it in a more clear and easy way.

Didem Turker Melek: Okay, Let’s go with another important tip. Drive the discussion. As the presenter, we are in the driver’s seat. It’s our responsibility to guide the attention of the audience to key points. Please remember that just because something is on a slide, doesn’t mean that the audience will notice it. You can use visual aids like the ones that I used in the previous slide, such as bold lettering, colors, boxes and circles. You can also use keywords such as issue, risk, meets, does not meet to grab the audience’s attention.

Didem Turker Melek: Okay, let’s look at another example. Here I am summarizing some results. This is basically a big block of text. There’s too much information packed in this one slide. It’s too busy and it’s not easy to digest. You may also notice that it’s inconsistent in the way it talks about results. I first see a number about some typical corner. Then I talk about something else meeting a spec. I throw in some comments about some simulation set up or environment and then I throw in more numbers and more setup related material.

Didem Turker Melek: Instead, what I can do is divide this into multiple pieces such as first setup and then the results and clear it up. But there is one more problem that I want to show. I don’t know how many of you here even noticed this, but there seems to be a major issue and it’s buried in a small bullet in the text. Something does not work. If we want to talk about an important issue or make sure that our audience knows about an issue that we observe, this is really not the best way. Now let’s try a different way.

Didem Turker Melek: First, notice that I use the keyword in the slide, issues observed. Now, this will definitely get the attention of the audience and I. There is no doubt that now this issue will be noticed. Next up, I state the issue itself. On top of that, I add some explanation and a possible resolution. I also included data in a graphical format. Now, whenever we highlight a key discussion point, it’s very helpful to have the data to back that up especially in a visual form like this.

Didem Turker Melek: I do want to note that when you include graphs, please remember to include axis titles because again, they may be obvious to you, but it may not be obvious to everyone and it makes it much clearer this way. Overall, when I present the issue like this, it’ll help me highlight and make sure that I get the right feedback and it’ll facilitate the right type of discussion

Didem Turker Melek: All right. Let’s recap with some key takeaways. First, well-prepare, technical presentations are powerful tools to help you communicate your work better, and you can utilize them in your weekly or regular technical meetings with your own team too. Two, if you’re presenting data, do it in a clear and organized way, so you’ll be accurately interpreted. A bonus tip here was that well organized slides will actually help you too when you’re presenting.

Didem Turker Melek: Third, for effective communication, use your point of view and guide the audience’s attention to where it needs to be. I’d also like to add that this is a skill like any other and practice will make it better. Start preparing those slides, everyone. Okay. Thank you. Thank you for your time.

Angie Chang: Thank you. That was excellent. Now, I’m going to bring up our panel and introduce to you our moderator for tonight. Jeannette Guinn leads the demand generation marketing organization at Cadence. Her experience includes a 20 plus year career in B2B tech marketing, owning a floral business and performing vocals of various cover bands across the Bay Area. She has volunteered as a Court Appointed Special Advocate, CASA, to foster children and currently serves on the Child Advocates of Silicon Valley board of directors. Welcome Jeannette.

rishu misri jeanette guinn dimitra papazoglou karna nisewaner cadence girl geek dinner
Clockwise from top left: Rishu Misri, Jeanette Guinn, Dimitra Papazoglou, Karna Nisewaner.

Jeannette Guinn: Hello, good to be here. I’m sorry. My audio cut out when you started the introduction. I’m assuming we’re going to kick this off. Hello everyone and welcome to the Cadence Panel on Women Empowerment. My name is Jeanette Zelaya Guinn, and I’m the Group Director for the Demand Gen Marketing Team here at Cadence. It is a true honor to be here today and it gives us a wonderful opportunity to have our voices be heard and valued. I’m joined here on the virtual stage by three amazing Cadence colleagues. To get this, this discussion going, I’d like to take a moment for each of them to do a quick introduction. Karna, let’s start with you.

Karna Nisewaner: Hi, my name is Karna Nisewaner, and I’m a vice president and deputy general counsel in the legal department here at Cadence. I started my career as an engineer, studying engineering at Princeton before moving to Singapore to teach basic electronics and seed programming at one of the polytechnics there before I pivoted my career over to law.

Karna Nisewaner: I’ve been honored really to be able to work for a number of different technical companies and for the last almost 11 years here at Cadence. I feel like my background in technology makes me a better lawyer for the company and allows me to really engage with all of the different teams and people here at Cadence. To me, that’s one of the best things about starting out your career studying technology is you have all these different options available to you, both as somebody that’s designing the IPs to somebody that’s marketing and telling people about stuff to somebody that’s helping on the backend with the legal patent protection, IP protection, or just basic contracts.

Karna Nisewaner: It’s just really so exciting to be part of what I think of as the future of the world, which is technology. For me, it’s great to be at Cadence, a place that’s really helping all these companies out there build the future. I’m just so excited to see where things can go. That’s why I really love my job and my company.

Jeannette Guinn: Awesome. Thank you so much, Karna. Thank you for being here. Rishu, let’s go to you.

Rishu Misri: Thanks Jeanette. Hi, I am Rishu Misri Jaggi. I work with Cadence as a senior principal technical communications engineer, but that’s a very long title. Doesn’t mean that I do the most important job at Cadence, but what it does mean is that I work with an organization that is at the center of technology, that I work with a male-dominated workforce.

Rishu Misri: Being a woman and a mother working at Cadence, what it means is that I get to maintain a very good work-life balance. I get to spend a lot of time with my kids whenever needed. I can attend to the parent-teacher meetings. At the same time, I can also be at the [inaudible] working and supporting on technology advancements with my other male counterparts. I can volunteer for various Cadence-sponsored community outreach programs that are focused towards empowering other women, kids and students.

Jeannette Guinn: Wonderful. Thank you so much, Rishi. To close it out with Dimitra on your introduction.

Dimitra Papazoglou: Okay. Hi everyone. My name is Dimitra Papazoglou, and I’m an application engineer at Cadence. I support the analog and mixed signal front of Cadence tools. My base is in UK, so it’s a bit late for me, almost 2:00 AM. At the same time, I need to watch my daughter. She’s 12 months old. She’s sleeping, so that’s good. That’s good. We can go and continue.

Dimitra Papazoglou: I’ve been working with Cadence nine years. I joined Cadence straight after university. I can say that I built my career at Cadence. I want to share with you my experience so far. When I started, I realized very quickly how challenging it is to work in this male-dominated industry. I still remember my first visit when I visited customer site and there were 10, 15 men, very experienced, and I was on the other hand very young and with no experience.

Dimitra Papazoglou: Since then, I had been trying to find answers to questions like how should I… What is the right position to stand? How should I use my voice? How can I look confident? In the end, I found all these answers to these questions, and then the support that I needed through a women community that was built internally at Cadence. I had the chance to meet and listen to the stories of several women and quickly realized that these are the women that really inspired me, my female colleagues.

Dimitra Papazoglou: Through them and through their stories, I got also inspired how to get promoted to the next level, how to face my return back to work this January when I came back from maternity leave. I’m really happy to have my female colleagues and those are the ones that really have inspired me and motivated to continue and navigate my career.

Jeannette Guinn: Wonderful. Thank you so much, Dimitra. I wanted to kick off the conversation with talking about current advocacy and what each of us do to empower women and underrepresented groups and why you do it. Why is it important to you?

Jeannette Guinn: I’ll kick it off. I recently became involved with a couple programs that were important to me. In my intro, as Angie stated, I am a board member for the Child Advocates of Silicon Valley program. It’s a nonprofit organization that provides court-appointed advocates for neglected and abused children. I was a former CASA volunteer. If you don’t know what that is, either reach out to me or look it up. It’s amazing. I did that for about five years and it changed my life and it made me realize how badly I wanted to become a mother. That’s where I started off my volunteer work.

Jeannette Guinn: I currently lead the Latinx inclusion group here at Cadence. It’s an opportunity to provide education on the Latin community. I’ve learned a lot and we’re interacting and learning a lot from the other DE&I groups at the company, which is just fascinating. Also, a committee member for the women and tech organization here at Cadence.

Jeannette Guinn: Then in my spare time, I just joined my local Little League board. I have two little girls, six and eight years old, Mia and Zoe. I often call it the Mimi’s and Zozo’s show because that’s pretty much my life. They’re both avid softball players. This was the second year that the league decided to do both baseball and softball under one organization. I saw the lack of softball visibility, and the girls were definitely treated differently. Wasn’t going to sit back and watch. I joined the board and with another female board member, we elevated the softball side significantly.

Jeannette Guinn: Yes, I use my very loud voice when I coach Mia and Zoe’s green Yoda’s softball team. Yes, very involved in that organization. Why do I do all of it besides trying to go crazy? I found myself just constantly complaining about things that were happening around me, and I didn’t want to sit back and watch. I wanted to make a difference and I wanted to make a change. I also want to be an example to my girls. I’m proving that we can make an impact in this world. That’s why I do it. What about you, Rishu?

Rishu Misri: Well, yes, I think I started with saying that I do get a lot of opportunity at Cadence to volunteer for various community outreach programs. I’ve been a member of the Make A Child Smile Society. We do anything that can bring a smile to a child, organizing fundraising events to sponsor the education or painting their schools or looking after their healthcare, taking them out for health checkups, even emotional care. We could take kids out for a day trip if needed, whatever that can make them feel a little better.

Rishu Misri: I’ve also been a member of the FMA committee at Cadence, which works towards female welfare. Under this program, we partner with an NGO in India called Goonj. We sponsor and one of the initiatives which focuses on welfare. The initiative is called Not Just A Piece Of Cloth and it focuses on increasing the importance in awareness around menstrual hygiene. There’s a taboo around talk about it, so we’re trying to break that taboo. Also raise funds that can go into providing for safe supplies for women and underprivileged sections.

Rishu Misri: More recently, I’ve also been volunteering for the Cadence scholarship program. Here we interact with military students from underprivileged societies. These are kids who are very bright, very enthusiastic, clear about their vision. A lot of them want to get into STEM careers, and the Cadence scholarship helps fund their academic goals. As mentors, we try to give them support with confidence building, time management, communication skills, and sometimes just act as sounding boards because the kind of issues they face with their academic sites, they may not have anybody at home to give them the ear. We sort of just support them there.

Rishu Misri: Those are all the kind of things. Sometimes also go and volunteer outside at my personal level. That’s really all the kind of things that I’m doing. Talking about why it’s important to help empower somebody, every time I come back from these events or an interaction like this, I may want to say that I have empowered somebody, but I think what I hear is I am empowered. It brings a lot more energy back into me when I come back from an event like this. It is not just the beneficiaries’ win. It is my win as well. It strengthens me a lot. That’s why it’s important.

Jeannette Guinn: Awesome. Dimitra, what about you?

Dimitra Papazoglou: For me, some years ago I’ve been asked and I’ve been honored actually to build and lead an internal women community at Cadence. I had the great chance to travel and meet in person more than 50 women from Cadence in Europe and Middle East. I had a great chance to talk to them and listen to their stories, understand their needs, and also the challenges that they face working in this environment, in this industry.

Dimitra Papazoglou: We as community team, we wanted to listen first to women and then set the objectives and find the best ways to empower them. What we have done is a set of actions, events. I’m going to mention some of them that I think that they can be also beneficial to everyone here, for the audience. Very beneficial is the talks given by women. The woman can be from outside or inside the Cadence organization. It can be from any level, from senior level or from an early career woman.

Dimitra Papazoglou: I do believe that everyone… You can always learn from a woman, no matter the level that she is. I can tell you an example. Karna, she’s also part of the panel. She actually gave an inspiring talk to the women of our community. She talked about her story, her career, the obstacles that she faced and how she overcame these obstacles. As you see that listening to this woman, you actually get the strength and the confidence on how to navigate and achieve your career and achieve your goals.

Dimitra Papazoglou: Another thing is what we do very interesting is regular meetings where we talk about topics like leadership, work-life balance. We talk about the talents that those topics have, and we try to find solutions together. Again, we talk to each other and try to help each other through these regular meetings. Another important thing is the trainings. We have done career trainings, but also body language trainings. I totally recommend this one. It’s one of the best trainings that I have ever done.

Dimitra Papazoglou: It is all about position, the right position to stand in, how to do the best use of your voice. I think many, many people have these issues like how should I talk? How should I present? I totally recommend these kind of trainings. They definitely can help you to strengthen your confidence. Why I think the women community is very important? Because through the networking that offers you and also the set of actions and events that I mentioned some of them, you can find through a community the mentors. You can find the role models. You can find the sponsors.

Dimitra Papazoglou: You can find all the answers about how to navigate your career and how to go to the next level. It can certainly contribute on how to achieve your career goals. I think it’s one of the best way for all the women.

Jeannette Guinn: Awesome. I just have to say a side note, the fact that you’re able to complete sentences at 2:00 AM in the morning is just impressive within itself.

Dimitra Papazoglou: And having a 12 month daughter, right?

Jeannette Guinn: Huge praises to you and onto Karna, your thoughts.

Karna Nisewaner: I feel like one of the things that I get the most joy from and that really helps benefit the community is the mentoring that I do for people, both internal to Cadence and external. You don’t have to be in the same subject matter as someone to help be that person that bounces ideas off of. As Elena mentioned earlier, it’s important to go to your manager with a plan or ideas to be that person that helps people come up with those plans or ideas and helps them review things ahead of time.

Karna Nisewaner: I feel like the internal mentoring I do within Cadence, particularly during the pandemic… I think it’s been important to help people as they’re just dealing with a lot of different issues and to be that sounding board. I feel like the more I progress in my career, the more important it is for me to reach out and be there for people.

Karna Nisewaner: Now, in the past, one of the things I loved doing was traveling. I think three or four years ago for International Women’s Day, I did a talk at one of our India sites. I went to all of our India sites and did talks to the women’s groups there. I loved being able to reach out to Dimitra’s group and do a talk right before she left on maternity leave. I thought that was great.

Karna Nisewaner: For me, it’s that ability to reach out and connect with people internally and externally and help be that sounding board that helps them move forward. To me, that’s how you, as an individual, can help others. You don’t have to be more senior. You don’t have to be in the same area, but you can be that really good sounding board and person who can walk through the ideas with somebody or can brainstorm things to think about. In the greater community, one of the things that I’m passionate about is making sure that women are able to work.

Karna Nisewaner: One of the things that really makes it difficult is effective childcare and during the course of the pandemic was also having school, which is a place where many of us have our kids and that allows us to have time at home to work. I’m on the board of a childcare organization in my community that runs the afterschool program and several infant and preschool programs because if you don’t have a place for your children to go, the people that tend to stay home are the moms, not the dads. I just think it’s important that we don’t cut people out of the workforce because they don’t have the support necessary to be able to go into work.

Karna Nisewaner: Then I think it’s also important to support your local school. I’m on a school psych council and help planning to create those environments where achievement gaps are addressed in kindergarten, where you’re looking at why is one group behind in reading, behind in math and behind in writing. What can we do starting in kindergarten, first grade, second grade to really stop the achievement gap there, build the confidence of everyone there, so that by the time they hit middle school and high school, everyone’s excited to learn? Everyone has that same background and the necessary ground level education in order to be successful. That’s another place where I spend some of my time.

Jeannette Guinn: Awesome. Then I guess I want to take it to… For all of be, what advice do you have for other women based on some of your experiences, your influences? I know that a couple people mentioned the importance, and Elena talked about it too, importance of having a mentor. I agree. Being a mentor and having one, the benefits of that just are endless. Dimitra, you talked about being influenced by Karna. I can say that the same has happened for me, so thank you, Karna, for everything that you’ve done for me. Just working on confidence, how to present in front of executives, how to become politically savvy, all of that is so important to growth. Dimitra, how would you like to expand on that?

Dimitra Papazoglou: Okay. I’ll share advice not really coming from my experience, but again, from women that talk about their stories, their experience through the women community. I’ll tell you three stories and what I have got from them. The first story was about the new role. There was a new role in her team. However, this role was in a different location, very far away from her location. Her manager never thought of her as a candidate because of the location, but then what she managed to do is to persuade that she’s the best for her role. No matter of the location, she actually managed to take the role. They found, together with her manager, a solution about the location issue and she actually got the role.

Dimitra Papazoglou: The advice that I got from that is that don’t wait to be given the opportunity, just believe in yourself and go and just take the opportunity. The second story is mostly advice. I’ll talk about my experience. I thought in the beginning that in order to go to the next level and get promoted, my manager actually will see that I’m doing awesome things and she or he will offer me the role, the promotion.

Dimitra Papazoglou: But then what I got through advice actually from another women was that when you want the role, just go to your manager, make it clear about what you want. Ask what you need to do in order to get the next role and just make sure that you take all the bullets and then just go to your manager and say, “I do all of this, so I can get the role.”

Dimitra Papazoglou: On top of that, she actually told me that even when you take the role, when you take the promotion, even then, go to next day and ask what you need to do for the next promotion. That’s also good advice. The third story that I want to share is about a pay rise. She wanted to get a pay rise in the beginning. She couldn’t really get it. She thought that she should give up, but then one thing that you said about mentorship, she had a great mentor.

Dimitra Papazoglou: In Cadence, we have great mentorship programs. The mentor was very, very supportive. Also through the community and, again, listening to other stories about similar topics and negotiations, she actually decided to keep trying. She got the confidence and then in the end, she got the pay rise. I will say just keep trying and never, never, never give up. That’s all.

Jeannette Guinn: Thank you.

Dimitra Papazoglou: I want to say that this advice… I’m sharing this advice because this advice has also influenced me and also has affected how I navigate my career.

Jeannette Guinn: Yeah. Yeah. Karna, what about you, influences, experiences?

Karna Nisewaner: I think one of the most important things is really just your own internal confidence and knowing that you are the best, knowing that you are capable of doing things and knowing that even if you don’t check all those boxes, you can check all those boxes if you’re just given an opportunity to try. I think back to several of the jobs that I got, where people were like, “Oh, you only got that job because you’re a woman.” I was like, “No, I got it because I’m better than you. I have more potential than you. I’m smarter than you.

Karna Nisewaner: I think feeling that and knowing that… Yeah, we’re all absolutely capable and you just need to internalize how capable and confident you should be because you can do it. You can absolutely do it. One of the pieces of advice I give to people is really just know your worth, know how valuable you are, know how much you can really do and do that.

Karna Nisewaner: I happen to have been raised in a family by a father that just made me feel super confident. I think that’s the best thing everybody can do is work on that however it makes sense to work on it. The other thing I to talk about is really work on building relationships with others. It doesn’t have to be anyone specific, but building the relationships across an organization will really help you grow your career because you’ll hear about things that are going on that you might not otherwise hear about. You’ll be able to make connections and help other people. Then in the future, they’ll know, “Oh hey, maybe I should help Karna.”

Karna Nisewaner: The other thing I would say is ask for things that you want. I wanted different experiences. I was focused in one area and I was like, “I want more. I want something else.” I said, “Hey… to my manager… “I want something more to do.” Then they gave me something more to do, and I did a good job with it, so then they gave me even more to do. I feel like you have to ask for those things because people don’t know what you want until you tell them. They can’t read your mind. They might say no, or it might not be the right time, but at least they’ll have that in their head and you’re no worse off by sharing what you want than you would be. You’re worse off not sharing really.

Karna Nisewaner:I just feel like raising your hand to say what you want, getting yourself out there… Being competent in your capability and ability to do any job that’s out there if just given the time and support to do it is really, to me, what I think is important that everybody kind of take away from this. Then as leaders and as members of the community, how can we help other people do that? How can we be the person that listens to what somebody’s saying in this, “Okay, this is what you can do. Let’s role play. Let’s make it happen.” I feel like that’s how we can really empower others is be that amplifier of other people’s voices. When somebody does something great, remind people, but then also shout out for yourself because you’re valuable.

Jeannette Guinn: I don’t know about the rest of you, but I’m pumped. I’m like, “I’m going to take over the world right now, Karna, that was awesome. Thank you so much and, Rishu, your thoughts.

Rishu Misri: I think pretty much whatever everybody else has already said, but my two cents will be just we need to make our tribe grow. For that, whatever it takes. Depending on where we are in our life and in our career path… If you’re in entry level, you will probably have to be focusing more on building your skills, trying to build the right networks. We’ve talked about mentorship and having that confidence. Like we say, that’s the most important thing, having the belief in yourself that you can do it, being resilient.

Rishu Misri: As you grow and are in a position to even be able to support others, then be compassionate towards the other women. Being a woman and being in a workforce, it’s not going to be easy. There are going to be times when it’s going to be tougher for you than it is going to be for your male counterparts. I mean, no offense there. I know everybody’s competent, but we’re going to be taking so many additional roles and nobody can take it apart from us.

Rishu Misri: I think it’s important that as a community, we stay more connected and we stay more compassionate towards each other and support each other in whatever positions we can and I think we also need to get more focused to bringing those women back who had to apply brakes to their careers. Be compassionate towards them. If there have been a lot of women who’ve applied brakes because they wanted to take care of children or they had had elder care to take care of or whatever other personal requirements…

Rishu Misri: If anybody had a career aspiration, a dream and we can help motivate those people back into the system, the workforce, I think that’s important. Just as everybody said, having belief in yourselves and just continuing to take the risks, I think that’s very important. Being able to try out new things and having the confidence that it’s… Tough times will be there, but I’m going to overcome them with my training, with my mentor support or whatever.

Jeannette Guinn: Yep, absolutely. Thank you Rishu, and as we wrap up this panel, last words of wisdom to women that are in the tech space that are working towards advancing their career… I’ll kick it off because it’s kind of wrapping up some of the things that you’ve all said. I say this to myself, to my team, to my family members. Don’t allow a struggle or a hardship to bring you down. It’s an opportunity or use it as an opportunity to grow stronger.

Jeannette Guinn: I could have a whole other session on my history, but I was financially on my own starting at the age of 17, and suffered years of abuse until I was about 23 years old. It sucked and you take each and every moment as learning opportunities and you make the best out of those crappy situations. Anything that I had to deal with in my 20s, as I was trying to advance my career, there were little nuggets of learning lessons.

Jeannette Guinn: If you want something, you go after it. Take that chance. There are going to be risks involved. There are going to be failures and that’s okay. You just don’t look back. You just keep looking forward. There’s a phrase that I use a lot. I say it a lot, but I was in a 12-month program with Women Unlimited, fabulous program. They taught me that you strive for excellence, not perfection because perfection’s just not possible. every day I just do my best and you strive for excellence. that’s my last words of wisdom. Rishu, any last words of wisdom from you.

Rishu Misri: I think I just continue build on what I said in my previous… I think it’s important that we continue to be resilient. That’s what is important. Just stay there, hang in, and if needed, seek support. There will be a lot of we people willing to help you. A lot of times, we may feel, “Am I doing the right thing being here? Is this where I should be? Maybe I should quit. Maybe this is not for me. Maybe… There’s so many questions that be come in to our mind. It’s not just for you. It’s for everybody.

Rishu Misri: Seek support. If you need to apply the brakes, do that. I’ve done that as well. When I had my daughter, I applied the brakes. Then when I had my son, I sought support. That’s ways I was able to continue doing what I wanted to do. I think that’s the other most important piece of advice that I have. That is whatever you choose to do in that moment. Do not be guilty about your choices.

Jeannette Guinn: Yes, yes, absolutely.

Rishu Misri: It was your decision. Don’t be guilty for whatever the choice you made. That’s important. Be resilient, seek support, don’t be guilty. That’s important. I think that’s all that I would say. Thank you.

Jeannette Guinn: Great. Thank you. Dimitra?

Dimitra Papazoglou: Yeah. For me, I’d like to actually say three things. For me, always have a career plan for the next two to five years and make it clear to your manager. Second thing, find the ways to strengthen your confidence. It can be this conference, it can be this panel. Find the Karna that will help you to have the confidence and say, “Okay, I’ll go for it. Karna said that. I’ll get all this confidence and I’ll go for it and I’ll take it.” The third is seek for opportunities. Don’t wait for them, okay? Don’t wait for others to give you the opportunities. You need to seek for them.

Jeannette Guinn: Thank you so much, Dimitra. And Karna?

Karna Nisewaner: I’ll build on what Dimitra said. It’s not just seeking opportunities. It’s being okay with change, being okay with saying, “This isn’t working out for me. I need to find a different environment, a different set of colleagues,” and having that community, having the people to support you.

Karna Nisewaner: I feel like you need to also be open to new things and maybe it’s a change in your role at a company. Maybe it’s a change of companies, but being flexible with yourself and not feeling like you’re stuck or stagnated into one thing, but that you can really do anything because I do believe that there are so many possible options for everyone. We just need to try and we just need to experience them. Sometimes things will be great. Sometimes they won’t be great. What can you change to make it better? Because you control your environment.

Karna Nisewaner: Yes, there are certain things we need. We need our paychecks, but you do control a lot of your environment and you need to create and find that environment that’s supportive, that’s there for you and that wants you to be successful. I feel like that’s what I found at Cadence is an environment where managers, colleagues, other people I worked with, they wanted me to be successful and they wanted to help me find that next thing.

Karna Nisewaner: You don’t find that in all jobs. If you’re not finding that, find people that will help you. Find a new role. Find others that will really amplify the value that you’re adding and really appreciate the way in which you add that value. I feel like we control our future, but we need to be out there saying what we want, sharing what we can do for others.

Karna Nisewaner: We can all have great careers. I just love how many more women are engaged and how many more of the underrepresented minorities are engaged in the community here at Cadence, are engaged in the Bay Area and are engaged worldwide. It’s great to see that growth. I just really hope it continues and that we continue to really show everyone that we are amazing. We are the best. We’ll rule the world, right?

Jeannette Guinn: Yeah, absolutely. Absolutely. I love it, Karna. Thank you so much, Karna, Dimitra, Rishu. It’s been a pleasure. On behalf of Cadence, thank you all. I hope this was helpful. Angie and Girl Geek, thank you for this opportunity. It was a wonderful experience. With that, go onto networking. Thank you so much.

Karna Nisewaner: Thank you.

Rishu Misri: Thank you.

Dimitra Papazoglou: Thank you.

Angie Chang: Thank you for being a part of that panel. I feel very empowered and ready to dig in. Now, I want to just really quickly plug that Cadence is hiring. They’re hiring for engineering jobs in cities like San Jose, California, Cary, North Carolina, and Austin, Texas. Now, we’re going to move onto our Girl Geek X networking hour. There’s a link that will be put into the chat. If you click on that, it’ll go to Zoom meeting, and we’ll see in a Zoom breakout room very soon.

cadence girl geek x speakers zooms
Cadence Girl Geek Dinner on March 16, 2022.

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Posted on

“Economic Justice and Cryptocurrencies / Web3”: Jen-Mei Wu, Community Organizer at PaRTEE4Justice (Video + Transcript)

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Sukrutha Bhadouria: I hope you’ve been having a good session so far. Good time so far in the conference and we are ready for the next session. Thank you all for waiting for us. Rashmi is here to give us our next talk. Rashmi is a Manufacturing Test Engineering Manager working at Robotics and Digital Solutions at Johnson and Johnson. In her career over the last 13 years, she has worked on consumer products, trained signaling, and more recently robotic applications and medical devices. She’s passionate about making an impact on our society with technology and helping fellow women in tech in their journey. Welcome Rashmi.

Rashmy Parimi: Thank you for the kind introduction. Hi everyone. I’m Rashmi. I am part of the robotics group in Johnson and Johnson currently working on the manufacturing side of one of our new robotics products soon to be released to the market and through the stock a Dr. Robot. Now see you. I’d like to transport you to this future vision where this will be a more accessible reality for a lot of people. <Laugh>.

Rashmy Parimi: I want to go back a little in the history before I transport you to where we are today and what the future looks like. A lot of you must have seen this picture on the left of an early operating room where surgery was more of a spectator show. Antiseptics and anesthetics were not something of commonplace. There was no concept of sterilization and for a lot of, I would say decades back then, laughing gas was a commonly used anesthetic.

Rashmy Parimi: Even that was not highly recommended because you know, there was mixed feelings either by the patients or the doctors to use it. A dentist came across ether being an effective anesthetic and he compelled the rest of the medical community to conduct a clinical trial to give more substantial data. And that was one of the starting milestones of making anesthesia a regular process of surgery.

Rashmy Parimi: I think the data convince people that one anesthetics are good. They’re not necessarily something that take you out of control. And also convince surgeons that they didn’t have to resort to methods like strapping down the patients to, you know, help them go through the surgery because you know, without an anesthetic the pain will make them move and that’s not something ideal. And they also felt that having a PA stable patient would give them more dexterity and stability to operate.

Rashmy Parimi: That was a very fast history of surgery back then. But from then to now, like there’s so much, you know, medicine has gone grown from deeps and bounds increasing human lifespan by at least 30 years. And even today, I think the whole fascination with watching surgery has not gone away, but it’s a little more, I’d say refined from how it was in the photo depicted on the right towards, sorry, on the left to where it is on the right where there is more advanced rendering of the surgical procedure, either during to help other specialists participate in it or to a surgeon or a medical team in a far away location to help add more perspective to a complicated situation.

Rashmy Parimi: From a very low out like low outcome pain causing and a long recovery method to introduction of laparoscopy and endo, which has improved patient outcomes and reduced the recovery time and also improved the accessibility to a lot of people for complicated procedures. So this is where I think with this is what most people are familiar with and laparoscopic was what sewed the seeds for the first ever use of robotic surgery.

Rashmy Parimi: This particular arm is maybe familiar to a lot of people as something used in, you know, large industrial assembly houses for large scale manufacturing, more like you know, car assembly facilities or other large equipment facilities. But you’ll be surprised to learn was this was one of the first experimentations of whether robotic surgery can be used or not. And you will be even more surprised to learn that the area in which this was used was brain surgery. <Laugh>. This was used to guide a percutaneous needle to do brain biopsy back about more than 25 years ago. And then this concept was further expanded to a colostomy and TransU urethral resection to further peak people’s re and research group’s interest to develop the concept of robotic surgery even more and work towards bringing it from a lab prototype to more of a reality. In 2000. one of the pioneer companies of robotic surgery, Intuitive Surgical, they broke the ground finally when their system, the first ever Da Vinci system got FDA approval for general laparoscopic surgery.

Rashmy Parimi: It was this innovative device with lo robotic arms with visual systems and also they had help from nonprofit scientific research organization, SRI, to help them advance a lot of these initial prototypes. And that’s was how most people today, if they are familiar with robotic surgery, I think this is the one name they recognize instantly.

Rashmy Parimi: Let’s talk about what are the advantages of robotic surgery that makes it so attractive to use when, you know, everyone would admit that laproscopy already takes us through a good bit of path onto, you know, smaller incisions and all of that. So we still get the same advantage as oscopy that is a smaller incision, which means quicker healing, lesser hospitalized time, which I’m sure all of you will, you know, relate to the expensive insurance bills and not having to deal with that. And also it is co like the cost saving and also the body will recover faster through a smaller incision since the amount of trauma is less. The other advantage is the precision the instruments can reach into hard to reach places of the body without having a wide incision with accurate precision and stability, which is a lot of, which makes a big difference in terms of your outcome of the surgery. And also with this precision al the comes with it, it adds an extra, I’d say boost to the surgeon’s abilities and gives them the confidence to tackle some really tricky procedures.

Rashmy Parimi: One of the important things of having a successful surgical outcome is good visualization. When you know you cut a part of the body, there is obviously going to be blood involved and in typical surgery it could a lot of times block the view of what is going on there, but with the time your incision smaller cuts, that disadvantage can be overcome and it leads to a better outcome. And also there’s a good example that I would like to use for what, how pressure virtualization you know, improves the surgery. So having robotic vision is like if you want open surgery is like using a flashlight to look through a window into your house while robotic surgery is like opening the door, turning on the lights, and then trying to look at your house. You can see it’s evident, which is a better way to look at your house.

Rashmy Parimi: And that advantage is offered to by the advanced imaging that comes with robotics surgery and with, in addition to all of these, the other advantage is exceptional dexterity. So everyone is, you know, familiar with how surgeons have these long schedules and if things do not go as planned, there is a lot of fatigue on them with the long hours and that can lead to that showing up on the surgery itself. But with robotic surgery, one of the things that can be controlled is to remove the tremor and other fatigue related impacts so we can reduce these inadvertent, you know, punctures or nicks which can cause unwanted bleeding into the body. So let’s look at few of the areas where today robotic surgery is used in one way of the other heart surgery where these very precise repairs that are needed is done using robotics stomach, though it looks like a big area, there is a lot of fine precise procedures that can be done in a better fashion using robotics.

Rashmy Parimi: General surgery of course, is another area where with a smaller incision and the precision offered, you can do a lot more compared to non robotic surgery. And same goes with the area of GY gynecological surgery where there is, you know, access issues and you want to make sure you don’t impact the healthy tissue or healthy organ parts. Same thing goes to lungs where the access is extremely difficult and with kidneys where the, the areas so delicate important that you want to make sure you do not cause unwanted damage to the existing parts. In the area of orthopedic surgery, robotics have given an added advantage of very precise cuts and placement for implants and you know, it’s popularly used I think in hip replacement and knee replacements, which has become very common place in the society today. In the area of dental surgery, there is a product in the market today which help with dental implants and there’s, I’m sure there’s a lot more research going on.

Rashmy Parimi: And as I explained in my first example brain surgery, it started off <laugh>. The whole idea for this was sewn with brain surgery and it is still an area of widely researched today and they are trying to develop products in that area. So here I have some examples of some popular players in the market today. So roughly going over that, the first one is Johnson and Johnson’s robot monarch, which is, which has f d a approval in the lung cancer and kidney stone management space. Below that you have Medtronic’s robot Hugo, which has approvals in the general surgery space. And the picture below is Intuitives DaVinci. It’s a newer generation of it, which also has approvals in general surgery and a lot more areas on the right hand side. The first one is the Yumi robot, which is used in the dental surgery field. Their application right now is in the area of implants. The one below from Striker is the maker robot used for I think the orthopedic area. I, I don’t want to guess the wrong thing, but I think in the, a place of hip replacement probably. And the one below is from Siemens and this is a robot used in the cardiovascular area.

Rashmy Parimi: Now that I’ve peaked your interest on how, what, what are the advantages that come with this novel application? I’m sure all of you must be curious how do you break it into this field? What are your pathways? Is it something very niche? Do you have to, you know, is it very a small circle, small exclusive circle? Well, I’d like to walk you through my own career path to kind of show you it’s really not all that difficult.

Rashmy Parimi: In the next slide I will also kind of walk you through during the various stages in the life cycle of a product development, what are the different functions that interact and how, you know, different disciplines come together to successfully build a robotic surgical product.

Screenshot at .. AM

Rashmy Parimi: I started off by education as an electrical engineer, but using that as my foundation, I have worked on firmware for different products, electricity meters, crane systems, small devices which include wearables, thermostats.

Rashmy Parimi: If you see here my, I went into this was not through either medicine or robotics. I started from a very normal field, which I’m sure most of you feel <laugh> a little easy to relate to. I did have a small ex in brush with medical devices early in my career where I was working as a part of a team on a prototype of a U USB based E ECG monitored. If any of you have noticed the E ec G monitor today used in the hospitals is, it’s a big piece of equipment and it’s not portable. If it, you know, there is a, it’s used in a remote location and they want to share the data around for more opinions. It’s not easily done. There is that accessibility issue. But if it were in a USB form and the data can be collected wirelessly and shared across seamlessly without the boundary of a physical location, it it would be a B great blessing to bringing healthcare to rural areas where accessibility is a big issue.

Rashmy Parimi: The proposition of that product was very interesting. And back then I was, you know, I wanted to continue in that but then again it was just one research project. But in, as I grew in my career, one of the chances I encountered was to be part of the startup verb surgical, which was working on a soft tissue surgical platform. Today surgical has been acquired by Johnson and Johnson and that team is continuing the work on that platform. Hopefully soon that will be in the market helping people improve their quality of lives. And even if you notice through my career, the job duties I’ve done has varied from pure research projects to some integration to what I do today, which is manufacturing test. So, and all of this is more about applying your skills, existing skills across different areas. I have not taken any new courses.

Rashmy Parimi: I have always maintained this curiosity to upskill myself on the job and try to, you know, read more on things I don’t much, that was how I was able to work through different domains within the same company. So next I want to talk about what are the various disciplines and roles that participate together during the development of a product. So initially, you know, when you have, when you want to establish the user needs and make sure a certain product is feasible from a regulatory perspective, the team that typically ha does the groundwork the product managers who talk to the customers such as the physicians to make sure they understand what will help them. Then you have the systems engineers who translate those customer needs into some kind of actionable product requirements. And then the clinical engineers who also bridge the gap from a clinical perspective.

Rashmy Parimi: The regulatory affairs team helps trying to understand what, how the impact of that, you know, what is the burden of this product to make sure we are safe. And also how, how do we prove that this product is safe to use on human beings once the use case has been established And there is this clear requirements for the product. Then comes a design phase where you have design engineers and various arenas. You have electrical design engineers, mechanical design engineers, hu UI engineers, UX engineers, all coming together to build different pieces of the system and of course test engineers to test all that has been built. And for most large scale products, one of the things that has been the, you know, big made a big difference if the product moves forward in a given timeline or it does not launch off is the integration piece of it.

Rashmy Parimi: There is a lot of complex software and hardware coming together and integration plays a big role. We have the systems integration engineers trying to piece those puzzles, making sure two independent modules operate together as one big unit and also clinical engineers vein from time to time to make sure what physically was decided in the beginning is still what the goal of it is towards the end. As the product goes into its future stages, the burden is to val validate and verify it so that we have the essential documentation for FDA approval. But before that, the manufacturing team and the supplier make sure they work with various vendors and internally and to build up these units that will provide the data for FDA to review and approve the device. And once that is done during the commercialization phase, you have marketing team, the sales team, the service team to make sure the product is supported within the customers who are using it and also provide the feedback to support the next level of iteration of design and all of these resulting in a complete cycle.

Rashmy Parimi: As you can see, quality is something which is critically important through the whole process and weigh in in all of the design phases and the later validation and commercialization phases.

What is the future outlook for this field? This is illustration from before the pandemic and you can see just few years ago there’s been 77 companies and these are only the companies that are have gone public. There are a lot more stealth companies who maybe close to finishing their product. So the number of companies have increased from a few million in the beginning of last decade to a lot more billions now. So it’s a fast growing industry and there has been a lot of acceptance to make sure this field is supported. And in general you’ll see these are the two areas where there has been a lot more progress in terms of adding new procedures and support in terms of surgeon’s interest and also success rates in the field.

Sukrutha Bhadouria: Rashmy, we can wrap up. It’ll be great.

Rashmy Parimi: Yeah, so I think this is my last slide, <laugh>. So with this I hope a lot of people I know, I’m sure you have a lot of questions. I’m happy to answer that later. I please feel free to connect with me on LinkedIn. Thank you everyone for your time and thanks for having me here, <laugh>.

Sukrutha Bhadouria: Thank you so much Rashmy and thank you to everyone for attending and you know, posting all your comments and sharing your insights. Thank you.

Rashmy Parimi: Thank you.

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Posted on

“Why Knowledge is the Future of Data”: Michelle Yi, Senior Director of Applied Artificial Intelligence at RelationalAI (Video + Transcript)

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Angie Chang: So our next session is Michelle Yi from RelationalAI. She is a Senior Director of Applied AI, and she’ll be speaking about harnessing knowledge and data and show us relational knowledge graphs in action.

Michelle Yi: Great. Thank you so much, Angie for the introduction. I’ll just go ahead and screen share, make sure everything is going smoothly. And let me make this big. Here we go. Okay. I think we are good to go. Okay. Yes. So happy International Women’s Day, everyone. And I, plus one, I saw in the chat, a comment about Reggie for president. So plus one to that.

Michelle Yi: So my name is Michelle Yi as Angie said, I’m super excited to share a little bit of perspective on why I believe knowledge is the future of data and how my personal experiences in the data space also align to this common vision that brought me to RelationalAI.

Michelle Yi: And so I thought I could start with sharing a little bit of context and background on myself and the journey that has brought me to RelationalAI. Our vision for what we’re doing really spoke to a lot of the challenges and problems that I saw in the machine learning and data space.

Michelle Yi: And actually to make this a little more fun and interactive, if you guys want to share a little bit about your own journey technology. I’d be curious to see what did you all study, whether it’s undergrad, PhD, masters.

Michelle Yi: What was your last educational focus? Something Heather said in the last talk actually really ties to the demo I have at the end of my talk, which is going to show a little bit of the backgrounds of the women that signed up for this conference.

Michelle Yi: So for me personally, I spent the last 16 years or so in the AI/ML space working with data from both our products R and D side, as well as a consulting perspective. So specifically, I don’t know if anyone will remember this, but in 2012, one of my first projects that I worked on actually aired as IBM Watson and this whole thing with Jeopardy where the computer was playing against the humans.

Michelle Yi: So that’s my one claim to fame. And then after that, I moved more into management consulting because I really wanted to understand the data and data science problems that many customers across many verticals were facing. And so through these experiences over the last couple of decades, I really got a lot of exposure to the impacts of the constantly shifting technical paradigms and how that impacted business.

Michelle Yi: So to give you an example, when I started at IBM ML 16 years ago, was on mainframe. This was before the Cloud. If you can even imagine an era before the Cloud. And then we were after we started getting migrated and pushed to go more Cloud oriented, moving away from on-prem, there was a big, no pun intended, big data movement.

Michelle Yi: Essentially saying, like, “Go collect all the things.” And we didn’t… We collected a lot of data without really always thinking about why did we need that data? And then we were sort of pushed to like, “Okay, well, if you want to use this big data thing and you want to make all those things that you collected useful, you need to go to MapR and Hadoop.”

Michelle Yi: And then what ultimately resulted was this data swamp architecture where we had data everywhere in different silos of many different types. And then that shifted into what’s now more of the modern Cloud data warehousing. So think about BigQuery, Snowflake, Redshift et cetera. And then after we consolidated all of these things, we’re like, “Oh, okay. We finally got it figured out.”

Michelle Yi: But then you kind of see another kind of paradigm around machine learning and for people to take advantage of that you need yet another patchwork tool chain. And we’re going to dig into this a bit more, but the question is why is it that every time we see a paradigm shift, or a new technology, or a new data structure that we kind of go through the same motions over and over again.

Michelle Yi: And so just to speak to a little bit of those problems, I don’t think this is going to be new to anyone in the data space, but basically with each iteration that we’ve gone through, we still see the same needs from the business and the technology side.

Michelle Yi: There’s this desire for kind of more data driven decision making across the board from your executive teams all the way down to the engineering teams. And then there’s this other problem of like, “All right, we went through big data and we collected all the things, but now we don’t really understand everything that we’ve collected.”

Michelle Yi: So we even today, I think many of us would agree that there’s really kind of a lack of understanding of the full extent of the data assets that an enterprise or even a startup has.

Michelle Yi: And then as a result of that, there’s this third bucket of problems where we’ve really seen a rise of just too many point solutions or too many point data applications that sometimes can be repetitive of each other.

Michelle Yi: I don’t know how many times I’ve [inaudible] this and seen to a customer and we’re like, “Hey, you’re interested in a fraud detection, no problem. Oh, by the way, they also built their own fraud detection solution over there in teams D or E.” And so we’re kind of seeing like this common theme across companies and across a long period of time. And again, we need to ask ourselves what’s the root cause of this.

Michelle Yi: And ultimately I think what I saw over and over again is that there’s really something missing from this modern data stack. If we’re really evolving the way that we think about data, why are we seeing the same problems manifest over and over again? And so this is the question I really want us to kind of hone in on and specifically around this concept of knowledge and I’m going to share because you’re like, “All right, knowledge.” That can mean so many different things to basically everybody on this call.

Michelle Yi: And I’d be curious how many data scientists, more on the ML side we have in the room today versus more of the software engineering data app side, I’ve lived in both sides of those worlds. And they’re converging in many ways, right?

Michelle Yi: Because a lot of intelligent data applications today at the core of them, they really are having embedded machine learning whether that’s a machine learning model that you and your teams build or managed service that you receive from a vendor that you buy.

Michelle Yi: And so from my personal experience, I wanted to share an example of a day in the life of a data scientist or a software engineer working on an intelligent application and really hone in on this question using a workflow example of like what happens to the knowledge. And tell me in the chat, let me see.

Michelle Yi: I want to make sure I have it topped up in the screen, but please tell me in the chat if you resonate with this, but one common thing that I think people really have experienced is that we tend to spend like 80% as a data scientist or someone building an intelligent app.

Michelle Yi: We spend like 80% of our time productionalizing things and maybe 20% of our time really modeling, collecting the requirements and the data, et cetera.

Michelle Yi: And if I go into this just like one more level deep and not to get too trapped in the weeds, but just to really hone in on the pain point and why knowledge and embedding knowledge in a workflow is so important is let’s say like all of us are on the same team together.

Michelle Yi: And we want to build this fraud detection application. And at the heart of this application is a machine learning model that gives some predictive score of like, “Yes, that transaction is 50%, 60% likely to be fraudulent.”

Michelle Yi: Well, let’s think about this. So step one, what do we really go do? We let’s say one, we get a sense of our own intuition of what kind of data we need. We probably need something about transactions.

Michelle Yi: And we probably need something about accounts and people related to these transactions and maybe that lives in, I don’t know, BigQuery, let’s say it lives in Teradata, and then it lives in Excel because how many of us store data… Plenty of us store data at Excel. And then let’s also say that we probably need some information from the public web because when people steal things, they need to go sell them and make money.

Michelle Yi: So we get this intuition, we make a list. And then we ultimately, what we end up doing is we go to the business owners or the business experts and saying, “Okay, does it make sense to have this kind of data? What are we missing? Oh, I see, this data has this flag that has a transaction type one. What does that actually mean?”

Michelle Yi: And so we spend a lot of time upfront collecting and gathering data. We work on a subset and that in this 20% bucket of data science work, in that 20% of time, we get a model working that we’re pretty happy with.

Michelle Yi: Let’s say we use Python and a Jupyter Notebook. steps one and two are done. We’re happy. And then we need to scale this up to production. And then what we end up spending 80% of our time on is rewriting everything that we learned in terms of collecting the knowledge from different business stakeholders and our own data science knowledge.

Michelle Yi: And we rewrite that in like Java, Spark and much more heavier imperative programming languages, just so we can productionalize what we already did in steps one and two.

Michelle Yi: So the question is why can we not preserve knowledge across the data, across this entire workflow end to end. And that’s where I really kind of started to think more about this problem, because imagine how many like teams, how much time it would save if I could just preserve all of my learnings that I collected up front from the business about the relationships between transactions and customers, and accounts, and then also like the different constraints.

Michelle Yi: So for example, if I am looking for pictures of cats, I know that cats have two ears. I shouldn’t even think or waste any time processing things with four ears or five ears. I mean, this is a toy example, but I think you get the idea. And then 0.3 is really like, “Okay. If I on team A, I’m building this fraud detection app, why can’t I just easily share this knowledge with somebody in team D so that they don’t have to go do the same requirements gathering?” Because you know, that happens in any organization. And so when we talk about knowledge, it’s how do we preserve these relationships and really save ourselves time and.

Michelle Yi: We preserve these relationships and really save ourselves time and make that accessible to more than just one team. So, there is this concept of a knowledge graph and so you’re like, “Okay, well, yeah. I’ve heard about knowledge graphs.”

Michelle Yi: And there’s sort of like this way of structuring and thinking about data that can somewhat solve this issue, but not exactly and let’s… I want to get into that a little bit really quickly.

Michelle Yi: And so, one of the things is that, here is just an example of a knowledge graph concept, right? And the thing about this picture is even if I don’t give you all the details of like, oh, this lives in inquiry [inaudible], this one lives in another database.

Michelle Yi: Conceptually, you can kind of get that a product has a brand and a product has a category where shoes is an example of a category and a company sells products. It doesn’t matter if you’re an engineer or a business person, you can pretty quickly see what this is.

Michelle Yi: And now imagine if you could actually just query your data as easily as you can read this picture. The thing with knowledge graphs though is that they’re actually not necessarily a new concept.

Michelle Yi: So, it was coined by Google when they created the Google knowledge graph. They wrote this paper that came out in 2012, over 10 years ago now, and it’s been a core competitive advantage to them.

Michelle Yi: So if you ever wonder why search is so powerful at Google, this is one of the secret sauces to that. And when you’re shopping on Amazon, if you’re like, “Wow, my recommendations are amazing.”

Michelle Yi: That’s also another reason why they’re so powerful, is that they’re using this thing called knowledge graphs. And so a lot of other companies have really adopted this thing called the knowledge graph. And you’re like, okay, you can do all these cool things. You can express your business knowledge in the same place as you would do your programming or your data querying, why isn’t everyone else adopting this?

Michelle Yi: Well, the problem is that, and there’s many, many problems, but there’s kind of like three that all high level boil it down to. But one of them is that yes, knowledge graph expertise is kind of rare and not everyone is Google or Facebook or LinkedIn, and they can’t hire hundreds of engineers to go build these things for them, right? There’s not enough people out there to do this.

Michelle Yi: And the second thing is that building and scaling knowledge graphs is really difficult because a lot of the existing solutions are built on really old paradigms. So like the Google knowledge graph paper came out 10 years ago, a lot of the commercially available systems today make it hard to use.

Michelle Yi: Some of these systems are based on theories that came out in the seventies in terms of navigational systems, right? And so it’s really, really hard to use any existing thing to build your own knowledge graph if that’s really what you want to do. And so similarly, operating and maintaining them is really challenging as well.

Michelle Yi: So it’s an amazing concept that just really hasn’t been more commercially viable and accessible to a broader audience. And so, there’s one thing that I want to quickly over is we’re kind of taking a slightly different take and then I’ll show a really fun example to make this more real and in honor of international women’s day here.

Michelle Yi: But one of the things that we’re trying to do is say, let’s build that next generation thing. What does that really look like if we were to take a knowledge graph and make that supercharged and really available to a broader audience.

Michelle Yi: And one of the things that’s key is you see the word knowledge graphs, and then you see this thing called relational and RelationalAI. So I’ll share a bit more before jumping into the demo quickly and then wrapping up.

Michelle Yi: But essentially when it comes down to what we’re trying to do is build this next generation database platform that really gives you that infrastructure layer that’s going to help you consolidate and keep knowledge in the end to end workflow based on a solid shared foundation of a relational knowledge graph.

Michelle Yi: So one of the things that being a relational knowledge graph does is, and this is a bit of an eye chart, but I’ll summarize it in one point, which is that the relational paradigm, when you think about why SQL databases, for example, or you think about why snowflake or BigQuery or Redshift are so popular today is because it separates a lot of the what from the how. So you don’t worry about this huge list of super technical things in the middle, right?

Michelle Yi: A lot of that is actually handled for you. And so that’s something that’s really, really cool about a relational knowledge graph versus other systems. Because again, we share those same technical foundations of what you really expect from that modern data stack and including things like warehousing, et cetera. And so when you think about your favorite SQL system or your favorite database system, I guarantee a large part of that adoption is because your business users, not just your engineering teams, can use it.

Michelle Yi: And so in the future what we’d love to see is like, because we share these same fundamental architectural paradigms, we’d love to see that layer of knowledge that sits across and really pairs with and augments the work that many organizations have already done to consolidate and clean up their warehouse. Basically all the work that everyone’s done going from Hudu to cloud data warehousing, et cetera. This is the thing that we want to say is missing from that modern data stack and that we want to augment and really bring out the power of these things across your organization.

Michelle Yi: All right. So with that said, I’m going to take a look at the chat here and just see at some of the backgrounds. Okay. I love it. Business management, psych. All right. So, in the last three minutes or so I want to wrap up again with just like a simple example where we took some data, thank you to Girl Geek X for providing some of this as well.

Michelle Yi: But basically we took some data on the types of folks we knew would be presenting and attending the conference today. And then we also took some information that’s already… So, for those of you that don’t know about DIFA, we took some information from them. They actually structure all of the information on the public internet in a knowledge graph. And so it’s super easy for us to be able to leverage that in our system. And we took a high level view of kind of the women participating.

Michelle Yi: And basically what you’re seeing here is we put a visualization of what’s called the weakly-connected components graph, right? And so it’s a type of graph algorithm where what you can see quickly is like there’s certain densities and there’s certain areas that are less connected on the edge here.

Michelle Yi: And so we took a survey of sort of what did people study, right? And for women that are in engineering or technology, what did they study as the most recent education? And so what I thought was really fun about this is that when you zoom in, you can kind of see the clusters you might expect.

Michelle Yi: This is a New York if I remember right. And then in New York, there’s lots of people with computer science degrees, et cetera, et cetera. But when you get to the edges a little bit further out, you see a lot of really, really cool majors and folks of women that are in our fields and that have really, really diverse backgrounds.

Michelle Yi: And I love seeing this. So you see like economics, I saw English, English literature. I saw health informatics right here, design and art direction. And so I thought this was like a really fun way using knowledge graphs to quickly show that it doesn’t matter what background you have, but there is a place for you in tech.

Michelle Yi: And the thing is that when you are kind of one of these weakly-connected components, you might sometimes feel like you’re the only one. Right? But actually it’s not true. There’s so many of us that are out here.

Michelle Yi: And so I thought this was a fun way to show that using some real data. So yeah, I thank you so much for all of your time. I think we’re right at the 45 minute mark. And so, really appreciate it. And if you have any questions or you’re interested in graphs or the tech, please don’t hesitate to reach out. Thanks so much.

Angie Chang: Thank you, Michelle. That was very informative. I love the chart and the graph and for explaining everything so clearly. 

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!