Best of 2019 – Elevate Videos

By Angie Chang (Girl Geek X Founder)

Elevate showcased 22 amazing speakers and 7 mission-aligned sponsors at our virtual conference in celebration of International Women’s Day for the past two years. We received rave reviews for the content and accessibility of the online program, and are looking forward to another in 2020!


Here are the most popular talks from past Elevate virtual conferences based on attendee ratings of the sessions:

#1 – Always Ask For More (video)Leyla Seka (Salesforce Executive Vice President), Jen Taylor (Cloudflare Head of Product Management)

#2 – Being Unapologetically You (video) Sandra Lopez (Intel Sports Vice President)

#3 – From Office Manager to Chief Product Officer (video)Shawna Wolverton (Zendesk Senior Vice President of Product Management)

#4 – Building High Performance Teams (video)Nupur Srivastava (Grand Rounds Vice President of Product Management), Citlalli Solano (Palo Alto Networks Director of Engineering, Colleen Bashar (Guidewire Software Vice President), Gretchen DeKnikker (Girl Geek X Chief Operating Officer)

#5 – CTO’s Lessons Learned from Software Developer to IPO (video) Cathy Polinsky (Stitch Fix Chief Technology Officer)

#6 – It’s Not Them, It’s You: Self-Awareness & Ego (video) Minji Wong (At Her Best Founder)

#7 – Creating An AI For Social Good Program (video) Anna Bethke (Intel Head of AI for Good)

#8 – Engineering Leadership: From Cat Herder to Air Traffic Controller (video) Laura Thomson (Mozilla Director of Engineering, Rija Javed (MarketInvoice Chief Technology Officer), Miriam Aguirre (Skillz Vice President of Engineering, Vidya Setlur (Tableau Software NLP Manager)

#9 – Using Statistics for Security: Threat Detection at Netflix (video) Nicole Grinstead (Netflix Senior Software Security Engineer)

#10 – The Art of the Interview: How Would Your Candidates Rate You? (video) Aline Lerner ( Founder)


We invite the Girl Geek X coommunity from around the world to participate in Elevate to share the latest in tech and leadership with fellow mid-and-senior level professional women.

Sessions may reflect the theme of this year’s conference – “Lift As You Climb” – and content typically covers the following topics:

  • Lightning Tech Talks – Dive deep into an area that’s unique/critical to your business or role (i.e. machine learning, security, usability, UX/UI, ethics in building product, data analysis, etc.)
  • Technical Skills & Tactics – Tutorials, walkthroughs, or deep dives into a skillset or tactical approach to how you solved a real-world challenge.
  • Learning and Development – Topics include negotiation, job search, interviewing tips, being a better leader, self-awareness, career growth, management, etc.
  • Inclusion, Equality, and Allyship – Topics include being a better ally, lifting other women up, and actionable advice for individual contributors or managers.
  • Interesting Life/Career Journeys/Distance-Traveled Stories – Did you overcome socioeconomic challenges (i.e. first in family to go to college, raised in poverty/rural area/etc.) while giving back or contributing to the greater good?
  • Work on a unique technical project or have interesting insights you’d love to share with other other women & allies? We want to hear from you!

Tip: The best proposals include 3-5 key takeaways — what attendees will learn from your talk!

Submit your proposal for a talk and/or panel here by December 24, 2019 11:59PM PDT for Girl Geek Elevate virtual conference.

For conference sponsorship inquiries, please contact


We would love to have more Girl Geek Dinners at med/health companies, biotech companies, consumer-facing companies… We are interested in partner more with the scientific and ethical-minded companies out there in addition to our slate of tech companies hosting Girl Geek Dinners.

Here’s how to partner with Girl Geek X in 2020. We are currently working with sponsors for 2020 dinner dates, and excited to continue partnering with companies to host Girl Geek Dinners!

For dinner sponsorship inquiries, please contact


Girl Geek Dinners, Girl Geek Elevate, Girl Geek Podcasts, and much more!

Here are the best 10 Girl Geek Dinner videos of 2019.

And the most-downloaded Girl Geek Podcast episodes in 2019.

We’ll be releasing the “best of 2019” lists for more content soon, stay tuned!

“Enterprise to Computer (a Star Trek Chatbot)”: Grishma Jena with IBM (Video + Transcript)

Grishma Jena / Cognitive Software Engineer / IBM
Sukrutha Bhadouria / CTO & Co-Founder / Girl Geek X


Sukrutha Bhadouria: Hi everyone, I hope you’ve been having a great day so far. Hi, Grishma. Hi, so yes, we are ready for our next talk. I’m Sukrutha and Grishma is here to give the next talk. Just before we get started, the same set of housekeeping rules. First is, we’re recording. We’re gonna share in a week. Please post your questions, not in chat, but in the Q and A. So you see the Q and A button at the bottom? Click on that and post there. If for some reason we run out of time, and we can’t get to your questions, we’ll have a record of it and it’s easy for us to find later and get you your answers later.

Sukrutha Bhadouria: So please share on social media #GGXelevate and look for job postings on our website at We’ve also been having, throughout the day, viewing parties at various companies. So shout-out to Zendesk, Strava, Guidewire, Climate, Grand Rounds, Netflix,, Blue Shield, Grio, and Salesforce Portland office.

Sukrutha Bhadouria: So now, on to Grishma. Grishma is a cognitive software engineer at IBM. She works on the data science for marketing team at IBM Watson. So today her talk is about Enterprise to Computer: a Star Trek chatbot. I’m sure there’s a lot of Star Trek fans out there because I know I am one, and I can’t wait to hear about your talk, Grishma.

Grishma Jena: Thank you, Sukrutha.

Sukrutha Bhadouria: Go ahead and get started. You can share your slides.

Grishma Jena: Okay, I’m gonna minimize this. Alright, can you see my slides? Okay. Hi, everyone, I’m Grishma. As Sukrutha mentioned I work as a cognitive software engineer with IBM in San Francisco. So, a lot of my job duties involve dealing with a lot of data, trying to come up with proprietary data science or AI solutions for our Enterprise customers. My background is in machine learning and natural language processing which is why I’m talking on a chatbot today.

Grishma Jena: I’ve also recently joined this non-profit called For Her, where we’re trying deal with creating a chatbot that could act as a health center, as a resource center for people who are going through things like domestic abuse or sexual violence so I’m very interested to see you know, a totally different social application of chatbot. But for today we’ll focus on something fun. And before I begin, a very happy Women’s Day to all of you out there. So, yeah.

Grishma Jena: When was the last time you interacted with a chatbot? It could have been a few minutes before, when, you know, Akilah was talking and your Alexa probably got activated by mistake and you had to be like, “Alexa, stop.” It could be with Siri. We interact with Siri every day. It could be on a customer service chat or it could be on a customer service call.

Grishma Jena: Basically, there are so many different avenues and applications of chatbots today that sometimes it’s even hard to distinguish if are we talking to a human. Is it a chatbot in disguise of a human? And it’s quite interesting to see where chatbots have come in the past few years.

Grishma Jena: So, this was a grad school project that we did. Our idea was, okay, chatbots are amazing. We really like that they help take some of the workload off humans, but how can we make them seem a little more human, a little less mechanical? Could we give them some sort of a fun personality?

Grishma Jena: And we brainstormed for a bit and we finally came up with the idea, hey, why don’t we, I mean … Well, to be honest we weren’t that big fans of Star Trek, but we did become one during the course of this project and we were like, “Okay, let’s think of Star Trek”. It has a wide fan base and let’s try to not pick one single character from Star Trek but let’s take all of the characters and make this huge mix of references and trademark dialogues and see what kind of personality the chatbot would have.

Grishma Jena: So, like I mentioned, the motivation was to make a chatbot a little more human-like. And we wanted to have a more engaging user experience. So the application of this could be, it doesn’t have to be something related to, you know, like an entertainment industry. It could be also something like a sports lover bot so that would be very chatty and extroverted and it would support your favorite sports team. Or it could be something a little more sober like a counselor bot who is very understanding and supportive and listens to you venting out or asks you about how your day was. So yeah, we chose Star Trek infused personality.

Grishma Jena: So our objective with Star Trek was wanted it to incorporate references from the show. [inaudible 00:05:17] wanted to [inaudible 00:05:20] Spock and live long and prosper. We wanted it to be data driven model, we did not want to feed in dialogues we wanted it to just feed in a corpus and have it generate dialogues on its own. We obviously wanted it to give interesting responses and to have the user engaged because that is one of the things that a chatbot should do, right? So in really simple words, just think of a friend of yours or it could be yourself who is this, you know, absolutely big fan of Star Trek and just transfer that personality to a chatbot.

Grishma Jena: So this is what the schema of our bot look like. We had the user utterance which is basically anything that you say or that you provide as input to the chatbot. And then we had a binary classifier. I’ll delve deeper into why exactly we wanted it, but the main point is that we wanted it to be able to distinguish whether what you’re saying to the chatbot is it something related to Star Trek or is it something a little more general conversation like, “How are you feeling today?” Or “What is the weather like?” And depending on that we had on that we had two different routes which the bot would take to generate a response.

Grishma Jena: So before we begin, we obviously need some sort of data and we decided that we would take all of the data that was available for the different Star Trek movies and the TV series. You’d be surprised at how little data is available, actually. We initially thought of just doing a Spock bot, but Spock himself has very limited dialogues so we just expanded our search to the entire Star Trek universe. And that’s why we took dialogues from movies, TV series. We didn’t want to have any sort of limitations as far as the data was concerned. We ended up with about a little over 100,000 pairs of dialogues.

Grishma Jena: Then we also went and got this database, which is known as the Cornell Movie Database. This database was created by Cornell University, which has a collection of raw movie scripts. It’s just a really good data set to train your bot on, the way how humans interact and what kind of topics they talk about, what are the responses like.

Grishma Jena: And finally, we also had a Twitter data set because we wanted some topics that were related to the ongoing affairs in the world, the current news topics. Because we envisioned that if you had a chatbot then people do like to talk to the chatbot or ask for the chatbot’s opinion on something that’s happening in real time.

Grishma Jena: So the very first component of a chatbot was having a binary classifier. Like I mentioned, we had two different routes for our chatbot. One would be the Star Trek route and the other would be a general conversation route. So we had the binary classifier that would help us distinguish whether whatever the user is uttering or whatever the user is giving as an input is it related to Star Trek or is it general conversation which was getting handled by the Cornell Movie Database. So we used an 80:20, that is the training data set and the testing data set split. And the features that we used were we took the top 10,000 TF-IDF unigrams and bigrams.

Grishma Jena: TF-IDF stands for tone frequency and inwards document frequency. Tone frequency is nothing but how many times a given word occurs in your corpus and inverse document frequency,, it’s kind of a weight that is attached to a word. So think of a textbook or think of a document that you have. Words like prepositions, like the, of, and would occur multiple times. But really words that would be important that would have some sort of conceptual representation, perhaps like the topic of it. Compared to it would be a little rare in occurrence, compared to prepositions, compared to commonly used words, and that’s why they should be given more weightage. So that’s the whole idea behind TF-IDF.

Grishma Jena: Unigrams and bigrams are nothing but you divide the entire document that you have into words. An unigram would be one [bit kilo word inaudible 00:09:17] bigram would be a set of two consecutive words that occur in the document. There’s an example later on in the slide to explain it better. Stop words, when consider stop words are just filler words like I mentioned similar to the prepositions. And we were very happy with the performance of the binary classifier. We were able to get a 95% accuracy on the test set, and we decided that is good enough, let’s move on to the next one.

Grishma Jena: And finally, this is the main core of it, where deep learning comes into play. So with deep learning, we used a model called a Seq2seq which is a particular type of recurrent neural network. So if you can see the image on the right, it is a simplified version of a neural network where you give it an input and it gets an output and that output is also the input for the next cycle, so it’s kind of like a feedback looping mechanism.

Grishma Jena: First, the specific type of neural network that we use, Seq2seq. It was just two recurrent neural networks so just think of a really big component that has two smaller components, which is an encoder and a decoder.

Grishma Jena: So the encoder actually takes in the input from the user and tries to provide some sort of context. What do the words mean? What exactly is the semantics behind the sentence that the user has given? And the decoder generates the output based on the context that it has understood and also based on the previous inputs that were given to it, which is where the feedback mechanism comes into play.

Grishma Jena: So just to go a little deeper into it. This is a representation of what a Seq2seq with encoder and decoder would look like. So the input over here would be, “Are you free tomorrow?” and the encoder takes in that input and tries to understand what exactly is the context or the meaning of this sentence. And finally the decoders understands, okay, this is something someone is asking about either they want to take an appointment or someone’s availability or someone’s schedule. And that’s where the reply is something like, “Yes, I am. What’s up?”

Grishma Jena: So these are some statistics about how exactly we went on training this on AWS. We used a p2.xlarge instance with one Nvidia Accelerator GPU and then we had the Star Trek Seq2seq. So we had one Seq2seq for just Star Trek dialogues and we had another one, the Cornell Seq2seq which is on Cornell data, which is more for just a general conversation purpose.

Grishma Jena: So we went ahead, we generated some sentences, but then we realized that the ones for Star Trek were really good because you’re giving it Star Trek as input so obviously the output is also going to be Star-trekky. But for the general conversation ones, for things like, “What is the weather like?”, “How are you doing today?”, “What is the time?” it was a little difficult for us because obviously the input is not Star Trek related, right? So the output also wouldn’t be Star Trek related, but we wanted this to be a Star Trek chatbot.

Grishma Jena: So we brainstormed a bit and we thought, “Hey, why don’t we try something called a style shifting?” Which is basically like you take a normal sentence, a sentence from the general conversation, and you try to shift it into the Star Trek domain.

Grishma Jena: And the way we did this was, we went through the entire corpus, the data set for Star Trek, and we created a word graph out of it. A word graph would be, just think of it as you pass different sentences in the data set and each of the words would form a node and the edges between them would tell how they occurred in relation to one another. So if they occurred right next to each other or within the same sentence.

Grishma Jena: And along with the words in the node we also had a part of speech tag. So we indicated whether it was an adjective, or a noun, or a pronoun or a conjunction. So let’s say for example our sentence was, “Live long and prosper.” You break it down into four words which are the four different nodes and then we label them with a different part of speech tag and we connected them because they come one after the other in the sentence.

Grishma Jena: So what we did, was after we built out this really huge word graph, we looked it up to insert what could be appropriate words between two given words in the input. So once we had the sentence we would check for every two words in the sentence and see what are the words that we could insert in between to give it more of a Star Trek feel to it to just, you know, shift the domain into Star Trek.

Grishma Jena: We went ahead and we did that and these were the kind of results that we got. “I am sorry” was the input and then the word graph went ahead and inputted “Miranda” at the end. “I will go” and then it inputted “back” at the end of the sentence because “go” and “back” kind of occur very commonly with each other. And similarly for the start of the sentences, it tried to input names like “Uhura” or “Captain”. So one thing we noticed was it really good at inputting names at the start and the end of the sentence and using the character names from the show did end up giving it a slightly more Star Trek feel than before.

Grishma Jena: So we went ahead and we just randomly tried to insert words that occurred more frequently between two words but then we realized that some of the sentences were ungrammatical. So what do we do? We came up with this idea of let us use the word graph as it is and then let’s take some sort of a filter to our responses. So, like I said, we realized that the word graph was giving a few incoherent and incorrect responses. What we did was we went ahead and constructed an n-gram model.

Grishma Jena: So n over here would be unigram, bigram, trigram. You can see the example over here if n is equal to one, which is an unigram, you break down the sentence into just different words so “this” would be one unigram “is” would be another unigram. If n is two, which a bigram, you would take two words that co-occur together. So in this case the first bigram would be “This is,” second one would be “is a” and then similar for trigram it would be “This is a” and then “is a sentence”.

Grishma Jena: So we created an n-gram model which was just to understand what exactly is the kind of dataset that Star Trek has. And then finally we wanted to get a probability distribution over the sequence of words that we have had.

Grishma Jena: So once we get this, we start to filter the responses and we ran the sentences using the bigram models that we trained on the Star Trek data set. Because of this we kind of got a reference type for seeing that what structures are grammatically correct. We went ahead and we get them and the ones that were a little odd sounding or that didn’t really occur anywhere in the data set we went ahead and removed them.

Grishma Jena: Another metric that we used for this was perplexity. So just think of perplexity as some sort of an explainability metric. We went ahead and used that which would help us tell how well a probability distribution was able to predict it.

Grishma Jena: Finally, we have all of the things in place and we have to evaluate the performance of the chatbot. So we came up with two categories of evaluation metrics. The first one was quantitative metrics where we used perplexity, which was mentioned on the first slide. And the second one was we wanted to see often was it using words that were very particular to Star Trek that you don’t really use in normal day life, you know, like maybe spaceship or engage.

Grishma Jena: And the second category was human evaluations where we got a bunch of, user group and we asked them to just read the input and the output and see how good it was in terms of grammar. If the response actually made sense, if it was appropriate. And finally, on the Star Trek style. Just how Star-trekky did it sound?

Grishma Jena: And, we also came across another bot online which is called as a Fake Spock Pandora Bot which was contrary to the way we had. Our bot was data driven this was rule based so it was actually given an input of human generated responses.

Grishma Jena: We wanted to see how good would a data driven model perform as compared to a human generated one. So this is just what the Fake Spock Pandora Bot looked like. And these were the kind of responses that the Pandora Bot gave. If you said, “I’m hungry, Captain” it said, “What will you be eating?” So it’s giving really good appropriate responses because humans were the back end for this.

Grishma Jena: And then, what we did was we went ahead and evaluated the results. And we saw that our bot was performing better for Star Trek style and it also was a little more coherent. For grammar, Pandora Bot was much better and that’s not surprising because humans were the ones who actually wrote it out. For perplexity, the Star Trek perplexity started dialogues were 65, so that was our baseline number and we figured out that the kind of responses our bot was generating that are 60, 60.9 was a little closer compared to Pandora was like, way far off at 45.

Grishma Jena: So we were pretty happy with our performance. I’m just gonna give you a few examples of what the different bots generated. So the yellow ones are the Pandora Bot and the blue ones are the E2Cbot. So let’s see, if the user says, “Beam me up, Scotty” the yellow one, that is the Fake Pandora Bot, gives, “I don’t have a teleportation device” which is a good answer. And the blue one is, “Aye, Sir” which is also a good answer. A little curt, but nothing wrong with it.

Grishma Jena: In the second example if you see our bot answered, “Bones, I like you.” So the “Bones” part is actually come from the word graph which gives it a little more of a Star Trek feel. And the last one over here is the Fake Bot, the human generated one, just says, “I am just an AI chatting on the internet” which is kind of not the response that you are looking for.

Grishma Jena: A few more examples over here. The user says, “My name is Alex” and then the Fake Spock Bot says, “Yes, I know Christine.” I just told you my name was Alex, why would you call me Christine? But our bot says, “What do you want me to do, Doctor?”, which is a better response. And, yeah, these are the kind of responses.

Grishma Jena: I think some of our human focus group people said that they might be correct, appropriate responses, but they might not be factually correct, which was a challenge for us, as well as for the Fake Spock Bot. We didn’t really delve deeper into it because that would kind of dive more into having a question answering system and trying to check if it’s factually correct or not but we tried to make our focus group users understand that it’s just a bot at the end of the day.

Grishma Jena: So finally, we were able to generate Star Trek style text. We were very happy with that, we were able to use the data driven approach which meant we could automate it. And we did figure that it performed better than the human generated responses that Pandora Bot would give, at least on style and at least on the appropriateness. It still needs a little bit of improvement in grammar but we were pretty happy with it.

Grishma Jena: So that’s me. Live long and prosper. And feel free to reach out to me on Linkedin or on Twitter if you have any questions about this. Thank you.

Sukrutha Bhadouria: Thank you, Grishma. This was great. So just to close I just wanted to mention to everybody that you actually sent your speaker submission to us and that’s how we got connected. So thank you for doing that. We got a lot of comments from people who are Star Trek fans, but yeah, what inspired you to build this project?

Grishma Jena: Yes, so this was actually a grad school project. We were taking a deep learning course so all of us had to build a chatbot as an Alexa skill. We brainstormed a lot, and we actually thought that Spock because Star Trek has a really huge fan base so Spock would be a good idea to do. But Spock had very little dialogue in all of the movies and the television series and then we were like, “You know what, let’s not stick to just one character, let’s have the entire Star Trek universe.” And, the bonus was that during my semester, I could continuously binge watch Star Trek and say that, “Yeah, I’m doing research because I want to see how well my chatbot works,” but I was just binge watching to be honest.

Sukrutha Bhadouria: Nice. That’s awesome. Well, thank you so much, Grishma, for your time. We really appreciate it and for your enthusiasm in signing up through our speaker submissions.

Grishma Jena: Thank you so much, Sukrutha.

“Coding Strong at Age 60”: Akilah Monifa with ARISE Global Media (Video + Transcript)

Akilah Monifa / SVP / ARISE Global Media
Gretchen DeKnikker / COO / Girl Geek X


Gretchen DeKnikker: I’m so, so, so excited about our next speaker, Akilah Monifa. She is the SVP at ARISE Global Media, which is a digital media platform for LGBTQ folks of color and their allies. And she made an Alexa skill called Black Media–or Black History Everyday, which I really want to just make it Black History Errryday. But not everybody’s gonna put all the Rs in it.

Gretchen DeKnikker: I’m very, very excited for this talk and you guys are gonna love it. Please, welcome Akilah. All right.

Akilah Monifa: Thank you, Gretchen.

Gretchen DeKnikker: All right, thanks.

Akilah Monifa: Welcome, everyone. It kind of reminds me, the start kind of reminds me of in eighth grade watching a science film and the film broke, but it is 2019, so we did get it together. I am Akilah. I’m gonna talk today about my Alexa Skill Black History Everyday.

Akilah Monifa: Even though you can see me, just wanted to share a little about me and the skills. This is me. This is my wonderful headshot. One of my favorite shots of myself. This is me and my children. my son Benjamin who is 15 and my daughter Izzie who turned 18. This is Raya Ross who is my intern and is a high school student, and helps me work on the skill. I just wanted to show a picture of her. This is my friend Elan and myself. We are in our Black History is Golden tshirts from the Golden State Warriors, because, obviously, black history is near and dear to my heart. Elan also helps a lot on the site, too.

Akilah Monifa: Okay. Now, this is just a brief little video that I wanted to share with you that Alexa made about my app.

Akilah Monifa: My first skill is pretty simple. It’s called Black History Everyday.

Alexa: Patricia Bath, that first black woman to serve on staff as-

Akilah Monifa: It started to work at 5:00 AM, on April 3rd, 2017, which happened to be my 60th birthday. And I cried when it worked. I cried tears of joy. I want people to know that you don’t have to know the coding to do it. I didn’t know the coding, and I actually now have three skills. I think it’s very exciting. I mean, I don’t think that I can adequately describe just the thrill that all of these skills have, but particularly the first one. And to know that so many people can hear the skill and be as enlightened through sound and knowledge, as I was, it is, I think, very, very profound.

Akilah Monifa: My children jokingly say that that’s my commercial for Alexa.

Akilah Monifa: Why did I start the skill? The first thing was that, as we all know, Black History Month in the United States is in February, and it’s the shortest month of the year, lot of people have complained about that. 28 days, 29 in leap year.

Akilah Monifa: My other big issue was that I really wasn’t learning much in Black History Month. The same facts were being regurgitated over and over. So, what do you remember about Black History Month in general? I mean, we heard facts about Martin Luther King, George Washington Carver, Rosa Parks, and that was really the extent of it. That certainly was not sufficient for me.

Akilah Monifa: The first thing that I did was to develop a website which is I was actually amazed that the URL was available, but it was, so I developed the website. My thought was that every day I was going to put a different black history fact on this website.

Akilah Monifa: Here are a couple of examples. The website exists. A few examples of the facts that I put on the website, and they’re very short. I wanted them to be diverse. This is Isis King who is the first transgender model to compete on America’s Next Top Model in 2011. This is the Mobile Edition. This is what it looks like. Mashama Bailey, the first black woman nominated for Best Chef at the James Beard Foundation awards 2018. Glory Edim, she’s the founder of Well-Read Black Girl, an online book club and community.

Akilah Monifa: The other thing that I wanted was the oh wow factor, “Oh wow. I didn’t know that,” or, “I was unaware of that.” So, I really tried to have really interesting things. Since today is International Women’s Day, starting today through the rest of the month all of my facts are going to be about women, about black women.

Akilah Monifa: Now we go from, I have this website. Two years ago, someone gave me an Alexa, and I had heard about it, but I had not experienced it. I got it. I saw that there were all sorts of skills on Alexa, so I thought I should be able to have my website into an Alexa skill. That was my thought. I thought how difficult can it be. Actually, I thought I don’t know anything about coding, so maybe I can’t do it. But I googled how to do an Alexa skill, and found out there was something called the Alexa skills kit, and that was online.

Akilah Monifa: So, I went to the Alexa skills kit and got information that alleged that one could build a skill in minutes with no coding required. I said, okay, I’ll develop the skill. Basically, when I went to the Alexa skills kit, there were five different entries that I could make to help develop the skill. I suppose theoretically, it could have been done in minutes…skipping ahead. It did not take me minutes. And when I tried to fill out the form or I did fill out the form and I developed my skill, it got rejected. I lost count the number of times that it got rejected. After you submit it, you submit it for certification, and it was not successful. I think I submitted it between 75 and a hundred times. I joined Alexa developers groups to try to figure out what was wrong and talked to people and tweeted…. The shorter version of it is that finally, after all of this, it did start to work. And I just wanted to show you this is just the first page. It was almost fill in the blanks. But the key thing that was missing for me in developing the skill is that I thought that simply by having the website that I could feed the website into Alexa, and Alexa would be able to read out my website, and that in fact was not the case.

Akilah Monifa: It was finally when I, through a lot of research and trial and effort, realized that one thing that I needed was to get Alexa to talk to the website. It was pretty simple. I just had to find a device, and the device that I found is called Feedburner, Once I plugged my website into that, then Alexa could understand what my website said and read out the information, which was just wonderful.

Akilah Monifa: As I described in the video, it actually started working on my 60th birthday, which was two years ago, which will be coming up two years ago, so I was very ecstatic. I can also really, if you’re trying to build an Alexa skill, really recommend Feedburner. After that, it was very simple.

Akilah Monifa: I just wanted to show–The skill, I did a definition of the skill. The skill basically says that it is Black History Everyday in about a minute from Arise 2.0. Black history is no longer relegated to the shortest month of the year. A different black history fact presented daily, seven days a week, 365 days a year, 366 in a leap year. It’s prepared. I say, “Invented by the team at Arise 2.0,” which is mainly consisting of me and Raya with some help from a few other friends who give me information. Our mission is to tell our diverse stories.

Akilah Monifa: If you have an Alexa and you go to Alexa, you can enable the skill in the app. And there it is, Black History Everyday, actually with an old logo. Or you can actually just ask it to enable it. I just wanted to at least show you–and hopefully, Alexa will work–how it works.

Akilah Monifa: Alexa, what’s my flash briefing?

Alexa: Here’s your flash briefing. From Arise 2.0 Black History Everyday, Zarifa Roberson, CEO/ Founder/ Publisher of I-D-E-A-L magazine for urban young people with disabilities 2004 to 2015.

Alexa: Toni Harris is the first woman football player at a skill position, non-kicker, to sign a letter of intent accepting a scholarship to Central Methodist University in Missouri in 2019.

Alexa: Akilah Bolden-Monifa, Alexa pioneer, developed Black History Everyday Skill for Amazon’s Alexa in the website

Alexa: Dr. Roselyn Payne Epps is the first black woman to serve as President of the American Medical Women’s Association in 2002.

Akilah Monifa: The only glitch was that it was my intent to have one black history fact every day. What I found out with Alexa is that through my website Alexa would read out five facts a day. I had to basically then shift gears and make sure that I had five different facts a day instead of one. That’s my skill. Thank you.

Gretchen DeKnikker: Thanks. Looks like I was still muted. Thanks, Akilah.

Akilah Monifa: You’re welcome

Gretchen DeKnikker: That is so awesome. There’s other people. It’s the same. People [inaudible 00:11:48]. That’s making their Alexas go off just listening to you.

Akilah Monifa: Yes.

Gretchen DeKnikker: Which is awesome, because that’s what happened when we did the dry run for her speaker talk too. And so, we had one question come in. She keeps getting rejected, she’s saying with Google not Alexa. Because I think they don’t want to give me the name I want. It’s frustrating for an indie developer. How many times did you say you had to keep applying?

Akilah Monifa: I lost track, but I believe that I applied for certification between 75 and a hundred times before it was accepted. And I would say that the one thing–that it passed certification, basically.

Akilah Monifa: The one thing that I didn’t do was you can test it before you submit it for certification, and I didn’t do that. I foolishly just kept certifying it and submitting it through certification thinking that it would work, and it didn’t. If I’d tested it, I would have seen that it didn’t work, so I probably wouldn’t have submitted it for certification

Gretchen DeKnikker: Another question. What was the thing that surprised you most about developing a skill?

Akilah Monifa: I think that the thing that surprised me, what most, was how easy it was that I just had the idea. Before people told me that you needed coding to do it or you needed to pay someone to code you, so I thought I can’t do it. The surprising thing was that when I googled how to build an Alexa skill, yes, if you know coding you can build it, but you can build it without knowing coding.

Gretchen DeKnikker: Amazing. I think this is great. What I’m really hoping, this will be my request to you, is that next year you can come back and tell us about building it for Apple and for Google, so that we can all have it, because I do think that American school systems don’t do a great job of giving that information out. It’s amazing that you took the time to just share it with everybody.

Akilah Monifa: Well, and the good thing is that it is available to everyone because even if you don’t have the skill, if you don’t have Alexa, you can get the information through the website. Just go to, and all the information is on the website, which is good.

Gretchen DeKnikker: Awesome. All right, Akilah, this was great. Thank you so much for taking the time.

Akilah Monifa: Thank you.

Gretchen DeKnikker: All right.

“Unconventional Journeys in Tech” —  Girl Geek X Elevate (Video + Transcript)

Shanea Leven / Director of Product / Cloudflare
Farnaz Ronaghi / CTO & Co-Founder / NovoEd
Rosie Sennett / Staff Sales Engineer / Splunk
Angie Chang / CEO & Founder / Girl Geek X


Angie Chang: Great. Well, we’re all here. Welcome back to Girl Geek X, Elevate. This is our afternoon session. We will be talking about our unconventional journeys in tech, since it seems like that seems to be more common than people realize. The diverse pathways we have found to our jobs, that we get lots of people coming up to us, and saying, “Wow, you’re doing a really good job. That’s a successful career.” We’re like, “Oh, okay.” And people are like, “How did you get there?” So, hopefully, we’ll be able to share some of these stories, and I’m gonna ask each of our panelists to talk about themselves and share their backgrounds and their journeys and tell us their personal stories.

Shanea Leven: Sure, I’d love to start. Hi, everybody. I feel like I’m echoing.

Angie Chang: I think you sound fine.

Shanea Leven: Okay. Okay, cool. Hi. I’m Shanea, I started actually, my career, as an analyst and I was really interested in getting into tech. I actually started a digital marketing agency, way back when. That worked really, really well. I taught myself to code, for a number of years. The thing about myself, is that after starting, doing the entrepreneur thing, I wanted to get into tech in Silicon Valley, and so I ended up taking a job at Google, as a Program Manager.

Shanea Leven: Along the way, I really wanted to get into Product Management, except that the prerequisite at this time was that you needed a computer science degree. For myself, having gone on a ton of journeys trying to figure out how to navigate the tech space, one of the big questions was, am I technical enough? Do I have enough technical skill?

Shanea Leven: In order to get that product management job, I had a few options in front of me, which was to either go to a bootcamp, trying to gain additional technical skill, or get a computer science degree, or possibly get a technical Masters degree, which I know a lot of people face. For me, I ended up going back to school to get a Computer Science Bachelor’s Degree, while working full time at Google.

Shanea Leven: After, I became a Product Manager. I proved myself to gain that technical skill. I went on to work at Ebay, as a Senior Product Manager. I went on to work as Head of Product, at a startup, and now I’m a Director of Product at Cloudflare.

Angie Chang: Awesome. Farnaz, I think you have kind of the opposite story, where you started with the CS degree, you want to share a little about your journey?

Farnaz Ronaghi: Yes, of course. Hello, everyone. Yes, I come from a very traditional computer science background, but I did not start by choosing computer science as a major. They way education works in Iran is that for free. I’m from Iran, from Tehran, education is for free. However, we go through a very detailed, competitive University entrance exam. I was an A+ student, and I was very arrogant. I picked my top major to be double E, and I told everyone else, “I don’t need to pick more, that’s it.”

Farnaz Ronaghi: That was the paper I submitted. But my Mom, out of just being so kind, and out of love, filled out a few more degrees for me, a few more preferences. Her preference, her first preference for me was computer engineering. So that is how I ended up in computer engineering. Yes, I had that training, I had that Bachelor’s degree. However, the training did not make me a software engineer. It did not make me fall in love with writing code, and it did not make me confident that I am gonna be someone who will stay in tech.

Farnaz Ronaghi: But that happened, I came to the US for a graduate degree, and started a company. I learned through fire, by just being on the job, and doing it myself.

Angie Chang: Rosie, why don’t you share your story? I think you are definitely someone who has picked up things along the way, and has a very scrappy attitude to learning.

Rosie Sennett: Yeah. I was a Theater major, a technical theater major and I actually started out building props and costumes on Broadway, and then I kind of got distracted by coding–this is gonna show my age–coding macros in WordPerfect when it was green screen and it caught me.

Rosie Sennett: So, one day, I opened up the New York Times, and there it said, “Seven months. Learn to be a programmer.” So I signed up at Baruch College, and learned COBOL, and assembler language, and I got my first job at Information Builders in New York, answering the phones in technical support. In fact, I answered the phone so often like that, I would do that at home.

Rosie Sennett: “Information Builders, technical support. How can I help you?” I climbed that support ladder, and while I was doing that, the internet was born. So, I taught myself how to make webpages and marketing noticed that I was kind of messing with the website, and they decided I should be over there, and I discovered what marketing was, and what sales was, technical sales, and ended up as a Sales Engineer. I was there for 13 years and then I decided to go back to entertainment.

Rosie Sennett: I went back to entertainment, and I went west, and stayed and did some film stuff for a while. Now, I decided it was time for insurance and a paycheck again. Floated my way back in, and the last six years, I’ve been at Splunk, happy as a clam, doing sales engineering.

Angie Chang: Quick question. I think Sales Engineer almost feels like an insider term in Silicon Valley. Can you explain a little, for some people who may not be so aware, what is sales engineering?

Rosie Sennett: Sure. A sales engineer is the technical person on the sales team. So, sales engineers are the jack of all trades, the technical person who gets to do…. Our technical knowledge is super, super wide and deep, and the last bit that we concentrated on with the last customer. So, you know the products of the company, usually a vendor, that you work with, and then other stuff that you adapt.

Rosie Sennett: So, we’re the most technical really, that there is, ’cause you have to know a little bit about everything, and be able to present, and talk, and interface, and translate for the sales people.

Angie Chang: Cool, cool. What drives you forward in your career over the years? Like, what has kept you … I don’t want to say leaning in, but like, engaged to pull the levers, to figure out where to go in your career.

Rosie Sennett: I would like to say it was determination and a laser-like attention span. But actually, it was sort of distraction and curiosity, honestly. Now, in my sort of elder years, I have some very clear direction, I think. I’m actually really inspired by the young women coming in, and I’m really inspired by all the amazing stuff that’s happening at Splunk. I really landed in the most amazing place. Very lucky and feel really privileged to be working with the people I’m working with, and working in the programs that we’re doing at Splunk. Hiring like crazy, by the way, so come take a look at all the stuff, and apply.

Rosie Sennett: But yeah, I mean, right now, mentoring and being mentored by my mentees. The most amazing thing. No direction until I found direction, by watching all the amazing people who are now coming into the industry, honestly.

Angie Chang: Shanea, what has driven you in your career, and your product?

Shanea Leven: Yeah. So, a few things has driven me. I think from my perspective, I love building products. I like being able to have a lot of ideas all the time, about almost everything. I’ve found that product management is a really great medium for me, personally, between being an entrepreneur and working at a company.

Shanea Leven: I still get a lot of those same feelings that I got when I was an entrepreneur, with a lot less risk. Another thing that drives me is that, at this point in my career, I feel this personal responsibility to help the folks coming up behind, like, the generation that’s in right now. Some of us have to take a stand and stay and push through, so that we can make change for everybody else. So, hopefully our daughters won’t struggle with some of the things that we struggled through. That’s incredibly motivating to me.

Angie Chang: Farnaz, how about you, what do you think has been really driving your career forward, and the challenges? I know once, on the prep call you mentioned motherhood as something that has really changed the way you look at your career and managing your career and life.

Farnaz Ronaghi: Yeah. Well, and moving forward, when I started writing code on my own, the joy of creation was what was pushing me forward. But it was getting a little bit crazy. I was becoming, not becoming, I was the engineer who was taking six shots of espresso per day, and is coding 7 AM to like, 2 AM, and only sleeps five hours, just doing that all the time.

Farnaz Ronaghi: But then, motherhood, yes, it happened and while I was pregnant, I used to talk to my team and say, “Well, it’s no different. It’s just a baby. We will deliver it, and move on to becoming the same person.” Just like that. But when baby arrived, I actually learned that time is limited, there are other priorities in life, and I started prioritizing my work better.

Farnaz Ronaghi: The impact was actually surprising. It helped me make better decisions, because then I was always writing code, or working on something. I always thought time is infinite. So, there is a lot of time to get back to things, or to not close the loop on something. Or to not discuss it for today, leave it for tomorrow.

Farnaz Ronaghi: When baby arrived, I had to be more on point, and I had to make harder decisions to pick one way versus another and to be actually more deliberate about what I learned, what I spent time on, what technology I explore, and what I don’t. It has been helpful in that way to give me depths in technologies that I need to have depth in and leave alone parts of the stack that are not necessarily in my expertise.

Angie Chang: Cool. I also have a question for the panel, which is, what is a challenge you’ve faced, and how did you fix that? Rosie?

Rosie Sennett: Do I look the most perplexed? What was a challenge I faced? That’s a long road.

Angie Chang: Okay, I have another way to put it. How have you learned from a career mistake that you thought was a mistake that has turned out to not be a mistake? I think one thing we hear a lot at Girl Geek dinners is, “How do I know I’m going on the right direction?”

Rosie Sennett: Mm-hmm (affirmative).

Angie Chang: Or Shanea, do you have-

Shanea Leven: Yeah. I fail all the time. I had this conversation recently, every time I start a new job, or every time I start a new thing, it always seems like right at the beginning, there’s always a setback. Always. It came up in conversation, because like, “Is it just me?” Am I constantly getting setbacks right when I feel like I’m taking two steps forward? I don’t think that that’s the case.

Shanea Leven: I had a couple of really big failures, at least I’ve categorized them in my mind as big failures. I took the risk, but ended up, after seeing it through, I kind of failed up and that’s perfectly fine. It’s okay to switch, it’s okay to change direction, it’s okay to move forward. Sometimes you don’t know until you try and completely failed.

Shanea Leven: When I was actually getting my CS degree, I tell this story a lot, which is, I basically cried almost every night. It was challenging. It was one of the hardest things that I’ve ever had to do. It challenged my identity to my core, and I couldn’t be happier that I did it. Every day it felt like the worst struggle ever. But it made me better.

Farnaz Ronaghi: Actually, to build up on that point, knowing what is gonna work and what isn’t gonna work is impossible. We never do. We have patterns that we have seen in life, we have interests and excitements. So, we just go with that, and we push and push and push. There may be a time that we realize, “Okay, this cause is not a cause that is worth pushing,” and sometimes we push and we push, you know, what essentially makes you stronger. You come out, out of fire, feeling like you are unbreakable, because you just did that.

Farnaz Ronaghi: Guess what? Just a few months after, something happens that feels like the worst event of your life, and if we don’t let ourselves feel that we have failed, if we don’t feel ourselves burned and emotionally challenged by something, we will never learn. We will never have any opportunities to grow, because we are always staying in our own comfort zone.

Farnaz Ronaghi: When it comes to computer engineering, and writing code, it’s just everywhere. It’s a very male dominated industry, and as a result of that, in many places, the culture may not be as positive as some of us like it to be. Or, it may not be the culture that we are used to engage with and that by itself used to be a big emotional burden on me. “If I don’t belong here, why am I pushing?” But at the same time, when you push, when you show up, you change the ones around you, and you actually find your own people. Not everyone is the same. You’ll find a group of people who are like you, and you work, and you have fun.

Shanea Leven: Mm-hmm (affirmative).

Angie Chang: That’s a good point.

Rosie Sennett: I saw not too long ago, a cartoon that had two pie charts. One was completely red, you know, a circle, and it is said, “Life ending disaster that it felt like,” another pie chart was a little slice, it said, “Actual problem,” and the white thing, it said, “The thing I will learn from.”

Rosie Sennett: I discovered in my 30s that I have ADD and what that manifests as is a bunch of blockers, things that will stop you from being able to complete tasks, or keep your attention in places. It’s considered a neuro-divergence, right? It’s things that require methodologies to complete tasks that other folks just do. Paying bills, getting through an exam focused, practicing piano. For me, I don’t necessarily look at things as failures, because hitting walls, or for a long time, getting really crappy grades, for me, was like, “Mm-hmm (affirmative), okay,” and I got used to it.

Rosie Sennett: I would think, “Okay, well, I have to figure out a way around this.” More than having something be a failure, it was like, “Okay, that’s what that is. Now I need a way around something.” Or it was a lesson learned that I hadn’t realized before. Sometimes it was a hurdle I didn’t notice was there, and I went … stumbled over. Having recognized that hurdle, once I knew what was going on, I would then find a way over it, the next time. Right? By like, putting a lot of pillows, you know, and padding it.

Angie Chang: That’s a great way to look at hurdles and failures as ways to tell, this is an opportunity to try something new and figure out a new pathway to what you want. I’m wondering, what is advice you would give your younger self? Like, any resources, or something you would have done differently? A way that you looked at your careers?

Rosie Sennett: Listen to your elders. You can learn from them. Yes. That is good advice.

Shanea Leven: I think one of the things that has really hit home for me in recent years is, “Dare boldly. Take bold steps,” and they are scary. It’s really, really scary to put yourself out there, but ask really bold questions and do bold things, because you can completely surprise yourself.

Angie Chang: Do you have any examples for that?

Shanea Leven: Yeah. A lot of times in tech, we get beaten down a little bit, right? I was just leaving Google, and I was convinced that I could not do product management. So, I decided to go on a confidence kick. I was like, “I’m gonna get my confidence back.” I used to be a very confident person. What happened?

Shanea Leven: One of the things that I read, in, I think, The Confidence Code, was asking bold questions. I went to a product management meetup with all of the female VPs of Facebook, and I stood up, announced like, “How are you guys helping,” I didn’t get into Facebook at the time, but, “how are you guys helping move Facebook forward, because if you guys couldn’t get through the process when you interviewed,” which is what they all talked about, “how are anyone else supposed to get through, today? Like, what are the steps that you’re actually taking?”

Shanea Leven: Before I did that, I had to stop myself, because I think I blacked out a little bit, I was so nervous. But absolutely, it was a starting point for getting my hand raised, getting a good question answered. It started a really great discussion. And then, after, I took another bold step, and I basically just walked up to Deb Liu, and said, “Can I interview for you, again?” And she said yes.

Shanea Leven: I went through the interview process and didn’t decide to go to Facebook, but that step of just asking her for, in front of hundreds of other people, and getting a yes, was my first big step into that, and to this day, I continue to kind of live by that. ‘Cause you never know what the other person’s gonna say, unless you ask.

Angie Chang: That was a terrific point. Is there more-

Rosie Sennett: What’s the worst that could happen? Right?

Shanea Leven: Yeah, yeah. Exactly.

Angie Chang: Does anyone have a story of the time they asked for more? I know this was kind of a theme through the day, we had some people-

Rosie Sennett: That was really cool. That was great, when Leyla that was talking about that. That was amazing. [crosstalk 00:23:08]

Angie Chang: I’m wondering if there’s a specific advice–

Shanea Leven: I ask for more all the time. I actually had a conversation with another fellow woman in product management this afternoon. When I started, I made a certain amount at Google, and I didn’t know, I had no idea what the context was for asking for more, and asking for my worth. I started working there a little bit, and I realized, I’m like, “I think I’m underpaid.”

Shanea Leven: I vowed that I would never ever do that again, because I felt like I had missed my opportunity. In my next role, and the next time that I was able to negotiate for salary, I asked for a lot more, and I got it. The next time, I asked for a lot more than that, and I got that. Because again, taking bold questions and just asking the right question and being prepared, you never, ever, ever know.

Shanea Leven: Sometimes, you ask bold questions, and bad stuff happens. But it’s again, what is, all of the things can be fixed with proper communication. All of the things, if you’re able to kind of dare to, you know, put yourself out there. Then things can be mitigated and things can be fixed. But yeah, and also, as a PM, it’s kind of a skill to keep asking for more, and more, and more, and more all the time.

Angie Chang: It’s a great work skill, definitely.

Rosie Sennett: Yeah, it’s true.

Shanea Leven: Yeah.

Angie Chang: I am curious more, it seems like, there’s different types of people that we meet, that attend Girl Geek X events, some of them are new grads, some of them are doing the mid-career bootcamp career change. Some of them are moms coming back. Is there specific advice, or is it the same advice you’d give them on how to come into tech, or like, people always ask us, “How do I get into tech?”

Angie Chang: It seems like, to me, tech has changed in how people view it over the last decade or two. It’s become a lot more intimidating. Have things really changed that much, and is there any advice you would give to people who are looking to change their job title, or come into a new role?

Rosie Sennett: I did not have a standard background, as we said, in a time when you needed one. I was a total unicorn, and it was spoken of lots of times. The guy who hired me held on to my letter and my resume, and walked around, ’cause he was a character, and if I could do that then … you know. In fact, I gave talks about it at the school I went to, ’cause it was weird, that I actually made it through, to them.

Rosie Sennett: Now, it’s not so weird. If it’s of interest to you, and I said this to lots and lots of women at Lesbians Who Tech, who came up to us last weekend, asking the same question. You know, “But I do this, can I do that?” You will not know, unless you boldly go forward, as Shanea said. You’ve gotta just push through. I mean, we live in a world now, where you can actually just teach yourself stuff, and teach yourself enough stuff to boldly say, “I know how to do this,” and as we learned today, it’s just women who think they need to know all of it, in order to say, “I know how to do that.” Yes, you can do that, by just saying, I know how to do that. Right?

Shanea Leven: Yeah.

Rosie Sennett: And then eventually, you will know how to do that.

Angie Chang: Anyone else have advice for any of these Girl Geek X community people?

Shanea Leven: I think that some advice is, if this is the goal, and you really want this goal, just be mentally prepared that there are probably gonna be some challenges. If I could go back, I would try to have a support system for not doing it on your own, and asking for help when you need it. I read in the book, really recently, Dare to Lead, by Brené Brown, to have like, written on a card, what actually really matters to you? Whose opinion about you actually matters? Because you might get a lot of things …

Shanea Leven: I switched careers several times, and I was doing product management work before I had the title of product manager, and it took me eight months to find a role as product manager, even though I was already doing the job. That one simple, tiny little thing was enough, that no one wanted to take a risk. I’ve heard that it could take a long time to get what you want. Or, being able to test, get data back and user test like, how you craft the story of your transitioning in. Just be patient with yourself and have the support system to vent if you need to, or test some things out, or role play, or just taking those baby steps.

Shanea Leven: But don’t compromise what your goal is, which is why the first part of it, this is your goal, stick to it. A lot of people sometimes think that it’s good to kind of weave around, and “I’m gonna take this job, ’cause it’ll lead to this job,” and sometimes it’s just better to just be a little bit patient and just get the job that you want. I learned that, where a lot of folks said, “Oh, you’re transitioning into product management, you should go take a step backwards and be a junior product manager, because you’ve actually never had the title.” I was like, “Absolutely not, I’m not gonna do that. That’s crazy.”

Shanea Leven: Like, there’s no way that I’m gonna do that, because I’ve already been doing it. There’s something that I’m not articulating, this is not the right company, or something like that, where giving yourself the leeway to try things and then I went from Program Manager straight to Senior Product Manager and I did just fine. It’s just all right, it’s fine. All good.

Angie Chang: I’m gonna move on to some questions from our audience. What types of roles have you seen former educators move into when they transition into tech, and similarly, what advice would you give educators who want to move into tech?

Rosie Sennett: Well, a direct one would be education at a vendor. That would be an easy slide, internal or external. It would be the straight up, “I can do that.” If you want to take your resume and say, “I know how to teach.” ‘Cause trainer education, internal and external, that would be the way in, “I want a job today,” way to go. But any kind of customer facing role is gonna be actually on that. If you can teach and interact, pick up whatever … I’ve been in vendors this whole time, that’s my thing. So, the shortest from point A to point B, to me, is gonna be right in-

Angie Chang: That froze, unfortunately.

Farnaz Ronaghi: Yeah, I actually, I agree with Rosie. For a teacher, there are a lot of tech companies that would have good … not customer support, customer success roles, professional services roles, program management roles that basically are, you are building programs on technology. So, they are very technology oriented, and you will learn a lot of technical things, as if you join us at NovoEd, you will learn. But your core skillset, which is teaching, is critical to being successful in that role.

Angie Chang: That’s good advice. I think when I talk to people who are looking for jobs, a lot of times, people are looking, but at the same big brands, at their roles. But also, there’s a lot of early stage and medium sized companies that we don’t necessarily think about. They can be found on AngelList, or just by browsing on the internet, you can find smaller companies that will take a chance on people with resumes that show more different experiences, work experiences, life experiences. I actually recommend that path, as well. Then, after a few jobs, or years, maybe you will be at that big famed company, with the brand name. But to maybe start, and thinking about the smaller companies, and joining those roles for experience.

Angie Chang: Let’s see, another question we have here is, Shanea, knowing what you know now, would you still do a computer science degree?

Shanea Leven: Yes. Absolutely. For me, I briefly mentioned that it was identity, it went right against who I thought I was, by getting this computer science degree. My options were, the bootcamp, I already have a Bachelor’s in Business, and the CS Bachelor’s degree. What I found was that I’d gone as far as I could, teaching myself to code. I had done literally every online class that I could possibly do.

Shanea Leven: The ironic thing is actually, I actually taught, created online coding training, which is the crazy thing. For me, there was a different problem, that I didn’t realize that I had had, until I went to get the computer science degree. Which was, I grew up in inner city Baltimore, and my school system, I did not realize, failed me. There was a lot of things that with getting a computer science degree, and the way it’s taught now, that they assumed that I knew, and I did not.

Shanea Leven: I needed to go back to school, and I also have a little bit of dyslexia. Who knew? Until I was reading programming documentation for a long time, so I absolutely needed the CS degree to fill a lot of the holes and really, to ground my foundation of the theory, in addition to the actual syntax of languages. I would probably do it again. I don’t think the job that I have now, that I could have gotten it, without having the CS degree, and not having a solid foundation in the theory.

Angie Chang: Thank you. So we have a question here about, was networking a big part of your career pivots, or successes? It seems like everyone talks about networking like that’s the only way, but people feel it’s dirty networking for professional reasons. Can anyone give some thoughts on networking? Or what’s the best way to get a job or promotion?

Shanea Leven: Networking was important for me. When I decided to take the role at Google, it was because I actually kind of stalked a recruiter. Like, a friendly stalking. I called all the time, I was like, “Oh, we have to be friends now,” and we’re still friends to this day, ten years later. But just building relationships with folks, the job that I have right now was actually through a referral from two conferences that I met this person at.

Shanea Leven: I know Angie through friends of friends, and so I think networking is important. You never know what kind of doors we can open for each other, what kind of opportunities, and the mistake that I made was trying to think that I could do everything alone, and you really can’t. It’s about building a community, as opposed to thinking about it as networking, like, I have to go and do this.

Shanea Leven: It’s about building a community and you being a part of a community, as well. ‘Cause we’re kind of all in this together, we’re all trying to move forward together.

Angie Chang: Definitely. Farnaz, have you found networking to be important in your career?

Farnaz Ronaghi: I know it is important, however, that’s why I’m staying quiet, because I cannot be the one recommending people to do networking, ’cause I am the worst at it. I’m actually the one who will sit behind her computer for as long as possible, the introverted of introverts. Don’t see me talking like this, we are behind a computer, just … we can’t see each other in person, and we’ll see. But I do agree that it is about–just going through the startup experience, founding a company, and six years into it, I actually even, I look back, and every single people that we raised money from, people who became our customers, those relationships that I basically had to build because of my work, for advancing the actual work that I was doing, not that I was at that–

Farnaz Ronaghi: They are the ones that I go back to for, “Oh, I want to look around, where should I look?” Or, I’m looking for a partner, I’m looking for an engineer, so it is very important to be able to build that network. However, many of us have challenges with the concept of going out and talking to what feels like strangers. I don’t have any medication to help with that. But I do recommend everyone to go out there and just practice it, because I was very, very introverted. Now, at least when I take tests, my extroversion is more than introversion. So, it changes and you start doing better.

Angie Chang: Rosie, do you have anything to say about networking?

Shanea Leven: I do. I think in general, it helps to have, it’s interesting, having sort of kind of bopped back and forth, I mean, over long periods of time, between the software industry and the film industry, building a network of people who, I guess, in essence, they are acquaintances. They’re people who you know from cocktail parties, in the film industry, or conferences in the tech industry, who you see at just these things, and you recognize them visually and you don’t necessarily have anything to give or take from them, but you see them and they’re familiar.

Shanea Leven: You do, one day, get to call them up and say, “Hey, do you know this person? I’m looking at their resume,” or, “Well, how’s it going over there?” You get to actually make that, pull that ticket. It’s a good thing to have. So far, it’s been where people say, “Hey, do you know this person? They seem to be connected to you, do you actually know them?” And I’ve helped people get jobs.

Angie Chang: Okay. All right. We’re gonna be wrapping this up, just, I think we’ll take one more question. I think we have a few more minutes. I think this one may be good for Farnaz. We have an attendee who’s found that it’s hard for former educators to move into edtech, as many of the roles require sales experience, or highly technical skills. Even hard to get PM and customer success roles, any advice?

Farnaz Ronaghi: Even customer success and PM roles?

Angie Chang: Yes.

Farnaz Ronaghi: Well, I think, first of all, try sending your resume my way, and then I think, my advice would be, it would be good to look for instructional design or product, or program management roles in smaller companies, in startups that are more willing to invest in people who may not have the track record of success, but who will be able to be there and put their heart and soul in owning the results. We actually do that, with people who are in our customer success and professional services team, they come right from being a teacher to being in a position of working with our customers directly and building programs for them.

Farnaz Ronaghi: It’s actually very interesting. Like, with a bit of onboarding, they do such amazing work. I have no doubt that it works. However, try smaller companies.

Angie Chang: Okay, one final question. This’ll be a fun one. What’s your advice for bouncing back when you do ask for more and it backfires? Like, what is your coping mechanism, self care, how do you get back up?

Farnaz Ronaghi: Well, I think, the thing is that first of all, I think when you want to ask for more, just ask it with data. Meaning, when you feel like you are underpaid, there are Glassdoor reviews that give you average salaries and top of the market, below of the market. There are job descriptions from all sorts of companies with salaries on them that you can use as a data point that, for example, you are underpaid.

Farnaz Ronaghi: Or if you are asking for resources, like just go with data that backs your story, and if you really have a point, but you get the push back, for no reason, what seems like no reason, I actually think in those situations it’s good to just get it out of your system, talking to friends, getting coaching and mentorship. If you actually reiterate the conversation with a mentor, with someone who has, not just a mentor, but someone who has asked for a salary change before, and you feel like you had the right, and you feel like they also agree with you, that your argument was good, your manager didn’t have anything good to say in response, I actually think that it is fair to not think of yourself as just a disposable resource that can be treated in any way.

Farnaz Ronaghi: Just go out there and interview and look for other jobs, and then bring them an offer. You know? Push as hard as you want to push, just as long as you are doing it logically, you have data, and you are sharing your experience with someone else, in the role.

Angie Chang: Does anyone else have last feedback for how to bounce back from a potential rejection?

Shanea Leven: I think Farnaz is totally right. Doing data, and having the conversation. I think that moving, or separating in your mind the emotion of the rejection from the actual negotiation, or the thing that you’re asking for, is something that I really struggled with. They’re two different things. Right? Like, maybe there’s a reason, maybe there isn’t a reason, or maybe there’s a reason that someone isn’t telling you. But that doesn’t necessarily mean that you shouldn’t deal with your emotion around it.

Shanea Leven: That could just be, for me, it’s talking to someone, it’s venting. It’s reminding myself that they didn’t reject me personally, but they rejected the thing that I proposed. Or, it’s maybe not … it’s a no right now, but not a no forever. Or, like Farnaz said, get another job. Just separating those things out is helpful.

Farnaz Ronaghi: You know, I have one more thing to add. You made such a good point. I personally, I find having cheerleaders in life, very, very helpful. I have one cheerleader. One best friend, who rarely tells me I’m wrong. So, whenever I’m going with my emotions high, so angry at the whole world, because of the unjustice that they just did to me, for whatever reason, he will tell me that I was right, that I’m smart, but I have to work on saying it better.

Farnaz Ronaghi: Or, “Why don’t we say it this way?” Like, I just empty the emotions somewhere, and we get to problem solving together. I think finding yourself a friend, a cheerleader would be very, very helpful to deal with the emotion, because it’s very hard not to feel the rejection. But you need to move on.

Rosie Sennett: Yeah, yeah. I think being really clear that your argument, if we take it out of the salary area, broaden it for a second. Being really clear that you know that whatever argument you’ve brought for this decision was delivered to the person you’re delivering it in a way that they can hear it. You’re gonna deliver it to that person, in a way they can understand. If it’s somebody who can understand an emotional argument, then go ahead and deliver that. But if it’s somebody who can not, do not deliver an emotional argument, because it will fall flat. If it’s someone who needs statistics, then show up with statistics. If it’s someone who rejects whatever it is outright, and you feel that you have presented it, you know that this is a logical argument and you’re being dismissed, then yeah, you have a bigger decision to make.

Rosie Sennett: If you’ve been waved off, that’s a whole ‘nother ball of wax. Don’t confuse that with your proposal being rejected. Two different things, really. I think that’s the clear thing, is that one thing is personal, one thing’s not and none of it, in the end, is really personal. As Shanea said, 90 percent of the time, that decision, especially if you’re an individual contributor asking for something that is directly affecting only you, you may have no idea how or why that decision was made, and it may never come to you. So, that information should never necessarily turn into us and them. It should never then go out and become announcement of us and them, that has to be processed, because that’ll mess with you, too. That was what I was thinking while everybody else was kind of talking.

Angie Chang: Okay.

Shanea Leven: I’d also read Crucial Conversations. It’s super helpful.

Rosie Sennett: Yes, yes. Very good, very, very good.

Shanea Leven: Very tactical of like, how to go about these things and some of the tactical things we’re talking about, splitting emotion and like, the actual execution of the conversation. Changed my whole life. I’d recommend starting there.

Rosie Sennett: Super cool.

Angie Chang: Thank you all for joining us today. I know it’s Friday. Thank you so much. We’re gonna be … just tweet at us ggxelevate and we are gonna be moving into our next session, so thank you so much for joining us. See you later.

Rosie Sennett: See you.

Farnaz Ronaghi: Thank you all, bye.

Rosie Sennett: Bye.

“A/B Testing Cheap & Easy with Open Source”: Dena Metili Mwangi with Sentry (Video + Transcript)

Dena Metili Mwangi / Software Engineer / Sentry
Sukrutha Bhadouria / CTO & Co-Founder / Girl Geek X


Sukrutha Bhadouria: Hi, Dena, how are you? You’re muted, so you need to unmute. So, hi everyone. I’m Sukrutha. A couple of housekeeping notes as we’ve been doing through the day. We’re recording all these sessions and we’re going to have the videos ready for you in a week. I saw some of you asking questions about the previous sessions. We will also have the slides for you to be able to view. Please share the fun that you’re having, whether it’s through the content, or selfies of your viewing party, or you watching at your desk on social media using the hashtag GGXelevate.

Sukrutha Bhadouria: We’re going to do Q and A at the end if we have the time. So, please post your questions. At the bottom, there is a button for Q and A. If we don’t have time for it, we’ll do it over social media, and I’m sure Dena would be willing to do that for us. Also, we have a job board on our website, So, please check it out.

Sukrutha Bhadouria: Now our next speaker is Dena. I’m super excited. She’s a software engineer at Sentry where she works on the growth team. Fun fact, she graduated from Hackbright where she learnt–did a ten week program studying Python. Before that she was a graduate from Duke University and was working as a research analyst at World Bank. Her talk today is about A/B Testing: Cheap and Easy With Open Source. And I’m sure everyone is excited to learn more about this.

Sukrutha Bhadouria: So, thank you so much, Dena, for taking the time.

Dena Mwangi: Hey, thank you so much. I’m going to go ahead and bring up my slides. Can you hear me okay?

Sukrutha Bhadouria: Yes, we can hear you.

Dena Mwangi: Excellent. Okay. Hi, guys. It’s so nice to be here with all of you. Happy International Women’s Day. Today we’re going be chatting a little bit about A/B testing and specifically how to do it cheap and easy with Open Source. You can find me on Twitter as Dena Mwangi, or on Linkedin. Feel free to connect.

Dena Mwangi: So, before I jump in, just a little bit about me. As was mentioned, I’m a software engineer at on the growth engineering team. I did go to boot camp, and that’s how I got into tech. I went to Hackbright and I think there are few Hackbright grads in the audience today. So, hi to all of you. I’m also a data enthusiast. I am into quantified training. So, I really like thinking about data and-

Sukrutha Bhadouria: Dena, sorry to interrupt you, they are soft on your volume. Can you speak up or move the mic over.

Dena Mwangi: Yes.

Sukrutha Bhadouria: You may need to start again.

Dena Mwangi: Okay.

Sukrutha Bhadouria: Yeah, this is better.

Dena Mwangi: This is better? Okay.

Sukrutha Bhadouria: Yeah.

Dena Mwangi: Thank you so much for the heads up.

Sukrutha Bhadouria: Thank you.

Dena Mwangi: Awesome.

Dena Mwangi: Yeah, so software engineer, bootcamp grad, studied economics. So, I really like thinking about the world in terms of data, which is how I ended up in the role that I’m in now. I also really like thinking about diversity, and inclusion, and how to do tech for good. So, I really liked the talk that we just had about AI. So, if you want to talk about any of those things, feel free to connect as well.

Dena Mwangi: Our agenda for today, we’re going to go through what and where is A/B testing? We’ll talk through the general MVP requirements, if you want to build your own. And then we’ll talk a little bit about PlanOut, which is an Open Source framework that you can use to help you out.

Dena Mwangi: So what is A/B testing? Simply put, it’s just a way of comparing two or more versions of a thing to determine which performs better. And the magic sauce that lets us do that is we are able to randomly assign samples of people to each variation and use statistical analysis to evaluate how legit our results are. If we do this correctly, we’re able to take the insights that we get from our small samples and say something meaningful about our larger population, which is what we’re really interested in. You’ll also hear this called split testing or bucket testing.

Dena Mwangi: Now, where is A/B testing? And the answer might freak you out. It’s everywhere. So, as you are using your applications, as you’re surfing the web, tons and tons of organizations are running A/B tests on us all the time. But, for the most part, it’s because they want to make sure that we’re getting the best out of their products that we can possibly get.

Dena Mwangi: So, one example of this is Netflix. So, while you’re Netflix and chilling, Netflix is running tons of experiments. One of these is what image they show when you’re surfing and trying to figure out what show you want to watch. So, they’ve played around with the title they show you, the image that they show you, and they run experiments to see which one gets the most clicks and which one ends up with more people watching it. The quick example of that is with a show that I love called Sense 8. If you haven’t seen it, you should.

Dena Mwangi: So, they ran this when they first had this show out. And this is just three of quite a few variations and buckets of this that they experimented with. So, you’ll notice that they’re playing around with the text, they’re playing around with the image that they’re showing, and they set this through all their markets. So, if you look at this, try and think about which one of these appeals to you the most. And, in the U.S., if you chose the middle one then you’re in the majority.

Dena Mwangi: So, most people in the U.S. ended up picking the middle one. So, most people who saw this ended up clicking on it and actually watching the show, which is what Netflix cares about. But as with A/B testing, you’ll find that, once you start digging into the data, there is often quite surprising insights to be found. So, while the middle one did the best in the U.S., all of these were winners in different markets. So, the last one won in Germany, the right one won in Brazil, and this actually tends to make a big difference.

Dena Mwangi: So, they saw, once they started running these experiments, a 20 to 30% lift in engagement with people clicking on these titles and actually watching the shows. So, you can make a difference.

Dena Mwangi: One more example for that, quick note, this stuff is hard. Computers are really hard. They do this with tons and tons of different shows and this is one where it kind of went awry. If you’ve seen Tidying Up with Marie Kondo, maybe this is the vibe you get, probably not. But this was a case where they kind of mismatched the image that they were showing in their tests.

Dena Mwangi: So, one other quick example of where A/B testing is is an example that’s kind of famous with Google where they weren’t quite sure which shade of blue they were going to use. And I think things like this are why A/B testing actually has a bad rap, because people think, really? Are we going to spend our time thinking about shades of blue? And actually, yeah, we are. Because this actually translated, by figuring out which one worked the best for their users, it translated to an increase of 200 million dollars in ad revenue. So, A/B testing can end up being quite profitable.

Dena Mwangi: So, if I’ve convinced you that perhaps A/B testing is something that could be useful for your organization, what do you do next? How do you even begin? So, let’s talk through some of the MVP requirements. Really, it boils down to two things. You want to think about how you’re going to bucket people and how you’re going to do it correctly. And the second thing you’re going to want to think about is your data, the data that you’re getting out, because you need to know which of your variations performs the best.

Dena Mwangi: So, for the first bit, you want to think about randomization. You’re going to be randomly assigning your users as they come through, but they’re going to come through your website multiple times, hopefully. And so, you want these randomization, these assignments to be deterministic and counting is hard. So, this is a nontrivial task. As you scale out your experiments, you’re also going to want to account for parallel or iterative experiments. So, if you have a user that is going to be exposed to multiple parts of your site, you want to be very intentional about what you’re showing them.

Dena Mwangi: As far as the data, you want to think about how you’re getting the data out for analysis so you can actually decide who wins. You want to think about how you’re linking it to your internal metrics. So, like with the Netflix example that we saw, they really care about people actually watching the show, and they really care about the people who are paying them, how much they’re paying them. So, you want to have a way of linking the success of your experiments to your internal metrics like activation and paid users. And you want to think about how are you going to be seeing this? What does your analysis look like? Do you need dashboards to make that easier for you and your team?

Dena Mwangi: When we thought about this, we had to make a decision between whether we were going to build something or whether we were going to buy something. And there’s pros and cons to both of these situations. So, with buying, of course, it costs money. That’s a downside. These can be pretty pricey. They run up to 40 to 60K sometimes. But, on the plus side, they’re almost ready out of the box. Bit of a negative is you have to do a little bit of extra work to link them to those internal metrics, and you have to also think about do you want to send all your sensitive information about your users out to a third party? If not, which you probably don’t want to do, how do you get that information from the experiments back and connected into your internal metrics?

Dena Mwangi: As far as building it, the downside is, well, you have to build it. So, you have to customize it to your exact use case, which is great, but that takes engineering resources, building and maintaining it. And again, counting is hard, so you have to think about how you’re going to be implementing that correctly and validating the results that you’re getting.

Dena Mwangi: So, when we thought about that at our institution, we decided to use Open Source for the first section, for the first problem of how to bucket people correctly. We don’t want to think about that. We figured if there was someone who has already done the work of implementing that, why reinvent the wheel? So, for the first part we used Open Source, but for the second part, we kind of had our data pipeline already in place. And so, we were able to leverage our existing infrastructure and just hook that into place.

Dena Mwangi: So, what did we use? We ended up using an Open Source framework called PlanOut. It’s Python-based and it’s from Facebook. It’s been around for a few years. So, it’s had various ports from other teams. For example, HubSpot built one for JavaScript. But the best thing about PlanOut, and the thing that really sold us is that it’s low entry but high ceiling. So, you get the bare minimum to get you started running experiments very quickly, but it’s extensible. So, you’re able to scale it out to lots of users, and you’re also able to have lots and lots of add ons.

Dena Mwangi: So, things that you get, you get random operators, you get deterministic assignments for your hashing, you get name spacing. And to do all this, it’s really simple. If you’re familiar with Python, you’re able to create new experiments simply by inheriting from a base experiment class and modifying the assignment logic.

Dena Mwangi: But, what you don’t get is you don’t get a GUI. So, everything is code based and every time you want to create a new experiment you have to write it out in code and write it out in Python. You also don’t get any post experiment analysis assets. So, the nice dashboards to help make your analysis life easier, those don’t really exist, and that’s something that you have to implement on your own.

Dena Mwangi: So, I find it best to learn about a new tool by walking through an experiment. So, we’re to walk through a really quick one with a pet adoption profile. So, suppose you had an app that was trying to get a pet adopted. Suppose it’s this guy. And you think that, if you play around with the image that you’re showing, we’ll be able to have more interest and more clicks on this lovely cat’s profile. You also want to have a blurb with it because why not? So, we’re going to have these two images and these two blurbs, which gives us four options that we’re experiment and randomly showing to our users as they flow through.

Dena Mwangi: If we wanted to run this with PlanOut and actually have an experiment up and running, this is pretty much all that it would take. Put some code, it’s always scary when you see code on your screen, but don’t fear. We’ll walk through it really quick. So, basically what this is is it just pulls from a simple experiment class from PlanOut, and it gives you all your random operators all in this one thing.

Dena Mwangi: What you have to do on your end is tell it the required rules of engagement. So, tell it what you’re trying to do, who you’re trying to experiment on. In this case it would be a user ID. Tell it what your varying. In this case we’re varying an image and a blurb. Tell it also how you want to vary this. And, in this case, we’re going to be using uniform choice. We don’t really care, 50/50 split with each one. And that’s all it really needs to know.

Dena Mwangi: But where does this actually go in your code? So, if you played around with Flask, for example, wherever it is that you’d be using this image and this blurb, regardless of what language you’re using, that’s where this would go. So, in this case, if you have a route, then you just throw in your assignments and you’re able to pull directly from them and put them into your template.

Dena Mwangi: But okay, so you did the thing, but where’s your data? For this, all you have to do is tell it how to do the logging in your setup. So, you tell it where you want to log all the things, what file you want to send it in, whether or not you have a data pipeline or not, you have this option of just throwing into JSON. So, as people are flowing through your website and seeing all the options that you’re showing them as you’re randomly assigning them into particular variation, all of this is getting logged and put into a JSON that looks like this that will make it easier for you to pull from it later on.

Dena Mwangi: And the important bit here is that you’re able to see what the image was that they were assigned and what the blurb was that they were setting. Also who they are and what time it was, but really these are the two main things that you care about. And that really is it. That’s your first experiment, and you’re ready to go forward and A/B test all the things.

Dena Mwangi: But, before you do, I will leave you with a few A/B testing sanity tips that we’ve learned on my team that have made our lives a lot easier. The first being, you really want to have well defined metrics of success before you start running your experiment. I think a lot of teams get really excited and they think, obviously, this is going to be great. I’m sure it will be a success, but they’re not very clear on what success looks like. So, before you run any experiment, be very clear to write this down and know what your metrics of success are.

Dena Mwangi: The second thing I would advise is to make sure that you’re doing all your experiments in small, measurable iterations instead of doing large sweeping changes. Sometimes this isn’t always possible. For example, if you have an experiment that’s being run that requires a lot of design or it’s very greenfield, then you might have to do a lot of front end work, a lot of front end cost work. But, for the most part, you really want to be doing this in small measurable iterations. That way you’re able to attribute what exactly changed to give you the lift that you might be seeing in your data. Otherwise, it gets very confusing. Was it the button color that you changed? Was it the language that you changed? It’s unclear. So, do small, measurable iterations.

Dena Mwangi: The last thing is, A/B testing is not a silver bullet. Data is one thing in your toolbox. It’s not the entire tool box. So, this really should inform your decisions. It shouldn’t be the one guiding light. So, if you see a lift in an experiment, for example, you really want to think about it and look at it in context of the whole picture of your application and what you’re trying to answer. So, with the Netflix one, for example, they could have said, oh okay. This one particular one won in the U.S. Let’s do this everywhere. But instead, they dug deep and they were able to desegregate and see that, actually, they had different winners in different markets. And they were able to leverage that information and go forth with that and be a bit more successful.

Dena Mwangi: Thank you so much for your time. It’s been so great chatting with you. Happy International Women’s Day. If you have any questions, I’m happy to answer them.

Sukrutha Bhadouria: Thank you, Dena. This was amazing. What a fun image at the end. All right. So, there are a few questions. So, let’s roll through them. Susan asks, do the logs include the end action that is the click event of user wanting to adopt the cat?

Dena Mwangi: That’s a great question, and no it doesn’t. So, this really logs the exposure. So, you would have to do the extra step, which is sometimes non trivial, of having to connect the exposure with the actual action of interest. So, what we do is we have lots of different analytics events. So, in addition to the exposure one, we think about what we want them to do, and what success looks like, and we log that as well separately. And, when we run the analysis, then we combine the two.

Sukrutha Bhadouria: Got it. How do you know what is a good sample size of data to test with?

Dena Mwangi: So, this actually, there’s equations that we run. It’s pretty standard like statistical modeling that you can just like put number in, figure out what power you want, figure out what level of statistical significance you want that you’re comfortable with, and play around with that. And that will spit out what sample size you should be going with, at the bare minimum.

Sukrutha Bhadouria: All right. And is there any project too small for A/B testing to be useful? For example, a small app in private data with only 20 users?

Dena Mwangi: Yes, unfortunately. So, with the statistical significance to be able to say something that’s truly meaningful, you would have to run the numbers and see like what the minimum number would be. But I think 20 would definitely be too small. You want something in the hundreds and up.

Sukrutha Bhadouria: Yeah, that makes sense. And finally, what did you find most challenging when you transitioned from bootcamp grad to working full time as an engineer?

Dena Mwangi: That’s a great question. That should be a talk all on its own. I think for me it was still like a steep learning curve, but it was getting really comfortable asking questions and asking one more question than you feel comfortable asking, and getting over that fear of being seen as not knowing enough or all the imposter syndrome things that come with being a bootcamp grad and being in your first tech role. I think, honestly, that was the biggest thing is just getting over that and saying, it’s fine. I just need to learn the things. So, I’m going to ask the questions.

Sukrutha Bhadouria: Thank you so much, Dena.

“Office Manager to CPO – Keynote”: Shawna Wolverton with Zendesk (Video + Transcript)

Shawna Wolverton / SVP, Product Management / Zendesk
Sukrutha Bhadouria / CTO & Co-Founder / Girl Geek X


Sukrutha Bhadouria: Hey, Shawna.

Shawna Wolverton: Hello.

Sukrutha Bhadouria: All right, so gentle reminder of a few things, so I’m Sukrutha. I’m CTO of Girl Geek X. We’re recording these videos. They’ll be available for you in a week. Post your viewing party, selfies of you watching this, and any other learnings that you have on social media with the hashtag GGXElevate. We’re going to do a Q and A at the end, if we have the time. So, use the Q and A button at the bottom to ask questions. If we don’t have time, we’ll answer the questions later tomorrow or later this week. So, please check out our job board on Yeah, that’s it.

Sukrutha Bhadouria: So, let’s enjoy Shawna’s talk. Shawna, Senior Vice President of Product Management at Zendesk. Before which she was the chief product officer at Planet. Prior to that, she spent 14 years at Salesforce, going from the first localization manager to growing into being the Senior Vice President of Platform Product. So, that was a great growth, and Shawna is going to be talking to us today about her growth from office manager to CPO in over a thousand steps. Thank you, Shawna, for making time for us. I’m super excited.

Shawna Wolverton: Thank you. Awesome. I will just jump in, since we’re running a little late on time.

Shawna Wolverton: Welcome, everyone. I hope you’re having a great day here. Thank you so much to the Girl Geek X crew for hosting today. I’m really honored to be here. And, as Sukrutha mentioned, I have had quite a career, not always the one I planned. It’s been an interesting odyssey, and I thought I would share a little of that with you.

Shawna Wolverton: And really, this is the only good place to start, because so much of my career really does come down to this. I think there is some myth out here that we can all– we’re self-made, we’ve worked hard, every accomplishment we have came–just sprung forth from our amazing intellect and crazy persistence. And I don’t want to discount that, but the universe is large and we are all very, very lucky to have been born in this place and time. And so many people have helped me along the way, and I’m incredibly grateful to them.

Shawna Wolverton: But really, when I think about career plans, we just heard a little from the good crew of Grand Rounds about planning. And so much of life is really a fantastic stochastic kind of adventure. And we can’t always get all of the steps right for how we want to get there. But, at the end, there’s some really great goals and great milestones that we get there.

Shawna Wolverton: So, in my lifetime, we had an entire industry come and go. We had entire things– dotcom dissolution, number one. We had a grand financial crisis. Entire industries are gone. So, we really have to think about agility. We do it a lot, in terms of how we do our work. But I think we sometimes kind of get a little locked in and forget that we wouldn’t make a solid waterfall five year plan to do anything else in our lives. And being agile in our careers is really critical.

Shawna Wolverton: And there’s no one right way to go, and sometimes things change. I spent the first 20 years of my life assuming I was going to be a physician. It turns out my university transcript and I had very, very different ideas about that future. And there was no product management Barbie set when I was a kid. And coming out of school with my fantastic degree in Russian studies and political science didn’t set me up for anything really obvious. And it took quite a bit of experimentation and curiosity. And I think that early curiosity is what has also kind of driven a whole bunch of my career. A strong desire to learn new things, and an absolute hatred of being bored has been probably the two biggest things that have driven my career to date.

Shawna Wolverton: And I think we think a lot about driving our careers. We hear about this all the time, right? What are you doing to drive your career? What are the activities that you’re doing? I think we get a little lost sometimes and lose the journey. I like car racing, maybe you don’t, but I thought this was a really amazing analogy, right? There is a fantastic race that goes from Paris to Dakar in Africa, and you have this amazing adventure. And you have a whole crew that comes with you, because you assume your car will break down, and you will go the wrong way, and it will take much longer than you anticipated. And it’s glorious. And the sort of alternative is this ring of never ending struggle.

Shawna Wolverton: And I think, when you think about your career and how you progress through it, kind of an adventure attitude and a fantastic kind of see what will happen is a great way to approach things and make sure that you’re not missing out on the amazing experiences that come sort of between those promotions. I think we sometimes put milestones in the ground about where we’re supposed to be at certain points in our lives. And, when we don’t get there, we can be really disappointed. And I always think that’s really unfortunate, because there is so much to learn along the way on these journeys.

Shawna Wolverton: My career was clearly not a straight line. I did start out as a localization project manager. You can see I did that job three times in my career. Sort of moving on from it, finding myself in a position where it was skills I needed to rely on to kind of go back into the job market when things had changed. I certainly didn’t expect to learn much that would help me in my career, taking that nine month apprenticeship as a handbag manufacturer with an Hermes trained designer. But, my goodness, did I learn a tremendous amount about human nature, about satisfying the wants and needs of customers in a way that I don’t think any other technology job would have given me.

Shawna Wolverton: And it was a tremendously long time between that first localization manager job and that SVP of Product manager job. I spent 14 years in the same job–or not in the same job, but with the same company, at Salesforce. And I think it’s another thing we think a lot about is, have I stayed too long? Have I stayed long enough? And I think that a lot of those thoughts are silly. If, like the previous speakers were saying, if you find the thing that matters and that gives meaning for you, then you keep going, and that’s what gets you up in the morning.

Shawna Wolverton: I like to tell people that product management is a job that by all measures is pretty terrible, right? If everything has gone swimmingly well, you’ve reflected that back out on your developers, your sales engineers, your sales team, your marketing team. And, if it goes absolutely sideways and the organization is at a loss, you stand up in front of the room and it’s all you. But what I found is that there was this fire in my gut that was about helping customers help their customers. And I couldn’t imagine doing anything else. And I think that love of my customers and that sort of obsession with helping them is the other sort of huge part that grew my success.

Shawna Wolverton: And then, I think this is a big thing that we don’t talk a lot about and that’s the–well, we do. We talk about it a lot. Who am I kidding? But having it all is a lot, right? You can have a family, and children, and a job and it’s not easy. But I think a lot of the things that we tell ourselves and a lot of the things that the media tells us about what it’s like to be a working mother are really bananas. There is this myth that we’re going to go out on maternity leave and we’re going to come back and we’re going to be distracted and we won’t be as good as we were. And there is this amazingly strong body of scientific research that’s happening about what happens when, not just women, but men and women come back from parental leave or after the birth of their children.

Shawna Wolverton: Amy Henderson, who is the founder of a company called Tendlab that really focuses a lot on sort of parenting in the workplace, has done this amazing research. And the women primarily that she spoke with, the senior women, all with hindsight, were able to look back and find an acceleration in their careers post childbearing. And I think, whether you have children or not is a totally personal choice. But I want everyone to know that it’s not, I think, the horrible, awful, career ending thing that we’ve thought it was for the longest time. And there is a fantastic thing that can happen. That little J curve was definitively, after my daughter was born. And, at 12, we’re having a fantastic time together in a way that allows me to be here and also there, and to be a fantastic role model for her about what it looks like to be a woman who is in the workforce.

Shawna Wolverton: And this one is important to me. It’s, success is not a pie, right? I think oftentimes we think for us to win or to get a promotion or to get ahead, someone else has to lose. And it’s not like there is this finite amount of success where we get our pieces of the pie, and we eat it, and then they’re all gone, right? It is really more of this sort of random, infiniteness that is more pi than pie. And so much of I got to where I am is about the people who put a hand out to me and who supported me in my career.

Shawna Wolverton: You heard from Leyla and Jen earlier. A huge part of my network of support in my career. And it’s incredibly important, I think, for us to think about how we turn around and put our hands up too for the next group of women coming up in the world.

Shawna Wolverton: So, yeah. That’s sort of my journey through my career. I think I went a little fast. I woke up this morning with a cold and I’m on a little cold medicine. So, maybe I’m going a little fast today, but it might leave us with a little more time for Q and A.

Sukrutha Bhadouria: All right. Shawna, that was amazing. My video feed is taking a bit of time. Hi.

Shawna Wolverton: Hi.

Sukrutha Bhadouria: You’re so awesome. I love the fact that you said that success does need to be shared, for sure. Sometimes we’re a little bit hard on each other, right? And we get a little competitive and we don’t help each other out enough. What has been, I guess, for you–I have a question. What has been, for you, the most fun role that you had while you transitioned over the years?

Shawna Wolverton: I think almost every job I’ve had I’ve found the fun. But I think I have a kind of warped sense of fun. Things that are really, really hard are fun. So, at Salesforce, we had a giant project to rewrite the entire front end of a 17-year-old software product. And it was an entire company motion, took me way out of my comfort zone, and it was hard. But, at the end, it was just so fantastically rewarding. And the fun part was really so much about the people and conscripting a sort of unwilling team onto team lightning, and then going out in the world and talking to customers about it.

Sukrutha Bhadouria: Yeah.

Sukrutha Bhadouria: There’s another question. What is a mistake you made in your journey that you could share with us?

Shawna Wolverton: I think one of the mistakes I made was thinking that I could go fast. Like there were times in my career I got a little ahead of my skis. Where I saw other people getting promoted, I let my ego get in the way, and, when I didn’t get a promotion I asked for, I took it really personally. I was devastated. And, in hindsight, I’ve realized that extra year spent in the job before I promoted was important, and I learned a lot. And I wasn’t ready when those things had happened. So, I think–yeah. Getting a little too attached to my own ego and letting some of that go.

Sukrutha Bhadouria: Yeah. It happens to all of us, I bet.

Sukrutha Bhadouria: Okay. We have another question that says, how did you get the opportunity to be a handbag apprentice for Hermes? How does one find additional opportunities like that, Shawna?

Shawna Wolverton: Strange things happen when you take City College classes for fun, and you meet interesting people, and when you’ve been laid off, and you have a little bit of severance, you can say yes to some things that you normally wouldn’t say yes to. I think I probably would have stayed longer, but my husband started sort of like, you know, it might be time, Shawna, this little adventure you’re on.

Sukrutha Bhadouria: So, what do you think kept you going when things were not going the way you meant for them to go? Like how do you keep yourself positive?

Shawna Wolverton: I mean, I think, for me, it’s about what’s always been important to me. And what I found when I became a product manager, and why I’ve never wanted to do any other jobs since I started is that connection to customers. And knowing that, even when things are really hard or things aren’t going great in the office, that the things that I do have impact on real human beings. And I was lucky enough, over the course of my career, to really get to know–I count a number of former customers as friends today. And getting to know how the things I did impacted their lives, and that really is the thing that keeps me going.

Sukrutha Bhadouria: All right.

Sukrutha Bhadouria: So, some more questions pouring in while I’m reading them out. So, what is the biggest leap mentally and leadership wise you think you experienced in moving from director, to VP, to SVP, to C-level?

Shawna Wolverton: Yeah. I mean, I think one of the craziest things is about–I mean, it’s funny, I talk about this a lot–and I noticed it the most when I became SVP, and that’s moving from what I call the person who was sort of, I had this brand as being the person who fought the man, right? I was speaking up for the customer, and I was really adamant, and I was pounding the table, and sort of advocating really hard, often against management. And then, I became the man. And it was a really interesting adjustment for me to understand that, often, there is a toll in this sort of, I disagree but I’m going to commit and go forward that is really required at an executive level. And that adjustment was probably the most interesting of my career.

Sukrutha Bhadouria: Wow. All right.

Sukrutha Bhadouria: Maybe last few questions. I’m curious because a little birdie told me you were best known for getting the customers to fall in love with you. What’s your secret?

Shawna Wolverton: Oh, listening. I think a lot of it was really active listening. And then, here’s the strange thing. I told them no. And I think sometimes we try to please customers, or bosses, or colleagues, and we say, oh yes, all the time. And then, we either can’t deliver, or can’t deliver in a timeline, or people make plans based on that yes. And sometimes a really, really clear, I’m not going to be able to do that is so much more powerful than our sometimes, like, extreme anxiety about wanting to be able to say yes.

Sukrutha Bhadouria: That’s amazing. I didn’t think of that at all.

Sukrutha Bhadouria: Just one last question, and then we should wrap. What was the pivotal moment that helped propel you into senior management roles?

Shawna Wolverton: Yeah. I mean, I think a lot of what I did to get propelled into senior management was taking on projects that no one else wanted to do. I’ve had the good fortune, I don’t know, to work in really fast growing companies where there was always more work on the ground than there were people to do it. And finding out which one of those things was actually really important, and then taking ownership of it, and showing management that I was the kind of person who could take on those hard things was a huge part of my career growth.

Sukrutha Bhadouria: Thank you, Shawna. This brings us to the end of your talk. Thank you everyone who posted questions and the amazing comments. By the way, you got a lot of love for your glasses.

Shawna Wolverton: Thanks.

Sukrutha Bhadouria: Thank you again.

Shawna Wolverton: Thanks, everyone. Have a good rest of your day.

Sukrutha Bhadouria: Thank you too. Bye.

Shawna Wolverton: Bye.

“Creating an AI for Social Good Program”: Anna Bethke with Intel (Video + Transcript)

Anna Bethke / Head of AI for Good / Intel
Angie Chang / CEO & Founder / Girl Geek X


Angie Chang: Hi, Anna.

Anna Bethke: Hello, how are you doing?

Angie Chang: Good. So we are back. I’m going to … there we go. Snazzy background. So we are recording the videos, this is a common question we get, and they will be available later on our website, Please tweet, the hashtag is GGXElevate.

Angie Chang: We’ve been sharing selfies of viewing parties, since it’s International Women’s Day, of women gathered, and allies, in rooms and offices, and coworking spaces around the world.

Angie Chang: Next up is Anna. She will be talking about how she developed the AI for Social Good program at Intel.

Anna Bethke: Cool, thank you. I’m super excited to be talking with everyone. I’m assuming my slides are showing, but let me know if that’s not the case. Awesome. I wanted to get into what does AI for Social Good mean, as well as what are some of the projects that we’ve been doing here at Intel, and things that I’ve seen elsewhere in the space, because it’s one that I’m super passionate about, and love.

Anna Bethke: But just first want to talk a little bit about how I got to where I am today. So I am from Colorado, and I also think of things in a geographic sense. Then I studied aerospace engineering at MIT out in Boston, and started on my career path as a geospatial data analyst in sorts, taking imagery from satellites, taking information from that, and then writing some algorithms to find different patterns of life, and anomalies.

Anna Bethke: Did something slightly different, but also geospatially related at Argonne National Labs, and that was in the Midwest, which was lovely, was really close to my husband’s family, but not quite the right fit for us. Moved over, and was doing a data science consulting type of gig at Lab 41, and landed at Intel doing data science work there.

Anna Bethke: So before I took up this role, I was primarily looking at natural language processing, deep learning techniques. How do we make these faster, what are the different things that we can do, what is the state of the art? It was very interesting, but I just have been so inspired by a lot of different projects, and I’ll talk about some of these groups a bit later.

Anna Bethke: Now I’ve been being a volunteer for Delta Analytics, as well as Data and Democracy, two groups that help pair you with some different projects that you can help a non-profit with, or help move the bar on what is important in the world.

Anna Bethke: That’s sort of how I define, and how AI for Social Good is defined, and it’s easiest to talk about specific projects than to say what this is, because AI is super nebulous. What is good is also something that has some subjectivity to it, but basically, the idea is how do we utilize AI hardware, and software, which is a lot of different techniques, and these technologies to really positively impact our world?

Anna Bethke: The thing that I find really promising and fascinating about this is that we can have a very large impact. This is a smorgasbord of some of the projects, and I’ll go into depth for a few of them. But there’s a lot of different verticals. Healthcare is a large one where we can start to be able to take these image segmentation networks, or object detection type of networks, and say, “Okay, well where is a potential tumor?” Or, “Where is a disease?”

Anna Bethke: “Is this something that looks benign or …” what’s the opposite? “Benign or …” sorry, I can’t say that word today. But basically, “Is this cancerous or not?” Taking these types of ideas from our research areas, and putting them into the field.

Anna Bethke: It’s very wide and varied. For earth or our different types of work there, we can do a lot of things, too. So ways to protect our natural resources, ways to protect, also, our man-made resources, so one of the projects was looking at restoring landmarks such as the Great Wall of China, or how do we map our structures and buildings so that we can have a better disaster response?

Anna Bethke: And then ourselves. How do we protect our kids against online threats, or physical threats? This is some work that the National Center for Missing & Exploited Children has been doing for years, and how can we help them with technology so that they can help protect and find potential perpetrators faster?

Anna Bethke: And then how do we help ourselves create these online communities that are better, so like preventing harassing text online, and even mitigating and stopping it.

Anna Bethke: That’s a high level. There’s more on the website, but the interesting thing is really once we get to dive into these projects. One of the ones that I think is really interesting, because it came from one of our software innovators. So these are basically entrepreneurials, individuals that are external to Intel, and they have these ideas of ways that can really help society, or these different projects, and there’s a link at the end of this presentation that you can get more information on this particular project.

Anna Bethke: Basically, this guy Peter Ma, he was looking at the issue that every minute a newborn dies from infection caused by lack of safe water or an unclean environment. This is worldwide, and it’s a very large issue. This was the World Health Organization, but the current systems that we have there are very expensive.

Anna Bethke: They require manual analysis, so you can’t just take your machine, and bring it from one village to another village, and that’s just not possible. But you want to make certain that the water everywhere in the community, and you’re going to need to be measuring this multiple times, because the water quality can change.

Anna Bethke: So what Peter did with some expertise help from Intel, is he built a convolutional neural network, basically. A computer vision model that is able to take a water sample, and using off the shelf products, as well as this Movidius Neural Compute Stick, which you can buy this commercially from a lot of different sites.

Anna Bethke: Don’t know if I have a link on it here. Oh yeah, if you go to the AI for Social Good website, then you can get to see more information on this product, and that has more information on how you can buy these NCS devices. They’re really low weight, you could actually see one in the image, I believe. That USB stick, basically.

Anna Bethke: He was able to build this entire prototype for less than $500, and now it’s even smaller, and less expensive, and it’s more than 95% accurate. So it might not be perfectly accurate, but it tells you a lot of information, and can really start to help communities know where their water is safe to drink, and where is problematic, which can greatly improve people’s lives.

Anna Bethke: Another really interesting project that I love is with a company called Resolve, and we’re building, with them, Trailguard AI, and the idea here is the camera that’s in the picture is this motion capture camera. Motion capture cameras are great. Scientists have been using them for a very long time to be able to monitor the health of animals, and where are animals located.

Anna Bethke: Park rangers are also starting to use this to be able to say, “Okay, when are there poachers in an area,” and how do we help this poaching epidemic, and really turn the tide on it? Because basically right now, National Geographic has identified that an elephant is poached every 15 minutes, or a rate of 3,5000 a year.

Anna Bethke: This presentation that I’m giving is about 15 minutes long, so in all probability, an elephant could’ve been killed during this, which is just really sad to think about. There’s not a lot of park rangers, it’s a massive area that they’re trying to cover, so what can we do more?

Anna Bethke: What we did with Resolve was embedded also the Movidius Vision Processing Unit, so this is the same chip that’s in that USB stick that was in the last project. But basically, we can run an object recognition network on the Edge, here.

Anna Bethke: Everything that is being processed is being done on this VPU, and basically what happens is an image is taken, that goes to the chip, the chip is able to run this CNN for object recognition, and in this case, we’re looking for people in particular, because this is what the park rangers are very interested in.

Anna Bethke: If there’s a person or a vehicle, then it’ll ping the park rangers, and this is really–

Anna Bethke: –Both reduce false alarms, about 75% of the images that the park rangers would’ve originally gotten, wouldn’t have anything in them. So basically, if a tree moves, then this camera goes off, because you want that to be very sensitive.

Anna Bethke: Now, by just sending the 25% that have a person or an animal, so I think people are only in about five or less percentage, depending on the camera, of course. You can greatly reduce the false alarms, as well as extend the battery life.

Anna Bethke: So we’re expecting these to last a year, which really helps, because then it’s harder for the poachers to find. If you’re always blazing a trail between all of these different cameras, then it’s pretty easy to figure out, a poacher can figure out where they are, and avoid them.

Anna Bethke: It’s really interesting. Something that we already have, both of these last two projects, as well as the next one, these technologies that we already have, like object detection is something that is getting more robust, that we’ve been using for a lot of different applications that we’ve been researching for a number of years now.

Anna Bethke: So how do we use it, though, for these really impactful purposes? One of the last projects I wanted to just mention before going into some of the stuff that I’ve learned while building up a program around these types of projects is the Wheelie. This is a project that we worked with a company called HOOBOX, and basically, they are robotics experts.

Anna Bethke: In the last example, Resolve are conservationist experts, so they bring deep knowledge about what the issue is, and we bring the technical expertise. So what HOOBOX saw, was there are a lot of people worldwide that are suffering from spinal cord injuries, and there’s more and more every day.

Anna Bethke: But the offerings for mobility devices can be expensive, complex, difficult to use, and there aren’t as many options as one would like. We developed the Wheelie 7 with them, and it lets users choose the most comfortable facial expressions to command their own wheelchair.

Anna Bethke: You can basically say, if I smile, go forward. If I raise my eyebrows, go backwards. If I open my mouth, go right. If I stick out my tongue, go left; things like that. This is really nice, because every person has different abilities, so by giving choices that extend the range of people that this’ll be really effective for.

Anna Bethke: And again, facial gesture recognition is built on a lot of deep learning applications that we have already looked at in-house, and so how do we apply it to this? What are the different things that we really need to think about in order to make sure that this product works for everybody in a very safe and reliable way, and that people’s privacy is protected, and all of the things that we really need to be considering while looking at this type of project.

Anna Bethke: This all of course is run on the wheelchair too, because you don’t want to be sending this information to the Cloud. That would take a long time, and if you are telling your wheelchair to stop, you want it to stop immediately. So this is run on the Intel NUC, it’s this little miniaturized PC with a customizable board. I think it’s four by four inches, so really small, can fit on the wheelchair, and it also doesn’t pull a lot of electricity, and the facial gestures are captured by this 3D RealSense camera, and that gives more information about the facial gesture than just a normal 2D camera.

Anna Bethke: Again, all these devices are things that we are commercially selling, which is great, because it’s things that are already built, and we can just improve them, make them better by seeing how they work in this new and different environment.

Anna Bethke: This project in particular was supported through a couple different projects. The Software Innovator Project as well, that was that CleanWater example that I gave a couple things ago, and our AI Builders Program. This one’s interesting, it’s sort of tuned for startup companies, and I’ll have a link for that one, too.

Anna Bethke: What does it really mean for me as being the head of this effort? I do a lot of different things. One is research and development, this is a slightly older picture of my cat, is a bit older, but I get to play with code, and do some literature research. That’s a little less now that the program is running, and getting a lot more interest in it, but it’s something that I try to carve out as well.

Anna Bethke: Sometimes I miss doing more of the technical work, but this is something that I am exceedingly passionate about, so I don’t mind not getting to code every day anymore. Another one is Connector. This was a breakfast ideation session last week, or a couple weeks ago, where I talked with a lot of different people like, “What can your companies do? What are the different ways that we can really just raise the bar, even just a little, with our technical expertise, with what our various organizations, or ourselves can offer?”

Anna Bethke: The last is Advocate. Talking at conferences, speaking to all y’all. One of the things I really hope to communicate is that this type of work is really important, and also really interesting and fun, and has a lot of very good business use cases that might not be as prevalent either.

Anna Bethke: I think a lot of us really want to do this type of work, because it’s the right thing to do, but there’s a lot of benefits too. It’s great for marketing, of course, as you could likely imagine, but it’s also really great for hiring and retention.

Anna Bethke: A lot of people want to be doing these types of projects, so the more that we offer them, the more that our companies can hire in this type of talent, and keep us all happy. Then the third for us specifically, being a hardware company, I think I eluded to it a bit, is that we really see a larger number of use cases of how can we apply technology, and then that helps us make certain that we are designing our technology in such a way that it is robust, and that there is a larger user base, basically.

Anna Bethke: So I’ve been learning all these different things along the way, but it’s interesting. There’s some other things too, so I wanted to share a few lessons. Asking for work that inspires you. The role that I’m in now didn’t exist, and I am so grateful for my manager, as well as the leadership here, that they’ve been really supportive in me taking on this position.

Anna Bethke: Helping me get the resources that I need, as well as helping to find what are the things that we can do now, in a couple months, where are the places that we really should be looking?

Anna Bethke: The second is that there are a lot of people who want to help, whether it’s helping plan meetings, whether it is doing the engineering work, being a contact coordinator. A lot of the projects that we have been developing are ones that one of my colleagues, or a colleague’s colleague has a friend who is doing this thing, and they are having this issue. Can we help out with that?

Anna Bethke: Those connections are wonderful. Low hanging fruit are wonderful as well. I say wonderful a lot. It’s very true. A lot of times we really try to go for, I think they’re called moon shots, but what is the coolest and the best thing that we could possibly do?

Anna Bethke: And while those are important too, there are things that we can do today, or in the next month, that are potentially quite easy for us to move the bar a little bit, but can have a really large impact in somebody’s life, or some animal’s life, the environment’s state.

Anna Bethke: Those are important to continue to look at, and to consider. But it’s also okay to say no, and this is one of the hardest things, and it’s really actually necessary to say no sometimes. I have been having to come to terms with the fact that I’m not able to do everything that I want to do, and we’re not able as an organization, or a company, to do everything, to help everybody, and so it’s sort of making sure along the way, that I’m preserving my own health and wellbeing, and sanity, and spending time with my cats, and my husband, and all of that at the same time, too.

Anna Bethke: And then who can help? Redirecting it to other resources, or saying, “I’m sorry. I can’t help at this moment, but maybe this person can.” Doing that redirect. But yeah, it’s hard. Then finally, you’re here for a reason. Whatever position you’re in, it’s awesome.

Anna Bethke: Imposter syndrome is one of my good friends now. I have definitely doubted myself along this way, when I was a data scientist, now as a head of a program, it crops up all the time. This is something that I remind myself of, and I think that we all should elevate each other.

Anna Bethke: I love communities like this, because I really feel like we do that. And then actually last, is that if you’re helping to debut hardware, there’s a high likelihood that they will take a picture of you holding the hardware.

Anna Bethke: This picture, I had no clue was going to be taken. This was right before the holidays, and I had just repainted my nails, and I had never painted them this bright and shiny, but I’ve come to terms with this too, and loving it.

Anna Bethke: I think it’s funny, so I have to laugh at myself a little bit, but I actually really like the super pink sparkly nails. How do you get started in this? Hopefully I have shared enough inspiration, and project examples. There’s more at AI for Social Good, at, and this really follows a similar process as most other projects, so getting your ideas, finding the partners.

Anna Bethke: So the partners are someone that has the ability to really implement this into action, and that really varies, and then getting your research together, the data, the compute. There’s some things that could help, like at this AI Developer Program, and that actually gives you links to both the Software Innovator Program, as well as our AI Academy, if you’re a student, but there’s some Dev Compute there, which could be helpful.

Anna Bethke: And then your algorithm development, so how am I actually going to analyze all this data, and make sense of the world? And then testing and deployment. This is really important, of course, to make sure that the system is working before it goes out into the wild. is the startup company connector, if that’s something that you’re in, and then one of the really important things as we’re doing this, is to talk about, and think about how do we do project management, project deployment in a responsible way, so there’s a bunch of different resources that are out there. There’s a lot of toolkits, so this goes everywhere from checklists like Deon from DrivenData, which just [inaudible 00:23:00]. These other things you should be considering as you go through, to more algorithmically based mechanisms like the IBMs 360 Fairness Toolkit, or the What If tool from Google.

Anna Bethke: Take a look if you are doing a project. Either for a socially impactful, or anything else, and there’s a large discussion around this right now, which I love. I’ll leave you all this sample of various social good volunteer organizations.

Anna Bethke: This is definitely a growing area of interest, so something in the Bay that I’ve been involved in is Delta Analytics. This is mostly San Francisco, but the other ones are either completely based in the United States everywhere, or also global, so DataKind, Data for Democracy, Code for America, Visualization for Good is really cool if you’re more on the visualization side, and then there’s a lot of different hackathons and challenges that you can join, too.

Anna Bethke: So yeah, that’s it.

Angie Chang: Thank you, Anna. This has been a great, very informative, resourceful talk. There’s been a lot of chatter and questions. I don’t know if we have time for … I’ll send you the questions, and you can maybe answer them on Twitter. I think you’re pretty active on Twitter, and we can get all the questions answered, with helpful links.

Angie Chang: Our next session will be starting soon, so thank you so much for joining us.

Anna Bethke: Thank you, yeah, I’ll definitely answer them there.

“The Gendered Project”: Omayeli Arenyeka with LinkedIn (Video + Transcript)

Omayeli Arenyeka / Software Engineer / LinkedIn
Gretchen DeKnikker / COO / Girl Geek X


Gretchen DeKnikker: Hey, everybody. Welcome back. Our next session here is with Omayeli Arenyeka. Arenyeka, tell me I’m saying it right.

Omayeli Arenyeka: Arenyeka.

Gretchen DeKnikker: All right.

Gretchen DeKnikker: This is important to get people’s names right. So she is a software engineer at Linkedin. She is also an artist and a poet from Nigeria. She submitted her talk to us through our speaker submissions on how she built a gendered dictionary and we thought it was so interesting that we invited her to come here and share it with you guys today, so …

Gretchen DeKnikker: Also, the videos will be available later. Don’t forget to tweet with hashtag, #GGXElevate. We’ve got the Q&A going in the bottom. And just after this session we will give away some more socks, so stay tuned.

Omayeli Arenyeka: That’s good?

Gretchen DeKnikker: Yep. I see you.

Omayeli Arenyeka: Okay.

Omayeli Arenyeka: I’m Yeli. Thank you so much for having me. My talk is about building a gender dictionary. But before we get into all the technical stuff I wanted to play a little word game. And it’s simple, you don’t have to do anything but think really hard.

Omayeli Arenyeka: So the game is, I say a word and you think of an image that’s associated with it.

Omayeli Arenyeka: Okay, here we go. Superhero. Ninja. Hacker. Rockstar.

Omayeli Arenyeka: And then now I want you to consider the images that came up, if they were of humans, whether those images were of a man or a woman or someone who doesn’t exist in those binaries. And this isn’t to shame anybody, it’s just an opportunity to reflect on biases because those biases we have they make their way into things that are supposed to be objective. So when given the option, translating from English to French, machine assisted language translation systems, like Google Translate, code the word nurse as feminine.

Omayeli Arenyeka: So in the Turkish language they use a gender-neutral pronoun that covers he, she, it. So when Google Translate goes from Turkish to English it has to decide whether the gender-neutral pronoun means he or she or it.

Omayeli Arenyeka: So this poem is written by Google Translate on the topic of gender, and is a result of translating Turkish sentences that use that gender neutral pronoun into English, so some of the lines are, he’s a teacher. He’s a soldier. She’s a teacher. He’s a doctor. She’s a nurse. She’s a nanny. He’s a painter. He’s an engineer. He’s a president. He’s an artist. He’s a lawyer.

Omayeli Arenyeka: And so, the algorithm in basing its translations on a huge corporate set of human language, so it’s reflecting the bias, a gender bias that already exists in the English language.

Omayeli Arenyeka: Another example of the effect of gendered language was highlighted be the augmented writing platform, Textio. They found that the gendered language in your job posting can predict a higher … can predict the gender of the person that you hire.

Omayeli Arenyeka: So thinking about this and other ways that our everyday gendered language communicates ideas we might not mean to, I decided I wanted to create something to allow [inaudible 00:03:45] for gender language, and that’s what this talk is about, Building a Gendered Dictionary.

Omayeli Arenyeka: So specifically, I wanted to make an API and a tool where you could find all the gendered words, you could find the equivalent of a gendered word. To be clear, what a gendered word is, they’re words that apply to a certain gender. So, lady/gentlemen, prince/princess, so some of them like lady and gentlemen, prince and princess, they have equivalents and some of them don’t. Some of them, like actor, are not gendered in definition but might be gendered in practice.

Omayeli Arenyeka: So the first question that I had to ask was, where and how do I get this data? So there are some existing data sets of gendered words. One of those examples is from a team of Boston … a team of researchers from Boston University and Microsoft Research. They created a data set that’s part of their work into removing the sexist biases that exist in corpus in data sets that train algorithms, like Google Translate. So they were trying to remove the bias from platforms like Google Translate.

Omayeli Arenyeka: But unfortunately, all the data sets I found, including that one, were not substantial enough. At most they had 1,000 words, and a lot of the words were false positives, so they weren’t actually gendered words. So I decided I would use these methods, API, static data, and web scraping to get the data.

Omayeli Arenyeka: So to start with, I had to determine what a gendered word was, so what I would I tell the computer that a gendered word was? So to start, it was all the words in the dictionary that have at least one of these terms in it, so woman, female, girl, lady, man, male, boy. For example, businessman has the word man in its definition and archeress has the word female, so both of them would count as gendered words.

Omayeli Arenyeka: Then I started looking for some APIs. So I found one of the largest … the biggest online English dictionary by number of words, Wordnik has an online API and it has a free … it has a reverse dictionary feature, which means find all the words that have one of those terms in their definition. So you can see on this screenshot, the reserve dictionary of woman is all the words in the dictionary that have the word woman in their definition. So airwoman would have the word woman in its definition, so it would count as a reverse dictionary term.

Omayeli Arenyeka: So Wordnik has a client for interacting with the APIs, so I just used that to make a call to their reverse dictionary. You can see that happening in line seven. I have all the terms and then I make the call to the Wordnik API in line 10.

Omayeli Arenyeka: So I got about 400 words back, which was kind of confusing because the API said that there were over 3,000 words that were … that had the word woman in their definition. So I had to find another data set, so I stored the 400 words I got from the Wordnik reverse dictionary API and then moved on to the second way of getting data, static data sets.

Omayeli Arenyeka: So I looked on GitHub and I found a dictionary in JSON format and I read that in using Python. So Python has a JSON module that you can just import. So I loaded that in for filtering and I got all the definitions of the word, as you can see on line six.

Omayeli Arenyeka: So, like I said, if a word has one of these terms in its definition, then it’s a gendered word. So how do we check that? With Python you can say, “If string in definition.” So if woman in definition, or female in definition, or lady in the definition, but then you have this long list of conditions. So instead of doing that we can use RegEx. So, for example, my name is Omayeli, but a lot of people often misspell it, so I could use RegEx to create one pattern that matches my name and all the misspelling of my name.

Omayeli Arenyeka: So I created a RegEx pattern for all of these terms, so I could search them in definitions and see if the word was a gendered word. So in RegEx the pipe symbol represents or, so this is saying match woman or female or girl. And then, if you find any of these strings … if you find any of these words in the string, you can see patterned at search definition, it’s searching the definition for one of those patterns. And if you find it in the definition, then we do something. But the issue with that is that it wasn’t looking for whole words, so sub-strings also count. You can see on the right the words that are matched, human, manhole, so these are not gendered words. They have the word man in them but it’s just a part of the word and not the full word.

Omayeli Arenyeka: So I had to use word boundaries, so word boundary allows you to perform a “whole words only” search. So now it’s looking for whole words and not just part of a word. So you can see on the right, it no longer matches manhole and manatee. It only matches man and boy at the bottom.

Omayeli Arenyeka: But then I also want words like grandfather, so what do I do? So word characters, which matches a word character. So anything from A to Z, zero to nine. So now it finds father and all the words that are combinations of father and another word in front.

Omayeli Arenyeka: So going step-by-step through the patterns, these parentheses are for grouping a pattern together as one. This character set says, “Match anything in this set.” So you don’t have to match all of them, but you just have to match one thing in that set. This is, like I said, matching a word character. This is matching a dash. And then this is saying it’s optional, so there can be something before the word but there doesn’t have to be.

Omayeli Arenyeka: And these are the final RegEx patterns. They’re pretty long. So after I finalized the pattern I went through the dictionary and for each entry in the dictionary, if the definition contained one of those terms then I added it to the list of gendered words. So in line eight it’s checking if any of those terms are in the definition, then we add that to our list of gendered words.

Omayeli Arenyeka: So when I add that together, the words from Wordnik and Webster and some other files, it came to about 8,000, which is great. Much more the 400 that I started with. But then when I went through the list there is words that did not belong there, words like lioness. So for my definition of what I wanted this gendered dictionary to be, it was a collection of gendered words for human beings, so not animals. So this was not a word that I wanted in my word set.

Omayeli Arenyeka: So instead … So I decided I would start to look for patterns in the incorrect words, so find … what were the common things in the definitions of the words that were not supposed to be in the set? So one of the patterns of incorrect words that I found was that in some of the incorrect words the definition included the gender term being used as the object of a preposition. So, for example, in the definition of waterfall it says, “An arrangement of a woman.” In the definition of Peter is says, “A common baptismal name for a man.” So it’s not a name that is describing a man. It’s a name for a man, so Peter shouldn’t be a gendered word. And you can see in the other definitions, “Short cape worn by woman,” or, “The position of a man.”

Omayeli Arenyeka: So how would I remove words that fit this category, and the category being the gendered word is being used as the object of a preposition? So first, I had to isolate the part of the string that I wanted to look at, and that was everything before the gendered word. So you can see, the highlighted portion is everything before the word … before and including the word man. So we can use … in Python, we can use RE module, which is for handling RegEx expressions. So the RE search method in line four will search through the text for any of those terms, any of our gender terms in line two. So in this case we have the string definition in line eight, so it’s looking for the word man and it will find the word man.

Omayeli Arenyeka: And then we get … When we get the location of where the word is, you get the end index. So in line nine you can see the search method in RegEx returns the index of where the word was found. So, from there we can get the end index and then we can use that to trim the string. So in line 11 you can see that the word, it’s now a common baptismal name for a man and it doesn’t include everything after.

Omayeli Arenyeka: So after we trim it, remove any punctuation, we use the string class in Python. The string class has a list of punctuations, so we use that to filter in line five and then we return the string without any of the punctuations. So if there was a punctuation it would remove it. So if there was a string in line seven, the return string would be line nine, so, which no punctuations.

Omayeli Arenyeka: So now that we have this trimmed definition we can use NLTK to find where the preposition in is the string. NLTK stands for Natural Language Toolkit. It’s used for processing the English language. So the first thing we do is we tokenize it, so tokenization is the process of chopping up a string into different pieces that are called tokens, and then throwing away certain characters like punctuation. So you can see on the right we pass in … This is an online version of a tokenizer. We pass in a common baptismal name for a man, and then it breaks it up into different tokens.

Omayeli Arenyeka: After we tokenize we can use something called a part of speech tagger, so I load in the part of speech tagger in line … in two, it’s part of NLTK. In line five I tokenize the string. So you can see, in line six, it has … the definition is chopped up into different pieces. And then in line eight we use the tokenizer from NLTK, which gives every token a part of speech. So you can see, common is an adjective, baptismal is an adjective, man is a noun. And I know those because I looked up what the tags were in NLTK. So you can see, NN represents a noun, so man is a noun. JJ represents adjectives, so baptismal is an adjective. And then after that, first of all, we remove the a’s and the and’s and the the’s because we don’t really care about them, so we remove them in line four and then we get … In line nine we get the word before the gendered word. So we know that the gendered word is the last word in the sentence. It’s the last word in the string, so we get the word before that, and then we check to see if that’s a preposition.

Omayeli Arenyeka: So in that case–this case, a baptismal name for a man, the word before man is for, which is a preposition so it returns false and says, “This is not a gendered word.”

Omayeli Arenyeka: Another pattern was that there were a lot of clothing items. So you can see skivvies, pajama, loose-fitting trousers, all of these are not gendered words. So I found a list of clothing items so I can remove any words that has one of these clothing items in the definition. Unfortunately, the website where I found the list, I had to apply for an API key. I did apply like six months ago and they didn’t get back to me, so I decided I would scrape their website. So web scraping is a tool for extracting information from websites that involves grabbing the html that makes up the website. And for doing this in Python there are two libraries I usually use, urllib.request and Beautiful Soup.

Omayeli Arenyeka: So the first thing you have to do is figure out how the data you want is structured in the dom, which you can do using the inspector tab of your browser. So we see that the data I want is in a link. That’s the child of a span element with a class TD. So I open the URL of the page in line two. In line three I add it to Beautiful Soup. In line four, and then in line five I look for the specific elements. So I’m trying to find links that are the children of spans with class TD. And then I get all the texts for them and I have the list of clothing items. So I use that to filter the dataset to remove any clothing items that are disguising themselves as gendered words.

Omayeli Arenyeka: And it came down to about 4,000 words. The last thing I wanted to do was find gender opposites, so I wanted to match words with their opposites, king/queen, father/mother. I can use that … I can do that using something called Word2Vec. So Word2Vec is an algorithm that transforms words into vectors. So back in 2013, a handful of researchers at Google set loose a neural net on a large corpus of about three million words taking from Google News texts. So the goal was to look for patterns in the way words appear next to each other. So you can see in the graph, microwave is close to refrigerator and it’s far from the word grass. Grass is close to garden, is close to hose and sprinkler. So the Google team discovered that it could represent these patterns between words using vectors and vector space. So words with similar meanings would occupy similar parts of the vector space and the relationships between words could be captured by simple vector algebra.

Omayeli Arenyeka: So these relationships are known as word embeddings and the dataset is called Word2Vec. It’s based on the idea that a word is characterized by the company it keeps, so a word is close to another word in space if they appear in the same context. For example, if we give the algorithm this text, since salt and seasoning appear within the same context, the model it creates will indicate that salt is conceptually closer to seasoning than, say, chair. And with that model, and with those word vectors we can do stuff like getting the similarity of words. You can see woman and rectangle are not very similar. Their similarity value is less than 0.1, whereas the similarity value for woman and wife is 0.8.

Omayeli Arenyeka: And word analogies, so you can do it for woman. You can do woman is to queen as man is to king.

Omayeli Arenyeka: In Python, if you wanna use models and the Word2Vec algorithm you can use a library called Gensim. So I mentioned earlier that they used … they set loose a neural net on a model of like three million words, so you can load that model of three million words into Python using Gensim. So it’s called Google News Vectors, so you can see in line two we have a model, Google News Vectors, and we load that in and then we have … we can call the models most similar method in order to get the equivalent of a word. And this isn’t perfect, but I think it works for most of my cases.

Omayeli Arenyeka: So in line nine we pass in woman, wife. So positive, woman is to wife, and then we pass in man, and then the results we get if the score is greater than 0.6 then we say it’s an equivalent. And so, we have …

Omayeli Arenyeka: And that was it. I got an initial word set using APIs and finding static dataset, and then I cleaned and filtered the dataset using RegEx, web scraping and NLTK, and then I used Word2Vec to find the equivalent for words that have them. And then I created a website and an API to house the data. So you can see, woman is to wife as man is to husband. So when navigating the site users can learn what words are specific to a gender, what words have gender equivalents, what words don’t, and which ones significant imbalances exist. And you can see what words have undergone semantic derogation, which is a process where [inaudible 00:20:25] take on more negative connotations. For example, the word mistress was more … was once the equivalent of the word master but over time it’s taking on a new meaning.

Omayeli Arenyeka: So in summary, the words that we don’t and do have matter. They reflect our biases and the ideas that we value [inaudible 00:20:42] risk reinforcing perpetrating those biases if we don’t [inaudible 00:20:42] the words we use and why.

Omayeli Arenyeka: Thank you.

Gretchen DeKnikker: Thank you so much, Omayeli. This was great. I wish you … Go back in and read the comments, because everyone was so excited about how you broke down this search methodology and the test you did at the beginning. Everyone was like, “Oh, I thought of a man, too.” So everyone really, really enjoyed it. Unfortunately, we don’t have time for Q&A and I know we’re missing those a little bit today, so don’t worry everybody. We have a list of the questions and we can go back and do more in-depth interviews with all of the speakers later. So your questions will get answered at some point. Thank you again.

“Tech Stayers & Leavers”: Lili Gangas with Kapor Center (Video + Transcript)

Lili Gangas / Chief Technology Community Officer / Kapor Center
Sukrutha Bhadouria / CTO & Co-Founder / Girl Geek X


Sukrutha Bhadouria: Hi, everyone. Hi, Lili.

Lili Gangas: Hello, hello.

Sukrutha Bhadouria: Thanks everyone for joining us, and also staying with us through the day. I want to just do a quick intro before we have Lili share her amazing wisdom. First I want to tell you some housekeeping notes. I’m Sukrutha, I’m the CTO of Girl Geek X. Housekeeping notes, yes, this is being recorded, the video will be available for you to view in a week. Please share all the information that you’re hearing that you’d like to share on social media with the hashtag #GGXElevate. I’ve been seeing a lot of you tweet your comments and questions there, so keep that going. We’ll have a Q&A at the end, so use the Q&A button at the bottom to post your questions, and we’ll make time for that.

Sukrutha Bhadouria: Also, our amazing sponsors have posted their job listings on our website, so you can go to to take a look at our job board. Now, for the amazing Lili. Lili Gangas is the Chief Technology Community Officer at Kapor Center. She helps catalyze Oakland’s emergence as a social impact hub, she advises inclusive tech entrepreneurship building activities in Oakland, such as Oakland Startup Network, Tech Hire Oakland, Latinx in Kapor Center, Innovation Labs as well. Lili’s also a proud immigrant from Bolivia, and her talk today is about Tech Stayers and Leavers. Thank you, Lili, for making time for us.

Lili Gangas: Thank you. Let me make sure you guys can … All you can hear me okay? Yes? Okay, awesome.

Sukrutha Bhadouria: Yes.

Lili Gangas: Well, hello, hello. I’m so excited to be here, this is amazing. I see a lot of all the different hundreds of women across the world, so let’s get started and jump right in. I’m gonna share my screen, and I have a presentation here. Let me make sure it all looks good on our end. Great. We should be able to see it now. Excellent.

Lili Gangas: Again, my name is Lili Gangas, I’m the Chief Tech Community Officer of the Kapor Center. Now, the Kapor Center, for those folks that may not be aware of what it is that we do, we really want to increase representation in tech and tech entrepreneurship. We want our communities to be representative of the demographics that we have. Our work is focused on US talent, and also US based companies, but we know that these companies in the community have a global impact, so our work really is trying to make sure that we are leveling the playing field.

Lili Gangas: Today, I’m here to share some of the problems that we’ve seen of why folks that have started in the tech careers leave, but then also, how could they stay, and do folks come back? Let’s jump right in and see what do the numbers show, what can we do about it, but before I do that, I wanted to share a little bit about me. As you heard earlier, I’m a proud immigrant from Bolivia. I’m not sure how many women here are from Latin America, but go Latin America. I immigrated to the US when I was about six and a half, and we take a look at all the stats that they shared with us about first gen, being the first to college, single parent home, all these different numbers that really say “You’re not supposed to be here.” But yet, thanks to my really just strong mother, fearless, and I’m just really blessed to have been able to have somebody like her just always push me, ever since … even from elementary school all the way to college and to pursue engineering, she’s always been the person behind me helping me make sure that I’m reaching the next goal, and also being able to make sure that I keep challenging myself.

Lili Gangas: This International Women’s Day, shout out to my mom, Sandra. Love you. I share that story because I think being able to come from a community that is very different than where I grew up, so in Bolivia, immigrating as a kid, you have the language barriers, you have the school, you have the cultural norms, it was challenging, but I think my love for math really gravitated me to math really being my language. That’s how I got started in my love for engineering. I went on to do electrical engineering. It started really my career in aerospace engineer. I think I’m gonna check to see if everybody can hear me okay. I think so. I started my career off electrical engineering, and specifically because I really wanted to solve problems that were meaty, that were big systems problems. I got to be able to work in a lot of satellite systems.

Lili Gangas: It was great, this was me going into tech and being able to just really nerd out in all the different types of technologies and teams, but I also started realizing that it was tough. I think as I started to manage teams, that’s where the team dynamic and the people dynamic became harder sometimes than the technical component. I think that that’s one of the topics I’ll go into a little bit more. Just so you have the real talk, what are some of those different issues that we all see? Those paper cuts that we also need to be aware of and how they’re impacting us, but also, what can we do about it? Now, in retrospect, from leaving engineering to then moving onto getting my MBA and really want to use technology for more social impact, and looking at social entrepreneurship is how I ended up here at the Kapor Center now.

Lili Gangas: After consulting with [inaudible 00:06:21], and the Excenture, and how we can use startups, public sector, private sector to really find new solutions and create new systems that are helping close gaps of access has been really what my career has pivoted to. I’ll share a little bit more about that as we go through some of the next slides. Lastly, I want to share also a little bit about me before, more personal. I love to run, and one of my long term goals has always been to run marathons, so in about three weeks, I’ll be running my third marathon, so I’m super excited. For all the women that are runners out there, keep at it, and for the folks who may not, I definitely encourage you, it’s a really great way to keep the body and the mind balanced. My last about me aspect is that I really started learning about meditation, and sometimes when we’re in different spaces, and sometimes we may be the only one that is like us in some of those spaces, or just balancing the difficulties that life brings, as a woman, as a professional, and just as a human. I think being able to find different ways to really allow ourselves to have that quiet time is something that I’ve learned really this past year, and it’s a blessing, it’s a gift that I wanted to share with you all.

Lili Gangas: Great. With that said, let’s get into what is really happening in the tech industry, and why are such talented people of color and women leaving? What is happening here? Just to give you a little context, in 2017, Kapor Center, along with the Ford Foundation and the Harris Poll conducted a study to specifically look at this, and over 2,000 tech leavers were surveyed, and the insights that I’ll share come out from that.

Lili Gangas: What did we want to study? We wanted to study what are the factors that are causing this turnover in the tech industry, specifically, what’s happening with underrepresented populations? Why are folks like me leaving? I left the tech industry to find different avenues, but there was also these paper cuts that I didn’t know they were paper cuts until actually I read the study, but also, what are some of the stories that we should be sharing so then that way folks feel heard and seen, and that we can do something that the policy inside the workplace can be done, but also, how do we provide support for those that we manage, or folks who are managing us? What are also some of the costs, and what are some of the practices that can be implemented to really change this culture around?

Lili Gangas: I’ll go to the next slide. For some background, the study was conducted over 2,000 professionals with this type of breakdown. This is a sample of the people that were surveyed. It’s a national representative sample, looking very focused on the intersectionality of the LGBTQ, the age, the race, gender, their previous role, the previous employers that they were in, and then over a set of 40 questions. Let’s get to the bad news. Some of this stuff might not be new for you all, it might be something that you may have lived. But this is the finding, and all those other professionals that took this survey found. We found that 37% of the surveyed professionals left because of unfairness, some kind of mistreatment in their role was really what turned them over to leave.

Lili Gangas: This is actually the highest reason why people leave, and it’s not rocket science to be able to say if you’re not treating me fairly, I’m not gonna stay. It just permeated across all the different groups as well. Specifically, underrepresented people of color were more likely to be stereotyped. Some surveyors responded that they were actually mistaken, if I was the only Latina, they were mistaken by the other Latina in the room. Little things like that really started adding up. Out of 30% out of those under represented women of color, they shared that they were actually passed, most likely passed for a promotion. LGBTQ also had some of the highest rates of bullying and hostility. One out of 10 women reported unwanted sexual attention and harassment.

Lili Gangas: Then, lastly, looking at some of these areas, some of the women reported others taking credit for their work, in addition to being passed over for a promotion, and sometimes even their ability was questioned at a much higher rate than men. The part that was interesting in all of the survey is that actually, white and Asian men and women reported observing a lot of these biases the highest, and they actually also attributed them leaving because of this reason. So, it’s not just impacting the under represented groups, it is really impacting the entire company. From that end, the key takeaways that the survey really showed is the unfairness that’s driving the turnover. The experiences differ dramatically across groups, and I think this is something that, especially since we have a community of global representation, to be very, very aware and mindful of that, that each group, even though you may not have felt it, some other group is feeling it, and it’s something to, when you’re working in cross global teams, but also very diverse teams, to be mindful of everybody’s experience as well.

Lili Gangas: The unfairness costs billions of dollars, but there is opportunity for this to change. We are able to find, and find ways to improve the culture if folks want to. I’m gonna go to the next slide.

Lili Gangas: Feel free, if you have any questions, feel free to add them into the Q&A, and I’ll jump into that after we’re done.

Lili Gangas: Great, I shared a lot of these problems, but what can we do? If you’re a C Suite at a tech company, or you’re a manager, there are ways that you can directly really help create a more level playing field for everybody in your workplace, and ultimately, women, we really just want to have equal pay. We cannot believe that we’re in 2019 and we still have issues that we’re still being underpaid. Specifically, Latinas in the US are significantly underpaid. They’re about 56 cents on the dollar compared to a white male. Second, improving company leadership is critical. Without having the C Suite, the CEO, but also the managers across the different angles being able to advocate and really create and put forward new policies. This is going to continue. We have to lead by example.

Lili Gangas: Again, promotion. This is an area where a lot of women that were surveyed, specifically here, expressed that this is why they were leaving, in addition to wanting to have a better work/life balance. If you’re not finding the opportunity internally, you’re going to leave, but sometimes if your job at the moment is providing you a great work/life flexibility, and it’s hard to make that change. Sometimes our careers start plateauing, but we have to be mindful that there are other opportunities and options. Ultimately, we just want to have a much more positive and respectful work environment. I know that here I’m preaching to the choir, because you all are probably feeling the same, but these are clear things, no matter the size of the company, that these are very doable, these are very trackable and measurable, and us, as being part of this industry, we have to make sure that we’re holding our leadership and our companies accountable.

Lili Gangas: Here we go, what can the companies continue to do, but what can you also make sure that your companies are doing? If you don’t see any of these aspects being done, this is the time to really start having these types of discussions across the chain, whether it’s with HR, or if you have a D&I, diversity and inclusion officer, or if you have your ERGs, but also starting to the high levels of the C Suite, being able to understand what does it mean to have a comprehensive D&I strategy? It really starts with that leadership, it has to be bold, unequivocal leadership from the CEO. We have to make sure that we are measuring the effectiveness of these strategies. One thing is to have it on paper, but the other part is to actually go and implement it, and measure it. Also just being honest if it’s working or not, which leads to the second opportunity for companies, and this is where we can all play a role, which is how do we create these inclusive cultures? What does this actually mean.

Lili Gangas: Make sure that your company has identified a set core of values. Make sure that there is a code of conduct. Ask if maybe you might be new to the company, and if somebody hasn’t directed you to a code of conduct or values, you should really ask, and that could actually also spur more discussions on this topic. Making sure that you’re always observing what’s getting implemented and if people are measuring it. For example, see if your company, and even if you’re a founder yourself, are you conducting employee surveys across the different experience that they might have, across the different levels? Are you doing it at regular intervals? Is your company doing these types of continuous studies? If not, maybe we should be bringing that up for discussion. Also, very, very important is to make sure that the data that is being collected and examined is intersectional. We want to make sure that you are giving voice to all the different groups by demographics, or the intersectionality or identity that otherwise wouldn’t be able to be shared out, and I think being able to have this intersectional lens needs to be intentional, and it needs to be measured regularly.

Lili Gangas: Ultimately, it’s really having a transparent culture about the issues that you’re having. There’s a really great resource for you all to check out that if you may not be aware, Project Include goes into these topics at even much more detail. There you could actually download and share some of this work with your team if you want to start having this discussion. Highly encourage you to do that. Lastly, developing an effective and fair management process. What does that mean? There’s actually a new sector of HR tech tools that are being developed that look at the people [inaudible 00:17:19] technology site where you could actually measure, let’s say through Asana or some kind of task management, who’s getting those types of tasks? Are women getting the technical tasks, or are they getting the more admin tasks? Who continually continues to get more work versus somebody else on the team? There are all these different types of tools now that exist that can help identify some of this bias that we may not be aware. It’s happening in the background, or some, for whatever reason, whoever is also managing a team may not even be aware of their own biases.

Lili Gangas: I definitely encourage you all to take a look at how are your teams being managed, how are you managing your teams, what type of technology are you leveraging to really be able to help create a much more fair process of managing the teams, the work, and being able to also audit what’s happening. Right now there’s a lot of, I think about two weeks ago, Google was in the news because they had their … my apologies … They released their compensation, and actually more white men ended up getting more raises than women. There’s questioning of even how was that invalidated? How was that managed? What were some of the processes? These are really, really tough discussions, but I think that we have to make sure that we also are being empowered. Without us, these companies won’t be able to continue to work. I think that the power of women in general, intersectionality and identity are different discussions that need to go from the talking to the doing, and to the implementing and measuring. That way we can start to do see of these toxic cultures really change. If you’re in a place where your company is actually thriving and doing really great in this side of the culture, protect it.

Lili Gangas: I think as you get bigger and larger, sometimes with the wrong hires, things can change. Being able to have this, being able to measure the things that are important, what you value, is critical. I encourage you all to do that. Lastly, again, this tech leaver study was released in 2017. There’s a lot of [inaudible 00:19:27], if you Google it, and you go to the Kapor Center Org website, you’ll see, you can go into more depth about the different insights. It’s a discussion that we need to continue to have. Hopefully I gave you a high level of what are some of the common pain points that you may also have felt but also how can we start to tackle them, so then that way we can continue to grow the talent and the leadership of women in tech.

Lili Gangas: With that said, I’ll see if there’s any more Q&A questions, but loved to have spent the time here with you all. Please feel free to contact me should you have any other more questions. You can find me on Twitter, connect with me on LinkedIn. There’s a lot of different other initiatives that I work on locally to help foster the tech talent to local tech talent as well as helping spur more entrepreneurship. We can have lots of different dialogues.

Sukrutha Bhadouria: Thank you so much, Lili. Your insight and your energy really, really made a difference, and there’s been great comments, especially because you said you meditate and you run marathons, I think that resonated really well with a lot of people. We have time for one question, just like a quick one. I’m gonna read it.

Sukrutha Bhadouria: I’ll summarize it, but there’s one that says “Can you share some tools that can be used to identify bias?”

Lili Gangas: Sure. Actually, in Kapor Capital there is different investments. One of the companies is called Compass. They are helping identify, using some data and AI, actually, in the backend to be able to measure the assignments of tasks. There’s also [inaudible 00:21:21] is another company that are we aware. Happy to share, I don’t want to blurt out names, but happy to share some links to a lot of different tools. When I did a talk last year on the pay gap, there was actually a huge increase of venture capital investments in this sector. There are so many different types of tools, different price points. Depending on the type of company that you are at, there’s definitely a lot. The question is, if you use it, how are you gonna measure the results? How are you going to do something about it? I think it’s understanding the what is it that you’re trying to tackle, which problem are you trying to tackle, and identifying the proper technology solution for that.

Sukrutha Bhadouria: Yeah, absolutely. Well, thank you so much, Lili. We’re going to end it here. Thank you to everyone who asked your questions and posted your comments, thanks. Bye.

Lili Gangas: Adios from Oakland, goodbye.

“Data Science And Climate Change”: Janet George with Western Digital (Video + Transcript)

Janet George / Chief Data Scientist / Western Digital
Gretchen DeKnikker / COO / Girl Geek X


Gretchen DeKnikker: All right. Welcome back everybody. We are here for the second section today. Darn it, I just cannot get the camera right. I’m going to turn into a millennial by the end of this, trying to get the right angle. Today we are recording these. You will be able to get access to them later. There is lots of chat activity going on if you want to hover over the chat button. Janet will have a Q and A session at the end of this, so use the Q and A button right there below. Hopefully you guys got some coffee and are ready, because this next talk is going to be amazing. Janet has 15 years of experience in big data, data science, working in labs, long before it was called big data, I’m sure, Janet has been rocking this. And she’s currently the Chief Data Scientist at Western Digital, here to talk to us today about data and climate change. Without further ado, Janet, please.

Janet George: Thank you. I’m going to start sharing my screen here in one minute. All right. Okay. Can you see my screen? Can everybody see my screen?

Gretchen DeKnikker: We can.

Janet George: Okay. Very good. I’m going to get started. I wanted to start out with a little bit of background about myself and how I came to be interested in climate change. Background, as she mentioned, I’m currently with Western Digital. I’ve also worked with companies like Apple and eBay and Yahoo in prior lifetimes. My educational background, I have a bachelor’s and a master’s degree in computer science with a focus on distributed computing, parallel processing, and specifically cognitive computing. My specialization is in artificial intelligence. I do a lot of stuff with CNN, convolution neural networks, RNN, which is recurrent neural networks, and also DNN, which now is deep neural networks, which has gained a lot of traction.

Janet George: How I came to do work around climate is related to my passion. And as some of you know, my passion is nature and sustainable ecosystems. I am a strong believer that we should leave the Earth better than we found it. I’m very interested in oceans, lakes, biodiversity, and really the preservation of natural habitats as we finish our journey on this Earth. With that, I’m actually going to talk a little bit about climate, climate data, and climate change.

Janet George: One of the questions, and I’m going to go through topics. How do we collect, normalize and parse data at the scale at which this data is available? And what data is really available around climate change? The data around climate change, we’ve got a lot of data around climate change. We have data that’s sitting in multiple data banks. This is historical data. We also have data that is found with the USGS. We have new data that’s coming up with sensors that are buried in the ground to watch our insects, to watch our birds, and to watch our plant population. We have lots of weather data from satellites and weather stations. We have atmospheric data, CO2 levels rising, heat waves, and things like that. We have very accurate data around sea level rise and precipitation and frost.

Janet George: More recently, we’ve also been getting data from National Geographic and other image data, which is actually quite new for us. This data’s coming–that’s from photographs that have been taken all around the world. And this gives us a very good idea about our shrinking glaciers. And the goal is to bring all this data, and so what I tend to do is we bring this data, we write agents, ingest agents, that can ingest this data into some sort of a data lake. And later, I’ll talk about what the size of this data lake should be and how large or small it should be. But the goal is to start uniting this data, because when you take data and you look at one dimension of the data, for example, if you’re only looking at insects, you may not get the whole picture.

Janet George: We want to get a full 360 degree view of our data with respect to insects, plants, heat waves, sea level rise, what’s happening to all different parts. And so the goal is to then bring this data into the data lake, so we have a unified mechanism to actually start looking at this data. And the data lake actually is an object store, so it’s a scalable object store. Then you build this data lake. You can just keep adding more notes to the data lake. Another advantage of having a separate lake versus a compute is the ability to allow storage to grow indefinitely and compute the grow indefinitely as the scale of data becomes much larger. And usually we start with the small scale, and then we can grow up to as big a scale as we want.

Janet George: The next topic I want to answer is around the focus. How do we focus? Climate change is so huge. It’s so big. It almost seems like, I call it the big, hairy, audacious goal. How do we tackle this big, hairy, audacious goal? And where do we focus? Should we look at the flora and the fauna? Should we look at weather, country, regions? There’s many, many variables that we can go after. And so what’s our focus area? Focus for me, because I come from a strong artificial intelligence machine learning background, I always look at the problem domain and form a hypothesis on what are the most critical variables that we need to watch for that directly informs us about predictions, or directly informs us about a metric that we can use to understand how we are doing in terms of forward progress or backward progress.

Janet George: If we see sea level rise, it’s a primary variable. And it’s somewhat of an independent variable, that is a very strong signal for many things that happen to climate change. So we spend a lot of time focusing on sea level rise and the consequence of the sea level rise and its direct impact for us. We also pick other variables like CO2. Now we have known that in countries, advanced countries, especially Europe, UK, many of these countries have focused on really taking action with reducing the carbon footprint, and have seen direct benefits. Those are some of the areas we want to focus here. Also, scientists are spending a lot of time trying to figure out how to harmonize carbon levels and how to make sure that we can reduce carbon levels by our actions. We know that these two variables are dependent on many things that are happening to us right now.

Janet George: For example, when the sea level rises, we see disappearing land. We see disappearing insect populations. We see tropical storms being much more severe than we’ve ever experienced in the past. We see our melting, ice melting. We see the loss of snow. We see drought that is persistent for years in a row. We see disappearing habitats. Heat waves take a different form. We experience heat waves like we’ve never experienced in generations prior. And we are seeing a lot of species, these invasive species that are surviving through these very high temperatures. And these are usually in the form of pests, which is not very good for the immune system of our habitat. They attack the immune system, the natural immune system of the habitat. Those are not good for us.

Janet George: And we are also seeing other things that are happening to us, like diminished plant population. And when we see that the plant population is diminishing, this has a direct effect on all of us because our healthy food sources disappear along with that. And so that’s something that we want to pay attention to.

Janet George: Next, the question was asked: What are my interesting discoveries around climate change? And so how would I ignore data that has a lot of false positives? And what have I discovered along the way? I think one of the biggest discoveries I’ve made as I’ve studied and looked into this data is that we have a lot of different species that are useful to us and help us along the way. Right? These species range from about 10 million to 14 million. And because of our history and where we came from and our infrastructure and compute, we only documented 1.2 million of these species that are captured. So this is a huge gap between what exists and what is actually captured.

Janet George: And today for the first time, we actually have this huge opportunity. We’re in this era where we can capture all of this. We can capture the current species. The problem with not being able to document these species, or being sporadically documenting these species, is the fact that we don’t understand how and when they become extinct. And when we don’t know the species and the rate at which it is reaching extinction, we are experiencing loss. And this loss is very severe. Now we can use big data and artificial intelligence. It is a right problem domain. We can use a lot of convolution neural networks. And I’ve been doing a lot of image analysis using convolution neural networks for insects, watching the different kinds and types of insects, classifying them, and also clustering the different species and documenting them so we can predict when they will be extinct, and the rate at which they are growing and why they are becoming extinct. What factors are contributing to their extinction and so on and so forth?

Janet George: One of the studies that have come up, and you can Google most of these studies, the one research paper that has come out that is very, very interesting is around the hyper alarming decrease in insect populations. Now you might know that insects are super critical. They’re a foundation for us in our plant economy. When we see 76% decrease in flying insects in just a matter of couple of years, we’re not talking a decade, we are not talking five years. We’re just talking year to year. That’s a crisis in our biodiversity. And there’s serious ramifications in habitat loss. Note that 35% of the world’s plant crops are pollinated by flying insects, so these are very, very important for crosspollination and maintaining the delicate balance of our natural ecosystems.

Janet George: What kind of infrastructure and what kind of investment is required? Is the problem so big that it cannot be tackled? Or is the problem bite size, and we can chew? And as a scientist, how do I come into the space? And what can I do, and where can I start? That is what I’m going to answer in this next slide. If you think about, we talked about the data lake, if you think about how economies of scale have allowed us to build very easy big data distributed computing stack, we can actually start very, very small. We can build on bare metal. We can use commodity hardware. There’s so much software that’s available to us, and AI algorithms that are available to us in open source. You can use Google’s inception for network. Or you can use Facebook’s PMASK CNN. You can use all of these technologies that are available to you. I am a big believer that you start small. And when you start small, you start with an investment of a couple million dollars. And based on how big your data becomes, if you have one petabyte of data, then you can do very well with a small compute infrastructure.

Janet George: And then you can grow out that compute infrastructure to as large as you want it to be. And that’s why the price tag is really based on the scale of data you want to process. But on the upper scale, if you think of processing all of those, like 14 million insect data, and much more than that, we’re not talking about a very large investment. We’re talking about up to 25 million in bare metal, compute and memory and storage, like data lake. This is not a very massive investment. Traditionally, building infrastructures with big companies and having a [inaudible 00:14:20] software that’s sitting in an IT department, organizations, enterprises tend to pay $50 million, to $100 million, to sometimes up to $200 million on infrastructure alone. We’re talking about $5 million, to $10 million, to $25 million dollars. And we can actually go at the problem and reverse the effect that is has on our ecosystem.

Janet George: For the first time, it’s a very doable problem. It’s something that can be attacked. Today, we don’t even get housing for a few million dollars here in the Silicon Valley. But we are able to actually create entire distributed big data computing stack with very, very small footprint. And so that allows us to do a very large amount of analysis, given the right compute and memory and storage.

Janet George: What are my lessons learned? Working with this data, working at scale, doing AI on insect images and trying to understand building prediction models on sea level rise, what I’ve learned is around the data collection and processing, we actually have to be very careful about how we collect the data. This is very important because there’s three components to what they’re trying to do. The first component of what they’re trying to do is the data itself. The second component is the infrastructure. And then the third component is: How do we actually take the models that we’ve built, and then how do we start to predict and use the predictions to make actual decisions for our future? My first learning around this is the KISS principle, which is really Keep It Simple. Get away from extract, transform, load, which are the traditional methods of loading a data and extracting the data.

Janet George: When we do the traditional methods, we actually cause loss of vital data signal, so we lose data in the process of trying to extract and transform. The best advice here is to really store raw data, and do the transformations for that data dynamically as you’re using the data, or learning from the data. I like to keep the data free from entanglements. And by entanglements, I mean schema. I don’t want to enforce a schema on the data because then you will have to spend a lot of time undoing the schema. You want a loose coupling with the format transformations. If you have a tight coupling, you will be in the business of trying to format and reformat data at scale, which will consume all the time and energy required, rather than do the actual analysis.

Janet George: We want to build a near real time processing capability, so what we’ve learned is when we have sensor data, and we are observing the plant, we cannot train on just old plant data. We’ve got to train on new real time data because we can see the plant behavior change. There’s a lot of variability in the data during the day when the plant is exposed to certain climatic conditions, or the plant is exposed to certain insect populations, and the plant starts wilting. Or if the weed is taking over, you can see how slowly the plant composition is changing. And in order for us to manage and monitor and learn, also train our machine learning models near real time, we want to be able to observe and train almost consistently and constantly.

Janet George: We want to assess the signal strength of the data at the time we are ingesting the data, not after we ingest the data. We have spent a lot of traditional time on trying to get on top of the data quality. And we want to try giving up controlling the data. We want to just work with the data in its natural form, so we try to understand the data as it comes to us, and especially at scale, petabyte scale. We’re not going to be able to control and manage all of the data quality. We just have to make sure that we have enough signals in the data that we can do the predictions with a great deal of accuracy.

Janet George: And my third most important learning is that when we build our infrastructure, we want to make sure that it’s future proofed, so that we don’t have to continuously keep rebuilding and re-architecting our infrastructure, rather, we simply add to our infrastructure as the scale of data grows, and also modernizing our platform and our technologies so that we can be ready for the amount of data, so when we go from one petabyte of data to 30 petabytes of data, we simply add compute and storage notes. But we don’t re-architect our infrastructure. Rather, we spend time on understanding the actual effects of the data.

Gretchen DeKnikker: Great. That was amazing, Janet. Thank you so much. [crosstalk 00:19:30]

Janet George: Key takeaway slide, one key takeaway I think is around how all of us can help transform the impact of climate in our daily lives. We are irrevocably connected as humans and Earth, and we can do our share.

Gretchen DeKnikker: Awesome. Thank you so much, Janet. Actually, we have time for one quick question. I don’t know how quick this question is, but we’ll give it a shot. Can you elaborate on the dynamic schemas? And do you have any advice on how to manage them?

Janet George: Yes. There is actually, within the Hadoop Ecosystem Stack, there is Avro, and Avro is a dynamic format. You can use Avro and you can do schema on read or write, so you don’t have to enforce a schema. You can do the schema as you’re trying to analyze the data.

Gretchen DeKnikker: Amazing. Okay. Thank you so much for your time today, and this wonderful, timely topic.

Janet George: Thank you. And I appreciate the interest very much.

Gretchen DeKnikker: All right. Thanks, Janet. Bye bye.

Janet George: Bye.