Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

• Check out open jobs at our trusted partner companies.
• Watch all ELEVATE 2023 conference video replays!
• Does your company want to sponsor a Girl Geek Dinner? Talk to us!

Sukrutha Bhadouria: Hi everybody. Welcome to this session. As you know, AI is such a hot topic and has been for many years. It’s gotten into an even hotter, hotter topic now. So I’m very, very excited to have Dr. Tonya here for us today. Wanted to remind everybody to share on social media, Twitter, Instagram, all the amazing learnings that you will have today. Don’t forget to also post your questions in the chat. Yeah, so anyway, <laugh> Sorry about that, the first part of the confusion that I had with my tech technical difficulties. I’m gonna go jump right into the intro.

Sukrutha Bhadouria: Dr. Tonya Custis is the Director of AI Research at Autodesk. She leads the AI research team at Autodesk that conducts the fundamental and applied research in AI and machine learning with the goal of unlocking a new era of AI-powered design tools. We’re so excited to hear her speak today and to hear about how roller derby has played a part in both her life and career. Welcome, Dr. Tonya.

Dr. Tonya Custis: Thank you so much. Hi everybody. It’s such an honor to have been asked to speak to you today. I’m really excited to be here. First off, in the spirit of International Women’s Day, I would really like to start by thanking and giving a shout out to all the strong women in my life from friends that I’ve had since kindergarten, academic advisors, roller derby teammates, to all of the brilliant supportive women in tech that I have and continue to encounter every single day in my career.

Dr. Tonya Custis: How is leading a roller derby team similar to leading a team of AI research scientists? Today I’m gonna answer this question by talking about my current position and the work I do as the director of AI research at Autodesk and about how I ended up there. Although I am not sure I would’ve realize it until much later, I guess the first step in my journey to getting here was learning to program in Logo in second grade, and then teaching myself BASIC on my Atari 400 computer when I was in fourth grade. But fast forward a few years, and the next step was obviously getting my college degree in music clarinet performance, and music theory followed by graduate degrees in linguistics and computer science.

Dr. Tonya Custis: Now, you might ask yourself, what do music, linguistics, and computer science have in common? At the time, I’m not actually sure, I could have told you. I maybe would’ve told you that it was very unlikely I was gonna get a job as one of two clarinets in any major symphony orchestra, which basically only opened when someone dies. I decided to go to grad school for linguistics. I was really interested in how the brain processes language and specifically how children learn language. However, as I advanced my linguistics grad school career, it also became clear that like the clarinet gig, in order to get a job as the linguistics professor, I would probably have to wait for someone to retire, and when they did, have to be willing to move to wherever that opening was. What I haven’t mentioned that is that in grad school, I was also married with two small children. They were so cute. Just picking up and moving to wherever was not really an option, giving my family situation.

Dr. Tonya Custis: Re-enter computer programming, I love doing it. As a kid, I’d taken some programming classes in high school and in college, but I had never really seriously considered getting a degree in it or making a career of it. Until my interests in how children learn language expanded to how can I get a computer to learn language? And I guess that’s when it all came together. What do music linguistics and computer science all have in common? Recognizing, producing, and leveraging patterns. I was able to take the skills from each discipline and creatively apply them to the others. I finished grad school around the time when Google and other search engines were really starting to take off. Although there were no actual computational linguistics or natural language processing degrees at the time, the coursework I did that combined linguistics and computer science put me in the right place at the right time.

Dr. Tonya Custis: My first job as a research scientist gave me opportunities to do machine learning research on information retrieval and natural language processing. It was my first real job and there were a lot of rules to learn. I needed to figure out how my research and my work fit into the bigger picture of the team I was on, but one thing that remained constant between academia and being an industrial research lab is that research means failing a lot. It’s part of the job, and it’s also part of the job to then pick yourself up and learn from those failures going forward.

Dr. Tonya Custis: A couple of years into my career as a research scientist, I went to my very first roller derby bout. As soon as I saw it, I knew I had to play. I immediately bought skates and gear and found out when the next tryouts were. Here’s me at tryouts. I tried out and I became a member of the league. But much like getting my first job, I now had to figure out what the rules were and how I fit into the team, and so became began the period of my life where I was an AI researcher by day, and a roller derby player by night.

Dr. Tonya Custis: I played roller derby, competitive roller derby, for 10 years and during that time I had the privilege of playing with skaters of all ages, all body types, and from all walks of life. The Women’s Flat Track Roller Derby Association (WFTDA) serves as the international governing and sanctioning body for bouts, rankings and tournaments. WFTDA currently has about 450 member leagues spread across six continents. There’s even a roller derby World Cup. The rules of roller derby are complex. They involve a lot of safety measures, where you can and can’t hit people, and with what parts of your body you can and can’t hit them with. But here’s a real high level description of how roller derby works. There are five players on each team, and the players skate counterclockwise on the track. Each team has four blockers and one point scorer, the jammer. The jammer is the one with the star on her helmet, and the jammer earns points earns a point for each opposing blocker she laps on the track.

Dr. Tonya Custis: What this boils down to is you have to play offense and defense at the same time. You have to prevent the opposing jammer from passing you and your teammates, while also helping your jammer through the pack to pass the players from the opposing team.

Dr. Tonya Custis: The most effective way to slow down, or stop players from the other team, is to knock them over, or to push them off the track. Roller derby is a full contact, physical sport. In addition to countless bruises, I’ve had a concussion, a broken leg, and a broken collarbone. This picture actually shows my collarbone being broken. But <laugh>, I have really never had so much fun in my life. I learned so much from playing roller derby, but maybe the best lesson was sort of like research. It doesn’t matter how many times you fall down, as long as you get up again.

Dr. Tonya Custis: A year into my roller derby career, I became captain of my team. I really did not want to be captain of my team, but somehow I got elected anyway. Being captain meant making decisions about people’s playing time, art training program, and the overall strategy. It meant being a player and being a coach at the same time. I was a reluctant leader, but I approached it as a puzzle, as an exercise in recognizing, leveraging, and optimizing patterns, skills that I was able to transfer from the other areas of my life. I learned that none of us had the same strengths or weaknesses, but we could strengthen each other on the track, and mitigate each other’s weaknesses by playing to our strengths, and as long as we worked together and trusted each other. Our team, which had never won a game when I joined the league, ended up winning several season championships in a row.

Dr. Tonya Custis: A few years later, when I was offered the job of research director, I was once again a reluctant leader. I loved doing research, I loved being a player on the research team, and I didn’t know if I really wanted to be a coach. But oddly enough, I was able to creatively transfer my roller derby leadership experience to my new role, which I again approached as an exercise in recognizing, leveraging, and optimizing patterns – and also one in playing offense and defense at the same time! What I’ve found is that being a leader means protecting your team, so they can concentrate on and have the space to do their jobs, so this is playing defense, but it also means playing offense by proactively getting your people opportunities for career growth and development, and making sure they have the resources they need to do their jobs.

Dr. Tonya Custis: So to be honest, leading a team of research scientists isn’t that different than leading a roller derby team. Everyone’s different, we have different strengths, different weaknesses, and not everyone plays the same position, but we all do better when we learn from each other and trust each other, working as a team to solve difficult and interesting research problems.

Dr. Tonya Custis: Let’s fast forward, back to my current role as the director of AI research at Autodesk, I lead a global team of AI research scientists and we work on machine learning research primarily in the 2D and 3D geometry space. So you might be asking yourself why someone with a heavy background in NLP and linguistics is leading a geometry-focused AI research team, and honestly I asked myself the exact same question when I was offered this job. What drew me to the design space was the opportunity to join language and geometry, to use language to communicate the semantics of geometry, in order to make design more intuitive. I wanted to take my background in NLP and use it to creatively solve problems in a new domain. Increasingly, NLP techniques, like large language models are finding more and more success when being applied in new ways and to new modalities.

Dr. Tonya Custis: The recent explosion of generative AI models like ChatGPT, Codex, Dall-E and Stable Diffusion is especially interesting in the context of design software. Autodesk, if you don’t know, creates design and make software. We serve the architecture, engineering and construction manufacturing and media and entertainment industries. We help our customers design and make everything, from smart cars to skyscrapers, and from bridges to the visual effects in films and video games. So if you’ve ever been in a building, driven a car, used a computer, sat in a chair, or seen a movie, you’ve likely experienced the designs our customers have made with Autodesk software.

Dr. Tonya Custis: My team’s research has three main areas of focus, and a big chunk of the work we do is in publishing papers at top conferences, often with academic labs. I’m briefly gonna outline some of our recent papers to give you a flavor of what we’re working on. In JoinABLe, which is work we did with MIT, we used machine learning to learn a bottom-up approach to parts assembly. So given a set of parts, can a computer learn to assemble them? To do this, the computer needs to be able to reason about not only the shapes of the parts, but also how they might be joined together. And as it turns out, this is a task people can only do correctly 80% of the time, so the graph shows the experimental results for JoinABLe with blue at 78% accuracy, against other state-of-the-art machine learning methods in green, and also, human performance in pink.

Dr. Tonya Custis: In the UNIST paper work with Simon Fraser University, we investigate using machine learning for style transfer. So for example, you might want a chair that matches the style of a table. This algorithm, given the 3D representation of a table, can generate a chair that matches it in style. Although style is somewhat subjective, it can be learned by neural networks, and the ability to transfer style can save designers a lot of time.

Dr. Tonya Custis: Let’s pause here for a second and talk a little bit about generative AI. I’ve been doing research on language modeling and computational syntax for the past 15 years. ChatGPT and other large language models are an incredible achievement towards a lot of the things I have been working on for my whole career. And advances in multi-modal generative AI models like DALL·E 2, Stable Diffusion, and Midjourney point to the desire to translate our ideas between language and other modalities.

Dr. Tonya Custis: Linguistic generalization and the ability to generate novel human-like language is a huge milestone for AI. Large language models are capable of generating strings of words that no human has ever said before. This is the power of linguistic productivity, using a finite number of grammatical structures to produce an infinite number of utterances. However, we do need to remember in the hype of all this, that large language models are trained to optimize for linguistic structure. There is no intent, no information, no reasoning, nor knowledge behind the words being strung together.

Dr. Tonya Custis: Any meaning perceived in the string of words supplied by the computer is, the meaning is supplied by the user. Said differently, there’s no actual creativity behind how large language models generate language, just probabilities. Nonetheless, one of the criticisms of large language models like ChatGPT, is that they make up and hallucinate facts. These models again we’re trained to generate syntax, not meaning and not facts. Having trained on over billions of word tokens, the text they generate puts words together that are related enough and seem plausible enough that they might sound like facts, but often they aren’t.

Dr. Tonya Custis: Why and how is this relevant to the design space? My team has been working on research in which we apply large language model techniques to 2D and 3D data. What if we could learn a grammar for how sketches and 3D models are generated by humans, and then mimic that grammatical generation with an algorithm? In the design space, having the algorithm hallucinate new designs is actually an advantage, not a liability. In this paper, we proposed two transformer based generative models that generate engineering sketches. This paper shows the ability to tokenize engineering sketches and represent them in a transformer architecture showing that it was not only possible but a promising research direction. Other works we’ve published that use transformer models for sketch and 3D generation are SkexGen and SolidGen.

Dr. Tonya Custis: In both cases, we are trying to generate designs that will be helpful to the user, which means they have to walk the line between being realistic and manufacturable, but also novel, diverse, and inspiring. SkexGen uses sequences of CAD operations to train a generative transformer model. Here we can see how the CAD operations in their order are broken down into tokens, or words, so that the transformer model can learn the structure, or grammar, of how the operations go together to create an entire 3D model, or sentence, if you will. SolidGen is also a generative 3D model, but instead of learning from sequences of operations, it learns a grammar of the 3D structures directly, by predicting the vertices edges and faces that make up an object, using transformer based and pointer neural networks.

Dr. Tonya Custis: These three papers are all great examples of how we can transfer and apply the ideas and mechanics of large language models to other types of data. Language is how we describe the 3D world around us, and we’re also working on multimodal generative models that allow designers to use language to communicate their designs, design intent, and specifications to computers. There has been quite a bit of progress recently in text to images, but similar progress in text to 3D shapes has leg behind because there isn’t a lot of data that pairs text to shapes.

Dr. Tonya Custis: CLIP-Forge proposes a machine learn geometry sensitive method for generating 3D models from natural language, so a prompt for a round chair generates a 3D box model of a round chair as opposed to a square. One CLIP-Sculptor is the newest paper from our lab recently accepted at this year’s upcoming computer vision and pattern recognition conference. It improves on the results of CLIP-Forge and other previous state-of-the-art methods, both in terms of diversity and accuracy, but also in speed of generation. Language is a tool we all share, and right now we’re starting to be able to use language to help shape and build the world around us in an actual physical sense. It’s really inspiring to me to think about how these technologies will evolve to support designers who use Autodesk software.

Dr. Tonya Custis: I have come a long way from almost no one understanding why I would possibly be interested in combining linguistics in computer science, and from when no one understood what my job was or why I was doing it, and so it’s incredibly validating to have reached this place where computers can recognize, leverage, and complete patterns at such a large scale that they are useful to people in so many different ways. What I want you to take away from this talk today is that technology is always changing. Life is always changing, but your strengths and the skills you’ve learned in one place are often transferrable to new situations. In my experience, leadership requires you to recognize patterns and creatively transfer what you know from one place to new situations, so that you can play offense and defense at the same time, so that your team has the space they need to on the track and the trust, confidence ,and resources they need to do their job successfully.

Dr. Tonya Custis: The ability to take what we’ve learned from one place in life and creatively apply it to new situations is what makes us human. Be curious, learn and discover. Fall down and fail. But remember, in roller derby and life, it really doesn’t matter how many times you fail, as long as you get up again. Thanks!

Sukrutha Bhadouria: Thank you so much. This talk really resonated with a lot of our attendees. We see some amazing comments. We are at time now. Thank you all, everyone, and thank you so much Dr. Castes for making time for us today.

Dr. Tonya Custis: Thank you all.

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

• Check out open jobs at our trusted partner companies.
• Watch all ELEVATE 2023 conference video replays!
• Does your company want to sponsor a Girl Geek Dinner? Talk to us!

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

• Check out open jobs at our trusted partner companies.
• Watch all ELEVATE 2023 conference video replays!
• Does your company want to sponsor a Girl Geek Dinner? Talk to us!

Angie Chang: Claudia Natasia is a director of product insights at Highspot and she’s exploring new ways to expand the impact research beyond product and to the business ecosystem. She’s passionate about improving accessibility of financial literacy and is excited to talk about quantifying impact. She also advises startups on growth and revenue. I’m really excited to hear her speak today about her topic. Welcome, Claudia.

Claudia Natasia: Thank you Angie, and welcome to my talk everyone. Happy International Women’s Day. I’ll get started. I grew up in Jakarta and this is a photo of the financial district of Jakarta, Indonesia. And I would like to say that I actually grew up in the financial district. My mom would pull me out of school sometimes and take me to business meetings that she has in different banks in the financial district. And sometimes I had the pleasure of actually being in the meeting room and watching her close deals. And what I learned from being in this environment from early on, even at the age of five, is that it’s important to build businesses and enterprises that last. And as I transitioned my career into many different things, starting in banking right after college and then eventually landing into research data and product strategy, I always bring that learning that I have from finance of how can we use, how can we as a company use research, use data, use strategy to build companies that last?

Claudia Natasia: And that is what my talk is going to be today. I hope that everyone here can come out of this talk with several different ideas for how we can build strategies for our companies to help our companies become more sustainable, particularly during more challenging economic climate. Like today, we’ll talk about a few things. The first valuation model called the discounted cash flow model. Then we’ll talk about two case studies of how being user driven drives valuation by way of the DCF model. And then we’ll wrap up and talk about evolving a revenue-generating data practice in your organization. I’ll share some tips for whether it you’re a founder or if you currently are employed in an organization, how you can structure your product revenue, data and research team to overall drive better and more healthier valuations for your companies.

Claudia Natasia: Let’s start with the discounted cash flow model. I’ll start by asking this question, is a hundred dollars today worth more or less than a hundred dollars next year and a hundred dollars the year after? I’ll pause for a few seconds and feel free to answer this question on the chat to start a discussion or write it down anywhere. Now, similar to that question that I asked earlier, let’s assume that you just won the Powerball. Congratulations, you won a huge windfall from the Powerball. And the question now is the Powerball usually does payouts in two different ways. The first way is a lump sum where they give you less than the amount you win, but they gave give all of it right now less taxes. Or you can also opt for this thing called an annuity that pays the, the, the funds that you win from the Powerball in a yearly manner. Would you rather take the lump sum so that huge chunk of money that comes right now, or would you rather take the annuity a smaller amount of money year on year until let’s say 30 years? That’s usually the timeframe of the annuity.

Claudia Natasia: I’ll give a few seconds so you can think about that as well. Alright, I actually ask this poll to a large number of people in my company the other week. And actually 72% of people answer lump sum and 28% annuity. And there’s really like, it, it’s a matter of preference at the end of the day, but also from a fiscal or financial sense, there is actually a right answer, even though at the end of the day it is a really a matter of preference. We can go debating this over and over again, but from a financial perspective, the answer is actually take the annuity. And I did a calculation to show why it’s important to take the annuity. Let’s assume that the Powerball, when I did this calculation, I assume that the Powerball was a hundred million. I don’t know what the amount is now, frankly, I’ve never been part of the Powerball, but let’s assume that the, that the Powerball is a hundred million dollars.

Claudia Natasia: If you were to take the lump sum payout, you’d get $52 million right now, less the federal tax at 24%, and then you’d get a net payout of around $32 million. Whereas if you get the annuity over time, over 30 years, you’ll actually net at $64 million, meaning you’ll actually get $31 million more, which is two x the amount of the lump sum pay out. That’s really amazing to see the difference between these two values. People often wonder why, how come the annuity, even though you don’t get the psychological reward of getting the money now, you actually net out with more. And what a lot of people don’t know is the Powerball annuity is actually paid through investments in bonds. The investments in treasury bonds actually has an interest rate of around 5%, so if you were to, let’s say, not invest in the stock market at all and you invest in government bonds, you’d net around 5%.

Claudia Natasia: Sometimes in good markets, of course, if you like let’s say invested in Bitcoin like four years ago, you’d probably net like three x higher, like I think at one point the increase was like 300%. But in certain markets there’s this thing called the risk-free interest rate where the for the risk-free interest rate in in particular, that’s the rate that you’d get if you invest your money in a safe product like government bonds. And the reason why annuity pays more and more each year is what Powerball assumes is if you were to take the money right now, you could invest it elsewhere. And the hope is that actually with the annuity, that if they were to keep the money and pay you on a yearly level, that they need to at least match the risk-free interest rate of 5%.

Claudia Natasia: Now let’s shift our thinking a little bit to the actual markets. I bet a large number of you are founders or part of a company that is actually VC-backed. By VC, I mean venture capital. And when you look at venture capital funding in the past just five years, you actually see an interesting trend in during the pandemic, we see venture capital funding actually increase drastically. And only recently because of the more challenging macroeconomic condition and rising interest rates and wars in in Europe, we actually start to see venture capital funding decline. And during periods when funding decline, VCs and investors, similar to if you are also an investor in the market, you become more picky on the type of companies that you want to invest in. And for the companies that you’re already invested in, you also pay more attention to fundamentals. You go back to very important fundamentals, business fundamentals of increasing sales while also being efficient and minimizing costs.

Claudia Natasia: And what VC is always used to calculate the valuation of companies is this thing called the discounted cash flow model. It’s actually a very, very simple formula that is a function of the cash flow that a business receives year after year. It’s a revenue less cost, that’s the CF and enumerator discounted or divided by a function of the interest rate. And this part is really, really interesting because as you all know right now, interest rates have skyrocketed. That’s why we’re seeing ramifications the mortgage market. It’s harder for people to get mortgage. If you could afford a home that’s let’s say $1.4 million last year with mortgage with the increasing interest rate, right now you could probably only afford a home at a lesser price because your mortgage is more expensive. And so when periods of r so r indicates interest rate in this formula.

Claudia Natasia: When r increases, when the Fed increases interest rates, then regardless of what cash flow you have, like assuming your cash flow is stagnant and even if it grows a little, that value will be eaten up by the increase in interest rates from this formulas perspective. And that’s the important thing for us all to know, as either founders or shareholders of our very own company as employees, is that we need to do more with less right now in this market, even if we’re able to grow our revenue by 20% this year because we were still able to maintain great sales velocity in a challenging macroeconomic climate. Note that a 20% increase in sales this year matters less than a 20% increase in sales last year because the interest rate has grown like nearly double at one point compared to two years ago. And that’s why it’s very important for companies to understand how we can build strategies that ensure that our cash flows in this DCF model are robust regardless of the macroeconomic condition in the world.

Claudia Natasia: What are the factors that typically drive strong healthy DCF models or strong healthy valuation? I listed a few different factors in the relationship to the DCF model sales. Of course, as you increase sales, you would see an increase in cash flow and a positive relationship to the DCF model. And a driver of sale is achieving product market fit. If you are starting a company and you are able to achieve product market fit earlier on and have customers that recommend your product or your experience to other customers and then drive like stronger horizontal growth on all of your product suite and continue to capture more and more of the market or even define your own market cap, then you can actually achieve stronger product market fit. And of course stronger sales and closely related to sales is growth. Growth also has a positive relationship to the DCF model.

Claudia Natasia: And companies typically achieve growth through these terms that you may have heard before, like acquisition, acquiring new customers, adoption, having customers actually use your product and using it more and more, which is engagement. And like what I said earlier is also recommending other people to use it and things like activation. Going through the entire horizontal and vertical growth of your product, using more of of your product’s, compliments that you also build in your company. And upselling and cross-selling, I’m actually getting ahead of myself because I just described the third driver of DCF which is actually retention. And retention also has a positive driver, like I mentioned it, it leads to cross sells and upsells. If you’re able to get customers to buy more of your product, like let’s say your company, what’s a great example of this… Like if your company sells a learning and management software, but also a conferencing software, if you’re able to get your customer to buy not just the LMS but also the conferencing software, then you’re essentially driving cross sells, which it’s the same customer but you’re driving more revenue from that particular customer.

Claudia Natasia: And then there’s things like lifetime value also, assuming that the customer doesn’t cross sell or upsell, they don’t buy more of your product suite, there is an opportunity of course fundamentally to just retain them, keep them year after year to come back and use your product and not jump ship to a competitor. And other drivers of DCF includes costs. There are several different types of costs and I’ve highlighted two major classes of costs, cost of goods sold. So if you were to produce a a product or a experience, what is the cost of actually producing that product? And then also operational costs of running your business. And then finally, like we’ve discussed in the previous slide, interest rates, which is driven by macroeconomic conditions and generally what the government decides to set the interest rates at. All of these factors work together with each other to ultimately influence the discounted cashflow model and the valuation of your company. And if you’ve read some tech news recently, that’s actually why we are seeing quite a bit of companies experience declines in their, their valuations recently because of the higher interest rates in the more challenging fundraising environment.

Claudia Natasia: Are there other ways we can hedge for revenue? Outside of what I’ve described on a DCF model? I’ll describe two case studies for how being user driven actually help hedge for revenue. The first is how a user-driven approach uncovers revenue generating opportunities. And I’d like to introduce everyone to this framework. It’s called the value frontier. There’s two different elements of this framework. One, if you’re a business, of course you want to drive value to your business. Like we saw in the DCF model, we want to drive more revenue, but also you want to drive value to your users.

Claudia Natasia: And the hope is that as you build more and more products or more and more features that you get closer to this frontier where you’re unlocking and creating value both for the business and both for the users. We were actually able to create that at this company I worked with in the past called Five Stars. Five Stars originally started as a loyalty company. If you’ve been to a boba store, you may have inputted your phone number and joined the Five Stars loyalty program. It was an amazing product and many, many people loved using it, but we wanted to do more with this product. And so what we did was we conducted user research. A lot of the researchers in the team went out to the field, went to all of the small businesses that used Five Stars to try to get a general idea, like an ethnographic perspective of how are people using the product and how can we improve the experience.

Claudia Natasia: We also mined revenue analytics data and created models against average revenue per user and closed loss feedback, which in sales terms is basically we weren’t able to to sign a deal with a customer, what was the reason behind their close lost deal. And we actually build a machine learning model that analyzes the text of all of the reasons behind the, like why we weren’t able to sign that customer. And then finally we use a combination of product analytics health metrics of all of our product suites to come up with an opinion of what can we do to actually drive more value to both five stars and our small business owners. And that ultimately led to the largest product pivot that I’ve frankly done in my career, which is to pivot the Five Stars loyalty program to become a payments provider. And we actually build all of this in-house. And ever since then, small business owners are able to take payments also that’s linked to their loyalty program and, and have this in store. And essentially what we did was we created value by driving this in-store loyalty program to become a mobile payment service that ultimately drove a multimillion dollar acquisition of five stars by a European point of sale company called SumUp.

Claudia Natasia: My next case study is on how being user driven reduces invisible liabilities and speaking about invisible liabilities. What are invisible liabilities? I like to use this example from friends. I don’t know if anyone here remembers this particular episode or even watch friends, but in this particular episode we, we all know that Monica is the character known to be very, very neat and very, very tidy. And her husband Chandler actually opened a door that he never realized was in their apartment. And behind that door was all of Monica’s mess just stashed inside. And essentially this really describes invisible liabilities because invisible liabilities are all of the things that it, it’s, think about it like tech debt or product debt. It’s all of the things that we don’t want to deal right now that we just stash somewhere. But eventually we will have realize that eventually they become a reality.

Claudia Natasia: In this particular example in this company that I worked with, customer support costs were rising and we didn’t know why. We were pretty successful at reducing the cost. It takes a support customers from a customer service perspective in year one. But in year two we started seeing it drastically increasing like in this hockey stick chart. And so what the team did was we mined 9,000 support call data, did one ml text analysis of all of the feedback on that support call data to find out the root cause and uncovered that these, the, the root cause of the increase in costs were actually three major user experience issues that were not fixed. It led to an average of five repeat calls of 30 minutes each. And assuming that the calls were $1 per minute, like the, cuz it, it takes of course we have to pay when we answer a call from a customer, it leads to $225,000 of extraordinarily loss not accounted for.

Claudia Natasia: We were able to fix this and of course reverse the loss that we had accumulated. But imagine if we had been more intentional earlier on to be more user driven to actually uncover and fix these UX issues, we would not have accumulated all of these costs. And that’s the scary thing about invisible liabilities is no one can see them until it’s too late. That’s why they’re called invisible. And so let’s update the DCF model. There are two things that ultimately drive discounted cash flow in addition to all of the things that I mentioned. New product pivots and invisible liabilities are also examples of things that could drive DCF.

Claudia Natasia: And both of these things are driven and discovered by strategic user research. I’ll end by describing how to build and empower a revenue generating research team or a revenue generating data and strategy team. And I’ll give a quick example of Uber. If you were to ask my mom 20 years ago if she would let me jump into a stranger’s car, she would definitely say no. But right now I’m going to Indonesia in a month and my parents actually told me to take an Uber from the airport. If Uber were to ask people 20 years ago if they would jump in a stranger’s car, probably the product would not have been built the way it was built today. Similar to the approaches that sometimes we take when we do research by building empathy maps like this, empathy maps are fundamentally not a great way to do research because they actually diminish innovation.

Claudia Natasia: When we’re asking people what they empathize with, what they do, what they see, all of this is limited to what currently exists in the world. If you’re building game changing products and experiences, you can’t ask people all of these things since their perception will be limited to what’s in the world today. And so instead of asking all of this, I would encourage founders, research practitioners, data, product managers, anyone in in the industry to think about data from a more strategic lens.

Claudia Natasia: And instead of asking what users want, ask these three questions. One, how can we influence user behavior? Two, how do users behave? And then three, is our strategy working efficiently and using a combination of both qualitative research, quantitative research, and also more machine learning and and advanced statistical analysis. And do not be afraid of combining all of those different types of data sources to ultimately form an opinion that will drive your revenue and your valuation strategies for your company.

Claudia Natasia: And I’ll end with the statistics. Two x higher growth is experienced by user driven companies as measured by a reset study from McKinsey. I encourage everyone today to shift your mindset away from just being product driven or sales driven and find ways to incorporate data to be more user driven to drive stronger and more sustainable valuations for your company. Thank you everyone. Please feel free to ask me questions on my Twitter account or I’ll be hanging out at the lounge after this. And happy to answer any questions that you may have. Thank you.

Angie Chang: Thank you Claudia. That was an excellent talk. I think people learned a lot and there’s definitely some comments and requests for slides, so if you can share them on your LinkedIn or put them in the chat, that person can get them. Thank you so much and we’re gonna start our next session. So yeah, good to see you. Thank you so much for joining us on International Women’s Day and we’ll see you at the next session. Bye.

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

• Check out open jobs at our trusted partner companies.
• Watch all ELEVATE 2023 conference video replays!
• Does your company want to sponsor a Girl Geek Dinner? Talk to us!

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

• Check out open jobs at our trusted partner companies.
• Watch all ELEVATE 2023 conference video replays!
• Does your company want to sponsor a Girl Geek Dinner? Talk to us!

Sukrutha Bhadouria: Hi. Hi everyone. We hope you’re having a wonderful time in the conference and you’ve been enjoying all our sessions. A few housekeeping notes. Make sure to tweet all the amazing things you’ve been learning in your various sessions that you’ve been attending. The hashtags are IWD2023 for International Women’s Day 2023 and hashtag ElevateWomen, which is, you know, the name of our conference where we want us all to lift as we climb.

Sukrutha Bhadouria: Next up, we have Maya. Maya is Director of Engineering at United States Digital Service in Washington, D.C.. Prior to working at U.S.D.S., she worked at Facebook as a senior software engineer and graduated from Stanford University with a computer science degree. Welcome, Maya.

Maya Israni: Thank you. And thank you so much for inviting me to be here today. Thank you all for joining. As she mentioned, my name is Maya. I’m the Director of Engineering at the U.S. Digital Service. I’m not sure how familiar y’all are with the Digital Service. During this talk, I’ll dive into a little bit of what government tech is and my journey here and the types of projects that we work on in government tech. Just because it is a little bit unique, especially as someone coming from private sector that transitioned into public sector and public sector work has been pretty awesome. Please do use the Q&A. I will aim to save as much time as possible to answer your questions. Again, use the Q&A and chat feature, and I will try to save time at the end to answer them.

Maya Israni: A bit about myself. I joined U.S.D.S. at the end of 2020. I started off as an engineer working in our nutrition safety net benefits space. For those of you that aren’t familiar, the United States government has a number of nutrition safety net benefit programs. Those include programs like WIC (Women, Infant, Children) or SNAP, which is formally known as food stamps. These are programs that are offered to low-income folks for cash and benefit assistance in, in buying food in other nutrition services. And we had a team over at the Department of Agriculture working on improving the service delivery of these benefits. I was an engineer in that team and then led that team for about a year, and then stepped into the director role at the beginning of 2022, and have been here since prior to US Digital Service, which is a small unit of technologists sit at the White House. I worked at Facebook, I worked in the private sector and I was an engineer on both integrity and privacy teams there and made that transition into government at the end of, of 2020. Let me dive a little bit into what engineering looks like at U.S.D.S.

Maya Israni: We have around 50 or 60 engineers and data scientists here at the US Digital Service from a wide variety of backgrounds. We have folks that are coming from public sector, from private sector, maybe that’s non-profits, other governments state governments, also folks with civic technology background. And like most places, we hire a diversity of engineers. We have full stack, we have front end backend folks with more security expertise, folks with more DevOps expertise, data scientists, data engineering. We generally hire senior folks with a generalist background, and that’s because we work on a really broad range of projects here. It’s important that folks are able to just jump in and, and plug in when, when needed and however needed. We also work on cross-functional teams.

Maya Israni: In addition to engineers, we have product managers, we have designers, and again, similar to private industry, those designers sometimes have UX backgrounds, research backgrounds, content backgrounds And then we also work with procurement folks, and that’s particularly important in government because oftentimes in government, we’re not necessarily building the thing, building the product. A lot of government services are run through having, bringing in a vendor or a contractor to kind of build out that service or build out that program, and so it is a key part of how we build government technology, and we have procurement folks to help us ensure that we’re building things responsibly, and baking in the best engineering and design practices into the products that we build.

Maya Israni: The day-to-day of U.S.D.S. engineering, like any job, there’s no specific day to day that I could call out, but we have about 20 different projects across about a dozen federal agencies right now. And the general mission of U.S.D.S. engineers is to bring the best practices in technology and in design into how we deliver government services and programs. Sometimes that might mean helping an agency set their technical strategy or hire in technical talent. Sometimes it actually is the technical implementation, and we do have some folks that have the hands-on keyboard coding and architecture work. Sometimes it’s helping set model engineering practices for an agency and, and kind of teaching, teaching folks how to fish rather than doing the fishing ourselves.

Maya Israni: I think it helpful and usually just to dive into some examples of projects so you can get a flavor of, of the breadth and scope that we work on, so I wanna dive into three examples with y’all today. The first is around our work with the CDC. So at the onset of the pandemic as we all experienced, there was a huge data crisis in terms of, we were testing folks for Covid and needed a streamlined, accessible sustainable way to report that testing data to public health departments. And so the US Digital Service partnered very closely with the CDC in building out a bunch of data reporting systems. One of them is called Simple Report, and you can learn more about this at simplereport.gov and the concept is quite simple. It’s a system that is used now across the country for testing organizations to report Covid test results to public health departments, and as you can imagine, that’s a critical point in the data reporting process and helping understand what is the severity of Covid cases or a number of Covid cases across the country, and helping us better understand how the pandemic was affecting different regions and what trends we were seeing. Again, this is something that we partnered very closely with the CDC on, and it’s a product we helped build from scratch and it’s a product that’s still used today.

Maya Israni: The second example I wanna wanna outline is related to an executive order that was signed in the end of 2021. In December 2021, President Biden signed what was called the Customer Experience Executive Order, and the goal of this Executive Order is to improve the customer experience, so the experience of folks like you and me have when we interact with government services and programs. And one of the core tenets of U.S.D.S. is designing with users and not for them, and bringing in the principles of human-centered design and helping better shape government services and programs to meet people where they’re at and be accessible for everyone. And so with this customer experience executive order, U.S.D.S. has taken on a bunch of different customer experience work across federal agencies. There are almost a dozen federal agencies that run programs that interface with people, the American public, and those are agencies like the Small Business Administration, the Social Security Administration, Veterans Affairs, and we’ve partnered really closely with those agencies to provide more accessible services to people and again, make sure we’re meeting people where they’re at and providing them with an accessible and dignified experience as they’re moving through the process.

Maya Israni: In particular, we worked closely with the Social Security Administration to modernize their landing page, so that’s ssa.gov. We launched a beta site at the beginning of 2022 and really focused on the user here, right? These are folks who are maybe more senior and there might be accessibility or connectivity issues when they’re accessing the website, and so we focused on accessibility and navigability and making sure that we were minimizing as much friction as possible when folks are landing right on ssa.gov and trying to understand more about the services or programs that they’re entitled to, and then how to actually enroll in them and minimize any sort of friction along the process. The third example of project I wanna highlight is one of our open source projects. This is one that was born out of the Justice 40 Initiative. In December, or excuse me, in January of 2021, just after inauguration, President Biden launched the Justice40 Initiative, and this commits to distributing at least 40% of certain federal funding to communities that have been disadvantaged, that have been marginalized, underserved, overburdened, particularly by environmental factors.

Maya Israni: And so after this initiative launched, US Digital Service partnered closely with, it’s called the Council on Environmental Quality (CEQ), which is a council out of the White House, and we built what’s called the Climate and Economic Justice Screening Tool. Again, this tool is, the code is all open source right now, you could check it out if you’re interested, and this is a tool that shows an entire map of the United States down to very specific local regions, and you can dive into a number of factors (environmental factors, housing factors, transportation factors) to better understand where each of those regions are scoring. And the goal of this tool is so that an agency can go in and help them identify communities that have faced environmental injustice, and that helps direct where they should focus and funnel funnel federal funding. It’s a tool that we built mostly off of census data, so data that’s available. And again, we built this tool end to end and open source, and in that way, we’re really trying to, as I mentioned instill engineering practices and set the standard and model of how we should build tools and products in government.

Maya Israni:I’ve talked a lot about what engineering projects have looked like at U.S.D.S., and I, I also wanna come, come forward and talk about the difference between maybe private sector, where maybe more of y’all are familiar with and public sector. And so I’ll drive into like different aspects of the organization that I think are important. The first is around culture. U.S.D.S. has developed a number of, of values that we work by and, and live by every day, and I’ll call out a few of the ones that are most resonate with me, and most resonate with the work that I think we do, because our culture is a little bit unique from the rest of government. Obviously the US government is a huge, huge entity and a huge bureaucracy and we’re just a mighty group of just over 200. But we do aim to maintain a culture that imparts those best practices as of technology and design, and one of the values that we live by is find the truth, tell the truth. You’re coming in as, as technical experts and you’re meant to, to hone into that expertise. And, and as you see things say things and, and call things out that goes really closely in line with our other value of optimizing for results, not optics.

Maya Israni: Every day, we use data and research to help inform the direction that we are designing and heading, and then the decisions that we make in building products and providing services. Another value of ours is to go to where the work is, and that means that while we sit technically at the White House, we are partnering very closely with agencies, and that means to meet your agency partners, meet your users where they’re at, go to where the work is, go shadow folks, go do field research, go sit in the offices of our federal partners, and empower the people in the agency who are oftentimes doing the work already. And again, designing with user is not for them, centering the human, centering the person in everything we do, designing for all people in America.

Maya Israni: A topic that became all the more exacerbated during the pandemic was this digital divide that not all of America might have access to smartphones or the type of Airmeet platforms that we’re using today, and how do we build services and introduce technology responsibly and provide folks with yes, an option to enroll in a program online, and also provide a paper process for someone who might not have access to the same resources. Building accessible websites, websites that are translated to different languages and using plain language, not government speak or legalese. Another topic I wanna dive into a little bit is just the pace of the work. Government does move slower, plain and simple. It’s a bureaucracy. That’s how bureaucracies are designed.

Maya Israni: Quite frankly, it can be irresponsible to move too quickly sometimes. Inherently as some of the work we do is a bit more retroactive. We’re responsive to events like a pandemic and how that changes how our country operates. That being said, things can go from 0 to 60 really quickly if we’re sprinting towards a launch. Like I mentioned, we worked on some Covid launches and on a few others, and things can move fast. And we try to build momentum and create that momentum as we go. I think metrics of success also look a little bit different in government. I came from private sector and was used to having a half over half product roadmap with dashboards and data, and could see the graph grow and, and ebb and flow over time. That’s not always the case over here, I would say.

Maya Israni:We oftentimes, sometimes we have those launches with metrics, but oftentimes we have more maybe creative metric success, and that looks like I mentioned, like hiring technical talent into agencies or setting the technical direction of an agency. The last piece I wanna touch on is the scope of work because it’s huge. The government is entrusted to provide critical services and programs to the public. And the types of services that we work on and the people that we’re impacting. The scope is enormous. We’ve worked on programs with veterans, with refugees and asylum seekers, with students, with small business owners, seniors, and focusing really mostly on, on the people who need it most. And I think that’s definitely worth calling out as maybe one of the unique facets of working in government.

Maya Israni: With that, I am going to take a look at some of the questions. First question I see right here is, “How did you discover the opportunities in U.S.D.S. and what made you transition from private to public?” I think that feeds kind of straight in from what I just said there, which was I’ll answer the the second part of the question first, which is why I made the transition from public to private, or excuse me, from private to public, is I never… I studied computer science in college. To me, computer science was always a tool. I wasn’t necessarily keen on building things to build things. It was always the why, and why does it matter, and how am I impacting people, and what am I doing in this world to make things a little bit better than… how leave things a little bit better than how I left, than how I entered. And again, I think the scope and impact you can have in government is just unparalleled and unmatched, quite frankly. In the United States, we entrust the government to provide these types of services and programs to people and set the policies, and I think that the type of impact you can have here was always very appealing to me.

Maya Israni: How did I discover it? That’s a good question. I think I discovered it a little bit by word of mouth. The story that most folks often hear is the healthcare.gov story. So when U.S.D.S. was born out in part when healthcare.gov launched and the launch didn’t go as smoothly, a bunch of technologists came in and helped stand it back up. And I think I’d kind of followed that journey and that story, and when the time seemed right and for me it was almost a year into a a pandemic, I decided to to make that shift.

Maya Israni: Another question I see. “What advice would you give to early stage UX designers interested in building a career in civic tech or government?” I’m gonna maybe broad broaden up the question and advice I would give to early stage technologists in general. There’s a lot of ways to get involved in civic technology that aren’t just working at, say, the US Digital Service or government agency.

Maya Israni: A lot of states and cities have digital services. I recommend you take a look at the Digital Service Coalition. There’s also volunteer opportunities and there’s also, we have what’s called the US Digital Corps hires early stage folks / early career folks to come in and do a tour of service in government. Definitely a lot of different options in how to dip your toes in and in the way that fits your personal and professional circumstances best. I see the host coming in here, which probably means I’m getting kicked out at this point. <Laugh>

Sukrutha Bhadouria: No, hardly getting kicked out. Just wanted to say thank you so much, Maya. This was wonderful. I see the amazing engagement and the questions. Please do take it to Maya, or you can reach out to her, how, how do people reach out to you, Maya?

Maya Israni: I think throughout the profile, I’ll add in my email address and you’re welcome to reach out to me there. We also have folks attending, sitting in the lounges and attending the different networking sessions, so please come talk to us. We’re eager to share more. We also have our incredible engineers and data scientists and designers and folks on the project teams attending those sessions as well, so please talk to them. And we’re eager to hear from you!

Sukrutha Bhadouria: Yes. All right. Just like one of the attendees said, this was rad. Thank you so much. All right.

Maya Israni: Thank you. Bye bye-Bye.

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

• Check out open jobs at our trusted partner companies.
• Watch all ELEVATE 2023 conference video replays!
• Does your company want to sponsor a Girl Geek Dinner? Talk to us!

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

• Check out open jobs at our trusted partner companies.
• Watch all ELEVATE 2023 conference video replays!
• Does your company want to sponsor a Girl Geek Dinner? Talk to us!

Angie Chang: We are going to be welcoming Kimberley Parsons, who’s the CEO and founder of Bamboo Teaming. She’s coming in from the Richmond, Virginia area. Please drop in the chat where you’re also coming in from today, it’s International Women’s Day! Thank you so much for joining us on this special day. Kimberley helps leaders develop and deliver on the full power of their leadership and so they can catalyze higher performance and results in their teams. Welcome Kimberley.

Kimberley Parsons: Yeah, thank you so much for having me today. I just wanna do a sound check that you can hear me. Yeah, there I am. Okay, great. Well welcome everyone. So great to have you here. I am Kimberley Parson, CEO of Bamboo Teaming. Just before I start and be off to the races, I just wanna share a little bit about me. I am joining you from Richmond, Virginia here to the East Coast where I hang out with my son and my husband and we make lots of fun memories together. I will tell you that I am a former technologist leader in technology after 10 years and really shifted my career through a women’s leadership development program to really helping teams and organizations be more effective. I was hooked through a lot of agile programming, agile development, agile leadership on the power of teams to really drive tremendous results within organizations. And so I wanted to do more thus changing my profession into HR coaching, executive coaching, team coaching, team development.

Kimberley Parsons: I’m here with you all today to talk about this topic of teams. Well let’s just jump right in to make the most of our time together. And my question for you all, have you ever led or been a part of a team that was full of highly talented people and yet it seemed like you could never hit your stride as a team. It was bumpy, you weren’t rolling, you look around and say, we should be cruising alarm together. You knew you had the potential to be more higher performing than you were, yet there was something that was holding you back, slowing you down, making it bumpy and capping your performance all together. So great from all the thumbs up and emotions and reactions.

Kimberley Parsons: I see that you’ve been a part of this team. Well, you are in the right place today because today we’re really going to focus on high level, quick touch three things. The first thing being what, how do you uncover your team’s specific strengths and opportunities in the four key dimensions that impact team functioning? Then we’ll go to well what does that mean for you as a leader? You will take some time to really think about like, well, what is your relationship and how your team is performing as a leader? And lastly, we will cover some common hot spots that can make incorporate team functioning and the corresponding team leadership behaviors you can do to cultivate kind of those areas that are hot spots for you today. That’s our roadmap. Quick housekeeping things. I will ask you to just drop questions in the chat.

Kimberley Parsons: My hope is that we’ll have some time to really go through those questions in the chat. And also just like you are doing right now, I invite you to engage with one another. If you hear something you’re like, aha, like that’s it. Like put it in the chat and keep the conversation going that way as well. With that, let’s just jump in to this first conversation, this first talking point around how do you know if your team is actually high performing? Not? Where are those places of strengths? Where are those places of opportunity? I love this quote, direction is more important than speed. You can go fast in the wrong direction. And so I work with teams who they are like, we need, they are the square tire, we need to have higher performance. And so they go and they just start picking and doing things.

Kimberley Parsons: My question is, well how do you know that the thing that you are picking and choosing to develop to grow is actually the right thing? It’s helpful when we have like a north star or a model for us of what high team performance looks like so that you can really do the assessment of, well, what are those strengths that we already have and what are those opportunities like, where do we need to head in terms of developing and growing ourselves as a team? In this first talking point here, really the goal is for you to think about where does your team have strengths and where does your team have opportunities in order to be higher functioning, higher performing as a team? And for this, the Caris team will is really useful. I like this as a model because it gives four dimensions of team functioning to pay attention to.

Kimberley Parsons: The first one being common purpose and goals, then roles and competencies, collaboration and cohesion, and then mutual accountability. Okay, so I’m going to give a really quick definition of these and as I go through, what I would ask you is to think about the team that you are closest to that you lead the team that you are most accountable for driving like big, tremendous audacious results in your organizations. Like think about that team and as we’re going, as I describe each of these dimensions, like make a mental note like is this a core strength for us as a team or is this an area of opportunity for us that we may need to address? Okay, are we good? Be thinking about strengths and opportunities for your team. First off, in the parentis team will at the center we have the common purpose and goal and common purpose and goal is really it addresses this question of why should we come together as a team?

Kimberley Parsons: What is our common purpose? Because rarely does a group of people, like do a group of people come together just for the sake of being together. There is something that is drawing them, connecting them, that’s really shifting them or causing them to need to be a team. So when a team has clearly defined their common purpose, it really gets to this question of, well, why should we come together? What is our remits? Why are we here? And having shared goals, it really speaks to what will we produce as a team. And so teams that have a strength in this area, they’re able to answer that question, what is our common purpose? And what are we producing together? Or this could be an area of opportunity, we come together, but maybe we’re not being as productive as possible because we’re not on the same page about our purpose.

Kimberley Parsons: We’re not on the same page about our goals. Again, think about is this an area of strength or opportunity for your team in this space of roles and competencies? Once we know what our shared purpose is, what our shared goals are, do we have the right people on the team and are we clear about the roles and responsibilities of each person on the team? Roles and competencies really answers this question of what are the contributions and expectations of each person on the team? Now I can imagine you’ve probably been a part of a team at least once in your life where it was unclear like who’s on first, as I like to say, what are the roles? What’s my area? What’s your area? How do we actually play together? How do we contribute together? That could be an indication, right? That this is a dimension roles and competencies that is an opportunity for that team to get clearer about so that it can drive higher team performance.

Kimberley Parsons: That’s just an example In this next section of collaboration and cohesion, how will we work together independently? What’s our rules of engagement for how we collaborate and how we connect to one another in terms of our relationships? Again, another space that could be a core strength for you or it could be a core opportunity. And lastly, in this dimension of mutual accountability, this is really about how do we hold ourselves and each other accountable for achieving results and upholding our standards. If this is a strength for you, you have practices in place that help you to hold one another accountable and you may not have practices in place or you need to strengthen them. So this could be an area of opportunity. I like really starting with the team will because it can give you a framework as to again, at North Star up, Hey, this is happening in my team, right?

Kimberley Parsons: This is an area of strength, this is an area of opportunity. And then the thing I like to say is, okay, leader or person who has influence in a team, what am I that they are that? Teams are systems. As the leader moves and shifts, it causes a reaction within the team. The question for you all today is, what am I that they are that in team performance, right? As you are thinking about how you were noting strengths and opportunities in your team as leader because you have such a weighty position within a team, such a weighty amount of influence within the team, what are you doing as leader to really catalyze higher team performance? Or where might you be frustrating that? And so the question for you to think about for yourself, right? Is really what are you doing to help drive a common purpose or share goals with the team?

Kimberley Parsons: Do you have the right people on a team? Are there clear roles and responsibilities? You leader, are there tools and practices in place to enable effective collaboration within the team? Are you helping to set or expecting setting the expectation that your team sets the conditions for inclusion, safety and productive interactions? And lastly, are there methods in place for the team to deliver results and address breakdowns when they occur? Right? So a lot of my work with teams leaders are really about how are you setting the conditions for accountability to be present within the team? Okay? As leader, again, what am I that they are that as it relates to team performance. And so lastly, the question for you to think about is which capability is most called for from you as a leader to catalyze higher performance in your team? If you said, Hey, my team has this area of opportunity, then turn the mirror on yourself, right?

Kimberley Parsons: What is it an area or a specific practice or behavior that you as leader can shift to really help catalyze that plant those seeds within your team? And so the last thing that we’re gonna cover here is for you to discover the common hot spots that can make or break team functioning and the corresponding team leadership behaviors that you can cultivate them. I work with a lot of teams, <laugh> I work with over a hundred teams and there tend to be some hot spots that keep showing up over and over within teams. I’m gonna go through these really quickly and I’m curious about which of these hot spots have you seen most often within teams?

Kimberley Parsons:The first one here misalignment on team purpose. And this is where the team, again, going back, they don’t have a clear purpose for why we’re working together as a team. If you ask the six people, the seven people that make up that team, you’d get seven different answers. That of course is the root the starting point of a team being able to function effectively. This is our number one starting point and I would work with you to get clear on what your team purpose is.

Kimberley Parsons: The second hot spot that I’ve seen is the inability to flex team leadership styles. And this is specific for the leader. I know that a lot of leaders really want to like, they have their preference for how they show up. I just wanna be participate participative with the team. Or I have leaders who are like, well, I just wanna like show up and like be directive. And this step back, what I would say is you need to really have an ability to flex your team leadership style based on what’s going on within the team, based on the people that are within the team.

Kimberley Parsons: The third hot spot is unclear roles and responsibilities. And so in this hot spot, this is where team members don’t have clear roles, responsibilities, there’s confusion or maybe there needs to just be a question around accountability around your space, MySpace, our space decision rates, what are those things, right? That is present there. And then in this fourth hot spot, ineffective decision making. This is where I work a lot with leaders on decision rights. Are you clear about who owns the decision here? Are you clear about the decision mode that you are using? Do you make decisions using consensus? Do you make decisions using authority? Do you make decisions using democracy? Does everyone vote? Does the leader just make a decision? And what I find often is that this isn’t clear in the team. It actually slows down. It makes decision making inefficient within the team.

Kimberley Parsons: Another hot spot that I see is polite fights. And so polite fights is where the team I hear often as we have a nice culture, so we don’t actually say the thing that needs to be said or we have disagreements about how we should move forward, but yet that doesn’t actually happen in the room with the team where you can harness and tap into the wisdom of the team. All of that happens outside of these teams because we didn’t wanna say anything, right? Or we say things but we’re not really saying the thing. There are polite fights there. And then the other hot spot that I see often is fuzzy agreements and commitments. It’s hard to have mutual accountability when you’re unclear about what the request was that you were asked to do. So what’s the agreement that we actually made?

Kimberley Parsons: You might hear it sound like soft ask of, it would be great if you could do this thing for me. Was that actually a request or not? Did I actually say yes or not? They’re fuzzy agreements and commitments and on a commitment side you get the I’ll try <laugh>, right? I’m not sure if that person actually committed to me or not. When breakdowns occur, it was really fuzzy and it’s hard for us to hold each other accountable. Okay, with that I am just going to share this poll really quickly. I would love to hear what are, what is the hot spot that you have seen most often within teams? And I’m trusting that you all can see the pole. I don’t know, can you see the pole? No, maybe? I don’t see the pole. Okay. Oh yeah, you see the pole? There we go. We’re getting some boats coming in. Okay, it’s going, it’s going. I don’t know if you all see the responses. Keep voting show on stage. Do you see it?

Kimberley Parsons: Okay, good. All right, awesome. Well it looks like the winner right now is unclear roles and responsibilities and then polite fights is coming in behind there. Fuzzy agreements and commitments. Okay, unclear roles and responsibilities. Wow, that is getting the majority here. I am going to close the poll. I don’t know if I closed the poll or not. I think I did technology. I love it. I go back to this question of what am I that they are that and if you are seeing like unclear roles and responsibilities has 23 votes here, right? What are you doing as a leader? Going back to you as leader turned the mirror around, what can you, what steps can you take as a leader to help have clearer roles and responsibilities within your team? One of the things I would suggest is really bring your team together and just do a mapping around accountability.

Kimberley Parsons: Do a mapping around decision rights. Do a mapping around roles, right? That is a, that is a simple activity that you can do with your team to help create more clarity, which leads to higher team performance, right? And so you can really go through any of the hot spots that you are experiencing within your team and ask yourself that question, what as leader can I do? What practices, what behaviors can I really do to shift, right, to help, to shift, to turn the tie, to have my team show up higher performing, higher functioning within this particular area. And so with that, I am going to go all the way. Keep going, keep going, keep going. Which hot spot needs the most attention in your focus team and what leadership moves might you make to shift the team’s effectiveness in that area? That is the question for you to walk away with. And then what’s the most powerful bite size action do you will take to impact your team’s effectiveness after this talk? That is the question for you, feel free to throw that in the chat box. I’ll open up the chat box so we can see some responses here. And Angie, I think that is my time. Thank you all so much for joining. I appreciate you being present. I’d love to continue this conversation with you.

Angie Chang: Thank you. That was an excellent talk. We think we have two more minutes actually, so if you want to answer any questions for two minutes and then we will wrap up.

Kimberley Parsons: Okay, perfect. Let’s do a question. If you have a question, throw it in a window. What about the manager not being able to manage the team properly, like if people step on each other and conflict management? Hmm. Thank you so much for the question. I think the response that I have, like my immediate reaction to that is what about the manager not being able to manage the team properly is to get to the root cause? Like I’m curious about if the manager is the manager of the team, why is manager not able to manage the team properly and maybe there’s some forensics that needs to happen in that space. Like if people step on each other in conflict management. Then the other part of what I would answer in that is I’m a big proponent of do some micro training with your team around a model, a method for conflict management and then practice and try it together and position it as we’re gonna do this experiment, right? First off is get to the same language around conflict management and how you do it and then position it as we’re gonna do an experiment and try out like this model that we just learned together. And it’s amazing because that can create like a safe space for you all to practice being more effective in this area. That’d be the second thing. Do I have time for one more question?

Angie Chang: Yes. One question.

Kimberley Parsons: What about management responsibility without authority? Wonderful question. The thing that I like to say is that we are all like, I firmly believe that we all have a place of influence within the teams that we are a part of. I think when you’re in this leading without formal authority or role, that’s where the relationship piece becomes super important. Really taking a time to go, go to one-on-one coffees, virtual coffees. If you’re not in person, really start to build the relationship and really practice like this move of inquiry. Get curious about, well hey, in this particular area I’m curious about like what your perspective is. Here’s my perspective. Really do this dance of inquiry and advocacy and build those relationships with each other. And I would also like just push, push back gently and say, I think we all have more influence than what we may give ourselves credit for as well.

Angie Chang: All right, great. We’re at time. Thank you so much. We are going to be moving into our next session, which is speed networking and hanging out in the lounge. Feel free to go wander over to the lounge. There are some tables there with either logos of companies that are hiring or there are also table topics like product engineering different teams based on the different job titles that you want to get or already have. Feel free to join those and thank you so much, Kimberley.

Kimberley Parsons: Yeah, thank you for having me.

Angie Chang: Thank you.

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

• Check out open jobs at our trusted partner companies.
• Watch all ELEVATE 2023 conference video replays!
• Does your company want to sponsor a Girl Geek Dinner? Talk to us!

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

• Check out open jobs at our trusted partner companies.
• Watch all ELEVATE 2023 conference video replays!
• Does your company want to sponsor a Girl Geek Dinner? Talk to us!

Angie Chang: Happy International Women’s Day. I hope everyone is having a great time in speed networking or meeting people at networking tables in air meet. We’re excited to be welcoming our afternoon keynote of the day. Jessica Sahagian is a director of engineering at ConnectRN and founder of Appraisely.io. She is a champion of women’s neurodiversity in tech and will be speaking about neurodiversity at work and previously held tech and finance positions at Fidelity, MITRE, and Raytheon. We’re excited to welcome our keynote speaker. Jessica, take it away. <Laugh>

Jessica Sahagian: Thanks so much, Angie. Hi everyone. Thank you Angie for the warm introduction and for having me. I’m really, really excited to talk to you today about one of my favorite subjects, neurodiversity. First so we can get to know each other better. If you could, could you type in the chat why this particular session was of interest to you? Both for my curiosity’s sake, but also because I wanna make sure that if you have a particular question or concern, that I can address it in the time we have together. Thank you for doing that and I’ll try to look at them throughout the session.

Jessica Sahagian: As Angie mentioned, I’m Jess Sahagian – very Armenian last name, <laugh>. A little bit about me. As Angie mentioned, I’m the director of engineering at ConnectRN. I’ve been in that position for a little over a year. I’m the founder of a real estate tech startup. I’m also an executive MBA candidate. I’m an owner of a luxury leather goods restoration business which has clients like Neiman Marcus. And I’m a mom to a wild two-year-old boy. Neurodiversity for the win with all of that <laugh>. Prior to ConnectRN, I spent two years at Fidelity Investments, five years prior MITRE Corporation, and prior to that I was at Raytheon and BNY Mellon so there was a little bit of finance in there, and some Department of Defense. Prior to that, I was an NFL analyst and I worked in higher education and technology publishing. In between all of that, I’ve been in situations where I’ve been unable to afford heat for my apartment. I’ve been in an abusive marriage. I’ve been divorced. That’s a lot, right?

Jessica Sahagian: A lot of role change, a lot of industry change, and a lot of ups and downs personally. However, being neurodiverse has actually been an asset once I figured out that I was neurodiverse. Since I like to follow my own advice, and you’ll see this later, I will first present an agenda to you before we’re gonna talk about. Agendas are very important, especially for NEURODIVERSE folks. <Laugh> I wanna give you an idea of what I’ll be talking about. That’s not to say I’m not gonna veer off topic here and there. It’s apropo of this conversation. I’m neurodiverse. I tend to go off and get really interested in something and then not be able to stop talking about it.

Jessica Sahagian: First, I’m gonna do an overview of neurodiversity. Then we’ll chat about hiring, how to attract a neurodiverse, and I’m gonna refer to neurodiverse. Instead of saying the whole word. I’m gonna say ND as I talk about different things, retention, how to create an ND-friendly workplace, and finally how you can be an ally and how you can succeed as an ND person in a work world and in a world that’s really set up for neurotypicals.

Jessica Sahagian: Let’s dive in. Neurodiversity, what is it? Neurodiversity is all around you. You’ve likely heard the term, it’s become both happily and begrudgingly a buzzword lately. Happily, because it’s getting a lot of attention and folks who have been marginalized are speaking out and speaking up more. Begrudgingly, because caring about neurodiversity shouldn’t be a fad or something that goes in and out of fashion like shag carpets or shiplap walls, right? We, we should care about the neurodiverse experience as part of DEI and that should be the standard, not the exception.

Jessica Sahagian: So let’s get into a definition. Neurodiversity refers to diversity in the human brain and cognition. For instance, sociability, learning, attention, mood, and other mental functions. And that’s straight from Wikipedia, friends, so you know it’s true <laugh>, but what constitutes neurodiverse, the list is very large and it grows every day, but it includes most popularly, autism spectrum, ADD/ADHD, dyslexia, dyspraxia, dyscalculia, OCD, and more. And while there are Googleable lists of all these symptoms for everything, I wanna make it very clear, none of these manifests the same across people affected. None.

Jessica Sahagian: In fact, up until the two thousands, it wasn’t even thought that women could have a ADHD. It was seen as something affecting hyper, misbehaved little boys. I remember as a little girl it, the little boy who lived down the street who was always jumping around, my grandmother said, “oh, he, he’s ADD, he doesn’t behave.” And autism’s really undiagnosed it, rarely diagnosed in women, especially adult women. There are reasons for this that I’ll get into in a little bit, but that this is why I went under the radar, right? I was well-behaved on the outside, as many of us probably were. I was at the top of my class. I brought home report cards glowing with praise, but I was masking. I was pretending. I was putting in hard work every single day as a little kid to be palatable to society and everybody’s expectations because of the lack of resources for young women and adult women.

Jessica Sahagian: Many neurodiverse women don’t know they’re neurodivergent, which is why consideration of neurodiversity really should be given to everyone. Also, a formal diagnosis – I do wanna mention this – a formal diagnosis is obviously valid, but self-diagnosis is also valid. In the United States, many doctors who diagnose neurodiverse conditions for adults do not accept insurance. In 2008, after years of struggling, my diagnoses cost me over $4,000 out of pocket at a time when I was working five jobs just to survive. Nobody would choose to be autistic.

Jessica Sahagian: If someone says, I think I’m autistic, you can just listen, you’re not there to solve a problem for them. You’re there to support. So now we’ve just established what neurodiversity is. What does the working world look like for neurodiverse folks? Well, miserable. That is, if we can even find a job. Our unemployment rate in the US is estimated at around 3.4% right now, overall. For the neurodiverse population that rate soared to 30 to 40%.

Jessica Sahagian: How do we fix this? We hire neurodiverse folks and we create environments conducive to their retention. Simple as that. We can’t just tolerate. We have to move away from tolerance, and we need to celebrate and accept. It starts with the hiring process. Make it short, make it sweet, and give feedback at each step. If you can’t tell in two to three interviews, maybe four, if someone’s a good fit, you’re bad at hiring. Plain and simple. You should be able to tell, and you shouldn’t be putting people through that arduous process.

Jessica Sahagian: Focus on, in your interviewsm focus on thinking and process, not perfect results. An example for engineering, I would give someone a code example. I’d walk through it with them. Why did you make these decisions? A candidate who’s neurodiverse may have a better idea, and may not have the the right answer at the end, but if you’re grading based on what your idea of right is, you’re excluding many talented people who may have a different way of getting there, and may be thinking outside the box.

Jessica Sahagian: If possible – I’m gonna harp on this a lot – Positions should be remote with an option for in-person if that’s how you work best. Focus on getting the most out of your workforce, not being you know, very prescriptive about where and how work gets done. That just ends up being good business sense. You know, you do have some positions that you need to have. And if you’re hiring for a factory position, people need to be in person unless they’re remotely operating a machine. We’re talking about positions that it can be flexible and it doesn’t require it. Have that be an option for folks to come into the office. Offer a remote option if possible.

Jessica Sahagian: Also, a big one that I harp on, despite being a person who constantly gets degrees because I love school – Don’t require degrees for certain jobs unless you need a medical doctor, in which case probably want them to have an MD. <laugh> Degrees often mean little, and they exclude not just neurodiverse folks, but all sorts of folks from underrepresented and marginalized backgrounds. This is how we lift people out of the cycle of poverty. By the way, if you can do the job and you’re a good human, you should be hired. Also, make sure in the hiring process that you’re asking about accommodations for everyone, when you have that initial screening phone call, ask, “Can I provide any accommodations to you on this during this process?” And give those without any sort of judgment or without marking something on someone’s resume that they have a check mark against them.

Jessica Sahagian: All right, so we talked about hiring. You’ve made a neurodiverse hire and now you want to retain them. Excellent. How do you do that? Focus on the following. Stim acceptance, a sensory friendly workplace, clear adaptive goals and asks, agendas – again, agendas on every meeting request, -and again, remote if possible. I do wanna dive a little deeper into the first two.

Jessica Sahagian: I mentioned stim acceptance and sensory friendly workplaces. What is stimming? So stimming by definition is a repetitive performance of physical movements or vocalizations. It’s usually a form of behavior by persons with autism or other neurodiverse conditions. It’s considered self-stimulation and it can serve a variety of functions. It can be calming. It can be an expression of feelings. What does that look like? Popularly? It looks like fidget toys, right? They became those, those little poppy things that everyone has. The fidget spinners. That became popular culture and that’s what became pervasive. But it also looks like picking at your nails. It looks like picking at your skin. It looks like a condition called dermatillomania, where you pull out your hair.

Jessica Sahagian: Repetitive hand stretching, and flapping, and consistently repeating oneself. I’m a nail picker. I’m a skin picker. I’m also a word repeater. Me repeating my words doesn’t mean I don’t think you understood me. It also doesn’t mean I don’t know what I’m talking about. It’s how my brain reinforces ideas and thoughts, and it’s a totally acceptable way to communicate. We need to create environments where we don’t get angry or irritated at folks who display these behaviors and push our own assumptions onto them, but where we assume that positive intent, unconditional positive regard, we want it, we should give it.

Jessica Sahagian: Also, please don’t stare – I have sensory issues with clothing and footwear. A lot of people on the spectrum do. I wear open toe shoes year round, which in Boston gets me a lot of looks <laugh> but when I wear socks or close- toed shoes, I get the heebie-jeebies, I literally feel like my skin is crawling. And in a previous job, I had a woman who was a skip level manager, give me a coupon for a shoe store and tell me I should consider getting heels to look more professional, despite the fact that her favorite direct report wore flip flops constantly. The only difference between myself and her direct report, I was on the spectrum and her report wasn’t. Other than that, we had the same competencies, the same skills. Mine were just constantly overlooked. We can do better as women supporting each other. Anyways, off that soapbox, let’s talk about the next thing.

Jessica Sahagian: Sensory friendly, quiet rooms are important. Let’s talk about how we can, I know a lot of times our offices now are all open. We have open architecture, there’s no not a lot of offices, and everybody’s out there in the middle talking constantly – that can be really loud. Try to offer quiet rooms. Try to take some conference rooms and put up – they have these things that kind of look like gym mats. Put them up around the outside so it creates a noise barrier so someone, if they’re feeling overstimulated, can book that room, go into that room, and be able to work, if they choose to come in, in person, Again, how do, how can you help this? Remote work. Somebody can go to a different room in their house, they can go to a coffee shop. If they maybe need more stimulation, they can go into their bedroom if they need a more sensory friendly environment than sitting at their desk. Give folks options. That’s really the key.

Jessica Sahagian: We talk about a lot about surviving. I wanna get neurodiverse folks, I wanna get us all really, to thriving. How do we go from surviving to thriving? And how you, as a neurodiverse person or someone who cares for a neurodiverse person, maybe your child is neurodiverse or a relative is neurodiverse. How can you put your best foot forward in a work environment or encourage somebody you are a caretaker to or a friend of? How can you encourage them to, how can you coach them to that? I have a couple tips and tricks that have helped me over the years. While I chat about them, I’d love to hear from ND folks on the call or folks who care for ND folks. If you feel comfortable, please share your strategies in the chat.

Jessica Sahagian: Okay, so my first tip, I use the old improv standard of “yes, and.” The idea being here that if someone asks you for something and is a neurodiverse person you feel afraid to say no to, but you know, you can’t complete and the time-frame desired, you use the “yes, and.” For an example, if I worked for an airline and my boss had just told me, you need to build a new airplane in two weeks. I’d like to say that is a ridiculous request and you’re not getting that. But what I would say is, yeah, “I’d love to build you a new airplane in two weeks – and it’s gonna be made outta popsicle sticks.”

Jessica Sahagian: This isn’t just an a neurodiverse tip. By the way a lot of us, especially as women, we get overwhelmed by being given extra tasks on top of what we’re currently doing. And we often feel that pressure to take on this work. Hit them with “yes, and.” Another example, my boss came to me and said, Hey Jess, can you take on X, Y, Z project? I would respond with “yes and an ABC project will have to be reassigned to someone else. Who should I sync with to make that happen?” Or,” yes, an ABC project will need to be reprioritized as a result. Its timeline is now moved. Can you please communicate to to stakeholders?”

Jessica Sahagian: You are in the driver’s seat. This is a theme we’re seeing throughout the day today, right? And we’ll see tomorrow. You are in the driver’s seat. That leads me to another tip. Lean into your neurodiversity. Don’t pretend you’re not living with it. I did that for a long time and it was a detriment to my success. Find the systems, tools and processes that work for you. This could be anything from automated note-taking software to the short meeting feature in Google Calendar, and I think Outlook has it too. Instead of scheduling a meeting for an hour or 30 minutes, it’ll schedule a meeting for 25 minutes in 55 minutes, and that gives you that five minute buffer to context-switch, to breathe, to get up, to get the synopsis firing again to, to go get some fresh air. And so work with these things that these tools and processes that are built to help you.

Jessica Sahagian: Pick a career that works with who you are, not against it. Personally, I get bored easily. I need a career where I get to have broad influence across an organization. I need to be able to dip my feet into engineering product, product management, business operations, and so on. That is what keeps my synapse is firing <laugh>. And it’s okay. It’s okay that I’m not quiet and dainty and satisfied with the status quo. It’s okay that I wanna climb the corporate ladder and run a startup. It’s okay that I’m ambitious. I learned being neurodiverse that I had to retake those reigns and steer my life in career in a direction that aligned with who I was instead of trying to fit fit myself into non-Jess-shaped situations. Like Stevie said this morning, own your story.

Jessica Sahagian: Another tip, project management software isn’t just for projects. I know we’ve all probably all heard of Jira, Asana, monday.com, Trello. I use a Kanban-style list keeper for my house and personal life to-do list. It has alerts built in. I’m a mom. I run a couple businesses. I’m also an employee. Oh my goodness. I have a lot of things going on. And it has alerts built in. I’m gonna forget something. It alerts me. I set these systems up. I automate these things to help myself. And an engineer, I can build things that help myself.

Jessica Sahagian: I so desperately wanted to be the type of person that could maintain a bullet journal. And if you guys know what bullet journals are, it’s like these very colorful, beautiful journals and everybody has like seven different colors of pens and they like do these sweet little doodles and bullet points and write, and everything’s color-coded and they check it off and it’s just so beautiful and perfect. So pretty, so neat. It’s not me. I’m not a bullet journal. I’m a massive handwritten chai-stained paper with doodles <laugh>. And again, that’s okay. I think it’s really powerful to know thyself, right? And to craft that life and career that goes with who you are, and not society, or your parents, or your family’s, expectations of you.

Jessica Sahagian: Final tip: schedule blocks. We’re talking about owning your stuff, right? Own your time. If you don’t work well with constant contact switching – I don’t – block off time in your co calendar every day where you can do heads down work. A lot of us can end up in those back-to-back meetings – every single day it seems like it can be that <laugh>. And you need to be a hundred percent on for all of them. That’s not sustainable. So block your time. Ask meeting organizers where you’re needed versus optional.

Jessica Sahagian: Your time is just as precious as everyone else’s. And remember one thing as as a neurodiverse person as you attain these goals, as you get promoted, as you get to higher levels, lift as you climb. That’s a central theme of Girl Geek X and Elevate. And it should be a central theme. I I think we’ve all seen the recent picture of Jamie Lee Curtis cheering you know, be the hype woman for other women. Lift as you climb. To whom much is given, much is expected, right?

Jessica Sahagian: Let’s be each other’s hype women across every spectrum, neurodiverse or not. So that’s pretty much what I wanted to touch on. I want to give everyone time to ask questions. We can have a dialogue. I appreciate you having me here today. This is a huge honor, by the way, to talk about this subject that’s near and dear to my and obviously many of your hearts.

Jessica Sahagian: I’d love to connect with folks on LinkedIn. My LinkedIn is up there. But in the meantime, I’m gonna check the chat and let’s see if we can answer some questions. Uh.. “currently work at the first place where I feel I can be authentic and open about ASD” – that is so important. I am as well. I’m at a company that allows me to be me and come with a hundred percent me to every interaction and every every meeting. And you know, I feel like I don’t have to dial myself down, down. I don’t have to be 70% Jess. My best friend described me as a brick of glitter to the face. I can be a brick of glitter to the face and it’s okay, you know, I can have that. And, I can also I can have days where I’m not my best self and nobody’s going to blink an eye, because they have those days too, and they they accept me for me, and I seek, I think, again, acceptance above tolerance. Just tolerating someone.. I never thought that was a good baseline. I thought tolerance is like, oh, I tolerate you, but I don’t accept you. Let’s get to acceptance and celebration. If you connect with me on LinkedIn, I can send you a lot of different resources. I would love to do that.

Jessica Sahagian: Outlook is my executive functioning tool. Yes. Schedule everything. I schedule everything in my Google calendar. I send alerts to my husband <laugh> he probably gets alerts. He doesn’t, he’s like, what is this thing? Whose birthday party? Whose child’s birthday party are we going to? But use the tool. There are free tools out there. Trello is free. Iit’s very easy. There’s an app called AnyList on my phone and I use that for my grocery list, but categorizes everything into dairy, bakery, fresh fruits and veggies, freezer.. so I literally can go to the supermarket and plan my route instead of just aimlessly walking around and being like, I guess I’m just gonna swipe the entire shelf of goldfish crackers, right? <Laugh>

Jessica Sahagian: Questions. “I’ve been a tech over 10 years in the process of getting an diagnosis and hoping to get more insight. Thank you.” So getting a diagnosis getting diagnosed with anything as an adult is really hard as a child. That’s easier as a woman, it’s near impossible. But there, there are places that do it. I do know if you’re getting an ADHD diagnosis, the Hallowell Center in Sudbury, Massachusetts does it. I don’t know if they do remote. I think they might have a branch in California, but it’s very difficult to get those diagnoses is an adult because a lot of places don’t accept, like I said, insurance for them. But there’s also a place that diagnoses autistic adults in Massachusetts. It’s called the Lureie Center and I think it’s division of Mass General Hospital, and I believe they also have outposts around the country, so definitely check them out.

Jessica Sahagian: Yes. Advocate for take home interviews. Have a take home test. Totally fine. Live interviews are nerve-wracking and I think that having that ability to take it home and to complete it in a time-frame, make the timeframe reasonable. Say, you know, if you’re asking somebody to de develop something, say, can you get this back to us in a week? And what you’re really looking for is not the right answer. I’m looking for how somebody thinks. I’m looking for, “Can you think critically?” I don’t care if you get the right answer. We all get the wrong answer from time to time, or a lot. I’m wrong a lot, but I like to think that my thought process is, okay, let’s think about, I wanna know if somebody is actually attacking the right problem. Does this person know how to know how to get to the root cause, or are they just trying to solve symptoms?

Jessica Sahagian: And do they ask, do they go through a five why analysis? Do they understand whether they do it formally and say, I know the five why’s. I know a Gantt chart. You know, they don’t need to know the names of it, but are they a natural? Are they naturally curious? Are they, you know, do they have that proclivity for “I wanna dig into a tough problem and work on it and learn.” The aptitude for learning. The ability to learn and collaborate is chief among anything. They took me from finance and turned me into an engineer, a software developer. If that can happen to me, anybody can, anybody can do it, right? Anybody can switch careers, anything can be taught, really, anything can be taught and a lot of things out there because of the, the one, the wonder and the horror of the internet, we can learn anything, right? We can learn anything for free. And so I think that really opens up careers to a broad swath of people who may not know the exact technical terms for something, but they can explain to you how it works and why, and they can reason. And I think that’s really important. So yes, absolutely to take home.

Jessica Sahagian: “How can we speak to our passion, fast talking and wide range of knowledge to articulate and focus our delivery of our capabilities as it can get lost in translation when limited on time?” Hmm. I would just be honest. I would say, I talk really fast and I know a a lot about a lot of different things, and I’m excited to talk to you today. And I might stumble, I’m gonna be nervous, and I’m going to say a lot of words, and if you have any questions and if I can make anything more clear for you, please ask me. Start with that. Own it, own it. Say, “Hey, I’m on the spectrum. I’m ADHD. I’m really nervous for our call today and I’m probably gonna stumble a little bit. Is it okay with you if we circle back at the end and I can clarify anything that maybe didn’t seem clear?”

Jessica Sahagian: “What are my favorite or life organizational tools?” So that AnyList app is really good. Trello, JIRA, but Jira costs money, so don’t do that. Get a free Trello. And I mean, I’m really bad at order. I have a calendar in every single room and each one has different things written on it, so I’m really not the person to, I am not a person to come in here and say, huh, I have discovered the fountain of of brain youth and I’m going to bestowed upon you. I’m still struggling and working through it, right? I’ve always been, I have a ton of journals, I have a ton of planners – paper planners – I’m a paper book person. I’m a writing person, not a typer. And I desperately want to be the type of person that keeps a journal and remembers to keep that and I don’t., so I would say get the AnyList app, get the Trello app if you are more of a digital person, and definitely use those to your advantage.

Jessica Sahagian: Productivity apps? Just mentioned those.

Jessica Sahagian: “What should tech company employee resource groups ask for in terms of information on how to better accommodate and neurodiverse employees?” Well, so it depends. Every company should have a neurodiverse advocacy group. I am the co-lead of the, what’s called it ConnectRN “the Brain Trust” and it’s neurodiversity and mental health. And so we are able to advocate for ourselves. We’re a group of folks who do do that advocacy. I think starting an employee resource group for neurodiverse folks is a great way to do that.

Jessica Sahagian: “I have trouble, I have ADD and I have trouble staying focused or getting motivation after the initial splurge of passion wanes. Any suggestions?” Oh, Sarah, I understand my friend. I have a closet full of hobbies that <laugh> that. I.. gosh, embroidery was a wonderful three month journey. If I’m not immediately good at something, I tend to put it away. I was actually not too bad at embroidery, but it hurt my fingers, and so that got pushed aside in favor of some other things. My spark gets re-lit when I look at things on Instagram so what I try to do is, I try to follow people who do the thing, or like a Reddit thread. After my passion kind of waned for that. I joined an embroidery subreddit and I see all the beautiful things other people are making and I’m like, oh my gosh, I wanna do this again. Not that I have time for you, but I wanna do this again. I think like actively seeking out and saying like, “Hey, you know, I really enjoyed this at one time, I’d like to get back into it.” And seeking out the beauty in it, whatever that is can often be enough motivation to start again.

Angie Chang: Awesome. Thank you so much Jess, for answering all of the attendee questions. We’ll continue to be chatting the next session’s coming up soon. I wanna say thank you again. We’re gonna wrap up this session. Jess, if you wanna eventually share, I don’t know if people ask for slides or resources. If you send in one person, can you copy me on it and then I can put it somewhere, somewhere people can get it or something. Thank you so much!

Jessica Sahagian: Absolutely. And if folks co connect with me on LinkedIn, happy to answer the questions I wasn’t able to get to.

Angie Chang: Thank you.

Jessica Sahagian: Thank you.

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

• Check out open jobs at our trusted partner companies.
• Watch all ELEVATE 2023 conference video replays!
• Does your company want to sponsor a Girl Geek Dinner? Talk to us!

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

• Check out open jobs at our trusted partner companies.
• Watch all ELEVATE 2023 conference video replays!
• Does your company want to sponsor a Girl Geek Dinner? Talk to us!

Sukrutha Bhadouria: Hi everyone. We’re back with the next session in Elevate. We hope you’re having a wonderful time so far. We now have Luiza who’s going to be speaking to us all the way from Brazil, depending on where you are all the way or right next to you. <Laugh>

Luiza Pena: Yes.

Sukrutha Bhadouria: Lead formal verification engineer at Cadence Design Systems in Brazil. She enjoys working in the semiconductor industry, driving usage, success and business across several top-notch US based companies remotely. In the spirit of International Women’s Day, we love to hear that she has worked with mentorship and career counseling, volunteer projects for women all across STEM. Welcome, Luiza. We’re so happy to have you here.

Luiza Pena: Thank you very much, Sukrutha. I’ll go ahead and get started here. Good morning, good afternoon, or evening, wherever you are right now in Brazil. It’s 6:30. It’s evening already. I’m very happy to be here to talk about this topic on the four allies you need to boost your career. I want to start this topic with a short story. If you watched Rebecca’s presentation earlier today, or maybe if you visited one of our booths in the virtual lounges, you probably know more about Cadence. But in a summary, Cadence is a tech company in the electronic design automation industry (EDA). It provides software, hardware, and system solutions to customers, mainly in the semiconductor industry, but also automotive and many others. And my role at Cadence is an application engineer, and I specialize in our formal verification of hardware tool called the Jasper.

Luiza Pena: I can say that Cadence today has over 10,000 employees, and the majority of this number composes those big R&D groups that we have across the globe developing different tools as part of Cadence’s portfolio. But as an application engineer, I am part of the sales and field operations group, and my goal is to make sure that our customers are properly using the tools we develop and they are also successful in their project. My group is like a top-notch customized platinum support. Let’s see.

Luiza Pena: My organization, even though it is not as big as R&D, it is still pretty large, maybe over 2000 people. And in this sales and field operations organization, there is a very important performance award given every quarter, two application engineers directly from our corporate vice president. It is a pretty big deal. And then one quarter while I was still, let’s say in the mid junior level, maybe I had two years in the company, I won this performance award and I was pretty happy about it, but also very surprised.

Luiza Pena: It was the first time someone from Brazil won, so as many of us women do when these situations happen, I asked myself, wait, why me? Then I asked my manager about it and he said that he’s, he just nominated me to our group director and he didn’t know how it goes after that. And well, that made me think a lot because if this award is given by the chief level vice president of our organization, it went through group directors, VPs, general managers and many other levels until we got there. How did I go so far? Maybe it was luck or something, but I didn’t believe that.

Luiza Pena: I started by reflecting upon what was the impact of the project that I was leading at the moment in your work context, positive impact maybe is providing the best technical solution or create new products, or maybe it is related to team building and developing the talents of your company. Maybe it is regarding effective management. Strengthen the company’s business position by making sure that the others are delivering what is important, right? And what all of these activities, whatever you do, can be or not relevant for the company. It depends of course, in your own competence and capacity to exceed the expectations, but it also depends on the opportunity that are offered to us and the projects that we are participating in.

Luiza Pena: Business acumen is something very important here. If you have it, it’s much easier to make decisions in a more conscious way when opportunities are reachable to you. Business acumen is also defined as just general business understanding. I like to think that it is the ability to take a big picture, view of a situation, weight it up, and make decisions that impact positively in the company. And to do this, you need to have a good general business knowledge of your organization and the market that your company is inserted in. You also need to understand how your team, your group, and the project impact on the business. Second, you need to develop important abilities that help you understanding cause and effect and make good decisions based on that.

Luiza Pena: You can ask yourself, is this project I’m working or this feature that I’m developing going to have a good positive impact in the overall company’s business? And finally, networking awareness. This is the focus of this presentation. While I’m pretty sure the previous two topics are covered by other presentations in this conference, right?

Luiza Pena: You need to build self-awareness by understanding your position and where you can go in the company – what are your strengths and where you can get with these skills that you can stand out? And you always need to work on your personal branding so people see what you’re doing and the results that you’re getting. And you need to understand who are the key people that are going to help you amplify your impact and take the next step in your career. This is stakeholder’s awareness and I like this idea a lot.

Luiza Pena: We need to be intentional about forming relationships and building our networks. This is something that men are doing all the time, and sometimes it’s not clear to us women that we have this power at our hands. And the question that we have here is, who are these people that we need to proactively connect with?

Luiza Pena: And then we reach the four allies idea. This is a concept that is widely spread in the corporate world. You can search about it on the internet and see different opinions about this same topic, but the general idea is that by connecting and collaborating with these four stakeholders, you can develop your knowledge skills and boost your influence.

Luiza Pena: The first of them is a mentor. Mentors are the ones who support, guide and advise you. This is a very flexible role, it can be someone in your company or out outside your company and can last like long or short. This guidance can be either technical or personal or even develop a soft skill that you’re lacking, but they are the ones that are going to share your joys and challenges professionally. Hopefully you are going to have many of them through your career along different levels of experience. Okay?

Luiza Pena: The next line in our list are the sponsors. Sponsors are the ones who provide you with different types of opportunities. These opportunities can be either a promotion, giving visibility of something that you’ve done, networking events – so they are generally part of the organization different from the mentor. That is like a more flexible role. They can be maybe a supervisor or manager or a senior level for colleague that can open doors for you. They are going to fight for you in forums that you don’t have access to, but they expect you to work hard and reflect positively on them as a result, since they are giving you opportunities. And because of that, sponsors usually give you honest and direct feedback, including the tough stuff. And I like to say that my mentors have made me the person I am, but my sponsors have gotten me to the position I am in today. I guess this is the main difference between the two of them.

Luiza Pena: And we go to the next ally which is the role model. Role models are the people who you may or may not know well, might be like a famous celebrity or something, but they demonstrate talents that you want to emulate and act habits that you want to develop. As for me, I have role models for technical expertise, mindset, or a ideological position, or they just have astounded me with their tireless advocacy for women and other important causes that I also fight for, or maybe they have shown me how to handle with a diversity, with resilience, assertiveness, and grace. This is also an inspiration for me, and they are crucial for your growth, even sometimes you don’t know these people, but because developing yourself as a person will make you proud and give you strength, inspiration, and motivation to take the next steps and succeed. Okay? They are also very important.

Luiza Pena: Last but not least, the best friend at work. <laugh> This concept has garnered some criticism, but in my opinion, there is no doubt that a friend at work help you. You know, it helps the day go by more quickly because they give you emotional support. They can boost your confidence on taking important steps and decisions in your career. And the message that I want to give here is that if you are lucky enough to find this person and truly connect with someone at work, don’t hesitate to do that just because they are in the workplace, okay? Because having this person is also very important for your career development.

Luiza Pena: And if we go back to our initial story, these are my guesses of what I think the stakeholders of the story thought about my nomination of that important performance award. Okay? Keep in mind that the idea here also applies to an important decision that your superiors are making about your career, like a promotion or deciding to give you a new project. And first, with my manager, he is the one who knows my job, right? He probably decided to nominate me and provided a lot of technical details to pass along. And when they came to our director, he still understands the technical aspects of what we are doing, right? But he probably thought that it was impressive. And by promoting that it will also promote his team overall. When it goes to a general manager or a group director, they might not even know who you are, and they are probably more focused about the project relevance or the customer importance.

Luiza Pena: This is why business acumen is important here, because you should be asking for opportunities matching two things you believe you can truly contribute to them with your skills, and you believe they have a good impact in the organization, and it is a relevant activity, okay? But the real magic happened here.

Luiza Pena: The vice president of my group was the person who idealized the creation of my team in Brazil some years ago. We didn’t have application engineers in Brazil before, and it was her personal project and showing the impressive things that we were down here would also reflect up upon her own success. She was the one who developed the talents here, and look what these young talents are producing. This is like great for her organization and showing the good work that everybody was doing. I think this was the most important part here.

Luiza Pena: And finally, in the very upper executive levels like corporate VP or a chief level officer, they will trust the judgment of their leaders in general, and they will always try to match the company’s culture and business. This person particularly is an amazing advocate for diversity, equity, and inclusion. And they believe in promoting young talents as it is also important for Cadence as a company. That might have impacted in the decision. If we translate this story, we can say that for your direct management is your hard work and your excellence that really matters, okay?

Luiza Pena: And for a unit level, you need to somehow exceed the expectations and do something that raises the bar for the team. And again, Rebecca mentioned it about knowing what you stand out and what are your key qualities really helps fighting your way here to, exceed the expectations. For the group level is the project relevance and a matching with the group’s goals that is going to matter. For the VP level, then it is business impact, customer impact. And in my case, it also had a matching with the goals of the the VP. And finally, in addition to business impact, the C level is going to be also worried about alignment with the company’s future and goals.

Luiza Pena: In this story, I can say that my manager was my mentor because he helped me a lot guiding me on leading this project but he was also a sponsor by doing the first nomination after all right? And I can say that my personal influence alone can reach from direct management to unit level maybe. And I want to mention that I had a very important friend during this phase that helped me go to through very challenging situations. And finally, I guess my main sponsor in this story was the VP and her influence was the one that made the nomination go through all the levels. Okay?

Luiza Pena: I think this is the important part that I could understand afterward, and I was doing the right thing by not knowing that I was doing the right thing. But it is very important to be conscious about this. And finally, one challenge that we have to connect with these four allies is when we work remotely. I’m based in Brazil, but most of my colleagues and customers are based in US.

Luiza Pena: Since many of us are remote, it’s even more important to embrace the opportunities to build our network within the company. One thing that we suggested to our manager, for example, is to pair juniors with seniors in our group in a official mentorship program, so every new person who joins the team has weekly or bi-weekly meetings with a senior colleague from a different location for a period of one year. This is official, and that encourages people, you know, to connect.

Luiza Pena: Another idea is if your company doesn’t give periodical updates on company’s goals and business position, you can ask for example, your manager to give like a quarterly update on what are the plans for the, the project that you are developing and the overall situation of the business. This is something that you can plant some seeds with some key people to provide more information to build your business acumen and finally be interactive with your peers. The opportunity to communicate are, you know difficult when we have this remote or, or hybrid work. Whenever you have the opportunity to have like a one-on-one or a meeting with a manager or a senior person in your team, make your intentions clear. Being assertive in your one-on-one communications is very important and show your intention on taking the next steps in your career.

Luiza Pena: Sometimes they make assumptions and they don’t even know that you are considering taking the next steps. So make your intentions clear so you they can you know, remind about you when these opportunities show up. And finally, don’t miss the in remote or in-person connection opportunities. My current most active mentor, I met him in the World Cup chat in the internal chat in the company. And we started, you know, talking there about football. And then I realized that he had awesome skills and then I asked him if we could meet like monthly just to discuss different topics. It was really out of the blue, but a very nice surprise.

Luiza Pena: To finish, support the women around you, mentor or sponsor if you have the opportunity. And maybe they are more approachable if you’re seeking for mentorship or sponsorship, but our presence in the STEM communities is still not as expansive as we wanted. Be open-minded ask for mentorship from men also, because otherwise we would be relying with few people. We want to expand our influence and also reach to places that some women are not reaching today. This is what I had to share. It is a lot of information, but I just wanted to plant this seed and open your mind that you start paying attention on those stakeholders in your own company and whatever step you are in your career. Okay? Thank you very much for attending.

Sukrutha Bhadouria: Thank you, Luiza, for the wonderful insights people have absolutely loved it, and you’ll see it in the comments. Thank you everyone.

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

• Check out open jobs at our trusted partner companies.
• Watch all ELEVATE 2023 conference video replays!
• Does your company want to sponsor a Girl Geek Dinner? Talk to us!

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

• Check out open jobs at our trusted partner companies.
• Watch all ELEVATE 2023 conference video replays!
• Does your company want to sponsor a Girl Geek Dinner? Talk to us!

Angie Chang: With us today, we have Joy Ebertz, who’s a principal software engineer at Split. She leads the backend team from a technical perspective and prior to Split worked at Box for years. In addition to designing software and writing code and running, she writes a blog, which I’ve really enjoyed reading over the years. I’m really excited to welcome Joy here today. Welcome, Joy!

Joy Ebertz: Thank you Angie. Awesome.I’m excited to be here. Today we’re gonna be talking about feature flags. And I know Angie just gave me an amazing introduction, but I’m gonna go through this real fast again. First of all, who am I? As she mentioned, I write a blog on Medium. I like to read, especially fantasy. I like to bake, especially pies, and I like to run especially really long distances on trails. But none of that is why you’re here today.

Joy Ebertz: Who am I professionally? On my blog, I write about running, but I also write a lot about diversity, equity, and inclusion. I write about career topics and I write about tech technical topics. Over the years I’ve worked at a few different places. I was at Microsoft, a really tiny startup. I was at Box and I’m currently at a company called Split.

Joy Ebertz: I’m primarily a backend engineer these days I mostly work in Java. I’ve spent a lot of time splitting monoliths into microservices and thinking about microservice architecture. I’ve also spent a bunch of time over the years working on REST API design and thinking about APIs in general. And I’ve also worked at both Box and Split on authorization and authorization frameworks. And why any of that’s relevant to the topic of feature flags is that Split happens to be a feature flag company. We might happen to think about feature flags a little bit more than the average user out there.

Joy Ebertz: Today I’m gonna be going through just a brief introduction of what are feature flags and then I’m gonna talk through some of the typical use cases we’re gonna go through and then we’re gonna go through some of the more unusual ones. We’re gonna talk about removing dead code load and stress testing, evaluating tech costs, parity testing, and logging.

Joy Ebertz: First of all, what are feature flags? There’s a few other terms you might have heard floating around feature toggles, feature flips, feature switches, conditional feature. And for some really odd reason we happen to call them splits, I don’t know, but basically all of these are different terms to indicate something that is con code, like code branches that’s controlled biographical graphical user interface.

Joy Ebertz: For example, it’s gonna look something like this where you have an if statement that basically just says if the split is turned on, or sorry, if the feature flag is turned on, then you’re gonna do whatever’s in that first block of code, and if not, then you’re gonna do something else. Then on the UI side, you might be able to set it up so that you know, your QA testers get the on treatment, or maybe everybody located in Boston also gets that on treatment. And then everyone else gets the off treatment. And this basically lines up so that then your QA testers and your folks in Boston get routed to that first block of code while everyone else gets that second block of code.

Joy Ebertz: What are some of the typical use cases around these? The biggest one is releasing features. I’m gonna get a little bit more into depth than that one in a second. But the next one is production testing. By this, I just mean you’re able to turn on a new feature or a new fix or something in production for only, like I mentioned before, your QA testers or maybe only the team that’s working on it so they’re able to actually test things on your production environment with your production hardware without impacting all of your users at once A/B/C testing. Being able to test out multiple versions of a website to see what resonates best with your users. And I have the C in there instead of just A/B testing, because sometimes you might have more than two different variations, right?

Joy Ebertz: Like maybe you’re testing a red banner versus a blue banner versus a green banner. That might be an option that you’re doing custom packages. This is basically just being able to give different feature sets to different sets of users. You might have a free set of users and you might have a paid set of users and maybe even have like a business package or an enterprise package. And each of these sets of users have a different feature set that they can see. This is one way to control which users see which features temporary UI customization.

Joy Ebertz: This is one of our, our more interesting features. Basically it’s the ability to actually change things in your UI without changing codes. How this might be used is if, let’s say there’s a storm in Boston, then maybe your customer success folks can update the feature flag to put a banner on the top of the screen that just says expect some delays, thank you for your patience or something, right? And you can then target that to only show up for your customers who are located in Boston. And then I mentioned I was gonna talk a little bit more about releasing features. What are some of the things there?

Joy Ebertz: The first piece is separating deploy from release. Deploy is like the, when you’re actually shipping your code to the cloud, right? It’s when you actually put that code on the servers versus release is when customers start seeing a and how this is useful is because these are both risky, right? And so having them together, it’s really hard to tell which part went wrong if something does go wrong. But by separating them, it makes them both lower risk and it means that you’re able to figure out and quickly roll back from either one if something goes wrong.

Joy Ebertz: The next one is canary or ramped release. Instead of just turning on a feature for all of your users at once, you can slowly roll it out. You can say, first let’s try with the freezer user free users, or maybe we try with 1% of traffic, or maybe we try with a few companies who have agreed to beta test something and then you can get feedback and check your metrics and so on and so forth. And if things look good, then you can increase that until you roll it out to everyone. And I sort of alluded to this already, but it’s also a big off switch. So something does go wrong, it’s very easy to turn that back off and roll back to a known good state. Okay? That’s basic feature flag use cases that are fairly common throughout the industry.

Joy Ebertz: But what are some of the more unusual or advanced use cases? The first one I’m gonna talk about here is using them to remove dead code. First of all, why is your code dead? You might have a method that’s no longer referenced, and I’m sure a few of you’re gonna say, oh, but my IDE can tell me this, right? But maybe you have an endpoint that’s never called, so it’s hard to tell from an IDE with an the actual endpoint, you’re like, your API endpoint is called or not, or maybe you have a parameter input parameter into a method that just never has certain values. Like an if block never gets hit because it just never has that value and these can all stack on top of each other. It might be the case that you have a method that looks like it’s called with certain parameter values, but maybe it never has some of those because the endpoint that would’ve passed those is never called, or maybe the method itself was never called because it’s called from do endpoints, right?

Joy Ebertz: it becomes very, very difficult to tell which pieces of your code are actually in use. One of the obvious ways to do to like try to solve this is to just pepper your code with log statements, right? We at Box, one of our developers actually came up with a lot of this Dave Shepper, but he had a tool that would just go in every branching statement. You add a log line essentially, and it’s just saying this code is not dead. And you have a unique identifier on there so that you can find which actual line was hit later when you’re looking through your logs. The problem with this is since most of us are in software, we know logs are expensive. <Laugh>, especially if you’re peppering all of your code, this is gonna get very, very expensive, right?

Joy Ebertz: One way to solve this is you can wrap the whole thing in a feature flag, right? And then you can just maybe only turn this on for 1% of traffic, right? And then maybe even only leave that on for, let’s say five seconds. And then you can go back to your logs and check which lines of your code were actually hit. And then when you go in there, you can then start sorry. And then you can remove those log lines from your code and then you roll it out to a little bit more and then remove more log lines and eventually any of those log lines that are still in your code are probably not in use anymore. And so you can roll it out to a hundred percent and leave it there as long as it makes you comfortable. For most of the features that we have, this is, you know, probably a couple of weeks maybe a month at most.

Joy Ebertz: When I was at Box, at one point we were working on some authorization code and looking at removing some of that and we were permissions, people get touchy about permission, so we actually left that one on for, I wanna say a year because we weren’t quite comfortable removing it until then. But anyway, once you’re comfortable, then at that point you can assume any of those log lines that are still not showing up in your logs are not being hit anywhere and you can go ahead and remove that entire code block.

Joy Ebertz: Okay, next, use case load and stress testing. I know a lot of, you’re probably familiar with tools such as Gatling or JMeter, and these tools are great. They allow you to basically write a set of test cases and run them repeatedly at like higher and higher rates against your systems, and this is a great way to tell or to load and stress tests. sorry, back up a second.

Joy Ebertz: Load testing is making sure that a new system can handle expected load. Stress testing is trying to figure out the point where that new system falls over. As you might be able to see by sending a bunch of test requests to a system, you can do both of these things. However, the big problem with those is you’re relying on a set of test cases written by a developer and it usually doesn’t reflect actual traffic patterns that you might be receiving through your site. Even if it does, you’re probably gonna have to spend a lot of time in order to get those realistic traffic patterns. One thing you can do instead is actually mirror your traffic and that basically means that you send your traffic like normal to your existing system and at the same time you send an asynchronous request over to the new system.

Joy Ebertz: And what’s nice here is that as I’ve kind of indicated with the arrow, you can start with that ramping like we talked about before. Initially you can just send 1% of your traffic to the new system and then slowly ramp it up. And what’s nice about this is then if your system new system does fall over, it’s much easier to see exactly at what point it fall over. And at least in my experience, it’s often much easier to debug what went wrong if you’re right at that inflection point rather than way over it. A lot of times when you’re way over, everything’s blowing up, everything’s falling over and it’s really hard to tell what the initial problem was. This allows you to find that exact point when things went wrong. And then stress testing. If you wanna go beyond your current traffic patterns, you might be able to start or you can start sending the same request multiple times to the new system.

Joy Ebertz: Maybe you sent it once and then you wait five seconds and you send it again. And this will allow you to send more traffic to the new system than you’re actually accepting your old system. And I fully admit that this is not quite realistic either, since you’re like, resending requests isn’t something that happens all the time with traffic, but it should still hopefully help with a slightly more realistic traffic pattern than you’re gonna be able to come up with in tests without a lot of time spent on those tests. Okay, that was load and stress testing. Next we’re gonna talk about evaluating tech costs. As you might have predicted, this is a very similar setup to that last one. The key for this one though is again, with the mirrored traffic, instead we’re going to just turn it on for a very specific length of time.

Joy Ebertz: Depending on what your traffic patterns normally look like, you’re just gonna pick something that’s representative. Maybe this is a week, maybe this is a day if your days look fairly similar – every site’s a little different, right? But the idea here then is, once you’ve figured out how much it’s gonna cost for that representative period, you can go ahead and then just extrapolate from there, and calculate what you think it will cost for the year, for example. One thing with this one, we’ve actually done this a few times at Split, you might wanna do it more the POC side of things. We’ve done this a couple times just as a, like, we’re considering this new technology, we have no idea if it’s a good idea, we’re just gonna like kind of gut check and you know, with a POC try to figure out what it might look like.

Joy Ebertz: But we’ve also done this… one of our more recent ones, we did this actually in pretty late stage. We were cleaning up an entire data pipeline system, replacing with something entirely new, and like we were pretty sure we wanted to go with this new thing, but we wanted to make sure we wanted this like last validation before we really started doing things. They weren’t going to completely blow the budget outta the water, right? It was a nice gut check at the end to make sure we were in the realm that we expected to be in.

Joy Ebertz: The next use case I wanna talk about is verifying parody or parody testing. This is also sometimes called tap compare testing and this is pretty similar. The other ones one of the key differences, while it doesn’t matter for the other ones, when you send the mirrored request. For this one, you wanna wait until you already have the response from your old system. And this is useful because then we’re gonna go ahead and send the request again to the new system, but we’re gonna compare the responses from these two systems to each other. And if they’re different, then we’re gonna go ahead and log it.

Joy Ebertz: And where the feature flag is useful is, again, because logs are expensive, right? We don’t wanna be spamming our logs and especially if something does go wrong, then we can go ahead and turn off the logging while we try to debug and figure out what happened or maybe turn it on just for a particular user while we try to debug exactly what’s causing the difference, while all other traffic continue can continue to flow to the new system to continue to load test it, but maybe not parody test it anymore. And this is one that we’ve used a few times at Box, when we use this, it was for our authorization system, we were replacing part of that, and we actually discovered that we had forgotten an entire feature, like an entire feature was something that somehow nobody noticed was missing.

Joy Ebertz: But because we had done this parody test, we caught it before this was the new system was rolled out to a single user, so we impacted zero users in the process, but we were able to find that we had a problem here. Another use case we had at Split recently is we were replacing a database with a completely different data system and it aas a nice way to ensure that the data coming out of the new database matched what was in the old one after we had migrated everything and done all of the various other pieces, it was nice to be able to verify that yes, the new data does match our old data and everything makes sense.

Joy Ebertz: Okay, last use case. Controlling logging. I kind of touched on this before, but logging is expensive. There’s this like tension between, do you log a lot and pay a lot of money versus do you log not very much and then when you know a problem happens, you’re totally in trouble, right? Like you don’t really wanna be in either of these situations, right? It’s kind of a problem in both cases, <laugh>.

Joy Ebertz: One way that a lot of people solve this is by basically, trying to reduce your logs, but then have a way to be able to through configuration, turn them on if things are going badly. And specifically, one way you can do this is through a feature flag. And I guess we’re currently working on doing some of this. We’re adding a library, like a wrapper library on top of the logging, which does a number of different things. But among the other things that it does is it enables us to put a single feature flag into that library that we’re adding. And this is nice because it allows us to quickly turn on and off the logging without having to push actual configuration in order to change those logging levels. And these are just to kind of give an idea of why that’s useful.

Joy Ebertz: In order to change the feature flag, and this is assuming that like maybe I need to get an approval on it and like, people aren’t quite instantly there. Like let’s say it takes 15 seconds in order for me to make a change and somebody else to come in and improve it versus this was one of our recent runs. It took 26 minutes of pipeline to get a change from, developer merged into our staging environment and then another six minutes to deploy that to production. And granted, an emergency situation, you might be able to bypass a few of those things in there, but we’re still talking on the order of seconds versus the order of minutes. And that can really matter when you’re talking about downtime or you’re talking about a real issue.

Joy Ebertz: The other nice thing about this is because it’s a feature flag, you can start to do more fine grain control too. So if you know that the issue is only affecting certain types of users or certain organizations, you’re able to then basically target only those people or only those organizations, which allows you, again, to limit the number of logs that you’re storing. But at the same time you know, you’re going to be able to get the information that you need or it might other be, might be other information maybe it’s certain profile types.

Joy Ebertz: Maybe it’s only affecting people in Boston which you can again use. Or it might even be that you happen to know that the problem is likely in the stack trace. So maybe you want to just log things from that stack trace and ignore all of the other, all of the other information out there. It’s also possible to do not just based on the user and the request, but also based on the actual classes and that side of things as well. So which, which pieces of code are you increasing the logging for? Awesome.

Joy Ebertz: This was a whirlwind trip through feature flags. Just a quick recap. I covered what feature flags are some of the typical use cases and then I went into some more interesting use cases around using them to remove debt code load and stress testing, evaluating tech costs, parody testing, and logging. I’m out of time actually, so if you have any questions you can go ahead and email me, or this is my blog as well.

Angie Chang: Thank, thank you, joy. That was excellent. I, like I said, I love Joy’s blog. I ever highly recommend to read it for both career and technical advice. And the feature flags topic has been always – I always see it on Twitter – so I’m really glad that this talk helped educate me on what it is. Thank you again, Joy.

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

• Check out open jobs at our trusted partner companies.
• Watch all ELEVATE 2023 conference video replays!
• Does your company want to sponsor a Girl Geek Dinner? Talk to us!

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

• Check out open jobs at our trusted partner companies.
• Watch all ELEVATE 2023 conference video replays!
• Does your company want to sponsor a Girl Geek Dinner? Talk to us!

Sukrutha Bhadouria: I hope y’all are having a great time so far in the conference. We’ve been having one great session after another. It’s been amazing for me at least. I do wanna remind everyone to please post on social media, all the amazing things y’all are hearing, learning, and listening to today. Use the hashta IWD2023 for International Women’s Day and the hashtag ElevateWomen because we want you all and us all to lift as we climb.

Sukrutha Bhadouria: All right, so now we have the wonderful Moriel, who’s a principal systems architect at Wikimedia Foundation in New York. She’s a physicist turned software engineer turned systems architect, currently work working on modernizing Wikipedia’s architecture. She’s also an open source enthusiast, and we are excited to hear about neutrality.wtf. Welcome!

Moriel Schottlender: Thank you so much. I’m so excited to be here. And it seems like I have gone from one thing to the other, to the other, and that’s I guess, part of changing, growth, I don’t know. But now today I’m gonna talk to you about neutrality.wtf, a case study in gendered language flipping tool, which is a mouthful but hopefully we’ll learn something and share something today. First of all, at me, I am originally front end development very much a open source enthusiast. I do a lot of localization work and support and languages. I hoard domains, which I assume a lot of you do too. And I’m a principal systems architect at the Wikimedia Foundation. We operate Wikipedia and I also apparently do a pretty good you know, self cookie that I was actually very proud of that one.

Moriel Schottlender: Let’s delve in. What is neutrality.wtf? We all know that the gender gap is real, but the gender gap is not just about how many articles we have about women or non men. The gender gap is also a little subtle. The gender gap is about how do we talk about both genders? And sometimes, or a lot of times, even when we do have articles about women, we tend to talk about them, not quite the way we talk about men, but it’s really hard to differentiate, to see it. And we hear a lot of people say kinda like, well, you wouldn’t say that about a guy, would you, wouldn’t you? I don’t know. And this is where the idea of that tool came up.

Moriel Schottlender: I was like, is there a way for me to kind of test this hypothesis to see if we flip around the gender terms and we read an article or read whatever it is online, can we identify problems or does it is not a problem? I don’t know. That’s where it came from.

Moriel Schottlender: This is what it looks like. This is the article about Ada Lovelace in Wikipedia. And so if you move it you know, pass it through neutrality.wtf, it’ll just replace all of the terms from male to female and from female to male. All of the gendered language will be just flipped around. And so now you can read about Ada Lovelace as if she was a man. And then now you can say, would I say these things about a man? And I’ll let you decide. I think, for the most part, probably okay, but there are cases where it might not be as okay. And this is the point of this tool to kind of flush those out. This is an example.

Moriel Schottlender: This is the beginning of the article matriarchy in Wikipedia. This is from 2018. The problem that I’m gonna show you today was actually fixed, which is very exciting. But I was thinking, okay, if I passed matriarchy through a neutrality.wtf, everything is flipped. Well, now it’ll describe patriarchy, which technically is the opposite, right? Would it look the same? Would it sound right? And so I did. And oh, it didn’t sound exactly right. There were a couple of things where if you went to the article about patriarchy, you didn’t see there, most notably in mammals, to the exclusion of females, probably not what we want. Good news is that these things were fixed, were removed from the article. But that is the point of this tool. To also clarify, we do have limitations and there are very known limitations. The first thing is gender itself is not binary, and this tool is not making any statement about the gender being binary.

Moriel Schottlender: On the contrary, there’s a lot of explanation that is exposing the fact that gender is not binary, but gendered language, especially in English, tends to be binary. This is only making the point about gendered language. There’s also a limitation about one-to-one replacement of language, which this tool is doing. This tool is dumb. This tool is just looking for words and replaces them, which means we have problems with language processing or language context. There are verbs versus nouns, for example, count can be replaced by Countess, Count Dracula, for example. But I also count from one to 10. Is that count test from one to 10? Probably not. The system is a little dumb. We do have those mistakes happen. The idea is to make it a little bit, you know, funny, also to make the point. We do know, this said, there’s also ambiguous words.

Moriel Schottlender: We lack context. And a lot of times the demeaning terms. So if you see something written about girls, when you actually talk about women, we know that it is demeaning. When we flip it around and suddenly talk about boys, it’s not as demeaning, so it might not always deliver, right, the actual impact, but it works.

Moriel Schottlender: I’m an architect. Let’s delve into the actual tech. How did this happen? Well, this all happened in the 2017 Wikimedia hackathon. As you can see, this is me in my telltale hackathon. Look, I am overly caffeinated, but barely awake, hackathon. This was an idea for the hackathon. We took articles from Wikipedia specifically. The tool works, by the way, outside of Wikipedia as well. You can do do any sort of article, but Switzer and PHP based on a one-to-one replacement, JSON dictionary, and it has a hackathon architecture.

Moriel Schottlender: What does that mean? Well, hackathon architect is a hopeful one. I started writing this and I was saying, I know that I’m gonna write something that I want to basically just have work and never deal with it again, so I want to make it right. I’m going to separate concerns. I’m going to create two repositories, one for the front end and one for the backend. I’m gonna use it as a submodule. I’m gonna like separate everything. And that was great, except it didn’t really work. While we were working on this in the hackathon, everything started kind of like mis-mashing together. The business logic was all over the place. The caching was everywhere. The front end was fetching and the business and the back end was kind of like, you know, interpreted. It was, it was crazy. It ended up being very, very, very, very tangled, which means it was a monolith in disguise.

Moriel Schottlender: I attempted to do decoupled. It was not decoupled at all. But look, as a hackathon project, it works, right? Like it’s just, it’s there. I put it out there. People use it, it works. It’s great. So why should I touch it? Well, it didn’t really work for me. The environment kept going stale. I had to keep going after kind of like upgrades and stuff and libraries. And then the caching layer was all over the place and started filling up my hard drive and moving to other things became really, really problematic. Managing the hosting was bad. And then monoliths in general, a very annoying because a little tiny bug, suddenly you have to fix everything everywhere because there’s no separation of concern.

Moriel Schottlender: I wanted to be lazy. I wanted to just let it live online without me constantly running after this. And the way to do that was to reconsider my architecture. Let’s think about this better, especially since it’s been a few years. Technology is new, there’s other options. Let’s figure it out.

Moriel Schottlender: How do you approach thinking about a new architecture to your thing? Well, the first thing is to think about what are the standards? What do I want? Like what am I aiming for, right? What I want is to make sure that it’s loosely coupled. It means that it’s made out of pieces that I can replace seamlessly. They’re not like directly touching one another so that I can, if something happens and I want to replace the middle layer or the top layer or whatever, I can just replace it. It needs to be easily maintainable. I needed to retain the same behavior, but allow for more features without constantly maintaining this. This was very, very important.

Moriel Schottlender: I had to make sure that I don’t run after everything all the time. And then there was another set of kind of standards that I wanted to add. This is open source. And because it’s open source, other people can come and just contribute. And that’s great. But it also means that I need to make sure that the separation of concerns are very clear. Because if someone else comes in and just adds code in, I need them to understand what is valid and not valid to add, right? And so it makes it everything a lot like easier to review and avoid diluting whatever I worked really hard for the con separation of concerns, even if changes, which is what I want to make sure, right? Like this is not a constantly like worked on project. I could leave it out there online and come back to it like months and months later. So I wanted to make sure that that works. All right.

Moriel Schottlender: Thinking about all of things, I came up with basically a three component, very broadly architecture. Okay? This is a plan. I’ll have a standalone Node.JS library. This thing is going to be relatively generic just to do replacement of terminologies inside HTML pages. And then I’ll have a microservice that takes the very, very generic and brings it into the specific, so it does all the translation needed to do a specifically neutrality.wtf replacing gendered languages thing. And I have a web front end that is completely decoupled and I can replace it any time. The standalone library, it has to be scoped and this is one of the biggest problems. We’re gonna touch on it in a minute. It has to be very, very well scoped. It does something very specific.

Moriel Schottlender: A black box, it goes in, goes out, HTML in, HTML out, does the replacement internally. It means also that it’s testable because it is very scoped. If I go in at some point in the future and I want to replace some functionality or add something, I can be sure that all my test pass and all my expectations are there, so it’s a lot easier than like a humongous monolith where who knows what I just touched on and what I did. And it’s open, so shareable. Some people can just take this library and do other replacements of other things. I don’t know. All webpages will have replaced cats to dogs or some images to other images, whatever it is. You have like a very kind of generic library or as generic as possible library to reuse. What it is due, it’s kind of a black box.

Moriel Schottlender: It accepts HTML string, it parses it into a document, performs all of the replacement. It’s accepted string and library sorry JSON replacement. It replaces everything. It serializes his backend, hands it over as a string. That is completely replaced. That is what it does. And then I have the microservice. The microservice needs to be the bridge between that backend, that like engine into the front end that can be like visible to users. And it does the product specific behavior. It validates all parameters. It does the fetching of the remote page because the library itself doesn’t know if it does fetching or whatever. It doesn’t know about HTTP or whatever. All it knows is about HTML documents. The micro server does it that. It handles all errors. It’s calling the library and it returns the output in a way that the front end can present it, which brings us to the front end.

Moriel Schottlender: The front end, straightforward. Decoupled. It can have multiple versions. I might have a web or mobile web or a browser extension, whatever I want, I could swap it around. And it needs to be accessible. It presents the interface. It has explanation pages on it. It accepts user input input, validation, all that kind of stuff. But then we need to consider complexity. There is an overlap a little bit between the microservice and the Node.js library here. What do we do about this complexity, right? Like I have overlap here. Which piece should do? What should the microservices always do the web specific stuff. It sounds right. But if I go with a very simplistic, then there’s a lot more complexity in the microservice, right? The microservice needs to do a lot more. If I go with a really complex library that can do anything, then now I’m risking making it unusable because it can do so many things that nobody knows how to use it, which we see online a lot, right?

Moriel Schottlender: But there’s another consideration here. The actual action of parsing and HTML string into a document is expensive. It takes time. And so if I do it twice, I’m actually doubling the amount of time that it takes the system to respond. I need to make sure that I only do it once, which means that if there are certain things that might be web specific but required to be done before I while I parse it, I might have to put it in the in the library, right? What do I do? You know, what happens? Where should the complexity live up, down in the middle? Well, like many architectural concepts, it depends. There is no clear answer here. I don’t have like a direct answer of this is certainly where you need to do it. What you need to do is consider your complexity and understand that no matter what you do, you never lose it.

Moriel Schottlender: People online say, I just reduced all complexity. You did not. You moved it somewhere. Okay? Either you did it consciously and you’re aware of the trade-offs or unconsciously and you’re going to encounter it later. When you think about complexity, you have to kind of think about it clear. What will help you with that is examine the trade-offs. What does it mean? Put it in here. Doing it more simple, less simple. Root yourself, right? With an actual use case. Yes, the standalone replacement library should be very generic, but it also is aimed at a use case. Root yourself with that. Because if you start thinking about anything it can do, you’ll never get away with it. Everything will be possible and then nothing will be possible. Root yourself with that. Consider your performance concerns. Those are external things that come at you that might need to kind of formulate where you put your complexity.

Moriel Schottlender: It dictates where that boundary exists. And that is totally valid. And then beware of diluting the actual meaning of the component you started out with. Like, you know, thinking about a microservice, a library and they have meanings. And if we lose track of where complexity lives, we can actually lose like the actual thing we want the library to do, right? We put so many things in it that now it can do so many things that it doesn’t actually do the thing we want it to do, right? Try to kind of like re remain with like what is the goal of this component? What do you wanna do? The trade-offs, the external stuff. And most importantly, document all your decisions. And this is important generally, but it is even more important when you have an open source thing that other people may come in and ask, why did you do this? Right?

Moriel Schottlender: With that said, here is the new architecture. As you can see, I have three main components to it. The web front end, Vue.js incidentally, but it could be anything else. There are clear boundaries here. The caching layer is very clear. The microservice does the operation to simplify the no, the the Node.js library. And I even added a little bit. If I want a browser extension, I can just use the npm package. It is generic enough to do that, right? And it is reusable. You can just use it. And this is what it looks like. You feel free of course to go neutrality.wtf to use it. It is open source. PRs are welcome. And that’s how it works. To summarize, always, always when you architect, when you think of code, even before you decide to re-architect anything, always consider your complexity.

Moriel Schottlender: Whoever tells you that you need to do something like complex misses the point that complexity always exists. And you need to do to have like a very conscious choice of where don’t just ask could it do this? Cuz that is very easy for engineers to do. We ask, could it do also this? And the answer is usually, of course it can. We’re engineers, we can do everything. Ask yourself, should it do this? Because if you get away at the end of it with a library or a component that does everything, then it’s probably not one component. It should be many or whatever you, you end ended up with like a tiny, tiny monolith again. So as so should do this it’s okay to want to be lazy. I know it’s counterintuitive. Everybody’s kinda like work hard. You should work hard, but you work hard.

Moriel Schottlender: And that’s true for engineers. If we actually think about it. Engineers are lazy in our core. We worked really, really, really hard on stuff to do stuff for us so that we don’t have to do them right? How can I end up having least overhead is a totally valid question that is architecturally sound. It is totally okay to do that. And then you decide what the trade-offs are so you know what to invest in. And it’s okay to decide, you know what, I’m going to take a little bit longer to do this thing but it will save me time later. That’s perfectly good. And that also means try to use external tools. Why make everything in-house? You don’t always have to. You have external tools. Preferring those usually is better. Not always. Sometimes you do need like very specific but make a conscientious choice.

Moriel Schottlender: It’s very easy to jump into directly. Oh no, nothing exists like that. Or whatever exists is not good enough. I’ll just create my own. That is usually what we call code smell. You need to really ask yourself if that’s actually what you need to be doing. And there’s another thing here underlying all of this conversation that I didn’t touch directly, but I touched a little bit. Working in the open, working in open source has a lot of benefits. A lot of benefits for the way that you work and for the architecture that you produce because it forces you to kind of think in a collaborative manner. Even if you don’t collaborate, even if no one ever produces a PR to your code, it doesn’t matter cuz you put it out there. So the first thing is that it kind of gives you permission to put it out there, even if it’s not really ready, you’re just out there.

Moriel Schottlender: And that’s important because we tend to only put out there things that are perfect. Don’t wait until it’s perfect, just put it out there. You get help from people. It forces you to put very clear boundaries on everything and it encourages you to document everything which is superbly important, even for yourself in a year from now to know why you just did what you did because you don’t remember. Trust me, I don’t remember what I did three months ago.

Moriel Schottlender: And finally, I think we don’t say that enough. Have fun with your projects! When you have fun with your project, it shows you’re excited about them, you’re passionate about them, and it shows also for hiring managers and for the public and for great products, and you keep on working on them. Find things that make you happy and passionate and work on those.

Moriel Schottlender: If you want more information neutrality.wtf is online. Feel free to use it, please. It’s on GitHub. PRs are welcome. Issues are welcome. And you can also talk to me. I’m “mooeypoo” almost everywhere, Twitter, Mastodon. And if you want to follow me around, go to moriel.tech to see what I’m up to. Thank you very much. It’s been a lot of fun. And go on and be passionate with your projects. Open source. I think I’m slightly ahead of time, which means I have time to look at questions. Potentially. There’s a lot of comments.

Sukrutha Bhadouria: Hi, this was absolutely wonderful. I was following the chats. <Laugh>, someone said, we all love your voice,

Moriel Schottlender: <Laugh>. Oh, thank you <laugh>.

Sukrutha Bhadouria: Anyway, I wanted to say thank you for, oh my gosh, I just noticed your handle. It’s so clever.

Moriel Schottlender: Ah-Ha. Yes.

Sukrutha Bhadouria: Yeah, your social media. <Inaudible> <laugh>. Anyway, thank you so much for your time. This was absolutely insightful and besides loving your voice, your content was amazing.

Moriel Schottlender: Thank you. Thank you. It’s been absolutely fun and I really hope that you know, we get more and more of these conferences. They’re really important and they’re awesome. I’ve been learning a lot,

Sukrutha Bhadouria: <Laugh>. Absolutely. We wanna keep this going. Thank you everyone. Bye.

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

• Check out open jobs at our trusted partner companies.
• Watch all ELEVATE 2023 conference video replays!
• Does your company want to sponsor a Girl Geek Dinner? Talk to us!

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

• Check out open jobs at our trusted partner companies.
• Watch all ELEVATE 2023 conference video replays!
• Does your company want to sponsor a Girl Geek Dinner? Talk to us!

Angie Chang: We’re here with Shanea Leven, the founder and CEO at CodeSee, a developer tool that helps developers and teams better onboard, refactor, and understand code bases. And before starting CodeSee, she led teams that delivered high quality products and features for leading companies including Google, Docker, eBay, and CloudFlare. Welcome, Shanea.

Shanea Leven: Thank you so much for having me. I’m really, really excited to be here. I’m excited to give this talk. As Angie said, I’ve been in dev tools for more than 10 years.

Shanea Leven: One thing that I know about the the experience is that it has always suck onboarding to a new code base. Every developer, whether you are at a new company or contributing to open source if your application isn’t hello world, you must onboard to that code base. And so therefore you must like learn the code base and learn how it works before you can actually modify it. And I showed up today with a secret <laugh>.

Shanea Leven: Are you ready? I figured it out. And the secret is, in order to learn a code base, you need to pair, you need to pair all the time for all time 24/7. And that’s the end of the talk. <Laugh> Mostly kidding. But for those of you who, for whatever reason <laugh> can’t pair every second of the day when you’re trying to learn a new code base, maybe, you’re just starting an open source or maybe you’re just getting getting your foot in the door in a new area, it’s not comfortable for you or you’re in a different time zone or you’ve got zoom fatigue.

Shanea Leven: I wanna share some tips with you all today and kind of make sure that you all know that it does suck and it does get better <laugh>. I’ll start by telling a story. This story is about Shane <laugh> and her and her first day at Acme. And Shane is, you know, bright-eyed and she’s bushy-tailed and ready to ship some code.

Shanea Leven: She’s ready to prove herself, but like I said before much like you all probably you’ve got, she’s got to learn the code base first. So pretty soon, Shane finds out that onboarding to this legacy code base of the company that she just joined totally sucks. It’s really hard, it’s time-consuming, and Shane is personally having a really hard time. Why is it in fact that onboarding is so hard?

Shanea Leven: Well, when you are onboarding to a new code base, this is all of the things that you need to know but, you know I have this saying that every code base is like its own unique plate of spaghetti. It’s not exactly the same every single time. But in general, some tips to follow should get you started. Tips are broken down into three major parts.

Shanea Leven: First, you need to know the code or the system. You need to understand how to work with work with the code or work with the team. And then finally, how do you just be a human? What is the reality of being a human being and how that affects your day-to-day life and engineer? First, you need to understand when you’re onboarding to a new company’s system you need to understand the code of the system.

Shanea Leven: You need to understand how actually the code is connected. What are all the execution paths? What are all the function calls? What other functions called other functions? What are the actual details? What is the high level architecture? What are all of the domain objects? What are all of the nitty gritty details when it comes to working with the team? You need to know what is the code review process?

Shanea Leven: What’s all the design patterns? What are all the testing patterns? What are all of those gotchas in the code? The things that you just kind of hold in your head or the team holds in their head, and it’s not really written down. What are all of the deployment procedures? And finally, you need to know and deeply kind of understand just the reality of being human.

Shanea Leven: As developers, our main goal is to shift things. It’s to write code <laugh>. It’s not meant to write everything down. And so when you don’t write all of these things down, when your team doesn’t write all of these things down it gets out of date and it’s very, very hard for you to understand. And many times that information is not in the code. Repetition and retention. Turns out that when you join a new company you’re trying to onboard to a code base for the first time, or you have a new team member who’s trying to do the same you’re only gonna retain about 30% of what you’re told in your first week.

Shanea Leven: Even though you know you’re likely going to really need to understand, or you someone told you, or you told your team member, you’re likely gonna need to tell them again. Turns out it’s really, really hard to learn code. The turns out that the way that we learn code hasn’t changed in about 50 years. The process of reading code and understanding in and of itself is insufficient. We read code one line at a time, and we imagine how our systems work. That’s what we do. That’s just how it has always been. There’s very little to help us make sense of the code. And the data could be abstracted with types, it can be completely missing if you’re writing, if your company happens to use an un typed language. And reading is actually the most manual way to extract information out of code.

Shanea Leven: Therefore, reading and understanding code then takes at least 60% of our time and effort, you know, everywhere from between like onboarding as we mentioned. But then you have to read in deep dive when you’re debugging, when you’re doing code reviews or when you’re doing future work. And then at the same time, we’ve got these huge, huge systems. Like we actually have 10 times more code than we’ve ever had. And those systems are constantly changing.

Shanea Leven: While you are trying to understand the code, your teammates are shipping code like at the exact same time. It’s literally changing from under you. No wonder that Shane, and probably, you all when you’re joining a new company is gonna have a hard time <laugh>. But that experience, your sucky experience, is literally about to get worse. Because onboarding unfortunately isn’t a single point in time, it’s constant, it’s constantly onboarding to new things. Here are a few examples.

Shanea Leven: During Shane’s onboarding, Tommy over here ships a new feature. And that feature needs to get integrated with Shane’s feature. Guess what? She needs to onboard to it. Then, you know, after a couple weeks, she gets added to an on-call rotation. And then at the middle of the night, your system goes down and surprise <laugh> when you are, are in the middle of the night and the system goes down. This is not a normal part of your area typically. You’re onboarding again, you’re just doing it at two in the morning. The only option at that point is to just figure it out when people leave. Because we are constantly always visualizing everything in our heads, those mental models that we keep go out the door when people leave, whether that person is leaving voluntarily or unfortunately in today’s market involuntarily.

Shanea Leven: They were typically the only ones who kind of knew how that cool widget feature thing worked. And there’s no documentation at all. And now it’s your problem <laugh>. You’re just onboarding again. In another case, restructuring you know, teams get moved around. An individual person, the flip side of that could get promoted. Like your org might be restructured and then therefore your team’s responsibilities have changed. You’re onboarding again. And in the promoted case, it depending on like how fast that team ships code, the person’s knowledge is actually gonna get outta date really, really quickly as they are trying to onboard to their new responsibilities. In fact, you’re just kind of stuck onboarding. Again big refactors, many people don’t know this or consider this, but when you’re trying to complete like a big refactor project, it’s just actually one big onboarding.

Shanea Leven: For instance, when you’re trying to refactor a monolith into microservices, you’re trying to like break things down, or you’re trying to, you know, go back from a microservices to a monolith or, you know, some gradation of service microservice, not so micro services. In between all of those things, you need to really deeply understand how all of the variations work, what are act, what’s actually kind of helping or happening under, under the hood. And so you need to actually onboard to the whole code base so that you can make sure that you know how all of the parts are involved and how they’re connected so that you can ensure that everything gets into the new system without breaking. And then finally every single time you do a code review when you’re working on your new team is like its own little mini onboarding experience.

Shanea Leven: Essentially you need to like read the code in your head. You’re trying to remember the related code and imagine how it all works together. And that’s just another mini onboarding. And then you are just taking that code and kind of blowing it out to years and years and years of legacy code, people coming in and out, maybe acquisitions, maybe someone completed that half refactor and there’s a bunch of teched. You’re just kind of constantly onboarding. You’re probably wondering at this point, Shanea, I thought you said that you would help <laugh>, so I’m gonna help you with some tips now.

Shanea Leven: Here’s what you can do to not have that sucky experience as much as you can. It’s broken up again into three parts. Advice for the team, advice for the onboarding, and advice for continuous onboarding. Again, advice is in general and there are always exceptions. You know, some people thrive where some other people struggle.

Shanea Leven: Advice for the team in general – Start small in scope and expand strategically. you wanna provide some small early wins. Choose some tasks that help your teammate build confidence and help you build confidence trying to require learning as few new things as possible for each task. Make sure that if you’re on the flip side, that you are trying to ask for, you know, compact things and make sure that you write things down. There’s actually a lot to learn when you’re learning simple things. It’s typically more than you realize.

Shanea Leven: Problem solving in a new code base is actually really tricky, and it’s sometimes really hard to feel confident that you’ve considered all of the edge cases. So goal is to have, you know, build your confidence as part of the new team. You can start them out in a small part of the code base as possible, learn all of the relevant pieces without getting overwhelmed. You don’t wanna start your new onboarding and a huge challenge that people have been avoiding. You can ask a lot of questions if to make sure that you are not falling into one of those cases as well, but also try not to stay on super tiny things for long. You wanna start small but build consistently. Whoops.

Shanea Leven: I like to have the first task to be updating the docs for something. There’s a lot of little details that you can use to familiarize yourself with just by creating the code review, filling out the code review template, getting it approved and merging it. You don’t have to worry about having to get feedback on your code. Also, correct anything that wasn’t right in the repo setup instructions or write them up if you know they don’t exist is another great way. You can write up a small and commit a small personal bio to the repo with maybe time zones, communication preferences like Slack or email technical experience or fun facts. It can be really helpful on our remote team and, you know, be creative to try to get through the whole process, see that whole process, get that out of the way, and then you can focus on your code.

Shanea Leven: Once you’re ready to get into the code, make sure that you don’t need to understand all of the details about how the entire system works to get the task done. That helps you to build confidence that you did it right in the first place. You can gain a bit of confidence just by be feeling like you’re an expert in some small piece. And then growing out from there, so few examples, a bug fix in an area that’s well tested a small team task that other folks know how to solve, but just haven’t gotten around to yet. You can walk through the code that they’re gonna touch and some of the patterns to use to change it. So all that you really need to do is get familiar, write the code and not have, actually have to figure out how to solve anything.

Shanea Leven:  Another example might be a very isolated feature with minimal risk of breaking anything. Like change a button on a link or have an endpoint return. One more attribute. Tech desk tech debt is really great. Early task, even, you know, some tedious things like upgrades or maintenance. As a new team member, you can feel like you accomplished something good and you’ve learned something and the team will really appreciate you for it. And then this is key. You wanna let each task kind of grow from there. Maybe more tasks in small tasks in new areas, but ideally, you know, slightly larger tasks in the same area. Again, your real job here is to help them feel or help yourself feel confident like you’re an expert in that area. And as soon as you can be kind of the go-to person the better.

Shanea Leven: Advice for you as the onboarding again. You’re wanting to look for a few specific things to kind of wrap your head around the code base when you’re thinking about the code and the system and how it works. So you wanna know what are the domain objects? What are the users in your system? What are the key flows?

Shanea Leven: Couple of examples might be you know, if you’re in an e-commerce platform, that might be understanding where, what, how do you define a product? How do you define the shopping cart? What’s the flow from pricing or discounts or adding something to the cart? In dev tools, it might be how do you define a code base? What is a dependency or a commit or a pull request in ed tech that might be, you know, an assignment or activity or a lesson or a grade.

Shanea Leven: And some user examples in your system for e-commerce might be like a customer or a seller. And in dev tools, it might be a developer or team lead. And in ed tech, it might be teacher or student. And then you wanna make sure that you understand the key flows. Key flows in e-com might be, you know, customer makes a purchase. What are all of those steps that they go through in the system? Seller lists the product. What happens in your architecture in dev tools, it might be, developer creates a new commit, what actually happens or system analyzes the code base, like what triggers what or what calls what, how do you understand those details? And in ed tech, it might be something similar like teacher creates an assignment or student answers a question. It might seem silly, but these are really good things to just get down and get a full end-to-end understanding of.

Shanea Leven: Some best practices you want to ask and be comfortable asking. Lots of people will know some of these things, but it’s better ask in the first couple weeks. Don’t be afraid and constantly ask questions because it will help you make your lives a lot easier. Exploring the marketing site might be a great way to kind of wrap your head around stuff. Oops. reading the docs. You might want to, whether that’s like user docs or dev docs, but you don’t wanna a hundred percent trust the docs. <Laugh> You wanna read to kind of get a big picture. But you don’t wanna necessarily a hundred percent think that every detail in the are gonna be correct. And whatever you do, please write stuff down. Draw it out so you don’t have to ask again or figure it out again. Again, you will forget 70% of the things that you learned in the first couple of weeks.

Shanea Leven: You wanna start from the key flows and then expand to the architecture. You know, make sure that you start there and maybe draw them out. What are the key components? How do they talk to each other? You wanna start piecing these things together to try to get a picture of the architecture. What are the most important components in your system? What is responsible for each of those domain objects or use cases? And how do they talk to each other?

Shanea Leven: This is an example of a monolith. Again, once you’re starting to put things together, you wanna make sure that you have an example like this that will help you to get a full picture of your system. Again, best strategies would be asking a teammate, reading the tests is a good way. Just reading the code and creating a diagram or diagramming things out yourself, and then double-checking with the team to make sure that it looks right.

Shanea Leven: Next you want to understand the low level things. What are the frameworks? Not just, you know, that your company uses React. You really want to know that they use Red React, React Router, React Query, functional components, and the React Testing Library. You want to kind of deep dive and get a detailed understanding of what are all of the underlying frameworks best strategies to kind of learn this. You wanna make sure that you at least go through some basic tutorials or guides for each framework and library. You wanna know how to use the tools that you’re using. You wanna be able to understand the coding patterns, what coding patterns are encouraged, which ones are discouraged? Every company is gonna be different.

Shanea Leven: Some best strategies to learn. This might be, you know, looking at recent code reviews to see how things are done most recently. You wanna read the code and then use a tool like GitLens to find that code review. When you’re working on a task and read to decision point, you wanna ask a teammate what that pattern likes to use before you say it’s done and ask for the code review. You want to understand the development process. Questions that you should probably look to answer might be, you know, how do we commit, how do we do a code review? How do we deploy, how do we plan a new feature? How do we give status updates and how do we retro? Next you wanna look out for code-based kind of tribal knowledge. You know, if you make a change here, you also need to make a code change over there. You don’t wanna touch this function because if you do without proper knowledge of how to do it, site performance might go down.

Shanea Leven: You wanna start to understand all of those things and look out for it. And then again, write stuff down. I’m gonna say this multiple times. It is gonna be very, very helpful for you. Don’t try to have it in your head. It will be again, cuz you’re gonna forget. Take notes. Drawing out diagrams is, like I showed before, is very, very helpful. Adding comments to the code will help everyone if they’re not there. And try to update the documentation as you go. So advice for teams. You want to make it easy to learn some of these things. If you’re on the team and you’re accepting in new teammate you want to know, have them understand what are the domain objects and the user flows and the key flows. You know, how do the key flows fits in the architecture? What are the frameworks? What are best coding patterns that we’re following and avoiding, and the development processes I mentioned. So here’s a thought.

Shanea Leven: There are lots of tools. You might be thinking, oh, there’s a tool for this, or there might be a tool for that. There’s a tool for notes. There are tools for reading code, but they’re all separate tools. And likely they’re still gonna get out of date because they aren’t connected to the main thing that you’re trying to learn, which is your code. So why should we actually have to draw out key flows by hand when the computer already knows how to run our code? Why in fact, do we have to keep reminding people about patterns and processes? You know, it’s really easy to miss stuff and we don’t on our current teams make it easy for our team members or you as the new onboarder to learn this stuff. And turns out that computers are actually much better at automating some of these things than the manual reading code line by line in our brains.

Shanea Leven: And again, every code review is a little mini onboarding experience, but our tools don’t really help us to onboard. They help us to just comment on a bunch of lines that are listed alphabetically.

Shanea Leven: For continuous onboarding, what if we could actually achieve that goal? What if all of the maps and the key flows and diagrams were all drawn out for you? What if you could just start from a file and expand until you understood the architecture? Or what if your teammates reminded you of the patterns that you should follow? Like right here, right there in your PR. What if you know every PR you could understand the impact that your code change has on the rest of the architecture? That is what we’re actually building at CodeSee. You could absolutely just use a tool like CodeSee.

Shanea Leven: There are lots of other tools as well in the code visualization space. It’s a new exciting space that’s really designed to help you feel comfortable and confident learning all of the code, even as as it’s changing out from under you. Try to invest, invest or advise your team to try to use, whether it’s CodeSee or not, some kind of tool to help you feel confident in the process. We can solve a lot of problems with having our tools help us to onboard a bit more. In conclusion, onboarding is hard. It sucks, but it can be easier. Thank you so much for listening to my talk.

Angie Chang: Thank you Shanea. That was excellent. You ran a little over, so I’m gonna have to stop us now. Thank you so much. And I know you’ll be back tomorrow with a workshop, so don’t miss out on that. It is one of the most what is it? Bookmark sessions if you look at the schedule. Amazing. We’re definitely excited. Really

Shanea Leven: Excited to see.

Angie Chang: All right, see you tomorrow.

Shanea Leven: Bye.

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

• Check out open jobs at our trusted partner companies.
• Watch all ELEVATE 2023 conference video replays!
• Does your company want to sponsor a Girl Geek Dinner? Talk to us!

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

• Check out open jobs at our trusted partner companies.
• Watch all ELEVATE 2023 conference video replays!
• Does your company want to sponsor a Girl Geek Dinner? Talk to us!

Sukrutha Bhadouria: Our next talk is gonna be given to us by Katie. Welcome Katie. Katie is a Senior Insights Analyst at Sony Interactive Entertainment. She’s a digital analyst focused on multi-platform customer behavior and insights. At PlayStation, she analyzes the customer journey across digital storefronts leads, A/B tests for digital merchandising and champions data-driven thinking across the organization. Welcome.

Katie Egeland: Thank you Sukrutha. Hi everyone. I’m Katie. And my talk today is about data storytelling and the skill that I think is most important in when you think about data storytelling and how it can really set you apart from from your peers. Before we get started, just a little about me. As mentioned, I am I work at Sony Interactive Entertainment, also known this PlayStation. I’ve been here for a little over five years, and I’ve worked on web analytics as well as commerce and customer journey analysis.

Katie Egeland: In addition to the video games, I also have worked in the international education and beverage industries, and I do enjoy gaming. Some of my, I hate picking favorite games, but some of my favorite game genres are action, adventure, term-based strategy. And I also enjoy any games that have plants in them. When we talk about data and data storytelling, you know, data’s everywhere these days and a lot of companies say, I wanna be data driven.

Katie Egeland: What does data driven really mean? It’s important to remember that just looking at numbers or having numbers doesn’t make you data driven. And as analysts, we often get asked for numbers, but those aren’t really the end goal. Nate Silver sums it up well here. He says, the numbers have no way of speaking for themselves. We speak for them, we imbue them with meaning, we being the analysts who are getting numbers and translating that to our stakeholders and businesses. But what kind of meaning are we looking for in the numbers? What is, what is the data for?

Katie Egeland: What is the point of getting those numbers and sharing them? The real point of it is to make decisions. And in order to help people make decisions from numbers, what do we need to do? We need to communicate. That’s how we really turn data into actions. And communication is more than just putting numbers in a page or building a dashboard.

Katie Egeland: We have to make sure we’re communicating data in the right context, making sure whether that we’re resonating with the people we’re sharing it with and what they care about, and actually addressing that, the problems that they’re trying to solve. And if we can do those things, then we are really being effective in our data communication. And I don’t think this is only for analysts. I think this applies to anyone who has to supply numbers or present data to others. We can use the same ways of thinking.

Katie Egeland: When we talk about communicating data and telling those stories, really the key to that is connecting with your audience. You’re trying to give them a message. In this case it’s data, what it means, what actions they might take from it. And when we do that, we often focus on a few questions. One is the what question, what data is needed? What are the metrics that we need to get?

Katie Egeland: What questions does our stakeholder need answered? The other question is, how will I get it? How will I analyze it? Where do I need to go to find this data? What methodology do I need to use to analyze it? What code do I need? How do I clean the data up so that it’s usable? And often we, we stop here, and these two questions are really important to get started on any project, but if we can go a little beyond just these what and how questions, that’s where we can start to dig into that effective communication of our data. And in order to do that, we can think about a few other questions.

Katie Egeland: One is, who’s going to see it? This is, you know, really key to understanding who you’re communicating with knowing how much they already might know about the topic what level of detail they might be looking for. If you’re communicating something to an executive, they’re probably gonna be interested in much more high level insights and not so much detail as an engineering or marketing team.

Katie Egeland: And then also, when you think about who, you can think about what other data might be relevant, even if it’s not something that they directly ask for, if you have an understanding of the angle that they’re coming at things from and the other things that they’re involved in, you might be able to, to find other data to, to give them supplementary information that makes the data that they’ve asked for more, more useful to them.

Katie Egeland: Lastly, the most important question in my opinion, is why, why do they want this data? And this isn’t to ask, you know, oh, why do you want me to do this work? But it’s really to help you as the analyst understand what problem is that data addressing and what actions or decisions are going to come off of this data request. In order to think about all of these questions, I wanna run through a couple of examples of common scenarios that analyst finds themself in and talk about how we can apply this framework.

Katie Egeland: First, reports and dashboards, these are really common. Any analyst is gonna have fulfill many requests for these. And these can actually be a little tough because often when you get a request to create a report or a dashboard, the KPIs and the metrics are already set. You might just have a, you know, a list of, here’s everything I want on this dashboard, but even still asking some of these questions can help you do a better job.

Katie Egeland: Who uses the dashboard? Who’s going to be looking at this on a regular basis, especially if it’s a recurring report. Understanding again this executive report. Is this for troubleshooting some kind of issues or monitoring for issues? Wfhat metrics are actually going to drive action? A lot of times in requests like this, you get a long list of metrics, but which of those metrics are going to, when they change, actually alert someone and say, we need to make a change because this metric changed.

Katie Egeland: There’s also a few things you can do when you’re building these and thinking about visualizations. Using those visualizations wisely, not just putting a bunch of fancy looking charts on it. Charts can really help you drive actions.

Katie Egeland: My personal favorite is trended charts, so showing something over, over a mid or long period of time. Often we might put some type of metrics on a dashboard and look at week over week, quarter over quarter, year over year changes, but those don’t really tell the whole story. If you put a metric onto a trended chart, you might see a little bit of a dip in your week over week numbers. But when you look back over six months, you can see an overall upward trend. Having that better context gives you more information about where things are are trending.

Katie Egeland: Another trick I like to use on reports and dashboards is using conditional formatting. It’s pretty common to see things go up or they’re good. You see green things go down and they’re red. But going a little bit beyond just that if you understand what are the benchmarks or what’s the threshold of something changing where we actually need to take action, as opposed to, just any up or any down is green or red. If we can incorporate those into the reports, it makes it a lot more easier for the person consuming that to understand, oh, this is, you know, red in, in a negative way that I really need to look into it and change something as opposed to this is dipped, you know, some very small amount since last week.

Katie Egeland: And then another, my last tip on here is checking in with the requester of your reporter dashboard after a few weeks or a few months that they’ve been using it. Especially again, if it’s something that’s recurring or they’re referencing often, and you know, asking them, what parts of this are you using? Are there areas where you wish you had more information? Are there actually parts that you’re not really looking at all, even though you wanted to see those, those numbers initially, and having that conversation to understand what is actually useful for them.

Katie Egeland: Lastly, I think remembering that having numbers isn’t the goal. Sometimes when you make a request for reports and dashboards, your goal personally might be, well, I need to send those numbers to my boss because they’re asking for them. But the point isn’t just having the numbers. It might be for you to put in an email, but ultimately those numbers are, are going to be used for something and to, to drive some kind of action.

Katie Egeland: And so even if you’re not an analyst, even if you’re not the one building that dashboard, asking those questions of the people that you’re sending it to so you can really understand how they’re using those numbers and what type, what type of actions they’re driving. Deep dives. These are the other, I think major scenario for a lot of analysts. A deep dive is a bigger analysis project. Usually it’s gonna be questions that explain behavior or make some kind of recommendations to your business stakeholders. And these can often take many hours, many days even, you know, months to, to complete. There are a lot of effort and, you know, asking these questions is, is just as important, if not more.

Katie Egeland: Again, that question of who, who’s going to see this analysis, how are they going to use it? Understanding if it’s an explanatory or an exploratory analysis. Explanatory is looking at, you know, what happened, why did it happen, what what factors contributed to it? Exploratory is looking at where there’s opportunities what potential solutions might be in making those recommendations and understanding the di the difference in what’s being asked.

Katie Egeland: Another big one for me is understanding if your requesters already have a hypothesis. Sometimes when you get a deep dive request, the people asking it might already have a hunch of why they think something is happening or, or where things are trending. And if you know what their kind of hypothesis is in advance, it’s not. You can tailor your analysis and cherry pick and, you know, just build something to support it. But if you, for instance, see something that might be contradictory to the assumptions that they have, you can anticipate the questions they might have and know that there’s going to be potentially additional skepticism if what you’re showing goes against what they thought was going to be the case. And then again, always connecting us back to what decision or what course of action is this deep dive going to to help influence.

Katie Egeland: Sometimes you might be in a situation where people just wanna know, know more about a topic, but it’s not quite clear what action might come off of it, given that these deep dives can take such a long time really pushing on what are the outcomes, you know, that we can expect from this and, and how might we take this information and use it after it’s finished When you’re putting together deep dives, there’s a few things I think are really important.

Katie Egeland: One of those is starting with a summary of your key takeaways. I like to say that spoilers are good. You shouldn’t hide hide your takeaways at the end. It can be tempting as you’re building a story to, you know, start with, here’s the question, here’s how we investigated it and, and wait till the end to reveal your, your answers to those.

Katie Egeland: But if you start with your key takeaways right up front, you really hook your audience, you get them engaged, you, they know where you’re heading and they’re gonna be more interested in understanding how you got there and how you’re proving your points. Also including context where you need to. And this can be as simple as just taking screenshots, providing in explanations of a particular feature or experience. Even if you’re presenting to people who are quite familiar with, with the product you’re talking about, they may not be completely familiar with the exact feature or nuance that you’re, that you’re covering.

Katie Egeland: And making sure you have that context so both they can understand it and you understand it, them yourself is really important. Lastly, using the appendix to tailor your content to different audiences. If you produce a presentation that has 30 slides, depending on who you’re presenting to, you may not need all of those slides. For every, every audience I mentioned earlier, executives, they’re usually not gonna be interested in really long detailed presentations. They’re gonna say, what’s the key takeaways? What do I really need to know? If they have additional questions, you can pull those things from your appendix. But being really mindful of who you’re presenting to and how to use a time wisely when you have with them.

Katie Egeland: And then I think engaging, asking questions when you’re on the receiving end of a deep dive is really important. Nothing’s worse for an analyst to spend many, many hours putting something together and kind of get nothing back from the people they present it to. But when you’re asking those questions, making sure you’re, you’re thinking about them and curating them a little bit for yourself and saying, okay, if wanna know more about this, how is that going to help me in my job? How is that going to help me take some kind of action in the future to avoid creating a lot of extra work of kind of nice to know questions that might be interesting but might not actually have a lot of value in the end.

Katie Egeland: What I want you to take away from this is that the key to communicating anything is making it resonate with your audience. And if you can make it resonate, it helps drive action. And in order to do that, we want to, you know, again, think about these, these four questions that I’ve been talking about. One of them is the what metrics actually connect to actions? What are the key takeaways? Not hiding those, making sure they’re really clear upfront because they’re more important than the actual numbers that support them.

Katie Egeland: The question of how providing context, providing the right level of detail depending on, on who you’re showing this to. And using visualizations with a purpose and as a tool to help people understand what action they should take. That question of who understanding who your audience is, working hard to curate what you’re presenting or showing to them, to their needs, their level of expertise and the actions that they can actually take themselves. And again, again, for me, the most important question is why asking questions. What it is you’re, what it is you’re trying to, to communicate. What’s the actual purpose of, of the analysis and remembering that the point isn’t just the numbers, it’s the action that comes off of them.

Katie Egeland: Now, I didn’t come up with all of this myself. This comes of course from many years of experience pulling data and presenting it and communicating it to people. But there’s a few resources I wanted to highlight here for you. One is a book called “Resonate” by Nancy Duarte. And she has many books about different styles of presentations. Some of them are data focused. “Resonate” is actually not about data at all. It’s really about visual storytelling and how to connect with your audience and break down your idea and what you’re trying to communicate and structure a story so that it really resonates with the people you’re talking to. The other one I really love is “Storytelling with Data” by Cole Nussbaumer, Netflix. And this is a little bit more straightforward data storytelling. It includes great tips for in improving your data visualizations, making them clear, making the action clear with what you’re trying to communicate, and also the importance of including context and making sure your audience understands that.

Katie Egeland: And once again, I’m Katie. If you have questions or you want to chat about data, storytelling and communication, feel free to hit me up on and thank you very much.

Sukrutha Bhadouria: Thank you Katie. People have so many amazing comments for you. We talk and even the reference to the book, the Storytelling with Data book being the best <laugh> book they have read, it’s really

Katie Egeland: Good. Yes.

Sukrutha Bhadouria: Yeah. Some people have shared their stories in the comments as well, so when you get the chance to take a look… Yeah. Well I wanna thank everybody for attending this session. It was very insightful. So thank you so much Katie, and thank you for taking time out on a workday, busy workday to have great Women’s Day with us.

Katie Egeland: Thank you everyone. Appreciate it.

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

• Check out open jobs at our trusted partner companies.
• Watch all ELEVATE 2023 conference video replays!
• Does your company want to sponsor a Girl Geek Dinner? Talk to us!