Posted on

“Leveling Up: Becoming a Manager of Managers” — Girl Geek X Elevate (Video + Transcript)

Transcript of Elevate 2020 Session

Gretchen DeKnikker: The inspiration for this panel was, you think it’s really, really hard becoming a manager until you become a manager of managers, and you don’t realize it’s just like another rung on the ladder. It’s like a whole different skill set and you’re lost, and it’s super hard, and so because we have such an amazing senior audience tuned in today, I thought this would be a perfect topic. And then, thanks to these wonderful ladies, we were able to put together the perfect panel, also.

Gretchen DeKnikker: So we have Ines Thornburg, who’s the Area Vice President of Splunk, works in their customer success arm. Arquay Harris is the Senior Director of Engineering at Slack. She actually got her intro at Slack through a Girl Geek dinner, so you should be coming to those dinners, because if you want to be Arquay, and don’t we all, you should do that. And then Bora Chung, who’s the Senior Vice President of Product Management at Bill.com. So, they’ve all worked at different sized companies. They’re at different sized companies now, so they have all of this amazing perspective. Bora’s going to come from product and Arquay’s going to come from engineering and Ines is going to come from customer success, and it’s going to be amazing, and all I have to do is basically sit back and let these women talk.

Gretchen DeKnikker: So, if we want to do a quick kind of round of intros, and why don’t you start, Arquay? And let us know kind of how many people that you’re managing now and a little bit about how you got where you are.

Arquay Harris: Sure. Hi, I’m Arquay, a senior director of engineering of essentially the growth team here at Slack. My org is about 70 or so people. I manage two teams. One is called customer acquisition and one is called expansion, and essentially they make up the product purchase funnel. How I got to Slack, as mentioned, I went to a Girl Geek dinner. I highly recommend that you go. It’s very rewarding. I’ve been here for about four years. I’ve watched the company grow from a company that was about 500, where engineering was roughly 100 or so, to now engineering is well over 700 and at our largest we were 2,500 people.

Gretchen DeKnikker: Bora?

Bora Chung: Hi everyone, I’m Bora Chung. I am SVP of Bill.com and I lead an organization of about 40 product managers and product designers. Just to give you context of the size of the company, we are about 13 year old company that do workflow automation for SMBs and our revenue’s about, I think last fiscal year was about 110 million. The entire company size is about 550, so product managers and product designers account for about 40 of them. We just went public December of last year so we’re going through a transition of being a private company to public company.

Bora Chung: How I got here, even though I manage both designers and product managers, my own professional heritage is more on the product management side, so I spent nine years out of business school at PayPal, four years at Apple, and then four years at eBay.

Gretchen DeKnikker: The quick unmute is not working. Ines, can you go?

Ines Thornburg: Absolutely. Thank you, Gretchen. So, Ines Thornburg. I am responsible for the Americas portfolio and customers for customer success at Splunk. My team is about 100 people, comprising of customer success managers as well as the renewal function and the renewal team that supports Americas customers. Been here about two years now and my career spans back to a series of different software companies where I started off as a consultant doing implementations, moved into presales, joined Oracle through the acquisition of Hyperion, so I went from a small growing company to a midsize company to a mega company. Was there for a while, learned a lot and then decided to try a venture startup.

Ines Thornburg: So, why I’m at Splunk, the technology’s very relevant in today’s data explosion as well as where we are in our journey in terms of maturity. And so Splunk is going through a pretty massive business transformation, shifting to a SaaS and subscription model, so that’s what really excited me. We’re still what I would consider a medium size company and really on a trajectory of growth, and that’s what I feel like I can make an impact on for our customers.

Gretchen DeKnikker: All right. So, you guys can see … Y’all, I’m trying not to say “you guys”. Y’all can see why I’m so excited about this panel. They have just an amazing set of backgrounds. It’s a completely different skillset, right? Ines, what do you feel like you kind of had to relearn in that very first time that you went from being a manager to managing managers?

Ines Thornburg: Gretchen, for me it was all about how I spent my time, really. And so going from being, as I mentioned, starting off as an individual contributor, doing the work myself, then being able to manage people doing work, then to manage multiple workstreams and priorities and making sure that those managers responsible for different workstreams not only were competent and experts in their field, but then, me balancing my time across the different responsibilities in a way that, frankly, I wasn’t getting too involved, I learned to trust the expertise on my team and learn what was good enough. And frankly, perfection is not always the end goal. We have to continue to progress multiple workstreams at one time in initiatives, and really making sure that no one gets left behind.

Ines Thornburg: And so, me figuring out that right balance between rolling up my sleeves and doing versus allowing people to do and coaching along the way was really that arc that we continue to perfect over time.

Gretchen DeKnikker: Bora, you were within the first 100 employees at PayPal, right?

Bora Chung: That’s right. When I first started, I started out as an MBA summer intern and the company was about 100 people.

Gretchen DeKnikker: What was that journey, along the same vein that I’m assuming it sort of started there where you were drinking from the fire hose?

Bora Chung: Sure, sure. I think the soft skill that I learned during that period was just mental agility. So, there were a lot of ambiguous situations when you’re a fast paced startup with just very few resources. You don’t really have a very well defined job description, so there were lots of ambiguous situations that hit you every day but just figuring out how to be a go-getter and get out of that ambiguity using mental agility was a skillset that I picked up in the early days of my career, and then if I could just connect that with the manager’s manager tradition, when I get to manager and then a manager’s manager, what I had to unlearn a little bit was when do I helicopter out versus when do I helicopter in. There’s absolutely no management course or management book written about how to do it, when to feel it out.

Bora Chung: So I think that’s a basic soft skill that you have to pick up very quickly and I struggled through that a little bit. I’m still learning it, but to me, the biggest difference between a junior employee in a very small startup versus a manager’s manager is learning how to do helicoptering in and helicoptering out at the right moments.

Gretchen DeKnikker: Right. Yeah, I like that analogy. Arquay, what’s a skill that served you really well as a manager and then you kind of had to unlearn?

Arquay Harris: Oh, that I had to unlearn? As an engineer, I got into engineering leadership in the way that most engineering managers get into engineering management which is you’re the most technically proficient person on the team and so your manager says to you, “Have you ever thought about management?” And you’re like, “No, but I’ll try it,” right? And so it’s a really hard transition because … It’s really hard because you know that you’re technically most proficient and so you just want to jump in there and do PR reviews and all of the stuff, and so you have to really make this transition from being able to be the person who was the peer on the team to the person who is the leader on the team.

Arquay Harris: And then when you make the transition from managing individual contributors for people playing bingo to manager, what happens there is you go from this very directive sort of supporting, coaching state of mind to managing to outcomes. So, when you have a person who is also responsible for managing other people on the team, you don’t want a person who is managing or doing things in the way that you would do them. Right? You want them to manage in the way that they do them and the way that feels comfortable for them.

Arquay Harris: And so I would never say to my manager, “Hey, I want you to do this and this is step one, two, three.” It’s like, “This is the outcome. How can I support you to get there?” You have to really trust them to be able to do it. And so the unlearning comes from this thing of wanting to be the person who is the hero, jumps in, saves the day, maybe writes the code, to really growing and empowering that next generation or that next level of leadership.

Gretchen DeKnikker: Yeah, absolutely. Maybe let’s go to Bora. What skills have you gained along the way that you feel like, “If I’d just learned that earlier, it would have been so much less painful?”

Bora Chung: Right, right. I think it’s doing skip level one-on-ones and getting the right communication done in those sessions. So, one mistake I made when I become a manager’s manager was I was just having one-on-ones with my immediate direct reports, but then they also have a set of teams and maybe not as frequent, but making sure that I check in with the team members and the delightful moments are when I hear some of the key themes and strategies being played back, I think that’s when things are going well. When you completely hear game of telephone being played and have a disconnected kind of direction and alignment, that’s when you know that things are not going well, so I think one thing that I recommend, and is a pretty tactical thing that you could easily do is maybe a little bit less frequent but do a skill level one-on-one check in and I think that I didn’t realize early enough but I picked it up and that has been serving me greatly.

Gretchen DeKnikker: Yeah, I’m taking that one back with me for sure. How about you, Ines?

Ines Thornburg: I think the one thing as you grow in your career and you become more visible, have more responsibilities, the one thing that I’ve learned is that when you speak or when you say something, the impact of what you’re saying really is that much stronger, that much more gospel, so to speak, and when you’re facilitating a meeting or when you’re communicating, you have to realize that, again, as your responsibility grows, is that people really listen. So you have to be careful, so if you’re trying to facilitate a brainstorming, for example, what I’ve learned is, facilitate the dialogue, get the conversation going, but I reserve what my opinion is until the end, because I don’t want everybody to just think that my opinion is the right one, because it’s certainly not. That’s why I bring together, and when I’m doing hiring, I always try to look for complementary skills.

Ines Thornburg: So I’ve learned to really be cautious about what I say and when I say it and to whom I say it, because I realize that, frankly, what I’m saying does affect and impact a lot of the folks on the team.

Gretchen DeKnikker: How about you, Arquay?

Arquay Harris: Things I wish I could have learned earlier?

Gretchen DeKnikker: Mm-hmm (affirmative).

Arquay Harris: This is a thing that I say all the time, which is … I say, “Be friendly, not friends.” If my team’s watching, they’re probably laughing about this because I say this a lot and it’s basically very early on in my career when I made that transition to manager, these people are your best friends. You hang out with them every night and when you are friends with the people who report to you, you cannot be impartial, right? You can’t say to your best friend, “You really screwed up on that thing. I need you to work harder in this area.” It can be really awkward.

Arquay Harris: And so what I really learned later in my career was how to set boundaries, because I do you a disservice if I’m not able to give you that really constructive and helpful feedback and help you grow. And that doesn’t mean that you have to be this monster who’s just a robot, but boundaries are really, really important and I just wish I’d learned that earlier.

Gretchen DeKnikker: Yeah, definitely. Definitely. You’d kind of talked a little bit before about another skill, about learning to delegate, and you had this example of quadrants.

Arquay Harris: Yeah. I kind of dug into that a little bit earlier. I wish I could claim credit but it’s essentially situational leadership. You can google it. There’s lots of videos on YouTube about it, but it’s basically about how when you are leading a large organization, or any organization, what a lot of managers will do is they will try to bend the team to the way that they lead. “I’m really introverted” or, “I’m super extroverted” or whatever it is, like the people need to fit into what I expect of them, but really, what a really good leader should do is you should make your management style situational to the person and to the stage that they are in their career.

Arquay Harris: And so it really just goes into this first quadrant, which is directive, which you might do to a more junior person. You might say, “Bora, I need you to log into this machine, do this work,” and then you move up into coaching, which is you have a little bit more skill and it’s like, “All right, you kind of know what you’re doing. How can I coach you through it?” Onto supporting, which is, “You know what you’re doing. How can I support you? How can I help you get to that next level?” And then the final magic kind of golden quadrant is delegation, and that’s just, “I don’t even really need to tell you what to do. You probably are bringing me the problem, telling me what it is that needs to be solved.”

Arquay Harris: And I think the thing that’s really interesting is it’s not really a straight line. You might kind of hover, depending upon your skillset, maybe in communication you’re in full on delegation mode but at technical proficiency maybe you need a little bit more support, and so I think that when I’m managing managers, I really try to think about it in that way, about what are the strengths and how do I help really, really uplift a person’s strengths and how do I help them really either correct for or counterbalance any weaknesses that they may have?

Gretchen DeKnikker: That is a good segue. Hiring is so different. All the skills that you learn to vet people when you’re a manager, and you’re just vetting them for do they have the skillset to do this role and do I think they’ll be the right fit with this team? But when you start hiring managers, what’s your suggestion there? Where do we start, Ines?

Ines Thornburg: One of the best practices that we have, and we really are very firm about it at Splunk, is at any role at this point, we have a panel. We have a select group of people that bring different questions to the table to assess skill. So, for example, we may have someone assessing the technical skill, we may have someone assessing behavioral type skills, situational skills, collaboration skills, et cetera. I always like to make sure I speak to the finalists.

Ines Thornburg: I like to know every single person on my team, a little bit about them, and really I have two primary questions that I’ve always asked as a leader doing hiring through every company I’ve been at, which are, number one, why am I talking to you today about this role, whatever the role is? Because what I’m looking for in that question is really what is their career journey? Why does this particular role fit into their long term career journey? I’m not looking for someone that’s just applying for a job because they may have seen something. I want somebody who’s put thought into how this role is going to help them along their long term career journey.

Ines Thornburg: Second, why Splunk or why whatever company? And to me, that shows me they’ve done their homework, they have a passion about what the company is we’re trying to achieve and we can have a dialogue. And from there, those two questions really help me take it on to the next level conversation, which is something that, frankly, how I always start those … And I’m not looking for skill, I’m not looking for technical proficiency. I’m looking for the long term drivers that really want that person to be on my team.

Gretchen DeKnikker: So, Bora, you’re at a company that’s one sixth the size of Splunk, so you might not quite have all the bells and whistles that Ines has at her disposal, so what is your process and how is it different?

Bora Chung: We start with the fact that interview is definitely a two-way street. We want to make sure that we evaluate the candidates, but candidate’s evaluating us, so we try to actually put an interview panel together that represents cross functional relationships, because teamwork, team play is an important element of culture at Bill.com, so we make sure that the candidate experiences the characters and the types of people that he or she will be working with. So, I think that’s one.

Bora Chung: The other piece is I think we have different seniority levels represented in the interview panel as well, so that I think some of the maybe early career folks could really test out the technical chops. You know, is this person a great designer? Is this person a great engineer? And then maybe someone like me could maybe test a little bit more about their soft skills, right? Can you actually influence the cross functional teams? Are you going to think more for the company versus your own output versus your own team’s output?

Bora Chung: So, I think we have a good balance of technical assessments and culture fit and teamwork elements going on. So I think we could definitely do more in terms of strengthening the recruiting process, but we’ve been hiring a lot of good talents through this.

Gretchen DeKnikker: Arquay, you’re kind of at the midsize between the two, but also, what did you have to change about how you interviewed? What skills did you need? What muscle did you need to build to be able to vet people to be managers as well as you did for ICs?

Arquay Harris: Having worked at very, very large companies where you have an interview process that is pretty set in stone and pretty precise, the cool thing about working at a hyper growth company like Slack is that I had the opportunity to really be involved in crafting that interview process and seeing it evolve over time, and we, right now, have a pretty defined rubric where we have pretty set slots where you’re judging people on things like teamwork and collaboration, ability to execute, strategy, and then we try to make it so that we have really diverse panels that are representative of gender and race and tenure and that type of thing.

Arquay Harris: But I think that the difference between evaluating an IC versus a manager is that to a certain extent when you’re judging an IC, there is the work product. That can be a really good weeding out factor, because if you do a coding exercise or you do … even when you come in and you’re doing white boarding exercises, not necessarily algorithms but something that shows technical proficiency, it’s a little bit easier to see whether or not a person can thrive or not thrive. It’s not perfect, but you have more signal, right?

Arquay Harris: When you’re evaluating a manager, it is, as I was mentioning, a lot more about the soft skills, and so you’re really trying to see if given certain scenarios, how they can fit and I think that it really does depend on your particular company and size and what you’re looking for, and so, for example, in those early days of Slack, one of the things that was really important was hiring managers who had experience or aptitude for scaling teams.

Arquay Harris: Because recruiting, if your engineering org is like 50 people or 100 people and we’re trying to grow to 7, 800 within a couple years, recruiting is going to be a very big part of it and do you understand to build strong relationships with recruiting? Do you understand how to really evaluate your pipeline? Fill gaps on your team? And so it’s these types of questions that we’re really looking for.

Arquay Harris: In terms of making it so that it’s a really fair and consistent process, we really make sure that we try to have our interviewers stick to the rubric, look to the way that people are answering the questions and that it’s not just subjective, like, “Oh, they’d be cool to hang out with,” kind of thing. We like to make it so that there’s some fairness and consistency built into the process.

Gretchen DeKnikker: Yeah, absolutely. You got me thinking, also, I think some folks really hesitate to hire that person with way more experience, right? Especially if you’re at the hyper growth company, because if you’re in a senior role at the hyper growth company, you’re gaining the skills at a rate that does not keep up. You might have perfected your job yesterday and you might be finally good at it, but the next day, it’s a different job and you’re not good anymore, and you’re constantly going. So how do you sort of fight that … I think some people get really nervous about, “I need to hire someone who knows what it looks like when we get there,” but that’s also a person that may know a lot more than you do, and I think people hesitate with that. How do you advise people to work with that?

Arquay Harris: Yeah, when I started at the company, my team was two people. Literally two people. And that was fine. I was like, “All right, let’s roll up the sleeves, let’s get it done,” but I was really excited about working for this particular company at that time and I think … You can suss a little bit of that out in the interview. If you’re interviewing someone and they haven’t done that exact thing and they can really describe to your their approach or their philosophy, what I really look for is, is this person a structured thinker? Do they have best practices or some kind of toolkit or some sort of methodology in the way that they approach leadership?

Arquay Harris: Because part of it is what you just said. It’s all intangible. The ambiguity is so high at a company like this, that I think understanding what type of leader you are and what you can contribute, that’s way more important than a very specific checklist of skills, because like you said, tomorrow it’s going to be different anyway.

Gretchen DeKnikker: Bora, you were nodding your head on that.

Bora Chung: Yeah, I was thinking about the early part of my career when you asked that question. I think when I was more junior in the early part of my career, I was thinking that I should be the smartest person if I’m the manager, and I was somewhat reluctant and afraid of hiring people smarter than you … smarter than myself, rather, but what I am realizing is that it’s absolutely cool to hire people smarter than me. It actually elevates the team. It improves the quality of the thinking and ultimately what we deliver to our customers is going to be much stronger. So I think I had to shed that a little bit of early stage career insecurity to really put together a strong team, so I think that was one.

Bora Chung: And then I think it goes back to one of the comments that Ines made earlier. I don’t have to be the perfectionist that knows all the answers. Sometimes a great value as a manager or manager’s manager comes from asking the right question, maybe asking the powerful question that nobody else is asking, because they are afraid or there’s a big elephant in the room. So I think a lot of wisdom I gained over the years is that it’s awesome to have team members that are smarter than you. They elevate you and your team and then, two, is you don’t have to have all the answers. Sometimes asking the powerful question could really be helpful as a manager or manager’s manager.

Gretchen DeKnikker: Absolutely. So, Ines, you have this huge organization. What do you have to add in? You have 400 people, right? Or something. It’s a crazy number.

Ines Thornburg: At Splunk, my team’s about 100 but in other jobs and other companies, it’s certainly been a lot larger, and that’s the thing. As our responsibility grows, you’ll have lots of different experts on your team in different disciplines, different business units, what have you, and it’s impossible just to chime in with Bora and Arquay. You can’t be the expert. It’s just physically impossible as your organization grows, and so what you do need to do is to be really, really comfortable working with these teams of experts in helping them accomplish their mission. And so, as a leader, really, my value to my team is making sure that we’re working towards the same goals and cascading those company goals down. Everybody understands those goals, that we’re progressing on those goals and frankly that we’re communicating our progress effectively in working together.

Ines Thornburg: Splunk’s a very technical company, like all these others, and am I technical? No, but I have a business degree and frankly we’re running a business at Splunk, and so my goal is to make sure that from a customer perspective, that those customers are getting value out of our technology so that they renew and we grow as an organization. And so, my value to my team is different than the value of them to our company and that’s what we have to make sure that we’re always balanced on so that together the team is stronger. So, that’s the way I think about it.

Gretchen DeKnikker: Okay. Time is going way too fast. We’re going to do one more question and then I see that we have some amazing, amazing questions in the Q and A also. So, your most cringe-worthy and your most exciting moments when you first made this transition, so that everyone can sort of go along with you. I can go first. My most cringe-worthy was like what Bora said. I thought I needed to know everything and I was so embarrassed when I didn’t know what was going on, and it took me a while to realize I’m just air traffic controller and actually the less information I have on a tactical level, the less opportunity I have to screw things up and I should just let the expert be the expert.

Gretchen DeKnikker: And then my most amazing one was when I walked in, I was planning this 10,000 person conference and there were hundreds of people setting up all of these little tiny details that we’d spent a year making and I knew the names of like six people that I could see at any given moment and I was like, “Okay, this is working. They have this. They’ve got it. I don’t even need to know what’s going on right now. This is amazing.” So, why don’t you kick us off, Arquay?

Arquay Harris: Cringe-worthy is definitely bad hires. Unlike hiring a bad IC hire, the blast radius is just so large when you have a bad management hire and it could affect the careers for quarters and quarters of the people in the team. Most amazing moment is really fulfilling and rewarding to see people grow, to see them go from kind of more junior manager to senior manager to director, to see them be able to come into their own as a manager, develop their own styles, and yeah, that’s probably the best thing.

Gretchen DeKnikker: How about you, Ines?

Ines Thornburg: Most cringe-worthy is when I feel like I’ve not done enough preparations and prepared my team, and so specifically, again, we’re all in some sort of technical discipline. Learning the technical skills, I think, is one aspect of the job, but let’s not forget about the soft skills. And so Arquay mentioned soft skills and looking at those in hiring, but also continuing to help the teams augment them. So that means communication skills, that means collaboration, meeting facilitation. It means executive presence, making sure that when you’re representing your company or your team, that you do it in such a way that you’re proud of that. So, when I know I haven’t prepared my team and I see a train wreck about to happen, that’s when I’m like … That’s the cringe-worthy.

Ines Thornburg: The most proud, frankly, Splunk just had our sales kickoff and we’ve been working really hard as a customer success organization over the past couple years to get to a point where we’re really ready to support almost 20,000 customers globally and the team recognition and what I saw … what my executives and the company recognized on the customer success team was just extremely rewarding to see the people on my team winning awards, being part of large contributions to customers, and frankly it just made me really warm and proud inside.

Gretchen DeKnikker: That’s awesome. All right, Bora.

Bora Chung: So, cringing moments. When you become a manager’s manager, naturally a lot more escalations hit your desk and escalations could stem from conflicts between people or conflicts between departments or sometimes goals are not aligned. Just having to resolve conflicts on behalf of the team, sometimes you’re successful, sometimes you are not so successful and disappoint the teams. So I think the escalation handling and conflict resolution, I think I had some rough spots at the beginning of my career, so I think that’s the cringe moments. The most proud moments, there are times that when you go on an extended vacation or extended business trip, you come back and your boss is basically telling you that, “Oh my God, Bora, your team was perfect. I didn’t even know that you were out of the office.” And at the beginning, again, you’re like, “Does that mean that I’m not adding any value? Did you not know that I was out of office?”

Bora Chung: Sometimes I would wish that some crisis would happen just so that they know that I was absent, but I think the real truth is that that means that you have a fantastic bench and you have a great top talent manager. So, my most proud moment is when I’m absent on a sabbatical or vacation and then the team doesn’t even notice that. I think that’s the ultimate success of coaching and grooming the right team.

Gretchen DeKnikker: Absolutely, yeah. Okay, so we have a ton of questions. The first one, and we’re going in order of their voting, as women of color, have you experienced any difficulty or veiled biases while managing male coworkers? What did you do to handle that situation? So, Arquay, Bora?

Arquay Harris: Sure, I’ll just jump in. I think one of the hardest things about being a woman, especially a woman of color, is just the big issue of low expectation. What happens to me a lot in particular is people think that I’m not technical. I’ve had interns be like, “Do you code?” Which is a ridiculous question that you probably never ask a male who’s a director of engineering. And so I think, yeah, you face that a lot and it’s really unfortunate. On the bright side, I think things are changing, particularly as we get more and more women in leadership positions, I think just having different voices in the room is really contributing to the conversation.

Bora Chung: For me, the usual stereotype where sometimes the hardship is, especially as an Asian woman, getting stereotyped into a bucket of, “Oh, you must be quiet, you must be an introvert,” so I think this is why I spent extra energy on developing what we usually call the executive presence and executive gravitas, because especially when you become a manager of manager, it’s not just your personal brand and personal reputation. It’s your team’s effectiveness that you have to be responsible for. So, I think those have been some tough spots, but I think I try to overcome it by being more vocal and representing the team more actively.

Ines Thornburg: Gretchen, I think you’re muted.

Gretchen DeKnikker: I need to unmute. Okay. Bora, this one’s for you. What are the things that you discuss during your skip level one-on-one? I’m thinking of setting up a skip level one-on-one with my skip level manager but I don’t know what we should discuss during those meetings.

Bora Chung: Right, right. So, I think it starts with just a very broad question of how are things going? And the other kind of check in is that, is there a certain expectation? So I try to also let the manager in the middle know that we are having the skip level. So I think the worst outcome is that if the manager in the middle gets alienated in this conversation, so I don’t really have an agenda. I think just like our services are getting more and more personalized, I think the skip level one-on-ones need to get personalized. So with some folks, I talk about just their career aspirations. With some folks, since I’m one level away, they could maybe ask more questions about the big picture strategy and whatnot, so it’s a little bit different, but the two things that I just always do is I let the team member drive the agenda. I just start by just checking in on overall things and I make sure that the manager in the middle is aware of the fact that we are having this conversation, and we’re not breaching confidentiality.

Bora Chung: There are some key things that I think the manager in the middle should know. I also make it pretty obvious and public as well.

Gretchen DeKnikker: Ines, do you do skip level?

Ines Thornburg: Oh, absolutely. Yeah, I even do double skips. Like I said, I want to be the finalist on all interviews because I really take pride in knowing people. One of the things, as a leader of a large organization, that I like to understand is, is career aspirations, as Bora mentioned, because honestly this is where we have a much larger purview of opportunity as a leader, and frankly if I have a conversation with someone and I understand really they want to be in another part of the organization at some point in the future, if I see that connection and see that match, I would love to make that match and keep that talent within my company rather than seeing people leave and take all that wonderful knowledge that we have, and great talent, to another company, frankly.

Ines Thornburg: So, I do that a lot and, frankly, when I’m looking … I don’t want people leaving my organization necessarily but at the same time, if we can promote from within and give people more opportunity within our organization, it just makes … frankly, people appreciate that and I love a team that culturally has a strong morale and knows that we’ve got each other’s backs.

Gretchen DeKnikker: I think this one’s for everyone, so we’ll have Arquay kick us off. What are the top traits and qualities you recommend focusing on for someone looking to get into a management role?

Arquay Harris: Adaptability for sure, because the thing about being an IC is that it’s a pretty defined trajectory to go from associate to engineer to senior to staff to senior staff, right? You might not know exactly what it is but there are some … some part of it’s mapped out. It’s a little bit more opaque when you’re talking about leadership because in any given moment you could have to deal with people’s emotions and you have to coach and you have to support and you have to discipline and you have to … It’s just all of these things that you have to do, and so you have to take, like we say, growth mindset. You have to be willing to iterate and change. So if you have these kind of qualities …

Arquay Harris: If you’re a person who’s really rigid and like things just so, you maybe want to not consider … Consider something else.

Gretchen DeKnikker: Or find people that are just like you.

Arquay Harris: Or that.

Gretchen DeKnikker: Yeah. How about you, Bora?

Bora Chung: I would say maybe two shifts and mixes, right? One is if ICs generally think about execution for the most part, I would say you have to start to blend in execution as well as strategic thinking, right? So I think that’s maybe the first shift. The second one is just how you think about time horizons, so let me maybe take product development as an example. Maybe when you’re an IC, you’re thinking mostly about next release, the release after that, but when you eventually become a manager, you think about maybe an annual roadmap or like a three year vision. I think those are maybe the difference in time horizon of your thinking, and there’s not a right or wrong. I think there need to be different parts of thinkers. Some people need to execute, some people need to think strategy, some people need to think next release, some people need to think about the three year vision, but I think those are some of the shifts that you start to … you need to have to transition into a managerial role.

Gretchen DeKnikker: Ines?

Ines Thornburg: Yeah, in addition … I mean, the adaptability is huge, and Bora’s comments, I think, were spot on. I will add onto those, communication, and, frankly, as you think about just rallying a team from what they’re doing at a macro level down to the micro, everyone needs to have a proper communication cadence and understand where we’re all marching toward. So, I think a lot about communication and different ways that we communicate, whether it’s quarterly all-hands calls, weekly cadence calls, the one-on-ones, the skip levels, Slack, we have Slack channels, we have email … I mean, we communicate in lots of different ways.

Ines Thornburg: We actually have spent the starting part of our year thinking about all the different communication … You know, the different communication means and important forums that we need to do to make sure, frankly, everyone is marching in line. At these high growth companies, things are moving so fast and, frankly, as a leader, we have to make sure that everyone is working towards the same goal. So, tops down, bottoms up, communication to me is super, super important and sometimes we just don’t think about it enough. So that’s one that I’ll add on.

Gretchen DeKnikker: Yeah, or the mindset that meetings are a waste of time. Meetings are your lifeblood when you get to a certain level. If you spent your whole day in meetings, unless they were just … you’re not careful with your time, if you spent your whole day in meetings, you were doing your job all day, and I think that’s a mindset thing that a lot of people really struggle with changing.

Ines Thornburg: Yep, agreed.

Gretchen DeKnikker: So, next question, I think this would be for Arquay. In engineering, what can we do as an organization to encourage more women in manager of managers positions? Was there anything specific that helped you get to where you are in your career and that is Katie coming from the Scotland, UK today so [crosstalk]

Arquay Harris: Thanks for joining. Part of it is basically making sure that there’s some sort of support system at your company and paying it forward and being that person who can encourage. So, for example, one of the things that I do at my company is every week I have office hours and I post it, and the women’s ERG … bingo … So I’ll post it in certain channels and get people to sign up and try to be mentor and support system when I can.

Arquay Harris: And then the other thing is, I think, really just having … When I was coming up, there weren’t a lot of people who look like me who did the job that I do, and so it just wasn’t a thing that I could even see myself doing. The idea of a CTO was Andy Grove, right? With the khaki shirt … I mean, a blue shirt and khaki pants, and so that’s part of it too. Just making yourself aware and available and aware to other people within engineering and letting people know that, hey, you are a source of information.

Arquay Harris: And then sponsorship is a big thing that people are doing lately. If there’s someone that you see who you think has potential, maybe encourage them, and if I had people on my team who show interest in management, maybe try giving them some tasks. Like, “Hey, maybe try managing this intern for a summer and seeing how it goes, or maybe you might want to run the sprint meeting.” That kind of thing. Really just give them these little nuggets to see if they have the aptitude and really kind of understand what management is.

Gretchen DeKnikker: I can’t believe we’re already at time, but I just want to thank you on behalf of everyone who’s tuned in right now because you guys just gave them most amazing session. So thank you again to Arquay, Ines, and Bora, and we will be back in just a moment.

Ines Thornburg: Thank you all.

Gretchen DeKnikker: Bye.

Posted on

Episode 18: Why Hiring is Broken with Aline Lerner of interviewing.io

Transcript:

Gretchen DeKnikker: Welcome to the Girl Geek X podcast, connecting you with insights from women in tech. This is Gretchen. I’ve been working in tech for over 20 years.

Rachel Jones: This is Rachel, the producer of this podcast. And along with Angie and Sukrutha, who are out this week, we’re the team behind Girl Geek X. This podcast brings you the best of Girl Geek X events, dinners, and conferences where we’ve been elevating women in tech for over ten years.

Gretchen DeKnikker: Today we’ve got a really special episode. We’re doing our first interview on this podcast with Aline Lerner, founder and CEO of interviewing.io. You might remember Aline from our Elevate 2018 Virtual Conference where her presentation on interviewing was the top rated session of the day.

Gretchen DeKnikker: Today we’re going into the data she’s compiled on why diversity quotas suck, imposter syndrome, and the dirty secrets of recruiting departments, and so much more. Welcome, Aline.

Aline Lerner: Hi, that’s a great intro. Thrilled to be here.

Gretchen DeKnikker: Awesome.

Rachel Jones: So you have a lot of expertise in hiring and interviewing as the founder of interviewing.io.

Aline Lerner: Aptly named.

Rachel Jones: Could you tell us a little bit about that?

Aline Lerner: Maybe actually I’ll tell you a bit about my background and then the company. So, I was a software engineer for about five years. I cooked professionally before that, which gave me a bad attitude and a lot of skepticism about things. Which served me both served me well later in life. And then I fell into recruiting kind of by accident, I think, as many people do.

Aline Lerner: One of the things that always frustrated me, both when I was an engineer and then later when I was a recruiter, was that one, there just wasn’t very much hiring data that was being used practically to make hiring decisions. And one very specific instance of that is how much we tend to rely on resumes despite the fact that the data shows that they don’t carry very much predictive signal at all about whether somebody is going to be good at their job.

Aline Lerner: So, I started interviewing.io to help make hiring more fair, and sort of make it more efficient in the process as well. And ultimately try to level the playing field in tech a little bit and give access to, opportunity to, people that are good but might not look so good on paper.

Aline Lerner: And in the process, by virtue of what we do, which I hope I’ll be able to get into a bit more, we collect a ton of data about technical interviewing and hiring. On our platform we regularly conduct technical interviews. I think at this point we have close to 50,000 that have happened. And we have all the audio, and all the feedback, and all the code people write and whether it runs, and stuff people draw.

Aline Lerner: And then of course we see where our candidates end up and how they’re doing there. That allows us to draw some really interesting conclusions both about recruiting and interviewing. And then also ultimately about hiring and outcomes as well.

Gretchen DeKnikker: So, for the people who are like, “Oh my God, that sounds amazing,” so, there’s kind of two parts to the platform, right? The, the folks who can do practice, and then clients of yours that are companies that actually use the platform to conduct their interviews.

Aline Lerner: That’s exactly right. The platform comes in several parts. Ultimately what we do is we’re a marketplace where companies can hire software engineers, but the mechanism for doing that sounds at first a little convoluted and weird, but it’s also our secret sauce. On our platform, if you’re a software engineer you can sign up and once you sign up you can actually book live anonymous mock interviews with engineers from companies like Google, and Facebook, and Microsoft, and Amazon, and Dropbox, and a bunch of other logos that make us really happy.

Aline Lerner: These interviews are completely free. So basically you grab a time slot and then when you show up at go time, there’s a senior engineer on the other end who meets you in a collaborative coding environment and just starts running you through a prototypical technical interview and gives you feedback at the end.

Aline Lerner: Of course we use this feedback. I’ll talk about how we use that feedback in a moment, but it’s really cool because for our candidates, interview practice is now completely de-risked. Rather than having to go and apply at a bunch of companies where you don’t want to work to sort of warm up before you get going [crosstalk 00:04:20]-

Gretchen DeKnikker: For those companies, too, right?

Aline Lerner: I guess sometimes it might be good for them because you might be pleasantly surprised. A lot of hiring is just interpersonal chemistry and you never know going in, but by and large it’s probably not the best use of either party’s time. It’s like dating, you want to go on some shitty dates if you’re just getting back out there to sort of break the seal before you go with the people you actually want to go with. Not to be cynical about dating.

Gretchen DeKnikker: Oh, we can do that. We can do follow up podcast on [crosstalk 00:04:51]-

Aline Lerner: I have a lot to say. So, we tried to give people something they couldn’t get anywhere else, which was this really high fidelity interview practice that’s very realistic. And the nice thing is we can use data from these interviews and how they turn out to surface people that are good in a way that we think is much more reliable and much more fair than how they look on paper. So, once you’re a top performer, you unlock what we call our jobs portal. And there you just see all the companies we work with, and we work with around a hundred, and you can say, “Oh I want an interview with Twitter tomorrow. I want an interview with Snap tomorrow. I want an interview with Microsoft tomorrow.”

Aline Lerner: You just click a button and you book a real technical interview with an engineer who works at that company. You don’t have to apply, you don’t have to talk to recruiters, you don’t have to try to find a friend who works there that’ll slip your resume on top of the pile.

Aline Lerner: You just press a button and then it’s still anonymous is the best part. You interview at one of these top companies, at the end, if things feel good, you can unmask and then they’ll shoot you straight to the onsite. The last thing I’ll say, because I’ve talked for quite awhile now is the really, really cool thing, the thing I’m most proud of is about 40% of the hires that we’ve made in the last three years, that’s how long we’ve been doing hiring, are people that don’t look good on paper at all.

Gretchen DeKnikker: Can you talk a little bit more about that?

Aline Lerner: So, by that we mean when we think about diversity at interviewing.io, for us it’s bigger than race and gender. Though, of course, those are both very important things. But for us, when we say nontraditional candidates, we mean primarily whether you look good on paper. So, did you go to one of a few schools? Or have you had the opportunity to work at one of the few sort of top companies that recruiters value when they decide whom to spam?

Aline Lerner: On our platform something like, I think we have 25% women, which is a little better than the pool at large, which we’re very happy about. I think 7% people of color, a number we’d to increase, but overall we have 40% of people who maybe dropped out of high school. Went to community college. Were a late stage career changer. Maybe they ended up going to a bootcamp, and then working in industry for a few years and sort of having to go through the school of hard knocks.

Gretchen DeKnikker: We focus on a narrow set of schools. You’ve done posts about this, the top five elite schools, you went to MIT, I went to Berkeley, these top schools.

Aline Lerner: We’re so great.

Gretchen DeKnikker: We’re so amazing.

Aline Lerner: We’re so amazing. These are top, top schools and top programs, and hiring from those same sources, when there’s a complaint about there’s a lack of diversity. It’s like, well maybe if you stopped fishing in the same pond that fishes in the same pond.

Rachel Jones: One thing that I think, Gretchen, you said on the episode that we did about bias and hiring was how companies are kind of off-sourcing the work of deciding who’s good onto these schools, and onto these other companies and just assuming, “Oh, they passed Stanford’s check. So that means they’re a great fit here.” But yeah, I think a lot of the data that you’ve uncovered shows that it’s not always a good fit. Just going by this kind of automatic check.

Aline Lerner: And there have been so many times when a candidate has applied to a company, gone in through the front door, they got rejected before they ever got to interview because of how their resume looks. Then they get on our platform, they crush it in practice. They book with this very same company, do really well in the interview, go on site, get an offer, and then the company looks in their ATS, their applicant tracking system, and they’re like, “Wait, shit. This candidate applied six months ago and we rejected them and now we made them an offer.”

Aline Lerner: We’ve actually had to update our contracts with our customers to say … Normally when you’re a recruiter, there’s this language in the contract that says if you’re already aware of this candidate from a different source, if you hire them, then you don’t pay us anything. We’ve had to update it to say if you’ve already interviewed this person and you rejected them, then you don’t owe us anything. That’s your candidate. If you rejected them based on their resume, and then we surface them, we actually added a lot of value and we hope we’ve forced a bit of an existential crisis in your hiring.

Rachel Jones: And you should definitely still pay us.

Aline Lerner: Please give us money because we think we did something good.

Gretchen DeKnikker: So part of, I think what I understood what you were doing is you almost … Not even almost. You end run the talent acquisition team, right?

Aline Lerner: It’s different at every company. I think that a great recruiter is worth their weight in gold because they can figure out what candidates want, they can shepherd them through the process. They can make sure that people have a great experience, they can help close, they can inform salaries. This is stuff that that is a very sophisticated thing and it takes a very specific set of skills. Right now, recruiting departments are spending a lot of their time and resources sourcing and doing all sorts of top of funnel things that don’t make a lot of sense.

Aline Lerner: Recruiters, in my opinion, should not be vetting candidates because they simply do not have the domain expertise to do it.

Gretchen DeKnikker: Well, they’re looking for people just like the people you already have.

Aline Lerner: That too.

Gretchen DeKnikker: They don’t take risks.

Aline Lerner: They’re not incentivized to take risks. They’re actually disincentivised to take risks.

Gretchen DeKnikker: Exactly.

Aline Lerner: We believe in sort of this brave new world where we, if you look at a job search, there are parts of it that are additive, there are parts of it that are neutral, and then there are parts of it that are shitty, and unpleasant and useless.

Aline Lerner: And we think that our goal is to amplify the additive stuff, automate away the neutral stuff and take away the shitty stuff. Right now in a hiring process, at the very beginning, once you’re in the process, you’re going to talk to a recruiter. We think that that call isn’t always the best use of either party’s time. Because, one, in a good funnel, maybe 25% of people will make it past the tech screen, but the tech screen comes after the recruiter call.

Aline Lerner: So, you’re selling like crazy to a bunch of people when three out of those four people will never make it through the funnel. Terrible use of recruiting time. And recruiters are not as good at selling. Unless they’re superb, they can not be as good at selling as an engineer, because when you’re talking to a peer you can be like, “Hey, what’s coming up on the roadmap? Why are you here? What projects got you excited? What’s your day to day like?”

Aline Lerner: You can’t do that with a recruiter. What we would love is to have that conversation happen later. And what we would love is to have vetting at the top of the funnel, not be made at the whim of some proxies like where people went to school, but actually based on what people can do and that is not something a recruiter can do either.

Gretchen DeKnikker: And everyone has their own shorthand. When I was a founder and we were hiring, I didn’t want people who had gone to top schools, and I didn’t want people who had worked at big companies with brand names because I felt this wasn’t going to be the perfect environment for them. Is that 100% true? No, but everyone takes their own shortcuts.

Aline Lerner: You have to, yeah. I think that data has the power to sort of free us from having to make decisions based on proxies like that. Although your heuristic is much better than I think most. The crazy thing too … I was going to mention this earlier, just about sort of putting that, that final nail in the university hiring coffin as it were … We were looking to see, speaking to biases and stuff, whether where you went to school could actually predict how you’d perform in interviews.

Aline Lerner: This is pseudoscience. It’s my blog. It’s not an academic paper. That’s the beauty of–

Gretchen DeKnikker: It reads very academically though.

Rachel Jones: [crosstalk 00:12:48]

Gretchen DeKnikker: Yeah, she fronts in the best ways.

Aline Lerner: The best thing about having a blog is there’s no peer review. It’s just your friends and your employees. They’re like, “You’re too long winded,” and whatever. But you can say whatever … Of course the stuff we say we think is true.

Aline Lerner: So, we basically took all our students and we sorted them into one of a few categories. There were people that went to elite schools, the MITs, and the Stanfords and the Cal Techs and so on and so forth. And Berkeleys. Then they were top state schools, and then I think we went by US News and World Report and ended up with basically four tiers.

Aline Lerner: I hope I’m remembering this right. I might be misremembering a tiny bit, but the idea was that there was no difference in interview performance between the first three tiers at all. And the fourth tier, I think it was a tiny bit worse, but it wasn’t even that big of a deal. And now of course one of the big … I posted this on Hacker News, and whenever you post anything on Hacker News, people come out of the woodwork and are kind enough to identify all sorts of shortcomings with your work.

Rachel Jones: So generous.

Gretchen DeKnikker: I would like to donate my time.

Aline Lerner: Yes. They deigned to do that. I’m very grateful. Because this was actually a really valid criticism. It’s like, well, there’s some selection bias here. Because the students that are taking their time to do practice interviews are super motivated. So, they’re the ones that chose to do all this stuff, and are more proactive and they’re more career minded or whatever. And that’s probably true. But even with that, it’s still insane to me that there was no difference in performance between the MITs, and the UC Santa Barbaras, which, I’m not trying to put UCSB on blast, it’s a great school. I just think hiring managers are generally less stoked about it than MIT. But that’s not actually the case in our data.

Gretchen DeKnikker: Okay. So, you have said that diversity quotas suck.

Aline Lerner: Yes. Yes I have.

Gretchen DeKnikker: One the one hand you can make a case that if people aren’t incented, meaning your bonus, your promotion, your whatever is not tied to some sort of metric … Right now diversity quotas to me look like, “Oh oops, we tried. Oh we interviewed, whatever.” But no one’s literally changing the way that they’re looking at hiring, sort of filters then everything that they’re personally bringing to it to change the ratio. So, if you don’t have quotas, then what do you do?

Aline Lerner: It’s a hard question. I don’t know that I’ve necessarily figured it out. I think there should be some metric. I think my biggest qualm with diversity quotas is that it feels like a very sort of lazy kind of low hanging fruit metric. I’ll try to think of a sort of controversial, inflammatory example.

Aline Lerner: Let’s say we have a world … And this is kind of one of those biblical choices, you have to choose one or the other. Like, would you rather. I don’t think it’s morally right for us to be in a would you rather situation where you’re deciding between a white male high school dropout and a white female student from MIT or something. They both actually bring a lot to the table in different ways.

Aline Lerner: If we’re going to talk about lived experience and diversity of thought, then we should mean it. And if we’re going to talk about nontraditional candidates, I would argue, you know, I’m a white female from MIT. I’ve had a much easier life than the guy that dropped out of high school. But I don’t even think it’s our place to make that choice. I want a hiring process where both of those people are welcome, and where they’re judged on their merits. That’s my view.

Aline Lerner: And I think if you create a quota that’s based around race or gender, people like me will always win, at least when it comes to if you’re choosing between two–

Gretchen DeKnikker: Because you’re a woman.

Aline Lerner: Because I’m a woman. And it feels like shit. I can’t tell you how many times people have asked me, “Do you think you got into MIT because you’re a girl?” And the fact is, I don’t know. I don’t know. It’s very possible that I did. I had friends in high school that I thought were much more qualified than me, and then they didn’t get in and I got in. And I’m very grateful I got in. It’s made my life a lot easier.

Aline Lerner: I met my co founder there. But it sucks. Now I’m walking around being like, “Does that …” And that’s not even a real problem. “Oh, I don’t know why I got into the best engineering school,” whatever. But it still feels shit. And this idea of people around me in college wondering, “Is she here because she’s smart, or is she here …” So, that is my view on this. And I just hate this idea that people who deserve good things are going to be questioned. I just don’t want people to have that experience.

Gretchen DeKnikker: Yeah. And the cognitive diversity, as people talk about it and they say gender, ethnicity, race are going to bring it, but what they’re always talking about is cognitive diversity. And so if everybody’s gone to the same school, even if there are different races and different genders, especially top tier schools, they are little machines that teach you all to think the same and tell you you’re special.

Rachel Jones: I actually just read an article that was really similar to this. It was a black engineer saying that a lot of the times the black people that you see hired to these top companies, they all still kind of embody this kind of white style of being in the workspace. So, even as you’re filling a quota, you don’t have to challenge your processes or challenge the reasons why you weren’t diverse in the past. You can just have this number [crosstalk 00:18:53]-

Aline Lerner: Well, I think for companies it’s this ideal cop out. It’s like, let’s get all the women and URMs from Harvard. Woo. It’s like, well are you actually doing something meaningful there? I don’t know. It’s probably better than not doing it, but it’s not enough.

Gretchen DeKnikker: Right. Yeah. I think maybe in interviewing.io version 7,000 you can have a test for cognitive diversity. Actually there is, we’ve talked about on a previous podcast the Basadur Profile, I feel like … but I don’t know if you could put that in, but it’s how you problem solve, and just sort of figuring out for your team where people fit in. It’s B-A-S-A-D-U-R if anyone ever wants to look it up. But I found that one is great, but I always just use it when the team’s already there. Just to figure out where we’re going to have conflict and things like that.

Gretchen DeKnikker: So, because we have a predominantly female audience, you talked a little bit on your Elevate session about how men and women suffer equally from imposter syndrome. But then you had a blog post where you went on and talked about how women participate in the interview process a little bit differently. Can you give us a little more [crosstalk 00:20:16]-

Aline Lerner: Yeah. So, we did this experiment a few years ago where … we actually have a few patents on a real time voice modulation where we can make women sound like men and men sound like women. And we can make everybody also sound androgynous, which we stopped doing cause it creeped everybody out. This uncanny valley, apparently. Interviewers don’t like it. I don’t know.

Aline Lerner: But we did an experiment where we made women sound like men and men like sound women. And we tried to see what that did to their interview outcomes. One of the things that was really surprising is that making women sound like men did not actually make their scores better, and making men sound like women did not make their scores worse. So, that made us feel good. We’re like, we have a platform where people are not jerks.

Aline Lerner: But I looked at the average scores and median scores for women and men. And women were doing worse in technical interviews. That was just clear. Like, shit, okay, what’s going on here? And what was really surprising is why it happened. I’m like, well, fuck, maybe women are just worse at being engineers. If that is what the math says–

Gretchen DeKnikker: Oh no, James Damore is actually right.

Aline Lerner: If that’s what the math says … I just want to know what the truth is. I’m not tied to any … I want to understand. So, that was a hypothesis that we were entertaining. But I’m like, if I’m going to publish that, I have to be sure.

Gretchen DeKnikker: [crosstalk 00:21:54] be a lot of very helpful people [crosstalk 00:21:56] to give you feedback.

Aline Lerner: Before I published that post I dug into the data a bit more. And what was really interesting is that women were quitting after a bad performance seven times as often as men. They do their first interview on the platform. It wouldn’t go well and then they would never come back. That was skewing the numbers.

Aline Lerner: So, I said, “All right, why don’t we remove the cases where people quit after their first interview in both men and women and see what that does to the averages.” And then of course the difference went away entirely.

Aline Lerner: Both men and women feel they don’t know what they’re doing equally. It’s a source of consternation to both genders, but men just plow forward more often and women go into this ball of self-loathing, I think, and don’t persevere. That’s a blanket statement. Many do, but more often they tend to give up.

Aline Lerner: That’s something that we’ve worked hard to address. There are probably more treatments we can do on the platform. But one thing we’ve started doing is emailing people after their first interview and being, like, “Hey, congrats. You got that out of the way. Something like a third of our top performers mess up their first interview. So, you’re in great company, no big deal. Just go again.”

Aline Lerner: That’s actually helped with with retention quite a bit. And there are other things that we have in the works as well. But that’s just an important thing for people to know. Technical interviews are so arbitrary. People’s performance vary so wildly from one to the other for both genders. It’s just a shame that women peace out after one when that interview may not actually mean anything.

Gretchen DeKnikker: Well, and then you went in a little bit on the Dunning Kruger effect, and people not being able to evaluate how well they did or didn’t do. Is there a gender difference there?

Aline Lerner: In overestimating performance? That’s a great question. I don’t know. I should look that up.

Gretchen DeKnikker: That’s the quote of, like, “God, give me the confidence of a mediocre white man.” I play that in my head sometimes when I’m like, “If you were just an average white guy, you would totally go do this.” So, just stop doubting. And yeah, what’s the worst that’s going to happen?

Gretchen DeKnikker: So, we have a lot of women in our community that ask questions pretty frequently at Girl Geek dinners about, you know, “I went to a bootcamp,” and sort of the doubts around that, and, “What advice do you have?”

Aline Lerner: Yeah, so what I would say is, one, just accept the fact it’s going to be really, really hard. It’s going to take months and months. And it’s going to be hard work. But if you persevere, it will work out. You just have to sort of make peace with the fact that there’s nothing wrong with you, and it’s hard for everybody, and you just have to hustle.

Aline Lerner: There are two things that I think are valuable. One is don’t apply through traditional channels when you approach companies. Because companies are saturated with bootcamp applicants. Every bootcamp student is on AngelList matching with every company. It’s not in your interest to sort of be a part of this giant pile of resumes.

Aline Lerner: And if this is your first exposure to programming, chances are your resume doesn’t have very much on it. It probably just has a few projects that you did while you were in the bootcamp. So, what you can do instead is approach engineers or hiring managers who work at those companies directly and be like, “Hey, I read this blog post that you wrote,” or, “Hey, I saw this thing that your company just put out. I have questions about it.” And just ask for advice, and establish yourself as somebody that actually cares about the thing. You’re not just trying to network, have some very specific call to action that you want to discuss.

Aline Lerner: You’d be surprised by how willing people are to help. But because you came through a non-traditional channel, the burden is on you to have to hustle, and it sucks. That’s the reality. But if you do it, it’ll be okay. But you can’t just do the same thing that everybody else does and expect that it’s going to work because the odds are just not in your favor.

Gretchen DeKnikker: So, we have a lot of senior candidates and obviously engineering hiring managers. Do you have anything in the data on the interviewer side?

Aline Lerner: Yes we do. On our platform, whenever a technical interview takes place, whether it’s a practice interview, or whether it’s a real interview with a real company, the feedback after the interview is symmetric. So, that means that not only is your interviewer rating your performance as a candidate, like your coding ability, and your communication skills and so on, the candidate is also rating the interviewer on things like, “Would I want to work with this person? How good were the questions? How excited would I be to work with them? How good was this person at sort of shepherding me through the question and giving me the right amount of hints while not taking away my ability to solve the problem by just giving me the answer?”

Aline Lerner: What we’ve seen is that the best interviewers and the best interview questions are collaborative. The best experience that people have is when you take away this pseudo adversarial relationship between interviewer and interviewee and it becomes a, “Hey, can we be smart together and solve a problem?”

Aline Lerner: That takes a lot more effort as an interviewer to set up both that environment and to craft the kinds of questions where collaboration becomes easy. If you just ask somebody to write this function, that’s not very collaborative. But if you start changing up constraints and you’re like, “Hey, now that we wrote this function, we ran into this weird thing at work the other day where under certain conditions this thing didn’t work. Let me tell you about that and what would you do differently? And here’s what we did and here’s what, here’s what I tried.”

Aline Lerner: And then it just becomes fun, right? It also becomes a much better selling vehicle because you can give people some insight into the kind of work you’re doing. If it’s interesting, it’s going to stick in their heads. Then after they leave the interview, they’ll be like, “Huh. What would I have done? What if we tried this? What if we had tried that?” Then you’re in there like a little parasite. That’s really what you want, I think.

Gretchen DeKnikker: Right. You’ve won them over.

Aline Lerner: You’ve won them over.

Gretchen DeKnikker: Yeah. No, I love things that. I’ve been off and on looking for a job for three years. I love going in and I get so excited when we can problem solve something together.

Aline Lerner: That’s the best part. It gives you a, “Is this fun for me? Do I want to problem solve with this person for the next 2.7 years of my …” Whatever that average is.

Aline Lerner: The other thing I’ll suggest if you’re an interviewer or a hiring manager, it’s really hard to come up with interview questions. But one weird trick for coming up with those is if you start some shared doc on your team where every time you do something at work that’s interesting or unexpected, you don’t have to write a question about it. But just jot a little note being like, “Hey, I had fun solving this, this was a little a departure from my day-to-day and this is something memorable.”

Aline Lerner: And then you can go back and look at that doc later and turn those moments into questions. And you’ll just have all these seeds for question ideas that you can expand on. And then make it so it’s something that somebody can do in 45 minutes and strip away everything but the essentials. But at least then it’s real.

Rachel Jones: Yeah. So, interviewing.io is specifically for engineering hires.

Aline Lerner: That’s right.

Rachel Jones: And with technical interviews, it’s easy to kind of strip things back and get to an objective space. But how could you apply kind of what you’ve learned from that platform to places that don’t have a more objective measure when they’re doing these interviews?

Aline Lerner: Yeah, it’s tough. It’s really, really tough actually. I think there were two criteria for whether something interviewing.io would work in a given vertical. And one is, is there a huge shortage of labor? Because if there isn’t, then why the hell would companies interview people anonymously? That’s insane. They’re doing it because they’re desperate.

Aline Lerner: But then the second one is can you make a value judgment about whether a person knows what they’re doing based on very limited data and what they’re doing in front of you? In some cases that may not be the right thing. I don’t really know how to interview salespeople. Gretchen, you probably know much better than I do.

Aline Lerner: But, my impression is that one of the best indicators of future performance is past performance. So, did you exceed your quotas? If you didn’t, why not? That may be more effective than, “Sell me this pen.”

Gretchen DeKnikker: Yeah.

Gretchen DeKnikker: Salespeople, their environment is everything. You have to really understand the environment that they’re in and how it differs from the environment that you’re going to bring them into. And figuring out how to ask those kinds of questions too, of what motivates this person? Because, salespeople, motivation is everything.

Aline Lerner: It’s everything, right?

Gretchen DeKnikker: Yeah. That guy who’s going to get to the end of the day and got 40 nos, and then the guy next to him gets to ring the bell, and they’re like, “I wasn’t going to make any more calls today, but …” Because they’re in that hyper competitive thing, if it’s competitiveness that makes them make that last call, then if you don’t have a big team … You know what I mean? If you’re not structured in a way where they can get that motivation, then you’re going to be really missing … and you’re going to be doing them a disservice if you brought in someone.

Aline Lerner: Yeah. So, I think in some cases having a perfectly objective scenario based in the moment interview is not going to give you a full picture, and may also do the people a disservice because if you’ve accomplished a lot of stuff in your career, that should probably be part of your story.

Aline Lerner: But, I think that there is opportunity to do more of that with … There are a lot of cases where you can just give people scenarios. I mean that’s really what coding interviews are, right? It’s like, “What would you do in this situation?”

Gretchen DeKnikker: It’s like business school with code, just a case study and a case study.

Aline Lerner: So, management consulting, I think also in particular lends itself well to this approach.

Gretchen DeKnikker: Oh, definitely.

Aline Lerner: I don’t know whether there’s a shortage of management consultants, though. Maybe not.

Gretchen DeKnikker: Business schools are minting more every day.

Aline Lerner: More every day. People often ask about product management also, whether that’s a good one. I think scenarios there are good, but again, there is not nearly as much of a shortage of product managers. So, companies don’t have to resort to very bizarre strategies like using interviewing.io.

Gretchen DeKnikker: Unless they would like to diversify.

Aline Lerner: Or, I mean, the best thing about … It’s so much cheaper for companies and so much faster. Because it’s not just about diversity. It’s like, these people are better. So, you have to interview fewer of them, and we take away all this top of funnel stuff so you can get to a higher … We just went live with this huge tech company in LA, they went live two weeks ago. They just made their first offer to a–

Gretchen DeKnikker: Wow.

Aline Lerner: That’s crazy. And that’s what people are doing, and it’s because it’s more efficient. And then you trick people into also doing the right thing.

Gretchen DeKnikker: It’s hard creating a category. But then once people get a little taste of it, then you kind of have this tailwind. I think your tailwind is coming, Aline.

Aline Lerner: Let’s hope.

Gretchen DeKnikker: Well, this has been amazing. Thank you for being … You made it so easy on us. Our very first interview, we’re like, “Oh, I hope we get …” Especially without Angie and Sukrutha, who we miss very much today. But this was great.

Rachel Jones: Yeah, this was wonderful. Thank you so much.

Aline Lerner: Thank you guys. Thank you for asking some hard questions. I feel like I worked for it. Really, thank you so much for having me. This is the stuff, this is the stuff that makes people use our products, so I’m so grateful to you both.

Gretchen DeKnikker: Thanks for listening to this episode of the Girl Geek X podcast. Please rate and review us on your favorite podcasting app. We’ll be back soon with more advice from women in tech.

Rachel Jones: This podcast is produced by me, Rachel Jones, with event recording by Eric Brown and music by Diana Chow. To learn more about Girl Geek X or buy tickets to our next dinner, visit girlgeek.io where you can also find videos and transcripts from all our events.

Posted on

6 Ways You Can Be A Stronger Leader and Make Better Hires

Nupur Srivastava, VP of Product at Grand Rounds

Long before she ever started obsessing over product features and worrying about design deadlines, Nupur Srivastava spent her days — and evenings, weekends and holidays — obsessing over her jump shot and running drills in her hometown of Qurain. Her hard work and dedication to the sport took her all the way to the Kuwait National Basketball team, where she played from 1999-2002 and learned the value of teamwork and how fun it is to win!

After earning her Electrical Engineering degree from the University of Michigan, Nupur began her tech career as a Wireless Hardware Design Engineer at Cisco. She then pursued an MBA from Stanford and transitioned into product management, finding her passion in the health tech space. Over the past eight years, she managed teams ranging in size from 5 to as many as 50 people. Driven by her upbringing and desire to help people, she launched Impactreview (acquired by MaterNova), a community for reviews of maternal and child health products for the developing world.

Today, Nupur is the Chief Operating Officer at Included Health in San Francisco, where she leads the company’s product management and design teams. As Nupur explains, “the company is on a mission to raise the standard of healthcare for everyone, everywhere. The team goes above and beyond to connect and guide people to the highest quality healthcare available for themselves and their loved ones. By leveraging the power of data and technology, we create products and services that make it easy for everyone to get the best possible healthcare experience.

When the Girl Geek X team sat down with Nupur during our ELEVATE 2019 virtual event on International Women’s Day, we wanted to pick her brain and hear her biggest mistakes and learnings as a health tech product leader and people manager. She shared some great advice:

1. Hire slow and fire fast.

Nupur confessed that she made a lot of classic hiring mistakes with her first hire. She was at a small startup, strapped for resources (we’ve all been there!), and there was a lot of work to be done. Feeling stressed for help, she hired very quickly without thinking through the long-term impact.

“Basically, I hired the first person who I thought could do the job from a technical standpoint,” she shared, “…but one thing that I didn’t focus on was whether there was strong alignment with the company’s values and where we were  growing. Unfortunately, a year later, I had to let this person go because it was a mismatch. I really wish I had spent time understanding upfront whether they were a good fit for what the company needed at the time.”

The classic saying that you need to “hire slowly and fire quickly” rings true here.


2. Ask the right questions.

“A lot comes down to the types of questions you ask in the interview process as well as what you get from the references.” Finding the right fit is less about technical proficiency, and more about who they are as a person, why they have made the decisions they have in the past, and what they are optimizing for in their upcoming role.

You want to ask questions about how they’ve made decisions in their career to date, what drives them, what motivates them. What wakes them up in the morning? When they’re put in a difficult situation, what value system is driving their decision-making?

Nupur stresses that what you’re looking for in a team member will be different for different stages of the company, and for each company’s unique values and mission.

It’s important to tailor your approach to your individual situation, because the perfect hire on paper might actually be a perfect hire for a different environment, but a poor hire once your own values and needs are considered.

3. Hire for impact: seek out people who are hungry, humble and smart.

Many of Nupur’s favorite hiring and interviewing strategies came from a book that CTO Wade Chambers recommended, called Ideal Team Player. “It focuses on this notion of hiring people that are hungry, humble, and smart, and that concept has really resonated with me.”

“We want to raise the standard of care for everyone everywhere, so we need to make sure that people are hungry for that impact,” she explained.

“The humble component is self-explanatory. People that are low ego and prioritize the company above self are great to have on the team. In addition, if you’re hiring someone to work in healthcare, you need to be sure they appreciate that the patients we serve are suffering through things that we may not totally understand. They need humility to empathize with that struggle and build the right products for those patients.”

“And then smart is not actually what you think it may be. It’s not IQ smart, but rather people smart. There’s a base level assumption that you’ll be able to do the job, but it’s incredibly important that you do it in a way that brings people along — that makes you a teammate that people actually want to work for and with.”

One of the things Nupur has been using in her recent interviews is simply asking everyone, “What’s the hardest thing you’ve ever done?” Their response typically gives you a sense of their work ethic and insight into what they consider difficult. Sometimes they’ll even answer with a personal response, and it offers a good window into who the person is, and whether they’re someone you want on your team.

4. Accept that your top performers will always eventually leave.

“As painful as it is, top performers will leave you at some point. With all members of my team, I try to develop trust, care deeply about their career, and truly understand where they want to go long-term. This way, when they eventually decide to pursue another opportunity, I’m not surprised because there’s openness and transparency in the relationships.”

The week before we sat down with Nupur, someone she’d worked with for four years left the company. She was an extremely high performer, and she let Nupur know of her intentions to leave four months in advance because they were actively talking about where she wanted to go and what drives her. The team member had joined a 50-person company. With headcount now over 500, she was ready for something different.

“I think the most important thing is to have that level of trust with your team members, such that you understand what their career goals are and you’re together making the decision about when is the right time for them to leave. If you adopt this approach, you can prepare for their departure in a way that is not disruptive.”

“It can feel like a painful punch in the gut when someone tells you they’re leaving,” she lamented, “but I think the least we can do is just not be surprised by the decision. At some point, maybe for their own career growth or evolution, or other things that they are optimizing for in their lives, you want them to leave. And as long as you are open and honest with each other and there is trust and transparency, it’s not the end of the world.”

Nupur’s general philosophy is one we could all benefit from adopting: “Everyone has different goals in life. The most we can do is be an advocate and great manager for our direct reports when they work for us, and help influence what they do next, so that you and the business are prepared for employee departures.”

5. Create an environment that welcomes diversity of thought and personality types.

“One of my biggest learnings as a leader over the years has been … beyond diversity based on race and gender, there’s tons of diversity in personality types and the way people like to do work.”

The Head of Data Science asked various team members to take a StrengthsFinder questionnaire, then put everyone into groups of people that are alike so they could discuss things they wanted to teach other groups who were different from them.

The entire product team has also used the DiSC assessment to better understand their behavioral differences. “This exercise gives you empathy for how different people want to show up, and how they want to debate ideas.” 

“Not everybody is comfortable being presented a problem and immediately jumping in and giving their thoughts. Some people want to think about a problem, spend a day organizing their ideas, and come back with their thoughts prepared.” 

“For me,” Nupur admitted, “the first step in improving my communication and collaboration with others is simply awareness. Where do people fall either in the DiSC profile or with StrengthsFinder? What do I need to be aware of as their leader so that I’m creating a comfortable environment for them to speak up?”

“I can remember the first realization I had when I recognized, ‘Oh, everybody doesn’t like coming into a room and talking loudly about their ideas? That’s interesting. I thought everyone was exactly like me!’ and that’s obviously not the case.”

“Using some of these frameworks has been incredibly important because it not only helps you understand others, but it also helps you realize how your type may be showing up for that person and what things you may need to temper, especially as a leader, because you’re setting the tone for the team.”

Nupur has a team member opposite her on the DiSC profile, and she’s started running ideas by him to make sure that he can offer feedback and criticism before she takes it to the team, because as she says, “I’m just hyper-excited and trying to tell everybody everything as soon as the thought occurs.” And that freaks some people out. It is important to understand where others in your team sit in the DiSC profile so that you can personalize your leadership style with them.

6. Let people know where you want to go!

One of the questions we hear asked at Girl Geek X events time and time again is about how to get ahead or move into a management role when you don’t have previous managerial experience.

Nupur’s advice is to make your manager aware that you want to be a manager, and make your goals explicit. “If someone wants to be a manager, you need to make sure that there’s an opportunity and a business need, and an opening in the company for a manager. Have open conversations, and make sure that you have the skills, training, and support of your manager.”

“The biggest thing is raising your hand and making it clear that that’s the path you want to go. Then hopefully if you have a good manager, and you are ready, they’ll make that opportunity for you.”

If you’re having open conversations about your goals regularly — say once per quarter — and you find yourself in a situation where the promotion doesn’t feel like it’s ever going to happen, or you start to feel like you’d be better off somewhere else, you’ll be in a better position to move on gracefully and with a reference you can count on time and time again.

For more hiring and people-management advice from Nupur Srivastava and other Girl Geeks, check out the full video & transcript from her panel on “Building High Performance Teams” at Elevate 2019, and subscribe to the Girl Geek X YouTube channel!


About the Author

Amy Weicker - Head of Marketing at Girl Geek X

Amy Weicker is the Head of Marketing at Girl Geek X, and she has been helping launch & grow tech companies as a marketing leader and demand generation consultant for nearly 20 years. Amy previously ran marketing at SaaStr, where she helped scale the world’s largest community & conference for B2B SaaS Founders, Execs and VCs from $0 to $10M and over 200,000 global community members. She was also the first head of marketing at Sales Hacker, Inc. (acquired by Outreach) which helps connect B2B sales professionals with the tools, technology and education they need to excel in their careers.