“Integrating Inclusive Research into Design”: Kat Chiluiza with Google (Video + Transcript)

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

• Check out open jobs at our trusted partner companies.
• Watch all Elevate 2021 conference video replays!
• Does your company want to sponsor a Girl Geek Dinner? Talk to us!

Transcript

Angie Chang: She is a UX researcher at Google. She joined as a UX researcher, a senior UX researcher from Fitbit, which was acquired by Google. She has a background in digital media, entertainment, forensic, and health. Her specialization is in kids and family, digital products, and services. And with experience and years of research in curriculum analysis and evaluation, she worked at Nickelodeon, Televisa Foundation, and Sesame Workshop. So welcome, Kat.

Kat Chiluiza: Hi. Can everyone hear me?

Angie Chang: Yes.

Kat Chiluiza: Okay. Hi. Hi, everyone. Thank you for the introduction. And yes, so my name is Kat, and I’m a UX researcher.

Kat Chiluiza: And really, UX research is about speaking to users, in order to understand their needs, their motivations, their behaviors, and then, taking these insights, in order to better distill what it is that we need to learn and apply and adapt into your own products and services and features.

Kat Chiluiza: I am trying to find my deck, because this is normally the part in which I would then lead into the actual deck itself. So if you would just bear with me for a second.

Kat Chiluiza: Yes. So UX research is about making sure that we are the voice of the user. However, we know that sometimes that it isn’t always the case.

Kat Chiluiza: When we think about products that are out there, sometimes they aren’t accessible or inclusive of people of different needs. This is not only frustrating, but limiting, right?

Kat Chiluiza: If you’re unable to use something, you might actually decide to leave the product altogether. Oftentimes, what causes this is when users’ intersectionality isn’t taken into account. That’s really what I’ll be talking about today, inclusive design and research.

Kat Chiluiza: What exactly is inclusive research? Inclusive research is when research is conducted in a way that involves and respects people of diverse identities. This could be gender identity, your ethnicity, any disability that you might have, all of that fits into identities. When you think about that intersectionality, that’s where, oftentimes, products do tend to fall, and not fully fit the needs of the full person.

Kat Chiluiza: Most people commonly think that we can address this by making research and design more inclusive in diversifying our participant pools. However, the challenge really is that diversity for diversity’s sake isn’t automatically going to lead to inclusive research findings. Instead, inclusive products is an intentional act. Inclusive research is an intentional act, and something that occurs throughout the entire production cycle.

Kat Chiluiza: Today, I’d like to share some tips that empower folks in product and design, to think about ways that you can add more inclusivity in the production process. And I’ll talk briefly about, what are some things that you can do when preparing for research, and when delivering those findings, and I’ll spend most of the time sharing examples of how research questions can fail to address that full spectrum of a person’s identity, and give examples on how we can improve them.

Kat Chiluiza: So let’s start at the very beginning, preparing for research. What are some things that we can do at the start, in order to make sure that we’re starting off at the right foot?

Kat Chiluiza: When we start planning for research, it’s important to narrow down and really kind of understand what it is that we actually are trying to get to know. Three questions to ask yourself at this stage are, what do you want to know, what do you already know, and what are you hoping to do with this information?

Kat Chiluiza: These questions are both not just for the researchers, but for the stakeholders, as well. This can help inform who you recruit, identify potential types of edge cases, and then, also keep you from going beyond the scope of the work or the scope of the research, as it’s very easy to sort of get down different paths.

Kat Chiluiza: When recruiting for participants, think critically and use your own intuition about, what are the users that might be excluded when designing for this product or feature? Ask yourselves who are our marginalized users, or think about, who are the users that might be negatively impacted by a feature update?

Kat Chiluiza: Oftentimes, there might be trade offs when making a new feature update. And in those trade-offs, is there going to be a certain user base that is negatively impacted, or whose services might actually not be as supported as well as before?

Kat Chiluiza: You can also look at behavioral data. What are your analytics telling you, not about the patterns, but about the outliers? For example, if it were a standard deviation, who are those people at the margins? Once you’ve identified those groups, then you can have a better sense of who are the recruits, who are the people that you often don’t speak to, but might benefit from actually doing so?

Kat Chiluiza: So now let’s actually get into writing out your discussion guide or survey. Every time we conduct research, we’re looking for ways to ensure research rigor. We make sure to screen out any of those double-barreled questions or any questions that might be leading or loaded. And we’re also mindful about the type of bias respondents may have coming into a research project. In fact, that’s part of the reason why we are so thoughtful about the recruit itself.

Kat Chiluiza: But what about our own cultural bias? How often are we reflecting on the types of questions we ask, and looking to see if we’re being exclusionary or insensitive, as we’re asking these questions?

Kat Chiluiza: Researchers unfamiliar with working with diverse audiences may ask questions that inadvertently show bias, right, or stereotype, or even completely ignore context. Oftentimes, these are researchers with the best intentions, but it just so happens, all of us have our own bias, and that’s how these things sort of happen.

Kat Chiluiza: To put this in context, imagine this was a survey that was sent to young adults in the US, and it’s asking them to select their favorite sport. And now imagine that this is the survey, and these options are the ones that were presented.

Kat Chiluiza: Technically, the researcher did their due diligence when it came to writing out these this question. These options are actually the most popular sports in the world. And so, as a result, it would make sense to include this as, “What are your favorite sports?” available.

Kat Chiluiza: However, they didn’t take into account that this audience is from the United States and localized, what might be popular in that country. Had they done so, they could have adjusted the potential responses to make it more culturally competent and relevant. While this question may seem really obvious to a lot of folks from the US, the reality is that this type of bias exists in anything.

Kat Chiluiza: So let’s look at a question that might be heteronormative. “What is your gender?” Heteronormative questions are those that are written from a cisgender or heterosexual perspective. The reality is that sex, gender and sexual orientation are far more nuanced, and should be their own individual question.

Kat Chiluiza: Additionally, if we look at the options, we notice that there’s a specific option for cis female and cis male participants, yet all trans people are put together. It doesn’t actually give an option between trans women or trans men. And lastly, when we look at the word “Other” to use to describe an identity, this phrase itself, “Other,” is dehumanizing to folks that might not fall into the first three categories.

Kat Chiluiza: This is an updated version that actually breaks down the nuance between gender identity and sexual orientation. And since identities have evolved over time, there’s also some new options in there, added to reflect that nuance. And by breaking down these questions, there’s also the benefit that it can help researchers and stakeholders better pinpoint what it is that they want to learn, and tease out what’s not relevant in the project.

Kat Chiluiza: If we go back to the initial set of questions that you answered in the beginning, now you can actually see which of these questions actually is most relevant to your project. How are you going to use this information that’s been presented?

Kat Chiluiza: So let’s take a look at this other question. This was, again, these are all questions that have been out in the real world. All of these have happened. This is a survey that was directed towards women on a website. If we pause for a second, I’ll give you guys two or three seconds just to think about what might not be inclusive about this question.

Kat Chiluiza: While this survey possibly considered the participants’ sex, it didn’t consider their gender identity, or even that some folks might just have a broader range of shoe style preferences that can impact their answers. Not all women identified folks wear women’s shoes. But this question assumes that our respondent is going to give a woman’s size. But what about those that only wear men’s shoes, or only know their shoe sizes in men’s sizes?

Kat Chiluiza: It also only gives shoe sizes in the United States, which is an oversight, since many websites tend to have a broad international readership. By neglecting these factors, you’re now risking faulty data since participants don’t actually know what size to put down, or they have to convert the sizes. Now you’re asking your participants to do additional labor, and possibly feel ignored, since they have to do their own work in order to provide you with answers.

Kat Chiluiza: But if we are realistic, we know that users or participants in surveys don’t tend to take that time in order to make those types of conversions. What’s more likely is that they’ll probably just leave the survey. And so, now this group that was already underrepresented will continue to be so.

Kat Chiluiza: Let’s see how this can be addressed. This updated version has the sizes available in both men’s and women’s, and that also show conversion sizes across different countries.

Kat Chiluiza: So let’s take a look at this Eurocentric question. While the question about race is very common in surveys, often, the options don’t include some more common groups in the US. And this question, in particular, doesn’t reflect the most up to date language, when it comes to describing some specific races, such as with the Hispanic option.

Kat Chiluiza: Lastly, there isn’t an option for mixed race people. Before we jump into how we can address that one, let’s also take a second to look at this other Eurocentric question. This one in particular is one that’s been making trends in culture surveys across organizations. And really, while this question is well-meaning and comprehend, and appears to be comprehensive, the reality is that it conflates culture, religion, and skin colors all as one, and ignores that they potentially have different experiences, and potentially could be treated differently.

Kat Chiluiza: When we look at how these questions can be updated, again, we noticed that there’s this breakdown of different identities, right? Race, culture, skin color have all been broken down separately. There’s also more comprehensive options when it comes to the changing demographics in the U S. For example, with Hispanic, there’s now been added the Latina, O, and X.

Kat Chiluiza: Here’s a question that one might see when you’re applying to jobs. “Do you have a disability?” A few issues that commonly are raised with this question in particular is its vagueness. People don’t know how to answer because they aren’t really sure what’s considered a disability for the specific role.

Kat Chiluiza: Instead, it’s often suggested to contextualize this question, and clarify what it means when you’re saying a disability. And lastly, allude to the relevance of this question at all for your specific application. So in this updated version, it’s, “Do you have a physical disability that can impact your ability to lift 10 pounds unassisted?”

Kat Chiluiza: So what’s the takeaway? Be clear and explicit. Think about the context of your questions in a person’s everyday life. Because if not, you risk basing decisions off of bad data, alienating your user base, as they might want to leave, or just left with a negative taste in their mouth. Or you might have to spend more money and time redoing the research, if you realize that that the data that you’ve gotten was impacted by some faulty questions.

Kat Chiluiza: Now that we’ve seen a few different questions and thought about intention, intersectionality, you’re probably thinking, “When is it appropriate to ask these questions? Or what type of considerations should I take into account?” Or even, “How do I know if I’m going in the right direction? I do have the best intentions, but now I’m wondering if that, maybe, that just might not be enough.”

Kat Chiluiza: The first thing to do really is just to go back to what you were originally hoping to learn, and what you were hoping to do with this data. That information is going to be really key, as it’s going to help you narrow down, what are those relevant questions you actually need to ask?

Kat Chiluiza: Then once you have that sense of, “Okay, these are the things that I definitely need to know,” then you can begin writing the questions. And as you begin to write these questions, learn to look from others, research the topic online, see what are some trends or some updates or things that have changed along the way, such as, for example, with the Hispanic/Latinx option.

Kat Chiluiza: Or see what others have said about crafting questions to that specific target audience. For example, the HRC, a few years ago, launched their own recommendations for how to ask the question around sex and gender.

Kat Chiluiza: And lastly, check your ego. There’s going to be a lot that you’ll learn and unlearn along the way. As identity changes, as the way in which we speak about different marginalized groups changes, then that means that how we adapt is going to be very different, and what was relevant before might not be relevant now.

Kat Chiluiza: In fact, even that HRC survey that I mentioned, that was written in 2016, and since then, even I had to make some adaptions to it, just to make it a little bit more relevant for the 2020/2021 audience.

Kat Chiluiza: Once you actually have these questions, then you can begin drafting a survey, or piloting your research in some way. You can do this with a small sample. This way, you can make adjustments to language or approach, while still being able to retain that data integrity.

Kat Chiluiza: Even with just one or two people, you can see if there’s some areas where you veered off in the wrong direction, or maybe the participants themselves might call out something that doesn’t sound contextually correct. And then you can make those changes without having impacted your larger pool.

Kat Chiluiza: In the last few minutes, I want to briefly touch on data analysis and insights. So we know that in the data analysis stage, there might be times in which you’ll see some trends that are consistent with just the Latinx participants, or maybe just your LGBT participants, but aren’t trending across the entire sample.

Kat Chiluiza: Especially as you begin to increase your diversity pool of participants, you might be seeing certain trends in certain areas. But as we all know, sometimes stakeholders are resistant to making changes, especially if they feel that it’s only going to impact a small subset of users. While this can feel demoralizing, the reality is that you can advocate for these changes in the same way that you would do for any other type of UX or UI changes that you might be running across.

Kat Chiluiza: For example, you can conduct a second round of research for that specific demographic. Then you can continue to show that this is a pattern that impacts this group, and it’s very consistent that this impacts this group.

Kat Chiluiza: Or you can test design iterations to address the user concerns with a larger sample size. Oftentimes, changes that are made to impact one marginalized group tend to have a broader impact for the entire community.

Kat Chiluiza: If we think about on sidewalks, the small ramps that are there, initially, that was made to make, to make it accessible for people in wheelchairs. But now people with strollers, people who are on bikes or scooters, they all benefit from it. If you see in a larger sample size that other groups continue to benefit from it, then it’s clear that this is something that could have a potential powerful impact on your product.

Kat Chiluiza: You also want to include some examples from competitors implementing more inclusive solutions. You want to make sure your company is staying up to date. These are just the trends in the industry. And by taking these steps, you can show how your company can remain competitive, or become even more competitive. By tying it to some type of business need, whether it’s acquisition, retention, subscriptions, or so on, then you can further help boost that case.

Kat Chiluiza: This session really has only scratched the surface of what there is to learn and think about conducting research with diverse audiences. But I hope that you now have a greater understanding of how even a small change to a question can have a meaningful impact to the users answering them. It allows them that one moment to actually be reflected in the types of questions in the Internet in itself, that actually shows them for who they are.

Kat Chiluiza: We can all advocate for systemic change and user research. So let’s all fight for a world where tech reflects those needs and wants of all consumers, one user at a time. And so, thank you.

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

• Check out open jobs at our trusted partner companies.
• Watch all Elevate 2021 conference video replays!
• Does your company want to sponsor a Girl Geek Dinner? Talk to us!