This, is the Nielsen Norman Group UX Podcast.
I'm Therese Fessenden.
Now, our last episode, we featured a member of our UX Master Certified community.
To be a part of this community, in other words, to be UX Master Certified, you need to take
about 120 hours of UX training and pass 15 different course exams.
Now, that alone is already an impressive feat, but what's especially impressive is what
these folks go on to do after they receive their certification, often extending their
knowledge to others and changing the way that their teams do UX work.
Our next guest is Angie Lee.
Angie and I crossed paths when she first started at Nielsen Norman Group years ago, but since
then she's gone on to do lots of great things, working for a number of firms, including GSNF,
an advertising agency in Nashville, Home Depot, and eventually Assurion, which is where she
works now.
She's a Senior Manager of Product Design.
In this episode, we discuss what her work looks like now, what the difference is between
customer-facing UX work versus employee-facing UX work, and the special considerations with
balancing priorities like getting really solid buy-in while also protecting research
participants.
It's a tricky balance to strike, but we hope that you enjoy this episode and us talking
shop about UX.
Angie, welcome.
Thank you for joining us today on this horrendously rainy day outside.
How are you doing today?
Where are you right now?
Yeah, I'm in Nashville, Tennessee, right downtown where ... I'm actually in the office,
so I'm in the Assurion office, and we have flex days on Fridays, so I'm one of the only
people here.
Sometimes it can be a little bit lonely or it's kind of weird to have those echoey offices.
I remember there were some days we had flex days at Microsoft back in the day, and I would
walk in and there would be nobody in any of the cubicles around me.
There might be some folks over there, there might be some folks over here, and if I happen
to run into someone, it was like, whoa, hey, a person I know, thankfully.
I'm actually one of the people waving around to turn on the lights that automatically turn
off.
Yeah.
You got to just remind the building that you're in fact there.
Yeah, that's me.
Always fun.
You're at Assurion right now.
What do you do?
How would you describe the work that you're doing there?
I am a senior product design manager here, and we separate ourselves into domains.
Right now I support the expert workspace domain, which is helping our frontline agents with
selling, with supporting on tech-based questions.
Got it.
So a lot of internal support and making sure that employees are given the tools, techniques
that they need, ideally maybe improving their experiences, it's got to be a lot of fun and
kind of interesting.
Like, how has that work differed compared to say some earlier work you've done before?
Or is it the same?
Oh, I would say it's very different.
I think it becomes a lot more complex.
There's a lot of patterns that you can follow and learn and observe from public facing sites
or consumer facing sites.
You're also like the end user of things like, I don't know, my phone, I might have like
a Lyft, Uber, Instacart apps, all of these things that you can test and learn just by
downloading and using them.
When you talk about internal facing tools, you really have to get in the mind behind
like the business needs and the goals of that individual.
So I think it, I find them to be a lot more rewarding and that they're more complex.
Got it.
That'd be kind of fun to go from kind of predictable problems to maybe less predictable or more
complicated, challenging research scenarios as well as design scenarios.
So when it comes to like the user experience for your work, like you mentioned that you
have internal staff, like what kinds of users and situations are you considering in this
work?
Yeah, I might have mentioned this.
But what we call frontline agents are, we call them experts.
So they're the people that when a customer needs help because their phone won't connect
to Wi-Fi or they're having trouble with their Amazon Alexa.
So when they call in or when you call in, someone has to be able to answer that phone
and oftentimes they're managing a lot.
They're looking at the customer's account information, they're looking at their history
as a client.
So I help manage those tools so that they can do their job easily.
And we're looking at augmenting that with generative AI to make that even more seamless.
That's exciting because certainly I feel like everyone's talking about generative AI and
there are so many different applications of it.
There's generative AI creating imagery, creating designs.
So I would love to hear if however much you're allowed to share with us, I would love to
hear what that ultimately might mean as far as the things you're designing.
Yeah.
Yeah.
So okay, one of the things, one of the challenges just as a client commitment and obligation
is that we have a lot of sites locked down so that our frontline agents who we are calling
experts can't access them.
So I'll just give you an example.
Let's say they can't go to a Reddit thread or they wouldn't be able to go to a Reddit
thread, they wouldn't be able to go to Google, but we still need to give them resources to
feel empowered and informed to answer tech related questions about pretty much anything
that has a power button.
So generative AI allows us to have some boundaries and control, but also widens like a plethora
of information that could come in seconds.
Yeah, that's really cool.
So it's interesting that yeah, people can't necessarily go to Reddit.
I can imagine there might be some risk of discrepancy and information given to people
if you're like, yeah, just find the info on Reddit.
It's totally fine.
Right?
Like maybe there is good information on Reddit, but there might be some need to like to your
point control and kind of have a more consistent experience both to the actual staff member,
but also to the people who are on the receiving end.
So it sounds like you have like interactions, you're not just accounting for what tools
are staff using, but also the interactions between staff and customers or end consumers.
And we want to protect their information too.
I think that's really at the heart of why some things are restricted.
It's unfortunate because you would think like, well, if we give them everything that they
need to access, then they would be able to do their job more easily.
But the control piece is protecting us as a business, protecting our client's name and
protecting the customer's information.
So we don't want the ability to like take PII and put it somewhere in the cloud.
So there's, you know, it's just pretty standard across a lot of tech companies now you see
it everywhere.
Yeah.
Yeah.
The data protection piece is really crucial.
And I can just already imagine how many people you have involved just by virtue of the things
you've just said, right?
There's creating designs for internal staff.
There's creating designs for the customers who are interacting with the staff.
There's generative AI that gets factored into this.
There's also data protection.
So how does your team organize all of this work?
Like how is the how is your org structure?
Can you give a kind of cliff notes version of it?
Okay, so that's a great question, because we just went through a little bit of a change
where we're trying to be more intentional about how we partner with data science.
So for a while, data science org has been supporting and getting input from what we
find out in the field and applying that into the generative AI models.
But we found that there could be a more effective way that we partner so that there's less lag
between all the communication channels.
So one of the things that we've done is one product team, which I think is pretty standard,
you would see a product lead, you'd find a product design lead, and then an engineering
lead and several engineers to help execute the work.
Well, now we have that same framework of the three product, product lead, product design
lead, engineering lead, and now a data science lead embedded in the same team.
So that's pretty much a snapshot of what one team would look like.
It seems a bit more decentralized.
So you have product design, you have product development, perhaps you have engineering,
you also have data science.
So they're all kind of working together as teams as opposed to being in kind of these
larger departments.
Is that correct?
I mean, they're probably still are, are they?
Yeah, they still all report up their verticals and through their org, but the fact that our
product teams are working together and we meet weekly, we have standups together that
really then helps us operate as one team, although we're representing like four different
areas.
Got it.
That seems like a much better way to kind of keep that consistency and that flow as
opposed to, hey, we have a project and then there's this other project fitted in when
you can.
Right.
Cause that can sort of stretch out project timelines naturally.
Yeah.
Yeah.
I think it is good.
It's brand new.
So we're still learning how to, to work in the right sequence and cadence.
Cause you know, you might've seen things like design ahead and then the design will be a
sprint ahead.
Well, data science also needs a lot of lead time because the complexity of the models
themselves.
So we're still trying to figure out how to weave in the best practices to support everybody.
Yeah.
Yeah.
That's definitely something to factor in as well.
And I know for me, at least for design, we have this perspective of lead time with research
and also lead time with the actual design process and factoring that in.
But to your point, data science itself needs a bit of time to work these models, which
are very complex and have a lot of variables and things like that.
So, okay.
We have generative AI, we have research with employees and you've recently done some research
with employees.
Right?
Yeah.
Our team, all of the designers in this domain, there's five of us went to Orlando recently
to do some research and it was really eye opening just to sit next to our frontline
agents, our experts, and look at them and how their day goes.
So that work, was it like, I don't know how much you can share of what it's like to be
an employee, but is this like you would go to a call center or is it like you would go
to people's houses because people are working remote?
I'm just kind of curious what that looks like.
So we do have both types of experts, we have our, and it depends on kind of like the client
and the effort, but for this one in Orlando, we were in a call center and we would sit
next to people who, you know, they'd have their headsets, they'd have, like they're
set up with the two monitors and looking back and forth, maybe something, some handouts
and material to reference.
So we really got to a true feeling of their work because we have other tools.
We might have screen recordings, things that they share, but it just doesn't replace being
able to sit side by side with someone and also witness, because it's a call center,
the background noise because people are having conversations and support calls all around
them and kind of that distracting environment.
Yeah.
It's a witness, like the actual context that someone has to do their job in.
And I can imagine that being really loud, I mean, especially you've got calls going
on all over the place, right, and everyone's sort of trying to resolve the issue, speak
at the right volume so that their customer, whoever's on the other line, can hear it,
right?
But at the same time, maybe not too loud, so they're not like disturbing their next
door or next, you know, couple cubicles over, whatever it is.
So yeah, how would you say that research is different from research you maybe have done
with customers in the past?
So research is different when you're dealing with internal employees or experts or front
line agents, because they often work with trainers, with managers about performance.
And sometimes there are audits because we have client commitments to, you know, doing
the right thing and compliance.
So there's also the risk that you are perceived as somebody who might like, you know, report
on bad behavior or tell on somebody.
So I think making sure that you go in sharing that your intent is only to make improvements
to their day to day job and observing the tools that they use that they don't use is
really different than when you're working with an end consumer.
I think it's pretty common practice to understand what a focus group or a user test is.
And they have no like relationship with you as being a co-worker.
I can totally see talking to someone, a customer, I'm going to say like, hey, I'll give you
50, 75 dollars for your feedback.
There's no reason for them to feel like they're going to hurt your feelings as much as someone
you work with.
Yeah, I can imagine the stakes a lot higher when you're working with co-workers.
And like you're saying, there's often this context of audits where people are observed
and have gotten used to being observed, but not for the purposes of, I have a say in the
design that gets updated, right?
It's usually more, oh, I'm observing your performance, right?
And that can be super stressful.
And people might feel the need to perform during this usability test or whatever sort
of research session it is, even though maybe that's not the intent.
Yeah.
Yeah.
I think I've noticed that.
So in the beginning, if we're just warming up to the conversation and it's a one-on-one
interview or discussion, there's some times that rapport building needs to happen in the
beginning because there might be a tendency to go like straight off the book, like this
is what I do and this is why I do it because I have been told to do this thing.
And instead I'm like, I maybe share a little bit about myself, my goals, and try to make
sure that everyone is feeling comfortable and safe to share and reveal a little bit
more.
Yeah.
I can imagine that being super helpful.
Like maybe that added warmup time that maybe you wouldn't normally need in a context where
it's a customer you'll see once and never again, or maybe you do see them again.
But generally that relationship is going to be a little bit different, a little bit more
casual versus this one, which is perhaps going to need that trust really well established
in order for that honesty, in order for someone to admit like, okay, this is what the book
says.
It's not what I do.
Right?
Yeah.
Yeah.
Sometimes these practices, these are good practices to have.
Building the rapport is great for internal employees and for customers themselves, but
it's even more so invaluable when someone feels like you're going to report on them
or they're going through an audit because an end customer doesn't feel that pressure.
Right.
So as far as like setting up studies for employees, like when you're actually studying these internal
experts, what would you say is like a common rookie mistake that either you've seen other
researchers do or that maybe you yourself have done and you kind of learned from it?
I'll tell you one that's I think pretty tactical and just like a logistics thing.
So when you're setting up an interview, you often have to reach out to a few people because
depending on how a frontline agent is paid, they might be paid per call, they might be
paid per hour, but they need to carve out that specific training time for this call.
So there are maybe two or three people involved before you can even get there.
Now one of the risks that people run is that invite just continues to get forwarded to
a few other stakeholders and then suddenly you've got, and I have my product team with
a data scientist, product lead and engineer lead.
So now we have 10 people invested in hearing this person share their insights and perspective
and that's just not the right environment to feel like you can have an open dialogue
about what you like and what you dislike.
So I'd say that's one of the things that I've learned.
I've made that mistake before, not knowing that things would just keep getting forwarded.
So trying to manage the people who are in the invite and finding other ways to distill
that information out.
Yeah, I've been there before where it's like observer sprawl, right?
Where it's just people, and on the one hand it's a good problem because it's like, wow,
everyone is super interested in what is being said here.
They're super interested in making things better and that sort of buy-in is wonderful
to have, but like you're saying, it's not really going to do wonders for honesty and
openness and vulnerability if there are 10 other people that this person doesn't really
know very well observing.
Yeah, I can see that as being really tricky.
How did you or how did you kind of learn how to manage that?
So there are a few things that I've done.
I mean, if I set up the invite, I might make sure that then near the closest stakeholders
to that individual, it might be their, we call them coach, someone who they meet with
training weekly, that they understand that this is a one-on-one conversation.
If there's anything that I'd like that they need to know, then I'll share back separately.
So it's a whole bunch of like relationship building conversations that people understand
the process and the intent.
I have actually heard, someone said to me once like, oh, but I work so closely with
my team.
There's no reason that anything you ask them, you should be able to ask in front of me.
And it was really about building that relationship so that they can understand like, well, you're
their boss.
And although you guys seem to have a good working relationship, you need to understand
like our process is to do things one-on-one and I'll make sure to keep you in the loop.
So I'd say a tip for someone starting out to try to set this up would likely being to
manage those expectations, reach out to people, turn the forward, disable the forward option
on the email invite, and then make sure that the agent themselves even know, like just
because there's a calendar invite, what to expect.
You might say no prep needed.
This is really just about you and me and the design.
Right.
Yeah.
Just making it, making it a bit more insulated for the person who's participating, but also
not necessarily neglecting or just not inviting all those folks because we also want them
to kind of feel involved, especially if they have that buy-in to feel like they're getting
some knowledge out of this and still willing to provide access in a way.
Because like you were saying, you have to kind of get people to approve this time with
these employees.
And so we don't want to just say, oh, you're not invited and then cause this more adversarial
relationship where now someone's like, oh, well, this team is doing research and I never
know what they're doing.
They're taking away from my projects, right?
We want this to be something where those staff members feel like they're benefiting as well
and there's something in it for them.
Yeah.
And we also have this concept of, you know, we want to be conscientious of how many people
we're taking off the floor because that takes like a population of folks who aren't able
to answer calls.
And it depends on the time of day, like is that a high volume day?
All of those nuances.
So there's a lot of people involved.
One thing that I've helped to make people feel like they can have those insights if
they're like not invited to the event is just to do a readout.
So like taking clips of things or collecting themes and then sharing those out live.
It's not just like, here's a report, take the report and do what you want to do with
it.
I'm inviting them then to a follow-up call so we can have a discussion about how we can
make improvements together.
Yeah, that follow-up call has been really essential, especially for, I mean, granted,
I'm in a different context, more in like a consulting slash agency type of context for
a lot of our research projects.
We do individual or independent research for our own courses and such, but when we do these
type of consulting gigs, one of the key things that we tend to do is that follow-up call
and it does two things, one, we do want to show the findings and explain it, give a little
bit more context other than here's a PDF, go read it, but also we can put in these clips
and there's often this misconception and it's one of the things where it's like listen to
what stakeholders do, not what they say because I'll often have these conversations with these
clients or other stakeholders and they're like, we want access to all the recordings
because their intent is to perhaps watch it and to get some benefit out of it, but when
you're interviewing or doing usability research with 13 people, that's at least 13 hours,
if not more of footage, like is someone really going to spend 13 hours of their time other
than the researchers, of course, like is anyone going to really spend 13 hours of their time
looking through all these clips?
Probably not, right?
So if we actually want to make these findings more accessible and to make them more real
to people, sometimes all you need is a few clips.
You don't necessarily need to give access to all 13 hours of footage.
So I definitely agree and second that, having that little clip, that short 30-second to
two-minute clip, however long you want it to be, but short and sweet and just gets to
the point, can be really, really engaging and can really give that sense of empathy
that can otherwise be really hard to come by if I'm just like, here's a quote that someone
said, right?
Yeah, yeah.
And hearing it straight from either the customer or the agent that we have is a lot more powerful,
showing pictures of the environment, sharing like, you know, there's all this background
noise and here's an example of what that could be.
I would say when you're taking these clips, it's really helpful to theme them as well.
So that's synthesizing the research because now instead of having to watch, let's say
13, like you were talking about 13 different video clips and then try to make sense of
the three or to five different topics, then you can say like, well, this is a top theme
of their biggest challenges, challenge one, challenge two, challenge three, and everybody
has touched on them.
Right, right.
And it's kind of pre-analyzed for people in a way because otherwise we're kind of expecting
whoever's reading this report or reading these findings to like come up with that themselves
and a lot of people don't have the energy to do that.
It's a lot of work, right?
I mean, the research is a lot of work.
So yeah, now what comes to mind now is like, obviously research is a lot of work and there
are going to be groups of people that perhaps think, well, it's employees like, we really
need to do this much work with employees when ultimately our moneymaker or revenue
generator would be our end consumers or end customers.
What has been your experience, not just at Asurion, but also like past experiences?
What have you seen as far as like the appetite for research with internal staff?
I'm a little bit conflicted because of the demand for Gen AI.
There's also a need to work really fast, right?
So we want to be able to be one of the first people in the market to leverage that and
learn from things that are in production, but at the same time, I mean, they flew 10
of us out and the entire design team here to Orlando just for a few days to be able
to do some research.
So I'd say that the appetite is very high.
We just have to be able to make sure that we keep in check some of those balances of
like we're getting people off of calls.
So there's a lot of buy-in, but we have to balance with our, to be able to serve the
customer.
Right.
Yeah.
And it's nice to hear that too.
And sorry to put you on the spot for that question, but I'm only asking because I know
that in some other organizations I've been part of in the past and I won't name them,
but there have been some organizations where the appetite for research with internal staff
is low because the idea is just sort of like, oh, they'll kind of get it or they know the
tool.
They have to use it anyway.
They'll figure it out.
We just onboard them.
Right.
We'll give them a PowerPoint slide of screenshots and they can just figure out based on the
screenshots what they need to do.
And on the one hand, I think I get it because you may not have the immediate return on any
investment.
Right.
But on the flip side, there really is a return on investment that if we take some time to
think about it, can make that internal research with employees worth it.
Like what comes to mind is, you know, in the amount of time it takes someone to find an
onboarding document, read the onboarding document, practice it, learn it, and then finally use
this tool that's maybe very poorly designed, then in that period of time, we've wasted
money.
Because like you're saying, call center, just as an example, there's an employee who's got
some task that they're primarily paid for and they're not doing it, right?
They're doing this learning and this troubleshooting or button mashing to figure out how to use
this tool.
Right.
So in a way it is costly, but it's important to do that research to make sure that we do
save money over time, especially the larger the company gets, the more people have to
do this thing and the more money that's essentially costing.
Yeah.
I mean, we have a fully staffed org just focused on training.
Like they create the training materials, they schedule the calls with all of the relevant
folks to get them onboarded.
And one of our goals is to help streamline that process instead of making it like an
eight hour day.
How can we do that?
So how can we build systems that are so intuitive that they need as little training as possible?
Right, right.
Yeah.
It saves time too for the training staff.
I mean, not to say that they're not doing their job, but they're doing it in a more
efficient way and they're able to cover many more topics perhaps.
Yeah.
And they could focus on different things.
Right.
And so what comes to mind for me now is like domain expertise because obviously people
are training to get better at their job, not necessarily better at a tool, right?
There's like a little bit of both that come together, right?
Like to think about a designer.
Obviously a designer wants to get good at using Figma, but you also want Figma to just
become a tool like a paint brush is to a painter, right?
Where they're not thinking about the bristles so much, but instead they're thinking about
what they're doing with it.
So a question I have, this is kind of about domain expertise, right?
There's often this conflict of like keep it as simple as possible, but also make it complex
because there's domain experts who are using this tool and who want the jargon.
So how do you kind of negotiate that and what do you think of as far as like, do you think
you've seen teams like buy into the simplification or have you seen teams buy into, well, I think
we need the jargon there or have you seen a little bit of both?
I think we're, I love this question because we are definitely positioned right now where
we want to make everything as simple as possible.
We have our frontline agents, our experts have made, they've managed with the tools
that they have right now and they've become really efficient.
Like let's talk about efficiency and effectiveness, right?
So they've become super efficient at using less than ideal tools and they've been able,
they know their shortcuts, they have things bookmarked, they have their process, but is
it ideal?
Could it be better?
A hundred percent, thousand percent.
And now with generative AI, that opens up a lot of doors.
So to your initial question, which is, is there buy-in to simplify and streamline processes
for domain experts who already kind of have their ways and methods?
I'd say that there is a lot of buy-in and we know we can shave off minutes, several
minutes off of a session that can allow an agent to be able to, you know, have a little
bit of time to think about their process.
They have a little bit of time to think about like going in between calls because it's call
after call after call.
So if they are really going super fast on one thing, it's just a lot of context switching.
Yeah, I can, I can imagine that is really difficult, right?
Where it's like good luck trying to improve your process if you have calls one minute
after another and you barely have time to like get up and use the restroom or, you know,
just as an example.
I mean, of course, even if it is allowed to do that, that's still mentally something that's
very taxing and very difficult.
So that's great to hear that, you know, this process has ended up yielding these sorts
of outcomes and that research has ultimately really been paying off in that way.
Yeah.
Yeah, it is exciting.
Yeah, and so as far as like how research, you know, you've talked about doing a lot
of research, going to Orlando, working with these call center employees.
How is that guided design choices that your team or other teams maybe have made?
So one of the commitments that our team has made is to try to minimize the amount of UI
elements that we have, because what we have observed is we've got, let's imagine like
a web page and then on the other screen there's another tool, a client tool, and they're looking
back and forth and we've observed a lot of the things that they're ignoring.
So we want to remove some of the things that at some point along the line had committed
to being important.
We're realizing that they're not important, that they're not useful, and how can we remove
them, take away some of the cognitive load to look at all of that and simplify the UI.
So that's just one kind of takeaway that's big because this is a legacy system, it's
kind of baked in and we have reporting built off of this.
So when I'm saying like turn off these UI elements, it's a lot of a bigger undertaking.
Right.
Yeah, it's not like let's just delete this.
Like something you might do in Figma or Sketch, it's like, I can just delete that, right?
And it's like, no, we can't because that exists and there's data associated with this.
Yeah, metrics and events are tied to these systems.
So we have to figure out how to get buy-in to adjust the reporting, maybe challenge why
some reporting streamlines are even set up and then go from there.
Yeah, that's certainly something that's like easy in theory, but hard to do in implementation.
Maybe takes a while.
The other thing I was thinking about is with legacy systems, people kind of have learned
patterns as well.
It's like I remember in the past I did a podcast episode with Paige Laubheimer and one of the
things he said was, when you organize or change these legacy systems, reorganize them, that
it's almost like someone goes into your house and reorganizes your kitchen cabinets.
There might be things you've learned to avoid and like certain pockets where you pay attention
to things like your mugs and your drinking glasses and if they all of a sudden moved
somewhere else, even if it was hypothetically more accessible, right now there's like this
additional learning curve.
And so sometimes a design change is a lot easier in theory and might be harder to do
in implementation.
Yeah, so we have a few work streams right now.
We have one that is like our North Star where they're looking at our end goal because it's
such a large undertaking that they're putting that together.
But we have another effort where we're working kind of in place.
Like what are the things that we can start removing, toggling off or like improving and
simplifying so that we can leverage that and take those learnings and maybe utilize the
same like Gen AI model and put it in our end goal.
So I guess said differently is like I hear what you're saying, taking somebody and just
plopping them into a really nice, effective, like well-designed house would be great.
But then you don't know where the forks are, right?
So just really basic things that you need every day.
And we're being cognizant of that in the way that we're testing in the current environment,
simultaneously building like this big, brand new thing.
Yeah.
I mean, on the one hand too, sometimes it is better to just start new so that you can
just instead of having to overcome existing habits, right?
You're building new ones.
But yeah, it's a constant challenge of where you maybe do a little bit of testing to see
does it work better in this format or does it work better as something totally new?
So it sounds like you guys are working on some really exciting things and that this
work that you've been doing with these internal experts has been really helpful, not just
for your company, but even maybe more fulfilling in the sense that you're doing these really
complex redesigns and challenges.
Yeah, it's been really exciting.
I think we're still getting to learn and come to understand how to partner in these larger
teams because I mentioned data science being part of the product team, but that's not one
person.
It's a lot of people and a standup is almost impossible.
So we're figuring it out and we're ready to take on those challenges.
Awesome.
Well, I think we covered a lot of really cool things and hopefully there are others listening
to this episode who are about to embark on this process of doing research with internal
tools or maybe they already are doing research with internal tools.
If you could offer any advice to someone who is brand new to this sort of work or whether
that's maybe they were previously doing more work within customers and now they're finding
themselves in this new space.
What advice could you give people to make this, I guess advice you would have given
yourself if you had to reflect on when you first started?
I would just say when you are running a usability test or for the first time, let's say, always
pilot.
Always pilot because, and you can pilot with a friend, you can pilot with a coworker that
you already know as it's likely that you guys are working on different projects.
So that gives you some space to reflect and make adjustments and make sure your test questions
aren't biasing the answers, that they're unbiased and that you're able to manage a dialogue
because you're not always going to get direct answers to what you want and how to respond
to those things because there might be a tendency to try to point something out and say like,
but you didn't notice this one thing and so just practice with your peers and get those
reps in.
Absolutely, and I appreciate what you said too.
There's always that temptation, especially for me, a lot of, for example, my family,
if they're struggling with using a particular app, I just tell them where to tap.
I'm like, go to that button and sometimes I do let them struggle a little bit, but it's
really hard to let people struggle through an answer, whether that's an interview question
answer or a usability testing task, and we see that they're having a hard time.
On the one hand, there is a time and place to interject as a facilitator to maybe get
to a particular part of a system that you need to test, but on the other hand, because
you're limited on time.
We don't have all day.
We can't just let them sit there forever, but we do still want to get some semblance
of, is this something someone's ever going to find without our help?
Sometimes we need to give a little bit more time than we're comfortable with.
I know for me as a novice, that was something I really struggled with, is just letting people
go for a while and not feeling this sort of benevolent urge of like, I want to help because
you are helping just in a more long-term sort of way.
Yeah, I think one of the questions when I was running these interviews for feedback
would be like, someone would ask me, I'd ask them a question like, okay, and can you describe
to me what you're looking at?
And they would look to me for confirmation.
Like I think that this is XYZ.
And you have to bite your tongue and be more like, can you tell me more about what you
think it might be?
Or something like that instead of confirming that or denying that question.
Instead of saying, yes, exactly, that's it.
We can't give them the, yes, you're correct or no, that's wrong.
We're like, yes, you got it.
Or even if it's wrong, we don't want to make them feel bad either.
We kind of want to just have that poker face, which is just hard.
Yeah.
Oh, and I have another tip actually.
I'll say that in terms of building the rapport, especially for testing with internal employees,
find ways to connect.
If you happen to be in their physical space and I would see somebody with a drawing or
a photo, I might ask them about it.
And that's a really easy way to just start building a connection, making someone feel
like a little bit less anxious and just really aligning to that human element.
Yes, for sure.
I think that's a great technique and a good way too, because there is a challenge at the
same time where we want to build rapport, but like you were saying, we don't want to
give people this sort of confirmation position where it's like people are looking to us for
the right answer when that's not why we're here.
We're here just to listen.
And sometimes that might also mean we can't say much about ourselves or we don't want
to cause someone to doubt their position or the validity of whatever it is they're saying.
And so sometimes we have to say less about ourselves intentionally.
And for me, I overshare all the time.
So that can often be really hard to, if someone says, oh, I'm from this place and I happen
to be from the same place.
Sometimes it's a game time decision.
It's like, do I really want to say I'm from there as well to make that connection or am
I going to kind of let my enthusiasm stew in my chest, but find another way to connect?
It can often be a bit tricky to find the right balance there.
Yeah.
Maybe like something that's a little closed.
Like one of the experts I worked with had several Rubik's cubes on his desk and I was
just like, oh, that's really cool.
And we just connected on that a little bit.
I'm not a big Rubik's cube person and it wasn't like super personal that went into a lot of
dialogue.
So I thought that's just an example.
Yeah.
But it does give people a chance.
It gives people a chance to get excited about something they are excited about.
It gives you a chance to make a connection based on something they care about as opposed
to something you care about that you're kind of pulling them into.
Right.
Exactly.
Exactly.
Well, I think that's great advice.
So yeah, thank you for sharing that.
And as far as like, if people want to learn more about running studies or about you, do
you have any resources or social media or places you can point people to?
Yeah.
Well, you can find me.
I'm on Instagram and I have my site, Angie Lee, A-N-G-I-E-L-I dot com.
My Instagram is not professional, but you can DM me there.
It's like my life, all things.
It's not just UX, but I'm happy to answer any questions in there too.
Awesome.
Well, Angie, thank you so much for your time today.
This has been so fun, been a lot of fun geeking out about research, but especially just catching
up with you as well.
So hope you have a great rest of your day and hopefully we get a chance to talk again
soon.
Yeah.
Thanks for having me.
That was Angie Lee.
If you want to learn more about her work, check out the links in the show notes.
Also, check out our website where you can find thousands of UX articles, videos, reports
about UX design, research strategy, even UX careers.
That website is www.nngroup.com.
On that note, if you want to stay up to date on our latest research and publications, we
do have a weekly email newsletter, which features our latest articles, videos, and upcoming
courses as well.
And of course, if you enjoy this show in particular, please follow or subscribe on the podcast
platform of your choice, maybe even YouTube.
This show was hosted and produced by me, Therese Fessenden.
All editing and post-production is by Jacob Swing and the Larimore Production Company.
Music is by Tiny Music and Dresden the Flamingo.
That's it for today's show.
Until next time, remember, keep it simple.