Amy is the Director of EMEA Marketing at UserTesting.com. She joined Andrew on the podcast to discuss how user testing helps businesses to understand why certain things are happening within the business.
She also explained how testing can be used by almost any department within a business and how it can help startup businesses before they’ve even launched a product!Also available on: or from your smart speaker.
Andrew Veitch: Hello, and welcome to The Joy of Marketing with me Andrew Veitch. This week, I’m joined by Amy Kelly, Director of EMEA marketing at UserTesting. I’ve long been a big fan of user testing and I’ve actually used user testing for many years as an E commerce solution. So it’s great to be joined by Amy, welcome to the show.
Amy Kelly: Thank you, Andrew, it’s lovely to be here. Thank you for inviting me.
AV: So I guess a great place to start, in fact that there’s a quote from Jim Barksdale, which I really like, which is, if we have data, let’s look at the data, if all we have are opinions, let’s go with mine. But I think that is a part of a bigger question about culture, isn’t it? Because you can have a culture of going with opinions. Or you can have a culture of test and learn, I guess, can’t you?
AK: Definitely I think it comes down to you know, how your organisation tries to rally around the customer. You know, like, what we try and do at UserTesting is build that culture of empathy. So just around, you know, what you’re saying there, Andrew, like test and learn, it’s just, you know, it’s avoiding that conversation of saying, you know, who’s the hippo in the room and who gets to make that final decision, doesn’t it? I think it’s definitely just about getting your organisation to think always about the customer to really understand that, you know, it’s not always just going to, as you said, your opinion’s going to be what launches this campaign, or what launches this product. It’s have we checked this with our key audience, because you should always start with who you’re serving, you should always go with, you know, who is this customer? What are we trying to do? And having that front of mind at all times. a
And I think that it does have to come from the higher up, you know, we’ve got a fantastic CEO, Andy MacMillan, who very much does focus on the customer at all times they think that does trickle down into an organisation, so that’s how you can obviously have a C level be really quite important in driving that culture of the test and learn understanding customers, and having someone always say, okay, have we tested that? Have we checked that with our key audience? Have we went ahead and even checked that this is something they want to have, you know, even before we build it? And I think that, you know, definitely while I’ve been at UserTesting, I’ve noticed that we are, as you say, practising what we preach, we say, drink our own champagne, you know, we’re launching our own campaigns through the UserTesting platform all the time as well. And I think it does, it makes a massive difference.
AV: Drinking your own champagne is a much nicer way of putting it across than the traditional eat your own dog food, which we used to say, in the software business.
But yeah, I totally agree. I think it is incredibly important at this top level, not to bring your own opinions to the table. I mean, something I found in 15 odd years in marketing, is that my personal views and the customer’s personal views are generally radically different.
In fact I was virtually at the stage, I think, when I worked for surfskate fashion business, the buyers would speak to me and ask them for my opinion, because if I liked something, it’d be a certainty it wouldn’t sell. If I didn’t like something, it was likely to be a top seller. So I think yeah, I mean, you’re absolutely right. If you if you bring your own personal opinions too much to the table, you know, it’s just not going to work.
AK: I think it’s definitely a balance for sure. Because if you think of all these folks, that they’re professionals, they know, they’re, they know what they’re doing, you know, when we think of digital designers, or anyone creating an online experience, you have that trust in your employees that they’re doing the best job they can. And it’s it is a guess, a balance of having your own opinions, and then also making sure that you are gaining the insights from the customers directly. Because that way, you know, you’re not going to lose them, you know, if you’ve got the professionals are doing their job and then you’re also validating things with your audience. You know, that’s really where you’re going to build that positive culture of test and learn.
AV: Yes, yes, you’re absolutely right, because opinion isn’t a bad way of thinking of something to test as long as you test it before you actually run with it.
So maybe just stepping back a bit. I mean, obviously, I’m aware of usertesting.com which you know I said I’ve used for many, many years. But perhaps people in the audience aren’t familiar with it. So could you maybe just step back and explain what it is you do?
AK: Of course, yeah, UserTesting is a leading human insight platform. And what we mean by human insight is we’re helping teams gather feedback at scale, and really helping you make decisions with you know, all the insight you need to make that decision with confidence really.
And the way it works is, I’ll walk you through an example where you know, if there’s a Digital Designer or UX designer or someone that’s creating an online experience, they may have got to the point where they feel – I’m happy with what I’ve created, I’m just going to test this now with my audience. So you go into the platform, and you zone in on that demographic first of all, so you try and find the participants, we have got a worldwide contributor network where they’re already embedded in the platform. So you don’t have to go and try and find these participants, we already have them there for you. And you will select that URL or wherever you want them to start. And then you will detail the test and exactly what you want them to do. And you’re trying to say, for example, they might be trying to test the usability or it might just be feedback and what do you think of this experience? Or what do you think of this creative? What do you think of anything at all, you know, give me that feedback and what I’m trying to find the insight and test on and, and then really, the results are video based feedback and video based feedback is so powerful, because you get to see that emotion. So we actually can have the camera on the participants, so you can see facial expressions, like if they’re scratching their head thinking this is a really difficult experience or you might just have the verbal feedback and they will be telling you exactly what they’re doing while they’re doing it, how they feel about it, their expectations, and probably could shine some light on something that’s really went wrong there like what pain and that feeling to help you then go back and iterate and make that experience even better.
So ultimately, as I said, it’s a scalable way to get feedback and you know, for traditional methods been a bit different for focus groups are in lab, in lab testing, this is just a faster way to to get that insight.
AV: Yeah, absolutely. I mean, I had somebody in the web team at Diet Chef who actually just used to go and sit beside someone as they went through the website, which was effective but obviously, the difficulty with that is trying to scale that up to 20/30 people and trying to have it people from different demographics, different countries, obviously, all got gotten very difficult. But I think, you know, I think that this product is I mean, that’s essentially what it’s doing isn’t that you’re just sitting beside somebody virtually.
And I guess it’s not only that you see problems, but it’s much more specific, isn’t it? So rather than just knowing there’s a problem on this page, which a tool might tell you, this is telling you specifically, what the problem actually is.
And again, actually one of the from Diet Chef, actually, we, when we launched the US site, we used user testing. I remember one of the things that came up is we had a card expiry date. And of course, in British English, you would say expiring and US English is expiration. And of course, the Americans hated seeing expiry, they thought it was maybe a fake website. But I mean, that’s, there’s no way that we could have got that insight through any other method, apart from just getting some Americans to test it and, and give us that feedback.
AK: Yeah, it’s definitely about breaking assumptions, Andrew, because I think that’s what we’re talking about earlier, it’s just thinking that you are the professional, you know how to do this job, go ahead. But at the end of the day, you know, like, if you are just taking it all for granted, and not not able to sort of break those assumptions, you know, that’s where things do end up in rabbit holes. And if you’re not getting that insight, you don’t know for sure if this is going to work. So yeah, it is I use the platform all the time for marketing campaigns. And just the insight and feedback I get, I think helps all of our campaigns improve massively.
AV: So Amy, who is it within the business that should actually be doing user testing?
AK: So I think typically, a lot of people think of user testing, and might immediately think that we sit within the UX box, you know, we have definitely worked with UX researchers and UX designers for a very long time and being critical to their tech stack. But now what we’re seeing is because of, say, for example, what the pandemic did, is just push everyone to this digital first. And now, you know, if we take the industry of E commerce is, you know, having to have that really great experience online, I think people really came to around the power of customer experience, and the power of the voice of the customer. So actually, now we’re seeing a lot more marketers, a lot more product managers, and C suite getting involved with listening.
So I think when we take the word research out of it, and we just say, Listen, it’s just getting feedback, or it’s listening to your customers, simplifying that and helping people understand that doing this will improve how much empathy you have for your customers, and ultimately, building a better business.
So I think that’s quite exciting at the moment, especially for myself as a marketer. I had never thought about getting my own feedback like that, you know, I would rely on agencies or people to tell me what the customers thought, you know, when I worked at Facebook, or I worked at YouTube for a while, and it was you know, we were getting an insights team coming to us and giving us reports, but I never was going out and testing anything myself or seeing a video of someone giving feedback on something that I created or something I wrote or creative I worked on whatever that was. So since I’ve been at UserTesting for the last three years, I’ve really completely changed how I market you know, and I do a lot of testing myself. And I love getting that feedback, because I might be so excited about something and think, oh, this campaigns amazing and really excited about this creative that we’ve generated and worked with the brand team on, and then we could sit, test it out and just get the most horrific feedback, and go okay, right, we definitely not thought of the customer during this whole excitement thinking that we’re great and we’re geniuses, when really, actually we should have probably started with the customer.
So I think what I was saying earlier, about the startup founders as well, is, is that whole element of understanding your customers when you’re when you’re building the product, or even before you’ve done it, we’ve got an amazing customer, John Talbert, who heads up Purple Dot, so he works in the E commerce world and he said that he actually takes the feedback from his customers of what they say they think that Purple Dot is, and he used that narrative as how to even sell his product. So exactly what the customer says he put on his website, because that’s how they interpreted the product. So I think when you’re thinking of a start up founder thinking, Okay, I’m going to test this with my audience, hearing feedback of what someone thinks of the product, and then actually using that in your marketing material. That’s pretty cool.
AV: Absolutely, absolutely. There’s nothing more powerful than actually just seeing the video of the customer, using the product and talking about it.
So where does UserTesting fit in with all of the other testing products and solutions that we’ll be using?
AK: So I think there’s an array of different testing solutions of what you’re, depends of course, you’re testing. But if you are complementing testing with, the platform UserTesting, you might have something like Qualtrics, or Survey Monkey, where you’re getting the big data, right. So that’s always important that quantitative data. But with that, you’re understanding perhaps what that is, but you don’t understand, understand why.
So when we talk about the why. That’s where UserTesting helps you have that more human approach. So I think that there’s big testing with big data and there’s different things like eye tracking, and there’s loads of solutions to just help enhance experience, you know, the experience software kind of market is much bigger now, because now organisations realise the impact of really top CX, top CX customer experience is just such a game changer now for being a leader or being a laggard, you know, who’s getting it right and who’s not. And definitely, with user testing, it does complement, you know, we’ve got customers use an array of solutions, which we’re partnered with as well, and we help with integrations to make that easier.
But, yeah, I guess it comes down to what they’re testing. But with user, UserTesting platform, you are getting the why, you know, when you are trying to understand something that is going to help you understand why something is happening, or why someone feels that way and you’re going to have other testing solutions to help you perhaps understand what that is.
AV: I mean, when do you think it’s a good idea to do testing? And is it when you sort of, is it something you should do like regularly, like, once a quarter? Or would you more link it in to when you make changes?
AK: I think it’s, you know, connected to the first question around culture, you know, if if you’re thinking regular testing, regular feedback, I think some people might get put off by the word research, for example, because it could just be as simple as just gaining feedback at key stages. So you know, if you’re creating a software product, or something that’s like an app, or you know, you’re gonna have regular iterations. And I think at any key changes, when you are making changes to the product, that is when you have to test it. So integrating those key testing stages throughout the product development lifecycle. So that’s definitely something we’d say.
But one thing that I love, talking to different startup leaders is, you know, testing before you even launch the idea. Now, there’s an amazing chief product officer as and also CEO, Arielle Kilroy, who’s the Chief Product Officer over at Dado HR, and she talks always around validating idea stage as much as product stage. So even if you were to put your ideas down on a slide, or you know, the most roughest form of the idea, send that out and even if you don’t have UserTesting, and you’re just sending it to a friend or anything, it’s just validating – Is this a good idea and do you think there’s a need for this because you, perhaps are within this demographic that we’re trying to serve. And I think that’s a powerful thing for startups to start to think about is, let’s test before we get too far and we have a really expensive change later on before we even start building, let’s test it now. And I think from that offset, especially if you’re a smaller organisation, and you’re starting to scale. You’re going to build that culture of test and learn right from the beginning, you know, if that’s, okay, we’ve tested it before we’ve launched and then we’ve tested it, you know, at this key stages of this, you know, design change, or whatever it might be, or, you know, we’re testing it when we’re seeing a drop in behaviour or sales conversions, you’re measuring whatever that may be, you know, just just having that as part of your narrative and your product team and your digital team have, have we tested that? Have we tested that? And I think that’s when it becomes a regular thing.
There’s no, there’s not any strict rule, I guess, on when to do it. But I would say the better you know, the more the better for sure. Yeah.
AV: You know, thinking about one of the big drivers in my E commerce business was actually settling arguments, I think, because quite often, me and my business partner would have an argument about something, but fortunately, you know, we were both data driven people. So if we, if we had a disagreement, we would put together a test, and then we would look at the data and we would actually end up with with an absolute proper answer, rather than I said, just just just diving in with what we thought.
But yeah, you mentioned that about testing, because I suppose that is another driver is kind of testing when when the performance dips, and obviously in E commerce, you know, the big thing that people are always thinking about is conversion rates isn’t it?
So I mean, how would you use testing to improve conversion?
AK: So I think it comes back to that question, why? So perhaps if you’re measuring conversions, and you see that drop, and that’s a concern, and everyone gets together, and we say, we’ve got these figures, we can see this graph, there’s a massive plummet, having something like user testing, or doing that actual feedback with with the customer, and getting them to explain why, is so powerful because they might shine a light in the fact that like that check out button is not even working guys, you know, it might be as simple as that. Or, you know, I just didn’t like the fact that sent me back to that page or whatever that that why is, and getting that real human insight can completely change that design of the experience and help improve conversions.
If you work in conversion rate optimization, it is so critical to not only get the big data and to manage that and see what’s happening trend wise, but also to understand what is the reaction to customers. What are they saying? How are they feeling? And building that full picture can help you make informed decisions with confidence. And at the end of the day, you’re going to be able to look into perhaps why some things went down or why some things went up, you know, that’s also great news and, and really figure out how to optimise on that. Because you’ll be getting that why you’ll be getting that insight and think, okay, this is how we have to make that change, this is what we’re going to do going forward. And ultimately, you should be able to increase conversions by having that fully informed decision making power.
AV: Yeah, absolutely agree. The conversions start with knowing what the problems are. So at Diet Chef, what I did is I had a list of the top 10 reasons people weren’t buying, which we compiled once a month, and the moment you have a list of the top 10 reasons people aren’t buying, conversion rate optimization suddenly gets a lot easier, because you know what it is that you’re actually trying to fix. Whereas I’ve seen quite a few people dive into conversion optimization, without really understanding what the problem actually is.
But moving on, just from the sort of core transactional behaviour of someone coming to the website and buying, I mean, can we use testing to sort of see some of the bigger picture too?
AK: Of course, I mean, really, that the platform itself is there for you to use, however you see fit. And a lot of the time let’s say when we’re actually going through things like brand refreshes, we will act, we’ll just want someone to kind of give us their opinion, generally, it’s all about how you phrase the questions.
So if you have a test that is based around usability, you are probably going to say, you know, for example, you’re shopping on ASOS, I would like you to go to the dresses page, and I’d like you to try and find a red dress. And then I want you to go through the experience and add shoes and all that. And that would be a test that you are very much asking questions around that step by step experience.
But if you’re actually just wanting to understand sentiment, then that’s also a way that you can use things and and marketers, you know, definitely I think it’s important before any massive creative changes, is just getting that insight and feedback on how someone feels about design or, you know, how do you feel about this change that we’re making to the brand? Or what do you think of this messaging? And just letting someone speak out loud, freely, it’s an open ended question, and be careful of how you build those questions.
But I think that that’s where you can really test anything, you can check anything you want. So I think that’s the powerful aspect of it is that you don’t have to just be checking usability. You can understand sentiment or there’s even employee teams that are using it to understand how their employees are feeling within the business and why they like working there and why they don’t you know, you can really test anything.
AV: Yeah. And certainly, again, Diet Chef, I also used it really early in the journey. So we would actually ask, sometimes we would ask people to start at Google searching for a delivered diet product and then we’d follow the chain all the way from, you know, right from that early stage, even before they arrived at the website. So you could just really understand what they were thinking when they were presented with all these different adverts.
So have you got any other sort of tips for what’s good to test? For sort of the direct to consumer listeners?
AK: I think it’s really important to align your focus around testing to your business strategy. So I think it’s, you know, understanding that there could be a million things you could test, but you’ve got to prioritise.
So if you’re looking at how to enhance conversions, or make your brand better, or just reach more people with a more convincing message, it’s really comes down to, you know, what are you trying to do within your organisation? What’s your focus areas, and what are your teams trying to do right now. And I think that’s where you can inform the, say, the testing roadmap, or however you want to position it, as something that’s critical to the culture, critical to the team’s, you know, not just the digital teams, but getting your CEO to use the platform, or, you know, do some user testing themselves to understand who the customers are. So they’re always bridging that gap.
And what we talk about is the empathy gap. Because if you’re not understanding your customers, you’re creating experiences blind. So if you’re not able to understand how your customers are feeling, or even just who they are, you know, put a person behind that persona, you know, don’t just like assume that someone’s this way, because you’ve written it down on a piece of paper, you know, Amy’s 34, she lives in Edinburgh, you know, she’s works in marketing, but you still don’t know me. So you have to speak to me, you have to go and understand what it is I want, like, what it is I need and how are you trying to help me as an organisation?
So aligning all of your testing focus around your business strategy, and just how the different teams are going to integrate that testing. And because you know, the UX designer might have a completely different test than the product manager. So just making sure it’s all relevant, and people are doing the correct testing to help them within their roles.
AV: Yeah, I can absolutely see that. I mean, I used it for everything. I mean, obviously, they’re just a practical way of, you know, other problems with the website, are there things that don’t work? Are there things that aren’t clear? Which is all very obvious.
But I think also some of that bigger picture. How do I compare with the competitors? So actually one of the things again, I used to do was, obviously doing it anonymously, so that testers weren’t aware of who it was, but I would test our competitors websites and our own websites. And then I would get the view, the view of how they compared, which again, I found enormously valuable.
AK: Yeah, comparison testing is really powerful for sure. Definitely, just getting that understanding of how people feel against one or the other, for sure. I really, really like that, like, do that with our own company as well.
AV: Yes. Well, you know, I’m just thinking, we definitely need to do some some user testing Machine Labs.
AK: Definitely, it’s good for the inset. I love just to understand even you know, when I’m not talking about using the platform, I love speaking to customers and the first thing I will say is why UserTesting? Why did you come to UserTesting? You know, what is it about UserTesting.com that’s like came out to you? So that I can, you know, just get more of an understanding of what it is that we’re good at, and perhaps what we have to improve on as well. And ultimately, that helps us market to those folks better as well.
AV: Yeah, well, I’m definitely glad to know that you’re doing user testing, too. Thank you very much, Amy. I think that’s been really helpful. I’ve learned a lot and I’m sure the listeners have to. So thank you very much for coming on.
AK: Thank you for having me.
AV: If you’re in E commerce and would like to get to know your customers more, then install Machine Labs. We’re on the Shopify app store or you can use our API if you’re on other platforms.
We’re a marketing automation platform that can help you really personalise the messages for your customers. We look at more than 950 variables, everything from age to income, to rural or urban, to hobbies and interests to household composition. And then we can help put together a really personalised recommendation, treating each customer individually.
So hope you enjoyed the show and I’ll see you next week on The Joy of Marketing