I’m doing things a bit differently for this episode of Experiencing Data. For the first time on the show, I’m hosting a panel discussion. I’m joined by Thomson Reuters’s Simon Landry, Sumo Logic’s Greg Nudelman, and Google’s Paz Perez to chat about how we design user experiences that improve people’s lives and create business impact when we expose LLM capabilities to our users.

With the rise of AI, there are a lot of opportunities for innovation, but there are also many challenges—and frankly, my feeling is that a lot of these capabilities right now are making things worse for users, not better. We’re looking at a range of topics such as the pros and cons of AI-first thinking, collaboration between UX designers and ML engineers, and the necessity of diversifying design teams when integrating AI and LLMs into b2b products.

Highlights/ Skip to

• Thoughts on how the current state of LLMs implementations and its impact on user experience (1:51)
• The problems that can come with the "AI-first" design philosophy (7:58)
• Should a company's design resources be spent on going toward AI development? (17:20)
• How designers can navigate "fuzzy experiences” (21:28)
• Why you need to narrow and clearly define the problems you’re trying to solve when building LLMs products (27:35)
• Why diversity matters in your design and research teams when building LLMs (31:56)
• Where you can find more from Paz, Greg, and Simon (40:43)

Quotes from Today’s Episode

• “[AI] will connect the dots. It will argue pro, it will argue against, it will create evidence supporting and refuting, so it’s really up to us to kind of drive this. If we understand the capabilities, then it is an almost limitless field of possibility. And these things are taught, and it’s a fundamentally different approach to how we build user interfaces. They’re no longer completely deterministic. They’re also extremely personalized to the point where it’s ridiculous.” - Greg Nudelman (12:47)
• “To put an LLM into a product means that there’s a non-zero chance your user is going to have a [negative] experience and no longer be your customer. That is a giant reputational risk, and there’s also a financial cost associated with running these models. I think we need to take more of a service design lens when it comes to [designing our products with AI] and ask what is the thing somebody wants to do… not on my website, but in their lives? What brings them to my [product]? How can I imagine a different world that leverages these capabilities to help them do their job? Because what [designers] are competing against is [a customer workflow] that probably worked well enough.” - Simon Landry (15:41)
• “When we go general availability (GA) with a product, that traditionally means [designers] have done all the research, got everything perfect, and it’s all great, right? Today, GA is a starting gun. We don’t know [if the product is working] unless we [seek out user feedback]. A massive research method is needed. [We need qualitative research] like sitting down with the customer and watching them use the product to really understand what is happening[…] but you also need to collect data. What are they typing in? What are they getting back? Is somebody who’s typing in this type of question always having a short interaction? Let’s dig into it with rapid, iterative testing and evaluation, so that we can update our model and then move forward. Launching a product these days means the starting guns have been fired. Put the research to work to figure out the next step.” - (23:29) Greg Nudelman
• “I think that having diversity on your design team (i.e. gender, level of experience, etc.) is critical. We’ve already seen some terrible outcomes. Multiple examples where an LLM is crafting horrendous emails, introductions, and so on. This is exactly why UXers need to get involved [with building LLMs]. This is why diversity in UX and on your tech team that deals with AI is so valuable. Number one piece of advice: get some researchers. Number two: make sure your team is diverse.” - Greg Nudelman (32:39)
• “It’s extremely important to have UX talks with researchers, content designers, and data teams. It’s important to understand what a user is trying to do, the context [of their decisions], and the intention. [Designers] need to help [the data team] understand the types of data and prompts being used to train models. Those things are better when they’re written and thought of by [designers] who understand where the user is coming from. [Design teams working with data teams] are getting much better results than the [teams] that are working in a vacuum.” - Paz Perez (35:19)

Links

• Milly Barker’s LinkedIn post
• Greg Nudelman’s Value Matrix Article
• Greg Nudelman website
• Paz Perez on Medium
• Paz Perez on LinkedIn
• Simon Landry LinkedIn

Transcript

Brian: Welcome back to Experiencing Data. This is Brian T. O’Neill. Today we have a new format we’re trying out on the show. I’ve got three panelists here, all from the field of user experience design and research, particularly focused on artificial intelligence.

And my friend Simon here, we just got to know each other from Thomson Reuters—Simon Landry, welcome—you had prompted me to, like, have a conversation about designing for probabilistic systems, right, when basically we are now out of full control over what these systems are doing. And I thought, “Oh, this would actually kind of be a fun topic to get a panel together.” So, I have Paz here from Google, and my friend Greg, who’s been writing a ton of material. You've already have books out, and I know you have a book out on UX. Is it called UX for AI?

Greg: It is, yes.

Brian: That’s the name of the book. Cool. Tell us about that at the end. But I wanted to just jump in. We’re going to have a casual conversation here. I’ve got some questions, and I think we’ll be probably throwing questions at each other here. But first thing I wanted to ask, this is probably totally obvious right now, but what do you all think the state of user experience is right now around, particularly, large language model implementations.

And I’m trying to focus this less on the foundation model, the websites that we all use, ChatGPT, Claude, and these kinds of things because they are effectively trying to solve everything for everyone. So, in terms of a little bit more narrow scope, have you all found some practical tools, applications that are leveraging these in a way that’s actually been useful to you from a user experience standpoint, like it’s substantially made your experience with the product better? I don’t think that I have yet. I’ve seen some nominal impacts in some of the work. Zapier has been a little bit helpful here and there. Code writing has been helpful for me personally. That still takes a while, and I don’t do a lot of coding anymore. But I just wanted to throw this at you all. What do you think the state of maturity is right now around this?

Everyone’s quiet. Oh, no one wants to say it’s crappy [laugh].

Simon: [laugh].

Greg: I’ll jump in, sure. I think code writing is, as you mentioned, great. The Code Copilot, GitHub, is definitely up there as far as usefulness, I’d like to think that something like Grammarly, despite its obvious shortcomings, is actually a very nice tool, although it seems like that this thing that it does now multiple passes, it’s extremely annoying, and I think it’s part of the justification of UX that a lot of the products are experiencing now that we’re kind of adding these AI features. And I like to think that the Sumo Copilot that we just released GA is actually pretty good and solves a real problem for our customers, that is writing a query in natural language, and then it translates it into the SumoQL, and then runs it. So, there are different sporadic things, but there is not really patterns of use that I think we can call out with certainty and say, “Yeah, this is absolutely how you do it,” the way that we’ve learned to do with mobile and web. I think that’s still in the works, very much.

Brian: Yeah. Paz, what’s your take on this?

Paz: I think as an industry, we are trying to catch up. We are extremely green, still. There’s so much to learn. We’re trying to find our footing. That’s how I feel. In terms of process, yes, there are tools, as the one Greg was mentioning, and we have, like, Miro, like, all these, like, [tableaus 00:04:33] that will help you summarize insights and expedite the process that you are doing right now.

But that’s not going to be the big game changer, in my mind, is how we are going to be shaping models, how we are going to be creating products that we don’t even imagine right now. And for that, we need to learn, as UX designers, and we’re not there yet. So, it’s exciting what’s happening. There’s, like, cool products out there. I personally use a lot of NoteLM. I think it’s fantastic—Notebook—but saying that we are very much at the beginning, and there’s so much more to come.

Brian: Yeah, yeah.

Simon: I think I just want to touch back on Grammarly. Greg, you mentioned Grammarly. I think Grammarly has the best—and I haven’t tried everything, obviously—but the best LLM UI I’ve ever seen. They provide buttons, not chat first, which is, I think, fantastic. And just this idea of when ChatGPT first came out, it feels like the whole industry conflated the UI with the technology, and we were like, well, if you have an LLM, it needs to be a chat UI, where the reality is just, like, a really effective trick that got us all to look.

Paz: Yeah, I mean, like, those chat interfaces, they’ve been very popular because you can put it on the side and it doesn’t break the main UI. So, I think, like, most companies have embraced it because it’s just it doesn’t mess with their business. But that’s not where the true innovation is going to come from.

Greg: Yeah, I think it’s—AI is such a transformative technology, and I think we need to start thinking AI first, very much like we were thinking mobile first, and that really revolutionized how we approached UI design. I think we should do the same thing with AI and really start thinking AI first, about our business models, about our relationship with customers, about what the jobs to be done are with our product and how to expand to new markets. I think that’s really the future, and that, unfortunately, is not something that a lot of UX people are equipped to do at the moment, so I’m really glad that, Brian, you’re doing this because I think this will help a lot.

Brian: Yeah. Simon, you look like you’re itching to comment on something there [laugh].

Simon: Yeah. I just want to put words to this, like, AI-first designing and, Brian, go back to what initiated our conversation in the start, but it’s this idea of AI-powered experiences are fuzzy in that they can go in so many ways. And as product owners or designers, we cannot control—there is no happy path because you don’t control the model and you don’t control the user. So, it’s just a fuzzy experience. Whereas our traditional SaaS, where it’s just like a user interface, then it’s a much more crisp experience.

And these are not my concepts. I didn’t invent these two categories. It’s someone from Anthropic that put those words, but I really like it. And so, the idea is, as product people or designers or what have you, we need to understand, how do you design an experience where you cannot anticipate what’s going to happen, and how do you make sure that is a good product experience, so that our customers aren’t like, “I don’t know what I’m supposed to do here. I’m just going to keep on doing things the way that I used to.” Because ultimately, that’s the competitor is, just the way they’ve always done it.

Brian: I guess I want to challenge one thing about the whole AI-first thing. I guess—and this is similar to the concept of mobile first. So, to me, the goal really wasn’t to be mobile first. The whole mobile first thing was a way to reduce things down to, like, the basics. What do you have to get basically correct most of the time to have a great product?

It helped you remove all the other stuff that’s not absolutely necessary, but really that’s because someone—product teams—didn’t want to be disciplined in making good decisions about what is most important to focus on. The screen forced us to [laugh] do that work. So, the same thing for me with AI. And I know where you’re coming from, Greg, and I do think there is a lot of potential for the technology, but I really come back to, like, it has to be benefits first. What is the benefit that you’re going to give to the customer?

And is AI or not, or some mixed experience, which I think most of the time, products worth paying for are going to be a mixed experience of deterministic software enhanced with artificial intelligence at the right time in order to give a benefit that is so much better than whatever the status quo is. That’s really what the game is. And I think if we try to use AI first all the time, we may fail because in this—the data science community talks about this all the time; you know, in large enterprises, they’re getting asked all the time, what is our—now it’s, what is our GenAI strategy? It used to be, what is our AI strategy? And a couple years ago, it was, what are we doing with machine learning?

And you can keep backing this stuff up. Then it was like regular analytics. They’re just watching this thing, the words change, and it’s all tactics first, as opposed to what benefit or what problem are we trying to solve, and what benefit are we trying to get? And then is a large language model implementation really the way to do that? Maybe a small language model would be better. Or maybe you don’t need any of that at all, like, or some combination in between. I just want to throw that [laugh] throw that at all of you, just for reaction.

Paz: I mean, I would say, when you say, like, mobile first, sure it was about, like, having experience that is a little bit more, kind of, like, targeted and concise, but also there was a lot of new possibilities that were happening because you were now, like, moving in the world, things that you couldn’t do with that desktop. So, it’s the same thing. At the beginning, we’re putting, like, this, there’s the web experiences into a mobile, kind of, like cramping everything there, and then we figure it out, like, new possibilities. We are doing that with AI right now. We’re using the experience that we have, and we’re kind of cramming in AI to see where we can, like, put something, just because it’s coming from everywhere, in different companies, but at some point we’re going to see these new capabilities for what they are, and there’s going to be, like, new paths that we’re not seeing just yet. We need to keep pushing as an industry to find those.

Brian: Yeah, I’m sure you all are doing experiments, and I love experiments, but the question is, like, are we simply trying to put out a feature and saying, if we ship a feature that has today’s hot implementation, pick whatever it is, then we now can say the product has X, but none of that actually says our benchmark was this, our benchmark for quality, or a benchmark for delight, or our benchmark for revenue—or hopefully all those things, if it’s a business—was X, and we’re trying to move it to Y, and did this feature actually do that or not? What I feel when I run into these experiences is the product team, or somebody, said, we need to do something with this. So, let’s throw a text box in and let’s summarize your LinkedIn post for you, or let’s summarize all your LinkedIn posts for you. Like, because I need to do what? And, like, and again, what was the benchmark as a user of that service? What was my satisfaction level before and after?

The point is you have to have a measurement of what quality is to know if you made an improvement. And what I feel like we’re doing is throwing stuff at the wall. And, like, Jared Spool is, like—I feel like—I was just listening to his thing, and we talk once in a while, and he’s pretty negative on a lot of this. He feels like this will be great for UX because we are going to make an effing mess, and then people are going to—not just us, but, like, all the people that don’t have user experience resources, are going to make a huge mess, [laugh] and guess what the problem is going to be? It’s not going to be the technology because it’s rapidly advancing to the point where you can almost do anything now. So, what’s going to be left? Cleaning up the mess to make it actually easy to get the value. I don’t know, like, argue the other side of this. I would love to push back [laugh].

Greg: I love that because I think the—Chris [Noessel 00:12:48], he wrote a perspective for the book that’s upcoming, and one of the things he said is, “AI is a bullshit generator.” And I absolutely love that. So, it’s exactly what it is. You can tell it to do anything. It will connect the dots, it will argue pro, it will argue against, it will create evidence supporting and evidence refuting, so it’s really up to us to kind of drive this.

But I do want to defend my AI-first thinking because if we understand the capabilities, then it truly is an almost limitless field of possibility because these things are no longer programmed, as Simon said, also. It’s not about a deterministic flow. It’s not a program. And these things are taught, and it’s a fundamentally different approach to how we build user interfaces, that they’re no longer completely deterministic. They’re also extremely personalized, and I do mean absolutely, to the point where it’s ridiculous.

You can personalize the entire e-commerce experience, completely give you everything that is completely relevant. You can do—in the recent article I wrote, I said, “Look, you know, let’s say you’re into fishing. You can go and analyze your previous purchases, figure out you’re into fishing, figure out what you’re missing.” In fact, I went ahead and did that as a quick little experiment, and it absolutely could figure out that I got two reels and only one rod in the last year. And so, you absolutely need a light rod. It recommended correctly the light rod that I was actually looking at buying, and then it then fed into Midjourney. It then created a background showing me actually fishing with all this equipment that is going to make me buy, or offer to buy.

Brian: Actually with a fish, for a change? It actually showed you catching a fish.

Greg: Exactly.

Brian: Instead of not catching a fish [laugh]?

Greg: As opposed to just fishing, exactly. Exactly, yeah because it’s not always catching, right? It’s not called catching. It’s called fishing. Yeah, so, but I guess what I’m saying is our normal sort of e-commerce approach is really out the window because you can so hyper-personalize it, including images that show you actually using the items. It is to the point where we did not have this capability before. So, if we think AI first and really think about what are we trying to achieve from kind of the UX perspective, we can push it to absolutely the next level.

Simon: I’m a little bit more, maybe, skeptical of, kind of, this AI future and this AI-first world. And some of the reasons why I feel that way is that even for that e-commerce experience, example, Greg, like, to put an LLM into a product means that there’s a non-zero chance it’s going to go just totally crazy, and your user is going to have an experience that they’re going to no longer be your customer. And that is a giant reputational risk. And there’s also just the costs associated with running these models. You know, maybe in the future, we’ll get this and that and the cost is going to go down to zero, but as a researcher, I like to just live in the, like, the possible, right, the real.

And so, it kind of feels like, right now, we’re thinking about—you know, we’re talking about, like, mobile first, and if you had only experienced a desktop, there’s no way in the world. You could imagine Pokémon Go. You couldn’t imagine a world where there’s a bunch of people that get together are on their phones, but it’s not about the phone. It’s about the community, and that world just couldn’t imagine exists. So, like, I think we really need to take a—this is a bit of shop talk—but more of a service design lens when it comes to this, and be, like, what is the thing somebody wants to do, not on my website, but in their lives, and what brings them to my property, whatever it is, and how can I, like, imagine a different world that leverages these capabilities, as we’ve talked about, to just help them do their job, where I understand that what I’m competing against is them just doing it the way that they used to do it because it probably worked well enough.

Brian: The status quo is always the enemy. Or it’s your friend, especially if we’re making it [laugh] worse. But the status quo is tough to change.

Brian: I mean, you guys are probably doing—you probably have more design resources on the AI space than most companies do. And in fact, it’s quite rare in my experience, to find a designer working with a data science team, let alone being assigned to a group that’s working on solutions that are wrapped up in machine learning and AI. So, what’s your take on that?

Paz: Yeah, I mean, I love how Simon has put it because it’s changing the framework that we are using currently, and that’s where the magic is going to happen. I think besides just thinking of the LLM as an output making, we also have to think AI is going to be making decisions for us. So, how we as designer can design for the shared agency, where both us as humans and AI are coexisting, and using any kind of interface, any kind of product? We have never seen that before, so I don’t have all the answers to how, what the kind of, like, shape that that’s going to have, but I think Simon was on the right path when he was talking about, like, you cannot imagine until you’re, like, working on it. And that’s what we’re trying to do. Yeah, so there’s, like, multiple resources here, as you can imagine. There’s, like, whole teams working on trying to figure it out, what is the future of our practice?

Brian: Yeah, yeah.

Greg: Yeah, absolutely, I agree. I think one of the key value-ads that we provide as designers is this ability to do vision prototyping, which is kind of, you know, as designers, we kind of got very complacent in the last, I don’t know, five, ten years. It’s always been like, okay, it’s this pattern or that pattern. If it’s a web to a mobile, we’re just going to do a column drop, and this is the typography. It’s always Helvetica, and, you know, on and on and on, right? It’s all very regimented. It became very scripted, if you will.

So, this is a new opportunity for us to shine, to really design. Exactly as Simon said, if you’ve only seen the desktop, how can you imagine what a Pokémon Go would look like, what that experience would actually look like? Well, we can. As designers, you know, we kind of got to rediscover, I think, our DNA of going back and really designing, which means imagining something that is technologically available but hasn’t really been seen yet. And that’s where I think a lot of the value that we bring to even serious enterprise companies can come from, particularly now that, as Paz mentioned, these are agentic flows.

This is augmented intelligence. This is, like, Iron Man, essentially. You know, everyone can be an Iron Man, and have their own Jarvis, and do superhuman feats. And what does that really look like? Well, we don’t know yet, but I bet we can help imagine that. Who’s better?

Paz: Yeah, and I would like to just make a little point there. The UX practice also has, like, researchers and UX writers. There’s, like, a few different, like, players that we all work together to make, like, better products. Right now those are shining in this space because we are seeing, within the model training, content designers, UX writers, they are fantastic at writing instructions and system prompts. And things that we’re seeing are more effective than just plain training, which it is also important by itself.

And same with researchers. We’re seeing for, like, creating evaluations, it’s just so much more they’re bringing to the table. I think the visual aspect, we still need to figure it out because we don’t know where the products are going, but it’s all of us, the ones that we are finding new ways together.

Simon: As a researcher, I just like to say that I agree we’re extremely important, and everybody should hire—

All: [laugh].

Paz: Yes you are.

Simon: At least ten UX researchers.

Paz: You are.

Simon: But like, one of the things that I’m. Maybe not struggling with, but I find extremely challenging, and I love this kind of challenge, is this idea of, with these fuzzy experiences—and I’m curious to see what everybody here take is going to be on this—but with these fuzzy experiences where you have natural language input, which means that the same person, there’s basically unlimited affordances. So like, the space where you can say what you want is unlimited—it’s impossible to know what somebody will say—knowing that, and knowing that different people express themselves differently, and have different expectations for a product, and that you do not know how the model is going to process the person’s input, so in effect, if the model’s output is the user experience, then you do not control what the user’s experience will be.

And so, with that reality, when it scales, let’s say you have 50,000 user interactions for your product in a month. You’re stuck with 50,000 interactions that, like, the signals, like, a short interaction, either means somebody really killed it and got exactly what they wanted first try, or they couldn’t figure out how to use it, and got garbage output, and will never use our product ever again. And so, like, have you seen approaches to make sense of 50,000 different ways that people express themselves, where only they know whether it was good or not?

Greg: Sure, yeah. I think the what you already mentioned Simon, which is research. I would call myself a researcher. I’m kind of a jack of all trades. I’m a generalist UX person who likes to use the research to then create a better UX. So, it’s a rapid, iterative testing and evaluation, so I think you almost need to do this.

And it’s a different approach. I think another way to think about AI first, it’s fundamentally different. When we go GA with a product, that means we’ve done all that research, we got everything perfect, it’s just, it’s all great, right? That’s what GA used to mean. GA today means we can now orchestrate the very first step in the interaction. So, GA literally means it’s a starting gun.

Simon: What’s GA, Greg?

Greg: GA: general availability. Like, when you actually release a product, right, when you launch a product. That means we’ve taken the first step to give you the tool, so now you can start playing with it, and we can start learning from what you’re doing. And part of the learning is the qualitative, and part of the is quants. So quants, it’s very difficult. You’re absolutely right. We don’t know unless we give them, you know, the usual thumbs up, thumbs down, which most people not going to click anyway, unless it’s really bad or really, really good.

But I think massive research effort is needed, and I mean qualitative, sitting down with the customer, watching them use the product, to really understand what is happening, and then to leverage that for improvement. But also really collecting the data. What are they typing in? What are they getting back? Is somebody who’s typing in this type of question is always having a short interaction?

And then, let’s look into that. Is somebody who’s typing in this other type of question, are they always getting a long interaction, or majority of them long? Let’s figure that out, right? Like things that are outliers on the spectrum, let’s really dig into it with rapid, iterative testing and evaluation, so that we can update our model and then move forward. Again, launch a product these days, literally means the starting guns have been fired. Put the research to work, right, to figure out the next step. That’s my two cents.

Paz: And I agree with that. I think it’s going to be impossible to really understand this, like, 50,000, 100,000, a million interaction from our users, but we can have a level of control. As UX designer, we need to get involved in the training of the model, and that’s something that, as a community, not everybody fully understands, yet. This is raw materials, these models that can be shaped, and to do so, you had to work with engineering. It’s this idea of understanding the context that you’re giving to the model, meaning the data, the instructions, the prompts that the user is now seeing, but we are crafting for them, and having really good evaluations, to Greg’s point, and really understanding whenever I put, like, a new system prompt here, if I took my instructions, what kind of result am I getting? Is this getting better?

And it’s going to be a neverending cycle. I don’t think it is a work that is never going to be done, but it will be more control. Hopefully we wouldn’t have, like, this crazy evaluation—sorry, not evaluation. Hallucinations. That’s what I was trying to say. These crazy hallucinations that really make for horrible user experiences where the user is not going to go back.

And also it will change the expectation of the user because he will understand it’s a model, and maybe if they ask a question in a different way, they will get a different result. I think, like, our expectation when using a product with AI is also going to change, and it’s going to become a little more fluid.

Greg: Yeah, you know what occurred to me? I just got an idea based on what you were saying. Could we not gamify hallucinations, and kind of, you know, almost like, elevate that? Hey, find us. Find us when it, hallucinate it and send it to us. The best hallucination gets swag.

All: [laugh].

Greg: And then you can even celebrate it, right? You can, like, create this, like, hey, I found the hallucination, Grammarly hallucination, just to use Simon’s favorite app. Like, you can have a shirt. I found the—you know what I mean? Like, one of those security shirts that we have, like, the white hats or the gray hat shirts. Am I, like, a total security nerd? Am I, like, the only one?

Brian: [laugh]. You are a security nerd, yes.

Greg: Oh, my gosh, yes. So, you know, I’m a white hat, I’m a gray hat, that kind of deal. So, I’m a hallucination spotter. Terrible name. We’ll have to work on that.

Brian: My feeling, Simon, first of all, on the quant side, you probably could do a lot of research on the transcript data. If you’re really just providing a large data set and putting an LLM interface on it, there’s a lot you can probably study by looking at the past transcripts. And if anything to me, that’s a place where I think UX Research can provide a lot of value by classifying what are people actually trying to do with this product, so that we can now double down on that. But also we may antagonize prompts or queries that are sent in the system that we know the system is not designed well to do, and it probably won’t ever do it better because we will never train the model on that information. So, let’s say, you know, like, I don’t know this is, like, I see this has information on the CRM, but it doesn’t have information on inventory, and when people are trying to line those up—do they need to buy more? Can I sell to this customer or not?—the model understands where the limitations of it are, and it’s surfacing those limitations to the customer.

Like, to me, because the problem space is so big, in a way, the success is going to come by narrowing the playing field down and hitting a home run on a smaller set of problems. The wider the problems that you’re trying to address, it’s like, exponentially, it gets harder to do a good job. There are techniques. I’m not a technical person, but Retrieval Augmented Generation or RAG, knowledge graphs are another way to improve confabulations or hallucinations with information, particularly if you’re, like, trying to retrieve fact-based information.

How many customers do we have right now in Arizona? Like, that really should be a factual information that can be properly queried in some system without having to write SQL and come back. There’s some things like that where there is an objective truth, but understanding, how does a customer validate—or a user, maybe, an internal user—validate that that information is correct, let alone—then there’s the bigger question: how do we know which class of problems this tool should be really good at delivering? And if we’re just never looking at any of the transcripts, we’re never doing qualitative research, then we’re just back to the, like, here’s your ChatGPT. Do whatever you want. The world is your oyster.

That has ramifications in certain contexts. That could cause real problems if people are actually acting on the information and it’s wrong. Forget about bias and all the bigger societal issues that go into bias and all of that, but just factually operating on wrong information. And this is something that the average data-first team that’s approaching AI from a data lens, this is the work they are not doing because they don’t have time to do this. It’s not that I don’t think they care because no one wants to put out shit nobody uses. I think most people are tired of that.

Even engineers don’t want to put out stuff no one uses because life’s too short, and isn’t it more fun to work on stuff people care about? I think that’s, like, generally more true now than it used to be [laugh]. And so, where the help is needed, I think, is in defining the problem space more concretely and making it absolutely clear, like, here’s where we could provide a great benefit, if our LLM, or if our system, or whatever it is that we’re making, could do X, Y, and Z. This is the class of problems that customers at Thomson Reuters actually trying to do. Even if we didn’t design it for that, we have these hints now, the queries, the prompts, just like search engine logs, we can see what the intent is. There is signal here that we are not responding to that, and we need to double down on this thing and stop working on that thing. You know, I don’t know. I’m throwing that out to you all to comment on.

Simon: Yeah, I think, I think you’re right, and I feel threatened. Are you coming for my job, Brian? [laugh].

Brian: No [laugh].

Greg: [laugh]. That’s pretty good [laugh].

Brian: I’m no longer employable, actually. So…

Simon: Those are like great insights. But something that came to mind while you’re talking, and also touches on what Paz was saying about the value of content designers is, before this whole LLM moment, I used to build skills for Alexa, and so I have a conversational design background. And it really feels like when LLMs came out, we were like, maybe the design, or just kind of products leaders in general, were like, “This is magic. We don’t need to design proper conversational experiences anymore. The computer will do it for us.” And the technologists were like, this is technology. Wonderful, great.

And so, it’s almost like we forgot the decades of experience in phone system design. And we know how to design good conversational experiences. I suspect that maybe it’s because conversational design was a very women-led field, and tech is men. Like, I suspect there’s some gender dynamics there, but just the idea being that we don’t need to just have a massive prompt that just kind of does whatever, and we cross our fingers. We know how to design good experiences where there is a back and forth, and it’s about marrying those two, and I think that’s where the future of experiences really lies.

Greg: Yeah, I think you touched on the very important point. I think that having diversity on your design team, including gender diversity and experience diversity, is critical, particularly having women be part of this experience. I know this is all kind of tech bro-driven in many cases, and we’ve already seen some terrible outcomes of this, with multiple examples where LLM is crafting horrendous, horrendous emails, and introductions, and so on. I’m going to not call out anything specific to protect the guilty here, but you just need to check my LinkedIn feed and you’ll see tons of examples.

This is exactly why UXers need to get involved. This is why diversity in UX and on the team, your tech team that deals with AI, is so freaking valuable. It’s like absolutely number one advice, get some researchers. Number two advice, make sure this is diverse. Your team is diverse.

Brian: For my audience listening to Experiencing Data, why do I need to go get some researchers? Like, explain to me in a few sentences with no, like, UX jargon? How is that going to benefit my data team, for example, that’s responsible for showing some value with AI in my organization, or my product team that’s led by an engineering founder and, you know, technical founders, we kind of do it ourselves. Like, I can just, I can have AI do my design work for me. Why do I need a researcher? What am I going to get for that? Like, sell me on that a little bit. And it’s not because I’m skeptical. I just want people to understand, how would that benefit me in an AI context? Anyone, anyone.

Simon: Ultimately, my job is to advocate for the customer, the user. And so, when there’s discussion, when there’s a product leader that or a product person that needs to make a decision, they have inputs from the sales, from tech, from all of these different inputs, and my job is to make sure that the user’s perspective is compelling, and that has weight when it comes to making whatever product decision. And so, it’s not so much that my job is, like, interesting—and I will find interesting things, and it’s going to be neat, and everybody’s having a good time, but the real value is to have a strong voice of the customer at the table for you make decisions that are not informed by what you think the user wants, but what the user wants.

Brian: Paz, you wanted to say something.

Paz: I guess, we had, like, switch a little bit the conversation, but to what you guys were talking about, it’s extremely important to have, like, UX talks, researchers, content designers, designer with the data teams because the data itself, yes, has bias, but also can be aligned with the intent of the user. I think it’s so important to understand, like, the user is trying to do this, it has this context, it has this intention. The data team is focusing on the technical aspect, which is fantastic, but we need to work hand-in-hand. Like, we need to help them understand the type of data, or the types of, like, golden prompts that were feeding the model in order to, like, train it. Those things are better when they’re written and thought by people that are really understanding where the user is coming from, and that is our domain.

So, if we work together—and we’re seeing that already. We’re seeing that engineering teams that are open to working with particular content writers that are writing really good preambles, and really good instructions. Those are getting so much better results than the ones that are working in a vacuum. So, the value they’re going to see when they start working with people that are going to bring these nuances to the data.

Greg: Yeah, I think all you have to do is just Google [BLEEP], and Google some of the stuff that [BLEEP] has been sending to folks, especially a Milly Barker post on LinkedIn that’s got probably a million impressions. It’s pretty striking what type of crazy hallucination style, really terrible type of interaction that AI can really do. I mean, everything that’s misogynistic, that is, you know, basically the worst of the worst of the worst, if you can imagine that, distilled into an email, and then sent to absolutely the wrong person, who is absolutely the opposite of what she wanted to hear that is put in the email, and that’s what she got from this company.

So, if you want to avoid that type of experiences in your own practice, that is why you should hire women, and minorities, and researchers on your team. This is exactly the type of stuff that is dystopian, and it’s not going to work for you, and it’s going to damage your brand irreparably unless you do that.

Brian: Yeah, it’ll keep you out of the AI—if you guys know the AI Incident Database [laugh]—if you want to stay out of there [laugh], it’ll help you stay out of there. I guess the way I see it, I think bringing those stories in the context to the problem space is really helpful because if you’re tasked all day with, like, just getting things into production, and monitoring model drift, and making sure models are accurate, you can build what I call technically right, effectively wrong solutions. So, they pass muster on accuracy testing against some pre-canned test that we decided upon, but it doesn’t necessarily mean that we improved someone’s life in a way that makes them want to pay us or keep paying us there. And to me, how do we connect the dots between that end customer and the technical accuracy of the system? To me, that’s the bridge that I think user experience design and research together can kind of help close that. I don’t know, what do you guys think?

Simon: I have hopefully a perfect example to capture what you just said there, Brian. So, the product I work on has an intent resolver, where it selects which, let’s call it a function that the user is presented with. And this problem of intent resolution, it’s a very hard problem. Like, how do you know what the person wants, and how do you surface that? That’s why Alexa just kind of never really quite gets that skill that you want.

And I was talking with the ML team, and they found that it was an interesting problem and challenging, and they were working on it, but I knew from a user perspective, that when your request is not met with the right function, it just kills the experience, and you probably just won’t use the product again because you’re like, “Well, this is a garbage. It just didn’t do the thing I wanted it to.” And so, when the ML team understood the user impact of the ML problem that they were working on, they were like, “Whoa. This is actually not some, like, theoretical thing. It’s like, this really has an impact.” It kind of energized them, and they saw, like, okay, cool, I’m motivated to work on this. And, yeah, it just opened their eyes to something new, and led to a better customer experience, ultimately.

Greg: Yeah, there’s an article me and Daria wrote on value matrix, and that is exactly the type of exercise that helps you do this type of valuation. The idea is, you would sit down with your data scientists and start the conversation: what is the outcome of getting this wrong? What is the cost of false positive or false negative? What is the benefit of true positive, true negative? And then if you assign an actual dollar value to these, then you can build this, [called 00:40:20] value matrix.

So, it’s beyond things like accuracy or recall. Because an accurate AI can also be a terrible choice for your product. And why? We can’t go into it now because we don’t have time, but you should probably read the value matrix article on uxforai.com. And by the way, I have a book coming in April, so check it out.

Brian: [laugh]. That’ll be great. Yeah, we do have to wrap here. This probably is a nine-hour episode that we need to do to get through everything, but this is a great just start to have this conversation and panel with you all. Can you just quickly tell me—I know Greg, we can find you at uxforai.com. Paz, where’s the best way to get in touch with you? And I know you write and stuff like that. Where can people find you?

Paz: So, either they can read my articles on Medium, where I talk about how UX is changing, or LinkedIn, they can connect with me. On LinkedIn, it is pazperez.

Brian: Cool. And Simon?

Simon: I’m on LinkedIn. You can search ‘Simon Landry, PhD.’ Lots of shit posts, but also lots of real good content on there.

Brian: [laugh].

Greg: [laugh].

Paz: Awesome.

Brian: Well, thank you so much for coming on, Experiencing Data. It’s been great. I look forward to getting this out to everybody. And listeners, please chime in. Hit me up on LinkedIn, or shoot me an email. If you liked this episode, we can do some more content on this topic. So, it’s a big space, and I’m glad that we’re starting to have these conversations. So, thank you everyone.