In this episode, I sat down with tech humanist Kate O’Neill to explore how organizations can balance human-centered design in a time when everyone is racing to find ways to leverage AI in their businesses. Kate introduced her “Now–Next Continuum,” a framework that distinguishes digital transformation (catching up) from true innovation (looking ahead). We dug into real-world challenges and tensions of moving fast vs. creating impact with AI, how ethics fits into decision making, and the role of data in making informed decisions.

Kate stressed the importance of organizations having clear purpose statements and values from the outset, proxy metrics she uses to gauge human-friendliness, and applying a “harms of action vs. harms of inaction” lens for ethical decisions. Her key point: human-centered approaches to AI and technology creation aren’t slow; they create intentional structures that speed up smart choices while avoiding costly missteps.

Highlights/ Skip to:

• How Kate approaches discussions with executives about moving fast, but also moving in a human-centered way when building out AI solutions (1:03)
• Exploring the lack of technical backgrounds among many CEOs and how this shapes the way organizations make big decisions around technical solutions (3:58)
• FOMO and the “Solution in Search of a Problem” problem in Data (5:18)
• Why ongoing ethnographic research and direct exposure to users are essential for true innovation (11:21)
• Balancing organizational purpose and human-centered tech decisions, and why a defined purpose must precede these decisions (18:09)
• How organizations can define, measure, operationalize, and act on ethical considerations in AI and data products (35:57)
• Risk management vs. strategic optimism: balancing risk reduction with embracing the art of the possible when building AI solutions (43:54)

Quotes from Today’s Episode

"I think the ethics and the governance and all those kinds of discussions [about the implications of digital transformation] are all very big word - kind of jargon-y kinds of discussions - that are easy to think aren't important, but what they all tend to come down to is that alignment between what the business is trying to do and what the person on the other side of the business is trying to do."
–Kate O’Neill

" I've often heard the term digital transformation used almost interchangeably with the term innovation. And I think that that's a grave disservice that we do to those two concepts because they're very different. Digital transformation, to me, seems as if it sits much more comfortably on the earlier side of the Now-Next Continuum. So, it's about moving the past to the present… Innovation is about standing in the present and looking to the future and thinking about the art of the possible, like you said. What could we do? What could we extract from this unstructured data (this mess of stuff that’s something new and different) that could actually move us into green space, into territory that no one’s doing yet? And those are two very different sets of questions. And in most organizations, they need to be happening simultaneously."
–Kate O’Neill

"The reason I chose human-friendly [as a term] over human-centered partly because I wanted to be very honest about the goal and not fall back into, you know, jargony kinds of language that, you know, you and I and the folks listening probably all understand in a certain way, but the CEOs and the folks that I'm necessarily trying to get reading this book and make their decisions in a different way based on it."
–Kate O’Neill

“We love coming up with new names for different things. Like whether something is “cloud,” or whether it’s like, you know, “SaaS,” or all these different terms that we’ve come up with over the years… After spending so long working in tech, it is kind of fun to laugh at it. But it’s nice that there’s a real earnestness [to it]. That’s sort of evergreen [laugh]. People are always trying to genuinely solve human problems, which is what I try to tap into these days, with the work that I do, is really trying to help businesses—business leaders, mostly, but a lot of those are non-tech leaders, and I think that’s where this really sticks is that you get a lot of people who have ascended into CEO or other C-suite roles who don’t come from a technology background.”

–Kate O’Neill

"My feeling is that if you're not regularly doing ethnographic research and having a lot of exposure time directly to customers, you’re doomed. The people—the makers—have to be exposed to the users and stakeholders.  There has to be ongoing work in this space; it can't just be about defining project requirements and then disappearing. However, I don't see a lot of data teams and AI teams that have non-technical research going on where they're regularly spending time with end users or customers such that they could even imagine what the art of the possible could be.”

–Brian T. O’Neill

Links

• KO Insights: https://www.koinsights.com/
• LinkedIn for Kate O’Neill: https://www.linkedin.com/in/kateoneill/
• Kate O’Neill Book: What Matters Next: A Leader's Guide to Making Human-Friendly Tech Decisions in a World That's Moving Too Fast

Transcript

Brian: Welcome back to Experiencing Data. This is Brian T. O’Neill, and today I have maybe an 11th cousin—something like that—Kate O’Neill, on the line. Welcome. Kate. How are you [laugh]?

Kate: I’m good cuz [laugh]. How are you?

Brian: Yes, it’s proper spelling, the two L’s, all of the works. You got the works going on. I love it.

Kate: The way it ought to be spelled.

Brian: [laugh]. Exactly. It rarely is, unless you are into wetsuits, and then maybe you know how to spell it [laugh].

Kate: Right.

Brian: So—

Kate: No, good to be here. Thanks for having me.

Brian: Yes, I’m really looking forward to talking to you about it. I think of you, this tech humanist, this person that’s drawing on human-centered design to make sure that technology is serving the people properly. And you do a lot of speaking on this, you’ve written books on this. What did I get wrong right there? Do you want to correct anything so—

Kate: [laugh].

Brian: Just so guests know, our listeners know who I’m speaking to today.

Kate: No, that’s right. I have a background that comes up through technology. You know, I worked in various fields at Toshiba, at Netflix, at Magazines.com, HCA Hospital Corporation of America. I’ve done a lot of different things inside of IT organizations and over different kinds of projects. And so, I’ve been very much a generalist.

And then before I started KO Insights, which is now 11 years ago, I had an agency called MetaMarketer, and we are an analytics and strategy agency that worked directly with big companies to help them think about their—how to model their business in data basically, so that we could think about, how are you going to validate the decisions you’re making? How are you going to test some of your hypotheses and so on? So, it's a very new kind of thinking for people at that time. And this is 2009 that we founded. So, it was very much, as far as I can tell, as far as I knew at the time, it was the very first strategic analytics agency.

And so, we got to work with some pretty heavy hitters, Adobe, Symantec, and folks like that. And so, that ought to be kind of familiar to the folks who are listening, that sort of thing, I imagine. Unless folks are listening who are very, very much younger than me and they’re going like, “What the heck? You were doing this already, like, 30 years ago?”

Brian: [laugh].

Kate: Yes, yes, yes. I look much younger than I am [laugh].

Brian: No I think they will—

Kate: And I sound much younger.

Brian: No, that this will all definitely resonate. We have a lot of pretty seasoned people who have been in the field for a long time. They’ve gone through the Big Data Era, and then there’s machine learning, and then they rebranded it AI, and now—

Kate: Right [laugh].

Brian: —blah, blah, blah, and it’s like kind of the same movie [laugh] just slightly—

Kate: Yeah. Yeah. We love coming up with new names for different things. You know, like, whether something is cloud, or whether it’s like, you know, SaaS, or, you know, we have all these different terms that we’ve come up with over the years. And it is fun.

You know, when you spend this long working in tech, it is kind of fun to laugh at it. But it’s nice that there’s a real earnestness, I think, that’s sort of evergreen [laugh]. People are always trying to genuinely solve human problems, which is what I try to tap into these days, with the work that I do, is really trying to help businesses—business leaders, mostly, but a lot of those are non-tech leaders, and I think that’s where this really sticks is that you get a lot of people who have ascended into CEO or other C-suite roles who don’t come from a technology background.

In fact, I think you know intuitively, you all listening, will know how rare it is outside of, you know, very tech-heavy fields, to have the CEO be somebody who comes out of a tech background. So, you know, these kinds of sweeping decisions that are often tech-powered are being made in the C-suite and in the boardrooms. And you know, folks don’t necessarily have a really good handle on scale and scope and capacity that comes from AI and other emerging tech. So, it’s a really important conversation, and I’m really gratified to be doing it.

Brian: Yeah. I just had a conversation with a friend of mine that runs a large data science and engineering consulting firm, and frankly, a lot of the work that’s happening, it’s not just come in and build this thing that we need. It’s, we don’t even know what we should be doing with all this data, but we think that we have a lot of it because we’ve been saving it, and maybe we’re a legacy—an older company that has been storing all of this stuff for a long time, and now I’m a leader, and I’m supposed to be doing something with all this. So, it’s like a solution in search of a problem. There’s a lot of FOMO in this space.

The data people tend to understand this better, but they need to go extract what business problem needs to be solved with all this data that’s there. And it’s kind of this tennis match of, like, “Well, what could we do?” “Well, what are your objectives?” “Well, what’s the art of the possible?” And I just see this tennis match going back and forth.

And when you add, like, human-centeredness and ethics onto it, I feel like that sounds to a non-technologist who hasn’t built a bunch of stuff that didn’t work and has gone through the process of realizing, like, adoption matters, that sounds, like, hand-wavy slow stuff that, oh yeah, and it should be ethical too, and it should be human-centered, and all that. UX stuff is important. But tell me about, like, when are we going to get some value? Or, when can I replace my customer service team? How do you position this idea, like, putting aside making sure we have a clear business problem to solve with machine learning or analytics or AI or whatever the proper implementation method is in that space, how do we make space for the story that adoption matters and human experience matters because it actually does have to do with the financial ROI eventually.

How do you approach that topic without it sounding, like, yeah, we’ll get to that at some point, which really means we won’t because we’re racing to get something out the door that uses ChatGPT or some large language model solution. How do you think about, like, framing that so it doesn’t sound like just a slow down, a slow, like, hard to measure, how will we know it really—like, what did I get for all of that work and go talk to customers and research? It just sounds very hand-wavy, I think to some.

Kate: [laugh]. Yeah, no, it’s a really good question. It’s a fair question, and it’s one that shows you know, your familiarity, obviously, with the space and with the types of projects we’re talking about. Because that’s true. There is kind of this disconnect on all sides of the equation you just described.

And I love the metaphor the tennis match because that actually does feel, in many cases, you know what it’s like, that you’re coming from in some ways below the data, where you’re saying, like, “Oh, we could build it up this way,” and in some ways you’re coming from above the data and going, like, “Well, I don’t know. We could pull this out of it.” And one of the ways that I think that makes sense and helps to frame the discussion so that it’s a little more graspable and has less of the hand-waviness about it is that I—in What Matters Next, I came up with this continuum of the Now-Next Continuum, and so trying to think about these kinds of things on a timeline of, you know, the past, the present, and the future. But also, one of the things I think we tend to overlook, in the years and years that I’ve been working in this space, I’ve often heard the term digital transformation used almost interchangeably with the term innovation. And I think that’s a grave disservice that we do to those two concepts because they’re very different.

So, digital transformation, to me, seems as if it sits much more comfortably on the earlier side of the Now-Next Continuum. So, it’s about moving the past to the present. It’s about saying, “Hey look, we have things that are, you know, technical debt.” We’ve got, you know, projects we need to catch up on. We need to transform the organization to be more current with things that have already happened. You know, the externalities in the marketplace, the customer expectations, you know, what our competitors are already doing. That’s transformation.

And innovation is about standing in the present and looking to the future and thinking about, you know, the art of the possible, like you said. Like, what could we do? What could we, you know, pull out of this unstructured data, this mess of stuff that’s something new and different, that could actually move us into green space, into territory that no one’s doing yet? And those are two very different sets of questions. And in most organizations, they need to be happening simultaneously.

They need to have some crossover. Somebody needs to have some visibility into what’s going on—usually that’s the CTO or CIO or somebody like that in an organization—but they also need to both have the same sort of—at some level—a sense of governance and ethics that’s part of the ethos of how that’s done. So, we’re always remembering that there are intersecting goals here. We need the business to be more effective, we need to be able to, you know, kind of speed up, catch up, do whatever it is we’re trying to do, and we need to find the ways in which this intersects with what people are actually coming to us to do at some point in the process, right? Like there’s some customer somewhere, some customer, user, guest, patient, student, whatever the role that the human is in at the time when they interact with us in a transactional way, like that person is trying to accomplish something. There’s a job to be done.

And so, our job to be done in making the business more efficient or more profitable or whatever needs to align with that person. And that’s really what it comes down to. And I think the ethics and the governance and all those kinds of discussions are all very big word kind of jargon-y kinds of discussions that are easy to think aren’t important, but what they all tend to come down to is that alignment between what the business is trying to do and what the person on the other side of the business is trying to do.

Brian: I’m going to ask you about ethics in a little bit, but let’s step back to this idea of innovation. I liked your framing about the transformation work being closer to now than, kind of, next. It’s not a challenge my thinking about how to come up with what’s next, what would be beneficial to the humans in the loop, whether this is internal stakeholders that we’re building solutions for or it’s downstream paying customers or beneficiaries that are even downstream from that, right, someone that doesn’t even know that they’re inside of a system that’s being affected by software or data or AI? Doing that, in order to get to innovation, my feeling is, if you’re not regularly doing ethnographic research and having a lot of exposure time directly to customers, the people, the makers, have to be exposed to the users or the stakeholders. There has to be ongoing work in this space and it can’t just be what are the project requirements, and then we disappear.

But I don’t see a lot of data teams and AI teams that have non-technical research going on where they’re regularly spending time with end-users or customers, such that they could even imagine what the art of the possible-what could be better—if you don’t know what’s wrong with now, like, besides genius design and randomly guessing and getting lucky on something, how will you ever know what the next best thing could be if you don’t understand what’s wrong with now? Is there some other way that I’m missing? Is that the wrong framing? Or what do you think? Like, how do you close that gap if you don’t know what’s going on? Is customer research and exposure to the people from the makers, is this essential, or is there another way to do this work?

Kate: I mean, I think that’s a pretty substantially well-understood phenomenon, right? Like I think that people in any kind of product development role, whether it’s more internal or external, all understand that at some point you need feedback mechanisms, you need a loop of validation. You need some sort of iteration. You need some sort of process whereby, you know, you’re kind of making sure that your idealized workflow actually works, you know, once it meets reality. That all has to happen, and you know, I think we’ve always had the dilemma of, there’s a—[laugh] there’s always the dream of the technologist is, you know, to have these kind of perfect systems, that everything is neat and orderly, all data is structured, it’s so beautiful, like, you know, it all looks great, it all flows well. And then, you know, the reality is that the technology solutions or the data fields or metadata specifications or whatever never fully encompass all of the edge cases. And that’s where you got to have things coming back on a regular basis with feedback.

And that’s one of the things I think we are seeing in the space of you know, conversational AI deployments, for example, you know, where you have a lot of organizations doing customer support replacement type of functions with conversational AI, and you have a lot of, you know, the sort of basic cases being fielded by your basic chatbots. And, you know, those are kinds of things like, “Oh, I lost my password,” or, “I need new access to my account,” I’ve got to do these really, you know, obvious and 80% cases of what gets handled by customer support teams. And so, to replace those tasks or those questions with conversational AI is an absolute no-brainer. Like, not even the human people who are in those roles necessarily resent that’s being done by chatbots at this point. But what happens is, of course, you get those edge cases, and more edge cases start to show up, and you start to be able to capture those, and be able to recognize, oh okay, there is a pattern here, you know?

There’s this edge case, and whenever that edge case comes up, we can capture it this way, and we can document around that. And what it tends to fall to is that the human people who are operators in those roles start to evolve into becoming experts in being able to handle edge cases and document edge cases, and being able to feed that, sort of, proactively feed ethnographic research, in a sense, back to product teams so that they can better tweak what’s being done with the AI, so that, you know, more and more of those edge cases are being handled as part of the basic cases, and that we’re getting more sophisticated, and we’re being able to build brand and culture and voice into what’s happening, and we’re able to, you know, make that all feel like a harmonious part of the whole.

Brian: Yeah, prior to getting feedback because getting feedback presumes that something has already gone into production and now we’re responding to things that are coming in. So like, let’s step back to the earlier pre-work, the earlier product work. How do you, in a human-centered way, decide that, hey, if we’re going to—we want, like—and I think it’s fine. Businesses are experimenting with GenAI technology, particularly these language models. I think that’s okay to place bets here, and some of this is a little bit tech first, let’s find a problem that it goes with, and we need good translators to help close that gap.

How do you, in a human-centered way, decide that customer service is a domain that we should explore using this technology, and B, when we talk about human-centered, there’s the agent, the worker, the employee of our own company, there’s a concept of human-centered for them, which is, what work should we be replacing or making better? And then there’s the paying customer that needs a password change, or they need some data about their situation, or what’s my balance on my checking account, or whatever the questions they’re asking. How do we do that work in a human-centered way, before this chatbot gets out into production, such that we know ahead of time what we’re replacing or what we’re augmenting with technology, such that we don’t have a revolt in the customer service team or an FU or attrition, or whatever the down—the negative externalities. Talk to me about that.

Kate: Yeah. So, some of that is about that tennis match, right? Some of it is about, you know, first of all, there should be some sense of leadership present in every part of the decision, from the very earliest parts of the decision. You should be talking to the people who are going to be part of the group that’s affected by the experimentation that’s happening and it should be clear that this is not about, you know—hopefully it’s not about pure cost-cutting measures, and we’re not looking to, you know, cut 15,000 jobs, and you know, that’s not what’s happening here. What we’re looking to do is figure out what is the right way to take care of our customers and make sure that they get fast, reliable answers that, you know, they come to depend on, and that we’re also, you know, making sure we’re providing emotional support and the kinds of the nuance that people need for the harder, complicated cases.

And that’s what we’re looking to do, is evolve it, make it more sophisticated, make your job more interesting, you know? But all of it is in service of what the company exists to do and is trying to do at scale. And so, I think that piece can’t be skipped. You know the piece where you actually have to have a very clear and succinct understanding of what the company’s strategic organizational purpose is. And that is an exercise that it sounds very fluffy, but it’s really, really important to the steps that follow, being able to articulate that strategic organizational purpose in, let’s say three to five words, so that it’s really crisp and really believable and graspable by everybody in the organization, who can then feel like, oh, I see how this then plays out.

Like, if you map that out—I have a digital transformation canvas that I use for this—and you know, once you’ve got that purpose statement, you’re also looking at things like your articulation of values, your articulation of the annual goals, the quarterly goals, everything that kind of ladders down, in a sense, from that. And then the brand experience and culture articulations, and how you kind of think about those aspirationally, how you think about the processes that go into those and how do those marry up to the purpose statement and everything that you’re—it’s almost like balancing a checkbook, in a way. You’re trying to make all these things kind of reconcile together. And only then, after you’ve articulated all of that stuff and kind of thought through that organizational design element of this, should you be able to then say, all right, and then how are we going to use data to model all of this and be very effective at how we’re bringing back agility into the decision-making we’re doing across the company? How are we using technology to then amplify and accelerate the alignment we now have with what we’re trying to accomplish, with people outside and inside the organization, with the culture we’re trying to create, et cetera?

All of that is important to have done at least once. I encourage my clients to do that, you know, quarterly, semi-annually, whatever is the right rhythm and cadence for them, but then, if that is part of the definitional process, then it makes it a lot easier for these so-called sort of tennis matches to take place, where you can begin to do the kind of experiments where you’ve got ChatGPT just being experimented with in the field, and people are trying to figure out, if I use it this way, what do I get? And then if people are trying to figure out from sort of top-down on a goals basis, like, a company wants to accomplish this big enterprise goal by the end of the year, and we’ve got this big initiative, and these things start to marry up together. But it only really happens in a way that feels orderly and can be kind of harvested into an inventory of solutions if it really comes from the same place, if it comes from the same originating sense of mission. And that comes from that, kind of, coherent—the values and the purpose articulation that has to come from leadership and that has to be communicated well and bought into all throughout the organization.

Brian: Do you ever see a friction between when that work is done to articulate this vision and what the values are that as soon as it comes to doing the work in that way, well, we need to get out fast, though, and we need to prototype it, and we need this. And, yeah, we’ll get to all that, but it’s like the organization just has this, like, rubber band of it’s like zoom, the culture pulls it back into its way of working because what’s being managed is schedules—

Kate: Yeah.

Brian: —and time and how fast and when will we get it out? And let’s not wait for perfect. Let’s just get it out quickly. Is that, like, part of this process of, like, how we’ll work differently to actually accomplish those things?

Kate: Because that means we didn’t reconcile the checkbook right, right? You know, because that means we weren’t intellectually honest about our culture. Our culture has that sense of acceleration and urgency and impatience for solutions, and so we need to capture that and make sure that’s reconciled in what we’re talking about. Yeah, I absolutely see that happen, pretty frequently. And once we’ve captured it, once we’ve said, like, look, we are dealing with—maybe we want culture change. Maybe we actually want to pivot the culture toward a more sort of reflective culture.

But I should say at this point that ethics and governance and this kind of human-centricity isn’t necessarily slow. Sometimes it’s slow and it’s intentionally slow, but I think it’s only slow if we haven’t done any of this work, and it has to insert itself into the process in an intentional way. Once we’ve done some of this work, we have some of the vocabulary, we have some of the concepts, we can hearken back to those in conversation, we can start to have a cadence of repeat meetings and discussions about this, where we go, like, oh okay, if we had followed the way we’d articulate this, you know, six months ago, then we might have done it, like, XYZ, but we actually did it, like, ABC. So, now we have to think about, does that mean XYZ was wrong or ABC was wrong, or is our whole articulation wrong? And it’s just this kind of self-awareness.

It’s not necessarily slow. It’s just the kind of intentionality, like, a sense of pace, of bringing yourself back to why are we making these decisions? Why are we doing it this way and not this way? And if it’s important to us that we be fast, that we be agile, then that has to be captured in the process. But it doesn’t mean—fast and agile doesn’t necessarily mean unethical or ungoverned, right? Like, we just have to have an Agile process for, you know, clearing it against the ethical frameworks and the governance policies that we’ve put in place.

So, all of this can be accomplished in any kind of organization. And I often say, you know, one of the things I captured in What Matters Next is, some people have the luxury of making decisions more slowly. And I think when we do have that luxury, we should take it. You know, it’s a really nice thing to be able to do, to sort of luxuriate in the fact that, well, this doesn’t need to happen today, so let me think about what’s the right way to do this.

But many organizations do need to make rapid decisions. And what happens is that going through this process, understanding the Now-Next Continuum, understanding this process I talk about in the book about better questions that lead to partial answers, that lead to insights, which also lead to foresights, going through that process a few times begins to instill a more disciplined way of thinking, no matter how fast or slow you’re doing that process. And so, as you start to become really versed in it, you can be in a really impatient, urgent kind of accelerated environment and still be thinking in a very disciplined way about the different types of components of the decision that need to be made, and be able to still gather the right kinds of—using the insights you’ve already inventoried, using what you already understand about the culture of the organization, the dispensation of what you’re trying to accomplish, and you can still spin off foresights, in a sense, where you’re setting yourself up for greater future readiness as time goes on with each cycle of this process.

Brian: I always kind of come back to it’s like, fast as compared to what and to what outcomes you’re trying to get. Because you can be really fast, but then it’s like, well, did you create—especially product people—did you create the benefits and the outcomes that you promised or that were desired? The tech might have shipped fast, but what matters is, what is the downstream effect of the technology is really what the game is, if you’re playing it right, in my book. Do you agree with that, like, that framing?

Kate: Yeah, yeah. I think that all sounds very sensible. There are organizations that I’ve been part of that are able to be super nimble, super fast, and have, you know, these kinds of disciplined approaches. I mean, Netflix was one of them. Like, being at Netflix, even in the time that I was there in ’99, 2000, 2001 you know, obviously the sophistication of the technology was not what it is today, but we were still using an awful lot of algorithmic optimization, a lot of database decision-making and, you know, rolling out all the technologies that sort of set the precedent for e-commerce today, and even in doing all of that at a really good clip, you know, being this, kind of, upstart company with an 800 pound gorilla breathing down our neck in the form of Blockbuster, you know, like, that was all still happening in a way that was very centralized, very kind of—well, I should say, decentralized and centralized because we had one central metric we all knew we were aiming for, and that was our customers, like, how many customers we had, and then every department had their own metric that they were centrally, you know, accountable to that we were reporting each month.

We were gathering at the Los Gatos Movie Theater. They rented it out one Friday every month, and we had our metrics meeting, and every department head was presenting their metrics reports while the rest of us sat there and ate popcorn and drank coke in the audience there. And it’s a really fun way to do it because you’re, like, okay, well, I am accountable for these metrics, I need to share them, and, you know, be transparent about them, and also it’s super on brand that we’re doing this, and it builds this culture of accountability, and also builds a culture of that sort of cadence of, you know you’re going to need to report on this monthly so you got to make a lot of progress each month. And that sort of trickles down into, what does that mean within each of the groups? How do we break up the projects that we have in front of us? How do we show some movement over the next month?

And that can get kind of chaotic and fast-paced, but you’re still doing it within this architecture, this really clear metrics-based architecture, and all the data is very connected, you know, we’re all trying to make sure that we’re making the best decisions with the best data that we have. So, all of that feels like there’s really no need for the decisions to be made, for any product to be launched or rolled out without necessarily aligning with those metrics or with the communication process that flows within that company. So, I see that as kind of the platonic ideal of it all, and I see very few companies today who are rolling out sort of similar approaches to that, in spirit at least. It feels like sometimes they’re doing some portion of that. But I am such an adherent, such a fan, of what I saw Reed Hastings and Marc Randolph do in those days at Netflix, and obviously it worked [laugh]. So, I definitely feel like there’s a lot to be learned from the wisdom of those early years, and I try to share that with the folks that I speak with and that I consult with.

Brian: Yeah. No, thanks for sharing that. So, your book title, What Matters Next: The Leader’s Guide to Making Human-Friendly Tech Decisions in a World that’s Moving Too Fast, talk to me about, how do we measure that the solution was human-friendly, especially if that feels like a more intangible go—I think most of our listeners probably feel like something like human-centered, human-friendly, usable, useful, these sound like things that are hard to quantify, whereas 38% of level-one ticket requests can now be answered by the bot; I can show a hard financial metric or progress there. But let’s say you’re a customer at an insurance company. You had a car crash, and you want to know when you’re going to get your check to fix your Volkswagen. How do we measure that customer experience, the human-centeredness part? If we’re going to deploy a chatbot, say, to answer questions about my claim, if that’s, like, the technology that we’re rolling out to a customer, how can organizations begin to think about measuring human-centeredness?

Kate: Yeah, so the reason I chose human-friendly over human-centered partly because I wanted to be very honest about the goal and not fall back into, you know, jargony kinds of language that, you know, you and I and the folks listening probably all understand in a certain way, but the CEOs and the folks that I’m necessarily trying to get reading this book and make their decisions in a different way based on it—and also whose language I’m reflecting when I say a world that’s moving too fast, like that’s the language of the CEOs that I’m talking to—I wanted to make sure that the language meant something to them and not just to us. So, when I think about how we measure human-friendliness, or human-centeredness, or any of that kind of stuff, and it’s obviously difficult. You can’t necessarily set up a hard metric for that, but you can, as you said, you can come up with proxies, right? You can think about ways that that alignment between what people are trying to do and what you’re trying to do as an organization can be dimensionalized. Like, how can we look at metrics that show that alignment, and what are we doing to make sure that those are getting better over time?

How can we make sure that—you know, I think when we think about proxy metrics, there’s, really, a lot of metrics that sort of sit in the periphery of user-friendliness or human-friendliness that are things like—like, if I think about meaningfulness, I want this to be a meaningful experience for people if I can possibly have that be true. If I can possibly create meaningfulness for people, how do I do that? How would I make that possible? And so, a net promoter score comes to mind as one that sits in the adjacency. Like, if it’s meaningful, there’s a very good chance that somebody is going to recommend it to others.

If it’s meaningful, people will, you know, maybe remember our brand. So like, these kind of brand memorability scores and things like that. You know, there’s things that sit near it that people might rate their experience in a customer support service interaction as having been helpful or something like that, those are going to be things that you just, kind of, create a circle around the goal you’re actually trying to hit, right? If user-friendliness is the goal, if meaningfulness is the goal, if memorability is the goal, if you know changing the culture, you know, if you really want to sort of set up to change the culture of X, whatever that might be, then you need to understand, how can you get the closest that you possibly can to measuring that, and point yourself directionally in that order.

And I think at the same time, we need to have the ‘both and.’ I’m a ‘both and’ kind of gal. I really want there to be kind of intellectually honest and emotionally honest discourse within the organization that says, like, look, we can’t measure all of this. We just have to make really good—we have to use our good instincts and use our good judgment to say every piece of this isn’t going to be measurable, so when we look at that and we say, like, okay, an accident claim, someone had a car accident, and their claim was processed, you know, 20% more effectively than it would have been last year before this AI project that we rolled out. Do we feel good about that? Do we feel like we did the right thing? Are we moving in the right direction? Can we honestly, you know, go to sleep at night knowing that we’re moving in the right direction, not only for the company, but for the people who are on the other side of the company?

And that, I think, is the piece that I’m really hoping that the book reflects the candor of. Like, I don’t think it’s useful to only talk about the metrics. I think it’s really, really important to talk about the metrics, to get the right metrics, and to make sure that you’re doing that alongside, you know, much more, like I said, intellectually honest and emotionally honest language about what it is you’re trying to accomplish and why you even exist as a company.

Brian: Yeah, the challenge is always, you know, these short-term thinking, short-term objectives, like, practical stuff, customer retention, acquisition of new customers, reducing cost, all this kind of stuff. It’s that tension, and I’m always trying to think about ways to help, you know, our product management audience that’s listening—particularly those that are in the analytics and AI space now, since there’s a lot of investment here and there’s a lot of hope and expectation here—how to balance those objectives and doing the work in a human-centered way, which is a long-term investment approach. And I agree with you, it doesn’t need to be slow. It’s only slow if you’ve just done—it’s like, if you’ve never done QA testing and you just ship first draft code, you ship your vibe code out the door, then QA is going to sound like it’s slow because you’ve never done it before. Why do we need that? That’s just, like, going to slow us down.

Well, as compared to what? Like, putting crap out and then paying for it later with angry customer service inquiries, or, you know, ethical things, which is kind of where I wanted to go next. My challenge with the ethics thing is, in order to do a good job at ethics, to make solutions—particularly AI solutions—ethical, I think for a if I put my audience hat on for this show, they’re like, well, define what ethics is, and being ethical as a company so that I have a target that I can align my product with. Because how are we going to measure that it was ethical? Otherwise, how can I be accountable to that? And I want to be accountable.

And maybe, personally, we all have our opinions about is this right or not, but the problem with ethics, it’s such a gray space. What one person sees as reasonable, someone else may be like, that’s not really we shouldn’t really be doing that with this data or whatever. Can you talk to me, like, about how you think about, does ethics need to be defined within the organization such that it’s a concrete target that product teams can, like, play that game—and I don’t mean it in that sense—but like, they’re playing a game where they know that’s a measurement that’s important. How do companies do that?

Kate: Yeah, I think—so, companies do it in a number of different ways, but I think it’s healthy to have a kind of, in one sense, a recognition of a more global sense of what ethics is agreed to be, like, when we think about the more global discourse about AI ethics. Like, I participate in the United Nations AI advisory body meetings, there’s a global discourse about what responsible AI looks like, what it means like to protect data, to you know, make sure that we’re not, you know, running afoul of IP, you know, things like. Those discussions are part of the responsible initiatives that are happening globally. But then, of course, for a given company, we’ve already made the effort to articulate our purpose and look at our values and think about our goals and culture, brand, and experience. We’ve done that exercise, and now what we need to do is make sure that what we mean by ethics internally, the ways that we think we may impact particular kinds of outcomes for people, that those are reflected in what we mean by brand experience and culture or by values, or whatever we may be talking about.

For example, I think, you know, one of the things that happens a lot is the data privacy protection kinds of stipulations within different companies. You know, this is—well, let me actually not go into that specific just yet because I think there’s even a better model that, actually, I think is going to be helpful to this discourse, which is again coming back to the Now-Next Continuum in What Matters Next, there’s another aspect of that model that I introduced, which is the harms of action and the harms of inaction. And at any given point you find yourself at a fulcrum, along this trajectory where you kind of go, like, I know enough to make an informed decision, or I don’t know enough. And what I’m informed about are the consequences, right? Like, I’m informed enough about how rolling out a product like this, or tech like this, or you know, a data set like this, how training with this data set, whatever, how this is going to play out for the people who are going to be impacted by this, within the company, outside the company, in the greater world at large, et cetera.

And so, you can gauge by harms of action and harms of inaction, when you need to make that decision. Like, if you don’t have enough information, you’re not ready to make that decision, you’re not ready to roll it out, and that’s ethics in and of itself. To be able to have the organizational stomach to be able to bring up a concern, to be able to say to your boss or to, you know, whoever needs to hear it, we’re not ready with this because we haven’t been able to contain—go back to the data privacy example, right—like, we’ve got an issue where if we do what we’re proposing to do, that’s going to put people’s privacy at risk. So, we need to solve that problem first, and then we can begin to layer this kind of solution on top of that. Or we know that every moment we’re delaying rolling something out is costing us, or is costing people the kind of solution that will actually make their experience that much better, and then there’s harms of inaction that we’re invoking by sitting on something that should actually roll out.

So this, I think, is a helpful framing, just in terms of, you know, bringing the idea of ethics and the idea of acceleration together in a way that’s actually, you know, discernible, that you can actually make sense of, that you can actually decide what to do in an actionable way that’s not about, oh, okay, Kate’s saying, you know, use good ethics and have a good governance process. I don’t know what that actually looks like in practice. What I’m saying is, you know, make sure that you are asking yourself those hard questions about if I have enough information to think about unintended consequences, to think about how this is going to play out when it rolls out, you know, all of the other things that we could list out here. And then, you know, if I have that information, and all is looking well, then we go. But if we know that we’re likely to hit an issue where you know we’re using a recruiting system, like Amazon’s old resume processing system, and we can tell that we’re getting a set of bias coming in there, that we’re only seeing certain resumes, and it seems to be reflecting some kind of hiring bias that’s sort of inherent, that it’s looking at the last ten years of perfect fits within the organization, which happened to be predominantly white males, you know, and so, like, of course, it’s skewed. The data set is looking for more white males to bring into the organization.

So, once we recognize that, we’ve got a problem. We’ve got a dilemma in front of us, and we have to say, like, are we going to go forward now, knowing what we know, or are we going to try to fix this and then roll out a fix and try to improve upon this? I mean, I think it’s got to be clear to most people, most people with a reasonable conscience that the right thing to do is try to fix it and then roll out, right? So, there’s really not, I think, too many ethical gray areas for reasonable people. Reasonable people can usually agree where most of those lines are.

I think it just comes down to execution in many organizations. And it is the pressure of acceleration and the sense of not wanting to slow down to have these kinds of conversations. But you’re only slowing down, as you said, relative to the larger span of time—if you look forward on that Now-Next Continuum—to what’s likely to happen if you roll this out, and what you’re going to have to contend with when you now have lawsuits, or when you now—[laugh] when you’ve now, you know, normalized something that you have to actually pull back across multiple systems in your whole enterprise architecture. I mean, it’s going to be a mess. That’s not only technical debt, it’s ethical debt, and it’s a really important thing to be aware of, and eyes wide open about. And that, I think, is where we have done ourselves a disservice in every respect. We think that ethics is really just about, kind of, policing ourselves, and it’s slow. It’s like it’s actually helping us be more agile in the future, if we make the right decisions at the right time.

Brian: Yeah. Do you think, let’s say you’re a product leader in your organization, maybe your senior management is not super technology savvy, particularly around data and AI, and yet, there are high expectations about what’s possible here, what can we what kind of value can we create, if you’re trying to get leadership to buy into ‘slowing down’ we’ll call it or introducing a more human-centered, or human-friendly approach to doing this product work, do you think the easier sell to get them to buy in is prevention of the harms?

Kate: Yeah.

Brian: Like, the reason to do this is to—you don’t want to be on the front page in New York Times, or go check out AI Incident Database and look at what happens when you do it the wrong way. Is that the right framing or do you feel like that’s too short term? Or is it more like we should be more aspirational about the art of the possible and, like, the new values we could create? Like my feeling is, like, it’s a shorter-term perspective, but harm prevention might be an easier sell if you’re trying to—

Kate: Absolutely.

Brian: Yeah. Is that where you would suggest people start, like, start this from?

Kate: Yeah. Yeah, I have for years framed a lot of these discussions in terms of risk reduction, risk management and risk reduction. But you know, my last book before What Matters Next was A Future So Bright, and one of the things that I talk about in that book is the concept of strategic optimism.

And that’s the premise that, you know, we—I ask this in audiences all the time. Like, raise your hand if you’ve ever been in a meeting where you talked about what could go wrong with a project. And like, of course, almost every hand goes up. Like, but raise your hand if you’ve been in a meeting you talked about what could go right with a project? And maybe one or two, but very few hands will ever go up. And my point is, we miss out on a lot of the opportunity by not framing things in terms of what is possible, what could happen if this is a really successful product, if things go really, really well.

Because we actually have risk on that side too, right? Like the servers may not stand up to it. If it’s wildly popular and we’re suddenly a viral hit, and then, you know, we’re not prepared with, like, a media message or whatever like, that’s a form of self-sabotage by not thinking about that kind of success. So, it’s important that we are managing risks and managing harms and also thinking about what’s possible, and how can we lean into the good outcomes we would really like to see happen, like, what we would most like to see happen. I think those are too often reduced to you’re either going to do one or you’re going to do the other, and it doesn’t make any sense. We absolutely have to do both of those things.

Every organization has risks that they need to manage, and every organization has goals and aspirations that they would like to achieve. And I think if you’re not doing both of those things, I don’t know what kind of an organization you’re in. I think most leaders are trying to do both of those things. And so, I think, you know, maybe one or the other of those positionings kind of speaks to you more depending on your personality, but I also think that the ‘both and’-ness of it, sort of the strategic optimism framing is resonant for many leaders, and that ability to say, “Okay, we get it.” Like, we need to manage the risks of what we could be rolling out if we introduce tech that hasn’t, you know, thoroughly been vetted or hasn’t really been thought through and in terms of what it could do, and there’s an upside to rolling this out.

Like, we could see an awful lot of more efficiency in the calls we can handle, or, you know, the claims we can process, or whatever, so we want to get it out as is possible. And that tension is healthy. We should have that tension. We just need to be talking about it in the right way, and we need to have the right, sort of, frameworks and vocabulary for having that conversation really effectively.

Brian: Okay, this has been super fun to talk to my cuz [laugh]—

Kate: [laugh].

Brian: For an hour on—

Kate: Likewise.

Brian: —all of this stuff. I’m going to put a link to your book, What Matters Next, the koinsights.com one, is that the best one to—

Kate: Yeah, yeah, that’s great.

Brian: Link up in the [show notes 00:46:47]? I have a question for you. If I was a CEO—I know you primarily were with the C-suite, like, when you were speaking engagements and things like this—if I was like, at a party and I met a CEO, what scenario would need to be going on in their organization such that I would say, “Oh, my gosh, you guys should go talk to Kate O’Neill. Bring her in.” What would be going on there?

Kate: Almost any [laugh]. No, I think it usually it’s about—many, many organizations come to me when they’re at a process where either they’re reviewing their product suite or their go-forward strategy on different things, and they’re trying to think about future-readiness. That’s, you know, one of the terms that I often use. And so, I’ve worked with very, very large organizations on thinking about the future-readiness of the products that they have and the solutions they have. But the other thing is, you know, just thinking about that, like, I talked about digital transformation and innovation as those, kind of, companion pieces on the Now-Next Continuum, you could be at any point in any of those and recognize, you know what, we’re not thinking about this well. We’re not making these decisions very well and we need a better vocabulary for this. We need better strategy for this.

And that is what I really love, to come in and work with organizations at a whiteboard or in a boardroom or whatever, and just hear, listen, you know, to what they’re dealing with and what they’re trying to accomplish, and where I’ll hear, you know, oh, I hear it. There’s the gap right there. And let’s see if we can’t address that gap and make things a little smoother for everyone.

Brian: Cool, cool. And where’s the best place for them to—if they did want to reach out and that’s true in their situation?

Kate: Yeah, just at koinsights.com. There’s a lot of information there about me, about my books. Not about Brian, though. But there will be. There on the press page, we will soon have a link to this podcast.

Brian: [crosstalk 00:48:37] Brian page [laugh]?

Kate: [laugh].

Brian: Too Many O’Neill Podcast. By the way, do you still run the Tech Humanist podcast? Is that—because you have a podc—you’re a host as well?

Kate: Yeah, I’m a host of the Tech Humanist show. So, folks are welcomed, invited to join in on that, as well. And we’ve had, yeah, we’ve had some incredible guests along the way. It’s a really cool thing to kind of pair up with being a keynote speaker because I’ll have—like, I did the Nevada Mining Association, which, you know, is such an arcane, like, specific thing, and then I had the CEO of that organization on to talk about how they’re using AI and technology in mining, and what direction that’s going. So, it’s a really cool conversation, but you get to, you know, kind of explore all different facets of tech there.

Brian: Got it. And by the way, folks, that was not data mining. Like… mining.

Kate: No, no, not data mining, although we did make that joke a lot, a lot.

Brian: That’s pretty low. Two out of ten on that one [laugh]. Kate, thank you so much. This has been really fun, and I hope to stay in touch with you, and I’ll put all those links in the show notes and everything for our listeners. So, thanks for coming on Experiencing Data.

Kate: Thanks Brian. Thanks to the listeners and thank you.