Speed is often confused with good product thinking. The idea is that if teams can ship prototypes, dashboards, and models faster, they will automatically learn faster. But execution speed alone doesn’t ensure a clearer understanding of what’s actually worth building.
Instead, teams often fall into a loop driven by demo feedback. They present working prototypes, and users respond to what they can see in the form of interface design, visualizations, or surface-level data behavior. While this feedback feels positive, it’s often misleading. Teams can end up reacting to presentation (UI) feedback only to find it does not change propensity to buy or increase user adoption.
The key idea today is that prototypes can either be used to clarify the problem space and user needs or to validate the solution presented. Where I see most teams fail is that every artifact or prototype is seen as a solution to validate, and they can miss the forest for the trees.
Another approach borrows from blue ocean thinking, which focuses on creating value by looking for overlooked opportunities in the empty space—beyond the known “problem space” your customer knowingly lives in now.
Because AI lets us move so fast with prototyping, I think there is an exciting possibility to explore the blue-ocean spaces where your product could evolve to produce value.
As always, we seek to go beyond building “technically right, effectively wrong”—which doesn’t make people buy, use, or refer your product. Today, we look at what AI can help us to do to see even farther beyond the immediate problem space.
Highlights/ Skip to:
- Where the idea for this episode came from (00:46)
- Why faster building of artifacts with AI doesn’t necessarily mean faster market validation (2:09)
- How understanding the problem space results in fewer prototypes being created (5:40)
- Using blue ocean strategy to arrive at new products worth paying for (08:23)
- Finding missed market opportunities blocked by cost, tech limits, or risk (12:39)
- How AI-assisted user research fits into blue ocean thinking (14:33)
- The big picture: winners will figure out what’s worth building before they build it (20:42)
Links
Transcript:
Brian: Welcome back to Experiencing Data. This is Brian T. O’Neill. Today, beyond deliverables, how AI user research and blue ocean strategy can help you discover analytics products worth paying for. Where did this episode come from? Blue ocean strategy? What are we talking about here?
Tonight, I was sitting at the drums. I’m just about to start, like, a six-week run of a new musical—or it’s a musical that was on Broadway that’s in production in Boston—and it’s often, it’s dark before we play the show. You know, we’re backstage, it’s quiet, the audience is talking, waiting for the show to begin, and a lot of times I just, this is where I do some of my thinking work and I started typing up this episode. The ideas were coming from other books I like to read, podcasts I’ve been listening to. Today’s episode is definitely informed by the work of Alex Smith from basicarts.org.
So, somehow I was thinking about some of the things I learned from him, and he would even say that this whole blue ocean strategy is not his own—and it’s not—but I was thinking about how this could help both anyone basically creating new forms of products, whether they’re analytics products or intelligence products or whatever, kind of, looking beyond obvious problem spaces to look for latent problems. So, that’s a lot of what I want to talk to you about today, as well as how AI fits into all this, given how a lot of teams are currently using it.
So, to begin here, the uncomfortable truth, I think, is that everybody, right now, that is working in technology and data, as you know people are—literally billions of tokens are being spent right now with generating code, generating designs, generating plans. Like, these tools are fascinating. It’s amazing. The productivity. I haven’t written this much code or worked on software, like, hands-on this much in the last ten years as I have in probably the last three weeks. There’s so much going on here and a lot of it’s super powerful, and I’m really excited about that.
But the question is, as you get really fast with faster delivery work, you’re able to generate deliverables, design mock ups, coding, assets, applications, prototypes, clean data pipelines, whatever it is that you’re using it for, if you’re trying to build a monetizable product, the question is, are your sales metrics, your user adoption metrics, are those business KPIs also moving faster now that you can do all this? And if you are actually getting stuff out, and you’re trying to validate that and you’re not seeing the business metrics move as fast, then the question really isn’t about whether we can execute fast enough. Like, this idea of speed being the issue, maybe that’s not the issue. You can accelerate solution delivery and testing, but that’s not necessarily the same thing as learning more about the customer problem space.
So, the trap is this idea that faster building means faster and better validation. Lots of generated stuff with the tools, which turns into, ‘well, let’s just build it and then show customers and get feedback.’ But I think what this can lead to, particularly if you don’t have trained researchers and designers on your team to help understand the types of questions you’re asking, where there can be bias in those questions, and how to interpret the feedback you’re getting, then customers end up reacting a lot to what they can see. And I’ve heard this before with prospects I’ve talked to who’ve come to me for help. They’re getting feedback on charts and visual design and the AI responses and questions about data accuracy or how fields were calculated and is the data real time and they really, like, in the weeds kind of stuff.
And while some of those may be valid questions, I think sometimes people are just reacting to what they can see, but they’re not actually thinking about their problem space anymore. They’re just trying to analyze your solution and try to back it into their world. And so, on top of all this, you can get false positives, feedback about how cool your idea is or, ‘I could definitely see someone in our company finding this really valuable,’ or this kind of feedback here basically what is vague interest, that doesn’t necessarily signal a willingness on their part to move towards some kind of commitment with you, whether it’s a financial commitment, a reputational commitment, or a time commitment like, ‘I would love to get on a second call and talk about this further,’ you know, things that are proxies for knowing that you’re actually making forward progress here. You can actually end up wasting time with customers that you can’t get back. And what I mean by that is if you’re trying to build a product that needs to be bought by someone who’s fairly senior inside an organization, you may not get a whole bunch of at-bats, to use the baseball analogy.
You can’t just keep coming back to them because you can come up with six different products. They’re probably not going to give you the time to look at six different versions of something, whether it’s all in one sitting or whether it’s in another one. And frankly, I would generally advise you not to go bring four or five versions of something to a customer because you’re going to basically invite them to give you design feedback at that point, and that’s actually not going to help you know whether or not you’re actually moving forward with your business metrics. So, the more you understand the problem space—which doesn’t necessarily require you to have a solution in hand to validate—the fewer prototypes you actually need to create. And so, you can actually reduce the chance portion of your product bet.
So, if you’re pre-revenue, or you’re working on new features that you hope will generate more revenue, or some kind of transformation or change, then there’s always going to be some element of chance that, will the market want this, and you can research it to death and feel like we have completely—you know, we know exactly what everybody needs, and then they still don’t pay for it. There’s always a window here of chance where we just don’t know how something is going to be received until it goes out. So, the question is, well, how much of that is chance and guessing, and how much of the bet, this increment of product you want to put out and get validation from the market from, how much of that is evidence-based? And I think if we’re always in solution delivery mode, we’re not actually increasing the evidence and knowledge of the problem space inside of our bet. So, more of that bet is going to be chance-based and less evidence-based.
And the reality is you probably just don’t need to build that many different prototypes if you know exactly what the needs are or where the opportunities are. And that’s not necessarily going to come from just constantly generating more prototypes and attempting to go get feedback on them. So, the big idea, you know, prototypes are easier to create en masse, but it’s easier to misuse these if you frame each solution concept as something that you need to test for market acceptance in the form of a purchase or an intent to purchase or something like this.
So, what’s another way to think about this, about how to arrive at high-value features, or literally, if you’re a startup, a product that’s worth paying for? And so, I’ve been following, again, this guy, Alex Smith, who from basicarts.org who talks a lot—he’s a strategist, and I think he works with a lot of, like, CPG brands, physical product companies, on their strategy. And he’s borrowing the blue ocean—what’s called the blue ocean strategy. I’m sure some of you are familiar with it.
So, like, the key points here, the best analytics products don’t answer existing questions better. They’re going to enable questions to be answered that customers couldn’t ask before. So effectively, you are creating value sort of out of thin air, not by doing the old thing better than the other service provider or product company does, necessarily, but potentially by showing that there’s a better way or occupying a space that no one else is even inside of right now. So, let’s just give some examples to make this more concrete. You know, before dashboards and data was captured, no one was really asking what happened last week, beyond just their gut feeling about it or what the boss said or whatever. They’re not quantitatively going out and defining that because there weren’t dashboards.
And before self-service analytics, no one expected to be able to go in and explore the data themselves in order to validate a hunch or some feeling they had and to go and search for data to back that up or to challenge it. And before we had, you know, large language models, people used to search for web pages that they would then read in order to get answers. But now people have gotten accustomed to just wanting to get the answer with some backed up evidence. But it’s less about finding different web pages that I have to read to then go and draw some conclusion or answer from.
The two examples I really like—these are both from the physical product world—about how blue ocean can work is, imagine Disneyland. You know, prior to Disneyland existing, no one said, ‘Boy, we just really need this theme park with these imaginary, different lands that you can go to and rides and characters and music and junk food and, you know, the people-mover and all the great stuff that Disneyland has.’ Nobody asked for that. Like, you couldn’t research your way into that. Disneyland created value basically out of thin air.
Another example of this is something like Liquid Death. Liquid death is a water brand. You’ve probably seen it. Their cans that—they serve this in, tall boy cans like you see with, like, microbrewed beer, at least here in the US, and they have this very heavy metal branding vibe. And it’s literally water in a can. But people were willing to buy this because when you go out to a club or to listen to music, or you’re at the bar, you can look cool drinking this funky product. And it’s just water, but it’s signaling something to the people around you, and people were willing to spend money on this. And nobody was asking for another water company, let alone a water company with this, like, heavy metal, anti-marketing kind of vibe to it, right?
I think for the most part, no one’s waking up asking for new analytics or intelligence features, per se. This isn’t where people are sitting most of the time in their head space, but they do change their business when a product makes a previously impossible, high-value decision easier or safer or automatic. I mean, look this is getting kind of meta, but just look at what’s going on with all these Copilot tools and code generation and things like this. I mean, in a matter of weeks to months, it’s massively disrupted how software engineering is getting done and democratizing that.
So, blue ocean doesn’t mean inventing random AI capabilities with your product. I think it’s about finding, inside your market, decisions—in terms of analytics, at least—decisions that should be happening, whether that’s by humans or machines because we can automate some of that, but they aren’t because they weren’t technically possible or they costed too much or they were unreliable or they were hard to de risk. There’s a lot of different reasons, but it’s really looking at that. And then, of course, that has to map to some type of business value as well because it can’t just be any decision, like, should we install a seventh urinal in the men’s bathroom, or not? That’s nice. Most people inside a big office building probably aren’t sitting around answering that. So, even if you could answer that question, it has to be something people actually value, right?
So, I think this whole concept of the blue ocean strategy and kind of thinking in the negative space, what could exist in the negative space there, it’s challenge—I’m not going to say, like, it’s challenging even to get your head around this because even myself, I talk a lot to my listeners about being very problem-centered and problem-oriented, and I still think that there are places, there’s lots of opportunities to use a problem-directed strategy here. And it might be solving a product in a better way than they currently do, but there’s other ways to create value that aren’t necessarily having that deep, kind of, problem focus and looking at, like, oh, we could automate this process that’s currently manual, or reduce the labor of it and this kind of thing, right?
So, let’s talk about the AI piece here. And what I mean is AI in the product development process, your product development process. AI doesn’t just speed up building so it expands what could be asked or what could be automated or what could be decided. So, I think many teams—and frankly, I’m doing this on my own, like, home projects, like, my own applications that I run, some to manage my business, both my music career and Designing for Analytics, you know, my consulting business that you all know—a lot of teams, I think, are also using it to execute faster on the known ideas, the known problem spaces, less to explore new problem spaces or to think about how to reframe what they’re doing, right?
So effectively, this is kind of what we call, like, swimming in the red ocean, right? How do we remove labor from the workflow? How do we clean up the data? How do you enable talk to your data—you know, which presumes users always know what to ask in the first place, and that a chat interface is necessarily the right interface to begin with. But I don’t want to rat-hole on that, and I’m not saying that talking to your data is definitely the wrong way to go. I think there’s a space for analytical answer engines out there, but I actually don’t think that’s the be-all, end-all.
The real shift that I think needs to happen here is that if building is cheap now, if delivery and engineering work is cheap, you can be and you should be running more problem discovery, not more solution demos. So, prototypes are for exploring how things get done, not whether people want to do them, right? You need to understand how to ask the right questions if you’re going to bring a prop in, a prototype, for example. The prototype isn’t always the right way to test for demand, so we need to understand whether or not we’re even remotely in the ballpark of problems that people have right now or value that they could imagine based on if you give them imaginary scenarios, what may seem like a fictitious scenario, and exploring that. If we’re not using AI to look at this and instead just focused on delivery, well, you’re just pushing more things out the door, but we’re not actually learning any faster, and so those business metrics are not moving as fast as the delivery work is.
So, we’re not necessarily increasing the outcomes—to put it in my parlance that I use with outcomes and outputs—we’re not necessarily accelerating the pace of outcomes, positive changes in users’ lives or financial metrics for your target customer or for your own business, right? So, instead of, you know, would you use this or pay for this or what features do you want, try to understand what your buyers and your users, what decisions could they be making faster or better. Why is their current status quo, tool sets, workflows, why is that stuff the way it is? Why is the reporting cycle monthly and not weekly or real time? Is it monthly for a reason or was that imposed because that’s what the tools gave them?
Why are they moving data out of, you know, this piece of software and loading it into Excel and then transforming it and then sending it back to some other tool? Why does some process need manual review versus letting an algorithm decide? Why can’t that be automated? What decisions are they trying to make, or wish they could, but they typically just, they take too long or they’re seen as too political or they’re seen as too high risk? So, the tactical shift here is, I think I’m all for using these Copilots for user interface portion of design, which isn’t the user experience portion, but the UI part, and for engineering, assuming you have qualified technical leaders to check the code that’s being run and all of that. And I’m not an engineer, but I know enough about the crap that these things can generate that you still need to have your human engineers in the loop. And I think most of you probably know that stuff much better, so I’m not going to say a whole lot about that.
But I am a fan of using these tools on the product delivery side here. But you can also be using these tools in AI to generate questions and scenarios, not just interfaces. So, these tools can help you form customer research plans, what questions to ask, what you might want to try to go out and learn, what hypothesis could you validate or invalidate and how should you go about doing it, what signals might you want to listen for that could be really relevant, but might not come up with your particular prototype. Your prototype is not going to surface those unless you prompt the users for it. But you don’t even know what you don’t know, so you don’t even know to go into that topic space.
This is all stuff that I think AI is really good at because it knows enough stuff about all these different domains in the world that I think it’s a really powerful research aid. At least it helps—it can get you from zero to one to understand what questions to even ask the human user in the first place. So, I’m not thinking about it as much as giving you the answers; instead, it’s helping give you the questions that you might want to ask, and it gives you a chance to interrogate those questions before you bring them to your customers. At which point now you’re getting factual data from a human being, the person that’s actually maybe going to spend money with you and this is much better intelligence, much better signal than you’re ever going to get from Claude or whatever your favorite foundation model is that you’re using in your day-to-day work.
So, big picture today, the winners won’t be the teams that are just building fastest; they’re going to be the ones that figure out what’s worth building before they build it. Because everyone is moving faster on the delivery side, but I think you should know that, like, this also means your customers might be thinking about building their own stuff, too. So, now it becomes, is there enough value in what you’re making? Is it worth building in the first place? What is the definition of quality from the perspective of the customer?
AI is changing what we can put into analytics and B2B intelligence products, but it’s also changing how we can do both discovery and product delivery work. So, it’s not just about using these capabilities in the product, but it’s also about how we use AI in the product development and design process. So, I’m hoping this episode gave you some other things to think about besides code generation and user interface artifact generation. I’ll tell you right now, just as the time I’m recording this, Claude Design just dropped. I think it’s impressive the number of products they’re putting out on top of their platform. It sounds like it’s answering a lot of challenges that people in the design community and even outside the design community have been having sitting at that intersection of code and visual design. So, I’m very excited to go and try that, but leveling up we have to think higher than just the user interface piece if we want to build products that sell themselves.
So, that’s it for today. Thanks for listening to Experiencing Data. If you’re trying to figure—crack this nut and you feel like you need an outside perspective on this and you’re not sure how to trust what—I’m sure you’re using large language models to help you with some of your own strategy. I do that as well, but sometimes what you need is the empathy of a human ear, another human that’s gone through these kinds of things before, that can help you understand, are you getting BS back? Can you believe any of this stuff?
Like, if you don’t, if you’re not an expert, how do you know that the information you’re getting back is actually pointing you in the right direction? So, if you think a human being would help, and you’re trying to build a data product that will sell itself, that’s what I try to help my clients do. I want to help you turn analytical complexity into commercial clarity so you’re shipping benefits and outcomes, and you’re improving your customers’ lives, and you’re making them pay for something that they just can’t live without.
So, if you’d like some help doing that. You can head over to designingforanalytics.com/contact and you can set up a free discovery call with me. I’ll do a rapid assessment of your situation one on one, and I’ll try to give you some advice on how you might want to move forward, whether it’s getting help from me or someone else, if it’s not something I work on or maybe you don’t need anything further than that. So, feel free to grab one of those if you’d like, and see you on the next episode.