As is typical for most web technology since I have been in the space, engineering and tech will always lead the race—and design and business will chase behind it, trying to make it useful, usable, and trustworthy—and monetizable of course.

So—how do we address the monetization part as product leaders?

Well, the lucky few of you will stumble upon a problem that is so excruciatingly vivid and easy to address, that throwing even modest tech at it will create something worth paying for—regardless of how much friction, risk, or issues of trust may occur. The rest of you will live in the world of the majority—where it seems like so much is now possible, but now, the game is to see what is actually worth building. We've sped up the delivery work with AI and will continue to do so with AI—but the hard work (for technical PMs in particular) is actually figuring out what is worth building in the first place.

If you're jumping into the space of agents, there are a number of considerations you should probably be thinking about—and some of these aren't really "new," but they might be more important than they used to be because of the non-deterministic nature of the software being created.

Here are 5 ideas to consider:

• Figuring out the right amount of transparency in the UX about what AI is doing - there is not enough, and too much, and a big "middle." These gray areas are why design exists, because your grey area probably won't match the next team or product's—so you can't just copy blindly and apply somebody else's thinking to your interface and UX. The other tricky thing here is that the level of transparency you may need to expose can change over time as trust increases. Which brings us to the T word:
• Trust in the system's accuracy, ethics,  data privacy promise, and reliability: when we think of things like "brand" trust, and why a plain white Chanel t-shirt might cost $500 vs. that trusty Hanes, we're not in the domain of data and logic anymore. Trust is something that happens over time, and it can be broken quickly. A lot of small details go into making people trust the system—and the developers behind it if you work internally. Who is overseeing all of that? Is anybody giving any of that intentional focus—or are you just hoping that having "accurate data/responses" is enough? And the topic of ethics? Well, if it's relevant in your context, I almost hesitate to mention it; everyone wants this and believes it's right; but the problem is that it often requires a firm definition of what "ethical" means such that one can know when the model has crossed the boundary.
• Model/system feedback and control: this is similar to transparency about "how it works," but goes a bit farther. How much and when do we surface the work of an agent to the user? Of course, in some situations, users need to be able to interrupt, re-route, stop, or modify the work of an agent—so how much feedback should we give users without making the process overwhelming? On top of the challenges of productizing an agent, if the system allows the user to have some control into the building of the agent itself, then the level of control needed will likely change over time as the user of the system evolves from a composer/developer to an actual user (beneficiary).
• User<>Agent relationship management, memory, and context: What does the system remember (or not) about the user, their context, their preferences, etc? Does this expire? How many times have you wondered if you need to keep prefacing your prompt with "Act like the world's best _____." Do the foundation models remember this preference ? For how long? In your  data product context, should the model be remembering this? How and when we expose any of this knowledge or metadata to the end user?
• Incident remediation / avoidance in future: when the model generates an unexpected response, and action is taken, what is the product-user-brand experience? What can the user do about an unexpected response? What, if any, feedback can be given to the agent to prevent this in the future?

Nobody has all this figured out; myself included - it's a really new space with the tech, but the human factors aren't all that different from technical software product design—so we can extrapolate what's going to matter and where attention needs to be given if we want these data products to produce business ROI.

Good luck, 007.

Photo by ADEX PHANE on Unsplash

.