Episode Description

Jeremy Forman joins us to open up about the hurdles– and successes that come with building data products for pharmaceutical companies. Although he’s new to Pfizer, Jeremy has years of experience leading data teams at organizations like Seagen and the Bill and Melinda Gates Foundation. He currently serves in a more specialized role in Pfizer’s R&D department, building AI and analytical data products for scientists and researchers. .

Jeremy gave us a good luck at his team makeup, and in particular, how his data product analysts and UX designers work with pharmaceutical scientists and domain experts to build data-driven solutions..  We talked a good deal about how and when UX design plays a role in Pfizer’s data products, including a GenAI-based application they recently launched internally.

Highlights/ Skip to:

• (1:26) Jeremy's background in analytics and transition into working for Pfizer
• (2:42) Building an effective AI analytics and data team for pharma R&D
• (5:20) How Pfizer finds data products managers
• (8:03) Jeremy's philosophy behind building data products and how he adapts it to Pfizer
• (12:32) The moment Jeremy heard a Pfizer end-user use product management research language and why it mattered
• (13:55) How Jeremy's technical team members work with UX designers
• (18:00) The challenges that come with producing data products in the medical field
• (23:02) How to justify spending the budget on UX design for data products
• (24:59) The results we've seen having UX design work on AI / GenAI products
• (25:53) What Jeremy learned at the  Bill & Melinda Gates Foundation with regards to UX and its impact on him now
• (28:22) Managing the "rough dance" between data science and UX
• (33:22) Breaking down Jeremy's GenAI application demo from CDIOQ
• (36:02) What would Jeremy prioritize right now if his team got additional funding
• (38:48) Advice Jeremy would have given himself 10 years ago
• (40:46) Where you can find more from Jeremy

Quotes from Today’s Episode

• “We have stream-aligned squads focused on specific areas such as regulatory, safety and quality, or oncology research. That’s so we can create functional career pathing and limit context switching and fragmentation. They can become experts in their particular area and build a culture within that small team. It’s difficult to build good [pharma] data products. You need to understand the domain you’re supporting. You can’t take somebody with a financial background and put them in an Omics situation. It just doesn’t work. And we have a lot of the scars, and the failures to prove that.” - Jeremy Forman (4:12)
• “You have to have the product mindset to deliver the value and the promise of AI data analytics. I think small, independent, autonomous, empowered squads with a product leader is the only way that you can iterate fast enough with [pharma data products].” - Jeremy Forman (8:46)
• “The biggest challenge is when we say data products. It means a lot of different things to a lot of different people, and it’s difficult to articulate what a data product is. Is it a view in a database? Is it a table? Is it a query? We’re all talking about it in different terms, and nobody’s actually delivering data products.” - Jeremy Forman (10:53)
• “I think when we’re talking about [data products] there’s some type of data asset that has value to an end-user, versus a report or an algorithm. I think it’s even hard for UX people to really understand how to think about an actual data product. I think it’s hard for people to conceptualize, how do we do design around that? It’s one of the areas I think I’ve seen the biggest challenges, and I think some of the areas we’ve learned the most. If you build a data product, it’s not accurate, and people are getting results that are incomplete… people will abandon it quickly.” - Jeremy Forman (15:56)
• “ I think that UX design and AI development or data science work is a magical partnership, but they often don’t know how to work with each other. That’s been a challenge, but I think investing in that has been critical to us. Even though we’ve had struggles… I think we’ve also done a good job of understanding the [user] experience and impact that we want to have. The prototype we shared [at CDIOQ] is driven by user experience and trying to get information in the hands of the research organization to understand some portfolio types of decisions that have been made in the past. And it’s been really successful.” - Jeremy Forman (24:59)
• “If you’re having technology conversations with your business users, and you’re focused only the technology output, you’re just building reports. [After adopting If we’re having technology conversations with our business users and only focused on the technology output, we’re just building reports. [After we adopted  a human-centered design approach], it was talking [with end-users] about outcomes, value, and adoption. Having that resource transformed the conversation, and I felt like our quality went up. I felt like our output went down, but our impact went up. [End-users] loved the tools, and that wasn’t what was happening before… I credit a lot of that to the human-centered design team.” - Jeremy Forman (26:39)
• “When you’re thinking about automation through machine learning or building algorithms for [clinical trial analysis], it becomes a harder dance between data scientists and human-centered design. I think there’s a lack of appreciation and understanding of what UX can do. Human-centered design is an empathy-driven understanding of users’ experience, their work, their workflow, and the challenges they have. I don’t think there’s an appreciation of that skill set.” - Jeremy Forman (29:20)
• “Are people excited about it? Is there value? Are we hearing positive things? Do they want us to continue? That’s really how I’ve been judging success. Is it saving people time, and do they want to continue to use it? They want to continue to invest in it. They want to take their time as end-users, to help with testing, helping to refine it. Those are the indicators. We’re not generating revenue, so what does the adoption look like? Are people excited about it? Are they telling friends? Do they want more? When I hear that the ten people [who were initial users] are happy and that they think it should be rolled out to the whole broader audience, I think that’s a good sign.” - Jeremy Forman (35:19)

Links Referenced

• LinkedIn: https://www.linkedin.com/in/jeremy-forman-6b982710/

Transcript

Brian: Welcome back to Experiencing Data. This is Brian T. O’Neill. Today, I have Jeremy Forman on the line. We met at Chief Data Officer IQ, the CDOIQ symposium, what, last July, I think it was—

Jeremy: Yep. Yeah, last July, yeah.

Brian: In Cambridge. Yeah, yeah. And it was great to meet you, and I was interested in talking to you because you gave a talk there, and of all the talks that happened there, I’m generally like looking for things around the space of data products and product management and user experience design, and rarely do I see those words talked about too much at CDOIQ or a lot of data conferences, the experience of using these solutions that we’re putting out. And so, that was on your slides there, and I enjoyed the session that was there. So, that’s why I wanted to bring you in today, and that’s why I’m excited to share your insights from Pfizer, and maybe you could talk about leading up to Pfizer, what happened. So, where were you before that because that’s somewhat new, right? Like, you’re kind of in a newish role.

Jeremy: Yeah. Pfizer, I’ve been in role since April 1st, so pretty, pretty recent. Pfizer acquired the company I was working for, Seagen, which was a mid-sized biotech in the Seattle area. And I was doing similar work, AI data and analytics for Seagen, just the difference is that at Seagen, I covered the whole company, and at Pfizer, I focus solely on R&D.

Brian: Got it. Got it. How is the change there between going from the [laugh] mid-size wide gamut of work down to what you’re doing now. Like, what’s that like?

Jeremy: Yeah, I mean, the breadth of pharma R&D, just the potential there, especially a large pharma, is just immense. Even though, you know, I’m not focused on commercial anymore, or finance, or HR, the breadth of R&D at a large pharma across all the different therapeutic areas, oncology, immunization, vaccines, it’s just massive, and it’s actually where my passion is, especially on the pre-clinical research side. So, the transition has been great because I can focus on the areas that I’m most passionate about. I really think we can, with data specifically, can make a huge difference.

Brian: Tell us a little bit, like, what’s your—just to give people, kind of, a sense of your universe, like, are you managing a team? And if so, what kinds of roles are on the team that you have?

Jeremy: Yeah, so managing a team… or a really new team, and I’m approaching the team similar as I’ve learned over the years how to approach building an AI analytics and data team, so the types of roles that we have—and going to try to stay as generic as I can, so I don’t go into any specifics within Pfizer that might, you know, get people upset—

Brian: Got it. Understood.

Jeremy: —but we’re functionally organized as a team, which means I have an engineering side, I have a product side, I have an AI innovation data science side, and then a solution architecture side. But that’s different than how we actually operate and go to market, if you will. So, I have the functional alignment so that we can build really good, strong career paths for, say, data product managers or engineers, but then when we, kind of, deploy our team and actually go to market, if you will, we’re organized in squads. So, product manager, solution engineer, lead engineer, data product analyst, and designer are deployed in a persistent capacity for a specific domain. So, kind of taken from a team topology’s mindset, if you’re familiar with that, the idea of source-aligned or stream-aligned teams, we have stream-aligned squads that are focused on specific areas of the business, like regulatory, safety, and quality, or oncology research.

It’s so that we can create both the functional career pathing, but also through the deployment of our squads, people limit context switching and fragmentation. They can become experts in their particular area, and they build a culture within that small team that helps them to execute efficiently over time, and hopefully find a little joy in the work. One thing to add to that is part of the reason to do that is because data only has value in context, and if you don’t understand the context—

Brian: Right.

Jeremy: —it’s really hard to build good data products. And pharma R&D, you know, you have to—you just have to have an appreciation and understanding of the domain you’re supporting. You can’t take somebody that’s, you know, got a financial background, and put them in an Omics situation, or an in vivo/in vitro experiment context. It just doesn’t work. And we have a lot of the scars, and the failures to prove that out [laugh].

Brian: Right, right. I would assume, though, that at some point the individual skill areas that you have—the designers, the data product manager, the data scientists, whatever—did not have an oncology domain experience at all. So, at some point, they were at zero. So, talk to me about how you build that up. I mean, this is common in anything with tech, as I’m sure you know.

It’s like, I mean, I work on new domains all the time that I’m exposed to, and there’s definitely value in when you can carry forward knowledge. But how do you deal with that? I mean, or do you only now hire with domain experie—like, if you hired a data product manager today, it would be, like, someone with proven oncology experience. We won’t bring in someone that’s coming at zero. Or how do you see that?

Jeremy: Yeah, absolutely. Especially in the data product manager side, we opt for people that have domain experience. And oftentimes we’ve hired people with domain experience that don’t have product management background, just because that interaction with the end-user, if you don’t have the language, you just won’t get anywhere. But on the other side, like, data scientists or some of the designers, we do try to find people with domain experience, but a number of times, we’ve brought in people who are just really good at what they do, and they learn the domain. The product manager side, it’s just not an option, right? You just can’t bring somebody in and have them sit down with a bench scientist and think it’s going to go well if they don’t have the language or understand the processes, or what it’s like to be a bench scientist.

Brian: Do you grow these TPMs—since you guys are in pharma, do you grow them in a Petri dish? Like—

Jeremy: Yeah [laugh].

Brian: —how do you find these people? Or they’re like, oh, it’s an engineer or data scientist that happens to have worked on the oncology side for eight years, and, like, they’re willing to, like, move into a business mindset and go beyond writing code all day. Is that—you’re kind of growing them in the house? Or—

Jeremy: Yeah.

Brian: Yeah, okay.

Jeremy: A little bit, right? You know, there are people that have project delivery experience, maybe some have been product owners before, could be somebody that’s just really skilled at facilitation, and kind of can get their minds around product strategy, product road mapping, and then we grow them. It’s very rare to find somebody that has both, although we have been lucky to find a couple of resources that have product background and understand oncology, but most of the time it’s hey, there’s somebody that’s been doing delivery work of some sort, technical delivery work, and really in a domain, really understands domain, and then needs to be scaled up on product management.

Brian: We already started talking about product here. And was that a given when you walked in the door? Is this something that you wanted to bring this, kind of, software product management operating model into Pfizer? Was it there when you walked in the door, or moving that direction? Where did it come from?

Jeremy: Yeah. I’ve been down this path three times, and every time it gets refined. And I am just convinced—and maybe I have blinders on or biases—but I’m convinced it’s the only way to go about delivering AI data and analytics in 2024 going out to 2025. I just, I don’t—I think the pace moves too fast. I think you have to have the product mindset to really deliver the value and the promise of AI data analytics.

I also think that small, independent, autonomous, empowered squads with a product leader is the only way that you can iterate fast enough with the business. And so yes, it is something that I’ve brought in, and it’s something that we did at Seagen, it’s something that we did at the company I worked at before, and just tried to refine that over time. But it is a pretty big change management challenge, and so that’s partly what we’re going through right now.

Brian: That had started at Pfizer before you came in the door?

Jeremy: So, there is definitely a product mindset and product thinking that I was not the first to bring that in the door. There’s a product management team, they’re very skilled, that are thinking enterprise-wide. I think it’s the data product thinking specifically that might be a little bit unique for what we’re bringing in, or maybe just accelerating some ideas people have had in the past.

Brian: And these data products we’re talking about, I’m assuming, internally-facing, non-commercial—you’re not selling this as a software offering from Pfizer for other companies to use—

Jeremy: Exactly. Yep.

Brian: Correct? Okay, so this is all internal. Do you guys have a common sense of what a data product is in the Pfizer definition of what that is, or a way that you help someone understand that? Because I’m guessing, the bench scientists probably don’t care, don’t know, really, what a data product is. Like, and I’m usually like, don’t use that language with them. Like, we talk about it over here in this little field because we know it helps us talk about what we’re doing, but it’s not customer language. So, what is it in this—in our little [laugh] bar that we’re sitting in here?

Jeremy: Yeah, no. It’s really interesting—and I’ll talk more about in the context of—because it’s so new here—at Pfizer, for me, I’ll talk about what we did at Seagen where, you know, at some point, a proud moment, I guess, or… a little bit of, like, feeling like we’re progressing, is when the research team started using the terminology of data products, right? Like, I thought that was great. They’re talking in terms of data products. They’re prioritizing in terms of data products.

I think that the biggest challenge we had, and that I think we still have, and I’m sure other companies deal with this too, is when we say data products [laugh], it means a lot of different things to a lot of different people, and it’s really hard to articulate specifically in a way that many people understand what a data product is, right? Is it a view in a database? Is it a table? Is it a query? I think people struggle consistently in the teams I’ve managed.

You know, over 18 months, we’re talking about data products, and I’m realizing, oh my God, we are all talking about the same thing—I mean different things. We’re all talking about it in different terms, and nobody’s actually potentially delivering data products. Or one team is, and some others. I think, kind of the definition or understanding of what a data product is, is going to be an ongoing challenge.

Brian: Yeah, yeah. We talk about this in the DPLC community, just this whole, like, debate space about what this term means. I’m almost kind of done with it. It’s like, that’s 2023.

Jeremy: Right, [laugh] exactly.

Brian: I’m just kind of tired of it because it’s really not what product management is about. Ironically, product is really about the benefits of the product. It’s not about the output, it’s about the outcome, and if we just talk more about the benefits, that’s really the point. It just it helps us to clarify here that we’re not talking about any digital deliverable of any kind; we’re really talking about things that foundationally use data.

Jeremy: Right.

Brian: So, it’s almost not that important. And I think most people listening to the podcast know, probably—to this podcast—know what you’re talking about already here. I’m curious about the moment when you said something about maybe one of these bench scientists, or an end-user of your solution, started using your language. And I agree, that’s always a powerful moment. It is with design as well, when you hear, like, an engineer start talking about, “Well, that’s not what I saw in the research that we did when we interviewed that guy. That’s not what the scientist is doing. That is not what his workflow. That’s—and he did not say that.” When they start using the language of a skill area, particularly in user experience research or in product management research. So, tell me about that moment was there, like, a specific one you can remember in a meeting or something where you’re [jogging 00:13:08] the floor. Like—

Jeremy: Yeah. I just remember, like, I think the team invited me to attend a workshop, and it was kind of a cross-functional workshop, and I was just sitting, I think I was sitting in the back row, and they were kind of having conversation, and somebody from the non-IT person, somebody—the user—just talked about, I think it was in the context of, like, well, we need to make sure we have the right data products, or prioritize the right data products to something like that. And I just thought, “Oh, man, that’s so cool.” Right? Like, one is, it’s, you know, using language, but two, that’s great. They’re talking about prioritization, or making sure we have the right ones. That was the moment.

Brian: Yeah, yeah. How do you guys work with designers? Like, one of the ironies I hear a lot, there’s a lot of discussion about data product management right now in the space and data products. A lot of places, when I look at just—I’m not looking for a job [laugh] in data product management, but when I see job descriptions that come across my desk, mostly sounds like a technology implementation role with a little bit of business-facing stuff, and they’re still really talking about technology deliverables. And so, they’re not really adopting product because a core part of product is technology, design, and business.

Like, the product management function, you need to have at least three of those, if not also the fourth leg, which to me, is data science, particularly in the AI piece. You need all four of those skills if you want to consistently deliver benefits there. So, you’re kind of an exception if you have user experience design in your team, in my experience, at least. I’m curious, was that there, when you came in, how did they decide that was worth funding, that we should spend money on that to make the experience for a bench scientist easier? That could sound like fluff, hand-wavy to some people. Like, why does it need to be easy for them?

Jeremy: Yeah so, no different in the current team I have versus historically, it’s we don’t have actual designers as employees on the team, through, usually consulting contracts project deliverable, we’ll bring them in. But on the team side, a similar as conversation we’re having about product management, we’re trying to skill up our data product analysts to actually own the design piece as well. Not maybe all of it, not to be, you know, go out and get degrees in human-centered design, but to be able to at least understand how to facilitate the right questions, conversations, that sort of thing. But largely what we do is through project funding, is we’ll bring people in.

And I think it’s still a mixed bag. I think when we’re talking about, specifically about data products, you know, there’s some type of data asset that has value to an end-user, versus separating that out from, like, a report or an algorithm, right, the data itself. That is harder, I think, from a UX perspective, to facilitate. I think it’s even hard for UX people to really understand how to think about an actual, like, a data product—again, not a report, not an algorithm, but just the data itself—I think it’s hard for people to conceptualize, how do we do design around that? And it’s one of the areas I think I’ve seen the biggest challenges, and I think some of the areas we’ve learned the most, and also failed quite a bit.

But usually with product, even a report, you iterate, right? Like, you can get better at it over time. You can deploy something three months later. You know, you’re building your backlog of features, getting user feedback. If you build a data product, and it’s not accurate, right, and people are getting results that they either are incomplete, they don’t agree with, the metadata isn’t proper, it’s not interoperable in a certain way that they expect which wasn’t part of the original design, people will abandon it really, really quickly—

Brian: Oh, of course.

Jeremy: —right? And so, that’s one of the areas that I think that—just a learning and I think, you know, an area that we’ve struggled with is to make sure we have the right design going into the initial launch of that product in a way that it’s hitting the viability, feasibility, desirability side with room for improvement, but that’s been a hard mark for us to hit.

Brian: Is it the level of effort that’s required to do it? Is it a cultural thing? Like, what’s the blocker to making sure the first increment, it is small value. It’s not a partial product, it’s not a half-assed product, it’s half of a product.

Jeremy: Right [laugh].

Brian: It’s halfway there with half the value. It’s not half-assed; all the way complete, but half-assed.

Jeremy: Right, right. Yeah.

Brian: You know. So, what’s that about? Tell me why that’s challenging.

Jeremy: I think one of the challenges is that some data products that you build, somebody’s very excited about it, and you work with an individual, and it solves a specific problem for that person, you know, six months ago. But then somebody else comes along and wants to use that data product, and it doesn’t meet any of their needs, even though the implication of that definition of that data product should provide value to that person downstream. If this makes—I don’t know if this is making sense. Like we’re—

Brian: Wait, they think that it should work for them? Or you and your team think that it should work for them?

Jeremy: We thought when we were building it, we’re covering what we needed to for interoperability, and all that kind of stuff, but—

Brian: Future-proofing it.

Jeremy: —when we turn around, it was like, oh man, we’re missing all this metadata. Why did you miss this metadata? You know? Oh, great. Now, it’s going to take me all of my time as a user to provide you insight to what metadata we need to improve that data product. It’s that type of thing. Which I do think goes into the product mindset of, hey man, we delivered something, it had initial value, now we need to iterate on it, versus it was way off, right? Total fail. So, trying to balance that, so we’re trying now to figure out how to do better, you know, fortification of our data products initially with ongoing, kind of, enhancements of them. But it has been really hard.

Brian: I mean, it’s trade-offs, right? Because race, race, race, race, race, speed, speed, speed, speed, speed. Well, speed usually means the first thing to go is talking to customers and doing research because that’s slow, and you can’t see that you’re building anything, and working code and prototypes and outputs looks like progress; talking to people does not—or doing analysis of customer interviews does not look like progress.

Jeremy: True. Yeah, yeah.

Brian: So, it’s like you’re going to pay for it either later by redoing it the second time. And the joke, like, in a lot of consulting is, like, there’s always money to do it a second time. There’s just never money to do it right the first time. So, for listening—I think, if you’re listening like this, is that trade-off between, do you want to go fast or do you want to go right with future-proofing built in? If I hear this, “Oh, we need to scale,” I mean, I’m talking to a training client right now, and they’re talking about, we want to teach our team to be able to design data products at scale for the enterprise, which I think is coded language for, “We want them to work for other people who are doing similar tasks and workflows, and not a one-off for everybody.”

But then the challenge is that they’re being told, or they are encouraging speed as, like, the number one criteria. I don’t know how you reconcile that because speed means you’re literally guessing. You have to fill in the blanks by guessing. Or you just ignore that whole segment of the audience, and you make assumptions that everyone doing oncology also does it the same way that in vitro does it. You just have to make that decision, or you pretend like it doesn’t exist. Like, I don’t [laugh]—

Jeremy: Brian, that’s actually a great point because that was our approach, is get products into users’ hands as quickly as possible.

Brian: Right, right.

Jeremy: Right? And then we learned from that. Then now it’s realizing, oh, you know, blank, maybe we should have taken a little bit more time, or maybe we should have done something differently to just follow that instead of trying to maximize the number of data products we create and get into users’ hands, maybe we should have focused on a smaller set, and really followed them through closely. I don’t know. We’re still trying to figure that out. And this is, you know, it’s not unique to my current team. It’s something I’ve seen in the past. Because you’re right, speed matters, but you have to make that trade-off. And that balance is tough. Clearly we haven’t nailed it, but we’re learning.

Brian: Oh yeah, and I’m sure, like, nailing is probably an impossible target.

Jeremy: Right, true. Yeah.

Brian: It’s really just like, there’s somewhere on this dial between speed, and future-proofing, and scalability, where you have to dial it in. And it might be different for different kinds of solutions that you’re working on because the deviation in science might be huge, but the deviation and, like, bring the drug to market or go through the trials process, well, trials is fairly predictable, regardless of the kind of drugs, so there’s a place where we don’t maybe need to do as much user research because the way we do the trials is roughly the same from a big process standpoint, whereas the science is, like, Wild West, maybe, you know, for each different domain area. I mean, I’m just riffing here. I’m not trying to solve it, but it’s not an easy factual answer. There’s just—it’s seven. The answer is seven.

Jeremy: The answer is seven, [laugh] or 42 right, right?

Brian: Right [laugh].

Jeremy: Even in the trials process, the process might be similar, but each trial could be run differently, and then you’re dealing with regulatory, you know, issue challenges, and so the precision of that data product or—and the reuse, right? That is another thing is reuse might not be realistic in some instances because the data is highly regulated.

Brian: Yeah, yeah. Tell me about how the… it doesn’t matter if they’re agencies, contractors, whatever. The point about the UX, thing I was talking about earlier, is more that there is an investment in that, whether it’s in training your data product analysts, whether it’s hiring outside resource or employees. The point is, like, someone’s spending money on it because they feel like there’s a benefit to that. I’m just curious how you go about justifying something that might—and maybe it doesn’t. Maybe it doesn’t seem, like, unnecessary, or is that really where we should be spending our dollars?

I think a lot of data teams, it is a place where it doesn’t seem really necessary to have that. Shouldn’t the product manager be doing that entirely? Like, it’s just dashboards, or it’s just a tool. I mean, you showed me this—maybe you can link this up to the talk that you gave about this generative AI prototype that you—actually, I don’t know if it was a prototype or not; it might have been in production, but maybe tell us about that. Like, how did you decide that the UX needed to have some attention given to it deliberately in order for it to bring value?

Jeremy: I think once you see what good looks like, and you see the impact it could have, you realize how important it is for the UX. And I’ll go back to my days at the Bill and Melinda Gates Foundation, where we had such an exceptional, human-centered design team of UX folks that were embedded within the data work that we’re doing, and made a huge difference with our end-users, but also the quality of our product. And after seeing that, I’ve just become passionate that it needs to be part of the mix of AI data and analytics delivery. And without it, you know, you end up with maybe a lot of output, but not a lot of outcome that you’re trying to get to, to something you said earlier, and so I’ve just always been, since that time, I’m just convinced it needs to be part of all of our delivery. And so, it’s just been something I’ve incorporated into projects.

It’s not always easy, too. Like, I think that UX design and combination of AI development or data science work is a magical partnership, but they often don’t know how to work with each other, right? And so, that’s been a challenge. But I think investing in that has just been critical to us. And even though we’ve had struggles like I was sharing with some of the data products we’ve created, I think we’ve also done a good job of actually building in user journeys, really understanding the experience and impact that we want to have. And on the prototype we shared, which is more or less in production right now, it was very much driven by user experience and trying to get information, knowledge in the hands of the research organization to be able to understand some portfolio types of decisions that have been made in the past. And it’s been really successful.

Brian: That’s great. I want to get into that, but let’s go back for a second to your time at the Gates Foundation. Can you explain for listeners here, who maybe haven’t worked with design or user experience as a discipline, but maybe they are feeling some of the challenges, like, we’re pushing stuff out the door all the time, no one’s using our stuff? It’s too hard, they don’t care. Apathy.

What was the before and after like because it sounds like there was almost a moment for you where there was a period where you’re like, this is just how I make bread now. I just always add salt. You just, I don’t know how to make bread without salt. You just do it. But there was a point where maybe something switched at the Gates Foundation. What happened there? Was there a moment? Or [laugh]—

Jeremy: Yeah, for me, it was—the before is, you’re having technology conversations with your business users, and you’re focused on the technology output, and just build that darn report.

Brian: Right, whatever their deliverable was, yeah.

Jeremy: Right. Post human-centered design, it was talking about outcomes. It was talking about value. It was talking about adoption. And by doing that, we actually, I feel like we lowered the amount of reports—because this is very much about reporting back then, the traditional business intelligence—lowered the number of reports that we just needed to churn out and really focused on impact. And it changed the conversation with our end-users.

And that was, you know, fundamentally, such a shift for us, and just having that resource, that expertise in the team, it just transformed, kind of, the conversation, and focused—like, I felt like our quality went up. I felt like our output went down, but our impact went up, and our quality went up. And then people loved the output, right? They loved the tools. And that wasn’t what was happening before.

You know, there’s a reason people are always asking for new reports that look eerily like the ones that were already produced, right? It’s because they weren’t really hitting the mark. And I just felt like with that team, they just really changed the game on our delivery. And it built a lot of confidence, too. And at that point, it was going from a team that had really low credibility to a team that was believed to be successful and a good partner. I credit a lot of that to the human-centered design team.

Brian: Yeah, yeah. So, fast forwarding to Pfizer, then—connect this if you want, if it’s relevant to this prototype that you showed at CDOIQ—tell me a bit about the dance between data science and UX working together. Was that a rough dance for a while on this particular project. And, like, what does that look like, such that maybe someone listening to this can understand, oh, if I was to, like, leverage UX, and I want to get some of that mojo that you got at the Gates Foundation, that sounds good. Like, my team’s status is going up, we’re showing value, the customers are happy, they’re using the stuff. That sounds good, but what am I paying besides money? What’s the cost look like? What’s the dance going to look like there that I need to know about?

Jeremy: I think on things like the prototype you saw, which is, like, a GenAI application user interface, I think it’s a little clearer how UX fits in that because you’re building something that users are interacting with, you’re building an application, essentially. When you’re thinking about automation through machine learning or building algorithms to, say, find adverse events in your clinical trials, that’s where it’s, I think, becomes a harder dance between kind of data scientists or analysts and human-centered design because I think a lot of times this data scientists feel like, oh, this is—you’re just going to go ask them questions about what they want to do. I can do that. Like, I’ll just go, you know, find out what needs to happen, and then I’ll just build something. There’s, I think, a lack of appreciation and a diminishment of—or a lack of understanding, maybe—of what UX could truly do.

And it’s because I think a lot of people think of it as just that UI that gets built, right, and I think that’s where we’re stuck as an industry, in some sense, where human-centered design is, you know, just that empathy-driven understanding of users’ experience, their work, their workflow, really understanding what the challenges they have. And I just don’t think there’s an appreciation, really, of that skill set. So, I think it’s a dance that we still do. Like, trying to embed a user experience person in a data science team is still hard, unless you have the leadership that’s just going to stick with it and prove it out. And I think on projects, it’s a little bit easier.

But we’ve still seen challenges. What I would love to see, which I haven’t been able to accomplish is, within a combined team, having within my practice human-centered design team, as well as data science, and then just embed that human-centered design skill set within each squad. That would be really powerful. I just haven’t been able to—don’t have the funding for it. And also it’s hard for, I think, the technical engineers, the data scientists, to really understand the value there.

Brian: Do you think it’s because they see it as more of a tax on—

Jeremy: Yeah.

Brian: —it’s just slowing us down. It—yeah—usually—

Jeremy: “Oh, I could do that.”

Brian: —slowing it down is the thing you hear. Yeah [laugh].

Jeremy: Yeah, yeah. “I can do that. Give me a half an hour with a user, I’ll figure it out,” right?

Brian: Yeah, yeah. It’s the, I don’t know what I don’t know about it because you can’t see it, like [laugh]—

Jeremy: Right. You can’t see it.

Brian: Right. You don’t see it because you don’t know what you don’t know about it, which is okay. And I’m actually a big fan of, like, oh, a data scientist wants to go talk to—great. Like, where it used to, you know, used to be kind of like, no, no, no, no, that’s what we do. Don’t touch that, you know? Now, it’s like, please go do that. Let’s do it together. Let’s learn how to do it better through education. And you can teach some of these skills that I think you need to really want to be able to do it.

Jeremy: Yeah, right.

Brian: I know other leaders that say, like, wants not enough. Like, there actually is, like, a skill, which, like, you either have it or you don’t [laugh]. I happen to know someone that both of us know on this call that has changed his perspective on that and really feels like it’s not enough to just be willing. Which I used to think is it is enough. I still kind of do tend to think that it’s possible.

Jeremy: I think it’s a matter of degrees, right? Like, I think that the best human-centered designers are like, there is a capability intrinsic. Like, just they have a skill set that I don’t think you can just train anybody for, even if they’re willing. But I do think that there are some basics, right? I think they’re basic tools, if somebody’s willing, it will improve their work and improve the outcome, but I wouldn’t say they’re user experience designers, you know?

Brian: Yeah. It’s talent, and skill, right? I believe talent does exist. It’s an accelerant for skill. It’s how fast you can acquire new skills and apply it. That’s kind of what talent is. And I agree, there’s talent in every field. Like, we’ve all met—I’m not an engineer, but it’s like, certain people you talk to, it’s just like, okay, this person’s, like, on another le—I’m not even an engineer, and I can tell that other engineers think that guy really know—or gal—knows exactly what she’s talking about. I think that exists and that’s okay.

We don’t need everyone to be like that. There’s definitely, like, learning applicable skill can still accelerate the work that we’re doing and all that. What did we look at on the screen? Maybe you could tell us about this application there. And did you have to have design and data science working together on this generative AI application? How did that go? Can you tell us what that looked like?

Jeremy: Yeah, we—it was at Seagen, and we had a lot of unstructured data that covered, you know, the 25 years of research that Seagen had done, but all that data was just locked in on a—stuck on a shared drive, that sort of thing. And so, in order to find out a decision made on a certain molecule ten years ago, you had to know somebody. But that’s really rich information for people to understand, and so that was the problem we were trying to solve, is, how do we create—get the knowledge available to the Seagen research organization? And so, yeah, there was a user experience person involved in really understanding the needs, the use cases, the types of questions people would want to ask. I’m not sure if we actually had a specific UX designer at that point when we started. I think we did from our consulting partner.

And then we had a product manager, we had some data scientists, software developers that worked together to kind of build this out and iterate on it over time. You know, it was really successful. I mean, to be able to use a large language model, and, you know, a RAG approach to be able to answer questions that we could have never answered in the past unless you knew somebody and that person had time to do the dumpster dive into the shared file system to answer these questions, it wouldn’t have happened. And so yeah, I think that was relatively successful. Something we’re carrying forward today into Pfizer.

Brian: Is there a way that you’d know, even if it’s just anecdotally, but how did you know it was successful? Like, when you look back, it’s like, I’m glad I put those resources on that because I can see X. How did you know it was worth it?

Jeremy: I think, kind of, voice of the customer, user feedback, adoption. Are people excited about it? Is there value? Are we hearing positive things? Do they want us to continue? That’s really how I’ve been judging success. Is it saving people time, and do they want to continue to use it? They want to continue to invest in it. They want to take their time as end-users, to help with testing, helping to refine it.

Those are the indicators. Because, you know, we’re not generating revenue or anything like that, and so it’s really kind of, what’s the adoption look like? Are people excited about it? Are they telling friends? Do they want more? When I hear that the ten people that were the initial users are happy and that they think it should be rolled out to the whole broader audience, I think that’s a good sign. The broader organization.

Brian: Yeah, yeah. If someone, like, wrote you a check for, I don’t know, ten extra million dollars for 2025, what problem would you throw that out on your team right now?

Jeremy: Oh, boy, [sigh]. I would really double down on self-service platform for an end-to-end kind of modeled through data self-service experience. And then with that, I’d be investing, as well as in that platform, curating data products for reusability across the organization, across R&D. That’s what I do. I think most companies under-invest in the data they need for AI, but you can’t just build the data—it’s not just building a data warehouse; you really have to build that experience out and build that self-service experience out. From being able to find that data, to model that data, to access that data, you have to build out the end-to-end flow, and I haven’t seen that done well.

Brian: Got it. Sounds like too many ad hoc, like, small project requests still coming in, and, like, oh, I wish they could just go to get that themselves without needing to come through us? That kind of thing?

Jeremy: Yeah, it’s still like, oh, just curate all this data. That’s one kind of side of it, which has historically never really worked for us as an industry. I mean, that’s your legacy data warehouse type of mindset. So, I kind of think that’s one thing. Or it’s just a one-off to solve this particular problem for this one team.

But I just still like this whole self-service experience, like, true self-service, where, I don’t know if it’s a great analogy, but like the Amazon Marketplace, I think a lot of people have thought about this, or I think about this, the Amazon Marketplace internally to an organization of models and data. And how can you have a GenAI interface to that where I’m asking a question about maybe a portfolio decision we made, and then it shares with me the data products that help to answer that, so then I can maybe take those products into a compute area where I can do my own analysis, build another model, and then that model and that data feeds back into the marketplace so that other people are learning. Because there’s a lot of across, you know, it’s not just unique to Pharma, there’s a lot of, kind of, modeling, and analysis, and data curation, and pipeline development that happens very much in silos, and so there’s a lot of redundancy. And so, part of the challenge that I would love to solve is, how do you learn and share across an organization with insights models and data that would help us be more efficient with our time and focus more on insights versus the amount of time people take to just pull data together and to start from scratch.

Brian: Understood, understood. Jeremy, this has been a great chat with you. I want to figure out where people can get in touch with you, but just, I wanted to kind of give you the closing last word. Any advice for people that are adopting product management or working with designers in-house on these enterprise data teams. Any, “Don’t do this again. I did it this way,” [laugh]—

Jeremy: [laugh]. Yeah.

Brian: Whatever. Just some, if you’re talking to your old self from ten years ago, maybe, what would you advise Jeremy of minus ten years?

Jeremy: Oh, man. Let me see if I can—I’m going to try to think of three things. One is that change within a team and bring in new team together takes time, and you really have to invest in the culture of your team. It’s not something you can just throw a bunch of people together with the right titles, even, and the right skill set, and expect that it’s going to work. It can take six months to a year to really hit that efficiency, and I didn’t appreciate that earlier in my career. It was more of a, hey, if we just put all these great people together, you know, it’s going to work out. And it’s just not the way it works.

I think the other is UX, right, really think about the experience in everything you’re doing, right? If it’s a data product you’re building, think about that experience, and the value, and the outcome, and try to anticipate how it’s going to evolve over time.

And maybe the last thing is really look internally to find some people that could play product manager roles or learn some skill set around user experience. There are people in your organizations that are probably, you know, diamonds in the rough that could really help accelerate this kind of shift to product and also feel like re-energizing their career. They those people do exist. I think it’s worth taking the time within your organization and the risk to invest in them.

Brian: Cool. Jeremy, this great advice. Thank you so much for spending time chatting with me. Where can people—can they get in touch with you? LinkedIn—

Jeremy: Yeah, please.

Brian: —or somewhere public, uh—

Jeremy: Yeah, I’m on LinkedIn, so feel free to connect with me on LinkedIn. Send me a message. Happy to chat.

Brian: Okay, cool. Well, excellent. I look forward to sharing all of this recording with the audience, and it was great to finally get a chance to chat with you. So, thank you.

Jeremy: Yeah, you so Brian, thanks for the opportunity. This is fun.