Bill Saltmarsh joins me to discuss where a modern CDO gets the inspiration to “operate in the producty way” in his domain, which is healthcare. Now Vice President of Enterprise Data and Transformation and the Chief Data Officer at Children’s Mercy Kansas City, his early days as an analyst revealed a gap between what stakeholders asked for vs. the outcomes they sought. This convinced him that data teams need to pause, ask better questions, and prioritize meaningful outcomes over quickly churning out dashboards and reports.
Bill and I discuss how a producty mindset can be embedded across an organization. He also talks about why data leaders must set firm expectations. We explore the personal and cultural shifts needed for analysts and data scientists to embrace design, facilitation, and deeper discovery, even when it initially seems to slow things down. We also examine how to define value and ROI in healthcare, where a data team's impact is often indirect.
By tying data efforts to organizational OKRs and investing in governance, strong data foundations, and data literacy, he argues that analytics, data, and AI can drive better decisions, enhance patient care, and create durable organizational value.
Highlights/ Skip to:
- What led Bill Saltmarsh to run his team at Children’s Mercy “the producty way” (1:42)
- The kinds of environments Bill worked in prior that influenced his current management philosophy (4:36)
- Why data teams shouldn’t be report factories (6:37)
- Setting the standard at the leadership level vs the everyday work (10:53)
- How Bill is skilling and hiring for non-technical skills (i.e. product, design, etc) (13:51)
- Patterns that data professionals go through to know if they’re guiding stakeholders correctly (20:54)
- The point when Bill has to think about the financial side of the hospital (26:30)
- How Bill thinks about measuring the data team’s contributions to the hospital’s success (30:28)
- Bill’s philosophy on generative AI (36:00)
Links
Transcript:
Bill: Hi, this is Bill Saltmarsh, vice President of Enterprise Data and Transformation, and Chief Data Officer at Children's Mercy, Kansas City, and you're listening to Experiencing Data with Brian T. O'Neill.
Announcer: You are now experiencing data with Brian O'Neill. Experiencing data explores how product managers, analytics leaders, data scientists, and executives are looking at design and user experience as a way to make their custom enterprise data products and analytics applications more useful, usable, and valuable.
And now here's your host, the founder and principle of designing for analytics. Brian O'Neill.
Brian: Welcome back to Experiencing Data. This is Brian T. O’Neill. Today I have the Chief Data Officer from Children’s Mercy in Kansas City, Bill Saltmarsh with me. How’s it going, Bill?
Bill: Going really well. Thanks, Brian. How are you doing?
Brian: I’m doing great. I was talking to one of your staff, Anna, who I know you guys have worked with in the past—or I actually didn’t know that until she told me—and she was talking about running this kind of product operating model at Children’s Mercy in Kansas City. And I’m always interested in these origin stories and where this starts. She said, “Well, Bill decided he wanted to, kind of, run the team this way.” And I’m always, like, people don’t just wake up and decide to do this one day.
So, I wanted to get your origin story and kind of understand where you’re coming from with this because it’s not the normal way that analytics and data science teams are typically doing the work of building these decision support solutions and analytics solutions and things of this nature using data to streamline operations, et cetera. So, welcome to this show. Tell me a little bit about the origin story. Like, where did this start [laugh] ?
Bill: Yeah, I love this. This makes me think of challenging times, but really hopeful, kind of like, discovery moments for me. I think about, I mean, I cut my teeth as a professional as a data analyst and as an analytics engineer before it was really a real thing. So, one of the things that I experienced in those roles as individual contributors is, I just wanted the ability, the systematic and consistent ability to connect my work, my tasks to something that’s impactful and meaningful organizationally. And so, I always felt like there was this gap between, like, what people requested and what people actually used, and being on the front line of like that dynamic was enlightening because it allowed me to, sort of like, experiment with how can I get a better version of the original request and kind of push back a little bit and kind of interrogate, like, the problem space a little bit before I start working on this thing.
And lo and behold, I started discovering that this is a common viewpoint in terms of, like, analysts who, kind of like, think critically about the work that they do and try to connect it to something bigger. I started finding people that had similar sensibilities about the work. And then it became known to me that there was, like, a data-as-a-product mindset and a developing framework in an emerging way of practicing this work that was much more meaningful to me. And so, from an early time in my career, I always was attracted to these sets of concepts of data as a product. And then as it became a leader of data functions and then enterprise data and analytics functions and then AI strategy and those types of things, kind of bigger picture organizational constructs, it allowed this concept of data as a product became a framework by which I could help my team members understand that there’s a better way of utilizing their skills and their experiences and their expertise to drive value for their stakeholders and their users and their colleagues, by just empowering them to kind of push back on what was requested of them, give them a little bit more leeway and latitude to, kind of, use critical thought and understand the problem space before they start developing solutions, to become much more of a contributor, a meaningful contributor, and an impactful contributor rather than a producer of widgets, dashboards, data sets, data models.
All of that stuff is important but a lot of it misses the mark if you don’t interrogate the problem itself that we’re trying to solve. And so, became this, again from a leadership perspective, it’s like this ability to kind of connect the work to a higher degree of autonomy and then create a better set of valuable assets accordingly, which to me is, I mean, that’s a big manifestation of my leadership principles, is connecting my team’s work to the most impactful outcomes that we can produce.
Brian: Were any of your past employers or positions that you were in, were you in a climate where this work was happening or were you doing this work individually and then when you reached the position you’re in now, you decided to, like, scale this downwards and w— [laugh] and wide-words—wide-words [laugh] ? Whatever the word is for that; you know what I’m saying—to disseminate this way of thinking. But I’m just curious if this is kind of like I’m working in isolation but I think this is the right way to do it, and then you’re proselytizing? Or were you in an environment where others were doing this? Like, tell me a little more about that.
Bill: So, I was the first, sort of, data team member hired at an advanced startup. And so, I was in this, like, position of, I had this skill and capability that a lot of people needed because they need access to data to drive you know project decisions or finance decisions or whatever the things were, right? But I had this very finite capacity. And so, I was at this crossroads as a contributor. Like, do I just kind of churn and burn, like, whatever I can do just to, like, get it out the door or do I just try to figure out a way of maximizing the utility of my time and my resources? What can I do to make the most possible impact?
And so, that’s kind of how it started with me is, like, I kind of—based on a lot of things about me and my sensibilities about work and that type of thing, I gravitated more towards a ladder which is, like, how can I just maximize my contributions? And so, that became, like, a set of practices. And again, like, as I, kind of like, employed these practices and learned that I’m not alone in this space and in the data and analytics world, they became refined over time and then I met people that are really much more adept at this whole thing, too.
Brian: [laugh] . Yeah I want to ask about how you scale—how you’re kind of scaling this across the team and whether or not everybody adjusts how they work, or you have, like, a small group of product managers and you leave some of the implementation people kind of in their original working mode. I want to talk about that in a second but before I do that, I know one of the challenges that, like, some, we call them data product managers where they have the title or not, this pushing back, this asking why this way? Why not this way? What decisions are you trying to make? That can introduce some flack.
Like, that’s not your job. Why does it matter? What difference does it make what the value is? I need this to get my job done, et cetera, et cetera, et cetera. Can you talk to me about, is that a cultural problem that needs to be dealt with management or is that a necessary stepping stone maturity that, you know, all people, like, yourself you have to cross over that, like, you know, like, the knight crossing the—Indiana Jones where you have to walk across the invisible bridge to get to the other side. Like, [laugh] tell me about that [laugh] .
Bill: I think, I mean there’s two things that come to mind with that. One is—and broadly speaking, this is, like, industry agnostic—if you’re leading an enterprise data function and you believe that data as a product as a philosophy and a framework and a set of behaviors is the right way to do this, then you do have to have the consistent dialog with other stakeholders and leaders of the organization and say, “This is an expectation. When you reach out to us, we’re not just going to take your request as you threw it over the wall and then do something and then throw it back over the wall and hope you catch it. It’s not the right way for us to work, it doesn’t make sure that we’re hitting the mark and giving you the value that you need. This is not about a dashboard. This is not about a report. This is not about a model. This is about what are you trying to do with the information that we’re going to give to you in a curated form.”
Right? And so, when you start to get into the conversation about, like, why do you even have a data team? Like, it’s not to churn out reports. Like, it’s about providing information to you at the right time in the right context so you can make better decisions. When you kind of change the conversation in that way, it gives your team an allowance, a buffer so to speak, so they can actually do this pushback, this perceived pushback or this interrogation of the underlying problem or challenge or opportunity from which a request may come to the group, right?
And so, it does, like, from a relationship management and relationship-building standpoint, it does require a leader like myself to kind of like, “Hey let’s talk about how this works and why it’s working this way and it’s for your benefit.” Right? And then the other piece of this that comes to mind, Brian, is, like, when you think about all of this work in the context of health care, right, and specifically as a health care provider in the US, so I guess the US is—that’s where I am so I kind of speak to that from that perspective—I mean care delivery is everything for health care, right? And so, when you think about when you’re a data team member at a hospital or a large health care system, the biggest, most important objective all the time is to solve the problems and support the folks that are providing care to patients. And the idea of what to do with data and how to maximize the utility of the data team and produce the best, most ambitious analytical endeavors and support those, like that is a very secondary concern, historically.
And so, that’s an interesting dynamic in and of itself because the clinicians are nurses, are doctors, are researchers, the folks that are on the front lines of delivering care to our patients and their families. Like, they don’t want to have this pushback. Like, they’re worried about bigger problems than you, like, building the right data model to support a subsequent, like, dashboard or predictive model. And so, it takes some maneuvering and definitely some relationship-building and trust-building so that they know that when you do say, “Well, like, you said you need this report, like, and it requires these things but, like, when do you need it, and like, what are you trying to do with that? Is there a better way for us to kind of achieve this goal for you? Can we automate this in some way so that it’s not, like, a reinvention of the wheel every six months or whenever the thing requires it?” It does require some massaging of that, like, dynamic and making sure that people trust you specifically on the clinical side.
Brian: Yeah. How much of that is the individual data product managers—or whatever your title for the people that kind of oversee these different product initiatives—how much is getting this right their job doing that in a day-to-day basis versus you coming in at a senior level and setting some expectations about the throwing-it-over-the-wall model and how we’re not going to do that? If you had to trade in one of those [laugh] tell me about the relative importance of, like, setting the standard at the leadership and management level about we’re not a data drive-through, so you don’t order and then it comes out of the window, no matter what it is, we’re going to give it to you in a bag and you’re going to drive away. How important is that relative to the individuals doing the pushback work and the day-to-day?
Bill: Yeah, yeah. Well, I think it’s probably more important to think about the ladder but I don’t think it’s—it’s not either/or. And I don’t mean to—
Brian: It’s both.
Bill: —dodge the question because it is both. I mean, like, if you kind of really zoom out and you think about, like, what I do for a living, I’m a chief data officer. Like, the statistics for chief data officers is not good, like, if you think about, like, their tenure and, like, the perception of them at organizations, the turnover, you know? And I think I mean I have a set of theories about why that is but my unifying theory is, I think that there are varying definitions of success for a data function at an organization, at every function of any scale, right? And the thing about it is, like, every definition is right to some degree.
And so, it takes a lot of communication and a lot of clarity and clarifying communication with folks that have differing definition of success and transparency has to be by default, accordingly. And so, like, there’s this, like—that leadership component is important because it has to set the stage for the behavior that comes out of the team, but the behavior is, like, I think it’s a personally—personally I think it’s a way better way of working as an individual contributor as an analyst, as a data scientist, as a data product manager, as a data engineer to be able to take a request and say does this align with, like, an objective and key result or a strategic imperative and how does it align and what is my role in contributing, right? I think that’s a—that’s just a better job, from my perspective, to have than to just, okay there’s the next ticket that came in and let’s figure out who has to do work now. So, I think it’s both. But I don’t know that it’s—even if I wasn’t having, like, the relationship-building and kind of build the relationships and helping our leaders and our colleagues understand the way that the group is working, it’s still more important—it’s sort of like if we’re talking about a zero-sum thing—it’s still more important that our data team members approach the work in, I think just, like, an objectively better way.
Brian: Yeah. How are you skilling or hiring for these non-technical skills? Because you talked about, “I think it’s a better way to work,” and all this, and I’m in fierce agreement over here, cheering inside my head, and then the other—after talking to you know 150 people on this show—I know that’s not how a lot of analysts and data scientists are wired out of the box. This is not default wiring. Default wiring is, I was paid to develop these complicated models and to do very complicated math and to build these things that no one else can do, and we’re going to spend 80 percent of the time futzing around, talking about what we’re going to make, and then the 20 percent of the time making it? Like, this is not default wiring.
Now, I’m stereotyping a whole class of people that are really sharp but I just wanted to throw that at you and understand, do you believe in rewiring or do you, like, no we’re going to change how we hire and we’re going to hire for different skills or nope everyone can be upskilled. Bring them in if they have the tech chops and then we will wire in these new skills. Talk to me about that.
Bill: It’s interesting. I mean the folks that you have on your team are incentivized for a variety of reasons. They’re incentivized by, like, with the skill set that they bring to the organization and the work that they do and what has previously provided this sense of value for them as an individual. Like, what makes them good at their job? Like, they have this coming in. So that’s, like, an implicit sort of like set of motivations.
But, like, you’re also motivated by what your boss tells you, right? And so, I think this is, like, somebody in my position, who’s again sort of thinking about this as a philosophy that I want to manifest in the data team that I lead, it’s about changing the roles and responsibilities of our leaders of our data team, of our data functions. For a long time, particularly in what I would say is, like, a little bit behind the curve industries, like, health care, the leaders of data functions or analytics functions have been seen as, you know, you have to do everything for this team. You have to gather the right work, you have to mentor these folks, you have to skill them up, you have to continue coach them, you have to contribute as an individual contributor still, you have to, like, be hands-on and all these things, and from my perspective, if you think about data as a product, as a function, and you really want to move in this way as a as an enterprise data function, you have to change the job of the leaders of those data functions.
Like, it now becomes imperative that they focus on the development of the skills and the capabilities and the capacity of their team and not about, like, managing intake and prioritization and, like, telling people what to do at a given time. You have to make them better at the work itself and not tell them what work to do, right? And so, it’s not just a switch you flip. There are moving parts if you—kind of, if you want to move in the direction of data as a product and have a data product function as part of the enterprise team.
Brian: Yeah, you described a bunch of project management stuff there and that’s not product management. I remember a time in my career when I was like ah it’s all the same thing [laugh] . No it’s not.
Bill: It’s not. It’s not.
Brian: It’s definitely not the same thing. So, this skilling, though, is this—how do you bring them to—your individual data leaders than these different departments you have, are you hiring people that have some kind of product skilling already or training them or how do you develop this skill, especially if they’re conceptually, like, “Okay, Bill’s smart and he has this point of view that’s maybe new to me or I’ve heard about this other teams doing that but we didn’t do it that way.” So, conceptually they’re still figuring it out and now they’re actually supposed to do this work and spread it down to their team. Tell me about how do you skill for that?
Bill: Yeah. I mean, like, you have to have a practical approach here. And so, one of the things that we’ve done is, like, try to introduce practices and behaviors that result in this different approach and ultimately a better set of outcomes and outputs. And so, some of the things that come to mind are, you can’t be doing off-the-books work, right? So, like, all of the things that are being requested of you, they should go through our intake and we should take a look at them.
Is there a previously delivered solution that already accounts for X percent of this new requested thing, right? That’s part of the data-as-a-product mindset, right? Let’s reuse components. Let’s solve complexity as minimally as possible, like, as minimal times as possible. Let’s modularize our solutions. And so, intake is a big deal for us. Like, nothing off the books. Like, I know you’ve been here for 25 years and that radiologist loves you and you know the things that they need all the time and they go directly to you not through our ticketing system, but we need to understand what you’re doing so that we can leverage all of the amazing work that you’re doing across all of the things that you do not see, right? So, it’s that type of thing.
Another thing that we talk about is having design conversations before development, right, which speaks to the heart of data as a product, right? Like, you need to think about, like, what are you trying to achieve with whatever you need to develop before you start actually writing code? Are you trying to provide specific insight at a certain juncture to a certain set of users? And if so, like, how might that look? And let’s get clear on how that might look before you start designing—I’m sorry, before you start developing, rather—because the code that you write may fundamentally change based on whether you need something delivered on a monthly basis or something that aggregates across 40 different dimensions or whatever the thing is. But getting clear on the design is really key for us. So, I mean some of, like, these principles that add up to behaviors and outputs and outcomes, that’s what we think about.
Brian: You have design-ers doing this facilitation or who does that kind of work?
Bill: Right now, it’s split across. I mean, we, as a nonprofit freestanding pediatric hospital, we don’t have all the money to hire all the people we would love to have and fulfill all the roles, just to kind of state the obvious out loud. And I work at Children’s Mercy as I alluded to earlier. And so accordingly, we have to spread those design responsibilities across our analysts, our data scientists, as well as our data product managers. When our data product managers are involved in the initial like interrogation of a potential use case or, you know, eliciting requirements, they get involved with design, but they can’t be everywhere, and so therefore, we have to democratize that skill and that capability across our analysts and our data scientists as well. And so, we try to help them and help them navigate those conversations so they can shepherd them with our stakeholders.
Brian: Got it. Is there a pattern you see when someone from a data background begins doing that work and, like, they come to you and maybe they complain about something or tell you how difficult this was, and you’re, like, “Yep, that’s first hump one, and there’s three more coming and then it’s going to be smooth sailing.” Like, is there a routine or a pattern that you see that this transformation that someone coming from more of a data background has to go through to get to the other side before this is, kind of like, a normal way of working and it’s not this imposter syndrome or I don’t know if I’m doing it right?
Like, these are some of the things that I hear with the teams that I’ve worked with that come out of data is this imp—like, it’s not as clear if you’re doing it right as writing code. Did it compile or not? Did it return the right data or not? You get these black-and-white answers all the time with that kind of work. You don’t get that with the fuzzy work of design, interviewing customers, making decisions that are six months ahead of where the build is going to be. So, any patterns there that you could see and talk about like that’s normal and here’s how we navigate it or just kind of inviting an open-ended response?
Bill: Yeah. I think that there’s a lot of muscle memory, especially specifically for folks that have done analytical type work for a long time. And I go back to, like, there’s a psychological side to this, which is immutable and non-trivial, which is, like, as a professional, as an analyst—let’s use a data analyst as an example—you have probably derived a lot of value for yourself, like, sort of self-esteem, your place in the professional world, as well as value for your organization by just solving problems as quickly as possible as they’re defined by the folks that are asking you for help. Like, there’s a lot of value in doing that, particularly if you’re at a small outfit or you’re sort of isolated and kind of like the one-stop shop for all data analytics problems and questions. Like, I can see that.
That model works to a certain degree, but when you put that person on an enterprise data function, like that muscle memory gets in the way of better work, right, because what you have to do is as a member of an enterprise data function, you have to think about, can I reuse previous work? Can my go-forward work be more useful than this particular use case? I think that’s the biggest thing is we see repetitively this reflex to, like, start the work quickly and not think about the bigger picture because of this psychological component of, like, well, this person needs the thing and I can give this thing faster if I just start working now. I think that’s one of the things I see constantly is just starting before we’re actually ready to start.
Brian: Well, it’s a feeling that if you got your code editor open, you’re pipelining the data and all this kind of stuff, it feels like progress, right? This is work I can see. Like, if someone asked me where we’re at, well, we were at zero yesterday and now we’re at this. It feels really good [laugh] .
Bill: Yeah. Yeah.
Brian: You know? But sometimes it takes—you have to slow down to speed up, you know, to make sure you’re delivering quality and not just something, just an output on a schedule. And we hear, “I’ll go faster,” and, “There’s got to be a way to do this faster,” and, “Speed wins,” and all these other kind of tropes and I’m kind of like, no, benefits win. Ship smaller benefits is okay, but the game is really in outcomes and benefits. It’s not in shipping tech outputs faster. That’s not helping anybody and it’s actually probably slowing you down over time if you keep repeating it [laugh] as you keep [happening to hold] all this legacy code and just baggage that you carry around with you, now that it’s like, oh, we built the architecture that way and it was the wrong one because we didn’t really know we were going to do with it, but we built all the plumbing, you know [laugh] —
Bill: That’s right.
Brian: This is the common engineering versus design argument, you know? And it’s not to say you can’t start both of them, but I’m always like, I just don’t understand how you build a good foundation without knowing what you’re going to do with it later. And those are usually expensive things to change, you know? [laugh] .
Bill: That’s right. That’s right. And what I would say in response to that, Brian, really quickly, what comes to mind is a lot of organizations overindex on more analyst type roles, especially when they’re growing, right, because they solve more acute problems, right? And when you focus on more of an analyst type skill set, then you don’t get, necessarily, an engineering mindset or an engineering set of output, right? And so, you end up solving complexity many times over, right, and you don’t think about foundations.
Foundation is, like, a subjective term, right? Between different data functions. You know, I think that comes to mind a lot. And as somebody who leads an enterprise data function, I think about my job is—like, if I kind of abstract it out—my job is to ensure that we work on the right things all the time. Because, like, we’re surrounded by this finitude, this finite time and resources and labor and capacity.
And how do I ensure that my group is doing the right work all the time? And then you solve that problem, but then you have to solve—you have to maximize that capacity. And that’s when it gets into, like, more of an engineering mindset versus, like, solving every problem on the surface, right? And that’s when you get into foundation building, true foundation building, platform orientation, which goes hand in hand, from my perspective, with data as a product.
Brian: Tell me a little bit about how you—this is a classic chief data officer problem—but ROI or value creation, at some point economics kick in here, even in the nonprofit space, and I’m wondering, especially, uh, like, in a hospital and a care environment like yours at what tier do you have to think about operational cost savings or whatever the goals are on the financial side? Is this really something for you to aggregate up, or you alone to be thinking about, and you’re really focused on improvement and care? Like, the people below you that are doing the day-to-day project and product to work that they’re really focused on care outcomes, but you kind of have to answer to the finance people, so to speak? Or is that everybody’s job and then the attribution problem, which is classic in this space, like, okay, we’re recommending this many nurses on these days, you know, and we can predict that minus two days before every national holiday, you want to staff up because people are drinking, and blah, blah, blah, like, you come up with some insight and it’s like, well, what’s your contribution to the care quality? How do we measure, like, what that predictive model about staffing was worth? Like, talk to me a little bit about this ROI thing that every chief data officer, I’m sure, is thinking about [laugh] .
Bill: Yeah, there is a chief data officer specific answer I could give, but what I would do is I kind of take a step back. I wear two hats at Children’s Mercy. So, I’m the chief data officer, and so by and large, I’m responsible for the enterprise data and analytics strategy and AI strategy, but I also lead our enterprise program management office and our office of change management, and so that is not technical. And that is not a… that’s not a typical chief data officer set of responsibilities, as far as I could see. And so, what that does for me is, like, it inextricably links my group to the value of the work that we do.
So I, by virtue of being able to lead the enterprise program management office and office of change management, I’m responsible for the goal-setting framework for the organization at large. So, OKR program, from the top all the way down to the frontline leaders, and that connectivity and that interconnectivity, the cascading nature and the laddering up, all of that is, like, part of a program that I help lead with my group. And it gives me a front-row seat to, like, what is the shared definition of success for everybody? Like, what could I possibly point my group to as a data leader that would produce the right output, the right outcomes at the right time to drive the right strategy and the right direction for the hospital at large, right? So, I have a very—I often say this—I think I feel like I’m the dog that caught the car, right, because for the longest time, as a chief data officer, as a leader of an enterprise data function, I’ve always wanted my group to be tied to the right work at the right time and make sure that I have visibility into that.
And now I get to see the inner workings of that and kind of build that machine and then latch on the data and analytics functions and our AI strategy to all of those articulations of success across the hospital. Which gets to your point, right? Like, when you think about how do we really move the needle for care delivery, if we’re trying to improve, like, wait times for the emergency room or trying to optimize, you know, discharge for inpatient or improve the quality of care and reduce infections across [unintelligible] different departments and units, like, I know exactly what our leaders are trying to achieve from that perspective. And then the challenge for the data function is, how do we do the right work, build the right solutions, and deploy the right products that drive those outcomes? But I get, like, maximum clarity on the outcomes that we need to shoot for, which is great.
Brian: I guess my question though is that the attribution of, like, when there’s a multidisciplinary team involved, there’s the people that are going to actually operationalize this down in the emergency room or whatever, is this an internal debate and it’s just one you have to swallow or you kind of like, no, we don’t worry about attribution of the value that we created, you know, for the hospital, et cetera. I just, I know this is a lingering problem for a lot of data teams is that, especially when it’s kind of ephemeral, like, decision-support information, like, what’s that worth to know how to staff better or when to—whatever, whatever the use case may be. How do you think about that?
Bill: So, the ROI specifically on the output of the data team? Is that what we’re—
Brian: Yeah, like the att—and the att—like, the credit. How do you figure out, like, well, you know, we had to talk to the office of change management, and then there’s the operational people who are going to actually make behavioral changes in the real world, the data team who provided the insights to do any of that. Like, how do we know what our contribution to this was? And especially if it’s like some big thing, it’s like, oh my gosh, we’ve been able to—like, we need half as many doctors on call because we found out half of them weren’t being used and we don’t need that many on call anymore, so we’re saving money and our care delivery is the same. So like, but what’s our credit, you know? Or are you, like, “No, no, no, that’s, like, the wrong game to play in the first place.” That’s a fair argument, too. But tell me a little bit about that.
Bill: Yeah, I think that there are some scenarios and some projects and initiatives where it’s much more linear and it’s like, you can see the cause and effect, but for much of the work that we do, it’s a little bit of a leap of faith, right? You think about data as driving better decisions to get better data to drive better decisions, like, the cyclical, like, flywheel, so to speak, right, and I think that’s an ethos that we have as an organization, and most organizations do have that, that claim to be data-driven. And so, it’s a little bit of a leap of faith to say that, like, well, if we use data to improve this outcome, then that’s going to be a better thing empirically for us or for our customers or for our patients or what have you.
And then there are, like, definitive, like, outcomes that the data team can support, right? So, if we do, like, a bottleneck analysis of the experience of our patients that become inpatients, right, from the time they enter the emergency room to the time that they are discharged, can we optimize staffing and can we optimize care delivery and the patient experience in a way that then shrinks the amount of time that they spend as a patient, which is a good outcome for them, right, that everybody wants to achieve for that patient and their family. Like, that’s the objective and like, we can do work analytically to drive that better outcome. And so, I think whenever we can do that, and we, kind of, create that very distinct and, like, explicit connectivity to the work, we do that. But again, it’s like, there is an underlying ethos that says, like, throw the data team at the problem, identify some success criteria, drive analytical endeavors towards those six success criteria, and that is a win. Like, so I think that it’s not always linear, but there are ways to create connectivity.
Brian: Yeah, yeah. You talked about AI a couple of times here; I wanted to kind of end in this space. So, I was thinking about our conversations. I think I was in for a soccer injury or something and I had to go to my PCP to get a referral for PT and all this kind of jazz, and so they’re using an AI note-taker in the room now. And I couldn’t—we were, I think the nurse was asking me something about how I’m doing and how’s the family doing, and I find myself mentioning—I’m probably making some of this up at this point—“Oh yeah, my kid was home sick with a cold and da, da, da, da.”
And then in my head, I’m like—because they had gotten permission to use this note-taker—I’m like, “Wait a second. So, now it’s going to say I am in here for a cold and an ankle roll and a whatever,” and I’m just, like, starting to think about the data that’s coming in and is this person going to actually read the transcript and correct all the notes or are they just, like, oh great, someone else is going to—the bot’s going to take care of the notes [laugh] . So, I was thinking about our conversation and actually, on a very tactical, just selfish or curious question that’s probably easy for you, like, is your team involved in, like, when you bring a tool in like that is the data team involved in deciding, like, whether or not to onboard a third-party? Because I’m sure they didn’t build this in-house. My guess is this is a bolt-on to the EHR or some third-party application that specializes in natural language processing and blah, blah, blah. Do you guys get involved in those kinds of buy decisions?
Bill: We do. We do. This is a very timely question on your part, right? Like we are actively moving to a new electronic health record in the coming months and part of our go-live deployment with the electronic health record is this bolt-on ambient listening technology that allows for us to capture the encounter notes, the clinical notes, and then make sense of them, right? And so, like, as a data team, we have a stake in how to make sense of that data, right? And we are definitely involved because what we want to do is we want to take that information, that unstructured information, and turn it into useful insights going forward. And so, that manifests across a variety of different contexts, both operationally and clinically.
Brian: Yeah, yeah. Is your team actively running genitive AI experiments in-house? And what’s your philosophy there? I know there’s some which find really high-value use cases and kind of go in all on that. There’s the, let’s find ten small experiments that don’t require us to boil the infrastructure ocean, so to speak, and try to see if we can provide some value in a small way. Do you have, like, a—or maybe you’re not doing anything with this, with HIPAA and all the other [laugh] challenges that are in the healthcare space, but talk to me a little bit about your AI work.
Bill: Yeah, certainly. Yeah, we were looking at a number of features. We’ve adopted some features already. We have our enterprise chatbot through Microsoft, so we’ve deployed that 15 months ago or so, and looking to further prove out the utility of that and increase adoption because we think there’s a lot of efficiencies to be gained from a productivity standpoint. But from a more of a clinical and operational standpoint, yes, we’re looking at a lot of different things that are built into the electronic health record system, things like note summarization.
From a data and analytics standpoint, there’s a couple of different features that allow for us to summarize dashboards and views that are built into the application, right? So like, it’s a little bit of, like, a translation type feature, like, so a normal—
Brian: Talk to your data? Yeah.
Bill: Responsibility of an analyst. You could just kind of say, well, like, what is this telling me? What can I summarize? What can I put into a slide deck? Or there’s natural language querying as part of that as well, built into the application that we’re deploying soon. And then there are things like translation capabilities through large language models, which is incredibly important in terms of the patient experience, especially for historically marginalized cohorts of patients and their families. So, you know, I think there’s a lot to be gained by looking at GenAI features that are being built into the workflows.
Brian: Mm-hm. Is that changing any of the, kind of, the product way of working or is this, kind of like, you could keep throwing new technology in and it’s not going to change our underlying ethos or you’re like, no, this is actually different enough that we are changing how we work? Any comments on that?
Bill: It’s interesting. It’s dependent on what angle you’re thinking about. The angle that’s very top-of-mind for me is our users, our stakeholders, our colleagues across the hospital are going to increasingly use things like Copilot to interpret information, right? And so, some of that is actual data and like, they’ll, you know, ad-hoc visualizations and ad-hoc analyses that they can get out of these tools and what does that do for the roles and responsibilities for the data team?
And, you know, I think that that—I think it’s okay. That’s a form of self-service enablement, but what it does to me is it heightens the importance of us having the right governance, capturing the right metadata, creating the right taxonomy, so that when we do have folks that kind of go off and do their own self-services with these tools, like, it’s attached to a knowledge graph that can make sense of the information for them without having somebody hold their hand, right? So, the way I think about it, it increases the complexity of the ask from the data team perspective, my perspective, making sure that the foundation is more solid because they’re not going to reach out to an analyst to get an ad-hoc question answered anymore; they’re going to just go to a tool. So, what can we do to prepare for that reality?
Brian: Yeah, yeah. That totally makes sense. Bill, this has been great. Do you have any closing thoughts you want to give to—we have a lot of people in data product management roles and leadership roles in this space that I know are always trying to find things they can take away from our guests and apply to their own environments. But any just closing thoughts you wanted to share with them?
Bill: Yeah, certainly. I think what I would say is, I would invest as much as possible, as a data team, in literacy programs for your colleagues, your stakeholders, your users, helping them understand how to use data, how to request information from the data team. I think it’s really important to kind of help them understand how to best leverage the resources they have at their disposal in their organization. Historically, what I’ve seen is, like, we don’t do a good job of communicating, like, how the data team can most derive value from the information and data that we acquire as an organization. I think just helping folks understand how to best utilize the data team would be a really great approach, from my perspective.
Brian: Thank you for that advice. Bill Saltmarsh, where can people find you? You have a great last name, probably not a lot of those, right, so I know you’re on LinkedIn. Is that the best place to connect with you?
Bill: Yeah, that’s the best place, I think so. Yeah.
Brian: Cool. Excellent. Well, thank you so much for coming on Experiencing Data. It’s been wonderful to chat with you and thanks for sharing all your experiences with us.
Bill: Happy to do it, Brian. Thanks for the conversation.