Today I talked with João Critis from Oi. Oi is a Brazilian telecommunications company that is a pioneer in convergent broadband services, pay TV, and local and long-distance voice transmission. They operate the largest fiber optics network in Brazil which reaches remote areas to promote digital inclusion of the population. João manages a design team at Oi that is responsible for the front end of data products including dashboards, reports, and all things data visualization.
We begin by discussing João’s role leading a team of data designers. João then explains what data products actually are, and who makes up his team’s users and customers. João goes on to discuss user adoption challenges at Oi and the methods they use to uncover what users need in the last mile. He then explains the specific challenges his team has faced, particularly with middle management, and how his team builds credibility with senior leadership. In conclusion, João reflects on the value of empathy in the design process.
In this episode, João shares:
- A data product (4:48)
- The research process used by his data teams to build journey maps for clients (7:31)
- User adoption challenges for Oi (15:27)
- His answer to the question “how do you decide which mouths to feed?” (16:56)
- The unique challenges of middle management in delivering useful data products (20:33)
- The importance of empathy in innovation (25:23)
- What data scientists need to learn about design and vice versa (27:55)
Quotes from Today’s Episode
- “We put the final user in the center of our process. We [conduct] workshops involving co-creation and prototyping, and we test how people work with data.” - João (8:22)
- "My first responsibility here is value generation. So, if you have to take two or three steps back, another brainstorm, rethink, and rebuild something that works…. [well], this is very common for us.” - João (19:28)
- “If you don’t make an impact on the individuals, you’re not going to make an impact on the business. Because as you said, if they don’t use any of the outputs we make, then they really aren’t solutions and no value is created. - Brian (25:07)
- “It’s really important to do what we call primary research where you’re directly interfacing as much as possible with the horse’s mouth, no third parties, no second parties. You’ve really got to develop that empathy.” - Brian (25:23)
- “When we are designing some system or screen or other digital artifact, [we have to understand] this is not only digital, but a product. We have to understand people, how people interact with systems, with computers, and how people interact with visual presentations.” - João (28:16)
Brian: Welcome back to Experiencing Data. This is Brian T. O’Neill. Today I have João Critis on the line from Oi down in Brazil. You’re in São Paulo, is that right? Or Rio? I forget.
João: I am based in São Paulo, but our headquarters is in Rio de Janeiro.
Brian: All right. Excellent. Excellent. Well, it’s been a long time since I’ve been to Brazil, but I’m due to go back and hope—maybe we’ll have a beer and talk about data products at that time. And that’s why I wanted you to come on the show here, to connect the dots between data products and design. So, you’re managing a whole team at a company called Oi. It’s O-I, which means ‘hi’ in Portuguese. Tell us what Oi is, and what is your team do, and what do you do it Oi?
João: Oi is a telecom company. We are [living 00:01:21] a big change now because we are separating our operation. So, the future of the new Oi is our company of internet fiber. So, we are connecting people with internet and change your lives with good connections and making the digital world easy. My team in Oi is responsible to all front-end data products. So, we design and develop all dashboards and reports and everything else about data visualization and information to take decisions.
Brian: So, you’re a design team with analytics people on it, or are you a design function inside an analytics team? Or some—option three, something else? How do you describe it? Because the reason I ask that is you know a lot of companies don’t have designers working in analytics, they have BI developers working in analytics that are working on a front-end, user experience is usually not a function traditionally, in this space. I think that’s changing. So, tell me a little bit about where do you fit into this organization?
João: Yeah. I think [unintelligible 00:02:32], Brian, because I am, like, an intruder in the data team here. I’m not the only designer working in the team, I have a UX designer and UI designer working with us. But we are a data team mainly, with Tableau developers and information designers, people who interview business to build some prototypes and documentations to developers, take this documentation and put on BI, too. And we have one more party in our team, people responsible for all web analytics data. They collect and present data from Google Analytics and other tools from our digital channels. So, information for navigation and user behavior in our platforms.
Brian: So, it sounds like, as is traditional in many internal data teams, you guys are primarily building services for internal business leaders, is that correct? Not customers of Oi. Is that correct?
João: Yeah. Mostly we are designing for our executives and directors, VPs, to take information every hour, every day to make decisions and drive your direction. We have operation team, operation sales team are very aggressive on streets, and so we have a big challenge to provide information for all levels to guarantee the beat of sales every day, every hour, every week. So, we have a big operation providing data for all levels.
But we have some products that people, our customers can access like our support apps and our websites. We have some pages and some features that we have to present some information, so you can find some charts hide in [laugh] support app or other parts of our website.
Brian: Got it. So, you guys specialize in the data components if there’s something that paying customers need to see as well?
Brian: They might call your team in for some help in that space?
João: Yeah, that’s it.
Brian: Got it. Got it. So, tell me in your definition—this term gets used a lot, I use it a lot too, and I don’t think there’s a concrete definition everybody shares yet—what is the data product to you or as defined at Oi? How do you think about what a data product is?
João: This is a good question, Brian. We discuss this every day because data products is a little difficult to define, or what is useful to someone. So, we define data product something that was called components. So, we have—I don’t know, if you know, like Atomic Design, when you will start from the molecule to a template, we use the same concept here. We have a small parts of data, and when we… put together as a component, we define that component as data a product. Can be a dataset, can be a prediction model, can be a report, can be a dashboard, can be push notification in, I don’t know, a message app like WhatsApp or Telegram or Teams, or other. We define data product something that people can use to transform in another thing or use as a final solution to make your tasks or make something new from that.
Brian: So, you kind of have, like, a whole library of these components that can be built out into full services or combined in different ways. And is that how you approach every new project is with that approach of this kind of reusable building blocks? Is that the idea?
João: Yeah. Is the big challenge because this is a big challenge for architecture, data architecture, because we have to put all our data together, splitted and separated by teams or by parts of our journey, customer journey. So, we are going to define our data, making a match with a customer journey, like a concept to market. So, it is a journey before a lead and before a sale. So, we have lead to [cash 00:07:04], so the journey starts in the lead and ends in the cash. And… in the end, we have the [unintelligible 00:07:11] to resolve so [unintelligible 00:07:13] to resolve is we are our client, we have to give you support and support our customers after the sales. So, we have three stages in our journey. This is our main subject.
Brian: Got it. Can you tell me a little bit about how you do the research to build these journey maps, right? Because you don’t just draw whatever you want; these are informed by customer research. I don’t think a lot of data teams are thinking about things this way, like, on a temporal timeline, where there’s a journey: a prospect becomes a lead who becomes a customer who becomes someone to sustain. Maybe there’s even a churn part of that experience, or they leave, or they’re happy and they leave, they’re angry and they leave. And you can take this all the way back around to new prospect again. How do you approach building one of these maps and doing the research for that? Who does the work? Are the business people involved? Are they open to being interviewed? Like, how do you do it?
João: Yeah we work as a design team. We are a data team, but we work as a design team because we make [discovery 00:08:21] in-house. So, we put the final user in the center of our process. So, we make some workshops with co-creation and prototypes, and we test how people work with data. This is a process very important because the dashboard may become unused if you don’t listen your users and what to your user have to do to make your job done. So, this is our core activity here.
So, we have to make sure we are providing to people opportunity to amplify their ability to make decisions. So, we need to know what people do at the moment the day starts, launching your computer and log off in your computer. So, what decisions, what systems they use, we have to take this information to provide data with [recency 00:09:23] people need, with frequency what people need, with quality people need. So, this is a long process because we have lots of resistance of business guys because they have an idea—
Brian: “Just give me my dashboard right now.”
Brian: “I don’t have time for this.”
João: I know everything I need. Just do it. [laugh][00:09:43]
Brian: [laugh]. Yeah. How do you respond to that when you get that?
João: Yeah, it’s very difficult struggling against this behavior because our response is every event we have [unintelligible 00:09:54], we need to talk people like… [unintelligible 00:09:59]. “Look, you do this analysis in two minutes. You have to open this application, this system, you copy this information, you make a calculation, and after that you base this information on other PowerPoint or Excel or other application. If we design something that take your job and automate these calculations and this opening this all the systems you do, we’ll make your job faster.”
So, this is one way we have to work. And the another way is how we display the information. This is another challenge for us because people have low maturity in analytics. This is a sad reality. [laugh].
I don’t know, I was a consultant about four years. I saw big companies with low maturity and big companies with manual process. And yeah, really manual process. This is real, this scenario is real. So, people don’t know how to choose the information have been display. How is the best chart to display that analysis is very important know what kind of analysis people are doing?
Because I choose the chart following the analysis. So, I don’t choose the chart because I like this is, I love pie charts. This is very beautiful and colorful. No, that’s not the point. The point is the analysis. What do you do with this information? What happen after you see this information? This is more important to us.
We have this approach. We are taking care about our process of people, our colleagues. And people open what happened after they see the data is very difficult because people don’t tell us what happened after. What do you do with this information? For what you need this information? People are resisting of these questions.
Brian: Do you cycle back with them? Or how do you track the success of your team’s work? Like, how do you know if you’re designing good data products or not? What’s the measurement? Or you’re a KPIs guy. What’s the KPI for the KPIs? How do you know that your work is working?
João: Yeah, we have some KPIs. I have designed KPIs, so we measure how many dashboards we made the prototype, how many dashboards we talked the final user, really final user or final group of users, we are measuring the time to develop. So, how many time you are spending developing the dashboard. And after the delivery, after a release. We have, like a NPS KPI, we make some one-on-one interviews with our product [unintelligible 00:13:03].
So, we talk with business to make sure we are delivering value, the dashboards are good or not. We have some surveys in every dashboard, have a survey with some questions giving a numbers one until ten, give us some feedback are this data is accurate or not, this data is not updated, this data is making sense today. Because the KPIs can change, this is very important. So, we have cycles of feedbacks we get every quarter. So, we are in our sixth cycle now getting feedbacks to improve our dashboard, so this is our process. This is a design process, just listening our users, this is very interested in listening our users to measure that our quality. This is the way we measure our deliveries.
Brian: Do you do any testing along the way before you get to production? To try to validate that the people are going to understand how to use the applications, the dashboards, whatever the design artifacts are? Do you do any testing of that alone? Not QA, but some type of design testing at all?
João: This is my dream. [laugh]. We don’t have yet user testing. We have to ask because with scenarios or, like, user research give us the frameworks. So, this is my next step. I think we need to do this because to understand deeper how people think, and how people make decisions here, we need to observe these behaviors.
Because the best moment for a designer is observation. How people are reacting to the dashboard. This is my dream, this is my next step, but we have workshops, we make some roadshows presenting and the making Q&A sessions to answer questions about the dashboards, why that KPI is in that dashboard, how is calculated that KPI? So, we are, how can I say, evangelizing. We are evangelizing how our colleagues have to use our KPIs and dashboards around the company. This is our work on that.
Brian: Got it. Got it. User adoption, is that a struggle? Getting people to use these solutions or not so much.
João: We have a big legacy here, so this is the challenge because every department built their solutions, and we are centralizing every data.
João: This is a struggle now. [laugh]. People have a solution. We are building new solutions, and we have [unintelligible 00:15:51] of solutions. And the adoption is a big challenge, but it’s a challenge now because we are in our big transformation; we are changing our business, so people are analyzing the legacy now.
But the new business is coming and the new business is coming in third or fourth quarter. So, this moment is very near now. So, people are changing, minds changing. This is a very complicated moment here, Brian, because we have to change the way we take decisions because the product is different, the business is different. The adoption now is a big challenge.
So, people are using more raw data now because people are understanding the new behavior of our customers of our products, and learning how is our new data. So, this is a new moment for everyone here. So, adoption is a big challenge for us.
Brian: Got it. How does your team decide which mouths am I going to feed? Because there’s probably more requests than you have time to work on, how do you prioritize projects, products, data products? What gets attention? How do you figure that out?
João: Ooh, this is [laugh]—
João: —this is a big war here because we have limited budgets and everybody wants data. This is a big challenge here. But we work on our agile project, so we make our [PI 00:17:25] planning. So, we plan development quarter and scoping quarter. So, these two tracks run together, and the same time you are developing, we are prototyping for the next quarter.
So, we have two tracks running together, developing and scoping new features and new data we have to present. So yeah, it’s very difficult, but the prioritization comes from our executive level, so they choose what features we will work in next three months. And we have to connect in this—how can I say this—cycle of the development. So, we have scoping, development, release. And after release, data analytics, take the data and make a data product. We have a long cycle here. We are talking about six months to a feature from final data product released.
Brian: One of the challenges some teams has is that you know especially if it’s coming very executive, you may get a request for a feature, as you would call it, and you’re not necessarily handed a problem to solve, you’re handed a solution to go build, which may or may not solve the problem or may not be properly scoped to really understand yes, we can abstractly give you a churn rate, but who’s going to use it and what are they going to do with this number once they have a churn rate? Are you trying to follow up with these customer—like, what is this for? Is that, like, a negotiation you have to have? And can you push back and say, “No, like, we need to have, I don’t know, a brainstorming session.” Or, like, how do you negotiate that? I’m sure it’s happened.
João: We have some data product owners that they are owners of our time. So, these guys are responsible for that. But we are a design team. [laugh]. We have to listen people. We have to make some adjustments and modifications. We have to deliver the best.
My first responsibility here is value generation. So, if you have to make some two or three steps back and make another brainstorm, rethink, rebuild something that not works, this is very, very common for us. Of course this is not good to business guy because we are talking about money, we are talking about our time, rebooting something or adjusting something. In my mind, this is the process. We ever be one hundred percent correct in the first time; we have to see the dashboards or data products as products, not projects.
João: Because products, you can improve the product. Projects, the project starts and the project end, and that’s it. Okay?
João: But product, no. Product is we can [unintelligible 00:20:25].
Brian: Is there a particular story or a particular increment of work that you guys worked on that has an interesting before and after story? Maybe you had a really resistant business customer and somehow maybe you changed their mind, or they became someone that was evangelizing your work after they had gone through the process together. Anything like that you could share with the audience because I think, what I’m trying to help people understand, there’s more than one way to build data products and to solve these kinds of decision support challenges that business users often have, but picturing it through a real example can be really helpful. So any, like, before, after story, you can share?
João: [laugh]. Yeah, I have a lot. Yeah, it’s very funny because I have to do that with my boss. This is a big challenge, is a real big challenge. So, when I land in the Oi, they didn’t know the value with conversation between the business area and data analytics area.
This is the key of the success because if you don’t know what you need, probably I will build a dashboard you don’t use. This is a big problem we can see everywhere. They have to accept a designer like me interview the VPs, or executive directors to ask, “Hey, what’s your relationship with our data? You see the data in your breakfast, in your home? In the taxi? Lunching? While you are eating?”
So, this kind of question is very uncommon for these people because they never were in front of a designer, and designers make some questions about behavior, about you, Brian. If you are my client, my customer, I will ask you what’s your relationship with our data? What do you do? What’s your pain? What do you don’t see now? Why you don’t see this information now? What do you need? Do you in your mobile phone? You need in your notebook? You need in your Whatsapp? What’s better for you?
So, the people was scared because this is very different. And in the start, we have some resistance from the medium levels of our—
Brian: Middle management, yeah.
João: Because the middle level thinks that they knows everything about other people, but this is not true. So, when we make this approach one-on-one, and from this approach one-on-one, you can take everybody’s involved in decision of that director, of the other executive. So, this is the big challenge we made here via human-centered data department.
This is a funny story because this is new, people don’t understand and don’t see value, but after the first, second release, people see the difference. “Oh, really? I have the information I need. This is good information. I have ever everything I need is here in this message on my phone. This is awesome.” So, this change of way of work is very funny because people don’t understand.
Brian: Yeah, I think what you said is you made a good point. Just for listeners out there, the issue with these proxy users with middle management or whatever management—a group of people who think they understand what another group of people wants or needs, there’s all kinds of bias and issues with that, right? A lot of times you’re either trying to summarize what a bunch of people need, you then filter it through your experience, and then you filter it based on the person you’re telling because you think oh, well, you’re a designer, so here’s the relevant information you need. It’s gone through so many filters at that point. And a lot of times what’s missing is the stuff that’s hard to get out of people, like, personal challenges that they have or, like, I don’t want to report bad numbers to my boss because we had a bad quarter and so I don’t want to get bad news, but I really need the bad news because I can’t possibly go forward without it, you’re never going to hear any of that stuff coming from a second party or a third party that’s gone different tiers of management and it’s summarized on behalf of somebody else.
Nobody’s sharing that kind of stuff, typically, in a format that really allows you to design something that’s going to change someone’s life and really make an impact. And if you don’t make an impact on the individuals, you’re not going to make an impact on the business. Because as you said, if they don’t use any of this stuff, it doesn’t make any value. You just spent money, you’re just a cost center, you’re not driving any innovation, you’re not increasing revenue, you’re not doing whatever the business objectives are, so it’s really important to do what we call primary research, right, where you’re directly interfacing as much as possible with the horse’s mouth—you’re talking to the horse’s mouth, no third parties, no second parties. You really got to get that, develop that empathy, and know what it’s like.
As you said, “What’s it like to be you? And what transparency do you not have? What information can you not see that would help you make a better decision tomorrow?” These kinds of questions can really get people to open up a little bit more and help them to realize I’m not here to just whittle dashboards and get my knife and tape and pencils out and make things for you. That’s not the goal. My goal is to arm you with the information you need to make better decisions.
João: Yeah, yeah.
Brian: That’s how I see it at least, and it sounds like you’ve had some similar—
João: I like to say that empathy is not a framework. You have to stay together, people, and feel what people are feeling. So, this is important. Do you know how our CEO feel about data?
Brian: [laugh]. Do you?
João: Not yet, but we are every day approaching closer of our executives.
João: So, we have VP layer with us. We know the VP layer pain point, we just get that. Yeah, that’s a big win for us. So, this is important because what this guy need? What this guy need? What’s matter for us [unintelligible 00:26:48], this is important. So empathy, we have to stay beside the people, watching people working, and feel the pain together.
João: This is difficult.
João: This is, like, a impossible thing, but we can do it.
Brian: So, do you know how the CEO feels about data?
João: [laugh]. That’s my dream, Brian, not yet, but we reached the VP level. We have very close from these guys and taking feedbacks with [unintelligible 00:27:22]. So, sent to them our prototypes to see new dashboards, new analysis, to contribute our [unintelligible 00:27:30] design process because they take decisions, so we need the feedback from them. So, the next step is some VPs to take us to the CEO room to have a conversation and listen about their needs. I am one year here in Oi, so this is my big win, reach the VP level. This is very important to us.
Brian: Cool. Cool. It’s been great to talk to you, João. I just wanted to ask you two, kind of, final questions. Is there one thing that data people, data scientists, analytics leaders need to learn about design? And then vice versa, is there something designers really need to understand better about data to create better data products?
João: Yeah, it’s important because we are talking about people. So, when we are designing or developing some system or screen or other digital or not—this is not only digital, but a product—we have to understand people, understand how people interact with systems, with computers, how people interact with visual presentations. So, information design is a discipline very important to understand that. So, it’s very important learn about business. Business is very important because the final objective, the final—the goal we have to reach is a better business for us, for our company. So, understand a little bit about business, how data can help business, this is the key.
So, we have three things to make sure we have in our pockets. First is, how people interact with systems and computers. This is very important to learn about that. It is very important to learn about information design, how we display the information, what’s the better way to display information. And the last thing is learn about business. So, study our business, make a deep dive in business areas, departments, what they need, why they work, what is the core business there. So, this is all very important. So, if you have these three abilities or skills, you will be very successful.
Brian: João Critis, thank you again for coming on Experiencing Data. Just wanted to give you the last word. Any closing thoughts you have about designing human-centered data products?
João: Okay, it’s my pleasure, Brian, thanks for the invitation. We need to listen people. Just put people in the center of your process, of your brainstorms, of your design process, this is very important to gather the pain point in the right way. So, the key is listen your colleagues, your leader, and our people in our company, we are designing for people.
Brian: Excellent, excellent. I love it. I’m totally with you there. Again, thanks for coming on the show. Where can people follow you? LinkedIn? Twitter? How can they get in touch?
João: People can get in touching LinkedIn my LinkedIn is /critis.
João: And my Instagram as well, critis is the name. [laugh].
Brian: Okay. Just your last name. Got it. Cool. I will definitely link those up in the [show notes 00:30:54]. And again muito obrigado.
João: Okay, very welcome.
Brian: All right. Take care. Thank you so much.