Transcript: Rachel Frenkel BenHanoch | Let's Learn Product Management

Cold open: the back door into product isn't enough

Rachel Frenkel BenHanoch: [00:04] Until recently, my advice to anyone who's an aspiring product manager and wants to get into an organization was: come in through the back door. Be CS, be support, be QA, get inside the company, learn how the company works, learn the product, learn the customers, and move up from there into product management. In other words, also set yourself a north star of where you want to get to. But today I think that's not enough. You need to bring those entrepreneurial abilities, create things yourself and show how you do it. Exactly, a pincer movement, I think so too, and I'd even add that it really isn't enough to just build the product.

Eyal David: [00:36] Put a few dollars into marketing. Hi, this time I'm hosting Rachel Frenkel BenHanoch, a seasoned product person, product consultant and product management lecturer. We talked about how she teaches product, the connection to entrepreneurship, product operations, a really interesting and cool case study from her days at AppsFlyer, and some insights about the future. Enjoy the listen. Hi Rachel, how are you? Hello. Rachel Frenkel BenHanoch is with us. Would you like to introduce yourself in a few words? Sure.

Intro: from VP Product to Product Ops at AppsFlyer

Rachel Frenkel BenHanoch: [01:09] So first of all, it's really fun to be here, I'm really happy, thank you Eyal for the invitation. I've been a product manager for very many years. I started when the profession was in its infancy. I drifted into it after a computer science degree and some work in IT, because I got really excited about the connection between business and technology, meaning how to help people and businesses with technology. I grew within the product management world. In the last ten years I was VP Product at companies, both startups and growth companies. In parallel, because I love the field so much, I started teaching in academia, and I teach product management to students both in entrepreneurship and in industrial engineering and management, basically the practice of what it means to be a product manager in the industry. Since I've passed through quite a few companies along the way, I bring them lots and lots of use cases and many cases straight from the field. It's not a theoretical course, it's a course that turns out people who already know how to do it hands-on by the end. My last full-time role was at AppsFlyer, I was in product operations. I founded and led the product operations group there. A fascinating field in its own right. I call it the little kid of the product world, because until recently VPs of Product did this role themselves, meaning tools, processes and methods for their product people. As product organizations grow, the VP Product has no time to handle it personally and needs someone to do it with them and for them, and that's what I did at AppsFlyer. Very fascinating. For the last year and a half I've been consulting startups, groups that want to improve their product management techniques and methodologies, busy VPs, and people who want to move to the next stage of their product management career.

Eyal David: [02:41] How lovely. Okay, so whoever is listening to this at more than 1x speed, great, you can slow down. Okay, so basically I asked you to come, Rachel, because you really have a broad view. Because you have a view both of people entering this world now from academia, and of course from all your broad experience across company sizes. And I'd actually like us to start with what, in your eyes, are the most important product skills in product people. Believe me. So first of all, I knew you'd ask this question, I prepared it at home, so to speak.

The three skills: communication, focus and data

Rachel Frenkel BenHanoch: [03:10] There are a lot of skills and I tried to compress them into three, and I'll explain for a moment. I think the most important skill is communication, the ability to articulate yourself. It comes from a lot of places. It comes from the fact that the product manager is at the center of the organization, really at the... I show my students: at the center of the sun. They talk to customers of course, but also to management and to development, with CS and with Sales, and they need to speak to each one in their own language. The product manager holds a ton of information in their head, but they have to bring the right information to the right person at the right time, very concisely. So communication is basic. Communication also helps when we need to lead without authority, leadership without management, and that's done with our tools for conveying messages in a very meaningful and impactful way. So that's the first thing. The second thing, and for me it was very hard as a beginner product manager, I hope today I'm a bit better at it, is focus. A product manager is flooded. Flooded with demands from every direction: from customers, from management, from development, from QA, from CS, from Support. They need to organize these things, prioritize them, and above all stay focused, because everyone pulls in their own direction. Sales want you to build the feature so they can close a deal. CS want you to build a feature so you'll keep the customer for them. Management wants you to build a feature that will outbeat the competitors, and so on. If you don't know how to keep a backbone and your focus, you'll get lost and won't reach your goals. Which is connected to communication too, right? Yes, yes, yes, but that sharpness of "right now you have no time" or "this isn't the focus." You can communicate to understand whether something is the focus, right, dig in and understand, right, there's a lot here, right, right, and give a very focused message, right, right. And first of all know it yourself, meaning know how to judge whether this is in my focus or not in my focus, and whether I can, need to, handle it now, yes or no. The last thing that I'll still include here, because I can't not, is to say we also have to know how to work excellently with data. There's no such thing as non-data-driven product management. Now, true, there are places with no data and you have to make a smart guesstimate, but as much as possible work with a lot of data, and with the research tools that exist today in AI you can no longer say "I don't have data," because even if data didn't reach you directly from customers, you have a way to do research by simple, cheap and fast means, and you must come with data-based things. That will also help you with focus, and it will also help you with communication, because to convince someone you need to bring them the numbers. And all these things together make you an excellent product manager. Wow, I totally agree, and I'd even expand

Eyal David: [05:31] with insights, not just with data. Exactly, like, bring the end result, the bottom line. Of course, of course, that's what I found, and therefore, of course, how much it costs. Okay, cool, so tell us for a second just about your path, basically. You're now in academia, you work, what does it look like where you work, how do people get into this field?

Product management in academia and guest lecturers

Rachel Frenkel BenHanoch: [05:55] Okay, so first of all, I'm thrilled there are academic institutions that teach product management. I think it's one of the most important professions of the future, with gen AI that lets us today basically create things without knowing code, and there are endless examples of this. I just got off a lovely podcast where the investor Lukach interviewed Maor Shlomo, of course, and he told about things people built, but about people who changed their profession: someone who was a restaurateur became someone who provides software to restaurants, because he knew the customers' pains and knew how to solve a real problem that exists. So I say that the product management world, if you know how to be a good product manager, you really can start your own business, and that's why it's taught, for example, at the School of Entrepreneurship at Reichman. That's where I teach. It's taught as part of industrial engineering and management at Afeka and at Shenkar, and Shenkar even opened a product management specialization now. Meaning, academia totally understands this is a profession of the future, and it also understands it's a profession of the future not at the academic research level but at the very, very practical level. That's why they always bring people from the field, of course at the places where I teach but also at other places, because you need hands-on experience, you need the practice, and you need to understand how it really works. Right, much more today than in the past, I think. And how, really, what people do they bring, what does it look like? So first of all, people choose the track before they choose product management, because product management isn't yet... No, I mean you said they bring people, guest lecturers. Ah, of course. Guest lecturers, I assume. Yes, yes, yes, guest lecturers. Meaning, there's someone who teaches the course, the methodologies, but they bring as many cases as possible from the field, from people who specialize in one area or another within product management, whether it's go-to-market stuff, whether it's prototype building, whether it's UX, whether it's PLG. Meaning, they bring people who bring their use case and their specific expertise into the course. But beyond that, the students are asked to build a real project of their own and really practice everything we learn, and come out with something that's a kind of portfolio, in which they deal with what you need to do as a product manager and with the product manager thinking we're trying to develop in them.

Eyal David: [08:03] Classic. Entrepreneurial thinking, basically, backed by business at the end of the day. Right. Entrepreneurial. So tell us, basically, who comes? Who comes? What personas do you see in class?

Who studies it and what they learn: PDLC and product mindset

Rachel Frenkel BenHanoch: [08:14] Women versus men? What does it look like? The truth is I don't see a difference between women versus men. Is it still men? Is it still 8200 types and the like, what does it look like? No, no, no, no, not necessarily. Meaning, no, the people who come are people looking for the multidisciplinary view and their ability to create things themselves. And I think both the entrepreneurship track and the industrial engineering and management track are that kind of multidisciplinary view, which fits like a glove for someone who wants to be a product manager, who has to have a very, very broad view. After all, a product manager deals with strategy and tactics, with marketing and sales and with the technical side, with user experience, of course with business. And to see this broad view you need someone who's interested in dealing with all these things, who doesn't want to be... Meaning, people who want to be programmers go study computer science, they don't go study product management. But people who want to deal with the field, with creating, with customers, come to study these things. Yes, do they understand at the point in time when they enroll that this is what they're going to do, these kind of integrators? So I know there are students, when I started teaching it wasn't like that, but I know there are students today who come and tell me, and this happened to me more than once: "I enrolled in the track because this is the course I want to take." They literally came and said "that's why I enrolled in the degree," I have this documented in emails. So yes, it's passed from generation to generation, and they already understand this is what they want to be when they grow up. Amazing, so what do you learn there, what does it look like? Okay, you said practice. Right, right. So first of all, of course, you learn the practice. I'll go over it really, really fast, but you learn the product development life cycle, yes, SDLC, PDLC, everyone calls it by their own name, to understand what this cycle looks like: from initiation, through understanding the market, understanding the customers, customer interviews, pain hypothesis, personas, customer journey, validation of the problem space. So far we've reached validation of the problem space. From here we do brainstorming of ideas, everything accompanied by the students' real hands-on work. Choosing the idea through reasoned prioritization, meaning you learn about prioritization methods, when to apply each method and how to connect it to the business KPI. Creating a prototype, you can't not do it today with gen AI tools that make insane prototypes. Validating the direction of the idea with customers, improving, meaning the build-measure-learn loop on that prototype. And of course you can't skip measurement. Within the course you can't measure success, but you can measure the level of engagement, scores that testers gave the prototype, saying from one to ten how much it solves their problem, how much they'd pay for it, and so on and so on. That's at least the measurement of "I'm in the right direction," and from here you can start building the product for real. So all of this is the practices. Okay. But I think what's important to me to teach in the course, beyond the practices, is the product mindset. And the product mindset includes product thinking that says: I'm not here to create technology, I'm here to create a solution to a problem. I need to understand who my target audience is, what their pains are, how I create a good, differentiated solution for them. Meaning, do the product strategy in a way that I understand what the big goal is that I'm trying to reach. I talk with them a bit about a company's vision, throwing out something far, huge. Just as an example, Google's vision is access to information in one click. Like, when someone looked at that 20 years ago it wasn't possible at all. Yes, but, but these things happen when you aim very, very high. And then you understand, okay, so who am I addressing, and you define your personas. Of course, afterwards you need to focus yourself on the first persona you'll build the solution for, not try to address everyone at the same time. And then you understand their pains with a customer journey map, and then you build your product, and then you define how you'll make a business out of all this. So that's product strategy, and the product mindset is of course connecting a real need to an excellent solution. But not only that. There are lots and lots of soft skills, traits that an excellent product manager needs to adopt. I think whoever chooses this field already has some of these things, as we said, good communication, the ability to focus, but some of the things you need to know and work on to get to that place. So I really teach them, one of the things, I'll give it to you methodologically. When I start talking with them about product strategy, I tell them that the title of it is "begin with the end in mind." I love this sentence. This sentence is taken from The 7 Habits of Highly Effective People, the seven habits of highly effective people. The book, this masterpiece, was written by Stephen Covey in '89, I checked so I'd know. Stephen Covey has not been with us for about 12 years now.

Eyal David: [13:02] But there are YouTube videos where you can hear him. One of the best-selling books, I think. Yes, it was a bestseller. I'm really curious whether Google's vision stayed like that, I haven't checked. Last question. I assume they're already, like, I assume they're already, like, one click, like, maybe it got a different perspective. Just before we continue: click the link to the Product Builder WhatsApp group in the episode description and join the community. There you can continue the discussion, ask questions and be the first to get updates on new Tachles episodes. We're waiting for you there. And now back to the episode. Tell me, and I'm really curious, how do you work on soft skills inside a lab space like that? For example, just the place to get feedback, that's probably already a place where you can work on soft skills, right, giving feedback. But how do you work on it? Soft skills, I think we can actually work nicely on communication, whether

Rachel Frenkel BenHanoch: [13:58] it's written communication or verbal communication. The students are asked to present things, sometimes in groups. I also do a group prioritization exercise, and then they work on the group work and on convincing people and so on, and I reflect back to them what their group dynamic was like and how they managed to reach results. Whether one person led, whether there was a joint decision here, and how we get people on board to listen to the things we want to bring for ourselves.

Eyal David: [14:28] Good ones, really, getting people on board, that's like what you said. I'll connect it back to connecting them, basically, to the "after," right, to that big vision, right. So please, you can continue.

Stephen Covey's 7 Habits as a PM toolkit

Rachel Frenkel BenHanoch: [14:39] I'll go back for a moment to that vision of "begin with the end in mind." So really, Stephen Covey revolutionized my life. I think I read the book, like you say, even separately from product management, it's really about how to make use of my life and get the best out of it. So "begin with the end in mind" is the second thing. He starts there with "Be Proactive," and I think I took that very, very much into my life when I told myself... People also learn this as the circles of influence versus the circles of concern, I think there are all kinds of models for this, that's also an interesting thing but I won't digress there right now, look up circles of influence and circles of concern. But I'm talking about "be proactive," and I took it into my life and I certainly took it as a product manager. As a product manager you're bombarded with information. If you don't know where you want to go, move the noise aside and proactively go toward the goals you're striving for, you won't get there, because other things will always be dropped on you. So I very, very much take this into the product management world, and it's part of the mindset I try to instill in my students. The second thing is of course "begin with the end in mind," and I tell them to keep the goal, the big vision, in front of their eyes, and even if you veer here and veer there, and certainly in the early stages of your company when you're chasing product-market fit and moving right and left endlessly, you need to always know where you're trying to get, in order to succeed in getting there, even if you need to make detours along the way. So that's very, very important. The third thing, if I remember correctly, that he says is "put first things first," which is classic prioritization for a product manager. Again, if we don't manage our time and don't do the truly important things first, we won't reach our goals. And you could go on, but it's really worth looking at The 7 Habits of Highly Effective People and seeing how it changes your product mindset. Yes, a great book, we'll really add a link. We'll also shift up a gear and really talk about product measurement and organizations, we'll get there, but let's stay in this area for another moment.

Eyal David: [16:33] How much do you feel that the mindset they come with, the expectations they really arrive at your class with, how different are they from the real world?

Undergraduates vs. experienced product managers

Rachel Frenkel BenHanoch: [16:41] Regarding the product management profession. So it really depends. I teach both bachelor's and master's degrees. People who come in the bachelor's are more clueless, they're really learning the basics of the profession in general, and few of them have gotten seasoned in the working world. Some work in support, a few, and that's lovely, work in CS, but some don't know yet, most, I'd say, don't yet know the high-tech world and how it operates. So from their point of view they're absorbing at this stage, they don't yet know how to tackle it with the things of the real world. So they come without many expectations, they come with an expectation to learn. When I teach the master's it's already a totally different perspective. People come after several years in the industry, some as product managers, a small part as product managers, a large part in professions of all kinds, even totally physical industry, not necessarily the software products industry. And then they tackle more and try to take all the practices we learn into their worlds, and they succeed greatly. Just a really cute anecdote: now, with the whole war, some of the students, certainly in the master's, in both degrees, couldn't be in class. A student in the master's who hadn't been in the course at all contacted me now and said, "I'm studying for the exam now, I'm asking to sit for the exam even though I wasn't at any lecture, because I was so-and-so many days in reserve duty." I told him, of course you can sit for the exam, no problem at all. And then I told him, how fun, I see on LinkedIn that you're a product management team lead at one of the companies. Tell me, while you're studying my recorded lectures, what are you learning and what are you taking from it? And he already told me a list of things he'd taken into his real worlds, and that's fun for me. Meaning, that I manage both with people who are basically a tabula rasa, to give them the understanding of "wow, product management is an amazing profession, this is what I want to be in life," and also, on the other side of the scale, to take product managers who are already product management team leads and give them insights and give them practices and give them methodologies they can take with them. So that, that makes me very, very happy. I'll say one more thing about this. I think we all basically know that until recently, until academia started training a bit, and today there are also courses outside academia, people came to product management from within the discipline. Meaning, they were CS at a company, or were even the type of customer, and understood the market and the need and the customers well, and then they were told, "you understand the customers so well, come be product managers." But then they learned the product manager role through on-the-job training, they didn't learn in an organized way. And then it may be that they've been doing it for two, four, seven years, but never learned things in an organized way. And then when you come and tell them, "it's nice that you do prioritization, but let's think for a moment about seven different prioritization methods and choose thoughtfully which method is right for your product at its stage, with the data you have or don't have," suddenly it really does something for them.

Eyal David: [19:34] I think it's going to come back. Like, while you were talking, very interesting, and by the way that's the reason you're here, because you can talk about both disciplines. I was thinking, first of all, about AI and how the role is going to change. Fine, let's talk about that. But also the thing where I got into product, I really understood the work, started getting into it and so on. And I think the world of coming to intern at companies in order to basically learn about an industry, learn about the business and so on, and after that, that's going to come back. Probably, or it'll be like in the United States. I think that's something sorely missing here, and I think it could really flourish. We talked about it before the recording, and I, about the basically values-driven value of bringing product people and new blood into the industry. Yes, so maybe that's a direction, I don't know, all kinds of internships, like, it's something that has to come in. There's a win-win here, we talked again about things from the book, so win-win is one of the things, right. But tell me what you imagine, really. A, really, maybe it's worth encouraging people, like, to start reaching out and trying to get them. I assume many startups and many companies would be happy for internships of one kind or another.

How AI is changing the PM role

Rachel Frenkel BenHanoch: [20:39] But tell me for a moment how you see this role in the future. So there's no doubt AI is making a change in the whole world, in the product manager role especially. One aspect, many people in the industry are already writing about it, is that product managers need the programmers less and less to do their job. Meaning, if until now for a prototype we needed a developer, we needed the designer and the developer to make a prototype so we could even validate with customers that we're in the right direction for the solution, so not anymore. We can run validation cycles with the customer without anyone else in the organization, and that very, very much changes our work. So I'll talk for a moment about three things. One was how we do our work better using gen AI tools. So one, the invention of prototyping. The second is research. The third is prioritization. The fourth is even ideas for initiatives based on the company's KPI, not to mention writing stories. Everyone makes themselves a copilot that writes stories with all the organizational knowledge. At one of the organizations I consulted for, we did a project and it came out amazing. We took all the Jira from the last year, all the stories, the epics that were written, exported them, uploaded them to GPT and said: learn, this is our style, meaning these are the things we address in stories, this is the content domain, these are the acronyms and so on, and now start writing us stories in this content domain. The results were instructive, truly instructive. The stories that came out were ones that needed tweaks and off to Jira. Meaning, amazing on the technical side. Okay, that's clear, maximize your capabilities. Totally, right. And in a moment I'll connect it a bit to juniors, but I want to look for a moment at another world of product managers, how AI changes them, that interests me much more. Okay, I don't know if I'm going where you're heading, but I'm talking about automations and about our ability to be... Meaning, there's doing the things we used to do professionally with help, and there's improving the processes we do. For example, making automations that manage our emails better, making automations that manage our Slack better, making automations that know how to collect feedback from users and distill it for us, making automations that, I don't know, summarize meetings I couldn't attend and feed me information. To free up time as much as possible, to be efficient, basically. Right. Now, this isn't something someone will do for me at the company. Meaning, as an employee at the company I need to know how to simply do 3x or 10x now with these tools, both in the profession itself and in the process side of things. So it may be that a talented product ops person will come in and do this with the product managers, but product managers have no choice but to jump into these worlds, not only in writing better PRDs but in automating their processes and simply producing more in less time. Totally. So whoever hasn't started, start. No, I actually took it in another direction, because I think there's another thing, and you touched on it.

Eyal David: [23:26] It's the more holistic, more entrepreneurial thing, basically. Because once you're now less dependent on stakeholders of one kind or another, you need to understand a lot more about a lot more. Okay? So, like, I think broadening horizons and curiosity is something that, wow, I'm really looking at now. Unequivocally, every person who stays in "this is what I know how to do and this is what I do" will be left very, very far behind.

Breaking into product today: build it yourself

Rachel Frenkel BenHanoch: [23:51] Wow, there was some ad when I was a kid, at the cinema, for some coffee. Something like "whoever doesn't go forward goes backward." It was in some deep accent like that. But I believe in it so much. Meaning, you, there's no such thing as not going forward. Standing still is going backward. You must learn the tools and you must use them. I want to get into something for a moment. You mentioned earlier the topic of juniors and how they can get in at all, together with the topic of curiosity. So I think that because of all these AI tools, and because of a person's ability, when they understand a particular content domain and a problem world and a world of customers in a specific domain, they can already try out the whole process of building the solution and the validation, and show product abilities even before they get in. Until recently, my advice to anyone who's an aspiring product manager and wants to get into an organization was: come in through the back door. Be CS, be support, be QA, get inside the company, learn how the company works, learn the product, learn the customers, and move up from there into product management. In other words, also set yourself a north star of where you want to get to. But today I think that's not enough. You need to bring those entrepreneurial abilities, create things yourself and show how you do it. Exactly, a pincer movement, I think so too, and I'd even add that it really isn't enough to just build the product, put a few dollars into marketing, right, so it meets something.

Eyal David: [25:12] Yes, but, but yes, right, meaning, feel it all the way. No, but also in the company's content domain. Meaning, even if it's a company that targets enterprise, then okay, create something that gives value to someone in that area and bring in traffic. Yes, you need to be very careful if you do things in the content domain of the company you work at. No, not even at the level of collecting data, at the level of giving basic value, show that you know how to do something real, right? Right. I think that really improves candidates, and then there's also a chance they'll accept you more easily into that intermediate role you mentioned, right? Right, right. And they'll go along with you on the fact that you want to move to product. Okay, cool. Any more insights about the future you're thinking about? I really think product managers will become much more entrepreneurial, and I think in very small organizations the product manager is going to integrate a lot of things, much more than today. That's really interesting. There'll be more product managers. Right, I'll add two insights to that. I totally agree with you. One, in the beginning I think the product management course within entrepreneurship was a little stepchild, and today it is the entrepreneurship. Meaning, in product management you basically learn what it means to create

Product management is entrepreneurship

Rachel Frenkel BenHanoch: [26:20] your company from zero to a hundred by yourself, because you really can. Okay, so you no longer, you maybe need a deck for investors and a pitch a lot less, because you simply do it yourself, like Maor Shlomo. Yes, so it can be done. That, I think, is one very, very important insight: product management is entrepreneurship. And a second thing, I no longer remember who said it, I have it on LinkedIn, you can look it up later, but someone very important said he wanted to do a reorg at the company and asked, like, how do we divide the money or the headcount among our people. And in the new organized group they asked him for a ratio of product managers to developers he'd never seen before. Meaning, if before they said, okay, for every five developers we want one product manager, now they wanted, I think, one to half, meaning twice as many product managers as developers. Like, some crazy ratio like that. Meaning, it completely flipped. Meaning, they understand that product managers can do so much without developers, that the developers do the finish, sorry for putting it that way, but you can do so much more. More product managers. What's certain is that the bottleneck becomes the product managers. Meaning, the developers are also already developing much faster, obviously, with all the copilots and the coding environments they work in, so they can deliver much faster velocity. So who's now the bottleneck is the product manager who didn't manage to do the research and didn't manage to do the validation, and therefore they want many more product managers. Curious, you don't know which company.

Eyal David: [27:47] Let's step up now, basically, to product adoption. Let's go. We talked about it, we'll connect it for a moment, we talked about it in the context of a case study you did, basically, at AppsFlyer, right? Just tell us about that move to ops, how you found yourself there. Sure. After many years of being in product and VP Product at startups at various growth stages,

AppsFlyer: how to measure product impact

Rachel Frenkel BenHanoch: [28:09] I, sorry for putting it this way, but I felt I'd kind of exhausted it. Meaning, going into startups again with the same growing pains and doing the same methodologies and succeeding already felt a bit like more of the same to me. I told myself I want to look at this from a different angle and help product managers be better in what they do, and not necessarily manage and be the VP Product responsible for all that string-pulling. Did you basically join AppsFlyer in this role? Yes, yes, I joined AppsFlyer as Director of Product Ops, who founded the product ops group at AppsFlyer. It's a big organization. Ah, maybe we'll say in a word what they do? Sure, gladly. AppsFlyer is a marketing analytics company. It basically helps marketers run campaigns in a smart and effective way, with a lot of tools that help them understand which campaign creates value and gives them the value on their ad spend, on the money spent on the campaign, how much of it yields good value for them. That means which campaign, on which platform, with which assets and so on. A comparison that's very hard for a marketer to do alone, working across several platforms, and it's amazing. Roi Anai was here in one of the first episodes, so we'll add it. Oh wow, well, he's the champion of champions inside AppsFlyer. Doing attribution, basically understanding the money that comes in at the end, where it came from, out of which advertising platforms. In short, AppsFlyer is a very, very large organization, international of course. The R&D organization is about 450 people, the product organization about 120 people, where product means product and UX and analysts, and there was a need to bring order there. AppsFlyer grew like many startups, with very, very autonomous products, and to make it one platform there's of course very significant UX work, but we also wanted to do work on methods. We wanted to create working methods, also best practices. Meaning, there's Roi Anai there and another VP Product there, and each internal product organization of a sub-product worked with its own tools, its own methods, and had its own insights on how to work best. But why not learn from ourselves and take the best of the best from all the groups and basically implement it across the other groups. One of the challenges we dealt with was how we measure ourselves. Now, how we measure ourselves in the non-classic sense. Not how we measure the company's success, everyone already knows that, money. But how we measure the impact of product managers, and how we measure the impact of a product within the company's product portfolio, when the products are at different levels of maturity. There are products that have been running at AppsFlyer and customers have used them for a decade already, there are ones that came out a year ago. How do we understand what the impact of that is, also at the different scales and the different numbers of customers? Because if you look, say if you measure the number of active users, then there's a product that's 10 years old, no wonder it has 100 times more than a product that's only lived a year, so you can't measure it that way. Sure, and how to build it when renewing a contract once, right? Right. That's exactly the point? Right. Okay. So what did we do? We decided to build something called a Product Adoption KPI, which can put all the products on one comparative level, in the following way. I sat with the various product groups and looked for champions within each such group, one Product Manager and one analyst, who would sit and define what deeper and deeper usage of their product means. Why deep usage? Just let's explain. Why deep usage? Because I want to see that users don't just log in to the system and do something basic in it, but that they understand the value that the system, that this product within the AppsFlyer platform, can give them, and they configure deeper things, ask smarter questions, cross-reference information more intelligently, and really get the maximum value out of the product. Now, of course there are different adoption levels for different customers. Some don't need deep adoption, but some do, and we wanted to try to understand a lot of things about it, also what the customer trends are. Meaning, customers who were in low adoption, what made them reach high adoption? In a moment I'll say how we calculated the metric and why it was so important, but it's very intuitive to say that the more a user is in high adoption, it means they find more value, it means they'll likely stay with us longer, and likely the cost of switching to another, competing product will be [high]. And also they'll probably expand their budget next year. Of course, in the end it's expressed in money, but these are very, very strong early indicators of whether they're going to stay with us, going to expand the number of users, going to do more transactions with us. How did they work up to that point, what did it look like? They always measured ARR, okay? We always want to back up the money coming in, but it was very, very hard to compare between the products. Of course they looked at how many accounts use each feature and how often, but there wasn't some metric you could track over time and see the depth of usage. And maybe we'll also talk about how much meaning it had for the rest of the functions in the company, right? Of course. I think one of the important things about this is that we started speaking a common language, not only within products but with the rest of the organization, mainly with go-to-market. Look, CS want to keep the customers with us, they want them to manage to find the value, so they have an interest in high adoption. Sales also want, after they've sold, to be able to sell more, to do upsells, so they want the customer to be happy and able to use it. But these organizations, when they didn't look at the same metric and at optimizing the same thing, could work in very, very different directions. Meaning, CS, for example, ran a campaign for South America in the gaming vertical, because they felt there was potential there to retain customers better. And Sales did crazy deals for another region of the world, and product was focused right now on product-led growth, whatever. Meaning, they worked with full force, with full professionalism, but in very opposite directions. And we said, let's do it differently. Let's look at one metric and unite our groups to act in the same direction, so it'll be 10x impact, not that we're putting full gas in neutral, simply in opposite directions of investment.

Eyal David: [34:00] And tell me, if you can share, this is obviously a move that comes down from management.

Rachel Frenkel BenHanoch: [34:05] I don't know if it comes down from management. I think the CPO is the one who initiated it, and said, let's find a metric that reflects the impact of the product people and of product within the organization, one that's well ahead of ARR. We want to see that a user finds value, which is an early indication that they're going to be with us for the long term, and going to do upsells, going to expand and so on. So it came more from the CPO level. Once they saw the results, and in a moment I'll tell a bit about a campaign we ran, it was adopted at the management level. Adopted at the management level to the point that Product Adoption became a management-level KPI that's tracked from quarter to quarter, seeing how it progresses. Meaning, it didn't stay at the product level. I really interrupted you. So wait, let's, I told you this sequence. One second, we'll come back relatively, whether on that double down came in with campaigns.

Eyal David: [34:48] Take us for a second, really, how do you bring a move like this down to the ground? How did you start working on it? Okay, so like anything, you have to start small. I loved the supermarket analogy, you have to do it. I have to do it. Cool, totally fine.

The supermarket cart and the Compound KPI

Rachel Frenkel BenHanoch: [35:00] So really, after this thing made a huge buzz in the organization, I went and talked about it at a Product-Led Alliance conference, yes, the product ops of the world, in London, and presented our use case and talked with them about it. I told them the totally true story. At the end of a workday I go to the supermarket and put products in the cart. My goal is to finish the shopping as fast as possible, run home, see the kids before they go to sleep. I fill the cart, load it with stuff, and the more I push, the less it moves, until it gets stuck. Not good, I want to get to my kids on time. I look, I try to understand what's happening, I see the cart's wheels pointing this way and that way, cross-eyed in every direction, and I realize I have no way of getting home unless I either take the things in my hands and run to the cashier, or, or give up some of the things, make the cart lighter and run forward. But it really made a penny drop for me, and I said: I'm pushing with all my strength, the cart doesn't move. Why doesn't the cart move? Because its wheels are pointing in different directions. Let's take this for a moment to our organization and understand that if our wheels push in different directions, no matter how much force we put in, it won't reach our goal. And then I said, we have to look at the same goal and have all the wheels pointed forward, and that's exactly what we did with CS and with Sales and with marketing and with product. When we looked at this Product Adoption, and let me take one step back and say that beyond the fact that we started with one small group, obviously any move inside an organization has to start with a small group that's your design partner, both for the process and for the results and to show that it works, only then can you of course bring it into the rest of the product groups. But we also brought in CS people, who were incredibly excited that we showed them we'd be able to create early indicators. It's really, really crazy. Let's talk for a moment about how this Product Adoption worked. I sat down with the product manager and the analyst of each product organization, and we said: pull me data that reflects depth of usage. Not just how many times they log in, but which features they activate, what configuration they set up, which persona within the customer account is using this thing. All these things are very indicative of depth of usage and of the value they extract. And we built a kind of Compound KPI made up of weights for a series of actions an account performs inside the product, and we got some value from 0 to 1, like 0.2, 0.7 and so on. Now, when we looked at customers, at this value of theirs over time, we saw fluctuations. But it was very hard to understand what these fluctuations meant. If a customer went from 0.4 to 0.42, is that significant enough? Is it meaningful, do we need to address it, do we need to do something about it? We didn't know. So we decided to make buckets, to look at the picture a bit from above, a bit from afar. We said, let's split it into three buckets. The low bucket, no or low usage, like "we configured but didn't use," "we logged in but didn't do anything with it." Medium, they're doing something with it. And those who are fully, I don't know if fully adopted, but highly adopted. What's highly adopted? They use the advanced features, they get into it, they get value out of it, they pull reports and so on. Now, what I also asked of the analysts, and I think this is essential and a bit of a game changer in the value this thing provides: we asked that the high adoption range, say in one product it was 74 to 100 and in another product it was 89 to 100, each had its own number, be correlated to the customers' chances of retention, looking at past data. Afterwards you'll do a replay and understand how it can be done, but we looked at past data and said: customers we saw over time who were in this range of Product Adoption value stayed with us, renewed, did a contract renewal with us. And then it was a game changer, because CS said, we get it, if we have a customer who's in Low or in low... In short, you went backwards, let's explain this for a moment.

Eyal David: [38:53] So they won't have to run us at slow speed here, and so on. Basically what you did: you looked historically, according to the criterion that basically segmented into different buckets, you took certain users from a certain point in time who behaved properly, or the way you want. Right. And then you looked back at what they did and knew how to project it onto others. Right. Very right. Um... and... you're basically doing a weather forecast here. I've never been there, but could be. Yes, no, that's it. Love it. Totally fine.

CS prioritizes by bucket

Rachel Frenkel BenHanoch: [39:21] I think, though, that its strength came from several aspects. From one aspect, CS said, wow, now we understand our goal. We want to secure customers, we need to make sure they reach a level of usage above this bar, and then I know they'll stay with us for the long term. And then it also helped CS prioritize their time, because they say: those in the high spot, I'll attend to them once a quarter. Those on the verge of moving to high, let's push them to move to high. Those in the low, maybe we'll hold their hand more, because they have a higher chance of churning. Meaning, they knew how to prioritize

Eyal David: [39:53] You can never reach all the customers, and otherwise you couldn't incentivize them, you couldn't move them there. Worth saying that, right, to them, to Sales, to whatever. Right, right. You need, basically, it's not that you work with the sales organization, with the specific person, you need to go again from the top and explain why it's worth it. Suddenly you have a really good case in hand. Right. And then you built a product. Right. I'll just close the loop on two aspects. One aspect is also

Targeted campaign: 68% moved to high

Rachel Frenkel BenHanoch: [40:20] that the segmentation ability was very, very deep. Meaning, you could segment by geography, industry, time zones and so on and so on and so on, to find exactly the similar and different parameters between a customer who's low and a customer who's high, and push with a very specific campaign that touches exactly the point where they need to improve to move to high, and convert them to high. And we did that. Meaning, these really were jaw-dropping results. In one of our products we took and looked at a cohort of this medium bucket. We said, let's reach out to them, let's prepare very, very targeted marketing materials, because we know what they can be better at compared to those who were already high. Through CS we set up meetings, CS sat with them and explained the value to them. After a few such meetings over a span of three months, 68% of the customers we approached with these precise marketing materials moved up to the high bucket. Meaning, we made sure they'd stay with us for the long term. That was a crazy success, because it's the precise materials to the precise people at the right time. That you know will also work.

Eyal David: [41:25] Yes. I'm simply picturing your slide deck as you talk. Amazing, like, pre-AI, all this craziness. Right. What were the reactions in the room after that talk, how was it?

Reactions in London and how long rollout takes

Rachel Frenkel BenHanoch: [41:35] In London, I have photos of it. First, people photographed my slides the whole time, I have a picture of people photographing my slides. I think it really, really opens minds and says "we need to do that," because it gives crazy impact, both in that Sales, CS, marketing and product all look at the same goal, we want to improve Product Adoption, and also because it gives very practical tools to all these groups to act in a focused and effective way, and also because in the end it contributes to the company's ARR in a very, very clear way. We brought more people into retention, we grew the ARR, you can't not do it. Amazing, and that's also basically [unclear]. Tell me, what did it look like, how long did it take end to end, this investment? Okay, so it really depends on what size organization you work in, but in an enterprise like that with 11 product groups it took quite a while. I think end to end, until it was embedded, it was nine months. That's quite a long time. The first pilot with the first group took three months. Meaning, when you have dedicated people, when you have an analyst and product who are with you and want to run with you and see the impact of it, you could already see results after three months. But it takes time to embed these things in an organization. Sure, like anything in a big organization. Wow, I also suddenly remembered who connected us, Lior Gerson, right, whose episode we'll also put in, a great guy. Very right. We worked together, of course, that's where we know each other from. We were at a company called Sibo, he was CTO and I was VP Product. It was fun working together.

Eyal David: [43:05] Excellent. So toward the end, it was a fascinating episode, we're moving to, like, the closer, and I wanted to ask for a second before that what things you're working on, like, maybe we'll give it a stage for a second.

What Rachel works on today

Rachel Frenkel BenHanoch: [43:14] Okay, I think the thing that's most fun for me to do now is to use all my years of experience and help companies, basically, improve their product management practice. I come into companies that have product teams at different levels of seniority and experience and help them, both with those wow moments of "wow, I didn't know you could do it that way," and with optimizing the product management work with their adjacent groups, and with tools, templates, like, let's give you an out-of-the-box method that will work best for you. And I simply do up-leveling for these product teams, whether in product or in product operations, and it's great fun. Where do you feel more need these days? More need inside organizations? Obviously implementing gen AI. Meaning, product teams are all struggling with the collection of tools and where to even start with it. Everyone's like, "wait, where?" So it helps a lot that I come in and say, let's find your low-hanging fruit. Meaning, where can you, I've already done several such workshops at several companies, where it's worth implementing in a way that will give you impact fastest, with which tools. It also depends on each type of organization, what environment it works in, whether it's Google or Microsoft and so on. But you can find the tools, the low-hanging fruit, and see results very, very fast. That's fun to do. What do you wish for yourself a year from today? Wow, first of all, I simply love doing what I do so much that I say, let's do it. I really love seeing the shining eyes of "we did it," and that we're doing it better, and that the product managers feel empowered now. So it's fun for me, whether it's someone who says "I want to be a product manager," whether it's "I'm now a product manager and I want to be a team lead now, what do I need to do to become that," or whether "I'm simply giving much more impact in my organization now." So I want to keep doing what I do.

Eyal David: [45:00] So I wish you that. Okay, so come back to us in a year and tell us if it worked out. I'd be very happy to. I'd be very happy to. Thank you very much Rachel, bye for now. Thank you very much. Bye bye. Hi friends, thanks for listening. If you found this podcast valuable, you can subscribe, follow us of course for more episodes on Spotify, Apple Podcasts or any other app. Of course, if you didn't find us on some app, I'd be glad if you wrote to us. We'd really appreciate five stars on every platform, and for you to follow us so more listeners can discover us and find the podcast. You can also find the previous episodes on any app or on the YouTube channel, we have links in the description. Until next time, yalla, be effective, and bye bye.