Transcript: "I Didn't Go In, I Went Up": How to Turn a Service into a Product | Orly Amrany (Justt)

Opening: monday's rebranding project and the secret of execution

Eyal David: [00:00] All good, I'm really happy to be here. Wow, it took a while, and I'm also really happy we're here. Joining us is Orly, who's CPO at Justt. Let's go, introduce yourself a bit. Actually, just, I'll say maybe start with the monday thing, I was really curious about it, because a second before this we talked about it.

Orly Amrany: [00:16] So I'll just tell the anecdote: I've known Roy, who's the CEO of monday, since the Wix days, when we sat together in a room and it started there. Over the years we went our ways, I ran into him on the street, and he'd just told me they wanted to change the name and that it was a very big project, and I happened to be, at that time I was pregnant and doing a few freelance jobs, and I said, yalla, I feel like doing a project. One thing led to another and I came in to do the end-to-end rebranding project, with everything that means, and it means a lot, and it was one of the things I think I enjoyed doing most up to then. It was completely different from a lot of other things I'd done and... So it was dapulse, and then suddenly it was on the Ayalon highway, right? Yes, and from the moment we really started until the moment it was a billboard on the Ayalon, after everything had already happened and we'd migrated all the users to the new name, the new domain, it was execution. Listen, they're animals there. I think four months, four, four and a half months. I've never seen execution like that in my life, really. It was like wow, it was wow. It was that feeling of everything, down to the smallest details: acquisition and the domain and what it means, and A/B testing on the, and how it affects SEO of course, and the product itself, like how users move over, and the best experience, and the communication, and so much more, and the little gifts they send afterwards, swag. It's so broad, and the whole company, they were about 100 people then, the whole company more or less worked on it, but it was execution. And were you even in product? Yes, and I was like, but I also came to manage this project. On all the aspects: when to release what, really when to tell the users who are more advocates, when to tell the community, when to announce, really building the flow of everything so it wouldn't create dependencies and so on. And in the end it really was, they did Pantheon-level execution there. Because, like, you're a person, I'm just hearing this,

Eyal David: [02:21] you're a person who loves systems that run like clockwork, you love, like, running events? Yes, I really love it,

Orly Amrany: [02:27] I'm also very operational by nature, and there was something very fun about it for me, and mostly being with people. I think it's simply the organizational culture. It doesn't happen at every company that something like this runs like clockwork. Because in the end it's not because I'm an excellent project manager. It's because there are people there who take ownership and simply do things end to end. Meaning, in the end, I think it also taught me a lot about when good execution works. In the end, when you go to a developer and there are tons of assets to replace, every banner, everything, he doesn't wait for the project manager to bring it to him; he goes to the designer and says, hey, I'm stuck on this, I'm missing an asset.

Will we see more company spin-offs in the AI era?

Eyal David: [03:04] Ah, it runs like clockwork. So tell me, I remember, we'll continue with the podcast, but a methodical pause: I remember they really came out of Wix. Do you think that's something that's about to happen a lot now? Company spin-offs? Interesting question, because now, especially now, everyone can do things on their own and fast.

Orly Amrany: [03:24] I think a lot of needs start out, you feel the need as a company. When you're working on it and you say, hey, I don't exactly have a product of my own. And a lot of companies with it in their DNA, like Wix, it was totally build, building on your own, so you build along the way. I think it's still not simple, because in the end you still need to raise money,

Eyal David: [03:44] and at the end of the day, even if you're not cyber, raising money isn't that trivial. Maybe there'll be a lot of bootstrapping now with... No, totally, but really, as a company, A, you're really aware of the needs. You experience the problems. B, you have people you can maybe now put on specific bets, right? Yes. Like, there's actually a good chance things like that will happen.

Orly Amrany: [04:07] I think the question you're raising is also a broader question. Almost every SaaS company today is talking about it and saying, wait, so they'll build us themselves, they won't need us, because everyone will build in-house. Nice, like, the build vs. buy that always existed, now it's easier to build, because at least the... I'm not talking about maintenance afterwards, but at least the initial build is cheaper and faster. You can also build it AI-first, and then... Of course. You still have to maintain it, right? There's still something that costs the organization, and that's a question, right? And maintenance, yes. I can't say. I think it's still very much a company's DNA. Meaning, some companies build everything in their DNA. A bit like Wix, where we built a billing system, which, you know, isn't obviously the core of the business, and they built HR systems, and built things and so on. But it's an interesting general question whether it's more, maybe the build vs. buy DNA will shift a little, it'll make it easier for companies whose DNA isn't building in-house to do it themselves. It probably, yes, depends heavily on the founders themselves, like whoever ultimately owns the thing. OK, but one second, you made an interesting segue to billing, so really, maybe introduce yourself a bit. So I'm Orly, I've been in product management for many years. I came from development; for many years, from the army, I was in Mamram, and I studied computer engineering at the Technion, and that's how I gradually rolled into the product industry. I think back in those days, I feel like an old-timer, or a problem, back then there was no product at all, and it wasn't very obvious what the value of product was once it started. Especially, I think industries started from heavier industries. Meaning, really, I was at Microsoft, where we did a firewall, and at Check Point, where in the end product, its role wasn't very well defined either. A lot of the time it would just bring requirements, you know, from B2B, from the companies, and was less really product as we know it today. And that's how I developed with the role, I learned what product is, what value it brings, an approach I came to on my own. It's very, and you've been at lots of companies, we'll put your LinkedIn afterwards, if you want, and they'll probably want to get in touch with you. But you're saying, like, how the role was, and now it's changing even more, right? Like, what are you seeing now? Really, it's a lot of conversations I'm having right now with lots of friends in the field, to understand how each one is approaching it, to get ideas and learn. I totally saw it change over time. Meaning, it started, if I look back, you know, somewhere in the early 2000s, when I myself, as a former developer, searched for the value: what will I, as product, bring to a product that's very technical? How will I bring value? What do I write? Specs for the drawer? Do I write documentation? What do I do? And I think only when more apps and internet companies started, you also saw that product people don't necessarily need a technological background. And then suddenly, I think, the understanding of how product thinks, that it's actually much more entrepreneurial, it looks much more at what the value is, what the impact is, how to really prioritize things correctly, how to do things fast just to test. Meaning, that whole package, which is much more than "now let's define something". And conversely, I think definitions became much less important. There are no more PRDs, people make mocks, and in the end it's supposed to explain itself. It changed into a mechanism that's much more of an entrepreneur who pushes. And actually, by the way, now with AI, I was just talking about this with a few people, I feel there's some return to more precise definition, really writing the definition. Because now, when you want to explain yourself, not to a person, who obviously understands things without you explaining, there's now a machine that codes. And if I don't explain all the context to it and give it everything, it won't do it well. Because yes, precision today is really everything.

Who is Orly: from development and Mamram to product management

How the product role changed, and why precision is back now

Eyal David: [08:04] Yes. So I have to ask you, but are you also talking, say, these are conversations around me, about what happens when AI decides, right? Where am I, where does my role change?

Orly Amrany: [08:14] Is that also what you talk about in your circles? Yes, that's exactly part of what I'm doing now, conversations with all kinds of friends and VPs of product and CEOs too. So really, the question isn't, I think everyone is feeling their way and trying to see. I don't think about where it'll replace or not replace, I think about where it can make things easier for me, where it can help, where I can do better, where I bring myself and my abilities into a situation that, fine, you need to go with progress and not actually... Exactly, exactly, for sure. But I'm really, really curious to know what things you're hearing around you, what you're talking about. So first of all, right now, at least in the surface-level things I asked about, I haven't yet had, I'm just, tomorrow I have a conversation with Shirley, who's the CEO of Shapes, whom I also met at monday, about it. So I don't have many insights from there yet. But meanwhile, the things I mostly follow on LinkedIn are really usage: how you use it and what you do to make product work more efficient. So a lot of what I see now is really about productivity, meaning how I now take all the insights from Gong and turn them into this, how I build a flow in n8n that does something, or how I break a requirement into small stories with Claude. Mocks, of course, much faster, faster prototypes. Those are the things most people talk about, with all kinds of different tools and so on. You just touched on it earlier, that it's more around entrepreneurship and those areas. So I think there are things where I still haven't cracked whether AI will affect them or not, and it's actually not the work itself of let's produce a faster, better definition, but simply how you push things inside the organization, how you ultimately set the right priorities. That still hasn't, at least I haven't heard of the use cases where really AI or some agent, fine, obviously it can do it, it does analysis for us from all kinds of systems, from the support system and from Gong, and try to prioritize for me what you heard more of. That still hasn't given me insights where I said, wow, here, I don't need to think, it tells me what's more important than what. Because in the end, especially in the products I look at, which are more complex, a lot of the time it's not something the user asks for. It's not something someone asks me for and then I do it or don't. It's something much deeper: I understand a process that's a bit broken, and then I do something more correct, product-wise. And that's why I feel that here, people who can really think creatively still have

What product managers talk about today: where AI helps and where it doesn't

Eyal David: [10:52] the advantage over AI. Yes. Well, I hope you'll come back in a year and we'll continue the conversation. So we just dove into... I was wrong, actually, we have no advantage. I really, we dove, we dove really deep for a second. Let's take a breath for a moment, go back, and tell us a bit about Justt.

What Justt is: the war on chargebacks

Orly Amrany: [11:11] So Justt has existed since 2020. We've already raised 100 million dollars in three rounds, we're about 120 people, and we basically help merchants, meaning anyone who sells something online, broadly, in all kinds of industries, whether e-commerce, travel, every industry that sells something online, fight a phenomenon called chargebacks, transaction disputes. It's very common, by the way; in Israel I think they make it a bit harder for us, so it's a bit harder for us to do it here, though it's still possible. But basically anyone who bought something can call their card issuer and say, listen, I don't recognize the transaction, it's not mine, it wasn't me, I didn't get what I wanted, all kinds of claims, and basically ask for a refund. The issuer usually returns the money pretty much immediately, meaning in this case the merchant is guilty until proven otherwise; the default is to take the money from them. It's a very big loss for merchants, it can reach between half a percent and one percent of their entire turnover, that's a lot. Not to mention the fees, which are 15 dollars per chargeback, which is, like, sometimes you have a chargeback on 4 dollars, and the fee alone is 15 dollars. It depends on your business. So they lose a lot of money on it, and a lot of it isn't legitimate. Meaning, a very large percentage of chargebacks, for various reasons, aren't legitimate, and they have tons of manual work around it, not to mention that Visa and Mastercard can shut you down if you have too high a percentage of chargebacks. So what we do is a bit like a lawyer: we help them fight. We basically submit a statement of defense that proves the case: the user said they didn't receive it, and here we prove a tracking number, they received it. And then we get them their money back, we recover their money.

Eyal David: [12:59] And there's a lot of it. And it's, yes, obviously it's great, and you also sit on that great spot, at checkout, where money flows, and going forward it also sounds very logical. If we talked here for a second about AI and agents, on one hand they say purchases will go up, right, and even become easier; on the other hand it also sounds like a lot more businesses are going to open, right? And agents will buy freely without supervision, so it'll be even more like that. Right, and on the other side more businesses will open too, meaning there's a trend you've been seeing since 2019 or so. And I wanted to ask, wait, how long have you been working there? It's almost three and a half years now. A very fun period.

Three and a half years in: scaling without inflating operations

Orly Amrany: [13:43] And what have you seen, like, from when you joined the company until now? First of all, it's a lot of fun to work at a growing company, because really from the moment I arrived, just three years, we've roughly doubled ARR every year, which is really amazing. And what's always amazing to see is that we handle a crazy and growing volume of chargebacks, but the amount of manual operation we do around it has kept shrinking. That was one of the significant things, I think, that also enabled our scale. Meaning, it wouldn't have been possible to scale the way we used to work. And there's something that was nice to see, at least for me: there's something called automation, you come in, you see people doing some process, and let's automate the process. And there's something that says, wait, you don't need to give people tools to do it better, let's simply build a product. Let's think about the product. Yes, let's build a product that does this thing on its own. So that was something very interesting, and there was a real challenge in it. Because in the end you try to see it a bit like AI, but how people think, bringing the logic inside, and there are tons and tons of use cases, and how you don't get burned. How people think, meaning,

Eyal David: [15:06] the agents on your side, basically reverse engineering. But wait a second, before we, just to understand, you talked about scale, so your ICP today is large companies, could it be anyone with a store, like, what does it look like? Broadly we work mainly with enterprise,

Orly Amrany: [15:21] meaning that's our ICP, very large enterprises. We work with very large companies, like Meta, Spotify, really giant companies, DoorDash. On the face of it, it could work for any company, yes, there's no real difference or a different machine, but in terms of the investment, and you know, in the end there's CS and there's support,

Eyal David: [15:42] in terms of, ultimately, the investment, we work on... So when you talked about scale, at that point three years ago, you had a lot of American customers, I assume? Yes, there were always a lot of American customers, and the ICP moved up, meaning,

Orly Amrany: [15:55] in the end the customers kept growing to six figures, but you know, there were always relatively large enterprise customers. Even 300-500 thousand dollars is relatively large.

Eyal David: [16:09] I'm trying to understand the bottleneck that existed. So basically, to keep serving and getting more of this good stuff, more of these guys, you'd have needed to either hire a lot of people, yes, lots of analysts, and keep being a service company, basically,

Orly Amrany: [16:24] yes, or really, basically for everyone who needed to go live, they'd do a kind of adaptation. Meaning they'd say, OK, these are the templates we have, let's duplicate them, understand its nuances, understand its flow, change things inside the template. It was still, like, the sending and so on, a lot of things were automatic, of course, not everything, but some was automatic. But it required a lot of work, including manual work, which was very error-prone in the end. You change something in a template and now something breaks; now you want to change something you found that's wrong, go through everything now, each one has 15 templates, and you have, thank God, already a few dozen customers,

Eyal David: [17:01] it starts getting complicated. Complicated, and it would also cost you. You probably saved a lot of money with this move too, obviously, but also now to really become, at scale, a company,

Orly Amrany: [17:14] most of these people sat in Israel, how would that have looked to me? Yes, most of the people sat in Israel. Doing that kind of hiring in Israel was very complicated too. It was, and teaching them. In the end I think mainly, by the way, a lot, it's funny, because I keep... people say yes, obviously it's far fewer people, much easier, much faster, and so on, but in the end I just think about the amount of human error. Now, you work with enterprise, they find an error in a document, oh dear, you're not serious, you come across as unserious, you can't work with companies like that, and every so often a different error in a document. And that's something that happened, you're saying. Of course, when someone works it's interesting. What was the error rate? You probably measured it? We didn't even know how to measure it, believe me. Because every so often we found something else in some document, some error. Now, in the end, when you build a machine, there are errors at the start too, but then you fix them, at least you're done with it. Like, you fix one thing and it doesn't come back. You discover more things, and to this day we now run validations with AI to find errors, but at least I know that once I fix an error, I've fixed it. It's not a template I touched and broke something else. Right, you're quoting, yes, I really understand.

Eyal David: [18:16] So a second before that, really, there's also the other side, AI, which you probably brought in, like,

The AI layers: document generation, evidence quality and unstructured data

Orly Amrany: [18:24] where else does it show up in the company today? Where else did you become more efficient? So it was, I looked, there are several levels of efficiency. Meaning, first of all, really the creation of the document itself.

Eyal David: [18:34] The second thing, which is often also related to documents, is even their quality. What do you mean? Once, there's a lot... When you say document, I understand there are templates they worked with before, an actual defense document.

Orly Amrany: [18:44] But was there also some output for the customer? Like, are these two types of documents? There's one document that's submitted to the issuer to read, and hopefully the guy in India on the other side, who reads it in three minutes, and who knows if he even knows English, is supposed to approve it or not. But there's also the customer hub, where the customer can really see what's happening, understand their statistics, understand what's happening and what data needs to be brought, and so on. So the efficiency was, if we go back to what things: first of all, it's really the creation of the document itself. Another thing is even the quality, in the sense that there's a lot of evidence. For example, if a customer talked to support and told them, listen, I don't like this, and then said I didn't receive it, you probably understand you have a good case. But that's something unstructured, meaning, how now, images, a photo showing you received it. By the way, a lot of delivery systems give you a photo, really. So a lot of things that until now were hard to extract and understand automatically, unless a person looked at them and stuck the photo in, AI can now help with. So if you went to DoorDash, I have a photo, maybe a location, and I can cross-reference it. A photo of the food by the door. Yes, that's what you said, exactly. Yes, yes, really. So basically the ability to really take unstructured data, whether images or communication, conversations that took place, and extract what's needed and prove it, in the end improves my ability to win at scale. Meaning, obviously what I used to do manually, if someone now goes through one by one manually and extracts, they can still do it. But it's not possible at scale. So also not for the same merchant, I'm not talking about us cross-merchant. I'm saying... OK, well, yalla, tell us.

The bottleneck: a service that doesn't scale

Eyal David: [20:32] So you basically arrived at a situation like this. Again, there was probably tremendous growth after COVID, that's why the company grew so fast, they wanted as many customers as possible, as much as possible, and we basically reached a bottleneck. OK, how do you define the problem? Again, you said you could look either at improving what's happening or at broadening the canvas, so how was it? So I think when I arrived,

Orly Amrany: [20:57] I have to say that at first I looked, you know, I looked, I looked from the side, and I said, some of the things were already automatic. Meaning, there was already the customer hub, which was automatic, the templates were generated automatically and so on, and I said to myself, OK, you did nicely, so what's going on, what do you need me for, what am I doing here? And then I started a bit, yalla, I really sat, sat with the guys in operations, how they do the templates. I understood what things they go through, when they know every merchant who goes live and look at the flow and so on, what they do, what they change inside the template. I simply looked, examined it, and all the time I said, fine, OK, in the end it's kind of rule-based, there's logic: if it's a physical good, obviously you need to prove delivery; if it's a digital good, you need to prove provisioning. Fine, and there are, I understand there are tons and tons of use cases, I understand the complexity, but... It's managed, we can talk about good. Yes, but why isn't it simply something written in code, and clear. And I think gradually, I started, not that gradually, it was pretty quick, but a month or two into it, I said, OK... Where did the penny drop for you? Actually pretty fast. In the end, you know, I sat with them, sat with the people who did it, tried to understand the complexity they really have, and they worked with a thousand pipes, a thousand things, wow, you know, at first I said, God help us, what a nightmare. And then I said, why do I need to go inside, when I can not go in, go up a level and say, fine, whatever logic you have in your heads, I'll sort it out. I know how to build a machine that does it on its own. And I think the way, I immediately laid the idea out as some mock. I love drawing right away; I didn't have AI then that could draw nicely for me, so I made something ugly and said, yes, what's the problem, let's do it like this and this, and here, there are a few logic rules. I put in the simplest things I'd heard so far and said, anything that comes up I'll add on top of this, this is the base. And I started running around the company, also to get buy-in, especially from people who understand the domain very well and know it. We did a workshop, we went over roughly all those arguments, there really were hundreds of data points and arguments, to understand what they had now. And for everything, all the time, I thought: does this fit my pattern or not, how do I fit it into the pattern. And that's it, that was the first thing I did. I said, that's it, this is the thing we do. Forget everything else, this. And we really built it, it took us in total, really end to end, from the start, from the definition, to the end, eight months to stand up the new machine, including migration, which wasn't simple, moving all of this over there, it's a big deal, but we ran it like clockwork,

The aha moment: going above the complexity instead of diving in

Eight months to stand up a new machine

Eyal David: [23:53] and from there life was better. Wait, one second, one second, one second. What were the challenges when you approached something like this? In the end you're talking here about taking a service and turning it, to productize it, right? Turning it into a product. So the challenge, the biggest challenge, I think, was really,

The real challenge: working with people who hold all the knowledge in their heads

Orly Amrany: [24:11] in the end, for it to come out well, I need to work with the people who have the knowledge in their heads, and how I actually understand it, and they need to confirm for me that what I'm doing now meets the need. And that's very, very challenging, because with people who don't have that thinking, who aren't in that headspace, their mindset isn't a product mindset. Now, obviously there are tons and tons of use cases, and it's very complex,

Eyal David: [24:39] but the moment I come and present a concept, they say, yes, but here I have a use case like this, and here, here, and here, here, and it's impossible, it's specific to this, I have to understand that... So let's sort it out for a second. I'm probably imagining you in some meeting, presenting a few buckets, a few categories, which AI probably crunched for you in the end, and you concluded it's four categories, and the debate is that not everything fits in. Yes, so when people actually sit down now,

Orly Amrany: [25:07] there are working sessions, and with people, they say no, but it's not exactly like that, because there are cases like this and that. And I say, OK, so bring me the cases, I want to see them. Because in the end, what happens is that everyone always remembers some case, everything is very theoretical, they tell you, yes, there are cases. OK, what do I do with the sentence "there are cases"? Bring the case, we'll look at it on its merits, handle it, see how it can be brought into this, how the capability can be improved. And that's what I did: in the end I made them sit one by one on it and bring me use cases, bring me cases. For me, right now, this is the vision, and this is what's going to happen; bring use cases that contradict it, that's the goal. And then as they brought use cases, we even did joint thinking, it was a real brainstorm, on how it can be solved product-wise. A, everyone enjoyed it, because you know, people who aren't product are suddenly part of the thinking process. And in the end it took me time; at first they were a bit, there was some resistance, like, wow, it's impossible, impossible, but at some point they got really excited about it, because they saw, hey, how fun, everything suddenly falls into some order. So they were very pleased with it. And it really was a success. I won't tell you I didn't have fears at first. There's something scary about coming in, you know, to a domain where I don't know all the details of exactly the rules and the... I know it well enough now, but at first I came in not knowing much, I learned as much as I could, and in the end I said, hey, maybe this really won't work. Maybe I'll now do damage to the company, you know, I'll lower their recovery rate. Well, at the end of the day, like you said, it's not ripping off a band-aid, right? You did, tell us maybe how you rolled it out. Yes, so you need to be very, of course I was very gentle with it, it scared me a lot, and I really needed to get the buy-in of the people, including Ronen, who's the co-founder and a real expert in the domain. It was very important to me that he see it and challenge me on it, and that I feel confident this thing was going to work. Again, in terms of the domain, basically, at the end of the day. In terms of his expertise, yes, in terms of the expertise. In the end he'd challenge me in the sense of giving me use cases, because he really was an expert, and if I tell him, listen, now it's only going to be like this, you can't do in code,

The CEO's buy-in and why they went all in

Eyal David: [27:10] something specific for a particular merchant, you can't. How will it meet, like, give me examples that really... Like, on the alpha in the end you navigated, I'm just saying, 40-80 percent to that area, and everything that's an outlier, could you still keep handling it like before? Alpha.

Orly Amrany: [27:25] We could have, but I didn't want to. We did a 100 percent migration in the end. Maybe except for one customer, the first customer, who was already really complex and had manual things done for them, maybe manual to this day, I think. Hard to uproot, right? Yes, the only one, by the way, the only one. We have hundreds of merchants today, and we moved all of them over. OK, yalla, so tell us what happened as a result. OK, fast forward 8 months. So fast forward 8 months, we basically said, OK, we're ready. We did dry runs, meaning we saw how it generates our first document, did manual comparisons, a migration plan. Now, a migration plan, let's say, I started the migration plan even before the product was ready, because to me that's the most important thing: understanding how I even know this thing works. So we really made a plan, a testing plan: what we test, what's needed, what the steps are to do a migration, first one merchant, for one customer of ours. So we made a really clear plan for them, tried it on one or two, and then, after we understood the steps, after we understood how we test it, we said, OK, from here. And of course we made sure we'd also have an A/B testing system, meaning we'd first open it 50-50. We thought a lot about whether to start with someone relatively small, to break it in on them, but in the end someone small would take me years, so we said no, we'll actually go for the big one, take a risk, open a week with enough traffic. And remember, the worst thing in chargebacks is that the answer comes after 120 days, right? So you open traffic and now wait 120 days to see what happened. It was very challenging, but at some point we also did manual checks, opened documents, said, OK, this looks good enough, we trust it. So take us a bit to the results, what exactly came out? So broadly, the truth is we did an A/B test on two big merchants, and we saw the results were more or less the same, maybe slightly better for us, but in my eyes it wasn't significant enough. But that's what mattered to me. I said, first of all, don't break anything. Now, after we managed to do that, there's an opportunity to improve a lot more. Why is there an opportunity to improve? Because now I can say, OK, now I think that if we put all the evidence we had, if I have images, put all the images not inside the text but at the bottom, or at the top, it's easier for them to read, so it probably improves our success. Then we tried it and it worked, and then across everyone, all at once, no need to change templates, the system does it on its own. OK. Or whatever, changes we now make to everything. We discover another use case we suddenly found with merchants who can do, they say, we sometimes refund, but not as a transaction, we credit you within the app. Now, once a merchant has refunded, that's total proof, otherwise they'd have been charged twice. Now, that's, say, a use case we didn't have, because nobody knew they refunded in credit.

Migration plan, A/B testing and a 120-day wait

The results, and why once you haven't broken anything you can only improve

Eyal David: [30:29] It's like you said, unmeasurable things. But one second, we're running for a moment, I'm trying for a minute to understand. OK, now eight months, a year passed, OK? What did you actually see already?

Orly Amrany: [30:45] So first of all, maintenance really became, wow, significantly easier. Really no longer, by the way, the issue of how long it takes us to get a merchant live was much less of an issue already. Right, that's also something we didn't talk about at all. Right, there are, by the way, other issues, and you gradually move on to the other things, but in the end there are issues of how you tell them what data to bring, and what data you need, and what, to help them bring better results in the end, and that's also an issue to solve, and so on. Just in your case, how does it work?

Eyal David: [31:21] It doesn't matter, I'm... Always in Hebrew, I'll go back and forth. Yes, it speaks, it addresses women and men alike.

Orly Amrany: [31:29] So basically going live went from being a very big part, which people talked about, how you even produce the configuration and so on, which has now become nothing, the configuration, and now the part that takes more of the focus is basically telling them, OK, first of all what data we need you to bring, how you'll bring it, and... And tell me another question. OK, so I'm setting that aside. Fine, there was an increase, we understand. Do you communicate it to customers at some point? Because, like, it's your IP in the end. That's true. So let's put it this way: because it's something internal, there was no communication to customers saying, listen, you're now on a new system. Because you know, you say, OK, what did you do until now? But also forward to sales and so on, it's not something you need to brand it, you're the queen of branding. So what we did brand, and in my eyes it really was, I still think it's unique in the domain: the whole industry started with manual handling, and then semi-automatic handling with templates. That's what everyone does today. We said, and this we did brand: we have no templates. We do a dynamic document. Meaning we do it dynamically, every case has its own, every case: why exactly the user is complaining, and what I can say about this specific case. And your ability to build it dynamically, because a template really limits you, there's nothing to be done. There's no way around it, in the end you're stuck with something. So that's the thing we did brand strongly, the Dynamic Document, which, by the way, some competitors in the market later copied, because it really was innovative. Because it's something where talking about it being built completely dynamically, now, in the domain, in the AI era, seems trivial, but three and a half or three years ago it wasn't. And that we did brand. Meaning we didn't say we have a new system or whatever, but simply that we do a dynamic document. It was... That's exactly what I was looking to hear. And it was probably very well received. It was totally our elevator pitch. Because, God, what's the difference between you and that? With us everything is dynamic, it's tailored, it's tailored to your case. It's not a template of... Every child is special.

The "dynamic document": the differentiator they branded

Eyal David: [33:40] All of us, yes. We're all God's children. And basically now, so you're saying that you, OK,

Orly Amrany: [33:48] you broke down the significant thing, and now you're basically AI-first, that's your approach? Yes, now we're totally that in a lot of things, especially in the new things. I'll give a few examples. Because basically, say, the next challenge we had, meaning after we cleared this hurdle, was that we understood that what most affects our ability to win in the end is what specific information we bring. Because with all due respect to texts, in the end it's proof: I need to prove they received it, I need to prove it's not fraud. And to prove it, there are a lot of pieces of information that sit with the merchant, and I need them to bring them to me. And I think on one hand there was a lot of debate at first about how much it matters, and whether it matters, and it's very hard to estimate. It's from a lot of examples we saw and analyses we did. Meaning, we took merchants and wanted to help them optimize, and we did work on them, and in the end we told them to bring more data, and we saw that this is simply it, this is the thing, there's nothing else here. And our ability now to come and say, OK, a merchant goes live, I know the merchant well, I know their business, I fill out a business profile for them, because everything is automatic now. That onboarding, that onboarding, yes. Which is probably flying now, right? So the technical onboarding, meaning getting them live, flies. But, and I can, by the way, every merchant who comes in now, even on their own; we also thought at first about self-serve, it's totally possible, meaning I come in, connect the PSP, put in the API key, it works. Yes, it's a type of product that connects less for me. No, fine, by the way, we have competitors who started from SMBs, Shopify-style, where in the end you say, yes, I can onboard myself, I come in, connect, the PSP connects, the thing works. Now, it works, that's true, but to bring the extra mile of the recovery rate, or of our ability to win, you need data in the end. And my ability to tell you exactly what data you need to bring, and what data each case specifically needs, isn't trivial, and that, say, is what we really want to solve with AI now. Because until now what we did is, we have a machine, and there's configuration, and we say, OK, you're such-and-such a merchant, this is what you sell, these are your things, here, take 40 data points, good luck bringing them to us, it'll be fine. And in the end we discover that this small tweak, one more question I ask them, one more piece of data, is what makes the difference. And lots of features around it: if you suddenly have chargebacks from the same consistent disputer all the time, or a very, very large chargeback, on that one you'll bring the photo.

AI-first: automatic onboarding and why it all comes down to data

Eyal David: [36:17] So how does it actually work? Yes, I understand. So how does it actually work, you're not a system people log into actively on a regular basis, so how.

Orly Amrany: [36:26] So yes, right, unlike a lot of other products I've built, this is a product where in the end their goal is: give me the money. It doesn't really interest me, I want the money back. So I look at our customer hub; in the end I have users in every organization who usually log in to do two things, or to upload data to us if they don't do it via API. So again, it's checked, how does it work? Right, so we have several ways. The way we prefer, of course, is that they bring it to us via API, and then it works automatically. But some companies don't want to invest in an API, and then we tell them, OK, they have some process there for a CSV, and they upload it to us every week for all the chargebacks, you know? There's an operation on the other side that uploads the data, and that's it, and then we can generate everything and it works. There are other ways: we can connect to their systems, we're now connecting to billing systems, to CRM-type systems, and bring the data ourselves. Meaning, then they don't need to do an integration, we do the integration. We're basically trying, my goal is to bring as much data from them as possible and make it easy for them to bring it.

Eyal David: [37:32] Totally clear. And that's suddenly also, I also love the pricing of this product, everything is really simple. It's all win-win, which is the most fun. Yes, it's amazing. I wanted to ask, in addition to that, we touched on it a bit at the start and I'm a bit curious, because we really talked about it in a prep call we did, we did a few, we have much more time, war and all. So do you have some recommendation for what you do, or what you suggest doing, to stay relevant? For product or for companies? I think take it generally, because you also detailed it just now, when you talked about what your challenges were. You talked about, I'll call it, I heard it, at least to my ear, naysayers, like we came to do something now and you had a kind of block from all kinds of use cases, just, that's how I heard it.

How to stay relevant, for product and for the organization

Orly Amrany: [38:28] I think it's a difficulty that exists when you come to an organization that isn't product-oriented and is service-oriented. It's a state of mind. It's really people who find it hard to think, not in a bad way or resistance; I actually don't see it as naysaying in a negative sense, but really in the sense that they can't think differently. They know all the bits and bytes, and they're so immersed that they can't for a moment go up and look at it from above, also because they don't have the... they haven't honed that skill. Meaning, in the end that's what product does all the time, so it's already honed, and also some people have it more in their nature and some less. So I think the difficulty is first of all that in the end it's a company's DNA, and that was also one of the biggest challenges, because in the end if a company's DNA isn't product DNA, it's very, very hard. And I think at Justt, the thing that most, it also scared me a lot to come at first because of this. I'd also had less good experiences with companies that aren't product-oriented, and it wasn't successful. And I think really Ofir, the CEO, who is very, he's a technologist himself, and people who in the end, that's also their state of mind, and they're all in on the approach. And Ofir was totally all in, gave me a free hand. I really made a very big revolution there, and you know, there was no fear. He said come on, yalla, we moved the traffic, we did it, we really went all in on it, and that's also what enabled it. Because if there had really been fears and concerns and they'd stopped and said let's do A, this migration could have taken years. So I think... I think that character, when you understand there's no other way, you said it earlier. Yes, but it's also a state of mind, there are people who even when you understand it,

Eyal David: [40:12] I'm afraid, and I think it's a lot about state of mind and courage. But again, if we take this, bring us back for a moment, if we bring it back to some concrete advice.

Orly Amrany: [40:22] Advice for what? To turn a company, I think... again, there's the AI part and so on. I think in the part about how to turn a company from a service company into a product company, the first thing you need is simply to go for it, not halfway. Meaning, if you go for it, you need to know you have the buy-in of the CEO, of whoever's needed, of whoever ultimately makes the final decision, to press the button.

Eyal David: [40:48] And also, you left a lot of cushions. At the end of the day, even if it had crashed, you left cushions, right? Cushions in what sense? Because you left it,

Orly Amrany: [40:57] I thought about it, that if we need to go back, then OK. First of all you opened it up, decided, had discussions about which customer to open it on, how to open it. Of course, you do it responsibly, because I can't crash on that. Exactly. By the way, more than that, we could also go back to the templates we had before. Meaning, I knew I had a way back. It wasn't like destroying things. It was really done responsibly. But I think still going for this move, both the investment and going all in, and also when the organization isn't, like the people around you are less product-oriented and much more busy with their service, the first thing is his buy-in. And by the way, also when I joined, I remember, in the early conversations I told him, listen, if you're not into it, I'm not coming to persuade, I'm not, that's a bad war. Like, if you want it, I'm coming to do it. And what else worked at the organizational level? Again, for the organization to stay relevant in this case, what else worked? So I think, first was really his buy-in, the decision. And second, at least for me, in the product department, also to really tell all the people who dealt with it day to day, to keep it real all the time. Meaning, you're talking about maintaining the templates. Yes, these are the people I worked with in the end, and it was important to me that they buy in, that in the end they're the ones I brainstormed with, I brought them feature ideas. Not because, not in the political sense that they'd be with me, but really in the sense of... because it's not just something else. Yes, I need to know that what I think is right really is right. I need their approval. Did you think about a timeline from the start? What did it look like? Yes, yes, there was pressure from the CEO too, by the way. Right. When it happens, by the way, I threw out eight months, it happened to come out exactly that, no idea how, but there was very big pressure to do it even earlier, and as fast as possible. And really, in the end I told them, listen, I'm not proposing anything you don't approve. You're my approvers, I need you to approve for me. I really gave them the power

Eyal David: [43:09] to veto something, on... And in your case, in the end, the cost of tokens and everything still doesn't matter, because you needed to break through and work at this scale. Look, we did a lot of things that we did

When AI and when logic: visibility into everything the system does

Orly Amrany: [43:26] and it also wasn't fully AI yet. There are very specific parts, very specific models that use AI, and there are also a lot of companies today that don't want us to use it. Meaning we have the ability not to use it, it's still fully automatic, working rule-based. You mean customers, so as not to send data outside. Yes. So I think there's actually an advantage in having started building it before, because the initial build is a kind of rule-based. And so the... and the... we brought AI in at the very specific places we needed. And in models and prediction, and there are a lot of things we can now really advance at a high pace. But back then, until we started, it was less AI. It was logic and so on, but it was, and everything is like, everything also has a UI, everything is visible outside. You know, the thing that matters most to me is visibility, because in the end you need to explain what you're doing, and support may need to answer something, and there are questions, and this thing isn't sitting somewhere in the code where nobody knows what's going on.

Eyal David: [44:21] And that's something that's... There are a lot of other departments here that were also greatly strengthened by this move, right? If you talked about support for a moment.

Orly Amrany: [44:28] So yes, I think in the end our ability to enable, to externalize everything we do, the logic. And by the way, that's still the logic of the document, which is more complex things, but there's even logic, the more, I'm telling you, in the end, every payment service provider, for example, has its own world of mappings. Go figure out its life cycle and how it works, really logic that only they know how they invented, usually with no documentation either, it's reverse engineering on the files to understand. So say this thing of mapping, which in the past was, OK, someone writes the logic for me in some Excel, and someone writes it in code, and the connection between the Excel and the code isn't clear. Now there's a PSP mapper that we wrote internally, where all the logic is written outside. Meaning, if I want to know why now this, why now they decided it was won, I look at the logic and I see why, because I see their logic. And that gives visibility, it makes much less, it lets me debug myself and improve and make it more precise, and it's really simple and easy. I understand, very much, yes. Gorgeous, how lovely, a really interesting story. Now going forward, so what. For the coming period, what's your focus? So basically we have, besides the fact that we, we have a whole roadmap for how to really use all kinds of AI components to really improve our recovery and our prediction. You mentioned agentic commerce a moment ago. Yes, of course, wait, that's one thing, that's the core, we still need to improve, we still have a long way to go and how to improve it and how to pinpoint exactly what we need for each chargeback and so on. Agentic commerce is something we're really tracking now, because once it happens there will simply be chargebacks and they'll need handling. We're really already ready in the sense that we're at least monitoring all the traffic right now to understand whether an agent made it or not. And basically the goal is also to really understand. Now we need to understand; a lot of things still aren't written, meaning it's still not known who's actually responsible for what, how we, also in terms of liability. But in the end it brings in a whole world of content we need to know how to support. Meaning, if the agent bought something for me, how do I know what the agent's settings are, and who defined the settings, and who's responsible for those settings? And basically, did the agent read the terms and conditions or did its user read the terms and conditions, and who approved, and does the agent approving it mean the liability is on whoever bought. So there's a whole world there. So you're really waiting for regulation here, basically. So there's also, we're also waiting for regulation and so on, but once there's regulation, it'll be very easy for us to adapt, because we're already bringing the data now, because in the end what matters here is data. It's A, understanding that it's an agent, taking all the agent's data, and from the data we have we'll already know how to produce what's needed. Like, once I've flagged it, I know how to handle it.

What's next: a roadmap for recovery and agentic commerce

Eyal David: [47:24] Yes. Think about it. It's nice to say who's responsible for the whole thing in the end. Wow, Orly, I'm glad we waited, because this turned out to be a really interesting episode in my view. How fun. Really. I want, so, OK, come back later and tell us how it's going. I think my main takeaway,

Wrap-up: from service to product

Orly Amrany: [47:42] is really around doing the, a lot of companies will now have to make this change, really turning from service into product, so

Eyal David: [47:50] we didn't do a playbook here, but hearing your experience of it was really interesting. Thank you. I enjoyed talking about it. Me too, yalla. OK, thanks. Bye bye. Bye, thanks.