Transcript: How to Build AI at the Edge of the Device — Ori Katz, Hailo

Host: · Guest: Ori Katz · Back to episode

· Updated:

This is the full transcript of the Product Builder episode “How to Build AI at the Edge of the Device — Ori Katz, Hailo”, hosted by Yael with Ori Katz.
To listen, and for the summary and key takeaways, go to the episode page →

Ori Katz, who leads the AI SoC domain at Hailo, tells the story of how an Israeli chip company became a serious player in AI at the edge — smart cameras, surveillance, industrial, automotive and robotics. The conversation (hosted by Yael) dives into the era's biggest challenges: how to compress large models onto edge devices, how to deal with price, memory and power, and why hardware demands far deeper product management than people assume — fewer flashy features, more hard decisions about cost, supply chain, quality and time to production. Along the way: the Raspberry Pi partnership as both a distribution engine and a quality stamp, the memory crisis, the gap between China and the West, and why physical AI and robotics are the strategic bet for the next 3-4 years.

In this episode

  • Every hardware company is really a software company: Hailo has ~30-40 hardware engineers versus 100-150 software engineers, because the people who buy the chips are software developers who want to run AI on them — not to fiddle with the silicon.
  • The heart of hardware product management is the 'unsexy' part: price, supply chain, lead times, power consumption and quality. To manage a product that sells millions of units a year you have to master these obsessively — not the flashy features.
  • The Raspberry Pi community (the AI HAT) is both a distribution channel and a quality stamp: it lets Hailo support tens of thousands of developers without an army of customer success, and becomes a base of trust and innovation the company leverages into its next products (robots, drones).
  • The big challenge of Gen AI at the edge is economic as much as technical: compressing a 2-7 billion-parameter model onto a device, amid a memory crisis where DRAM costs ~$150 — and who is going to add $150 to a smart camera? Edge wins over cloud on privacy, latency, power and cost.
  • The strategic bet is physical AI and robotics for the next 3-4 years, with surveillance as the market exploding right now; China is already a year or two ahead of the West in running VLMs in production for surveillance, at lower cost.
  • Cost reduction is itself innovation: delivering the same performance at half the price is what cracks a real market like security cameras, where competitors fight over every socket and you also need stable quality, lead time and supply.

Yael: [00:04] Hey, it’s Yael. You’re on Product Builder. We’re about to start the episode, but first, go ahead and click Follow on Spotify or Subscribe on YouTube and help us reach more ears. Let’s begin. Hey Ori, what’s up? Hey Yael, all good, how are you? Overall, pretty good, definitely, without coffee though. So Ori Shubi, Product at Hailo, right? Nice to catch you between flights, right? You just got back from where? I just got back from Korea, and before that I was in Taiwan. If I could, I’d connect them, but I have kids, a wife, life. Things like that in the country for years pulled you back here to record? Yes, exactly. So come on, I want to brag for a second. What does Hailo do? Maybe so we understand where you actually operate. Sure. So it’s not a company that was founded nine-something years ago.

Intro: between flights to Taiwan and Korea

What Hailo does: accelerating AI at the edge

Ori Katz: [01:00] We are basically a hardware company at our core, meaning we develop and design chips for accelerating artificial intelligence. That means we perform computations. For artificial intelligence, we bring them to what are called edge devices. We’re not talking about the cloud, not about Nvidia, not Grok, and all those cool things. We do other cool things. Trying to and succeeding in also taking all kinds of devices—cameras, cars, and anywhere there are chips that are within a framework of. Low operation, so to speak, but still being able to run AI at insane performance. Okay, we actually started with AI accelerator processors. Where I think our most famous achievement maybe is that we’re basically, for those who know Raspberry Pi and the whole maker community, we are basically the AI HAT—that is, the AI accelerator add-on for Raspberry Pi, and through that. We sell and support tens of thousands of people who are basically developing tons and tons and tons of artificial intelligence models, artificial intelligence pipelines. On our chips and running them in real-world conditions. So that’s basically, basically, basically what the company does. Later on we also added what are called chips for smart cameras, system-on-chip for those who know, SoC. And that’s it, and it’s a lot of fun and it’s cool and it’s a lot of AI and it moves at a crazy pace, but in the end, in the end, in the end, it’s also hardware.

Raspberry Pi and the AI HAT: the community link

Yael: [02:21] Hardware, and you also told me just a second ago that you’re actually. A software company, basically mainly dealing with that. I think before we dive into that, we should also touch on the use cases and what you do; that was also somewhat evident from your homepage when I approached it, where the roughly 80-20 is, so maybe you can touch on that too. But let’s take a step back. You basically started—tell me, you’ve really been at the company for a long time already. Tell us a bit about your journey at the company, what that looked like. Yeah, sure. So look, I came in—I’m an R&D person by background, and originally that wasn’t so long ago.

Ori's path: from R&D to product

Ori Katz: [02:53] I’m a chip and hardware guy; I worked at Mellanox before that, then Nvidia. I came eight years ago following a friend’s recommendation—doesn’t matter—to lead one of the hardware development teams at Hailo. My first 4-5 years there I just made chips, and that was a lot of fun. I really love making chips; for those who want to, there are endless topics to talk about there. I really came in through R&D, and one of the nice things about a startup, a small company, is that there are lots of opportunities to move around; by the third chip I was already leading as project head. And then basically the CEO said, listen, Ori, it’s time, someone needs to take this thing—there wasn’t anyone here—come, let’s focus for a second because I can ramp up in software. First of all, how small are you? Let’s start with that. Today we’re not that small anymore; we’re 250, 280 people. When I joined, we were 15. And out of that, just to give a sense of how a hardware company looks, there are, say, 30 maybe 40 hardware people. In development and another 100-150 software people. Yes, all hardware companies in the world are basically software companies, and nobody will tell you otherwise. Developing a chip is hard and long and so on, but developing software on that chip is endless, so in the end all the users, all the customers who buy these chips are people. People who want to develop software, software developers, and they don’t care—they care at the power level, how the hardware works, but they want to. They want YouTube, they want sports, they want AI things, they want all the things I assume your listeners know and love, or don’t love. And that’s what most hardware companies look like. We grew over the years; we really started at around 20, more or less, but when I joined we were 15-20. And basically we started with the first chip we wanted to bring to the world; that’s what I told you earlier—an accelerator, meaning. A small team of 15 to build a chip and another 15 to build the software for that chip, and basically we went and started making accessible, so to speak, enabling as much as possible. Use cases and resolutions and segmentations and detections, and all the coolest neural networks in the world are still cool in my opinion. And we basically competed in benchmarks against companies like Nvidia, meaning Nvidia was running in the cloud. And we basically brought those amazing things to customers who had never dreamed of gaining access to Nvidia chips or being able to run on them. The power that Nvidia consumes and run those. Those same applications, just in tiny little boxes like these—really, really cool. And you did that from the first part, let’s call it, of the company; that really was the fact that we work closely with Raspberry Pi. That collaboration is really a big deal; it’s mainly a stamp of quality because in the end there’s a huge difference between. Supporting lots of customers and being an otto—an Oto, someone who doesn’t even know his customers, meaning once you download the software, additional ones, there’s also a very convenient distribution channel here, very convenient and very friendly, and we’re also good friends with it and it helps us a lot. How to say what I mentioned earlier: after that came the part where we wanted to do camera integration, meaning to make a dedicated and custom chip. that it’s no longer for all kinds of cases, but for huge companies, and that’s where we suddenly found ourselves dealing with image quality algorithms, and suddenly we were. We found ourselves in a situation of researching how AI improves image quality, which is something we were never required to do before; previously we dealt with analytics, use cases, and things like that. And then we said, okay, we have a chip, we have a lot of AI that we developed, a lot of investment in AI, let’s try to improve an image with it, and today. Five years later, we find ourselves in integrations in cameras with giant companies that actually use our chip to see better at night, for example I thought it would get there.

From 15 to ~280 people: how a hardware company looks

Why they chose an accelerator over the cloud

The value of the Raspberry Pi partnership

Moving to dedicated chips for smart cameras

Yael: [06:44] I think maybe also tell her what types of cameras in the field, maybe that will add value, because let’s say I’m thinking for a second.

Ori Katz: [06:51] About aviation, but security in general, so it was a completely different thing; it started as a kind of small item in the menu that it’s important to have, so you can do this bounding box thing that exists in all kinds of airports, as you say. And as time went on, and certainly with what’s happening now with the LLMs and LLMs and so on, which I haven’t talked about yet, we’ll get to that, that we also do deployment into cameras, actually. It became an inseparable part of these cameras, and one of the hardest problems the company encounters is simply, it’s an old problem, seeing in the dark, yes, it’s not something new; humans encountered it long ago too. And when we discovered, and really discovered in the communities and in the startups, that it could make it possible to see well at night, to see in the dark, okay, not IR LEDs and nothing like that. There are capabilities, again, around the right models to do what’s called denoising, cleaning the image from noise, and actually combine that with a camera chip, with the sensor that knows how to do what I called earlier analytics. And take everything we learned from the Raspberry Pi users and suddenly all of that comes together into a product that, on the one hand, has your scratch, your low-power AI. On the other hand, the advanced software you’ve been building for ten years, and you integrate analytics there and improve image quality, and suddenly this thing becomes a cheap chip. Powerful and significant, that starts doing this independently in the world without you, and somewhere in the middle of that is the part where I actually came in to lead product, to also look at what we did in the accelerator with major companies. What does the camera world need in order to be able to see? How much are we able to scope the investment through community, and the community has been a success over the past year and is going very well, meaning people answer each other, right? You know, like. You say, this isn’t ours, we didn’t invent it, and it’s so cool to see how something you built, never mind for a second the money, but just the fact that. People answer each other there when we’re not working for them, and then you say, okay, if maybe I can strengthen this, that means I actually need it for my other products too, the ability. To work in distribution and the ability to work with a community, and the ability to support a company without 250 customer success people for everything. So we decided to actually maintain this thing as a unit and establish a larger unit, which is what’s called the system-on-chip unit, where we really go after. Let’s call them monsters; sometimes they’re small monsters, sometimes bigger monsters, and there we really go after the big ones, some with spectacular success, some on a very long journey in the desert because. And some of it, we can also talk about it, is the less sexy part of product management and everything related to that. It wasn’t, and not at all, and nothing like that; it was the golden-olden stuff, you know, price and supply consumption, and price-to-supply and supply chains. And what it means to ship, and how much your lead times are, and all kinds of things that in the hardware world fall under product management categories. And I don’t want to say boring, but. I like them, but sometimes they’re less sexy than other things, yet to successfully manage a killer product in the world of millions of units a year. You have to control this thing at basically psychotic levels. How deep are you actually in factories, in automation? Tell me about the so-called industrial side, so in the industrial world I’ll give you. Just a little anecdote: a few years ago I was in Germany with a client from a large brewery. The bottleneck literally, yes, is bottling; that’s the bottleneck of the whole factory, meaning what limits them from producing more is the machine. There’s some kind of basin where all the bottle caps fall into a kind of trough, and they need to bottle the bottles; the caps fall 50/50 either on this side or that side, because that’s how the world is, that’s how physics works, and their hardest problem. Is being able to decide whether this thing, whether the cap, is facing up or facing down, and it’s a matter of quantities because you can look at them one by one and it costs a lot of money now, so that’s an example of industrial. What do I mean by industrial, I’ll say again from the perspective of a chip company, The problem with industrial is that, unfortunately, they need to sell in large volumes. Industrial solves extremely, extremely, extremely complicated problems, But they don’t need many chips for that, and therefore we decided to serve the industrial field through the first segment we talked about, actually through the acceleration segment, And to tell customers, folks, you want four cameras that will decide whether the cap is upside down like this or like that, Please download the site, order a Raspberry Pi, run it, and it really works, because we understood that these factories—some huge ones of German conglomerates, The fact that they’re huge companies doesn’t mean we can make a living from that, and that was one of the wake-up calls, and therefore the industrial field to this day, Is supported commercially successfully under the umbrella of the accelerator, a great field for anyone making chips, but it’s limited in volume, And where, once again, I say we found our place and looked a little to the right and left, including at the competition, Really security cameras, smart cameras, airports, streets, and now you also see it, by the way; I’m a lot in the East, as you said, Also in Israel, but mainly in the East, you see it entering the private market already, you see it entering what’s called retail and department stores and really everywhere, everywhere, people are aware, Also of the image quality they get from security cameras, and also the capabilities of the cameras, and here enters the topic of. What’s called Gen AI, yes, there is a huge advantage in what you can actually do with these things, To make cameras truly smart—not just say, “put a box around a person and a vehicle,” but to be able to talk to them, right? People see AI, they see GPT, and they want to say, “Come on, tell me, give me a summary of what happened in the store yesterday over the last 24 hours,” and they don’t want to go to the cloud. Right, they want everything to be fast, and they want privacy and all kinds of other things. So now we’re really in an era, and it’s hot off the press, I’m saying this because it’s not new—Gen AI is not hot off the press at all—but the deployment of Gen AI at the edge, meaning where Hailo sits, taking these enormous models and finding them the use case, the so-called killer app, and deploying them at the edge with low power, and making it meaningful enough that customers will want it, that’s one of the biggest challenges facing us today, and it’s really fascinating, so okay.

Seeing in the dark: AI for denoising

Building the SoC unit

The unsexy side of hardware: price and supply chain

An industrial example: the German brewery

Where Hailo operates today

Gen AI at the edge: cameras you can talk to

Yael: [13:19] A moment before we continue, click the link to the Product Builder WhatsApp group in the episode description and join the community, where you can continue the discussion, ask questions, and be the first to receive updates on new Tachles episodes, we’re waiting for you there, and now back to the episode. Let’s make sense of this for a second. So basically, what you’re saying, if I understand correctly, the challenge is essentially to take the huge model, train it on very, very specific data, and then basically bring up a model that is relatively not huge. There are, right, there are several challenges here, the main challenge is definitely the main challenge today: the models that

The technical challenge: compressing smaller models

Ori Katz: [13:55] companies, let’s call them, all the big ones publish, the smallest of them is basically 7 billion parameters. Yes, there are also 2 billion, but barely anyone pays attention to it; there is very little attention on it today, I believe that will change, taking and training the models, training the 2-billion smaller models, so to speak, and getting them to produce accurate results. Okay, so that’s the first challenge because we’re edge devices. I don’t know how familiar you are with the memory crisis and the dramas—I’m saying you should talk about it, but I’ll mention it anyway—because for those who aren’t in the field, and even for those who are, it’s one of the biggest crises the industry has seen in recent years, due to various processes also related to Nvidia’s purchasing of memory and their move to what’s called HBM for heavy processing in the cloud. The good old memories of devices, including our phones by the way, are becoming neglected, and as a result their prices are going up; fewer and fewer people want to manufacture them—Micron and Samsung, and so on—and then I, as a product manager, or Hailo for that matter, look at this and say, wait. I want to bring in a two-billion or even seven-billion model now, so I can run some cool generative thing on a camera, but this DRAM now costs $150, and who is going to add $150 to a smart camera just so it can run some VLM that will tell you exactly who entered the room yesterday at such and such time, okay. So, another aspect of product management—certainly hardware product management, certainly edge product management—is something that is on our plate these days, and we’re dealing with it. But really one of the biggest challenges is what it means to do generative at the edge; why not run everything in the cloud, right? Hailo is confronted with this question all the time. There are many reasons: privacy, latency, power, price, and a lot of other things, but the cloud is very, very strong, and we constantly find ourselves proving to customers. We’re four steps ahead of the customer, trying to show me the same thing—if you run it on the edge, you’ll save so much and so much and so much. And why it will be good for you, and why it will pay off, and why you’ll have, like, ROI, and what you’ll get out of it—so that’s another angle. A really recent angle from the last six months of how the pace of AI meets the world of hardware.

The memory and cost crisis (DRAM/HBM)

Why not run everything in the cloud

Yael: [16:12] Which, by its nature, is usually not a very special world, so okay, let’s go back a bit, see where we left off for a second. We talked about the commerce world, so basically it’s like a contract here—thanks for that. I’m very curious about the market; I don’t know the competitors well enough, but basically from all this talk about efficiency—now, as you said, before it was just contextual pins, but suddenly now I want to become more efficient as an organization, I want to basically count people and also be much more data-driven, as you basically said, I want to get to ask, hey, how many people were in the store? Just an example you gave. So you need to decide now, basically you need to place your bet on which industries you’re most invested in.

Ori Katz: [16:54] Wow, we’re not suddenly talking about security—not security—basically it’s more of an industry, yes? So first of all, as a company, as a strategy, and as the big house looking at the future. There’s no doubt we’re looking at physical AI; that is, it’s something from, let’s call it, the recent period. Robots are not a new thing, but when you look at who was at CES this year, and soon there will be ISE and ISC, and you look at the conferences, you understand that robots have always been there everywhere at those conferences we go to, but you can no longer ignore it. I mean, we are inside an AI company, a hardware company, and we look at the competitors. our giants, for example Qualcomm—code name—there are many others, and you can’t ignore the fact that the working point we developed and we look at what our assets are, what we have brought to the world so far: the ability to run AI in incredible software that tens of thousands of people use, but at low power and low cost. It has to be robots, yes. You look five years, four years ahead from today—it has to be robots. Okay, so today robots, of course, is a very broad range of things, or physical AI is a very broad range. Today you can talk about automatic vacuum cleaners, or if you go all the way to drones, of course, and when you talk about the really extreme things, it’s all kinds of humanoids like that which will take care of elderly people, disabled people, and all kinds of things like that, but when you talk about where three to four years from now, we absolutely have to place our bets; where should we place our bets today so we can explode in another 3-4 years? The answer is physical AI, where we’re operating today. Today I’m leading my AISOC. We’re talking about the surveillance market, which has always been the market that is exploding right now; I’m not talking about the surveillance market forecast. It doesn’t sound sexy, and that’s not in the sense of—I just have to say, I work with a lot, a lot of customers, and in fact the subject of facial cognition is almost the last thing I hear about. Still, most of the companies in the world I really work with want to do good and make the public space safer, with intrusion and detections and things. Classics, but at much larger volumes, much faster, with much greater accuracy. At the level of, okay, an accident happened, I want the accident, so that the rescue forces will arrive within. 0.01 seconds for them to know it happened and get to the scene, so if I look at Hailo and its three main layers: We have the base, where we operate, that’s the makers; we have the second floor, which is where I mainly operate today around the East and so on, which is surveillance. And you can go back to R&D today; the direction is to be there. We are already working on drones and robots today, I should say yes. Through the SoC, through an accelerator, but until there’s a dedicated chip for this, it’s clear that we—and until the market gets there.

Physical AI and robotics: the next bet

Surveillance as an exploding field

Yael: [19:42] So we’re working on it. Wow, I’m glad you said that because you talked about drones, but in the end, yes, I also think I suddenly imagined that same robot that helps. An older person or something like that, and that’s already like a completely different market. Now I understand a little more the play you’re making, basically, with the makers. Because it’s basically also some kind of, again, distribution, but also an enabler to actually make the play—absolutely, absolutely—and that’s also true for investors, meaning for the market, because you know today.

Ori Katz: [20:12] Investors, or also employees, we need to work on sensors, yes, robots, and everything. I’m asking: here are robots, see how many robot customers I have. Now they’re doing it through the accelerator, as you said, through our SoC, and that gives you so much noise and scale. Because, you know, you just talk to your customers, they already want to talk to you, you’re already working with them, you’re their supplier. So okay, they don’t buy a million chips a year, but they do keep the conversation going, so you have that base. And again, it goes right back to what you said: your thing is holding the community and the community as a basis for innovation, and as you said, you went with it. If in another three or four years the biggest drone company in the world isn’t our customer from today onward, then that’s it, so yes, so I kind of want for a second to put on the self-defense here for a moment.

Community and distribution as an adoption engine

Yael: [20:59] I want to tell a bit about trends—what you see at conferences in the industry, on your trips to the East, tell us a bit about that too. Before everything, yes, East and security, tell a bit about that. So look.

China vs. the West in surveillance

Ori Katz: [21:12] For those who don’t know the field, the world is. The United States and the West today are what you’d call the product manager of the world. Wait, I have to ask—security, surveillance, China, that’s basically the story, right? So look, no, I. I was exactly once—I’ll also tell you on a personal level, because I too, on that basis, decided I had been to China twice in the past year and a bit. And I understood, both at the Hailo level and on a personal level. Strategically, we, we don’t need to be there. I mean, I’m not talking about the ethics; I’ll put the morals aside for a moment, even though I still don’t know—again, in the end they don’t tell me what they do with those cameras. But listen, what these guys are doing in surveillance and security, the investment they put there and the things they do are simply unbelievable. Once, you know, the Chinese were like copiers and all sorts of things from the past that simply are no longer true. They are the most advanced in the world; they are running VLMs and have them in their production pipelines already for at least a year and a half, two years, okay, for less money than any. Motorola or a big American customer who is thinking about this, and that’s where Americans and Europeans, and let’s call the Koreans and Taiwanese, for that matter the West. Are trying to understand how to use them, and they are already nine levels ahead; God knows what they’re doing with it, I’m not even getting into that, and I can also tell you on a business level. For Western companies it’s very hard to operate in China, very, very, very hard. It also really comes from government directive. So that, and going there is less fun, but just—it’s really buzzing. So there is the world and there is China; that is completely the separation, because Korea and Taiwan. Which is really called the East for us, right? For that matter, in the West they are where most of the security cameras in the world are manufactured and developed, and Americans and Europeans are, let’s call them the product managers—they say, okay, we want this, we want that, do this here, do that there for us. The Taiwanese and Koreans basically manufacture, but China is a different game, it’s a different world; by the way, that’s also true in cars, I can tell you. Event-triggering and smart search and all sorts of things that we today implement on the cameras as a reference for customers to take and see how it works. If we don’t bring it to them with Gen [AI], they already solved it two years ago, a year ago, and before that, and in smaller models and cheaper chips and everything, Made in China as well.

Yael: [23:33] Maybe the thing we were exposed to last year with DeepSeek—the whole economic thing is very clear now in China.

Ori Katz: [23:41] Yes, the economic aspect and also the attempt to limit it. Meaning that the West—I’m not getting into politics here—and that the West. Perhaps, with some wisdom from its perspective, trying to prevent China from accessing advanced technologies, maybe that buys it, you know, a year or two or something like that. But it also encourages a lot of creativity, and the Chinese, again, I have to say on a personal level because I went—they’re simply amazing in terms of the developers. Simply amazing; it’s no longer just reverse engineering, it’s innovation entirely, and it’s inspiring, even a bit scary, but inspiring. I mean, I was at a conference. In Beijing, exactly a year and a half ago, and I saw there things in terms of advancement that I. Thank you, as a product person you go home afterward with, like, so many ideas, so many cool things, again cool and not cool, but. What progress. So in that sense the Chinese absolutely deserve a tip of the hat for everything that entails. Share with us more trends that you see.

Yael: [24:37] So again, in surveillance, so what do you think? I’ll leave you another question I wrote down on the side. Like, again, I think about the revolution it made in software development, where and how much it is reflected in hardware development.

AI in hardware development: Cadence, Synopsys and debug

Ori Katz: [24:57] Great question. Look, when it comes to code development, I mean. I wouldn’t say it’s exactly the same, but we’re there. I mean, our company group in VLSI, and if someone doesn’t know. Very-large-scale integrated chips, or chip design, as it’s called here, a pretty big industry in Israel. Everyone is already working with it, working with Cursor, working with Claude, working with whatever you want. One of the interesting things in hardware is that at the end you have to do physical design. There’s really this part where you take all that amazing logical code. Now you give it to synthesis, and synthesis inevitably does things—it turns it into hardware. Inevitably it does things you can’t later debug and understand, so in that sense it’s naturally fertile ground for generative AI because it simply speeds up those. Developments and the same things that classical synthesis tools did before, by the way, for anyone who knows Cadence and Synopsys. Two companies that are EDA tools for hardware development, the biggest in the world, worth between 80 and 90 billion dollars each—huge companies. These are companies that AI is part of. I’m talking to you about conferences from 5 years ago, even before there was Claude, they were already putting AI into all their tools. They’re very advanced in this, so in that sense of hardware development. It’s definitely there. I don’t even know if it’s even behind software development; maybe in some ways it’s ahead, thanks to Cadence and Synopsys.

Yael: [26:18] I’ll even say more than that, like, I think in software development even even. I’m using AI now, so as a company I produce much, much more output, and the market sees it much faster. Do you recognize that in your hardware the pace is becoming much faster, because it’s not the same—you understand what I mean? Absolutely, and I don’t have a good answer because the reason is that chip development takes.

Ori Katz: [26:42] Exactly what to do—a year, a year and a half—and this is an event that just happened, so I’ll be able to tell you, and we’ll all be able to tell you, not because I’m smart in hindsight. I’ll be able to tell you in another year, a year and something, whether this really—by the way, one of the things—but that scares. It scares software too, but in hardware somehow, because I’m hardware, it scares me more: how do you debug this thing? I mean, hardware. Debugging it is much, much harder, and you take this thing after it did tapeout, after the chip has already come back to the lab, and you work on it now and see what happens. If this thing was written mostly or entirely by AI, how do I debug it? I mean, I take the agent that wrote it for me and say, come help me—because without it, you’re done.

Yael: [27:23] What does that look like today? How much in companies like this is written by AI, right, or in similar percentages to software, right? I think I’ll honestly say maybe that puts us in an uncomfortable spot, but our last tapeout.

Ori Katz: [27:36] No, not a large percentage. Our last tapeout was a little over half a year ago, and I think even then there is still a significant problem here. There is a very significant problem here.

Yael: [27:47] And you can basically catch it, as you said, only when the chip is ready and I can test it.

Ori Katz: [27:52] Chip debugging post-silicon, for those who get to do it, is one of the hardest things in the world. And you open waves; there are all kinds of tools, by the way, worth millions of dollars, that Cadence and Synopsys themselves sell and make a lot of money from—what’s called emulators that try to emulate the chip again, and so on. Wherever it was, wherever this chip is, wherever the wires really are inside, the transistors—you won’t be able to read them because an AI agent wrote them. I’m sure the people at Cadence and Synopsys have already thought about this. It’s a problem you can’t ignore, because otherwise no one will write with AI. But apparently you need that same agent to sit next to you in the lab and debug the code it wrote, because otherwise—that’s what they do today: they bring in the hardware person to debug the code he wrote.

Yael: [28:32] You need to bring the agent; otherwise, very interesting, come back here, okay, give us some trends, some time to think about something.

Automotive and generative AI: talking to the car

Ori Katz: [28:42] One of the areas we’re operating in a bit less right now is really the automotive field, where you see especially in the East—by the way, Japan, Korea, the Japanese and Korean car companies. In the context of LLM and generative AI, there’s no doubt that there the company’s ambition—once we talked about automotive, and when I joined Hailo, we talked about autonomous vehicles and so on. Almost nobody is talking about that; the goal is to be able to talk to the car, okay? A full, meaningful conversation—that is, to talk to the car, and basically the next stage of this thing. Yes, just talk to it. Just tell me—now a customer of ours, I caught him before, I think I was in Japan two years ago—he told me, Ori, you’ll see what the Japanese are like. What they want is a bit like other trends. Japanese people want to sit in the car and talk to their car; they want it to tell them what’s up, how good you look today, and how your day was, and then, of course, the Japanese, like with everything, take things to the extreme. But the very basis of the basis, and you already see the Chinese are completely there, completely, completely, completely. It’s: open the window, I’m cold, I’m hot, how many kids are there, the seat belt—they’re not wearing the seat belt—wait, open this, turn on the heating, all of that is already basic, meaning it’s already entering everything today. All age groups and so on on their end, and in my opinion this will soon enter all cars in China. You can talk to the car, activate it, close it, turn it off, do all the things. So that’s a trend, it’s happening there very, very strongly; I believe it will happen with that too.

Yael: [30:10] So let’s go in there, yes, okay, this is a home, apparently not far off now, until Japan marries its car. Tell me, I actually took it—it actually sounds like something that’s close. Yes, so basically the most interesting thing here is replacing taxi drivers, in the end. First of all, there is that, yes, there is that.

Ori Katz: [30:33] The whole topic of autonomous vehicles—listen, I joined it, I’d say, eight or nine years ago. I don’t remember if people remember the scale of the investments that were flying around the world back then the way they do today. We even said among friends that it would be over by 2023, and yes, it was—I don’t remember what the company was called, they’d recruit them from Facebook, take them and pay them a billion dollars—it was the same thing, only about autonomous vehicles. Now those things are happening, and back then things were happening too, like. The car—I think there’s a very strong psychological barrier, you can see it, it’s very very strong, toward the stage the industry is at, but I mean the industry came and said. That psychological barrier—I respect it, and I’ll give it the time to develop; these are things that simply mature socially and psychologically regardless of the technology. Let’s let this thing develop, let’s get used to it, let’s focus on other things, let’s try to solve all the autonomous-vehicle problems tomorrow morning and it’ll arrive now. In my opinion it will certainly arrive, no question, but we’re still there, there’s that issue with it. People accept, without any problem, a person killing another person by accident, or whatever, presumably by accident, because of a car. People simply aren’t willing to accept a machine killing other people by accident; they can’t accept that.

Yael: [31:47] Yeah, I don’t know, I was kind of thinking as we go—yeah, a generation is changing here and that sounds to me like a very significant enabler, but okay, let’s put that aside, let’s continue. Do you have more trends that you see?

Ori Katz: [31:59] Listen, it’s not that—I have, I have a lot more trends, but that already goes into [other] worlds.

Yael: [32:04] Tell me for a second, where do you see all your competitors in the world, in your world?

Competitors: Axelera, DEEPX, Ambarella, Qualcomm

Ori Katz: [32:09] So first of all, in its field there are quite a lot, right—in the accelerators field we basically operate in two, in two or three divisions actually. In the basic accelerators division we dominate the market, okay, meaning we have competitors—there’s a company called. Axelera, a European company, Belgian I think, Belgian-German, sort of pan-European, very nice folks there. There’s a Korean, South Korean company called DEEPX that are competitors of ours, but today their accelerators market isn’t that big, and we today are de facto. Dominant. Our biggest competitor, actually. Is the SoC itself—meaning the SoC itself, for instance the Raspberry Pi in the earlier example, will deliver good enough performance even without an accelerator, and then. It’s a competitor of ours, because that creates a love-hate relationship with the company we do the integration with. In the AI SoC area we’re talking first of all about a company called Ambarella—that’s a company that’s. Not small, in the United States, dominant; it entered, started from the field of video encoding and ISP, and of course Qualcomm—Qualcomm is a company that, whether you look at cameras, whether you look at robots, at physical [AI], at drones. It’s basically a company also in the automotive field, an amazing company, Qualcomm, by the way, for anyone who doesn’t [know].

Yael: [33:21] Say—I’ll continue in a second, afterward you can leave me a link to Qualcomm below, I’m joking. What does the camera world look like today? You entered it, and it’s also a choice to enter—we didn’t talk about that.

The camera market, production and cost reduction

Ori Katz: [33:37] Part of a company’s maturing, certainly a hardware company—I won’t get into SaaS because I understand it less—part of hardware’s maturing is that generally users get into SaaS less today. Part of a hardware company’s maturing is moving from that stage of makers and accelerators and so on—what does it mean to enter a field that’s stable, strong, has millions of sockets, meaning you do TAM and SAM analyses, and whoever knows and looks at where the best product-market-fit is for me. You go after—you have to mature, you need quality, you need lead time, you need price, and you need supply, and you need data that’s orderly, and all kinds of things that. Take a lot, a lot, a lot, a lot of time, and it was a maturing process—on one hand [about] strength, on the other hand necessary—meaning today. Say two and a half, three years roughly, that’s roughly the time I’ve been flying around the world; yes, from the moment I started flying to all kinds of Taiwanese, Koreans and Chinese to look for opportunities for my product. Since then, until today, and today we can really say it took two years and we’re now in full production with several customers, and we just released, half a year ago, a product that’s a cost reduction. Of the previous SoC—that’s always a celebration; it’s significant, something like 50 percent, which is also, by the way, product management and so on, Product Builder. One of the most important things—there’s innovation, and there’s the innovation of how to give the same performance at half the price; that’s one of the most important things there is, certainly in the hardware world. IKEA built a company on that—that’s a company, that’s a company—and today I really see the fruits of what it means to bring a cost reduction. To a chip we released two and a half years ago, and it took it two years to enter the market and go into production, and today, as I go around the world, I’m already with a second chip, I’m already with a second kid, I’m no longer with the first one, so to speak. And also at half the price, and it’s stable and so on—that matters, because cracking a market that’s real is hard; competitors fight over every single socket, and companies come that can lower the price. More than you, and hardware is a world of supply chains and memories and things—so how far off is it today that the camera companies will really want to get into this.

Yael: [35:43] Meaning, to bring you in to them—what do you mean, we’re in? No, oh, let them come line up at my door.

Ori Katz: [35:49] Look, the small ones are already at the scale of Motorola—a company that deploys ten thousand cameras a year along the highways in Italy, okay, to do license plate recognition. So who are the big ones—just to set the ear straight—the big ones are Motorola Solutions, Honeywell, Axis, for anyone who knows the company here, Hikvision and Dahua, whom we work with less, but what’s here in every door and everything, all the smart security cameras in the world—that’s millions upon millions of units—those are the companies. They’re much more selective, much harder to get into, they demand certifications and they get deep into your guts and they check everything—it’s a game where you’re basically being examined. Look, the moment you’re there, first of all you always have to protect your spot, because that’s what’s called a very lucrative area, you always have to. Innovate and bring more, and so on, but yes, it’s a game where you pass, and we’re almost there—we’re already in several designs of. TikTok, a notch below in size, but the names are already significant names. Well, so listen, before we finish—it was all fascinating and interesting.

What really matters in hardware product management

Yael: [36:47] There’s a question I usually ask—I ask about products in general, but I’m asking you now—in general, what in your eyes are the most important skills in your world, in the hardware world? Wow, I really pressed on this when we talked in the prep call, who knows how long ago. You basically took it into the areas of the essential versus the peripheral, right?

Ori Katz: [37:16] I’ll tell you what’s not right—I won’t—I’ll say, I’ll look at it the other way: what people think product management is. Let’s say I’m interviewing people for product management, so there’s—let’s do lots of cool features, okay—there isn’t room to say how much that is not the significant part of product management. So if you’re asking, if I have to say what the significant skill is. It’s to look at all the aspects—but really all the aspects—in hardware, as I mentioned, a few things, aspects that aren’t about how convenient and cool and easy your software is to use, which, by the way, often between us. That’s not, that’s not, that’s not the crux of the product—the combination of content and price. The ability to take this thing, and that usually means very unsexy features, really dealing with things. You know, down to the level of where the pins sit, where they sit today, how you arranged it today, whether the architecture is on the right side or the left side—things that most product people, in hardware in general, product is a world. It simply, it simply doesn’t interest them; they deal with the wow, let’s put a VLM in the camera, a VLM in the camera—that’s one thing. So if I have to put my finger on it, it’s really that ability to deal with what’s truly important to the customer, which many times, in my experience at least, in hardware is the least interesting part of the product—that’s the part.

Yael: [38:36] The simple—in your case, hardware, totally hardware—but here, yes, and there’s also the dimension of quality, which for you is super super significant. Quality is hardware, meaning in the end, you know, a customer.

Ori Katz: [38:46] They’re interested, okay—what stress, how much—let’s see you, a product person, come define quality in your case, which I said—there’s the—I closed the—define it. And say to her, listen, I need our mission profile to be 24/7 now, not for a year but for two years—that’s not interesting, but that’s product management, because it’s coming and looking at your market and saying, listen, if we. Can declare that this thing is good for 10 years and not for eight years, then that gives you a kind of edge over the customers—and in a market that’s so much the surveillance market. Which is so cautious and hesitant, and you know wants [proof] on every little thing, and where you put AI and where—where you’re coming from—those are exactly the places you need to excel at, which is to show. That you’re deep in the iron, as always, and you’re not [that] just because you’re an AI company—and again, it’s almost a split personality for me and for the company. Rather, no—you’re there and you deal with the good and important things of old, and not with, once again I say, running a VLM with some LLM, which is just the cherry on top but really not the base of the cake.

Yael: [39:45] As I said, it was really really fun, and how great that you came. If people want to get in touch with you, how do you suggest? Questions—LinkedIn, we do LinkedIn—come on, Ori, what a joy—sorry, I had a really great time, thanks. Let’s do it again, we’ll cut this out, Diana, we’ll cut it in editing. Ori, how great that you came, it was really really fun, and thank you so much. Fun—it’s the jetlag, blessed be it. Friends, thanks for listening. If you found this podcast valuable, you can subscribe and follow us, of course, for more episodes on Spotify, Apple Podcasts, or any other app; and of course if you didn’t find us on some app, I’d be glad if you’d write to us. We’d love five stars on every platform, and for you to follow us so more listeners can discover us and find the podcast. You can also find the previous episodes on any app or on the YouTube channel—we have links in the description. So until next time, come on, be efficient, and bye bye.