How to Build AI at the Edge of the Device — Ori Katz, Hailo - Product Builder Podcast episode with Eyal David
Updated:

How to Build AI at the Edge of the Device — Ori Katz, Hailo

Host: · Guest: Ori Katz · About the host

Every hardware company is a software company: the people buying chips are developers who want to run AI on them, not tinker with silicon. According to Ori Katz, who leads the AI SoC domain at chipmaker Hailo, the heart of hardware product management is the 'unsexy' part — price, supply chain, lead times, and power consumption — not flashy features.

In this episode

  • Every hardware company is really a software company: Hailo has ~30-40 hardware engineers versus 100-150 software engineers, because the people who buy the chips are software developers who want to run AI on them — not to fiddle with the silicon.
  • The heart of hardware product management is the 'unsexy' part: price, supply chain, lead times, power consumption and quality. To manage a product that sells millions of units a year you have to master these obsessively — not the flashy features.
  • The Raspberry Pi community (the AI HAT) is both a distribution channel and a quality stamp: it lets Hailo support tens of thousands of developers without an army of customer success, and becomes a base of trust and innovation the company leverages into its next products (robots, drones).
  • The big challenge of Gen AI at the edge is economic as much as technical: compressing a 2-7 billion-parameter model onto a device, amid a memory crisis where DRAM costs ~$150 — and who is going to add $150 to a smart camera? Edge wins over cloud on privacy, latency, power and cost.
  • The strategic bet is physical AI and robotics for the next 3-4 years, with surveillance as the market exploding right now; China is already a year or two ahead of the West in running VLMs in production for surveillance, at lower cost.
  • Cost reduction is itself innovation: delivering the same performance at half the price is what cracks a real market like security cameras, where competitors fight over every socket and you also need stable quality, lead time and supply.

Episode Description

Ori Katz, who leads the AI SoC domain at Hailo, tells the story of how an Israeli chip company became a serious player in AI at the edge — smart cameras, surveillance, industrial, automotive and robotics. The conversation (hosted by Yael) dives into the era's biggest challenges: how to compress large models onto edge devices, how to deal with price, memory and power, and why hardware demands far deeper product management than people assume — fewer flashy features, more hard decisions about cost, supply chain, quality and time to production. Along the way: the Raspberry Pi partnership as both a distribution engine and a quality stamp, the memory crisis, the gap between China and the West, and why physical AI and robotics are the strategic bet for the next 3-4 years.

Chapters

  1. 0:20 · Intro: between flights to Taiwan and Korea
  2. 0:52 · What Hailo does: accelerating AI at the edge
  3. 1:40 · Raspberry Pi and the AI HAT: the community link
  4. 2:38 · Ori's path: from R&D to product
  5. 3:36 · From 15 to ~280 people: how a hardware company looks
  6. 4:38 · Why they chose an accelerator over the cloud
  7. 5:24 · The value of the Raspberry Pi partnership
  8. 6:03 · Moving to dedicated chips for smart cameras
  9. 7:18 · Seeing in the dark: AI for denoising
  10. 8:20 · Building the SoC unit
  11. 9:32 · The unsexy side of hardware: price and supply chain
  12. 10:21 · An industrial example: the German brewery
  13. 11:57 · Where Hailo operates today
  14. 12:28 · Gen AI at the edge: cameras you can talk to
  15. 13:42 · The technical challenge: compressing smaller models
  16. 14:19 · The memory and cost crisis (DRAM/HBM)
  17. 15:39 · Why not run everything in the cloud
  18. 17:03 · Physical AI and robotics: the next bet
  19. 18:30 · Surveillance as an exploding field
  20. 20:33 · Community and distribution as an adoption engine
  21. 21:12 · China vs. the West in surveillance
  22. 24:47 · AI in hardware development: Cadence, Synopsys and debug
  23. 28:42 · Automotive and generative AI: talking to the car
  24. 32:09 · Competitors: Axelera, DEEPX, Ambarella, Qualcomm
  25. 33:37 · The camera market, production and cost reduction
  26. 36:47 · What really matters in hardware product management
  27. 39:52 · Wrap-up
Read the full transcript
Full transcript with speakers and timestamps

Selected quotes

“All hardware companies in the world are basically software companies — and nobody will tell you otherwise.”
— Ori Katz
“There's innovation, and there's the innovation of how to deliver the same performance at half the price — that's one of the most important things there is, certainly in hardware.”
— Ori Katz
“If I have to put my finger on it — it's the ability to deal with what truly matters to the customer, which in hardware is often the least interesting part of the product.”
— Ori Katz

Questions & answers

What does Hailo do?

Hailo is a hardware company that develops and designs chips for accelerating artificial intelligence on edge devices — not in the cloud. Its best-known achievement is the AI HAT for Raspberry Pi (an AI-acceleration add-on through which tens of thousands of developers run models), and it later added chips for smart cameras (SoC). The company was founded about 9 years ago, grew from 15 employees to roughly 250-280, and operates in security cameras, industrial, automotive and robotics markets.

Why run AI at the edge rather than in the cloud?

Because of privacy, latency (response speed), power consumption and cost. The cloud is very strong, so Hailo constantly finds itself having to prove the ROI of running at the edge to customers — how much it saves, why it pays off and what they get out of it. Deploying Gen AI at the edge is one of the biggest challenges right now.

What is the memory crisis and why does it matter for AI in hardware?

Following massive memory purchases (among others by Nvidia) and a shift to HBM for heavy cloud processing, the 'ordinary' DRAM of devices (including phones) is being neglected and getting more expensive. When a product manager wants to run a generative model on a camera, DRAM that costs ~$150 makes it uneconomical — a clear-cut aspect of edge hardware product management.

Which markets does Hailo focus on, and where is it heading?

Three layers: the base is accelerators and the maker community (including industrial, which is supported commercially under the acceleration segment); the second floor, where Ori mainly operates today, is surveillance / smart cameras; and the strategic direction forward is physical AI — robots and drones — for the next 3-4 years.

What are the most important skills in hardware product management?

The ability to look at all the aspects — not just how convenient and cool the software is: the combination of content and price, quality, lead time, supply, reliability over years. Many product people focus on the 'wow' (e.g., a VLM in the camera), but in hardware the part that truly matters to the customer is usually the least glamorous one — and that's exactly what you need to excel at.

People & resources mentioned

People

  • Yael — host of this Product Builder episode
  • Ori Katz — leads the AI SoC domain at Hailo

Companies & orgs

Tools

  • AI accelerators / AI HAT for Raspberry Pi
  • System-on-Chip (SoC) for smart cameras
  • EDA tools for hardware development (Cadence, Synopsys) with built-in AI
  • AI coding tools (Cursor, Claude)
  • VLM / Gen AI at the edge
Product Management Hardware Edge AI Hailo Chips Physical AI Robotics Surveillance Gen AI