More Raspberry Pi Projects

Project 8 — Machine Learning at the Edge with the Pi AI Kit

Edge AI — Why It Matters

Most AI services work by sending data to a remote server — your photo goes to the cloud, a neural network analyses it, and the result comes back. That's fine for many uses, but it has real drawbacks: latency, cost, privacy concerns, and a dependency on internet connectivity.

Edge AI runs neural networks directly on the device — no cloud, no subscription, no data leaving your home. The Raspberry Pi has always been capable of some on-device ML, but the Pi AI Kit changes the game by adding a dedicated Neural Processing Unit (NPU) that handles AI inference 13× faster than the Pi 5's CPU alone, while drawing minimal power.

Hardware Options

What You Will Need

How the AI Pipeline Works

The key insight with the Hailo NPU is that the neural network runs on dedicated silicon — the CPU remains largely free to do other work (run a web server, save video clips, send notifications) while the NPU handles inference at full speed.

Step 1 — Fit the AI Kit (hardware)

• Remove any existing HAT from your Pi 5 if fitted
• Connect the flat flex cable to the Pi 5's M.2 HAT+ connector (the slot closest to the USB ports) — the cable inserts with the blue tab facing up, and the retaining clip snaps closed
• Connect the other end of the cable to the AI Kit's M.2 connector — same orientation
• Fit the AI Kit onto the Pi 5's GPIO header and secure with the supplied spacers and screws
• Connect a Pi Camera to either of the Pi 5's two MIPI camera connectors using a camera cable

Step 2 — Install the Hailo Software Stack

pi@raspberrypi:~$ sudo apt update
pi@raspberrypi:~$ sudo apt install -y hailo-all

The hailo-all meta-package installs:

• hailofw — Hailo NPU firmware
• hailo-tappas-core — GStreamer plugins for the Hailo pipeline
• hailo-pyhailort — Python bindings for the Hailo Runtime
• rpicam-apps — updated libcamera apps with Hailo integration

# Reboot is required for the PCIe driver to load
pi@raspberrypi:~$ sudo reboot

Verify the Hailo device is detected

pi@raspberrypi:~$ hailortcli fw-control identify
Executing on device: 0000:01:00.0
Identifying board
Control Protocol Version: 2
Firmware Version: 4.17.0 (release,app,extended context switch buffer)
Logger Version: 0
Board Name: Hailo-8L
Device Architecture: HAILO8L
Serial Number: ...
Part Number: HM21LB1C2LAE

You should see Hailo-8L (AI Kit) or Hailo-8 (AI HAT+). If the command is not found or the device isn't listed, see the troubleshooting section.

Step 3 — Run Your First Object Detection Demo

Raspberry Pi OS ships with ready-to-run object detection examples via rpicam-hello:

# Live object detection on a connected display (requires monitor)
pi@raspberrypi:~$ rpicam-hello -t 0 --post-process-file \
  /usr/share/rpi-camera-assets/hailo_yolov8_inference.json --lores-width 640 --lores-height 640

This opens a preview window with bounding boxes drawn around detected objects in real time. Try walking in front of the camera, placing objects on a table, or pointing it out of a window at passing cars.

The -t 0 flag runs indefinitely — press Ctrl+C to stop.

Available post-processing configs

pi@raspberrypi:~$ ls /usr/share/rpi-camera-assets/ | grep hailo
hailo_yolov5_personface.json      # detect people and faces
hailo_yolov6_inference.json       # YOLOv6 object detection
hailo_yolov8_inference.json       # YOLOv8 — 80 COCO object classes
hailo_yolov8_pose.json            # human pose estimation
hailo_yolox_inference.json        # YOLOX object detection

Step 4 — Python Object Detection Script

For real projects you'll want to capture frames programmatically, filter detections, and act on them. Here's a practical Python script that captures frames from the Pi Camera and runs YOLOv8 inference via the Hailo NPU:

pi@raspberrypi:~$ pip3 install --break-system-packages picamera2 opencv-python-headless

pi@raspberrypi:~$ nano ~/detect.py

from picamera2 import Picamera2
from picamera2.devices.hailo import Hailo
import cv2, time, datetime

# ── Configuration ───────────────────────────────────────────────
MODEL_PATH    = "/usr/share/hailo-models/yolov8s_h8l.hef"
LABELS_PATH   = "/usr/share/hailo-models/coco.txt"
CONFIDENCE    = 0.5           # minimum detection confidence (0–1)
WATCH_CLASSES = {"person", "cat", "dog", "bird"}  # trigger on these only
SAVE_DIR      = "/home/pi/detections"

# ── Load labels ─────────────────────────────────────────────────
with open(LABELS_PATH) as f:
    labels = [line.strip() for line in f]

# ── Setup ───────────────────────────────────────────────────────
cam   = Picamera2()
hailo = Hailo(MODEL_PATH)

model_h, model_w, _ = hailo.get_input_shape()
cam.configure(cam.create_video_configuration(
    main={"size": (1280, 960)},
    lores={"size": (model_w, model_h), "format": "RGB888"}
))
cam.start()
import os; os.makedirs(SAVE_DIR, exist_ok=True)

print("Detection running — press Ctrl+C to stop")

try:
    while True:
        frame, lores = cam.capture_arrays(["main", "lores"])

        # Run inference on the low-resolution frame (sent to Hailo NPU)
        results = hailo.run(lores)

        triggered = []
        for det in results:
            label = labels[int(det["class_id"])]
            conf  = det["confidence"]
            if conf >= CONFIDENCE and label in WATCH_CLASSES:
                triggered.append((label, conf))
                # Draw bounding box on the full-res frame
                x1, y1, x2, y2 = [int(v) for v in det["bbox"]]
                cv2.rectangle(frame, (x1, y1), (x2, y2), (0, 200, 0), 2)
                cv2.putText(frame, f"{label} {conf:.0%}",
                            (x1, y1 - 8), cv2.FONT_HERSHEY_SIMPLEX,
                            0.55, (0, 200, 0), 2)

        if triggered:
            ts   = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
            objs = "_".join(sorted({l for l, _ in triggered}))
            path = f"{SAVE_DIR}/{ts}_{objs}.jpg"
            cv2.imwrite(path, cv2.cvtColor(frame, cv2.COLOR_RGB2BGR))
            print(f"[{ts}] Saved: {objs} ({', '.join(f'{l} {c:.0%}' for l,c in triggered)})")

        time.sleep(0.1)   # 10 FPS — reduce for less CPU use

except KeyboardInterrupt:
    print("Stopped.")
finally:
    cam.stop()
    hailo.close()

pi@raspberrypi:~$ python3 detect.py
Detection running — press Ctrl+C to stop

Step 5 — Add Notifications

The detection script above saves images but doesn't alert you. Add a notification when a person is detected using Telegram (free and easy) or email.

Telegram notifications

pi@raspberrypi:~$ pip3 install --break-system-packages python-telegram-bot

• In the Telegram app, search for @BotFather and start a chat
• Send /newbot, give it a name and username — BotFather replies with a Bot Token
• Search for your new bot in Telegram and send it any message to start a conversation
• Find your Chat ID by visiting: https://api.telegram.org/bot<YOUR_TOKEN>/getUpdates

Add this function to your detect.py script:

import asyncio
from telegram import Bot

BOT_TOKEN = "your_bot_token_here"
CHAT_ID   = "your_chat_id_here"

async def send_alert(image_path, detected_objects):
    bot = Bot(token=BOT_TOKEN)
    caption = "🚨 Detected: " + ", ".join(
        f"{label} ({conf:.0%})" for label, conf in detected_objects
    )
    with open(image_path, "rb") as photo:
        await bot.send_photo(chat_id=CHAT_ID, photo=photo, caption=caption)

# Call inside the detection loop when triggered:
if triggered:
    # ... save image as before ...
    asyncio.run(send_alert(path, triggered))

Step 6 — Run as a Service

Make the detection script start automatically on boot:

pi@raspberrypi:~$ sudo nano /etc/systemd/system/aidetect.service

[Unit]
Description=Pi AI Object Detection
After=multi-user.target

[Service]
User=pi
WorkingDirectory=/home/pi
ExecStart=/usr/bin/python3 /home/pi/detect.py
Restart=on-failure
RestartSec=10
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target

pi@raspberrypi:~$ sudo systemctl daemon-reload
pi@raspberrypi:~$ sudo systemctl enable --now aidetect
pi@raspberrypi:~$ sudo journalctl -u aidetect -f    # follow live output

Software-Only Path — Pi 4 or Pi 5 Without AI Kit

If you don't have the AI Kit, you can still run object detection using the CPU. Performance is significantly lower but perfectly usable for snapshot-based applications or low-frame-rate monitoring.

pi@raspberrypi:~$ pip3 install --break-system-packages ultralytics opencv-python-headless picamera2

from ultralytics import YOLO
from picamera2 import Picamera2
import cv2, datetime, os

MODEL      = YOLO("yolov8n.pt")    # nano model — smallest and fastest
CONFIDENCE = 0.5
SAVE_DIR   = "/home/pi/detections"
WATCH      = {"person", "cat", "dog"}

cam = Picamera2()
cam.configure(cam.create_still_configuration(main={"size": (1280, 960)}))
cam.start()
os.makedirs(SAVE_DIR, exist_ok=True)

import time
while True:
    frame   = cam.capture_array()
    results = MODEL(frame, conf=CONFIDENCE, verbose=False)[0]
    found   = {results.names[int(c)] for c in results.boxes.cls}
    hits    = found & WATCH
    if hits:
        ts   = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
        path = f"{SAVE_DIR}/{ts}_{'_'.join(sorted(hits))}.jpg"
        cv2.imwrite(path, cv2.cvtColor(results.plot(), cv2.COLOR_RGB2BGR))
        print(f"Saved: {path}")
    time.sleep(2)   # 1 frame every 2 seconds on Pi 4

Project Ideas

Troubleshooting

Quick Reference