Raspberry Pi AI Camera with Sony IMX500 Sensor: On-Sensor AI Acceleration and Tensor Processing

Raspberry Pi’s latest AI gadget brings artificial intelligence closer to users with its innovative design. The new Raspberry Pi AI Camera integrates an AI accelerator directly with the image sensor, offering advanced AI capabilities through the camera connector.

Equipped with the Sony IMX500 imaging sensor, this module enables low-latency, high-performance AI processing for a variety of applications. It captures images and videos enriched with tensor metadata and supports on-sensor model execution using an integrated accelerator with 8MB of dedicated memory. This allows real-time object detection, segmentation, pose estimation, and more, making it a versatile tool for AI-powered camera projects.

The AI Camera offloads processing from the Raspberry Pi’s CPU and GPU, allowing them to handle other tasks efficiently. Supported by libraries such as libcamera, Picamera2, and the rpicam-apps suite, it enables users of all skill levels to create advanced AI vision applications.

At the core of this innovation is Sony’s IMX500 Intelligent Vision Sensor, a revolutionary CMOS sensor that integrates image processing and AI on a single chip. This design reduces infrastructure costs, boosts efficiency, and accelerates edge AI deployment. Its seamless compatibility with AI tools and services empowers OEM partners to deliver state-of-the-art vision AI solutions.

Raspberry Pi AI Camera Specifications:

• Image Sensor: Sony IMX500 Intelligent Vision Sensor
• Resolution: 12.3 MP (4056 x 3040 pixels)
• Sensor Size: 1/2.3” (7.857 mm)
• Pixel Size: 1.55 µm x 1.55 µm
• Integrated Features: IR Cut Filter
• Focus: Manual adjustable focus (20 cm to infinity)
• Focal Length: 4.74 mm
• Field of View:
  • Horizontal: 66° ±3°
  • Vertical: 52.3° ±3°
• Aperture: f/1.79
• Infrared Sensitivity: Not supported
• Image Output Formats:
  • Bayer RAW10
  • ISP output (YUV/RGB)
  • ROI and Metadata
• Tensor Input Size: Up to 640 x 640
• Data Type: Int8 or uint8
• Memory: 8 MB (for firmware, network weights, and working memory)
• Framerate:
  • 2028 x 1520 (2×2 binned) at 30 fps (10-bit)
  • 4056 x 3040 (full resolution) at 10 fps (10-bit)
• Cable:
  • Length: 200 mm
  • Connector: 15 x 1 mm FPC or 22 x 0.5 mm FPC
• Operating Temperature: 0°C to 50°C
• Dimensions: 25 mm x 24 mm x 11.9 mm

The left side of the image illustrates the architecture of a traditional AI camera system. In this setup, the camera captures images and sends them to the Raspberry Pi for processing and AI inference. These systems often rely on external AI accelerators or the CPU to perform inference tasks, adding complexity and resource dependency.

In contrast, the right side of the image showcases a system utilizing the IMX500 sensor. Here, the camera module includes a built-in Image Signal Processor (ISP) that converts raw image data into an input tensor. This tensor is processed directly by the AI accelerator within the camera, which generates output tensors with the inference results. These results are then sent to the Raspberry Pi, eliminating the need for external accelerators or neural network processing on the CPU. This streamlined approach enhances efficiency and reduces the hardware requirements for AI tasks.

Fully compatible with all Raspberry Pi computers and maintaining the familiar form factor of the Camera Module 3, the Raspberry Pi AI Camera integrates seamlessly with the Raspberry Pi software stack. It supports libraries like Picamera2 and libcamera, with full integration available for the Picamera2 Python library, enabling programmatic access to the IMX500 sensor for Python-based projects.

The Raspberry Pi AI Camera Module is now available to buy from the company’s official resellers and is priced at $70. Additional information can be found on the Raspberry Pi product and documentation page.

Please follow and like us: