Sony IMX500 and IMX501 incorporates AI processing directly onto the sensor
AI image processing is a pretty new field, but one which is rapidly growing. We aren’t talking about the AI photography on your smartphone, but rather AI image processing for surveillance purposes. Current implementations require image data to be captured by a camera and then sent to a cloud for image recognition and other processing to happen, adding latency to the entire process.
With the new Sony IMX500 and IMX501 sensors, that AI processing will be performed directly on the sensors. They will incorporate a Digital Signal Processor (DSP) dedicated to AI signal processing along with memory for the AI model, in addition to the standard on-sensor ISP that processes the image data.
By incorporating the AI processing onto the sensor directly, real-time image recognition is now made possible. This new design allows for ISP processing and AI processing in just 3.1 milliseconds on the sensor itself, with the entire process completed in a single video frame. This allows for high-precision, real-time tracking of objects when recording videos.
Depending on the implementation’s needs, system designers can now decide what to send to the cloud. It can be just metadata, the region of interest (ROI) in the images, the image in various ISP output formats or the entire captured image. That allows for a reduction in data transferred while also alleviating privacy and security concerns when the entire image isn’t sent to the cloud.
Aside from the on-sensor AI processing, the Sony IMX500 and Sony IMX501 sensors look pretty impressive themselves. They seem pretty similar to the Sony IMX412 sensor which was featured in the Sony Xperia XZ2 Premium. They can capture up to 12.3MP images and 4K60 video, with rather sizeable pixels, measuring in at 1.55 μm each. The difference between the IMX500 and IMX501 is the packaging, with the Sony IMX500 being a bare chip product while the Sony IMX501 offered as a 12.5 x 15 mm ceramic LGA package.
![[MWC2018] Nokia brings forth 5 new devices; joins the Android One family! - 26](https://pokde.net/wp-content/uploads/2018/02/Nokia-MWC-2018-cover.jpg "[MWC2018] Nokia brings forth 5 new devices; joins the Android One family! - 27")
![[CES 2024] ASUS Announces New Dual-Screen Zenbook, Plus Redesigned Vivobook Lineup - 30](https://cdn.pokde.net/assets/uploads/2024/01/1739342600-xvw261xn4h7mt0e4syiz.png "[CES 2024] ASUS Announces New Dual-Screen Zenbook, Plus Redesigned Vivobook Lineup - 31")
Sony doesn’t seem intent on marketing the new sensors in the smartphone segment. They are instead targeting IoT implementations in retail and industrial equipment industries, where the AI processing can be used to power various AI-related functionalities. For those of you out there who are interested to sample the new sensors, the Sony IMX500 will cost 10 000 JPY (~RM407) while the Sony IMX501 will cost 20 000 JPY (~RM813).
Pokdepinion: I wonder if these cameras will be used for enhancing the facial detection in systems like China’s Social Credit System.
