Camera Image Signal Processing (ISP)

SUMMARY

Cogent Embedded provides the tools you need to develop and optimize processing raw camera signal. Our ISP pipeline includes the embedded code that transforms raw video and PC-tool that help tune the pipeline.

RELATED PAGES

COMPATIBLE IMAGERS

  • Most common automotive CMOS imagers supported; contact us

COMPATIBLE PROCESSORS

BACKGROUND

When it comes to automotive cameras, customer expectations are high. Familiar with the performance of relatively inexpensive consumer products, customers expect an equally impressive HD view from their vehicle’s backup camera.

These high customer expectations can only be met with Megapixel cameras. Previously, those contained a CMOS imager chip and a separate image signal processing (ISP) chip.

The cost of a separate ISP chip can be avoided by moving the image signal processing instead into the infotainment headunit or the surround view module. There, an existing processor can perform the ISP function without incremental hardware cost.

ISP Pipeline

Architecturally, moving ISP from the camera to the headunit or surround view module simply makes sense. Execution, however, can be tricky and less straight forward than one might expect. This is where Cogent’s experience and existing code may help: We have been working with automotive cameras for many years and provide an end-to-end solution that addresses the pitfalls of implementing ISP on existing hardware.

Our embedded code is the real-time loop that controls the camera to provide a perfect view around the vehicle, no matter how bright or dark the environment. It performs the following functions:

  • Raw data correction (remove defective pixels)
  • Lens correction (correct geometric and luminance/color distortions)
  • Noise reduction (Apply temporal and/or spatial averaging to increase SNR (signal to noise ratio).
  • Demosaic (reconstruct colors per pixel)
  • Exposure control
  • White balance
  • Color correction (adjust color in different lighting conditions, including challenging low light conditions)
  • Noise reduction
  • Color space conversion: RGB or YUV output

PC Tuning Tools

Every professional needs good tools. This is certainly true when it comes to developing an image pipeline:  The large number of different settings that can be adjusted and their interdependence requires a sophisticated set of tools to get the job done. At Cogent, we have developed our own suite of ISP tuning tools. Based on camera profile data we can quickly provide an initial setting, and optimize those settings within a closed loop process: The PC “watches” the camera output and manipulates the embedded ISP on the fly through our proprietary Tweaktool API.