SOLVED IMX462M pixelformat on RPi5

[alex_ai]

@alex_ai said in IMX462M pixelformat on RPi5:

Hi @veye_xumm

Hi, I managed to decode mipi packed raw format in rawfile_to_opencv_show.py file and seems like problem is not only in pixel format. If you look to image bellow : it has 6 same small areas which overlaps each other

Also enabled "test image" in camera configuration and it looks similar:

It is most probably driver issue or hardware issue.
What is else I can try to modify?

[veye_xumm]

@alex_ai
I found this information on the Raspberry Pi official website.
Previously, we used the unpacked format as it was convenient for customers. However, it seems that the Raspberry Pi OS has now removed the unpacked format and only retains the packed format (or is there some way to configure this?). We need some time to investigate this further. The information below is for reference.

mode
Allows you to specify a camera mode in the following colon-separated format: <width>:<height>:<bit-depth>:<packing>. The system selects the closest available option for the sensor if there is not an exact match for a provided value. You can use the packed (P) or unpacked (U) packing formats. Impacts the format of stored videos and stills, but not the format of frames passed to the preview window.

Bit-depth and packing are optional. Bit-depth defaults to 12. Packing defaults to P (packed).

For information about the bit-depth, resolution, and packing options available for your sensor, see list-cameras.

Examples:

4056:3040:12:P - 4056×3040 resolution, 12 bits per pixel, packed.

1632:1224:10 - 1632×1224 resolution, 10 bits per pixel.

2592:1944:10:U - 2592×1944 resolution, 10 bits per pixel, unpacked.

3264:2448 - 3264×2448 resolution.

Packed format details
The packed format uses less storage for pixel data.

On Raspberry Pi 4 and earlier devices, the packed format packs pixels using the MIPI CSI-2 standard. This means:

10-bit camera modes pack 4 pixels into 5 bytes. The first 4 bytes contain the 8 most significant bits (MSBs) of each pixel, and the final byte contains the 4 pairs of least significant bits (LSBs).

12-bit camera modes pack 2 pixels into 3 bytes. The first 2 bytes contain the 8 most significant bits (MSBs) of each pixel, and the final byte contains the 4 least significant bits (LSBs) of both pixels.

On Raspberry Pi 5 and later devices, the packed format compresses pixel values with a visually lossless compression scheme into 8 bits (1 byte) per pixel.

Unpacked format details
The unpacked format provides pixel values that are much easier to manually manipulate, at the expense of using more storage for pixel data.

On all devices, the unpacked format uses 2 bytes per pixel.

On Raspberry Pi 4 and earlier devices, applications apply zero padding at the most significant end. In the unpacked format, a pixel from a 10-bit camera mode cannot exceed the value 1023.

On Raspberry Pi 5 and later devices, applications apply zero padding at the least significant end, so images use the full 16-bit dynamic range of the pixel depth delivered by the sensor.

https://www.raspberrypi.com/documentation/computers/camera_software.html#common-options

[alex_ai]

@veye_xumm
Thank you for documentation, I used wrong algorithm to decode 10 bit packed format.
According to doc:
10-bit camera modes pack 4 pixels into 5 bytes. The first 4 bytes contain the 8 most significant bits (MSBs) of each pixel, and the final byte contains the 4 pairs of least significant bits (LSBs).

I fixed it and can open raw image from python, but anyway need find way how to decode video in real time, because now decode one frame is about 3 seconds.

Same Image as above:

[veye_xumm]

@alex_ai
You did a great job.
I have a plan to use the libcamera framework to support RAW-MIPI-IMX462M, but I haven't started yet. Once it is integrated with libcamera, it will be possible to use the picamera programs.

[veye_xumm]

@alex_ai
To facilitate the application of the mv series cameras, we have released a beta version of the libcamera support package.
It doesn't do isp tunning, so you can try it out. Please contact me if you have any questions.

https://github.com/veyeimaging/libcamera
https://github.com/veyeimaging/raspberrypi_v4l2

[alex_ai]

@veye_xumm
Hi, thank you for update.
I'm stuck with compiling libcamera now, with same error as here : https://github.com/raspberrypi/libcamera/issues/151

Did not find solution yet, probably need play with source code, will let you know if any updates.

[veye_xumm]

@alex_ai
You can use the latest libpisp and try again.

[alex_ai]

@veye_xumm
Hi, it helped but I had to use 1.0.6 tag of libpisp, with the latest version I had compile errors.
Now for command : rpicam-hello -v
I have this error:

I used set media script before:

Maybe forgot something else.

[veye_xumm]

@alex_ai ./media_setting_rpi5**.sh is not necessary.

[alex_ai]

@veye_xumm
Hi, I noticed that version of libcamera different on my screenshot (it 0.2.0 but I installed 0.3.0)
I uninstalled rpicam-apps and rebuild it from source code again and it uses my version of libcamera now.
if execute rpicam-still -v I have next error:

In commit I see added json file only to vc4 folder (for RPi4) , can we add it to pisp folder (for RPi5) it should have name : "mvcam_mono.json"? it has different format and can't use same file from vc4.

Also I managed start video by this command:
rpicam-still --tuning-file /source/libcamera/src/ipa/rpi/pisp/data/uncalibrated.json -t 0
and it works!

But better run it without additional parameters
Thank you.

[alex_ai]

@veye_xumm
One more question, I see now all parameters like gain and exposure managed by libcamera software according to json file, Can we use internal camera ISP for it as before?

[veye_xumm]

@alex_ai
Since our camera supports many models of sensor, we are not using libcamera's built-in isp functionality at this time. The RAW-MIPI-IMX462M camera, therefore, can only be used in manual exposure mode.

[alex_ai]

@veye_xumm
Thanks,  it means only manual gain/exposure mode.

I have one more question but it is a little bit out of topic.
 
Yesterday I tried to take pictures of the night sky to compare with 462 color sensor, even with high gain value stars are not bright as for the same parameters on 462 color (but mono should be more sensitive) . It looks like there is no switching between  LCG and HCG modes or I need to tune some other parameters.
Can you advise what could be tuned for this case?

[alex_ai]

@veye_xumm
With metime and mgain parameters I increased sensitive, but it more noisy than color sensor with ISP, anyway issue with pixel format fixed for me, thank you.

[alex_ai]

@veye_xumm
By the way, I see that I can read registers of camera by snsreg command but can't find how to write.
Is it possible set register 0x3009 value to 0x11 to enable HCG mode?

[veye_xumm]

@alex_ai
Sorry, the ability to write directly to the sensor registers is not provided at this time to prevent confusion over the control of the sensor.
I will provide a version of the upgrade package to you as soon as possible.

[veye_xumm]

@alex_ai
Sorry for not following up with you on this in a timely manner. Please download an upgrade package with hcg mode support from the link below.
https://www.mediafire.com/file/313db29c5mb4260/RAW-MIPI-IMX462M_C1.06_20240806_upgrade_hcg_tmp.bin/file

[alex_ai]

@veye_xumm
Hi, thank you for firmware update.
Just to double check, is HCG mode always turn on now?

Also for register 0x3009 I got value 0xffff instead ox11 but seems like HCG enabled anyway.

I need wait for clear sky to test and provide feedback

PS:
After upgrade at the end it writes: "Failed to write register" message but upgrade done , can I just ignore it?

[veye_xumm]

@alex_ai
After the upgrade, re-power up the Raspberry Pi.
The actual version number corresponding to this upgrade package is 1.09， please confirm this.

[alex_ai]

@veye_xumm
Yes, version is 1.09 after upgrade