UNSOLVED Capture MV-MIPI-IMX178M with OpenCV

barbanevosa

I use MV-MIPI-IMX178M with Jetson AGX Orin to capture it with my Python script using OpenCV.

My pre-configuration v4l2-ctl commands are:

v4l2-ctl -d 0 -c roi_x=0,roi_y=0,preferred_stride=3136,frame_rate=22
v4l2-ctl -d 0 -v width=3088,height=2064,pixelformat=GREY

Format Video Capture:
        Width/Height      : 3088/2064
        Pixel Format      : 'GREY' (8-bit Greyscale)
        Field             : None
        Bytes per Line    : 3136
        Size Image        : 6472704
        Colorspace        : sRGB
        Transfer Function : Default (maps to sRGB)
        YCbCr/HSV Encoding: Default (maps to ITU-R 601)
        Quantization      : Default (maps to Full Range)
        Flags

In my Python script I use GStreamer and the simplified code looks like this:

import cv2

pipeline = f"v4l2src device=/dev/video0 ! appsink"
cap = cv2.VideoCapture(pipeline, cv2.CAP_GSTREAMER)

while True:
    ret, frame = cap.read()
    if not ret:
        print("Capture error")
        break
    cv2.imshow("Cam", frame)
    if cv2.waitKey(45) & 0xFF == ord("q"):
        break

cap.release()
cv2.destroyAllWindows()

It works fine!

Then I try to switch capture API from GStreamer to v4l2:

import cv2

cap = cv2.VideoCapture(0, cv2.CAP_V4L2)

cap.set(cv2.CAP_PROP_FRAME_WIDTH, 3088)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 2064)
cap.set(cv2.CAP_PROP_FOURCC, cv2.VideoWriter_fourcc(*'GREY'))

while True:
    ret, frame = cap.read()
    if not ret:
        print("Capture error")
        break
    cv2.imshow("Cam", frame)
    if cv2.waitKey(45) & 0xFF == ord("q"):
        break

cap.release()
cv2.destroyAllWindows()

And I see stripes in the video window. I guess the reason is how the v4l2 considers the frame width and the prefered stride.

I try to set:

cap.set(cv2.CAP_PROP_FRAME_WIDTH, 3136)

...and get 'Capture error'.

My attempts to change prefered_stride property by v4l2-ctl were not successful.

The only workaround I've managed to find so far is to reduce the width of the captured frame to a multiple of 64 so that matches the prefered_stride. The closest such value to 3088 is 3072. So I preconfigure the cam with width=3072,prefered_stride=3072. And set in my Python program:

cap.set(cv2.CAP_PROP_FRAME_WIDTH, 3072)

That works fine!

But what if I had to read all 3088 pixels of width? Is there any solution for v4l2 (not GStreamer)?

veye_xumm

@barbanevosa
The issue occurs because the camera sensor outputs a resolution of 3088 × 2064, but the receiving side (V4L2 buffers) aligns each line to a stride of 3136 bytes for memory alignment. When OpenCV is used directly with cv2.CAP_V4L2, it applies the width setting (3088) both to the sensor and to the buffer interpretation. Since the actual buffer has padding (3136 vs. 3088), OpenCV misinterprets the data layout, which results in visible image artifacts (stripes).

We recommend using a GStreamer pipeline to handle the stride and alignment internally, so that OpenCV receives a clean image with the correct resolution.