UNSOLVED ROC-RK3588S-PC and IMX462

natzguk

@veye_xumm
Yes please see my examples.

camera.open(
        "v4l2src io-mode=dmabuf device=/dev/video0 ! video/x-raw, format=(string)UYVY, width=(int)1920, height=(int)1080 ! appsink",
        cv::CAP_GSTREAMER);

Does not work. I can open the camera but the frames are always empty.

firefly@firefly:~$ GST_DEBUG=3 gst-launch-1.0 v4l2src io-mode=dmabuf device=/dev/video0 ! video/x-raw, format=UYVY, width=1920, height=1080 ! appsink                 Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
0:00:00.517057017 46983   0x5597da8c60 WARN                    v4l2 gstv4l2object.c:4735:gst_v4l2_object_probe_caps:<v4l2src0:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Invalid argument
New clock: GstSystemClock
0:00:00.517846562 46983   0x5597da8c60 WARN                    v4l2 gstv4l2object.c:4528:gst_v4l2_object_get_crop_rect:<v4l2src0:src> Failed to get default crop rectangle with VIDIOC_G_SELECTION: Invalid argument
0:00:00.517940188 46983   0x5597da8c60 WARN                    v4l2 gstv4l2object.c:3278:gst_v4l2_object_reset_compose_region:<v4l2src0:src> Failed to get default compose rectangle with VIDIOC_G_SELECTION: Invalid argument
0:00:00.522044833 46983   0x5597da8c60 WARN          v4l2bufferpool gstv4l2bufferpool.c:850:gst_v4l2_buffer_pool_start:<v4l2src0:pool0:src> Uncertain or not enough buffers, enabling copy threshold

natzguk

I did some more tests and tested all the examples from https://github.com/veyeimaging/rk35xx_firefly/tree/main/linux/samples/opencv/veye camera

Files 1-3 do not work. No frames are shown. Only v4l2_opencv_show4.py works but it only reads with 15 FPS.

Can you confirm that the python opencv examples 1-3 work with ROC-RK3588S-PC and IMX462 ?

natzguk

Hi @veye_xumm
Do you know why UYVY does not work from the examples?

natzguk

@veye_xumm Please help.

v4l2_opencv_show1.py
v4l2_opencv_show2.py
v4l2_opencv_show3.py

Do not work with ROC-RK3588S-PC and IMX462.

veye_xumm

@natzguk
I'm sorry for not replying earlier.
To be honest, we are not very familiar with OpenCV. I believe the low frame rate is due to the software-based UYVY to preview data format conversion, which involves CPU processing for each pixel, leading to inefficiency.
I will verify the samples and your script as soon as possible.

natzguk

@veye_xumm

Only format=NV12 (v4l2_opencv_show4.py) works with OpenCV but it requires videoconvert which is very slow.

I want to use format=UYVY (v4l2_opencv_show2.py) but it does not work with
ROC-RK3588S-PC and IMX462. I believe it is a problem with the driver for ROC-RK3588S-PC. The same code with format=UYVY works with Raspberry PI 4.

natzguk

@veye_xumm

Did you have any chance to check the samples

v4l2_opencv_show1.py
v4l2_opencv_show2.py
v4l2_opencv_show3.py

with ROC-RK3588S-PC and IMX462?

natzguk

@veye_xumm please help

veye_xumm

@natzguk

@natzguk said in ROC-RK3588S-PC and IMX462:

Only format=NV12 (v4l2_opencv_show4.py) works with OpenCV but it requires videoconvert which is very slow.
I want to use format=UYVY (v4l2_opencv_show2.py) but it does not work with
ROC-RK3588S-PC and IMX462. I believe it is a problem with the driver for ROC-RK3588S-PC. The same code with format=UYVY works with Raspberry PI 4.

I apologize for the delay in responding to your question. In fact, as mentioned here, the VICAP module of RK3588 does not support outputting the UYVY format, so please use the NV12 format instead.
Actually, I think using the NV12 format is a good choice.It will not lead to a decrease in system efficiency.

@natzguk said in ROC-RK3588S-PC and IMX462:

Did you have any chance to check the samples
v4l2_opencv_show1.py
v4l2_opencv_show2.py
v4l2_opencv_show3.py
with ROC-RK3588S-PC and IMX462?

Indeed, as you said, only sample4 works correctly. These samples are just collected examples of common ways to use OpenCV to access the camera, provided for customers as a reference.
As mentioned before, our expertise in OpenCV development is not extensive.

videoconvert is indeed an issue. I believe the internal data format of OpenCV is RGBA, and regardless of whether you use UYVY or NV12, conversion is needed.

natzguk

@veye_xumm

If I use NV12 as described in sample4 I can only read frames with max 15FPS from the camera. This will not work for our application.

Can you show me an example which can read frames with 30FPS?

veye_xumm

@natzguk I believe the key issue lies in videoconvert and the display, which has low efficiency. You can refer to our GStreamer for previewing.

natzguk

@veye_xumm
I am not talking about previewing the frame. I am talking about simply reading the frame. No processing.

Here is the example with sample4:

import numpy as np
import cv2 as cv
import os
import time

cap = cv.VideoCapture('v4l2src device=/dev/video0 ! video/x-raw, format=NV12, width=1920, height=1080, framerate=30/1 ! videoconvert ! appsink', cv.CAP_GSTREAMER)

if not cap.isOpened():
    print("Cannot capture from camera. Exiting.")
    os._exit(0)

last_time = time.time()
frame_count = 0

while True:
    ret, frame = cap.read()
    if not ret:
        break

    frame_count += 1
    this_time = time.time()

    if this_time - last_time >= 1.0:
        fps = frame_count / (this_time - last_time)
        print(f"FPS: {fps}")
        last_time = this_time
        frame_count = 0


cap.release()
cv.destroyAllWindows()

Simply reading the frame is currently only getting 15 FPS on ROC-RK3588S-PC. I believe this is a driver issue with NV12.

veye_xumm

@natzguk pls try this:
https://github.com/veyeimaging/rk35xx_firefly/blob/main/linux/samples/gstreamer/veye camera/preview.sh