VEYE IMAGING Forum
    • Categories
    • Tags
    • Recent
    • Popular
    • Users
    • WIKI
    • veye.cc
    • Register
    • Login

    ROC-RK3588S-PC and IMX462

    Scheduled Pinned Locked Moved Rockchip App camera
    18 Posts 2 Posters 8.7k Views 1 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • veye_xummV Offline
      veye_xumm @natzguk
      last edited by

      @natzguk

      @natzguk said in ROC-RK3588S-PC and IMX462:

      ROC-RK3588S-PC_Ubuntu_v1.0.6f_veye_mvcam_vbyone_230830.img

      Do not use this image.

      Use this one：
      ROC-RK3588S-PC_Ubuntu_v1.0.4b_221226_veyecam2m.img

      Questions will be answered as soon as possible, please be patient.
      如果你使用中文，请直接用中文提问。
      May the force be with YOU. (This is the translation of the mysterious Chinese symbol above.)

      N 2 Replies Last reply Reply Quote 0
      • N Offline
        natzguk @veye_xumm
        last edited by

        @veye_xumm

        It works now! Thank you.

        Maybe you can update the documentation. https://wiki.veye.cc/index.php/VEYE_CS_Camera_on_Firfly_Boards

        For me it was not clear and I just downloaded the latest image for RK3588S which is v1.0.6f

        1 Reply Last reply Reply Quote 0
        • N Offline
          natzguk @veye_xumm
          last edited by natzguk

          Hi @veye_xumm

          On RK3588S with opencv I only get slow FPS reading from IMX462 of around 15 FPS. The same code on RaspberryPi can read with 30FPS.

          Here is what I tried:

          1.)

          camera.open(
        "v4l2src device=/dev/video0 ! video/x-raw, format=(string)NV16, width=(int)1920, height=(int)1080,framerate=30/1 ! videoconvert ! appsink'",
        cv::CAP_GSTREAMER);

          Only 11-13 FPS

          2.)

          camera.open(
        "v4l2src io-mode=dmabuf device=/dev/video0 ! video/x-raw, format=(string)UYVY, width=(int)1920, height=(int)1080 ! appsink",
        cv::CAP_GSTREAMER);

          Can open camera but can not read frame

          3.)

          camera.open(0, cv::CAP_V4L2);
camera.set(cv::CAP_PROP_FRAME_WIDTH, 1920);
camera.set(cv::CAP_PROP_FRAME_HEIGHT, 1080);

          Can open camera but can not read frame

          Here is the output of dmesg

          firefly@firefly:~$ dmesg | grep veye
[    6.065238] veyecam2m 7-003b: veye camera driver version: 00.01.01
[    6.065262] veyecam2m 7-003b: Failed to get power-gpios, maybe no use
[    6.065422] veyecam2m 7-003b: supply avdd not found, using dummy regulator
[    6.065498] veyecam2m 7-003b: supply dovdd not found, using dummy regulator
[    6.065531] veyecam2m 7-003b: supply dvdd not found, using dummy regulator
[    6.065560] veyecam2m 7-003b: could not get default pinstate
[    6.065565] veyecam2m 7-003b: could not get sleep pinstate
[    6.065580] veyecam2m 7-003b: Success to get veyecam2m endpoint data lanes, dts uses 2 lanes
[    6.187925] veyecam2m 7-003b:  camera id is veyecam2m
[    6.190818] veyecam2m 7-003b: sensor is IMX462
[    6.190860] veyecam2m 7-003b: board type is ONE board
[    6.191449] rockchip-csi2-dphy csi2-dphy0: dphy0 matches m00_b_veyecam2m 7-003b:bus type 5

          And v4l2-ctl

          firefly@firefly:~$ v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
        Type: Video Capture Multiplanar

        [0]: 'NV16' (Y/CbCr 4:2:2)
                Size: Stepwise 64x64 - 1920x1080 with step 8/8
        [1]: 'NV61' (Y/CrCb 4:2:2)
                Size: Stepwise 64x64 - 1920x1080 with step 8/8
        [2]: 'NV12' (Y/CbCr 4:2:0)
                Size: Stepwise 64x64 - 1920x1080 with step 8/8
        [3]: 'NV21' (Y/CrCb 4:2:0)
                Size: Stepwise 64x64 - 1920x1080 with step 8/8
        [4]: 'YUYV' (YUYV 4:2:2)
                Size: Stepwise 64x64 - 1920x1080 with step 8/8
        [5]: 'YVYU' (YVYU 4:2:2)
                Size: Stepwise 64x64 - 1920x1080 with step 8/8
        [6]: 'UYVY' (UYVY 4:2:2)
                Size: Stepwise 64x64 - 1920x1080 with step 8/8
        [7]: 'VYUY' (VYUY 4:2:2)
                Size: Stepwise 64x64 - 1920x1080 with step 8/8
        [8]: 'RGB3' (24-bit RGB 8-8-8)
                Size: Stepwise 64x64 - 1920x1080 with step 8/8
        [9]: 'RGBP' (16-bit RGB 5-6-5)
                Size: Stepwise 64x64 - 1920x1080 with step 8/8
        [10]: 'BGRH' (18-bit BGRX 6-6-6-14)
                Size: Stepwise 64x64 - 1920x1080 with step 8/8
        [11]: 'RGGB' (8-bit Bayer RGRG/GBGB)
                Size: Stepwise 64x64 - 1920x1080 with step 8/8
        [12]: 'GRBG' (8-bit Bayer GRGR/BGBG)
                Size: Stepwise 64x64 - 1920x1080 with step 8/8
        [13]: 'GBRG' (8-bit Bayer GBGB/RGRG)
                Size: Stepwise 64x64 - 1920x1080 with step 8/8
        [14]: 'BA81' (8-bit Bayer BGBG/GRGR)
                Size: Stepwise 64x64 - 1920x1080 with step 8/8
        [15]: 'RG10' (10-bit Bayer RGRG/GBGB)
                Size: Stepwise 64x64 - 1920x1080 with step 8/8
        [16]: 'BA10' (10-bit Bayer GRGR/BGBG)
                Size: Stepwise 64x64 - 1920x1080 with step 8/8
        [17]: 'GB10' (10-bit Bayer GBGB/RGRG)
                Size: Stepwise 64x64 - 1920x1080 with step 8/8
        [18]: 'BG10' (10-bit Bayer BGBG/GRGR)
                Size: Stepwise 64x64 - 1920x1080 with step 8/8
        [19]: 'RG12' (12-bit Bayer RGRG/GBGB)
                Size: Stepwise 64x64 - 1920x1080 with step 8/8
        [20]: 'BA12' (12-bit Bayer GRGR/BGBG)
                Size: Stepwise 64x64 - 1920x1080 with step 8/8
        [21]: 'GB12' (12-bit Bayer GBGB/RGRG)
                Size: Stepwise 64x64 - 1920x1080 with step 8/8
        [22]: 'BG12' (12-bit Bayer BGBG/GRGR)
                Size: Stepwise 64x64 - 1920x1080 with step 8/8
        [23]: 'BYR2' (16-bit Bayer BGBG/GRGR)
                Size: Stepwise 64x64 - 1920x1080 with step 8/8
        [24]: 'GB16' (16-bit Bayer GBGB/RGRG)
                Size: Stepwise 64x64 - 1920x1080 with step 8/8
        [25]: 'GR16' (16-bit Bayer GRGR/BGBG)
                Size: Stepwise 64x64 - 1920x1080 with step 8/8
        [26]: 'RG16' (16-bit Bayer RGRG/GBGB)
                Size: Stepwise 64x64 - 1920x1080 with step 8/8
        [27]: 'Y16 ' (16-bit Greyscale)
                Size: Stepwise 64x64 - 1920x1080 with step 8/8
        [28]: 'GREY' (8-bit Greyscale)
                Size: Stepwise 64x64 - 1920x1080 with step 8/8
        [29]: 'EBD8' (Embedded data 8-bit)
                Size: Stepwise 64x64 - 1920x1080 with step 8/8
        [30]: 'SPD6' (Shield pix data 16-bit)
                Size: Stepwise 64x64 - 1920x1080 with step 8/8
        [31]: 'Y12 ' (12-bit Greyscale)
                Size: Stepwise 64x64 - 1920x1080 with step 8/8
        [32]: 'Y10 ' (10-bit Greyscale)
                Size: Stepwise 64x64 - 1920x1080 with step 8/8

          Please help

          veye_xummV 1 Reply Last reply Reply Quote 0
          • veye_xummV Offline
            veye_xumm @natzguk
            last edited by

            @natzguk
            Have you tried these samples here?
            https://github.com/veyeimaging/rk35xx_firefly/tree/main/linux/samples/opencv/veye camera

            Questions will be answered as soon as possible, please be patient.
            如果你使用中文，请直接用中文提问。
            May the force be with YOU. (This is the translation of the mysterious Chinese symbol above.)

            N 1 Reply Last reply Reply Quote 0
            • N Offline
              natzguk @veye_xumm
              last edited by natzguk

              @veye_xumm
              Yes please see my examples.

              camera.open(
        "v4l2src io-mode=dmabuf device=/dev/video0 ! video/x-raw, format=(string)UYVY, width=(int)1920, height=(int)1080 ! appsink",
        cv::CAP_GSTREAMER);

              Does not work. I can open the camera but the frames are always empty.

              firefly@firefly:~$ GST_DEBUG=3 gst-launch-1.0 v4l2src io-mode=dmabuf device=/dev/video0 ! video/x-raw, format=UYVY, width=1920, height=1080 ! appsink                 Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
0:00:00.517057017 46983   0x5597da8c60 WARN                    v4l2 gstv4l2object.c:4735:gst_v4l2_object_probe_caps:<v4l2src0:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Invalid argument
New clock: GstSystemClock
0:00:00.517846562 46983   0x5597da8c60 WARN                    v4l2 gstv4l2object.c:4528:gst_v4l2_object_get_crop_rect:<v4l2src0:src> Failed to get default crop rectangle with VIDIOC_G_SELECTION: Invalid argument
0:00:00.517940188 46983   0x5597da8c60 WARN                    v4l2 gstv4l2object.c:3278:gst_v4l2_object_reset_compose_region:<v4l2src0:src> Failed to get default compose rectangle with VIDIOC_G_SELECTION: Invalid argument
0:00:00.522044833 46983   0x5597da8c60 WARN          v4l2bufferpool gstv4l2bufferpool.c:850:gst_v4l2_buffer_pool_start:<v4l2src0:pool0:src> Uncertain or not enough buffers, enabling copy threshold
              N 1 Reply Last reply Reply Quote 0
              • N Offline
                natzguk @natzguk
                last edited by

                I did some more tests and tested all the examples from https://github.com/veyeimaging/rk35xx_firefly/tree/main/linux/samples/opencv/veye camera

                Files 1-3 do not work. No frames are shown. Only v4l2_opencv_show4.py works but it only reads with 15 FPS.

                Can you confirm that the python opencv examples 1-3 work with ROC-RK3588S-PC and IMX462 ?

                N 1 Reply Last reply Reply Quote 0
                • N Offline
                  natzguk @natzguk
                  last edited by

                  Hi @veye_xumm
                  Do you know why UYVY does not work from the examples?

                  1 Reply Last reply Reply Quote 0
                  • N Offline
                    natzguk
                    last edited by

                    @veye_xumm Please help.

                    v4l2_opencv_show1.py
                    v4l2_opencv_show2.py
                    v4l2_opencv_show3.py

                    Do not work with ROC-RK3588S-PC and IMX462.

                    veye_xummV 1 Reply Last reply Reply Quote 0
                    • veye_xummV Offline
                      veye_xumm @natzguk
                      last edited by

                      @natzguk
                      I'm sorry for not replying earlier.
                      To be honest, we are not very familiar with OpenCV. I believe the low frame rate is due to the software-based UYVY to preview data format conversion, which involves CPU processing for each pixel, leading to inefficiency.
                      I will verify the samples and your script as soon as possible.

                      Questions will be answered as soon as possible, please be patient.
                      如果你使用中文，请直接用中文提问。
                      May the force be with YOU. (This is the translation of the mysterious Chinese symbol above.)

                      N 2 Replies Last reply Reply Quote 0
                      • N Offline
                        natzguk @veye_xumm
                        last edited by

                        @veye_xumm

                        Only format=NV12 (v4l2_opencv_show4.py) works with OpenCV but it requires videoconvert which is very slow.

                        I want to use format=UYVY (v4l2_opencv_show2.py) but it does not work with
                        ROC-RK3588S-PC and IMX462. I believe it is a problem with the driver for ROC-RK3588S-PC. The same code with format=UYVY works with Raspberry PI 4.

                        1 Reply Last reply Reply Quote 0
                        • N Offline
                          natzguk @veye_xumm
                          last edited by

                          @veye_xumm

                          Did you have any chance to check the samples

                          v4l2_opencv_show1.py
                          v4l2_opencv_show2.py
                          v4l2_opencv_show3.py

                          with ROC-RK3588S-PC and IMX462?

                          N 1 Reply Last reply Reply Quote 0
                          • N Offline
                            natzguk @natzguk
                            last edited by

                            @veye_xumm please help

                            veye_xummV 1 Reply Last reply Reply Quote 0
                            • veye_xummV Offline
                              veye_xumm @natzguk
                              last edited by

                              @natzguk

                              @natzguk said in ROC-RK3588S-PC and IMX462:

                              Only format=NV12 (v4l2_opencv_show4.py) works with OpenCV but it requires videoconvert which is very slow.
                              I want to use format=UYVY (v4l2_opencv_show2.py) but it does not work with
                              ROC-RK3588S-PC and IMX462. I believe it is a problem with the driver for ROC-RK3588S-PC. The same code with format=UYVY works with Raspberry PI 4.

                              I apologize for the delay in responding to your question. In fact, as mentioned here, the VICAP module of RK3588 does not support outputting the UYVY format, so please use the NV12 format instead.
                              Actually, I think using the NV12 format is a good choice.It will not lead to a decrease in system efficiency.

                              @natzguk said in ROC-RK3588S-PC and IMX462:

                              Did you have any chance to check the samples
                              v4l2_opencv_show1.py
                              v4l2_opencv_show2.py
                              v4l2_opencv_show3.py
                              with ROC-RK3588S-PC and IMX462?

                              Indeed, as you said, only sample4 works correctly. These samples are just collected examples of common ways to use OpenCV to access the camera, provided for customers as a reference.
                              As mentioned before, our expertise in OpenCV development is not extensive.

                              videoconvert is indeed an issue. I believe the internal data format of OpenCV is RGBA, and regardless of whether you use UYVY or NV12, conversion is needed.

                              Questions will be answered as soon as possible, please be patient.
                              如果你使用中文，请直接用中文提问。
                              May the force be with YOU. (This is the translation of the mysterious Chinese symbol above.)

                              N 1 Reply Last reply Reply Quote 0
                              • N Offline
                                natzguk @veye_xumm
                                last edited by

                                @veye_xumm

                                If I use NV12 as described in sample4 I can only read frames with max 15FPS from the camera. This will not work for our application.

                                Can you show me an example which can read frames with 30FPS?

                                veye_xummV 1 Reply Last reply Reply Quote 0
                                • veye_xummV Offline
                                  veye_xumm @natzguk
                                  last edited by

                                  @natzguk I believe the key issue lies in videoconvert and the display, which has low efficiency. You can refer to our GStreamer for previewing.

                                  Questions will be answered as soon as possible, please be patient.
                                  如果你使用中文，请直接用中文提问。
                                  May the force be with YOU. (This is the translation of the mysterious Chinese symbol above.)

                                  N 1 Reply Last reply Reply Quote 0
                                  • N Offline
                                    natzguk @veye_xumm
                                    last edited by

                                    @veye_xumm
                                    I am not talking about previewing the frame. I am talking about simply reading the frame. No processing.

                                    Here is the example with sample4:

                                    import numpy as np
import cv2 as cv
import os
import time

cap = cv.VideoCapture('v4l2src device=/dev/video0 ! video/x-raw, format=NV12, width=1920, height=1080, framerate=30/1 ! videoconvert ! appsink', cv.CAP_GSTREAMER)

if not cap.isOpened():
    print("Cannot capture from camera. Exiting.")
    os._exit(0)

last_time = time.time()
frame_count = 0

while True:
    ret, frame = cap.read()
    if not ret:
        break

    frame_count += 1
    this_time = time.time()

    if this_time - last_time >= 1.0:
        fps = frame_count / (this_time - last_time)
        print(f"FPS: {fps}")
        last_time = this_time
        frame_count = 0


cap.release()
cv.destroyAllWindows()

                                    Simply reading the frame is currently only getting 15 FPS on ROC-RK3588S-PC. I believe this is a driver issue with NV12.

                                    veye_xummV 1 Reply Last reply Reply Quote 0
                                    • veye_xummV Offline
                                      veye_xumm @natzguk
                                      last edited by

                                      @natzguk pls try this:
                                      https://github.com/veyeimaging/rk35xx_firefly/blob/main/linux/samples/gstreamer/veye camera/preview.sh

                                      Questions will be answered as soon as possible, please be patient.
                                      如果你使用中文，请直接用中文提问。
                                      May the force be with YOU. (This is the translation of the mysterious Chinese symbol above.)

                                      1 Reply Last reply Reply Quote 0

                                      Hello! It looks like you're interested in this conversation, but you don't have an account yet.

                                      Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.

                                      With your input, this post could be even better 💗

                                      Register Login
                                      • First post
                                        Last post