Navigation

    VEYE IMAGING Forum

    • Register
    • Login
    • Search
    • Categories
    • Tags
    • Recent
    • Popular
    • Users
    • WIKI
    • veye.cc

    SOLVED IMX 287 opencv python多线程多相机硬触发帧率下降

    General Discussion
    2
    9
    1454
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • E
      EtherealHorizon last edited by

      如题,通过v4l2设置帧率为最大之后,单线程多相机的帧率与外部硬触发频率基本一致,但是波动比较大。然后使用如下代码进行多线程多相机采集,帧率远低于外部硬触发频率。

      import cv2
      import argparse
      import subprocess
      import time
      import numpy as np
      
      import numpy as np
      import cv2
      import threading
      from copy import deepcopy
      
      thread_lock = threading.Lock()
      thread_exit = False
      
      class VideoThread(threading.Thread):
          def __init__(self, camera_id, img_height, img_width):
              super(VideoThread, self).__init__()
              self.camera_id = camera_id
              self.img_height = img_height
              self.img_width = img_width
              self.frame = np.zeros((img_height, img_width, 3), dtype=np.uint8)
      
          def get_frame(self):
              return deepcopy(self.frame)
      
          def run(self):
              global thread_exit
              cap = cv2.VideoCapture(self.camera_id)
              cap.set(cv2.CAP_PROP_FRAME_WIDTH, self.img_width)
              cap.set(cv2.CAP_PROP_FRAME_HEIGHT, self.img_height)
              frame_num = 0
              timestampes = []
              while not thread_exit:
                  ret, frame = cap.read()
                  if ret:
                      #thread_lock.acquire()
                      self.frame = frame
                      if frame_num == 0:
                          time_start = time.perf_counter()
                      frame_num = frame_num + 1
                      #thread_lock.release()
                      print('get' + self.camera_id[-6:] + f'frame {frame_num}')
                  else:
                      thread_exit = True
                  time_end = time.perf_counter()
                  time_ms = (time_end - time_start)*1000
                  timestampes.append(time_ms)
                  if frame_num >= 3000:
                      thread_exit = True
                      timestampes = np.array(timestampes)
                      np.save(self.camera_id[-6:] + '.npy', timestampes)
              cap.release()
      
      
      def main():
          # Set up command-line argument parser
          parser = argparse.ArgumentParser(description='Real-time display of GREY image from mv-cam')
          parser.add_argument('--roix', type=int, default=0, help='roi start x (default: 0)')
          parser.add_argument('--roiy', type=int, default=0, help='roi start y (default: 0)')
          parser.add_argument('--width', type=int, default=704, help='image width (default: 640)')
          parser.add_argument('--height', type=int, default=544, help='image height (default: 480)')
          parser.add_argument('--fps', type=int, default=320, help='frame rate (default: 30)')
          parser.add_argument('--pixelformat', type=int, default=12, help='pixel format (default: 12)')
      
          args = parser.parse_args()
          
          v4l2_cmd = f"v4l2-ctl --set-ctrl roi_x={args.roix}"
          subprocess.run(v4l2_cmd, shell=True)
          
          v4l2_cmd = f"v4l2-ctl --set-ctrl roi_y={args.roiy}"
          subprocess.run(v4l2_cmd, shell=True)
          
          v4l2_cmd = f"v4l2-ctl --set-fmt-video=width={args.width},height={args.height}"
          subprocess.run(v4l2_cmd, shell=True)
          
          v4l2_cmd = f"v4l2-ctl --set-ctrl frame_rate={args.fps}"
          subprocess.run(v4l2_cmd, shell=True)
      
          v4l2_cmd = "v4l2-ctl --set-ctrl low_latency_mode=1"
          subprocess.run(v4l2_cmd, shell=True)
      
          v4l2_cmd = f"v4l2-ctl --set-ctrl pixelformat=XY{args.pixelformat}"
          subprocess.run(v4l2_cmd, shell=True)
      
          #v4l2_cmd = "v4l2-ctl --set-ctrl exposure_auto=1"
          #subprocess.run(v4l2_cmd, shell=True)
      
          #v4l2_cmd = "v4l2-ctl --set-ctrl exposure_absulute=20"
          #subprocess.run(v4l2_cmd, shell=True)
      
          v4l2_cmd = "v4l2-ctl --set-ctrl trigger_mode=1"
          subprocess.run(v4l2_cmd, shell=True)
      
          v4l2_cmd = "v4l2-ctl --set-ctrl trigger_src=1"
          subprocess.run(v4l2_cmd, shell=True)
      
          v4l2_cmd = "v4l2-ctl --set-ctrl vi_time_out_disable=1"
          subprocess.run(v4l2_cmd, shell=True)
      
      
          global thread_exit
          img_height = args.height
          img_width = args.width
      
          thread_0 = VideoThread('/dev/video0', img_height, img_width)
          thread_1 = VideoThread('/dev/video1', img_height, img_width)
          thread_0.start()
          thread_1.start()
          #frame_num = 0
          while not thread_exit:
              #thread_lock.acquire()
              #frame_0 = thread_0.get_frame()
              #print(f'get cam0 {frame_num}')
              #frame_1 = thread_1.get_frame()
              #print(f'get cam1 {frame_num}')
              #thread_lock.release()
              #thread_0.join()
              #thread_1.join()
              #frame_num = frame_num + 1
              #if frame_num >= 3000:
                  #thread_exit = True
              pass
          print('done!')
          # Release resources
          v4l2_cmd = "v4l2-ctl --set-ctrl vi_time_out_disable=0"
          subprocess.run(v4l2_cmd, shell=True)
      
      if __name__ == '__main__':
          main()
      
      veye_xumm 1 Reply Last reply Reply Quote 0
      • veye_xumm
        veye_xumm @EtherealHorizon last edited by

        @etherealhorizon
        请问你现在配置的帧率是多少?
        我们对于python层面不太懂,不过我觉得如果是调度问题,可以尝试增加驱动层buffer的数量。

        E 1 Reply Last reply Reply Quote 0
        • E
          EtherealHorizon @veye_xumm last edited by

          @veye_xumm 测试了200和100fps,实际帧率在67-69左右且不稳定。想请教如何在驱动层做调整?

          veye_xumm 2 Replies Last reply Reply Quote 0
          • veye_xumm
            veye_xumm @EtherealHorizon last edited by

            @etherealhorizon
            下面这点代码是我用工具辅助生成的,仅作参考

            # Function to set buffer count
            def set_buffer_count(device, buffer_count):
                fd = os.open(device, os.O_RDWR)
                req = v4l2.v4l2_requestbuffers()
                req.count = buffer_count
                req.type = v4l2.V4L2_BUF_TYPE_VIDEO_CAPTURE
                req.memory = v4l2.V4L2_MEMORY_MMAP
                fcntl.ioctl(fd, v4l2.VIDIOC_REQBUFS, req)
                print(f'Number of buffers set: {req.count}')
                os.close(fd)
            

            python进行内存拷贝方面,是否存在效率问题? 这个我不太确定。

            我们对驱动和c/c++语言比较熟悉一些。像yavta这种代码,是直接使用应用层和驱动层交换内存块的方式实现取图,没有内存拷贝——当然了,应用程序取到之后,进一步进行图像处理,则是另一回事了。

            1 Reply Last reply Reply Quote 0
            • veye_xumm
              veye_xumm @EtherealHorizon last edited by veye_xumm

              @etherealhorizon
              我查到一些信息,希望对你有一定的帮助:

              • python线程
                在Python中,线程确实只能实现逻辑独立,但不能真正做到并发运行。这是由于Python的全局解释器锁(Global Interpreter Lock, GIL)导致的。
                因此要做到并发,需要用多进程。

              • cv2的cap.read()函数,涉及到内存拷贝。
                读取到的帧数据通常以原始格式存储(例如 YUV 或 RGB 格式)。OpenCV 需要将这些原始数据转换为 cv::Mat 类型,以便在 OpenCV 中进行处理。 这个转换过程涉及到将原始数据复制到新的内存缓冲区中,并按照 OpenCV 的内部格式进行排列。这个地方应该是你程序的性能节点。

              如果你的嵌入式平台是多核,建议采用多进程来并发运算。

              E 1 Reply Last reply Reply Quote 0
              • E
                EtherealHorizon @veye_xumm last edited by

                @veye_xumm
                感谢提醒,更换multi-processing之后帧率就正常了。另一个问题是v4l2-python的版本已经多年没有更新,无法直接获取时间戳,不知道有没有解决方案或者比较成熟的c++案例?

                veye_xumm 1 Reply Last reply Reply Quote 0
                • veye_xumm
                  veye_xumm @EtherealHorizon last edited by

                  @etherealhorizon said in IMX 287 opencv python多线程多相机硬触发帧率下降:

                  感谢提醒,更换multi-processing之后帧率就正常了。另一个问题是v4l2-python的版本已经多年没有更新,无法直接获取时间

                  好。
                  驱动层中,v4l2_buffer->timestamp在接收到一帧数据时打上的时间戳。这个值通过v4l2获取buffer的接口是能读到的。你可以查一下v4l-python能不能正常获取此值?

                  E 1 Reply Last reply Reply Quote 0
                  • E
                    EtherealHorizon @veye_xumm last edited by

                    @veye_xumm 经过一番尝试后没能找到,最后是用python多进程为每个相机分别subprocess运行v4l2-ctl命令行然后读取--stream-mmap --verbose的时间戳,目前一切顺利。还想请教一下怎么做real time binning,就是实时地将2x2或者4x4的像素值求和再保存,我们的c水平比较烂哈哈。

                    veye_xumm 1 Reply Last reply Reply Quote 0
                    • veye_xumm
                      veye_xumm @EtherealHorizon last edited by

                      @etherealhorizon

                      @etherealhorizon said in IMX 287 opencv python多线程多相机硬触发帧率下降:

                      最后是用python多进程为每个相机分别subprocess运行v4l2-ctl命令行然后读取--stream-mmap --verbose的时间戳

                      这个时间戳不够准确,如果你的算法对事件准确性要求不高,也可以先这样用。

                      @etherealhorizon said in IMX 287 opencv python多线程多相机硬触发帧率下降:

                      还想请教一下怎么做real time binning,就是实时地将2x2或者4x4的像素值求和再保存,我们的c水平比较烂哈哈。

                      这个我们也缺乏实际经验。不好意思。

                      1 Reply Last reply Reply Quote 0
                      • First post
                        Last post