CS-USB-IMX307 OpenCV 連續採集影像造成 USB disconnct

coolmitch

@veye_xumm
希望有改版時，對於 uvcvideo 的參數也能夠支援。目前我無法對 power_line_frequency 做設定，對於不支援這些參數這點我非常困擾。以下是對 CS-USB-IMX307 (/dev/video0) 以及另一款 USB Camera module (/dev/video1) 的比較

v4l2-ctl -d /dev/video0 -list-ctrls
VIDIOC_S_INPUT: failed: Device or resource busy
brightness 0x00980900 (int) : min=0 max=100 step=1 default=50 value=50
contrast 0x00980901 (int) : min=0 max=100 step=1 default=50 value=50
saturation 0x00980902 (int) : min=0 max=100 step=1 default=50 value=50
hue 0x00980903 (int) : min=0 max=100 step=1 default=50 value=50
error 5 getting ctrl White Balance Temperature, Auto
aaeon@aaeon2-desktop:~$ v4l2-ctl -d /dev/video1 -list-ctrls
Video input set to 0 (Camera 1: Camera, ok)
brightness 0x00980900 (int) : min=-64 max=64 step=1 default=0 value=0
contrast 0x00980901 (int) : min=0 max=100 step=1 default=38 value=38
saturation 0x00980902 (int) : min=0 max=100 step=1 default=60 value=60
hue 0x00980903 (int) : min=-180 max=180 step=1 default=0 value=0
white_balance_temperature_auto 0x0098090c (bool) : default=1 value=1
gamma 0x00980910 (int) : min=100 max=500 step=1 default=440 value=440
gain 0x00980913 (int) : min=0 max=128 step=1 default=40 value=40
%(#ff0000)[power_line_frequency 0x00980918 (menu) : min=0 max=2 default=1 value=1]
white_balance_temperature 0x0098091a (int) : min=2800 max=6500 step=10 default=4600 value=4600 flags=inactive
sharpness 0x0098091b (int) : min=0 max=100 step=1 default=80 value=80
backlight_compensation 0x0098091c (int) : min=0 max=2 step=1 default=0 value=1
exposure_auto 0x009a0901 (menu) : min=0 max=3 default=3 value=3
exposure_absolute 0x009a0902 (int) : min=50 max=10000 step=1 default=166 value=50 flags=inactive
exposure_auto_priority 0x009a0903 (bool) : default=0 value=1
focus_absolute 0x009a090a (int) : min=0 max=1023 step=1 default=68 value=68 flags=inactive
focus_auto 0x009a090c (bool) : default=1 value=1

veye_xumm

@coolmitch
这个问题，可能是v4l2-ctl在一个一个查询参数配置的时候，遇到不支持的参数之后就停掉了。实际摄像头是支持power_line_frequency 的，要不然我们用pc的软件也会显示不支持。
当然怎样通过命令行去配置，我现在只能想到C语言调用v4l2接口一途。

coolmitch

@veye_xumm 一般是可以這樣設定
v4l2-ctl -d /dev/video1 --set-ctrl=power_line_frequency=2
不會報錯誤訊息

但我對 CS-USB-IMX307 設定時，回應是
v4l2-ctl -d /dev/video0 --set-ctrl=power_line_frequency=2
unknown control 'power_line_frequency'

對了，我去下載你說那個 potplayer 但畫面長不一樣，是不是版本不同？我找不到設定的地方....

veye_xumm

@coolmitch
可以参考一下这篇文章：
http://wiki.veye.cc/index.php/Windows系统下怎样查看USB摄像头视频

另外，我这边对opencv和python不太熟。使用你给的ffmpeg的方法，的确重现了一次问题。
现在我的结论是，并不是内存泄漏，需要进一步查找，困难度比预期的要高。
有没有更快重现问题的方法？

coolmitch

@veye_xumm
我用 ffmpeg 看到的錯誤訊息跟 python3+opencv 跑出來的有點不太一樣。但都需要跑很久（每次都大概需要 2~3hrs 左右）可以重現。（本來每分鐘跑一個循環需要 2 天半）。如果我給你 python3 的程式碼，你那邊能夠執行嗎？（需要有 opencv 安裝好）

veye_xumm

@coolmitch
我今天安装了一个3.2.0版本的opencv，跑你之前给的例程不行。
安装4.5.2的版本，失败了。

coolmitch

@veye_xumm
我重寫一個 python3 程式給你。照理沒有一定要 4.5.2，但你可以試看看 https://qengineering.eu/install-opencv-4.5-on-jetson-nano.html
我是照這個步驟編譯的。編譯過程一定要把 swap 打開，不然虛擬記憶體會不夠用。

veye_xumm

@coolmitch
我也是按照这个步骤做的。但是cmake阶段报错了。

coolmitch

@veye_xumm cmake 可能需要重新安裝新版。
參考這個（我有點忘記更新 cmake 的程序了)
https://stackoverflow.com/questions/49859457/how-to-reinstall-the-latest-cmake-version

----------------- BEGIN imx307_debug.py --------------------

import cv2

def run_once():

cap = cv2.VideoCapture('/dev/video0', cv2.CAP_V4L)
# cap.set(cv2.CAP_PROP_FOURCC, cv2.VideoWriter.fourcc('M','J','P','G'))
cap.set(cv2.CAP_PROP_FOURCC, cv2.VideoWriter.fourcc('h','2','6','4'))
cap.set(cv2.CAP_PROP_FRAME_WIDTH, 1920)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 1080)
cap.set(cv2.CAP_PROP_FPS, 30)

# 讀取 10 個 frames
for i in range(10):
    ret, frame = cap.read()

cap.release()

def main():
cap_count = 0
while True:
run_once()
print('-- %d --' % cap_count)
cap_count += 1

if name == 'main':
main()

----------------- END imx307_debug.py --------------------

coolmitch

重新把程式碼排版ㄧ下...

#! /usr/bin/python3
import cv2

def run_once():

    cap = cv2.VideoCapture('/dev/video0', cv2.CAP_V4L)
    # cap.set(cv2.CAP_PROP_FOURCC, cv2.VideoWriter.fourcc('M','J','P','G'))
    cap.set(cv2.CAP_PROP_FOURCC, cv2.VideoWriter.fourcc('h','2','6','4'))
    cap.set(cv2.CAP_PROP_FRAME_WIDTH, 1920)
    cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 1080)
    cap.set(cv2.CAP_PROP_FPS, 30)

    # 讀取 10 個 frames
    for i in range(10):
        ret, frame = cap.read()

    cap.release()

def main():
    cap_count = 0
    while True:
        run_once()
        print('-- %d --' % cap_count)
        cap_count += 1

if __name__ == '__main__':
    main()

veye_xumm

@coolmitch
请问你的cmake版本是？
我的是3.10.2

veye_xumm

@coolmitch
我更新了一下cmake，还是编译不过。
anyway，这事先放一放。
我现在改了一个版本的固件，用ffmpeg循环测试的方法，测试到6000+次数，还是稳定的。
请你也升级并老化测试一下：

下载链接：
https://www.mediafire.com/file/nnysng0map7jnf0/dimaging_upgrade.bin/file

升级方法：
http://wiki.veye.cc/index.php/USB升级固件方法

coolmitch

@veye_xumm
cmake --version
cmake version 3.10.2

CMake suite maintained and supported by Kitware (kitware.com/cmake).
我的也是 3.10.2
所以會不會是有什麼 dependent lib 沒有安裝完全？

coolmitch

@veye_xumm
升級後，開機看到這個
[ 48.421580] uvcvideo: Failed to query (GET_DEF) UVC control 9 on unit 2: -32 (exp. 2).
[ 53.659534] uvcvideo: Failed to query (GET_DEF) UVC control 9 on unit 2: -32 (exp. 2).
[ 53.672858] uvcvideo: Failed to query (GET_CUR) UVC control 11 on unit 2: -32 (exp. 1).
[ 53.681295] uvcvideo: Failed to query (GET_DEF) UVC control 9 on unit 2: -32 (exp. 2).
[ 89.889940] uvcvideo: Failed to query (GET_DEF) UVC control 9 on unit 2: -32 (exp. 2).
[ 410.317273] uvcvideo: Failed to resubmit video URB (-1).

不過我還是讓他去跑著測試了...
今晚先用 python3 的程式跑跑看...

veye_xumm

@coolmitch
我这边老化一夜，已经运行38000+次数，现在还是稳定的。期待你的结果。

coolmitch

@veye_xumm 我昨晚也跑了 22000+ 次，這樣應該就過了。所以是什麼原因造成的？

veye_xumm

@coolmitch
应该是我这边程序在处理close的时候，对资源的释放上有逻辑不完善的地方。希望你后面也继续测试观察吧。嵌入式端控制frequency的问题，我再研究一下。

coolmitch

@veye_xumm 瞭解。昨晚的還繼續在跑。我今天會做另一個測試站雙重測試。uvccamera 的參數看有沒有辦法讓有支援的都給出來，這樣在 linux 這邊會比較好處理（之前用 python 的那個 fourcc 的處理方式，因為沒有完整 uvc 的參數，搞了很久才找到那個怪方法...）

veye_xumm

@coolmitch
这个涉及到的改动太大了。暂时做不了所有参数都支持。现在我们只能是把摄像头默认参数调到最好。
另外就是你说的frequency hz设置的问题，需要解决一下。

coolmitch

@veye_xumm
對了，在 CS-USB-IMX307 新版韌體更新後，插入主機時會出現以下訊息，請問那些 Failed to query (GET_DEF) UVC control ... 是否就是 driver 要去讀取 uvccamera 參數時讀不到的錯誤？

[66391.767449] usb 1-2.4: new high-speed USB device number 5 using tegra-xusb
[66391.788802] usb 1-2.4: New USB device found, idVendor=12d1, idProduct=4321
[66391.788807] usb 1-2.4: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[66391.788810] usb 1-2.4: Product: CS-USB-IMX307
[66391.788813] usb 1-2.4: Manufacturer: Tianjin Zhonganyijia
[66391.788815] usb 1-2.4: SerialNumber: 2021-04-29-v1.5
[66391.790104] uvcvideo: Found UVC 1.10 device CS-USB-IMX307 (12d1:4321)
[66391.791116] uvcvideo 1-2.4:1.0: Entity type for entity Processing 2 was not initialized!
[66391.799329] uvcvideo 1-2.4:1.0: Entity type for entity Extension 10 was not initialized!
[66391.807507] uvcvideo 1-2.4:1.0: Entity type for entity Camera 1 was not initialized!
[66391.815533] input: CS-USB-IMX307 as /devices/70090000.xusb/usb1/1-2/1-2.4/1-2.4:1.0/input/input3
[66391.860569] uvcvideo: Failed to query (GET_DEF) UVC control 9 on unit 2: -32 (exp. 2).
[66391.868750] uvcvideo: Failed to query (GET_DEF) UVC control 9 on unit 2: -32 (exp. 2).
[66391.876889] uvcvideo: Failed to query (GET_DEF) UVC control 9 on unit 2: -32 (exp. 2).
[66391.885043] uvcvideo: Failed to query (GET_DEF) UVC control 9 on unit 2: -32 (exp. 2).
[66391.893245] uvcvideo: Failed to query (GET_DEF) UVC control 9 on unit 2: -32 (exp. 2).
[66391.901418] uvcvideo: Failed to query (GET_DEF) UVC control 9 on unit 2: -32 (exp. 2).
[66391.909611] uvcvideo: Failed to query (GET_DEF) UVC control 9 on unit 2: -32 (exp. 2).
[66391.917744] uvcvideo: Failed to query (GET_DEF) UVC control 9 on unit 2: -32 (exp. 2).
[66391.925863] uvcvideo: Failed to query (GET_DEF) UVC control 9 on unit 2: -32 (exp. 2).
[66391.933954] uvcvideo: Failed to query (GET_DEF) UVC control 9 on unit 2: -32 (exp. 2).
[66391.942076] uvcvideo: Failed to query (GET_DEF) UVC control 9 on unit 2: -32 (exp. 2).
[66391.950219] uvcvideo: Failed to query (GET_DEF) UVC control 9 on unit 2: -32 (exp. 2).
[66391.958366] uvcvideo: Failed to query (GET_DEF) UVC control 9 on unit 2: -32 (exp. 2).
[66391.966488] uvcvideo: Failed to query (GET_DEF) UVC control 9 on unit 2: -32 (exp. 2).
[66391.974603] uvcvideo: Failed to query (GET_DEF) UVC control 9 on unit 2: -32 (exp. 2).
[66391.982734] uvcvideo: Failed to query (GET_DEF) UVC control 9 on unit 2: -32 (exp. 2).
[66391.990897] uvcvideo: Failed to query (GET_DEF) UVC control 4 on unit 2: -32 (exp. 2).
[66391.999007] uvcvideo: Failed to query (GET_DEF) UVC control 4 on unit 2: -32 (exp. 2).
[66392.007219] uvcvideo: Failed to query (GET_DEF) UVC control 4 on unit 2: -32 (exp. 2).
[66392.015352] uvcvideo: Failed to query (GET_DEF) UVC control 4 on unit 2: -32 (exp. 2).
[66392.023488] uvcvideo: Failed to query (GET_DEF) UVC control 4 on unit 2: -32 (exp. 2).
[66392.031630] uvcvideo: Failed to query (GET_DEF) UVC control 4 on unit 2: -32 (exp. 2).
[66392.039743] uvcvideo: Failed to query (GET_DEF) UVC control 4 on unit 2: -32 (exp. 2).
[66392.047865] uvcvideo: Failed to query (GET_DEF) UVC control 4 on unit 2: -32 (exp. 2).
[66392.056021] uvcvideo: Failed to query (GET_DEF) UVC control 4 on unit 2: -32 (exp. 2).
[66392.064287] uvcvideo: Failed to query (GET_CUR) UVC control 11 on unit 2: -32 (exp. 1).
[66392.072494] uvcvideo: Failed to query (GET_CUR) UVC control 11 on unit 2: -32 (exp. 1).
[66392.080699] uvcvideo: Failed to query (GET_CUR) UVC control 11 on unit 2: -32 (exp. 1).
[66392.088904] uvcvideo: Failed to query (GET_CUR) UVC control 11 on unit 2: -32 (exp. 1).
[66392.097111] uvcvideo: Failed to query (GET_CUR) UVC control 11 on unit 2: -32 (exp. 1).
[66392.105311] uvcvideo: Failed to query (GET_CUR) UVC control 11 on unit 2: -32 (exp. 1).
[66392.113525] uvcvideo: Failed to query (GET_CUR) UVC control 11 on unit 2: -32 (exp. 1).
[66392.121728] uvcvideo: Failed to query (GET_CUR) UVC control 11 on unit 2: -32 (exp. 1).
[66392.129935] uvcvideo: Failed to query (GET_CUR) UVC control 11 on unit 2: -32 (exp. 1).
[66392.138162] uvcvideo: Failed to query (GET_CUR) UVC control 11 on unit 2: -32 (exp. 1).
[66392.146380] uvcvideo: Failed to query (GET_DEF) UVC control 8 on unit 2: -32 (exp. 2).
[66392.154495] uvcvideo: Failed to query (GET_DEF) UVC control 8 on unit 2: -32 (exp. 2).
[66392.162626] uvcvideo: Failed to query (GET_DEF) UVC control 8 on unit 2: -32 (exp. 2).
[66392.170738] uvcvideo: Failed to query (GET_DEF) UVC control 1 on unit 2: -32 (exp. 2).
[66392.178855] uvcvideo: Failed to query (GET_DEF) UVC control 1 on unit 2: -32 (exp. 2).
[66392.186981] uvcvideo: Failed to query (GET_DEF) UVC control 1 on unit 2: -32 (exp. 2).