UNSOLVED AR0234 on Rk3588s is unable to capture image

Megan

When I run the following command on my ar0234 camera on rk3588, it never exits, and the output file (y8-1920x1200.raw) remains empty:

radxa@radxa-cm5-io:~/rk35xx_veye_bsp/mv_tools_rockchip/i2c_tools$ v4l2-ctl -d /dev/video0 --set-fmt-video=width=1920,height=1200,pixelformat=GREY --stream-mmap --stream-count=1 --stream-to=y8-1920x1200.raw --verbose
VIDIOC_QUERYCAP: ok
VIDIOC_G_FMT: ok
VIDIOC_S_FMT: ok
Format Video Capture Multiplanar:
	Width/Height      : 1920/1200
	Pixel Format      : 'GREY' (8-bit Greyscale)
	Field             : None
	Number of planes  : 1
	Flags             : 
	Colorspace        : Default
	Transfer Function : Default
	YCbCr/HSV Encoding: Default
	Quantization      : Default
	Plane 0           :
	   Bytes per Line : 2048
	   Size Image     : 2457600
		VIDIOC_REQBUFS returned 0 (Success)
		VIDIOC_QUERYBUF returned 0 (Success)
		VIDIOC_QUERYBUF returned 0 (Success)
		VIDIOC_QUERYBUF returned 0 (Success)
		VIDIOC_QUERYBUF returned 0 (Success)
		VIDIOC_QBUF returned 0 (Success)
		VIDIOC_QBUF returned 0 (Success)
		VIDIOC_QBUF returned 0 (Success)
		VIDIOC_QBUF returned 0 (Success)
		VIDIOC_STREAMON returned 0 (Success)

Checking the dmesg I see the following,

radxa@radxa-cm5-io:~/rk35xx_veye_bsp/mv_tools_rockchip/i2c_tools$ sudo dmesg | tail -50
[168904.089384] stream_cif_mipi_id0: open video, entity use_countt 1
[168904.095199] stream_cif_mipi_id0: close video, entity use_count 0
[171834.286239] stream_cif_mipi_id0: open video, entity use_countt 1
[171834.300668] rkcif-mipi-lvds: stream[0] start streaming
[171834.302678] rockchip-mipi-csi2 fdd10000.mipi0-csi2: stream on, src_sd: 000000002b436c72, sd_name:rockchip-csi2-dphy0
[171834.302811] rockchip-mipi-csi2 fdd10000.mipi0-csi2: stream ON
[171834.302909] rockchip-csi2-dphy0: dphy0, data_rate_mbps 720
[171834.302963] rockchip-csi2-dphy csi2-dphy0: csi2_dphy_s_stream stream on:1, dphy0
[171834.302984] rockchip-csi2-dphy csi2-dphy0: csi2_dphy_s_stream stream on:1, dphy0
[171850.879291] rkcif-mipi-lvds: stream[0] start stopping, total mode 0x1, cur 0x1
[171850.879316] rkcif-mipi-lvds: get vblank fail, vblank_def 0, vblank_curr 0
[171851.404320] rockchip-mipi-csi2 fdd10000.mipi0-csi2: stream off, src_sd: 000000002b436c72, sd_name:rockchip-csi2-dphy0
[171851.404441] rockchip-mipi-csi2 fdd10000.mipi0-csi2: stream OFF
[171851.405589] rockchip-csi2-dphy csi2-dphy0: csi2_dphy_s_stream_stop stream stop, dphy0
[171851.405611] rockchip-csi2-dphy csi2-dphy0: csi2_dphy_s_stream stream on:0, dphy0
[171851.405651] rockchip-csi2-dphy csi2-dphy0: csi2_dphy_s_stream stream on:0, dphy0
[171851.409855] rkcif-mipi-lvds: stream[0] stopping finished, dma_en 0x0
[171851.417483] stream_cif_mipi_id0: close video, entity use_count 0
[171922.273442] stream_cif_mipi_id0: open video, entity use_countt 1
[171922.291475] rkcif-mipi-lvds: stream[0] start streaming
[171922.293546] rockchip-mipi-csi2 fdd10000.mipi0-csi2: stream on, src_sd: 000000002b436c72, sd_name:rockchip-csi2-dphy0
[171922.293679] rockchip-mipi-csi2 fdd10000.mipi0-csi2: stream ON
[171922.293779] rockchip-csi2-dphy0: dphy0, data_rate_mbps 720
[171922.293833] rockchip-csi2-dphy csi2-dphy0: csi2_dphy_s_stream stream on:1, dphy0
[171922.293854] rockchip-csi2-dphy csi2-dphy0: csi2_dphy_s_stream stream on:1, dphy0
[171938.510253] rkcif-mipi-lvds: stream[0] start stopping, total mode 0x1, cur 0x1
[171938.510415] rkcif-mipi-lvds: get vblank fail, vblank_def 0, vblank_curr 0
[171939.033950] rockchip-mipi-csi2 fdd10000.mipi0-csi2: stream off, src_sd: 000000002b436c72, sd_name:rockchip-csi2-dphy0
[171939.034072] rockchip-mipi-csi2 fdd10000.mipi0-csi2: stream OFF
[171939.035220] rockchip-csi2-dphy csi2-dphy0: csi2_dphy_s_stream_stop stream stop, dphy0
[171939.035242] rockchip-csi2-dphy csi2-dphy0: csi2_dphy_s_stream stream on:0, dphy0
[171939.035282] rockchip-csi2-dphy csi2-dphy0: csi2_dphy_s_stream stream on:0, dphy0
[171939.039471] rkcif-mipi-lvds: stream[0] stopping finished, dma_en 0x0
[171939.047702] stream_cif_mipi_id0: close video, entity use_count 0
[172297.000019] stream_cif_mipi_id0: open video, entity use_countt 1
[172297.014603] rkcif-mipi-lvds: stream[0] start streaming
[172297.016800] rockchip-mipi-csi2 fdd10000.mipi0-csi2: stream on, src_sd: 000000002b436c72, sd_name:rockchip-csi2-dphy0
[172297.016931] rockchip-mipi-csi2 fdd10000.mipi0-csi2: stream ON
[172297.017029] rockchip-csi2-dphy0: dphy0, data_rate_mbps 720
[172297.017082] rockchip-csi2-dphy csi2-dphy0: csi2_dphy_s_stream stream on:1, dphy0
[172297.017103] rockchip-csi2-dphy csi2-dphy0: csi2_dphy_s_stream stream on:1, dphy0
[172331.431364] rkcif-mipi-lvds: stream[0] start stopping, total mode 0x1, cur 0x1
[172331.431392] rkcif-mipi-lvds: get vblank fail, vblank_def 0, vblank_curr 0
[172331.954771] rockchip-mipi-csi2 fdd10000.mipi0-csi2: stream off, src_sd: 000000002b436c72, sd_name:rockchip-csi2-dphy0
[172331.954893] rockchip-mipi-csi2 fdd10000.mipi0-csi2: stream OFF
[172331.956044] rockchip-csi2-dphy csi2-dphy0: csi2_dphy_s_stream_stop stream stop, dphy0
[172331.956066] rockchip-csi2-dphy csi2-dphy0: csi2_dphy_s_stream stream on:0, dphy0
[172331.956107] rockchip-csi2-dphy csi2-dphy0: csi2_dphy_s_stream stream on:0, dphy0
[172331.960880] rkcif-mipi-lvds: stream[0] stopping finished, dma_en 0x0
[172331.969078] stream_cif_mipi_id0: close video, entity use_count 0

Could there be any parameters I overlooked in setting up my sensor/driver configuration? I would appreciate any hints for debugging this further.

Thanks in advance!

veye_xumm

@megan

I see that you have ported the driver to the Radxa CM5 board. Your I2C is already working, otherwise the camera wouldn't be recognized correctly. Another thing to check is that the MIPI DPHY and CSI interfaces are properly configured in the DTS.

With the correct DTS configuration, typically on the RK3588 platform, a script for capturing images would look something like this:

export I2C_BUS=7
export WIDTH=1920

export HEIGHT=1200

export FPS=50

v4l2-ctl -d /dev/v4l-subdev2 --set-ctrl roi_x=0

v4l2-ctl -d /dev/v4l-subdev2 --set-ctrl roi_y=0

media-ctl -d /dev/media0 --set-v4l2 '"m00_b_mvcam '"$I2C_BUS"'-003b":0[fmt:Y8_1X8/'"$WIDTH"'x'"$HEIGHT"'@1/'"$FPS"']'

v4l2-ctl --set-ctrl frame_rate=$FPS  -d /dev/v4l-subdev2

v4l2-ctl -d /dev/video0 --set-fmt-video=width=$WIDTH,height=$HEIGHT,pixelformat=GREY --stream-mmap --stream-count=-1 --stream-to=/dev/null

Megan

I attempted to run the script to capture an image but encountered the same issue—the final command does not exit.
(I am using 4 data lanes and tried both 60 FPS and 120 FPS).

Below is the DTS I used. I am not very familiar with configuring a DTS, so there is likely an error. Any help in identifying any mistake would be greatly appreciated!

// SPDX-License-Identifier: (GPL-2.0+ OR MIT)
/*
 * Copyright (c) 2022 www.veye.cc
 *
 */

&csi2_dphy0 {
	status = "okay";

	ports {
		#address-cells = <1>;
		#size-cells = <0>;
		port@0 {
			reg = <0>;
			#address-cells = <1>;
			#size-cells = <0>;

			mipi_in_ucam: endpoint@4 { 
				reg = <4>; 
				remote-endpoint = <&mvcam_out>;
				data-lanes = <1 2 3 4>;
			};
		};
		port@1 {
			reg = <1>;
			#address-cells = <1>;
			#size-cells = <0>;

			csidphy0_out: endpoint@0 {
				reg = <0>;
				remote-endpoint = <&mipi_csi0_input>;
			};
		};
	};
};

&csi2_dphy1 {
	status = "disabled";
};

&csi2_dphy2 {
	status = "disabled";
};


&csi2_dphy0_hw {
	status = "okay";
};

&i2c7 { 
	status = "okay";
	clock-frequency = <400000>; 
        pinctrl-names = "default";
        pinctrl-0 = <&i2c7m2_xfer>; 
	
	mvcam: mvcam@3b{
        status = "okay";
        compatible = "veye,mvcam"; 
        reg = <0x3b>; 
        clocks = <&cru CLK_CIFOUT_OUT>; 
        clock-names = "xvclk";
        //avdd-supply = <&vcc_mipi>;
        power-domains = <&power RK3588_PD_VI>; 
        pinctrl-names = "default";
        pinctrl-0 = <&cif_clk>;

        //power-gpios = <&pca9555 PCA_IO0_4 GPIO_ACTIVE_HIGH>;
        reset-gpios = <&gpio1 RK_PB1 GPIO_ACTIVE_HIGH>;  
		pwdn-gpios = <&gpio1 RK_PC1 GPIO_ACTIVE_HIGH>; 

        firefly,clkout-enabled-index = <0>;
        rockchip,camera-module-index = <0>;
        rockchip,camera-module-facing = "back";
        rockchip,camera-module-name = "NC";
        rockchip,camera-module-lens-name = "NC";
        port {
			mvcam_out: endpoint {
				remote-endpoint = <&mipi_in_ucam>;
                data-lanes = <1 2 3 4>;
            };
        };
    };
};

&mipi0_csi2 { 
	status = "okay";

	ports {
		#address-cells = <1>;
		#size-cells = <0>;

		port@0 {
			reg = <0>;
			#address-cells = <1>;
			#size-cells = <0>;

			mipi_csi0_input: endpoint@1 {
				reg = <1>;
				remote-endpoint = <&csidphy0_out>;
			};
		};

		port@1 {
			reg = <1>;
			#address-cells = <1>;
			#size-cells = <0>;

			mipi_csi0_output: endpoint@0 {
				reg = <0>;
				remote-endpoint = <&cif_mipi2_in0>;
			};
		};
	};
};

&rkcif {
	status = "okay";
//	memory-region = <&cif_reserved>;
};
&rkcif_mipi_lvds {
	status = "okay";

	port {
		cif_mipi2_in0: endpoint {
			remote-endpoint = <&mipi_csi0_output>; 
		};
	};
};
&rkcif_mipi_lvds_sditf {
	status = "disabled";

};
&rkcif_mmu {
	status = "okay";
};

//&rkisp {
//	status = "disabled";
//};

//&rkisp_vir0 {
//	status = "disabled";
//};

//&rkisp_vir1 {
//	status = "disabled";
//};

veye_xumm

@megan
First of all, please go to the following link to update the driver of mvcam. The latest driver will set the lane number in the camera according to the configuration of the DTS.

https://github.com/veyeimaging/rk35xx_veye_bsp/tree/main/linux/drivers/kernel_v5.10

Secondly, the DTS needs to be configured according to your RK3588 board, and I can't do it directly for you. You can make analogies and modifications by referring to the DTS of the camera originally supported by your board.