Using veye-mipi-327 with opencv

Nathan Sinclair

I'm piping the contents of this command to a python program which accepts a numpy array for displaying to screen.

veye_raspivid -t -1 -o -

In what raw data format is this command transmitting raw video?

veye_xumm

@nathan-sinclair
I think veye_raspivid transmitting h.264 encoded data.
I think maybe V4L2 mode is a proper option.
http://wiki.veye.cc/index.php/V4L2_mode_for_Raspberry_Pi

Nathan Sinclair

@veye_xumm
Has issues running on pi zero. When installing:
usage: sudo ./install_driver.sh veyecam2m/veye327/csimx307/cssc132
Enable i2c0 adapter...
./install_driver.sh: line 81: 741 Segmentation fault sudo modprobe i2c-dev
Add dtoverlay=veye327 to /boot/config.txt
Add gpu=400M to /boot/config.txt
Add cma=128M to /boot/cmdline.txt
Installing the veye327.ko driver
install: cannot stat './driver_bin/5.10.63+/veye327.ko': No such file or directory
install: cannot stat './driver_bin/5.10.63+/veye327.dtbo': No such file or directory
./install_driver.sh: line 116: 769 Segmentation fault sudo /sbin/depmod -a $(uname -r)

Nathan Sinclair

@nathan-sinclair @veye_xumm
Okay here's the progress i've made.
command uses MJPEG codec and rgb rawformat.
Pythoncode:
start_pipe

LiveCamera.pipe = subprocess.Popen(['/home/pi/veye/veye_raspcam/bin/veye_raspivid', '-cd', 'MJPEG', '-rf', 'rgb', '-n',  '-t', '0', '-o', '-'], stdout=subprocess.PIPE)

read from pipe

raw_image = LiveCamera.pipe.stdout.read(LiveCamera.width * LiveCamera.height * 3)
a = raw_image.find(b'xffxd8')  # JPEG start
b = raw_image.find(b'xffxd9')  # JPEG end
if a != -1 and b != -1:
    jpg = raw_image[a:b + 2]  # actual image
    bytesarray = raw_image[b + 2:]  # other informations

    # decode to colored image
    image = cv2.imdecode(np.fromstring(jpg, dtype=np.uint8), cv2.IMREAD_COLOR)

Doesn't read any data at the moment.

veye_xumm

@nathan-sinclair
You can try using the D-SDK:
http://wiki.veye.cc/index.php/VEYE-MIPI-290/327_for_Raspberry_Pi#D-SDK
program: capture2opencv or capture2opencv.py

Of course veye_raspividyuv is another option.

Nathan Sinclair

@veye_xumm I think from previous correspondence capture2opencv was only capable of capturing 1 frame per second, we're needing a live feed. I should have said that earlier. I'll have a look at it though. Trying out the YUV. will report what i find.

Nathan Sinclair

@veye_xumm
I'm not familiar with the raspividyuv program.
I am using the following command piped to my python program:

LiveCamera.pipe = subprocess.Popen(['/home/pi/veye/veye_raspcam/bin/veye_raspividyuv', '-t', '0','-o', '-'], stdout=subprocess.PIPE)

Got it going but it appears only to run maybe a few frames per second.
Also I'm neeeding a time frame in order to get this working. It's very important that we get this flipping going for a client. To be specific:
We're using a pi zero with a usb camera and MIPI-327 and both of these cameras need to be able to be flipped in the event the camera goes upside down. We're using python because a library exists for it that allows us to display the video streams without a gui. Arducam software needs to be able to run at 30fps at 1080p.
Isn't MJPEG an option that can be used? i just need a way to get the right data from the mjpeg stream and then i'm okay.
What data is being sent based on the codec option MJPEG or H264. I just need to know where the data is and use opencv to convert it to an image.

veye_xumm

@nathan-sinclair

You can use http://wiki.veye.cc/index.php/VEYE-MIPI-290/327_i2c/ mirrormode function to set MIPI-327 output flip image. No need to flip it via opencv.
MIPI-327 output format is UYVY(a type of YUV422), which is not supported by opencv originally. Using the cpu to do the format conversion resulted in a lower frame rate.
If you just need PREVIEW, why not use veye_raspipreview? It also allows for display position adjustment scaling etc. without the need for a gui.

Nathan Sinclair

@veye_xumm I was just looking at the i2c program. Call this resolved.