SOLVED Improve DE-coding on snapdragon devices
-
Snapdragon-based decoders have latency issues with streams where
poc_type is set to 0.
To reduce the end-to-end latency when using your camera, it would be beneficial to set poc_type to 2 when encoding -
this seems contradictory, but "lower is better" holds not true for this parameter
(in other words, value of 2 means: no picture re-ordering possible, what we want).
Read more about that: https://github.com/google/ExoPlayer/issues/8514Mfg,
Constantin -
Hello,
We have accepted the problem and arranged for engineers to study and solve it. Please wait patiently, thank you! -
Great ! if you are using v4l2, it should be simple:
https://linuxtv.org/downloads/v4l-dvb-apis/userspace-api/v4l/ext-ctrls-codec-stateless.html#c.v4l2_ctrl_h264_sps
pic_order_cnt_typeNote that it is close to impossible to just "overwrite" pic_order_cnt_type before encoding, it really needs to configured when encoding.
Nvidia jetson and rpi for example both allow setting pic_order_cnt_type before starting the encoding process.
I'd assume it's the same for other manufacturers of h264 ICs. -
Correcting my sentence, since I cannot edit the post anymore - should have said:
Note that it is impossible to just "overwrite" pic_order_cnt_type before decoding, it really needs to be configured when encoding.
-
@consti10
Hi,
I'm sorry to tell you that we can't modify the pic_order_cnt_type parameter right now. -
@veye_xumm
Hello, is this a HW or SW limitation ? -
If you take the rpi encoder for example, it defaults to poc_type=2.
The jetson omxh264enc defaults to poc_type=2 while the jetson nvvlh264enc defaults to poc_type=0 but can be configured to use
poc_type=2 in the latest release.
I'd suspect that pretty much all h264 encoder (no matter who produces them) should be capable of poc_type=2, just as they can be configured for different levels / resolution-framerates (as long as they are rated for the according level). -
@consti10
I think it's a hardware limit.
I will check again and get back to you later. -
I am not sure, but I read:
"H264 decoding parameters, must turn off the hardware decoder"
http://wiki.veye.cc/index.php/How_to_view_USB_camera_video_on_Android_OSturning off the HW decoder for h264 on android doesn't really make sense.
Maybe changing the poc type could fix this issue, too ?Rn I can observe two issues with this usb camera:
- When connected via USB (original cable) there can be artifacts (I suspect packet loss via usb or some driver bug)
- POC type as described above.
Else it's a really good product, the image quality is amazing and the end-to-end latency in 720p60 can be low if you got the right decoding HW (exynos).
-
@consti10 said in Improve DE-coding on snapdragon devices:
turning off the HW decoder for h264 on android doesn't really make sense.
Maybe changing the poc type could fix this issue, too ?I'm not sure about that.
- When connected via USB (original cable) there can be artifacts (I suspect packet loss via usb or some driver bug)
You are right. This camera will have a probability of packet loss. The higher the cpu performance of the host, the less packet loss.
This problem is still being studied and solved.- POC type as described above.
-
@veye_xumm said in Improve DE-coding on snapdragon devices:
I'm not sure about that.
Well, sw decoding is too slow for anything higher than 480p on mobile phones. If you see issues with the hw decoder, this is probably also due to packet loss - and the android sw decoder is often better in handling corrupted packets. poc-type=2 might help though, since it is also usually more resilient to misconfigurations on the hw decoder as a result of packet loss.
You are right. This camera will have a probability of packet loss. The higher the cpu performance of the host, the less packet loss.
This problem is still being studied and solved.I saw packet loss with an i7-8750H on my gigabyte aero 15X laptop, a really fast host CPU. Maybe the camera firmware has trouble re-sending usb packets.
-
When I search the firmware for the Hi3516v200 I find the following reference
h264e_set_poc
inside
hi3516ev200_h264.ko
poc pretty much always stands for "pic_order_count_type" in this context.So I doubt it is a hardware limitation by the encoder.
Especially since there is also the following comment:
pic_order_cnt_type err, should in [0,2]
e.g. poc_type can be in the range [0,2] where 2 is what I was asking for. -
@veye_xumm It would be nice to see some movement on this...
-
Hi, I am having the same issue. Could you please fix it as Consti suggested?
-
+1, the mentioned issue from @Consti10 is keeping away dozens of potential new customers. I really like your products as I own several Imx291, imx307 and imx327 models and picture quality is outstanding. So fixing this issue would convince lots of people to use it as fpv cams on UAVs!
-
Hi,
I'm also quite interest in this solution for future projects. Hope it gets fixed.
Cheers.
-
@consti10
I fully understand your thoughts and demands. The reply to you before is not easy, but after in-depth study.
I can't give you specific information about the chip. What I can tell you is that it's not 3516, but the firmware contains compatibility information for this scheme.With regard to poc, maybe the solution we are using now, as you said, the hardware itself supports the setting of poc. However, as a product development level, after communication with chip manufacturers, this parameter can not be set now.
The whole system is not completely opensource, nor can we directly control some registers and hardware.I'm sorry, but poc can't be modified for this camera. This is the end result.
-
@veye_xumm
Thank you for this conclusive response. Maybe your subsidary who provides the sw for the encoder chip can do something though -
From the official documentation for Hi3518 i found out that they actually use poc_type=2 by default.
So it is likely that the sw you are using is manually setting poc_type=0.For the OpenHD project, we test a lot of usb cameras. They all share the poc_type bug though (I refer to it as bug here, because it makes the camera unusable for low latency streaming on a lot of platforms).
Whoever is the first one to bring a usb camera to market with HiXXX chip and poc_type=2 basically has the lowest latency h264 usb camera on the market paired with rpi or snapdragon as decoder.
-
@consti10
Thank you for your detailed explanation.
But thanks to Trump, all Hisi chips have been discontinued. Don't expect anything from the Hixxxx chip in years.
