topic Gstreamer ingestor in Edge Software Catalog

Gstreamer ingestor

Lucho — Thu, 26 Aug 2021 21:23:13 GMT

Hi, we are having some problems using gstreamer .

After a couple of seconds executing EII with an AVI file as input, we are having the following error in video ingestion, using EII 2.4.2:

[Wed Aug 25 18:52:02 2021] INFO:run:145: Initializing Gstreamer pipeline [Wed Aug 25 18:52:02 2021] INFO:run:147: Gstreamer ingestor thread started 0:00:01.408703907 38 0x7fc24c009140 ERROR h264parse gsth264parse.c:2795:gst_h264_parse_set_caps: video/x-h264 caps without codec_data or stream-format 0:00:01.422327566 38 0x7fc240066cf0 ERROR h264parse gsth264parse.c:2795:gst_h264_parse_set_caps: video/x-h264 caps without codec_data or stream-format 0:00:01.442964822 38 0x7fc240066cf0 ERROR vaapi gstvaapiimage.c:104:vaapi_image_is_linear: FIXME: incomplete formats 444P 0:00:01.467961058 38 0x7fc240066cf0 ERROR vaapi gstvaapiimage.c:104:vaapi_image_is_linear: FIXME: incomplete formats 444P 0:00:01.509774979 38 0x7fc240066cf0 ERROR vaapi gstvaapiimage.c:104:vaapi_image_is_linear: FIXME: incomplete formats 444P [Wed Aug 25 18:52:02 2021] INFO:new_sample:257: Format: BGR, Size: 640x480 2021-08-25 18:52:02,480 : INFO : Insecure Mode : VideoIngestion : [area_detection_udf.py] :process : in line : [184] : Crop Area: [(0, 0), (640, 480)] Detection Area: [(0, 0), (640, 480)] [Wed Aug 25 18:53:33 2021] INFO:bus_call:176: End of stream [Wed Aug 25 18:53:33 2021] INFO:run:150: Gstreamer ingestor thread stopped

Ingestor configurations are the recommended in the user guide:

"ingestor": { "loop_video": true, "pipeline": "multifilesrc location=./video.avi loop=TRUE ! decodebin ! videoconvert ! videoscale ! video/x-raw,format=BGR,width=640,height=480 ! appsink", "poll_interval": 0.04, "queue_size": 10, "type": "gstreamer" }

It seems that the video is not starting again. We have changed run() function in gstreamer ingestor cpp file to add a loop as a work around:

void GstreamerIngestor::run(bool snapshot_mode) { #ifdef WITH_PROFILE auto start = std::chrono::system_clock::now(); #endif if (snapshot_mode) { m_frame_count = 0; } for(int i=0; i < 200; ++i){ LOG_INFO("Video loop iteration: %d", i); LOG_INFO_0("Initializing Gstreamer pipeline"); gstreamer_init(snapshot_mode); LOG_INFO_0("Gstreamer ingestor thread started"); gst_element_set_state(m_gst_pipeline, GST_STATE_PLAYING); g_main_loop_run(m_loop); LOG_INFO_0("Restart Gstreamer ingestor thread"); usleep(1000000); } LOG_INFO_0("Gstreamer ingestor thread stopped"); #ifdef WITH_PROFILE // This code block will execute only when g_main_loop ends // and it can be triggered by stopping the ingestor source auto end = std::chrono::system_clock::now(); int elapsed = std::chrono::duration_cast<std::chrono::seconds>( end - start).count(); LOG_INFO("GStreamer FPS: %d", m_frame_count / elapsed); char* str_app_name = NULL; str_app_name = getenv("AppName"); std::ofstream fps_file; fps_file.open("/var/tmp/fps.txt", std::ofstream::app); fps_file << str_app_name << " FPS : " << (m_frame_count / elapsed) << std::endl ; fps_file.close(); #endif }

Adding a loop in the gstreamer ingestor have fixed the problem, and we have a continuous execution, but this error keeps appearing in ingestion logs:

ERROR vaapi gstvaapiimage.c:104:vaapi_image_is_linear: FIXME: incomplete formats 444P

Is this a known issue? or maybe a problem with the configurations. Any feeedback on this will be appreciated.

Regards

Re: Gstreamer ingestor

JesusE_Intel — Fri, 27 Aug 2021 17:58:27 GMT

Hi Lucho,

Thanks for reaching out! Have you tried using the latest EII version 2.6 and see if the same behavior is observed? Also, I found the following in the multifilesrc doc, could you give that a try?

In case one notices general stream error with multifilesrc element when certain video files are used then transcode the video file to H264 video with .avi container format to ensure the compatibity of the format of the video file.

Regards,

Jesus

Re: Gstreamer ingestor

Lucho — Mon, 30 Aug 2021 20:41:48 GMT

We have tried using .avi container with H264 codec:

having the same results.

Video loop keeps failing:

[Mon Aug 30 20:24:19 2021] INFO:bus_call:177: End of stream
[Mon Aug 30 20:24:19 2021] INFO:run:151: Gstreamer ingestor thread stopped

with the same configurations:

"ingestor": { "loop_video": true, "pipeline": "multifilesrc loop=TRUE location=./video.avi ! decodebin ! videoconvert ! videoscale ! video/x-raw,format=BGR,width=1024,height=720 ! appsink", "poll_interval": 0.04, "queue_size": 10, "type": "gstreamer" }

we will try with EII 2.6.

Re:Gstreamer ingestor

JesusE_Intel — Wed, 01 Sep 2021 17:37:00 GMT

Hi Lucho,

Sounds good, please let me know the outcome of your tests with EII version 2.6.

Regards,

Jesus

Re: Re:Gstreamer ingestor

Lucho — Fri, 03 Sep 2021 15:19:06 GMT

Hi Jesus,

We have tried with EII 2.6, with the same results. Output:

[Fri Sep 3 15:07:59 2021] INFO:run:147: Gstreamer ingestor thread started error: XDG_RUNTIME_DIR not set in the environment. [Fri Sep 3 15:08:00 2021] INFO:new_sample:257: Format: BGR, Size: 1024x720 2021-09-03 15:08:00,061 : INFO : {} : VideoIngestion : [area_detection_udf.py] :process : in line : [184] : Crop Area: [(0, 0), (1024, 720)] Detection Area: [(0, 0), (1024, 720)] [Fri Sep 3 15:09:17 2021] INFO:bus_call:176: End of stream [Fri Sep 3 15:09:17 2021] INFO:run:150: Gstreamer ingestor thread stopped

Video ingestion config:

"ingestor": { "loop_video": true, "pipeline": "multifilesrc location=./video.avi loop=TRUE ! decodebin ! videoconvert ! videoscale ! video/x-raw,format=BGR,width=1024,height=720 ! appsink", "poll_interval": 0.04, "queue_size": 10, "type": "gstreamer" },

Is there any configuration that we can change to solve this issue?

Thanks

Re:Gstreamer ingestor

JesusE_Intel — Tue, 07 Sep 2021 18:12:49 GMT

Hi Lucho,

Let me check with my peers and get back to you.

Regards,

Jesus

Re: Re:Gstreamer ingestor

Murni_J_Intel — Thu, 09 Sep 2021 04:19:58 GMT

Hi @Lucho

For the videoscale element, it needs to be processed before the videoconvert element as the example shown on the guide: https://github.com/open-edge-insights/video-ingestion/blob/master/docs/usb_doc.md

can you try using the same configuration to see if you still encounter the same error?

Re: Re:Gstreamer ingestor

Lucho — Thu, 09 Sep 2021 16:23:25 GMT

Hi Murni,

We have removed videoscale due to an error tring to change the order. Config:

"ingestor": { "loop_video": true, "pipeline": "multifilesrc location=./video.avi loop=TRUE ! decodebin ! videoconvert ! video/x-raw,format=BGR ! appsink", "poll_interval": 0.04, "queue_size": 10, "type": "gstreamer" }

Video loop keeps failing:

[Thu Sep 9 16:16:33 2021] INFO:run:145: Initializing Gstreamer pipeline [Thu Sep 9 16:16:33 2021] INFO:run:147: Gstreamer ingestor thread started error: XDG_RUNTIME_DIR not set in the environment. [Thu Sep 9 16:16:33 2021] INFO:new_sample:257: Format: BGR, Size: 1920x1080 2021-09-09 16:16:33,712 : INFO : {} : VideoIngestion : [area_detection_udf.py] :process : in line : [184] : Crop Area: [(0, 0), (1920, 1080)] Detection Area: [(0, 0), (1920, 1080)] [Thu Sep 9 16:18:56 2021] INFO:bus_call:176: End of stream [Thu Sep 9 16:18:56 2021] INFO:run:150: Gstreamer ingestor thread stopped

Re: Re:Gstreamer ingestor

Murni_J_Intel — Thu, 09 Sep 2021 23:55:44 GMT

Hi @Lucho

Thank you for your feedback. May I check with you, is it working with the existing PCB demo video? is the video.avi that you used being place on the test_videos folder? if yes, please help to change the path on the configuration.

I would suggest you try out with the existing available demo video to see if the error still persists so that we can reproduce it on our side. You can also try without using any filter or classifier UDF by passing "dummy" to the UDF configuration in Video Ingestion and Video Analytics container. You can refer to the https://open-edge-insights.github.io/pages/video.html#working-with-video-data section on how to configure "dummy" UDF. This will help to see if the error caused by the ingestion pipeline

Re:Gstreamer ingestor

JesusE_Intel — Thu, 30 Sep 2021 18:03:01 GMT

If you need any additional information, please submit a new question as this thread will no longer be monitored.