Intel® Edge Software Hub
Get answers from community peers to your questions about building Edge Software Hub solutions for edge compute node.
Announcements
The Edge Software Vision Package for Red Hat Enterprise Linux is now available here.

Learn more about developing on Intel® Hardware and Software here.

Gstreamer ingestor

Lucho
New Contributor I
2,603 Views

Hi, we are having some problems using gstreamer .

 

After a couple of seconds executing EII with an AVI file as input, we are having the following error in video ingestion, using EII 2.4.2:

 

[Wed Aug 25 18:52:02 2021]  INFO:run:145: Initializing Gstreamer pipeline

[Wed Aug 25 18:52:02 2021]  INFO:run:147: Gstreamer ingestor thread started

0:00:01.408703907    38 0x7fc24c009140 ERROR              h264parse gsth264parse.c:2795:gst_h264_parse_set_caps: video/x-h264 caps without codec_data or stream-format

0:00:01.422327566    38 0x7fc240066cf0 ERROR              h264parse gsth264parse.c:2795:gst_h264_parse_set_caps: video/x-h264 caps without codec_data or stream-format

0:00:01.442964822    38 0x7fc240066cf0 ERROR                  vaapi gstvaapiimage.c:104:vaapi_image_is_linear: FIXME: incomplete formats 444P

0:00:01.467961058    38 0x7fc240066cf0 ERROR                  vaapi gstvaapiimage.c:104:vaapi_image_is_linear: FIXME: incomplete formats 444P

0:00:01.509774979    38 0x7fc240066cf0 ERROR                  vaapi gstvaapiimage.c:104:vaapi_image_is_linear: FIXME: incomplete formats 444P

[Wed Aug 25 18:52:02 2021]  INFO:new_sample:257: Format: BGR, Size: 640x480

2021-08-25 18:52:02,480 : INFO  : Insecure Mode : VideoIngestion : [area_detection_udf.py] :process : in line : [184] : Crop Area: [(0, 0), (640, 480)] Detection Area: [(0, 0), (640, 480)]

[Wed Aug 25 18:53:33 2021]  INFO:bus_call:176: End of stream

[Wed Aug 25 18:53:33 2021]  INFO:run:150: Gstreamer ingestor thread stopped

 

Ingestor configurations are the recommended in the user guide:

"ingestor": {
            "loop_video": true,
            "pipeline": "multifilesrc location=./video.avi loop=TRUE ! decodebin ! videoconvert ! videoscale ! video/x-raw,format=BGR,width=640,height=480 ! appsink",
            "poll_interval": 0.04,
            "queue_size": 10,
            "type": "gstreamer"
        }

 

It seems that the video is not starting again. We have changed  run() function in gstreamer ingestor cpp file to add a loop as a work around:


void GstreamerIngestor::run(bool snapshot_mode) {
#ifdef WITH_PROFILE
    auto start = std::chrono::system_clock::now();
#endif
    if (snapshot_mode) {
        m_frame_count = 0;
    }
    for(int i=0; i < 200; ++i){
        LOG_INFO("Video loop iteration: %d", i);
        LOG_INFO_0("Initializing Gstreamer pipeline");
        gstreamer_init(snapshot_mode);
        LOG_INFO_0("Gstreamer ingestor thread started");
        gst_element_set_state(m_gst_pipeline, GST_STATE_PLAYING);
        g_main_loop_run(m_loop);
        LOG_INFO_0("Restart Gstreamer ingestor thread");
        usleep(1000000);
    }
    LOG_INFO_0("Gstreamer ingestor thread stopped");

#ifdef WITH_PROFILE
    // This code block will execute only when g_main_loop ends
    // and it can be triggered by stopping the ingestor source
    auto end = std::chrono::system_clock::now();
    int elapsed = std::chrono::duration_cast<std::chrono::seconds>(
            end - start).count();
    LOG_INFO("GStreamer FPS: %d", m_frame_count / elapsed);
    char* str_app_name = NULL;
    str_app_name = getenv("AppName");
    std::ofstream fps_file;
    fps_file.open("/var/tmp/fps.txt", std::ofstream::app);
    fps_file << str_app_name << " FPS : " << (m_frame_count / elapsed) << std::endl ;
    fps_file.close();
#endif
}

 

Adding a loop in the gstreamer ingestor have fixed the problem, and we have a continuous execution, but this error keeps appearing in ingestion logs:

ERROR                  vaapi gstvaapiimage.c:104:vaapi_image_is_linear: FIXME: incomplete formats 444P


Is this a known issue? or maybe a problem with the configurations. Any feeedback on this will be appreciated.

 

Regards
 

Labels (1)
0 Kudos
9 Replies
JesusE_Intel
Moderator
2,587 Views

Hi Lucho,

 

Thanks for reaching out! Have you tried using the latest EII version 2.6 and see if the same behavior is observed? Also, I found the following in the multifilesrc doc, could you give that a try?

 

In case one notices general stream error with multifilesrc element when certain video files are used then transcode the video file to H264 video with .avi container format to ensure the compatibity of the format of the video file.

 

Regards,

Jesus

 

Lucho
New Contributor I
2,551 Views

We have tried using .avi container with H264 codec:

 h264codec.png

having the same results.

Video loop keeps failing:

[Mon Aug 30 20:24:19 2021] INFO:bus_call:177: End of stream
[Mon Aug 30 20:24:19 2021] INFO:run:151: Gstreamer ingestor thread stopped

 

with the same configurations:

"ingestor": {
        "loop_video": true,
        "pipeline": "multifilesrc loop=TRUE location=./video.avi ! decodebin ! videoconvert ! videoscale ! video/x-raw,format=BGR,width=1024,height=720 ! appsink",
        "poll_interval": 0.04,
        "queue_size": 10,
        "type": "gstreamer"
    }

 

we will try with EII 2.6.

0 Kudos
JesusE_Intel
Moderator
2,544 Views

Hi Lucho,


Sounds good, please let me know the outcome of your tests with EII version 2.6.


Regards,

Jesus


0 Kudos
Lucho
New Contributor I
2,529 Views

Hi Jesus,

 

We have tried with EII 2.6, with the same results. Output:

[Fri Sep  3 15:07:59 2021]  INFO:run:147: Gstreamer ingestor thread started
error: XDG_RUNTIME_DIR not set in the environment.
[Fri Sep  3 15:08:00 2021]  INFO:new_sample:257: Format: BGR, Size: 1024x720
2021-09-03 15:08:00,061 : INFO  : {} : VideoIngestion : [area_detection_udf.py] :process : in line : [184] : Crop Area: [(0, 0), (1024, 720)] Detection Area: [(0, 0), (1024, 720)]
[Fri Sep  3 15:09:17 2021]  INFO:bus_call:176: End of stream
[Fri Sep  3 15:09:17 2021]  INFO:run:150: Gstreamer ingestor thread stopped

 

Video ingestion config:

"ingestor": {
        "loop_video": true,
        "pipeline": "multifilesrc location=./video.avi loop=TRUE ! decodebin ! videoconvert ! videoscale ! video/x-raw,format=BGR,width=1024,height=720 ! appsink",
        "poll_interval": 0.04,
        "queue_size": 10,
        "type": "gstreamer"
    },

 

Is there any configuration that we can change to solve this issue? 

 

Thanks

0 Kudos
Murni_J_Intel
Moderator
2,463 Views

Hi @Lucho 

 

For the videoscale element, it needs to be processed before the videoconvert element as the example shown on the guide: https://github.com/open-edge-insights/video-ingestion/blob/master/docs/usb_doc.md

 

can you try using the same configuration to see if you still encounter the same error?

 

Lucho
New Contributor I
2,431 Views

Hi Murni,

 

We have removed videoscale due to an error tring to change the order. Config:

"ingestor": {
        "loop_video": true,
        "pipeline": "multifilesrc location=./video.avi loop=TRUE ! decodebin ! videoconvert ! video/x-raw,format=BGR ! appsink",
        "poll_interval": 0.04,
        "queue_size": 10,
        "type": "gstreamer"
    }

 

Video loop keeps failing:

[Thu Sep  9 16:16:33 2021]  INFO:run:145: Initializing Gstreamer pipeline
[Thu Sep  9 16:16:33 2021]  INFO:run:147: Gstreamer ingestor thread started
error: XDG_RUNTIME_DIR not set in the environment.
[Thu Sep  9 16:16:33 2021]  INFO:new_sample:257: Format: BGR, Size: 1920x1080
2021-09-09 16:16:33,712 : INFO  : {} : VideoIngestion : [area_detection_udf.py] :process : in line : [184] : Crop Area: [(0, 0), (1920, 1080)] Detection Area: [(0, 0), (1920, 1080)]
[Thu Sep  9 16:18:56 2021]  INFO:bus_call:176: End of stream
[Thu Sep  9 16:18:56 2021]  INFO:run:150: Gstreamer ingestor thread stopped

 

0 Kudos
Murni_J_Intel
Moderator
2,424 Views

Hi @Lucho 

 

Thank you for your feedback. May I check with you, is it working with the existing PCB demo video? is the video.avi that you used being place on the test_videos folder? if yes, please help to change the path on the configuration.

I would suggest you try out with the existing available demo video to see if the error still persists so that we can reproduce it on our side.  You can also try without using any filter or classifier UDF by passing "dummy" to the UDF configuration in Video Ingestion and Video Analytics container. You can refer to the https://open-edge-insights.github.io/pages/video.html#working-with-video-data section on how to configure "dummy" UDF. This will help to see if the error caused by the ingestion pipeline 

 

0 Kudos
JesusE_Intel
Moderator
2,500 Views

Hi Lucho,


Let me check with my peers and get back to you.


Regards,

Jesus


0 Kudos
JesusE_Intel
Moderator
2,348 Views

If you need any additional information, please submit a new question as this thread will no longer be monitored.


0 Kudos
Reply