Save inference results to influxDB

[Huang__Colin]

Hi intel,

We have an use case with ESH.

We want to save the inference result to influxDB.

Could you give me some ideas that we can easily save results and read them ?

Thanks 

[Jia-Shing_W_Intel]

Hi,

Here is a brief summary of how to store video analytics results in InfluxDB. I’ll use docker_setup/samples/docker-compose-video-streaming-timeseries-usecase.yml as an example as it has a wider coverage of EIS components.

• If you look at the ia_video_analytics section, you can find the EIS Message Bus topics it wants to publish and subscribe, how they are done (TCP or IPC, IP addresses and port numbers, and the allowed client list):

    environment:

      AppName: "VideoAnalytics"

      DEV_MODE: ${DEV_MODE}

      PROFILING_MODE: ${PROFILING_MODE}

      no_proxy: ${eis_no_proxy},${ETCD_HOST}

      ETCD_HOST: ${ETCD_HOST}

      # MessageBus Endpoint Configuration

      Clients : "Visualizer,FactoryControlApp,OpcuaExport,ImageStore,InfluxDBConnector,RestDataExport,WebVisualizer,TLS_RemoteAgent" --> a list of allowed clients

      CertType: "zmq"

      ZMQ_RECV_HWM: "1000"

      # Use IPC mode (zmq_ipc) when VideoIngestion and VideoAnalytics based

      # containers are running on the same host. If they are running on diff

      # host, please use TCP mode for communication.

      # Eg: Stream cfg for

      # IPC: zmq_ipc, <absolute_socket_directory_path>

      # TPC: zmq_tcp, <publisher_host>:<publisher_port>

      SubTopics: "VideoIngestion/camera1_stream" --> topic to subscribe

      camera1_stream_cfg: "zmq_ipc,${SOCKET_DIR}/" --> subscription method

      PubTopics: "camera1_stream_results" --> topic to publish

      camera1_stream_results_cfg: "zmq_tcp,127.0.0.1:65013" --> publishing method

As you can see, InfluxDBConnector is already listed among the allowed list.

• Next, look at the ia_influxdbconnector section. Here, you can see the reference setting already subscribes to the camera1_stream_results video analytics results.

    environment:

      AppName: "InfluxDBConnector"

      CertType: "zmq,pem"

      DEV_MODE: ${DEV_MODE}

      PROFILING_MODE: ${PROFILING_MODE}

      no_proxy: ${eis_no_proxy},${ETCD_HOST}

      ETCD_HOST: ${ETCD_HOST}

      # MessageBus Endpoint Configuration

      Clients: "Visualizer,DiscoveryCreek"

      Server: "zmq_tcp,127.0.0.1:8675"

      # Server: "zmq_ipc,${SOCKET_DIR}/"

      INFLUXDB_TLS_CIPHERS: ${TLS_CIPHERS}

      # Use IPC mode (zmq_ipc) when VideoIngestion, VideoAnalytics and Visualizer

      # based containers are running on the same host. If they are running on diff

      # host, please use TCP mode for communication.

      # Eg: Stream cfg for

      # IPC: zmq_ipc, <absolute_socket_directory_path>

      # TPC: zmq_tcp, <publisher_host>:<publisher_port>

      SubTopics: "VideoAnalytics/camera1_stream_results,DiscoveryCreek/dc_point_data_results"

      dc_point_data_results_cfg: "zmq_tcp,127.0.0.1:65014"

      camera1_stream_results_cfg: "zmq_tcp,127.0.0.1:65013"

      PubTopics: "point_data,point_classifier_results"

      point_data_cfg: "zmq_tcp,127.0.0.1:65015"

      # point_data_cfg: "zmq_ipc,${SOCKET_DIR}/"

      point_classifier_results_cfg: "zmq_tcp,127.0.0.1:65016"

If you do Python, each time you receive an EIS Message Bus message from this VideoAnalytics/camera1_stream_results topic, it has the following three elements:

1. The topic name, in this case, camera1_stream_results
2. Metadata associated with this message, for example, {'channels': 3, 'encoding_type': 'jpeg', 'height': 1200, 'defects': [{'type': 0, 'tl': [516, 160], 'br': [567, 201]}, {'type': 0, 'tl': [627, 568], 'br': [653, 607]}, {'type': 0, 'tl': [1098, 174], 'br': [1160, 210]}, {'type': 1, 'tl': [549, 519], 'br': [587, 556]}], 'img_handle': '661b3eb5bb', 'width': 1920, 'encoding_level': 96}
3. A BLOB of the video frame that was inferenced.

C++ and the Go language have their corresponding representations of this.

As EIS provides a reference and tries to be generic, the format of what get stored in the InfluxDB may not be what you want. You can parse through the metadata, extract what you want and store it.

If you simply want to store video analytics results in InfluxDB for later processing, you don’t even need telegraf, kapacitor, or the mqtt-broker. This is because InfluxDBConnector communicates with VideoAnalytics directly via the EIS Message Bus.

Regards,

Jia-Shing