Issue with running discovery tool.

[Jacky_Gupta]

I am trying to retrieve frames from  Image store  using discoverHistory tool,but get  only "2020-11-05_17:15:20.dat" files (Not Frames)inside eishistory folder where by default the frame should get saved.
 I am using a simple 2 line of UDF as shown below which takes  input frame ,convert it into gray and display using visulzer.
 
 ************************UDF********************************************************
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
gray = self.gray.reshape((self.gray.shape[0],self.gray.shape[1],1))
**************************************************************************************
I have followed all the steps mentioned in Readme.md inside DIscoveryHostory Folder.Also i am using Dev mode using IPC mode of communication.

Can you please guide me so that i can retrieve frames.
I have attached logs of  DiscoveryHIstory tool / entire_EIS logs / docker-compose.yml and eis_config.json  of  build folder/ query_config.json and eis_config.json  of discoveryHistory tool and also the UDFs

also i had changed to below configuration in IPC mode but reverted to ""type": "zmq_tcp", in eis_config of discoveryHistory as it was not saving even "2020-11-05_17:15:20.dat" files in this case.

    {
        "type": "zmq_ipc",
        "socket_dir": "/EIS/sockets/"
    }


Regards
Jacky

[Johnbelley]

Could you describe what doesn't work exactly?Are you running the 32 bit version or 64 bit?

[Jacky_Gupta]

When i run discover History Tool,i don't get frames saved in eishistory folder.

below logs are displayed and it exits.Also i am using 64 bit machine

*******************************************************************************************************

ds0xxx:~/EIS_2.3.2_new/edge_insights_industrial/Edge_Insights_for_Industrial_2.3.2/IEdgeInsights/tools/DiscoverHistory$ docker-compose up
WARNING: The NO_PROXY variable is not set. Defaulting to a blank string.
WARNING: The eis_no_proxy variable is not set. Defaulting to a blank string.
WARNING: The ETCD_HOST variable is not set. Defaulting to a blank string.
WARNING: The ETCD_PREFIX variable is not set. Defaulting to a blank string.
Removing ia_discover_history
Recreating 930eae6852b7_ia_discover_history ... done
Attaching to ia_discover_history
ia_discover_history | 2020-11-06 12:37:17,631 : INFO : Insecure Mode : root : [influxdbconnector_client.py] :query_influxdb : in line : [44] : Initializing message bus context
ia_discover_history | 2020-11-06 12:37:17,632 : INFO : Insecure Mode : root : [imagestore_client.py] :retrieve_image_frames : in line : [139] : Initializing service for topic 'imagestore_service'
ia_discover_history | 2020-11-06 12:37:17,632 : INFO : Insecure Mode : root : [influxdbconnector_client.py] :query_influxdb : in line : [47] : Initializing service for topic '{"influxconnector_service"}'
ia_discover_history | 2020-11-06 12:37:17,887 : INFO : Insecure Mode : root : [influxdbconnector_client.py] :query_influxdb : in line : [50] : Running...
ia_discover_history | 2020-11-06 12:37:17,887 : INFO : Insecure Mode : root : [influxdbconnector_client.py] :query_influxdb : in line : [51] : Sending request {'command': 'select * from camera1_stream order by desc limit 10'}
ia_discover_history | 2020-11-06 12:37:17,888 : INFO : Insecure Mode : root : [influxdbconnector_client.py] :query_influxdb : in line : [53] : Waiting for response
ia_discover_history | 2020-11-06 12:37:17,886 : INFO : Insecure Mode : root : [imagestore_client.py] :retrieve_image_frames : in line : [142] : Running...
ia_discover_history | 2020-11-06 12:37:31,808 : INFO : Insecure Mode : root : [main.py] :main : in line : [101] : Exiting...

*********************************************************************************************

[Murni_J_Intel]

Hi @Jacky_Gupta 

Let me check on this issue and get back to you as soon as possible, sorry to keep you waiting. 

[Jacky_Gupta]

Hi Murni,

Let me know if you need any help from my side.we have to make some urgent decision regarding the use  of discovery tool in our project.

If you can connect to my system using webex/Teamviewer/anydesk  etc it would be very helpful,as i can show you the problem .

Also,you can watsapp me at 7795591292.

regards

Jacky

[Jacky_Gupta]

Hi all,

i have debugged further and found it is unable to put data inside image handle queue as value of check is false which can be seen in attached snapshot.I have attached latest log for reference.

*******************************************************************

check = all(item in loaded_json['columns'] for item in valid_input)

************************************************************************

[Murni_J_Intel]

Hi @Jacky_Gupta 

This issue only happens if you are running on IPC mode or this will happen in TCP as well? may I know if you are using the PCB demo default use -case for running the tools? It will be easier if we can help to root cause the problem step by step.  

I suggest you try running the DiscoveryHistory tools by using the TCP mode with the default example (PCB) that has been given to ensure you have the setup right.  You will need to ensure the ImageStore and InfluxDB container is up and running.  After done with this, you can change to run using IPC mode with the same default use-case. 

[Jacky_Gupta]

Hi @Murni_J_Intel ,
As per your suggestion I have downloaded EIS2.3.2 again which had PCB demo and TCP mode as default use case .It had ImageStore & InfluxDB Container running and build Discovery Tool as per the
read.md of Discovery tool.It was able to save good/bad frames along with metadata in default path as well.
Later changed to IPC mode as shown below in config.json of discovery tool and rebuild it.But when i ran the tool using Docker-compose up it was not working even with PCB demo.

{
"type": "zmq_ipc",
"socket_dir": "/EIS/sockets/"
}

I have attached snapshot of logs of disccovey tool.

I request you if possible , we can connect over webex/Teamviewer at your preferred timing.

currently, my objective is to take any other video apart from provided in EIS package ,use a simple UDF (which takes frame and output it without any VA processing) and use discovery tool(in any mode IPC/TCP) for retrieving frame along with metadata.If you help me to establish this pipeline then i can proceed with my actual UDF.
Also,when i do provisioning it takes time ranging in between 1 min to 1 and half hour.Earlier it use to provision within minutes.

Regards
Jacky

[Murni_J_Intel]

Hi @Jacky_Gupta 

Thank you for your feedback from the previous post, from the log that been given seems like the tools having a problem to get inferencing frame.  Below is the changes step that you need to do in order to change the default TCP mode to IPC mode

Configure the docker-compose.yml file at [WORK_DIR]/IEdgeInsight/build directory

1. Change the video analytics container configuration as below

camera1_stream_results_cfg: "zmq_ipc,${SOCKET_DIR}/"

2. Change the visualizer container configuration as below

camera1_stream_results_cfg: "zmq_ipc,${SOCKET_DIR}/"

# user: ${EIS_UID} (Run on root)

3. Change the influxdbconnector container configuration as below

camera1_stream_results_cfg: "zmq_ipc,${SOCKET_DIR}/"

Server: "zmq_ipc,${SOCKET_DIR}/"

# user: ${EIS_UID}

 4. Change the imagestore container configuration as below

camera1_stream_results_cfg: "zmq_ipc,${SOCKET_DIR}/"

 Server: "zmq_ipc,${SOCKET_DIR}/"

 # user: ${EIS_UID}

Configure the DiscoveryHistory tool to run on IPC

open the eis_config.json at config directory

 {
"type": "zmq_ipc",
"socket_dir": "/EIS/sockets"
}

Now after done all of the changes, you can bring up the EII container at [WORK_DIR]/IEdgeInsight/build by using below command. After done you should see the PCB demo is up and running via visualizer. 

$ docker-compose up -d 

Then you can re-run the DiscoverHistory tool and see if the previous problem still persists. 

Thank you. 

[Jacky_Gupta]

@Murni_J_Intel ,Thanks I was able to run PCB demo along with discovery tool to retrieve and save images in IPC mode.

so what should be my next step to meet below objective as per my previous communication.

"currently, my objective is to take any other video apart from provided in EIS package ,use a simple UDF (which takes frame and output it without any VA processing) and use discovery tool(in any mode IPC/TCP) for retrieving frame along with metadata.If you help me to establish this pipeline then i can proceed with my actual UDF."

[Murni_J_Intel]

Hi @Jacky_Gupta 

In order to use your own video, you can include the new video in the same directory as the sample video (VideoIngestion/test/).  You can also create a new directory consist of the new udf and model that will be used. After doing this, you will need to build again the container to include the newly added video and udf by following the below step.

1. $ cd [WORK_DIR]/IEdgeInsights/build

2. $ dokcer-compose up --build -d 

After done the above step, you change the pipeline for videoIngestion config at etcd Web UI.  For VideoIngestion, you can use dummy udf and include the newly added udf in the VideoAnalytics configuration at the etcd Web UI. 

[Jacky_Gupta]

Hi @Murni_J_Intel  ,

In Continuation with previous communication, I have used new video and placed in VideoIngestion/test/ folder. Also i have replaced working PCB UDF with dummy UDF .I have  build the container as well.
I can see video output in EIS visualizer. But when i run discovery tool ,it exits after sometime. Although metadata is saved but no frames in saved path.

I have attached eis_config.json / Discovery_tool logs /EIS complete logs for your reference.

Note: EIS 2.3.3 was lately downloaded.

Regards
Jacky

[Murni_J_Intel]

Hi, 

Sorry for the delay, let me take a look on this issue and get back to you as soon as possible

[Murni_J_Intel]

Hi, 

can you help to provide the log for imagestore container? you can run below command to check it

$ docker logs -f <container_name>

can you also provide a screenshot for the directory /tmp/eis_history/data ?

[Jacky_Gupta]

Hi @Murni_J_Intel ,

I have attached screenshot and logs of image store as per your request.

[Murni_J_Intel]

Hi @Jacky_Gupta 

May I know if you already make the necessary changes in the docker-compose.yml ? Can you help to share the docker-compose.yml for inside the [WORK_DIR]/IEdgeInsights/build directory and at the discoveryhistory tools for me to look into details of this. 

[Jacky_Gupta]

Hi @Murni_J_Intel ,

I have made the required changes in docker-compose.yml as per READ.MD of Discovery tool.

It is also in prod mode .

I have attached docker-compose.yml from [work_dir]/IEdgeInsights/build directory and also the entire discovery tool directory for your reference.

If you need any other files/folder ,you can also watsapp me at +91-7795591292

[Murni_J_Intel]

Hi @Jacky_Gupta 

Previously you are trying to run in IPC mode, it is still the same now ?

[Jacky_Gupta]

Hello @Murni_J_Intel ,

It is in TCP mode of communication.

I have downloaded  EIS_2.3.3 and just replaced PCB UDF with dummy UDF and pcb demo video with 

any other  video.I can see the Video output in EIS app visualizer.All other configuration remains same.

Steps to reproduce the issue with discovery tool.

1.Download EIS_2.3.X(Working with PCB demo along with discovery tool)

2.Replace EIS_config.json with one provided in previous communication .

3.And finally replace PCB udf with dummy udf provided in EIS package.

Provision/build the Eis  and also build/up the discovery tool.

[Jacky_Gupta]

Hi @Murni_J_Intel 
I have debugged further and found it is entering "retrieve_image_frames" function of image_store_client.py at line 142 and from there it enters the while loop of main function at line 99 of discovery tool where it finds img_handle_queue to be empty and
finally it exits as can be seen from discovery logs which i have attached. we can also see image handle queue object at the logs.

Regards
Jacky

[WengWai_C_Intel]

@Jacky_Gupta 

If you are using own video file as input and change the udf to "dummy" so it will not run any analytics on the images, there will be no 'defect' information in the metadata. Based on the current implementation of Discovery Tool, if no 'defect' information it will not proceed to retrieve the images. This can be seen in the influxdbconnector_client.py file with the code below. The Discovery Tool is an reference tool to showcase how to use the interface of ImageStore and InfluxDBConnector. You can refer to it and apply to your own application.

valid_input = ['channels',
               'defects',
               'encoding_level',
               'encoding_type',
               'height',
               'width']
check = all(item in loaded_json['columns'] for item in valid_input)

thanks!