Intel® Edge and AWS Cloud collaboration on AI inference

Introduction

2020 has been a year of transformation. The global pandemic fundamentally changed the way people interact with each other. As COVID spreads, reinforcing social distancing becomes so important for us to create a safe environment. With AI and Computer Vision on the edge, the Intel and AWS team introduced a social distancing reference implementation for developers to scale the technology in your local community, with a one-click installation and moderate customizations. In this blog, we will introduce the benefits of edge-to-cloud architecture, the sample model powered by Intel and AWS offerings and many more use cases on the Intel® Edge Software Hub to advance human life.  

Intel Edge and AWS Cloud Collaboration

As COVID-19 broke out, numerous medical professionals around the world stated that social distancing is one of the most effective non-pharmaceutical ways to prevent the spread of the disease.

In order to adapt to current times, Intel has brought a powerful reference implementation that can help curb the spread of the disease by using computer vision inference through the Intel® Distribution of OpenVINO™ Toolkit in order to measure distance between people and store the data into an InfluxDB. This data can then be visualized on a Grafana Dashboard.

This blog serves as tutorial to navigate through the installation, deployment, and customization of this reference implementation.

1. Follow the documentation to install the social distancing reference implementation by following instructions in link below: 
   https://software.intel.com/content/www/us/en/develop/articles/multi-camera-monitoring-reference-implementation.html
   
   
2. Install AWS IoT python SDK on the machine where RI is installed by following instructions in link below:
   https://docs.aws.amazon.com/greengrass/latest/developerguide/IoT-SDK.html
   
   
3. Configure AWS component in cloud & download the certificates by following instructions in link below:
   https://docs.aws.amazon.com/greengrass/latest/developerguide/device-group.html
   
   
4. Modify code in “main.py” to connect and send data to AWS cloud.
   a. Add import statements.

 

               i. from AWSIoTPythonSDK.MQTTLib import AWSIoTMQTTClient

               ii. import json

            b. Add a code segment for additional command line arguments to integrate with AWS IoT.

                i.     # AWS MQTT client parameter arguments

                ii.     parser.add_argument("-e", "--endpoint", action="store", required=True, dest="host", help="Your AWS IoT custom endpoint")

                iii.     parser.add_argument("-r", "--rootCA", action="store", required=True, dest="rootCAPath", help="Root CA file path")

                iv.     parser.add_argument("-c", "--cert", action="store", dest="certificatePath", help="Certificate file path")

                v.     parser.add_argument("-k", "--key", action="store", dest="privateKeyPath", help="Private key file path")

                vi.     parser.add_argument("-id", "--clientId", action="store", dest="clientId", default="basicPubSub",help="Targeted client id")

                vii.     parser.add_argument("-t", "--topic", action="store", dest="topic", default="sdk/test/Python", help="Targeted topic")

            c. Set local variables with argument variables.

                i.      #AWS MQTT parameters

                ii.      host = args.host

                iii.     rootCAPath = args.rootCAPath

                iv.     certificatePath = args.certificatePath

                v.      privateKeyPath = args.privateKeyPath

                vi.     port = 8883

                vii.     clientId = args.clientId

                viii.    topic = args.topic

            d. Add code segment to initialize MQTT Client and set connection configuration.

                i.     #Setup AWS MQTT Client

                ii.     myAWSIoTMQTTClient = AWSIoTMQTTClient(clientId)

                iii.     myAWSIoTMQTTClient.configureEndpoint(host, port)

                iv.     myAWSIoTMQTTClient.configureCredentials(rootCAPath, privateKeyPath, certificatePath)

                v.     # AWSIoTMQTTClient connection configuration

                vi.     myAWSIoTMQTTClient.configureAutoReconnectBackoffTime(1, 32, 20)

                vii.     myAWSIoTMQTTClient.configureOfflinePublishQueueing(-1)  # Infinite offline Publish queueing

                viii. myAWSIoTMQTTClient.configureDrainingFrequency(2)  # Draining: 2 Hz

                ix. myAWSIoTMQTTClient.configureDrainingFrequency(2)  # Draining: 2 Hz

                x. myAWSIoTMQTTClient.configureConnectDisconnectTimeout(10)  # 10 sec

                xi. myAWSIoTMQTTClient.configureMQTTOperationTimeout(5)  # 5 s

                xii. # Connect  to AWS IoT

                xiii. myAWSIoTMQTTClient.connect()

            e. Add Topic and AWS MQTT client object to context class.

                i. #Adding topic and AWS MQTT client object to context so it can be shared across the code

                ii. context = Context(manager, worker, db, models, num_reqs, args.input_queue_size - 1, ch_min_dist, show_period, args.no_show, grid_sizes, resolution,topic, myAWSIoTMQTTClient)

            f.     Add variables to the Context class’s init function.

                i. def __init__(self, manager, worker, db, models, num_reqs, last_frame_id, min_distances, show_period, no_show, grid_sizes, display_resolution,topic,myAWSIoTMQTTClient):

            g. Initialize local variables with the new values passed to the init function of context class

              

                i. self.topic = topic

                ii. self.myAWSIoTMQTTClient = myAWSIoTMQTTClient

            h. Modify the update_social_violations function to send data to AWS IoT

                i. Add Timestamp in specific format to send it to AWS TimeStream database

                    1.      timestamp = datetime.datetime.utcnow().strftime("%Y-%m-%d %H:%M:%S.%f")[:-3]

                 ii. Add code section to create MQTT message and publish to AWS IoT

                    1. #This section will Publish  people count and violations to AWS IoT main topic

                    2. #creating the message

                    3. message_indivChannel = {}

                    4. message_indivChannel['channel_id'] = str(ch_id)

                    5. message_indivChannel['people_count'] = people_count

                    6. message_indivChannel['social_distancing_violation'] = viol_count

                    7.  message_indivChannel['timestamp'] = timestamp

                    8. #converting to JSON format

                    9. message_indivChannel_json = json.dumps(message_indivChannel)

                    10 #calling MQTT Client publish message

                    11. self.myAWSIoTMQTTClient.publish(self.topic,message_indivChannel_json, 1)

          5. Configuring AWS IoT to store data into Timestream DB.

              a. Add a new rule.

              b. Add rule query statement.

              c. Add action, select Timestream table.

              d. Configure Timestream action

              e. Create a new database 

              f. Create a new table.

              g. Set dimensions to channel_ id
                  Note that dimension cannot be “integer”

              h. Set the timestamp by parsing the data in the MQTT payload

                    1. Value - ${time_to_epoch(timestamp, "yyyy-MM-dd HH:mm:ss.SSS")}

                    2. Unit - MILLISECONDS

              i. Create the role.

          6. Setup Grafana

              a. Upgrade Grafana on the host machine to the latest machine.

              b. Add AWS timestream plugin.

              c. Configure AWS plugin with your credentials.

              d. Configure your dashboard.

          7. Update the run.sh file 

python3 main.py --person_detector "$PERSON_DETECTOR" -d1 $DEVICE1 \

 -m1_height $MODEL1_INPUT_HEIGHT -m1_width $MODEL1_INPUT_WIDTH \
--width $WIDTH --height $HEIGHT  -n_s $NUM_SOURCES -n_c $NUM_CHANNELS \
-n_th $NUM_THREADS -i_q $INPUT_QUEUE_SIZE -i "$INPUT1" "$INPUT2"  \
-min_social_distances $MIN_SOCIAL_DIST1 $MIN_SOCIAL_DIST1   -decode_device $DECODE_DEVICE \
-e "anee81iss8x57-ats.iot.us-west-2.amazonaws.com" -r "/home/vibhu/VibhuSocialDistancingData/AmazonRootCA1.pem" \
-c "/home/vibhu/VibhuSocialDistancingData/ac597af7e1-certificate.pem.crt" -k "/home/vibhu/VibhuSocialDistancingData/ac597af7e1-private.pem.key" \
-id "ieitank1" -t "esh/socialDistancing"

          8. Run the application with sample videos.

              Any social distancing violation will be marked in the videos and users can monitor the performance from the dashboard. 

More Use Cases and Software Offerings

Developers are eager to create customized AI solutions to solve a real-world problem. When problem gets identified, it becomes critical to accelerate time to market, reduce development cost and scale with robust ecosystem. Intel makes it all possible by offering developers deployment-ready, reusable containerized SW packages, and Use Cases on the Intel® Edge Software Hub. Developers can find reference implementations, including tutorial, sample code, and documents for tons of edge-to-cloud AI applications. 

Additional Resources

Follow the documentation to install the social distancing reference implementation by following instructions in link below:

https://software.intel.com/content/www/us/en/develop/articles/multi-camera-monitoring-reference-implementation.html

Install AWS IoT python SDK on the machine where RI is installed by following instructions in link below:

https://docs.aws.amazon.com/greengrass/latest/developerguide/IoT-SDK.html

Configure AWS component in cloud & download the certificates by following instructions in link below:

https://docs.aws.amazon.com/greengrass/latest/developerguide/device-group.html

Modify code in “main.py” to connect and send data to AWS cloud.

 

Notices and Disclaimers

Intel technologies may require enabled hardware, software or service activation.
No product or component can be absolutely secure.
Your costs and results may vary.
© 2020 Intel Corporation. Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries. Other names and brands may be claimed as the property of others.

Intel and the Intel logo are the trademarks of Intel Corporation or its subsidiaries