FUJIFILM VisualSonics Accelerates AI for Ultrasound with Intel® Technologies

Authors:

Christoper A. White, Principal Developer, Software & Machine Learning, FUJIFILM VisualSonics, Inc.

Abhishek Khowala, AI Solutions Architect, Health and Life Sciences, Intel Corporation

Introduction

Artificial Intelligence (AI) has been making inroads into medical research at an accelerating pace. An excellent example is a joint project undertaken by FUJIFILM VisualSonics (VSI) and Intel. We teamed up to advance the use of AI-based measurement tools that are extremely valuable for pre-clinical researchers analyzing small animal models in studies of human disease. Taking advantage of micro-ultrasound technology, these tools free researchers from time-consuming and error-prone manual procedures. For more details, read our whitepaper. 

Like many AI tools, VSI’s AutoLV Analysis software is highly compute-intensive. It makes significant performance demands, because high performance of automated analysis is necessary to keep results moving at a pace that is practical and user-friendly. Our objective was to increase the ease of deployment and performance of the AutoLV Analysis software on Intel® Core™ processors by optimizing Deep Learning models using the Intel® Distribution of OpenVINO™ Toolkit. The results of our optimizations far exceeded our expectations:

• Intel Distribution of OpenVINO Toolkit optimizations resulted in a huge performance improvement of 41.4x.¹ We were able to reduce the average time per inference from 129.6 ms. to 3.1 ms.
• Improved inference performance, ease of deployment, and the extension of processing across Intel hardware using the toolkit gave us the freedom to develop more complex models without fear of overtaxing the hardware.

VisualSonics, a subsidiary of FUJIFILM Sonosite, designs and manufactures the world’s highest resolution ultrasound and photoacoustic instruments. Used in many areas of pre-clinical research, VSI products enable researchers to study live animals in real-time, longitudinally, while eliminating safety issues and side effects encountered with other imaging modalities. VSI designs and develops tools for pre-clinical research, whereas Sonosite provides point-of-care ultrasound systems and medical informatics to physicians and clinicians, with the goal of enabling them to improve procedure efficiency, time-to-diagnosis, and patient outcomes. Sonosite and VSI work closely together to enable technologies to migrate from pre-clinical research directly to important patient care solutions. 

Left Ventricle (LV) Analysis

Cardiovascular researchers represent the largest base of VSI’s pre-clinical customers. The FUJI-Intel team took a close look at Left Ventricle (LV) analysis, which plays a crucial role in research aimed at alleviating human diseases. The metrics revealed by LV analysis enable researchers to understand how experimental procedures are affecting the animals they are studying. LV analysis provides critical information on one of the key functional cardiac parameters, ejection fraction, which measures how well the heart is pumping out blood and is key to diagnosing and staging heart failure. LV analysis also calculates several other standard cardiac function parameters, such as fractional shortening, stroke volume, and cardiac output. A thorough understanding of these factors helps researchers to produce valid, valuable study results.

In addition, LV analysis plays a growing role in the clinical care of human patients. The ability to display critical cardiac parameters in real time enables medical care providers to make a diagnosis more quickly and accurately during ultrasound interventions, without needing to stop and take measurements manually or to send images to the radiology department.

AutoLV Analysis Software

Efficient, reproducible analysis of imaging data is critical to research goals. AutoLV Analysis software, VSI’s Artificial Intelligence-based measurement product, is a fast and accurate tool for analysis of cardiovascular imaging data. AutoLV Analysis brings AI to the functional analysis of the left ventricle in small laboratory animals with a “one-click” solution for both B-Mode and M-Mode research. Reliable, reproducible measurement data is the key to understanding model animal anatomy and physiology, and for completing studies, publishing work, and all other aspects of small animal pre-clinical research. AutoLV Analysis software makes functional and anatomical analysis of the left ventricle fast, highly reproducible, and free from operator error.

An M-Mode ultrasound—the M stands for “Motion”—examines a line of motion over time. When used in echocardiography, M-Mode displays the movement of the myocardium, enabling accurate real-time measurements of the thickness of the heart wall, internal diameter, and heart rate. These measurements enable the calculation of key heart parameters, including ejection fraction.

B-Mode—B is for “Brightness” —ultrasounds differ from M-Mode in that they show a single image at a given point in time: a two-dimensional ultrasound image composed of pixels representing ultrasound echo intensity. As in the case of M-Mode data, researchers can use B-Mode images to measure and quantify anatomical structure.

Both M-Mode and B-Mode are critical in assessing cardiac function, and both present problems for researchers. The challenge is that acquiring M-Mode or B-Mode measurements manually is laborious, time-consuming, and subject to human error, especially considering that both multiple systolic and diastolic points must be measured to provide data for cycle averaging.

Figure 1. M-Mode thickness measurements of the anterior wall, chamber, and posterior wall—sometimes referred to as the “string method.”

Intel Technology Makes It Possible

As an AI tool, AutoLV requires a platform that is powerful, flexible, and reliable. Both VSI and Sonosite depend on systems equipped with Intel® technology, including Intel Core i7 and i5 processors, as well as Intel Atom® processors. Running these systems enables VSI software developers to take advantage of the Intel Distribution of OpenVINO Toolkit. The toolkit, designed for Open Visual Inference and Neural Network Optimization (OpenVINO), makes it possible to harness the full potential of AI and computer vision. Based on Convolutional Neural Networks (CNN), the toolkit extends processing across Intel hardware (including accelerators) to maximize performance on demanding workloads, such as those AutoLV Software generates.

AutoLV would not be practical (or user-friendly) without the performance Intel technology delivers. Prior to the toolkit’s availability, we could run deep learning solutions only as a post-processing operation with inference times on the order of hundreds of milliseconds per frame on the system CPU. With the introduction of the OpenVINO feature set, we can now optimize models directly for the CPU and integrated GPU. This has resulted in over a 41.4x improvement in inference time processing, as detailed below.¹

To help generate these improvements, we also used the Intel® Integrated Processing Primitives (Intel® IPP) library, an extensive set of ready-to-use, domain-specific functions that are highly optimized for diverse Intel® architectures. Using Single Instruction Multiple Data (SIMD) instructions, the library helps improve the performance of compute-intensive workloads and accelerates processing speed while simplifying code development.

In addition, we leveraged OpenCV, a complimentary toolkit optimized for Intel architecture. Working in conjunction with Intel IPP, OpenCV improves the processing of real-time images and provides additional analytics and deep learning capabilities.

The implementation of these Intel hardware and software technologies made it not only possible but practical to run real-time imaging applications at frame rates exceeding 20fps.

Establishing Performance Benefits

To establish the viability of AutoLV as a practical application, we performed tests using systems equipped with Intel Core i7 processors—hardware similar to that found in typical laboratory environments. We used Deep Learning methodologies trained on research images generated at VSI.

We then looked at the performance of the M-Mode AutoLV algorithm, comparing Deep Learning inference optimization using models converted for TensorFlow and for the Intel Distribution of OpenVINO toolkit.

Performance Comparison

The TensorFlow model test resulted in an average time per run of 129.6 ms. The OpenVINO test, on the other hand, resulted in an average time per run of 3.1 ms. That means the OpenVINO model, running on an Intel Core i7 processor-based system, ran 41.4x times faster than the TensorFlow model. That’s a remarkable 4140% increase in performance.1

As the tests outlined above make clear, Intel architecture-based systems are delivering on the promise of AI today, with much more to come. We have no doubt that the success of the Intel Distribution of the OpenVINO Toolkit model running on Intel processor-based hardware will prove to be as relevant for the clinical side as it has been for researchers.

_{1. Performance results are based on testing as of May 2020 by FujiFilm, and may not reflect all publicly available security updates. No product can be absolutely secure. Test system configuration: Intel Core i7 processor 6700HQ, 2.6GHz , Dell Model 0XC72F-A00 Motherboard, 16 GB Dual Channel memory, integrated Intel HD Graphics 530, Microsoft Windows 10 Build 14393.}
 

_{Notices & Disclaimers}

_{Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors.}

_{Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more complete information visit www.intel.com/benchmarks.}

_{Intel’s compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice.}

_{Refer to http://software.intel.com/en-us/articles/optimization-notice for more information regarding performance and optimization choices in Intel software products.}

_{Intel does not control or audit third-party data. You should consult other sources to evaluate accuracy.}

_{© Intel Corporation. Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries. Other names and brands may be claimed as the property of others.}