Camera technology continues to evolve, providing new opportunities to capture more of the world surrounding us. Increasing resolution sizes, larger storage capacities, and newer fisheye and panoramic cameras enable us to capture more of the surroundings.
There are several healthcare use cases that I believe could benefit from the rich data coming from a camera equipped with fisheye or panoramic lenses including:
- Telesitting - utilizing cameras in a room to monitor patient and visitor activity. One Telesitter can monitor multiple rooms (normally up to around 16) and alert onsite staff to attend to a room when conditions warrant.
- Workplace violence - monitoring a room or a hallway for potential threats or escalations.
- Enforcing hand hygiene policies - ensure that people who enter or exit a patient's room follow the hospital hand hygiene policy.
- Artificial Intelligence - building models to detect and alert as conditions in the room change. AI inferencing can monitor for tugging on IVs, attempting to get out of bed, identifying people as they enter and leave the room, reduction of patient wandering, monitoring for patient turning or repositioning to prevent bed sores and alerting staff if the patient does not do this frequently enough, and so on.
While these newer cameras capture more data and can allow us to scale impact, improve patient outcomes, and protect employees, there are challenges in processing the information quickly enough. In addition, the output from fisheye and panoramic cameras can be challenging for a human to view and understand.
For instance, see the following image captured from a panoramic camera (i.e., 360 degree view) and saved using the equirectangular projection image format, which maps spherical points (3D) to planar points (2D). Without assistance (or training) it can be difficult to process mentally and fully understand such an image.
Software can help by taking the full image, extracting a field of view to present to the user, and allow the user to pan, tilt, roll, or zoom interactively to look around. Alternatively, virtual reality headsets present these images in a highly intuitive and interactive way.
As an example of this challenge of contextualizing, this image was taken with the camera positioned northward such that the road extends out to the right and left sides of the camera. Figure 1 shows the image in equirectangular format where it is apparent how difficult it is to comprehend that.
Yet, if a person selects a field of view that looks eastward, then the software can extract a portion of the image and present that view, which is far more intuitive to process and interpret the image, as seen in Figure 2.
Likewise, comprehending the southward field of view shown in Figure 3 becomes obvious after extraction and is nearly impossible in the full, equirectangular format in Figure 1.
I have prepared an interactive blog hosted on GitHub.io to share more about equirectangular images, how they are represented, and how a flattened field of view can be extracted and presented.
Please visit the blog, Introduction to 360 Degree Representations, for the full experience.
This is the first from a series of blogs that covers the basics of equirectangular representations, open-source solutions that extract a selected field of view, and techniques and tools to optimize the extraction process.
The introduction also showcases the same image as this blog and provides an interactive viewing solution to move the field of view around and see different perspectives from the single frame. Try it out!
We would like to hear from you! Let us know in the comments – which use cases in health and life sciences do you think panoramic and fisheye lenses will have the greatest impact?
If you enjoyed hearing from the Health and Life Sciences team and want to hear more, give this post a like and ensure you subscribe to get the latest updates from the team.
About the Intel Health and Life Sciences
Reach out to Intel's Health and Life Sciences team at health.lifesciences@intel.com or learn more about what we do at https://www.intel.com/health.
About the Author
Doug Bogia received his Ph.D. in computer science from the University of Illinois, Urbana-Champaign, and works at Intel Corporation. He enjoys photography, woodworking, programming, and optimizing solutions to run as fast as possible on a given piece of hardware. Connect with him on LinkedIn https://www.linkedin.com/in/doug-bogia-4653696/ by mentioning this blog.