What is Stereo Vision?

Stereo Vision is a computer vision technology that uses two cameras to capture images and estimate depth information in 3D space. To achieve this, two cameras photograph the same object from different positions, and the captured image pair is used to calculate depth information. This involves calculating the disparity between the two images and using it to estimate depth. This technology is similar to the principle of humans using two eyes to estimate depth information. It allows for the identification of object positions and distances in a 3D environment and is used in various fields such as robotics, autonomous vehicles, video games, and image processing. It provides more accurate and practical results compared to conventional computer vision technologies, playing a crucial role in various applications.

Special Features of RTLS using Stereo Vision

Real-Time Location System (RTLS) using Stereo Vision is a system that analyzes visual information collected through two cameras to track and monitor the location of objects or people in real-time. Stereo Vision enables highly accurate location inference, ensuring high precision and reliability. Furthermore, because location information can be updated in real-time, it provides critical data for logistics, manufacturing, and construction. Additionally, Stereo Vision-based RTLS does not use tags, eliminating the need to purchase and maintain additional tags or signal-generating equipment for targets. This reduces system configuration and maintenance costs, making it cost-efficient. Furthermore, the high accuracy and reliability of target tracking combine cost-effectiveness with superior system performance. Therefore, Stereo Vision-based RTLS is a highly useful technology that simultaneously achieves cost reduction and performance enhancement.

How does positioning using Stereo Vision work?

Core Technologies of ORBRO's Stereo Vision-based RTLS

Original Image Streaming

To increase location tracking accuracy in RTLS, high-level Depth Map inference technology is essential. While humans can quickly and accurately perceive distance with rough outlines through experience and intuition, Depth Map technology focuses on image details, directly linking precise inference to location tracking. Therefore, we do not use lossy compressed video streaming methods typically used in general CCTV, but instead utilize lossless video to minimize the loss of low-level features. In this way, Stereo Vision-based RTLS minimizes the loss of location information.

By utilizing lossless original images, the surfaces of objects are represented more continuously in the Depth Map.

Inference Background

Original

Encoding

Advanced Depth Estimation

Utilizing stereo camera images for depth inference is of great interest in the field of computer vision research. In particular, the application of deep learning methodologies over the past decade has shown superior results compared to previous classical algorithms. However, classical algorithms are still used in many fields because they guarantee adequate accuracy and high real-time performance. This typically means processing is possible at frame rates of 15 FPS or higher.

Competitor Depth Map using Computer Vision

Depth inference results using classical computer vision algorithms faced issues where distortion occurred in detailed parts and object outlines. For example, parts like the top of a hand holding a mouse or a person's head obscured by a partition were not inferred. However, the deep learning-based model we developed has overcome these issues. This model is optimized for use in 'RTLS using Stereo Vision' and ensures real-time performance (>15 FPS) through lightweighting and fine-tuning. While the processing speed may slow down to approximately 4 FPS when processing inputs from multiple cameras, it still provides satisfactory location information considering the movement speed of people on screen.

ORBRO Depth Map using Deep Learning

Depth Estimation Finetuning Technology

Images taken under natural light versus artificial lighting have different characteristics, which can manifest even when lighting hits only one lens of a stereo camera. In this case, inconsistent characteristics between the left and right images occur due to differences in exposure values of the left/right image sensors.

Differences in characteristics between left/right stereo images caused by direct lighting exposure

In addition to these external environmental factors, internal factors that may occur during the stereo camera manufacturing process must also be considered. These include vertical alignment errors or rotation errors that may occur when mounting image sensors on a board, and magnification differences due to changes in the distance between the lens and the image sensor.

(a)

(b)

(c)

(d)

Distortion patterns by wide-angle lenses and individual deviations between Camera A and B (Green: 2.7m, Purple: 6.3m, Red: 8.8m)

To solve these problems, the process of accurately modeling distortion patterns and correcting them to match actual depth is a critical part of depth inference technology. This correction process must account for individual camera uncertainties, such as deviations occurring during lens production or assembly. Without applying a distortion correction model, deviations of 30% or more are typically observed in depth values inferred from wide-angle stereo cameras. However, applying a distortion correction model not only eliminates these errors but also achieves an average error rate reduction of about 50% in depth inference. Therefore, we perform calibration using our sophisticated, internally developed distortion correction model before every stereo camera is shipped. This ensures superior depth inference technology and provides customers with the highest performance and reliability.

Key Advantages of Stereo Vision