360-Degree Imaging Technology for Next-Generation XR

A 360-degree wearable camera and recognition AI that can inform users about their surroundings 

In situations where extended reality (XR) is used to a high degree, such as daily living assistance, rehabilitation of the disabled, immersive educational and training environments, and in professional and consumer telepresence applications, XR systems must respond according to the user’s expectations and present information in a way that reflects the user’s perceptions. 

However, the use of common cameras for XR creates problems such as missing nearby events or not being able to continuously track objects due to the limited field of view. This makes it difficult to gain a more comprehensive understanding of the wearer’s surroundings. 

Ricoh has developed the world’s first fully egocentric*1 360-degree wearable camera (as of June 2025, based on Ricoh research). Although there are currently some 360-degree cameras available that are worn around the neck or on helmets, this is the world’s first 360-degree wearable camera that is worn close to the wearer’s eyes. It allows for 360-degree recognition of surroundings. Equipped with a microphone and speaker, it also allows for voice interaction with LLMs*2. The proprietary AI can also warn of approaching objects in the vicinity and detect objects based on arbitrary text. For example, the wearer could ask for “people wearing helmets.” 

*1 egocentric: A view in which the user’s point of view and the virtual camera’s point of view coincide. The user can feel as if they are inside that world. 

*2 LLMs: Large language models. A technology that can perform processing such as answering questions in natural text or summarizing documents with human-like accuracy, and that can be easily trained. It can process ambiguities and fluctuations that exist in human spoken and written language (natural language) by determining the relationships between words that are far apart in a sentence and taking the context into account. 

Figure 1. Initial design of the first-person 360-degree wearable camera 

First-person 360-degree wearable camera 

Ricoh has developed a 360-degree camera that can be worn over the ears (Figure 2). This camera allows for a more immersive experience and is capable of recognition from the point of view of the user. 

Overview of first-person 360-degree wearable camera 

・ Lightweight (can be worn over the ears) 

・ Four cameras capture images 360 degrees around the wearer 

・ Inertial measurement units 

・ Microphone and speaker functionality 

・ Resolution: 3840 × 1920 (pixels) 

Figure 2. First-person 360-degree wearable camera  

Figure 3. Image capture sample (first-person 360-degree wearable camera) 

As shown in Figure 4, the device is controlled by a small single-board PC, making the entire system portable. The device can also communicate in real time with a server, allowing it to connect to LLMs and other state-of-the-art AI models and interact with them in real time via voice. 

Figure 4. Overall system diagram 

360-degree AI 

Open-vocabulary object detection 

We have developed a real-time open-vocabulary object detection method for 360-degree video. The user can specify what to detect by freely entering text on the PC, and this exciting function holds great promise in a variety of fields. For example, it could be used to manage safety in a factory, such as entering text to detect “people wearing safety vests.” 

Figure 5. Open-vocabulary object detection  
 

Collision detection 

We have developed an AI that detects approaching humans in a 360-degree video. “RICOH THETA,” a 360-degree camera, is installed on a forklift and notifies the driver of people approaching the forklift to help prevent unexpected accidents such as colliding with people in the driver’s blind spots. 

Figure 6. Collision detection