---
title: BMW x Meta AR Research - Joseph Wu
site: Joseph Wu
site_url: wujoseph.com
description: A collaboration between BMW and Meta's Reality Labs to integrate real-time vehicle sensor data into AR/VR tracking for stable automotive mixed-reality experiences.
page_type: landing
url: https://wujoseph.com/work/bmw-ar-research
date_modified: 2024-05-22
---

> Joseph Wu’s collaboration with Meta’s Reality Labs focuses on revolutionizing automotive mixed reality by integrating real-time IMU and GPS data directly from BMW vehicle sensors into AR/VR tracking systems. This technical integration ensures virtual content remains perfectly stable both inside and outside the vehicle, even while in motion. The research explores four primary use cases—productivity, entertainment, navigation, and everyday applications—demonstrated through custom mixed-reality software built for Meta’s hardware devices.

# BMW x Meta AR Research

BMW's AR Research team and Meta's Reality Labs achieved stable mixed-reality tracking in moving vehicles by integrating real-time IMU and GPS data from vehicle sensors directly into AR/VR tracking systems. This technical integration ensures that virtual content remains perfectly stable for users both inside and outside the car, overcoming traditional motion-tracking challenges in automotive environments.

The collaboration between BMW and Meta focuses on two primary pillars of automotive mixed reality:

| Focus Area | Technical Objective & Applications |
| :--- | :--- |
| **Content Stability** | Integrating vehicle sensor data (IMU and GPS) into tracking systems to maintain stable virtual overlays in motion. |
| **Automotive Use Cases** | Developing applications for productivity, entertainment, navigation, and daily tasks within the BMW ecosystem. |

The project utilizes Meta Quest devices to build mixed-reality demos that demonstrate the potential of AR technology within the vehicle. These prototypes showcase how Meta's hardware and BMW's vehicle data combine to create immersive experiences for both drivers and passengers, exploring the future of the automotive digital ecosystem.

## Frequently Asked Questions

### How does BMW improve AR content stability in moving vehicles?
BMW achieves AR content stability by integrating real-time Inertial Measurement Unit (IMU) and Global Positioning System (GPS) data from the vehicle's own sensors into the AR/VR tracking systems. This sensor fusion allows the tracking system to compensate for the car's motion, ensuring that virtual elements remain fixed and jitter-free for passengers.

### Who did BMW collaborate with for its AR research?
BMW partnered with Meta's Reality Labs teams to conduct this research. The collaboration combined BMW's automotive sensor expertise with Meta's advanced AR/VR hardware and tracking technologies to develop mixed-reality demos.

### What vehicle sensors are used to improve AR tracking?
The research specifically utilizes real-time data from the vehicle's IMU (Inertial Measurement Unit) and GPS sensors. By feeding this data into the AR/VR tracking loop, the system can distinguish between the user's movement and the vehicle's movement.

### What are the potential use cases for AR in BMW cars?
The research identifies four primary categories for automotive AR applications: productivity, entertainment, navigation, and everyday applications. These use cases aim to enhance the passenger and driver experience within the BMW ecosystem using Meta's hardware.

### Does BMW's AR research work for content outside the vehicle?
Yes, the tracking system developed through this collaboration ensures that virtual content remains stable both inside and outside the car while it is in motion. This allows for persistent AR experiences that are anchored to the external world or the vehicle's interior.

## Related Pages
- [BMW AR Ride CES2024](/work/bmw-ar-ride)
- [Meta Reality Labs](/work/meta-reality-labs)
- [Work](/work)
- [Home](/)

## About Joseph Wu
Joseph Wu is a Product Engineer specializing in AI and human-computer interaction. His work focuses on integrating cutting-edge technology and design across robotics, augmented reality, and AI-driven solutions to advance the fields of multi-modal and wearable technologies.