One of the biggest problems developers experienced when creating mixed reality content for the Quest was the lack of access to the headset's cameras. This is about to change.
At Connect this week, VR/AR Meta VP Mark Rabkin announced that developers will soon have access to a new camera Switching API, scheduled to arrive “early next year.”
Set to include tracked objects, AI applications, “fancy” layers, scene understanding and “much more,” Rabkin said this version is “expected to enable all kinds of cutting-edge MRI experiences.”
Testing is said to be happening “very soon,” which could mean that certain developers may choose to use the beta version of the Passthrough API in their games.
As it stands today, developers rely on things like geometric representations of the environment, body and hand tracking data, and user-defined furniture placement rather than raw camera access.
Providing greater access to cameras will allow developers greater detail in assimilating the user's individual space; This is something that's been missing so far, with studios being forced to turn to “low-hanging fruit” like moving from optional MR modes to VR games. on the wow factor of seeing your own room. There are several great examples of studios working within these limitations to create intelligent room-scale MR games, such as Creature's recently released mixed reality game. Starship Home (2024).
This also means Meta has to tighten up on camera privacy concerns; access was always the company's responsibility. Meta has not yet disclosed specific limitations to the Passthrough API or whether it includes raw access; Therefore, it is not yet known what level of anonymity the additional camera data will provide.
Additionally, the company announced in a blog post this week that Horizon OS with the launch of the Quest 3S on October 15 will include “simpler, more transparent, and easier to use Privacy Settings” such as activity status, privacy indicators when apps are run, and more. announced that it will update it. The Quest's microphone uses location and spatial data, as well as a new sensor lock feature that cuts power to your headset's external cameras and microphones in “certain situations,” such as when your headset goes to sleep.