What is XR?
XR (extended reality) is an umbrella term for the spectrum of technologies that blend digital content with the physical world or replace it entirely, spanning virtual reality (VR), augmented reality (AR), and mixed reality (MR). The letter X stands in as a variable that can be substituted with V, A, or M, and the term is used when a discussion or product applies across the whole spectrum rather than to a single modality.
The category exists because the underlying tools that build these experiences (real-time engines, 3D asset pipelines, spatial input, head-tracked rendering, streaming architectures) are shared across VR, AR, and MR. A developer authoring a spatial application typically targets XR as a class rather than committing to one specific hardware endpoint, especially with cross-platform runtimes such as OpenXR now standardizing the API surface.
XR is often described by the Milgram-Kishino reality-virtuality continuum, which places pure physical reality at one end and fully synthesized virtual reality at the other. AR and MR occupy the span between these points, with the practical distinction between them being how deeply the digital content interacts with the physical environment (occlusion, physics, semantic understanding).
The XR modality spectrum
Each point on the continuum has distinct hardware, input, and content characteristics.
Virtual reality (VR). The user is visually isolated from the physical world by an opaque head-mounted display. All content is rendered by the runtime. Typical hardware: Meta Quest, Valve Index, PlayStation VR2.
Augmented reality (AR). Digital content overlays the physical world, usually through a phone or tablet camera or through transparent optics on glasses. The physical environment is the dominant visual layer. Typical hardware: ARKit-capable iPhones, ARCore-capable Android devices, Snap Spectacles.
Mixed reality (MR). Digital content sits inside the physical environment with occlusion, physics, and often semantic understanding of the scene. Passthrough MR headsets combine outward-facing cameras with VR-style displays to composite the physical world into the render. Typical hardware: Apple Vision Pro, Meta Quest 3, Microsoft HoloLens 2.
Passthrough boundary. Modern devices increasingly blur the AR/MR/VR distinction by allowing users to switch between modes on the same headset. A single session may begin in VR, transition to passthrough MR, and momentarily surface AR content, all through the same runtime.
How XR works underneath
Regardless of modality, an XR experience shares a common set of underlying systems.
Spatial tracking. Six-degree-of-freedom (6DOF) tracking of the head, hands, or controllers, delivered by outside-in cameras, inside-out cameras, IMU sensors, or a combination.
Stereo rendering. Each eye receives a separately rendered view at the display's refresh rate, typically 72 to 120 frames per second, with the runtime handling reprojection to hide dropped frames.
Input. Controllers, hand tracking, eye tracking, and voice inputs are exposed through the runtime API. OpenXR standardizes these paths so an application can support multiple input modalities with the same code.
Compositor. The runtime composites application-rendered frames, system UI, and (for MR) the physical-world passthrough feed into the final display image.
XR and 3D content delivery
XR imposes some of the strictest performance constraints in interactive computing. A dropped frame in a headset can cause visible judder or motion discomfort, and the required frame rate is roughly 2x that of a flat-screen game running at the same fidelity. Content delivery matters accordingly: assets must load quickly enough to keep the frame loop steady, geometry and textures must fit within GPU memory budgets that are tighter than desktop equivalents, and streaming architectures increasingly play a role in serving high-fidelity 3D content into headsets without a large pre-download step.
See also
Augmented reality (AR) — The AR endpoint of the XR spectrum.
WebXR — W3C standard that brings XR experiences to the browser.
Spatial computing — Adjacent term emphasizing the compute paradigm rather than the display modality.
3D streaming — Delivery pattern used to fit high-fidelity 3D into XR frame budgets.
Additional resources
