The first time I tried to build a "simple" AR app, I thought it would take a weekend. Point the camera at a table, anchor a 3D model to it, ship it. The weekend turned into three weeks of tracking drift, lighting estimation bugs, and surface-detection edge cases that nobody warned me about. The model kept sliding two inches whenever the lighting changed. I learned more about computer vision in those three weeks than in the previous five years of web engineering.
That experience changed how I think about spatial computing. It isn't a UI paradigm you bolt onto an existing app — it's a completely different set of assumptions about where computation happens, how users interact with it, and what "correct" behaviour even means when your frame of reference is a moving room rather than a static window.
When most people hear "spatial computing," they picture someone wearing a clunky headset, stumbling around a living room with their arms outstretched. That image isn't wrong, but it captures maybe 5% of what spatial computing actually is. The bigger story is about an entirely new computing platform, one that replaces the flat screen with three-dimensional space. And it's already here, running on the phone in your pocket.
The Problem with Flat Screens
For roughly 40 years, every interaction with a computer has been mediated by a rectangle. Phones, laptops, monitors — they're glass rectangles that force three-dimensional ideas into two-dimensional space. This works surprisingly well for text, spreadsheets, and most productivity tasks. It starts to break down when you're trying to visualise a 3D model of a building, collaborate with a team across different locations on a physical task, train someone on a complex physical procedure, navigate a warehouse, or design a space that doesn't exist yet.
Spatial computing flips the relationship. Instead of pulling the world into a rectangle, it projects information into the world around you, or creates entirely new worlds you can inhabit. The rectangle becomes optional.
3D Space] -->|compress to 2D| B[Flat Screen
Rectangle] B --> C[User views
projection] end subgraph Spatial["Spatial Computing"] D[Real World
3D Space] -->|sense & map| E[Digital Twin
of Room] E -->|anchor content| F[Content lives
in real space] F --> G[User walks
through it] end style B fill:#1e293b,stroke:#f43f5e,color:#e2e8f0 style F fill:#1e293b,stroke:#4ade80,color:#e2e8f0
What Actually Makes Something "Spatial"
Spatial computing systems share four core capabilities that distinguish them from traditional screens. Spatial awareness means the device understands physical space — the shape of your room, where furniture is, how far away objects are. This is handled by a combination of cameras, depth sensors, LiDAR, and algorithms that build a real-time map of the environment. Positional tracking means the system tracks where you are and how you're moving, not just your head but your hands, fingers, and gaze. This is what makes interaction feel natural rather than requiring a controller or mouse. Anchored content means digital content can be pinned to specific physical locations and stay there — you attach a virtual label to a real machine in a factory, and anyone who walks past with the right device sees it. 3D rendering means instead of displaying flat pixels, the system renders three-dimensional objects that appear to occupy real space, with appropriate perspective, scale, and occlusion.
These capabilities run on a spectrum. An iPhone with LiDAR lets you measure a room — that's spatial computing at the low end. An Apple Vision Pro projecting a full virtual workspace into your living room sits at the high end. Most of the interesting industrial applications in 2026 sit somewhere in the middle, using purpose-built headsets with well-scoped tasks rather than consumer general-purpose devices.
The sensor stack that makes it work
The thing I didn't appreciate before building one was how much sensor fusion is happening under the hood. A single frame of stable AR requires the IMU (inertial measurement unit) to report device orientation at around 1000 Hz, the cameras to feed visual features at 30-60 fps, depth sensors to report distances, and the operating system to reconcile all of this into a coherent pose estimate — all within about 16 milliseconds to maintain visual synchronisation with the physical world. If any one of those loops drifts, the virtual content visibly slides. The reason my table-anchored model kept drifting was a well-documented issue with low-texture surfaces — the visual tracker had nothing distinctive to latch onto.
The Hardware Landscape in 2026
The most visible spatial computing devices right now fall into four categories. Mixed Reality Headsets like the Apple Vision Pro use passthrough cameras to show you the real world and overlay digital content on top. You see apps floating in physical space. You place a virtual monitor and it stays there. AR Smart Glasses like Meta Ray-Bans and Xreal devices are lighter-weight eyewear that adds spatial audio, a forward-facing camera, and sometimes a display overlay for simple information without the weight of a full headset. Mobile AR lives in your pocket — ARKit on iOS and ARCore on Android give apps the ability to understand physical surfaces, track movement, and anchor virtual objects to real locations. IKEA Place has been downloaded over 40 million times, and it's mobile AR, not a headset. Industrial Wearables like the RealWear Navigator and various purpose-built AR headsets are where spatial computing is generating real ROI right now — hands-free information access on factory floors, in server rooms, during field maintenance operations.
Meta Quest 3
~$500 to $3500] C --> C1[Meta Ray-Ban
Xreal One
Snap Spectacles
~$300 to $700] D --> D1[iPhone + ARKit
Android + ARCore
~$0, already owned] E --> E1[RealWear Navigator
HoloLens 2
~$2000 to $5000] style B1 fill:#1e293b,color:#e2e8f0 style C1 fill:#1e293b,color:#e2e8f0 style D1 fill:#1e293b,color:#e2e8f0 style E1 fill:#1e293b,color:#e2e8f0
Here's the important part most adoption coverage misses: industrial wearables are where the real money is. The Apple Vision Pro sold around 500,000 units in its first year — a respectable number but tiny compared to mobile. RealWear, Microsoft HoloLens in industrial contexts, and various purpose-built headsets have been shipping tens of thousands of units into field-service, oil and gas, and manufacturing workflows for years. The headsets aren't pretty, but they're paid for by a well-understood productivity delta.
Where Spatial Computing Actually Gets Deployed
Forget the flashy demos. Here's where spatial computing is earning its place in 2026. Manufacturing and maintenance technicians servicing complex equipment can see step-by-step instructions overlaid on the actual machine in front of them. Error rates drop significantly when you don't have to look away from the job to read a manual. Boeing reported wire-harness assembly error reductions of up to 90% after deploying AR work instructions. Architecture and real estate architects walk through buildings that don't exist yet; real estate agents let buyers tour properties that haven't been built. This isn't science fiction, it's the default workflow at many firms. Medical training surgeons practice procedures in spatial simulations before performing them on patients. The training fidelity is dramatically higher than flat video, and the American Academy of Orthopaedic Surgeons has published studies showing trainees who use AR sim complete procedures faster with fewer errors. Remote collaboration teams working on physical problems — a hardware prototype, a retail store layout, a facility design — collaborate in shared virtual spaces where everyone sees the same 3D model at the same time. Retail IKEA Place has been downloaded over 40 million times; customers who use AR features buy with more confidence and return products less often, with return rates dropping by up to 25% on AR-previewed items.
What broke in our first industrial pilot
When I advised on a factory-floor AR pilot (a wire-harness assembly task, similar to Boeing's use case), the thing that unexpectedly broke was lighting. The assembly area used overhead fluorescent lights that cycled at 60 Hz, which interfered with the camera's feature tracking. Our anchored instructions kept drifting by a few inches whenever someone walked past a light. The fix was non-obvious: switch the device's camera exposure mode to fixed rather than auto, and add more visual fiducial markers to the work surface. That debugging session taught me more about AR tracking than any tutorial. If you build for industrial environments, assume the lighting is hostile until proven otherwise.
For Developers: What This Changes
If you build software, spatial computing represents a genuine platform shift of the kind that happens maybe twice per decade. The UI paradigm is completely different from flat screens — there's no window manager, no z-axis ordering of flat panels. You design for 3D space. Content can be above, below, beside, and behind the user. Depth and scale become design elements. Input methods expand dramatically: gaze, hand gestures, voice commands, and physical controllers all become valid input channels. Designing for spatial interfaces means designing for how humans naturally interact with their environment, which is both more intuitive and harder to implement well.
Performance constraints are extreme. You need to render at 90+ fps per eye, with 6DOF (six degrees of freedom) tracking, while maintaining low latency so the real world doesn't drift relative to your virtual content. For context, that's roughly three times the frame rate of a typical game, split across two eyes, on battery-powered hardware. Spatial applications need aggressive optimisation from day one — there's no "we'll optimise later" path.
The development workflow
Here's what a minimal ARKit project looks like in Swift. This places a simple sphere 50 centimetres in front of the user's device when the app launches:
import ARKit
import RealityKit
class ViewController: UIViewController {
@IBOutlet var arView: ARView!
override func viewDidLoad() {
super.viewDidLoad()
let config = ARWorldTrackingConfiguration()
config.planeDetection = [.horizontal, .vertical]
arView.session.run(config)
// Create a sphere at 50cm in front of the camera
let sphere = MeshResource.generateSphere(radius: 0.05)
let material = SimpleMaterial(color: .orange, isMetallic: false)
let entity = ModelEntity(mesh: sphere, materials: [material])
let anchor = AnchorEntity(world: [0, 0, -0.5])
anchor.addChild(entity)
arView.scene.addAnchor(anchor)
}
}
That runs. When you point the camera around, the sphere stays anchored in world space. The first time you see it work on a real device, the "oh, this is different" reaction is immediate.
New frameworks are emerging quickly. Apple's visionOS (SwiftUI + RealityKit), Meta's Presence Platform, and open standards like OpenXR are becoming the development targets. If you've never written spatial code, expect a learning curve similar to moving from desktop to mobile in the early 2010s. Unity and Unreal Engine remain the dominant cross-platform options for anything game-like, while visionOS-native apps use SwiftUI with RealityKit for declarative 3D composition.
spatial apps"] --> Q1{Single platform
or cross-platform?} Q1 -->|iOS / visionOS only| Apple[ARKit + RealityKit
Swift / SwiftUI] Q1 -->|Cross-platform| Q2{Game-like or
business app?} Q2 -->|Game-like, 3D-heavy| Unity[Unity + AR Foundation
or Unreal Engine 5] Q2 -->|Business / productivity| OpenXR[OpenXR +
WebXR for web apps] Apple --> Ship[Ship to App Store
Vision Pro + iPhone] Unity --> Ship2[Ship to Meta Store
Vision Pro + Android] OpenXR --> Ship3[Works across
multiple platforms] style Start fill:#1e293b,stroke:#fb923c,color:#e2e8f0 style Ship fill:#16a34a,color:#fff style Ship2 fill:#16a34a,color:#fff style Ship3 fill:#16a34a,color:#fff
What the Docs Don't Tell You: Performance Realities
There's a gap between what Apple's and Meta's documentation describes and what actually happens when you ship a spatial app. I've collected the gotchas that cost me the most time.
The 90 fps rule is non-negotiable. visionOS throttles your app to 90 fps per eye and will drop frames aggressively the moment you miss the budget. On a Vision Pro, you have roughly 11 milliseconds to render one eye. That's an entire frame in a typical mobile game. You cannot spend it on a single expensive shader — you have to think in terms of GPU time budget from day one. A draw-call count of 200 is a reasonable ceiling for complex scenes; beyond that, expect compositor intervention.
Thermal throttling is a design constraint, not a runtime surprise. Vision Pro's M2 chip throttles clocks after roughly 20 minutes of sustained 90 fps rendering. Industrial headsets hit the same wall faster. Design the app to reduce visual complexity on a throttle signal — drop shadow quality, lower particle counts, disable secondary reflections — rather than letting the system drop frames.
Occlusion is where most apps break. Getting a virtual object to appear behind a real one (so you can't see part of the virtual desk lamp because a real person is standing between you and it) requires per-pixel depth comparisons every frame. It looks correct about 90% of the time. The other 10% is when users notice. There is no easy fix; you design around it, usually by keeping virtual content at a distance where occlusion artifacts are less visible.
The passthrough latency ceiling is 12 milliseconds. Beyond that, motion sickness hits predictably. This is the hardest constraint on mixed-reality headsets and the reason why adding "just one more step" to your rendering pipeline often isn't possible. If you're tempted to do an expensive AI inference pass on every frame, profile it against the 12 ms wall first.
Storage and distribution are weirder than mobile
Spatial app sizes explode quickly. A typical visionOS app bundles 3D models, textures, audio, and environment maps that easily push 1-2 GB. Vision Pro has enough storage that this isn't fatal, but the App Store's per-device download limits and over-the-air update sizes create real friction. Streaming-asset patterns (download models on demand from a CDN) become common, which means you're now also running a content pipeline.
Distribution for industrial wearables is another world entirely. RealWear and similar enterprise devices often don't go through a consumer App Store at all — you ship signed APKs via a device management platform (Workspace ONE, SOTI, Microsoft Intune). The deployment story is closer to embedded firmware than to app distribution. If you're coming from a mobile background, budget a month to learn the MDM tooling before your first industrial pilot goes live.
Accessibility can be the differentiator
Spatial computing's killer accessibility features are genuinely good. Vision Pro's eye-tracking lets users with limited hand mobility navigate entire apps with gaze alone. The same primitives that make immersive experiences possible also make completely hands-free interaction possible. If you're building for industries with workers in PPE, gloves, or restricted environments (field service, surgery, clean rooms), hands-free voice + gaze control is often the reason you win a contract over a traditional tablet app.
Test on the actual hardware, early and often
The single most common spatial-app failure mode is shipping something that works beautifully in the simulator and falls apart on device. visionOS Simulator does not render passthrough, does not enforce the 11 ms per-eye budget, and does not throttle. I have burned entire sprints on features that looked fine in simulator and were unshippable on device. Budget at least one Vision Pro (or equivalent target headset) per two developers on the team, and build your CI/CD pipeline to deploy test builds to those devices automatically. Anything else is theatre.
The Honest Assessment
Spatial computing will not replace your laptop next year. The headsets are still heavy, expensive, and socially awkward for extended use. Battery life is limited — two hours of continuous use is still the ceiling for most consumer headsets. The developer tooling is immature compared to mobile, and the App Store for visionOS has a fraction of the depth of the iOS App Store after more than a year in market.
But the trajectory is clear. Every generation of hardware has gotten lighter, more capable, and cheaper. The underlying sensor technology — LiDAR, computer vision, inertial measurement — is improving rapidly. For specific use cases in enterprise, spatial computing is already the best tool available, even if consumer adoption is slower than the hype cycle suggested.
The flat screen had a good 40-year run. Spatial computing is what comes next, slowly and then all at once. My rough timeline: consumer mixed-reality glasses (the kind you'd wear to a coffee shop without drawing stares) are probably five to seven years away at meaningful scale. Industrial spatial computing is here today and paying for itself. Mobile AR is already in everyone's pocket. If you work in a domain where 3D data matters — architecture, medicine, manufacturing, retail, training — spatial computing is already a fair part of the conversation.
What to Do Now
If you're a developer curious about this space, the fastest way to learn is to build something. Download RealityKit (iOS/visionOS) or AR Foundation (Unity, cross-platform) and pick up an iPhone. Build a simple AR app with ARKit — it takes an afternoon and you'll hit the gotchas fast enough to understand what the hard parts actually are. Follow Apple's WWDC sessions on spatial computing; they're free and unusually well-produced. Most importantly, look at your current application's core use cases and honestly ask: would this be better in 3D? For most apps, the answer is no. For a meaningful minority, it's a genuine competitive advantage.
Spatial computing isn't coming. It's here. The question is whether you'll be building it or playing catch-up when it matters.
Sources
- Apple — visionOS Developer Documentation (official platform reference for spatial computing development)
- Boeing — AR in Manufacturing Case Study (wire-harness assembly error rate reductions)
- IKEA — IKEA Place App (40 million+ download figure and usage impact)
- OpenXR Working Group — OpenXR Specification (cross-platform spatial computing standard)
- Apple — ARKit Framework Reference (iOS developer-level AR APIs)
This post is part of the AmtocSoft emerging-tech series. If you're building for spatial computing and hit a wall, drop a note at hello@amtocbot.com — I read every message.
About the Author
Toc Am
Founder of AmtocSoft. Writing practical deep-dives on AI engineering, cloud architecture, and developer tooling. Previously built backend systems at scale. Reviews every post published under this byline.
Published: 2026-04-17 · Updated: 2026-04-18 · Written with AI assistance, reviewed by Toc Am.
Get These In Your Inbox
Weekly deep-dives on AI engineering, no fluff. Join the newsletter →
Or grab the book ($39, ~100 pages) · Buy me a coffee
☕ Buy Me a Coffee · 🔔 YouTube · 💼 LinkedIn · 🐦 X/Twitter
No comments:
Post a Comment