sparksbudWebXR Input Design
A VR menu needs an escape route as much as it needs a pointer. We learned that while building Sparksbud's headset dock: a button can work with a trigger and still become unreachable when the panel fades, a controller disconnects, or the visitor has only gaze input. The useful design unit is the complete path from appearing to selecting to getting the menu back.
Start with WebXR input sources, then design each menu action for a ray, a press and a recovery path. Sparksbud uses controller selection events for buttons, press and release for sliders, and a timed gaze selection when no controller source is connected. The dock can be brought back after hiding. Hand input and Vision Pro still need device-specific checks.
What WebXR actually gives you
WebXR input sources describe where an aim ray comes from and how a primary action is reported. A controller, a tracked hand and a gaze pointer may behave differently. A browser can add or remove sources during a session, so inputSources is a changing list rather than a permanent pair of hands. The selectstart, select and selectend events are useful for different moments in the same action.
Sparksbud asks for an immersive-vr session from the Enter VR button and requests local-floor and hand-tracking as optional features. A browser can decline an optional feature and still open the session. That is why the dock also accepts controller selection and gaze. The WebXR session request must stay attached to the visitor's button action; moving it behind an unrelated asynchronous setup step risks losing the required user activation.
| Situation | Aim and action | Recovery when the dock is hidden |
|---|---|---|
| Controller source connected | Point a target ray and press the trigger | Press once to reveal the dock, then choose an action |
| Tracked hand exposed as a controller source | Aim the ray and pinch to select | Pinch to reveal the dock |
| No controller source connected | Look at a dock target for 1.5 seconds | Look away, then back at the old dock position for 1.5 seconds |
The controller and gaze paths have code and Quest emulator checks. They do not cover every controller model or prove physical hand input, gaze comfort or Vision Pro behavior.
Those paths share actions, but they do not share a control signal. In Sparksbud, a trigger or pinch calls select. A slider starts on selectstart, tracks the same target while held and finishes on selectend. A quick click handler alone would make the slider feel broken. When a second resting controller is present, the dock keeps the active one from stealing hover until the user moves or selects again.
The recovery lesson
The dock sits in world space, just below the visitor's view. It fades after controller inactivity. We found that restoring it on any gaze hit caused an immediate reappearance after Hide: the visitor was still looking at the same place. The current gaze path requires a look away before a look back. That makes recovery a deliberate action without drawing a permanent floating instruction over the scene.
For controllers, a first press when the dock is hidden reveals it and does not also activate the button underneath. Once visible, the visitor can choose a control. A grip squeeze recenters a visible dock. These are separate states in the input code, and the tests check both the hidden and visible paths. A timed gaze also remembers the last button it fired so a layout repaint does not trigger the same choice twice.
Put the dock away while the ray is already over a button, then select once. The action should restore the dock without firing the hidden button.
Keep the pose and the menu alive
The world motion can stop while the headset pose keeps updating. Sparksbud's render loop still draws a surround world when motion is paused, which lets the visitor look around and use the dock. A pause that also stops head tracking would be a broken VR state. Its menu input and rendering use the current XR camera pose, and its own state reports whether the dock is visible, which input path is active and what is hovered.
When the visitor switches worlds in VR, the old world remains visible while the new one prepares. The loading cue appears in the dock, then the next scene replaces the old one after preparation. That choice avoids leaving the visitor in an untracked blank view during synchronous geometry work. Our world lifecycle note follows the loading path.
Test the transitions, not just the click
| Change | What to check | Failure it catches |
|---|---|---|
| Visible to hidden | Hide, fade and idle timeout | A menu that blocks the scene or never leaves |
| Hidden to visible | First press reveals without selecting | An accidental action behind the hidden panel |
| One source to another | Connect, disconnect, then select | A stale pointer or slider drag |
| Press to drag to release | Move along a slider before letting go | A value that jumps or never commits |
| World switch | Keep pose, cue and exit available | A frozen headset view during loading |
We run browser checks and a two-eye Quest emulator for these paths. Those checks prove the application state and stereo render in that environment. They cannot prove how a physical Quest pinch feels, whether a real hand is exposed the same way, or how Safari on Apple Vision Pro maps look-and-pinch. We have not tested Sparksbud on Vision Pro. The dock currently binds two Three.js controller slots; a transient pointer on another device is a reason to test the actual input-source sequence before promising support.
After swapping from controllers to hands, drag a slider as well as tapping a button. The drag can expose stale input state that a single select misses.
What we would change next
For a broader device target, we would record the input-source profile, target-ray mode and select events on the physical device, without logging hand poses or personal data. Then we would make the dock's pointer policy explicit for transient sources. The aim is to preserve the same user outcome: see a target, choose it once, and recover the menu. The Quest entry guide covers the visitor's steps; this note is about the implementation behind them.
Questions developers ask
Should gaze always be active beside controllers?
No. Sparksbud enables timed gaze selection when no controller source is connected. With a controller present, gaze could compete for hover and fire an unintended action.
Why use selectstart and selectend for a slider?
They mark the held interaction. Sparksbud starts the drag on selectstart, updates the value while the ray stays on that slider and clears the drag on selectend.
Does optional hand tracking guarantee hands will work?
No. An optional feature may be unavailable or declined, and devices expose input differently. Test the source and event sequence on the physical headset you support.
Why render while world motion is paused?
The visitor still needs head tracking, the dock and Exit VR. Sparksbud can stop scene travel while continuing the XR render loop and input updates.
Has this input design been verified on Vision Pro?
No. Its WebXR input model needs a physical Safari session test. The Quest emulator and code review do not establish Vision Pro behavior.
