XR & UI Input Devices

XR & UI Input Devices

Unity supports specialized input interfaces like head-mounted VR/AR devices and character input engines for internationalization.

XR Input Actions

In virtual reality and mixed reality, input is tracked through spatial hardware sensors:

• Tracked Pose Driver: Attached to cameras or virtual hands, it directly maps the physical rotation and position of an XR headset or hand controller to a GameObject's Transform.
• XR Interaction Toolkit Action Map: Binds actions specifically to XR device layouts (e.g. Meta Quest controllers, HTC Vive controllers, or Apple visionOS hand tracking gestures).

IME Input Support

Input Method Editor (IME) allows users to type complex characters (such as Chinese, Japanese, or Korean) using standard keyboards:

• In Unity, you can check if a user is currently composition-editing characters by checking Input.imeIsSelected and managing the composition string position with Input.compositionString.

UI Event System Raycasters

The Unity Event System uses Raycasters to detect inputs on screen widgets:

• Physics Raycaster: Detects hits on 3D objects with Colliders.
• Physics 2D Raycaster: Detects hits on 2D colliders.
• Graphic Raycaster: Detects clicks or touches on Canvas UI elements.

TL;DR

• Tracked Pose Driver maps XR controller coordinates directly to GameObject Transforms.
• IME support allows text input for complex languages like Japanese or Korean.
• UI Event System uses Raycasters to decide which widget receives click/touch events.