**Name String**

`XR_ANDROID_raycast`

**Extension Type**

Instance extension

**Registered Extension Number**

464

**Revision**

1

**Extension and Version Dependencies**

[`XR_ANDROID_trackables`](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_trackables)

**Last Modified Date**

2024-10-02

**IP Status**

No known IP claims.

**Contributors**

Spencer Quin, Google

Nihav Jain, Google

John Pursey, Google

Jared Finder, Google

Levana Chen, Google

Kenny Vercaemer, Google

## Overview

This extension allows the application to perform raycasts against trackables in the environment. Raycasts can be useful for detecting objects in the environment that a ray would intersect with. For example:

- To determine where a floating object would fall when dropped, by using a vertical raycast.
- To determine where a user is looking, by using a forward-facing raycast.

| **Caution:**   
| **Permissions**   
|
| Android applications**must** have the`android.permission.SCENE_UNDERSTANDING_COARSE`permission listed in their manifest as this extension depends on[`XR_ANDROID_trackables`](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_trackables)and exposes the geometry of the environment. The`android.permission.SCENE_UNDERSTANDING_COARSE`permission is considered a dangerous permission. (protection level: dangerous)

## Query supported raycast capabilities

The[xrEnumerateRaycastSupportedTrackableTypesANDROID](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_raycast#xrEnumerateRaycastSupportedTrackableTypesANDROID)function is defined as:  

    XrResult xrEnumerateRaycastSupportedTrackableTypesANDROID(
        XrInstance                                  instance,
        XrSystemId                                  systemId,
        uint32_t                                    trackableTypeCapacityInput,
        uint32_t*                                   trackableTypeCountOutput,
        XrTrackableTypeANDROID*                     trackableTypes);

### Parameter Descriptions

- `instance`is the[XrInstance](https://registry.khronos.org/OpenXR/specs/1.1/html/xrspec.html#XrInstance)from which`systemId`was retrieved.
- `systemId`is the[`XrSystemId`](https://registry.khronos.org/OpenXR/specs/1.1/html/xrspec.html#XrSystemId)whose supported trackable types for raycasting are being enumerated.
- `trackableTypeCapacityInput`is the capacity of the`trackableTypes`, or 0 to retrieve the required capacity.
- `trackableTypeCountOutput`is a pointer to the count of the array, or a pointer to the required capacity in the case that`trackableTypeCapacityInput`is insufficient.
  - `trackableTypes`is a pointer to an array of[XrTrackableTypeANDROID](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_trackables#XrTrackableTypeANDROID), but**can** be`NULL`if`trackableTypeCapacityInput`is`0`.
- See the[Buffer Size Parameters](https://registry.khronos.org/OpenXR/specs/1.1/html/xrspec.html#fundamentals-buffer-size-parameters)section for a detailed description of retrieving the required`trackableTypes`size.

[xrEnumerateRaycastSupportedTrackableTypesANDROID](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_raycast#xrEnumerateRaycastSupportedTrackableTypesANDROID)enumerates the trackable types that support raycasting by the current session.

### Valid Usage (Implicit)

- The[`XR_ANDROID_raycast`](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_raycast#XR_ANDROID_raycast)extension**must** be enabled prior to calling[xrEnumerateRaycastSupportedTrackableTypesANDROID](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_raycast#xrEnumerateRaycastSupportedTrackableTypesANDROID)
- `instance`**must** be a valid[XrInstance](https://registry.khronos.org/OpenXR/specs/1.1/html/xrspec.html#XrInstance)handle
- `trackableTypeCountOutput`**must** be a pointer to a`uint32_t`value
- If`trackableTypeCapacityInput`is not`0`,`trackableTypes`**must** be a pointer to an array of`trackableTypeCapacityInput`[XrTrackableTypeANDROID](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_trackables#XrTrackableTypeANDROID)values

### Return Codes

[**Success**](https://registry.khronos.org/OpenXR/specs/1.1/html/xrspec.html#fundamentals-successcodes)

- `XR_SUCCESS`

[**Failure**](https://registry.khronos.org/OpenXR/specs/1.1/html/xrspec.html#fundamentals-errorcodes)

- `XR_ERROR_VALIDATION_FAILURE`
- `XR_ERROR_RUNTIME_FAILURE`
- `XR_ERROR_HANDLE_INVALID`
- `XR_ERROR_INSTANCE_LOST`
- `XR_ERROR_SIZE_INSUFFICIENT`
- `XR_ERROR_SYSTEM_INVALID`
- `XR_ERROR_FUNCTION_UNSUPPORTED`

## Perform a raycast

The[xrRaycastANDROID](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_raycast#xrRaycastANDROID)function is defined as:  

    XrResult xrRaycastANDROID(
        XrSession                                   session,
        const XrRaycastInfoANDROID*                 rayInfo,
        XrRaycastHitResultsANDROID*                 results);

### Parameter Descriptions

- `session`is the[XrSession](https://registry.khronos.org/OpenXR/specs/1.1/html/xrspec.html#XrSession)that performs the raycast.
- `rayInfo`is the[XrRaycastInfoANDROID](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_raycast#XrRaycastInfoANDROID)that describes the ray to cast.
- `results`is a pointer to the[XrRaycastHitResultsANDROID](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_raycast#XrRaycastHitResultsANDROID)that receives the results of the raycast.

The application**can** perform raycasts by calling[xrRaycastANDROID](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_raycast#xrRaycastANDROID).

- If a raycast intersects more trackables than[XrRaycastHitResultsANDROID::maxResults](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_raycast#XrRaycastHitResultsANDROID), the runtime**must** return the hit results that are closest to the[XrRaycastHitResultsANDROID::origin](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_raycast#XrRaycastHitResultsANDROID)of the ray.
- If a raycast intersects a trackable of type`XR_TRACKABLE_TYPE_ANDROID_PLANE`, that is subsumed by another plane, the runtime**must**return the hit result for the subsuming plane only.
- The runtime**must** return the hit results in closest-to-farthest order from[XrRaycastInfoANDROID::origin](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_raycast#XrRaycastInfoANDROID)along the[XrRaycastInfoANDROID::trajectory](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_raycast#XrRaycastInfoANDROID)vector.
- The runtime**must** return`XR_ERROR_TRACKABLE_TYPE_NOT_SUPPORTED_ANDROID`if the trackable type corresponding to the[XrTrackableTrackerANDROID](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_trackables#XrTrackableTrackerANDROID)handles in[XrRaycastInfoANDROID::trackers](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_raycast#XrRaycastInfoANDROID)are not enumerated by[xrEnumerateRaycastSupportedTrackableTypesANDROID](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_raycast#xrEnumerateRaycastSupportedTrackableTypesANDROID).

### Valid Usage (Implicit)

- The[`XR_ANDROID_raycast`](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_raycast#XR_ANDROID_raycast)extension**must** be enabled prior to calling[xrRaycastANDROID](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_raycast#xrRaycastANDROID)
- `session`**must** be a valid[XrSession](https://registry.khronos.org/OpenXR/specs/1.1/html/xrspec.html#XrSession)handle
- `rayInfo`**must** be a pointer to a valid[XrRaycastInfoANDROID](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_raycast#XrRaycastInfoANDROID)structure
- `results`**must** be a pointer to an[XrRaycastHitResultsANDROID](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_raycast#XrRaycastHitResultsANDROID)structure

### Return Codes

[**Success**](https://registry.khronos.org/OpenXR/specs/1.1/html/xrspec.html#fundamentals-successcodes)

- `XR_SUCCESS`
- `XR_SESSION_LOSS_PENDING`

[**Failure**](https://registry.khronos.org/OpenXR/specs/1.1/html/xrspec.html#fundamentals-errorcodes)

- `XR_ERROR_FUNCTION_UNSUPPORTED`
- `XR_ERROR_TRACKABLE_TYPE_NOT_SUPPORTED_ANDROID`
- `XR_ERROR_VALIDATION_FAILURE`
- `XR_ERROR_RUNTIME_FAILURE`
- `XR_ERROR_HANDLE_INVALID`
- `XR_ERROR_INSTANCE_LOST`
- `XR_ERROR_SESSION_LOST`
- `XR_ERROR_OUT_OF_MEMORY`
- `XR_ERROR_LIMIT_REACHED`
- `XR_ERROR_POSE_INVALID`
- `XR_ERROR_TIME_INVALID`
- `XR_ERROR_FEATURE_UNSUPPORTED`

The[XrRaycastInfoANDROID](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_raycast#XrRaycastInfoANDROID)structure is defined as:  

    typedef struct XrRaycastInfoANDROID {
        XrStructureType                     type;
        void*                               next;
        uint32_t                            maxResults;
        uint32_t                            trackerCount;
        const XrTrackableTrackerANDROID*    trackers;
        XrVector3f                          origin;
        XrVector3f                          trajectory;
        XrSpace                             space;
        XrTime                              time;
    } XrRaycastInfoANDROID;

### Member Descriptions

- `type`is the[XrStructureType](https://registry.khronos.org/OpenXR/specs/1.1/html/xrspec.html#XrStructureType)of this structure.
- `next`is`NULL`or a pointer to the next structure in a structure chain. No such structures are defined in core OpenXR or this extension.
- `maxResults`is the`uint32_t`maximum number of results to return.
- `trackerCount`is the`uint32_t`count of the`trackers`array.
- `trackers`is the array of[XrTrackableTrackerANDROID](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_trackables#XrTrackableTrackerANDROID)that the casted ray**should**be tested against.
- `origin`is the[XrVector3f](https://registry.khronos.org/OpenXR/specs/1.1/html/xrspec.html#XrVector3f)that the ray is cast from.
- `trajectory`is the[XrVector3f](https://registry.khronos.org/OpenXR/specs/1.1/html/xrspec.html#XrVector3f)that the ray is targeted at.
- `space`is the[XrSpace](https://registry.khronos.org/OpenXR/specs/1.1/html/xrspec.html#XrSpace)that the ray is cast in.
- `time`is the`XrTime`the ray is cast at.

The[XrRaycastInfoANDROID](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_raycast#XrRaycastInfoANDROID)structure describes the ray to cast.

- The[XrRaycastInfoANDROID::trackers](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_raycast#XrRaycastInfoANDROID)array**may**contain trackers of different types.
- The[XrRaycastInfoANDROID::trackers](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_raycast#XrRaycastInfoANDROID)array**must** not contain multiple trackers of the same type, otherwise the runtime**must** return`XR_ERROR_VALIDATION_FAILURE`.

### Valid Usage (Implicit)

- The[`XR_ANDROID_raycast`](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_raycast#XR_ANDROID_raycast)extension**must** be enabled prior to using[XrRaycastInfoANDROID](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_raycast#XrRaycastInfoANDROID)
- `type`**must** be`XR_TYPE_RAYCAST_INFO_ANDROID`
- `next`**must** be`NULL`or a valid pointer to the[next structure in a structure chain](https://registry.khronos.org/OpenXR/specs/1.1/html/xrspec.html#fundamentals-valid-usage-for-structure-pointer-chains)
- `trackers`**must** be a pointer to an array of`trackerCount`valid[XrTrackableTrackerANDROID](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_trackables#XrTrackableTrackerANDROID)handles
- `space`**must** be a valid[XrSpace](https://registry.khronos.org/OpenXR/specs/1.1/html/xrspec.html#XrSpace)handle
- The`trackerCount`parameter**must** be greater than`0`
- Both of`space`and the elements of`trackers`**must** have been created, allocated, or retrieved from the same[XrSession](https://registry.khronos.org/OpenXR/specs/1.1/html/xrspec.html#XrSession)

The[XrRaycastHitResultsANDROID](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_raycast#XrRaycastHitResultsANDROID)structure is defined as:  

    typedef struct XrRaycastHitResultsANDROID {
        XrStructureType               type;
        void*                         next;
        uint32_t                      resultsCapacityInput;
        uint32_t                      resultsCountOutput;
        XrRaycastHitResultANDROID*    results;
    } XrRaycastHitResultsANDROID;

### Member Descriptions

- `type`is the[XrStructureType](https://registry.khronos.org/OpenXR/specs/1.1/html/xrspec.html#XrStructureType)of this structure.
- `next`is`NULL`or a pointer to the next structure in a structure chain. No such structures are defined in core OpenXR or this extension.
- `resultsCapacityInput`is the capacity of the`results`array, or 0 to indicate a request to retrieve the required capacity.
- `resultsCountOutput`is a pointer to the count of`results`written, or a pointer to the required capacity in the case that`resultsCapacityInput`is insufficient.
- `results`is a pointer to an array of[XrRaycastHitResultANDROID](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_raycast#XrRaycastHitResultANDROID)structures. It**can** be`NULL`if`resultsCapacityInput`is 0.
- See the[Buffer Size Parameters](https://registry.khronos.org/OpenXR/specs/1.1/html/xrspec.html#fundamentals-buffer-size-parameters)section for a detailed description of retrieving the required`results`size.

The[XrRaycastHitResultsANDROID](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_raycast#XrRaycastHitResultsANDROID)contains the array of hits of a raycast.

The runtime**must** set`resultsCountOutput`to be less than or equal to[XrRaycastInfoANDROID::maxResults](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_raycast#XrRaycastInfoANDROID).

### Valid Usage (Implicit)

- The[`XR_ANDROID_raycast`](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_raycast#XR_ANDROID_raycast)extension**must** be enabled prior to using[XrRaycastHitResultsANDROID](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_raycast#XrRaycastHitResultsANDROID)
- `type`**must** be`XR_TYPE_RAYCAST_HIT_RESULTS_ANDROID`
- `next`**must** be`NULL`or a valid pointer to the[next structure in a structure chain](https://registry.khronos.org/OpenXR/specs/1.1/html/xrspec.html#fundamentals-valid-usage-for-structure-pointer-chains)
- `results`**must** be a pointer to an array of`resultsCapacityInput`[XrRaycastHitResultANDROID-based](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_raycast#XrRaycastHitResultANDROID)structures
- The`resultsCapacityInput`parameter**must** be greater than`0`

The[XrRaycastHitResultANDROID](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_raycast#XrRaycastHitResultANDROID)structure is defined as:  

    typedef struct XrRaycastHitResultANDROID {
        XrTrackableTypeANDROID    type;
        XrTrackableANDROID        trackable;
        XrPosef                   pose;
    } XrRaycastHitResultANDROID;

### Member Descriptions

- `type`is the[XrTrackableTypeANDROID](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_trackables#XrTrackableTypeANDROID)of the trackable that the raycast hit.
- `trackable`is the[`XrTrackableANDROID`](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_trackables#XrTrackableANDROID)that the raycast hit, or`XR_NULL_TRACKABLE_ANDROID`if the trackable`type`was`XR_TRACKABLE_TYPE_DEPTH_ANDROID`.
- `pose`is the[XrPosef](https://registry.khronos.org/OpenXR/specs/1.1/html/xrspec.html#XrPosef)that the raycast hit.

The[XrRaycastHitResultANDROID](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_raycast#XrRaycastHitResultANDROID)contains the details of a raycast hit.

The[XrRaycastHitResultANDROID::pose](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_raycast#XrRaycastHitResultANDROID)for a plane hit**must**be such that X and Z are parallel to the plane, and the Y axis is normal to the plane.

|-----------------------------------|----------------------------------------------------------------------------------------------------|
| **Type of trackable hit**         | **Description**                                                                                    |
| `XR_TRACKABLE_TYPE_PLANE_ANDROID` | Hits horizontal and/or vertical surfaces to determine a point's correct depth and orientation.     |
| `XR_TRACKABLE_TYPE_DEPTH_ANDROID` | Uses depth information from the entire scene to determine a point's correct depth and orientation. |

### Valid Usage (Implicit)

- The[`XR_ANDROID_raycast`](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_raycast#XR_ANDROID_raycast)extension**must** be enabled prior to using[XrRaycastHitResultANDROID](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_raycast#XrRaycastHitResultANDROID)
- `type`**must** be a valid[XrTrackableTypeANDROID](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_trackables#XrTrackableTypeANDROID)value

## Example code for raycasting

The following example code demonstrates how to perform raycasts.  

    XrSession session; // previously initialized
    XrTime updateTime; // previously initialized
    XrSpace appSpace;  // space created for XR_REFERENCE_SPACE_TYPE_LOCAL.
    XrPosef headPose;  // latest pose of the HMD.
    XrTrackableTrackerANDROID planeTracker; // tracker for plane trackables.
    XrTrackableTrackerANDROID depthTracker; // tracker for depth trackables.

    // Perform a raycast against multiple trackers.
    XrTrackableTrackerANDROID trackers[] = {
      &planeTracker,
      &depthTracker,
    };
    XrRaycastInfoANDROID rayInfo = {XR_TYPE_RAYCAST_INFO_ANDROID};
    rayInfo.trackerCount = sizeof(trackers) / sizeof(XrTrackableTrackerANDROID);
    rayInfo.trackers = trackers;
    rayInfo.origin = headPose.position;
    rayInfo.trajectory = CalculateForwardDirectionFromHeadPose(headPose);
    rayInfo.space = appSpace;
    rayInfo.time = updateTime;

    uint32_t totalHitResults = 0;
    constexpr uint32 NUM_DESIRED_RESULTS = 2;
    XrRaycastHitResultANDROID hitResult[NUM_DESIRED_RESULTS];
    XrRaycastHitResultsANDROID hitResults = {XR_TYPE_RAYCAST_HIT_RESULTS_ANDROID};
    hitResults.maxResults = NUM_DESIRED_RESULTS;
    hitResults.resultsCapacityInput = NUM_DESIRED_RESULTS;
    hitResults.results = hitResult;
    XrResult result = xrRaycastANDROID(session, &rayInfo, &hitResults);

    if (result == XR_SUCCESS && hitResults.resultsCountOutput >= 1) {
      // Hit results are returned in closest-to-farthest order in
      // hitResults.results[0] .. hitResults.results[hitResults.resultsCountOutput - 1]
    }

**New Enum Constants**

[XrStructureType](https://registry.khronos.org/OpenXR/specs/1.1/html/xrspec.html#XrStructureType)enumeration is extended with:

- `XR_TYPE_RAYCAST_INFO_ANDROID`
- `XR_TYPE_RAYCAST_HIT_RESULTS_ANDROID`

**New Structures**

- [XrRaycastInfoANDROID](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_raycast#XrRaycastInfoANDROID)
- [XrRaycastHitResultANDROID](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_raycast#XrRaycastHitResultANDROID)
- [XrRaycastHitResultsANDROID](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_raycast#XrRaycastHitResultsANDROID)

**New Functions**

- [xrEnumerateRaycastSupportedTrackableTypesANDROID](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_raycast#xrEnumerateRaycastSupportedTrackableTypesANDROID)
- [xrRaycastANDROID](https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_raycast#xrRaycastANDROID)

**Issues**

**Version History**

- Revision 1, 2024-10-02 (Kenny Vercaemer)
  - Initial extension description

*** ** * ** ***

OpenXR™ and the OpenXR logo are trademarks owned by The Khronos Group Inc. and are registered as a trademark in China, the European Union, Japan and the United Kingdom.