Cannot configure to stream with CloudXR from Omniverse Kit

Omniverse Core Platform Extensions
, ,
1

Greetings!

I am trying to test streaming using CloudXR following this documentation: Cloud XR User Manual — Omniverse Extensions

However, “Selected Output Plugin” dropdown does not have a “CloudXR” option.

I’ve tried adding the following packages to the .kit file and autoloading them, which did not help.

"omni.kit.xr.advertise" = {}
"omni.kit.xr.core" = {}
"omni.kit.xr.profile.ar" = {}
"omni.kit.xr.profile.common" = {}
"omni.kit.xr.profile.tabletar" = {}
"omni.kit.xr.profile.vr" = {}
"omni.kit.xr.system.openxr" = {}

Any help is greatly apprecited. Thank you!

2

Can I get a bit more information please. What template are you using and what version? Composer 107? Also what headset do you have?

3

Hi @Richard3D, sorry for the late reply. Unfortunately, I just found the notification email in junk.

I am using Editor with version 107. I expect the VR headset to arrive this week, until then, I wanted to try CloudXR Client Sample locally, which already works with the local server sample. But I could not move to that part yet.

In fact, I have been reading various documentation and tutorials to understand how to stream the kit applications using CloudXR. Therefore, also to receive some guidance, I asked about that option being missing from where it is shown in the CloudXR Manual documentation (screenshot from local given below).

Thank you very much!

image
image563×376 15.6 KB

(valid for AR and VR tabs as well)

4

Yes this is because you are using the Kit Base Editor. You are better off with our flagship template, USD Composer. But even then you have to make sure you enable the XR/VR extensions from the Extension Manager.

1 Like
5

Trying to reproduce with a Composer application. The CloudXR option is still missing, but I am not sure if I am doing the CloudXR client sample part right.

With CloudXR Tools for Testing, I am able to stream the server samples by providing local IP on a windowed Windows client sample. However, I am a bit lost at connecting the Composer app to the client sample.

I thought of starting the stream from the Composer app itself, then providing that IP to the client sample using CLI options, just like connecting the CloudXR server and client samples. In the Composer app, there is only “Join With a Session Link” option on clicking LIVE ⚡ button at top right.

There must be something I am missing, but need your guidance on finding that :)

Thank you!

6

Have you tried to use the VR tab not the Tablet AR tab? In your graphic you sent your own looking in the tablet AR tab. You have the VR tab right next door.

I think you might be getting confused between XR,AR and VR. If you’ve ordered a VR headset, then you need to be in the VR system. It has nothing to do with cloudXR.

Go to the VR tab and either select openVR or steamVR. For example, I use a steam headset so I use steam VR.

Make sure you have all the VR extensions enabled an extension manager.

7

Maybe I am not explaining myself correctly, sorry. I have some experience in using XR devices and developing WebXR application. Please ignore the screenshot being taken from Tablet AR, I am actually testing over VR but just shared that screenshot since all three panels had the same visuals and only OpenXR output plugin to select, unlike how it’s shown in Cloud XR User Manual.

I have a Meta Quest 3S just arrived, but I am encountering common Quest Link issues. I am able to connect the headset to Steam VR, yet again on Omniverse, an option for SteamVR does not appear as well.

What I am actually trying to understand is, how to stream the Composer using CloudXR and test it online with a VR headset and mobile.

Hope I am not missing any. I will be checking everything again. Thanks!

8

Ok I see the issue now. OK so all you have to do is go to your Steam VR Settings and make your “runtime” use “OpenXR” not SteamVR. Then stick with “OpenXR” in kit and now they should connect just fine.

Here is an article about it
How to set SteamVR as your OpenXR Runtime – LIV Help Desk

image
image802×632 28.4 KB

1 Like
9

Okay, that worked, thank you! I am able to see the output on the headset.

My real curiosity was about how and where to configure and use CloudXR, to stream the kit application; instead of locally connecting a headset. Forgive me if it is actually obvious and I am missing it.

I greatly appreciate your support!

10

Well, once you have the tablet AR system working in the cloudXR mode you can just set whatever IP address you want. It could be a remote IP or local IP depending on how you’ve configured it.

image
image767×619 40.4 KB

11

I already have the following extensions in my_company.my_composer.kit file.

I assume the image you provided is from CloudXR User Manual document. The issue is that I don’t see those panels and settings in Tablet AR window (or VR/AR). Should I look elsewhere? The same guide requires omni.kit.xr extension, which does not exist (see extensions above). I also assume the guide is a little outdated, so I opened the thread.

12

Actually no. The image I shared last time is directly from my own Composer.

Ok I have an idea here. It seems the best way to troubleshoot this, is to not use your own app for now. Let’s just get the workflow right with the standard off the shelf Composer. There are too many variables otherwise.

I use VR and XR and AR all the time, easily from USD Composer, made from the kit app template. Just make a fresh USD Composer and try it there. See if that all works. Then we can work backwards.

Once you have a default Composer template, just go to Extension Manager and add AR Tablet Experience and VR Experience

13

image
image1296×795 115 KB

image
image1299×802 117 KB

14

@Richard3D , before anything, thank you for your patience. I greatly appreciate it.

I have just cloned a new kit-app-template repository, checked out at 107.2 branch, created a USD Composer from the template, and added VR/AR/Tablet AR profile extensions you with the same versions in your screenshots (107.2.4). I was using 107.0 before and there the extension versions were 107.0.320 (see the screenshots). I tested with and without application layers.

(Previously used in 107.0)

image
image388×294 15.2 KB

image
image409×275 16.1 KB

Using extension versions 107.2.4, I am able to see “CloudXR Runtime” option in “OpenXR Runtime” dropdown (see the screenshot). However, “Selected Output Plugin” dropdown still only has “OpenXR” option; and I am still unable to see the panels you’ve shown in the screenshot (General Settings, Display Settings, World Scale Settings etc.). Do I need to look at another window instead of VR/AR/Tablet AR for those panels?

image
image565×466 23.3 KB

Again, thank you very much for the attention!

15

Look in the ADVANCED section of each of the TabletAR and the VR. But I am curious, did the basic VR headset work? Did the basic Tablet AR work? For an iPad you use our App from the App Store. The Nvidia Omniverse Streaming Client

16

The basic VR headset worked after setting SteamVR as OpenXR runtime (I understand USB connection by “basic”). I could not test with an iPad yet, which I will try as early as tomorrow.

All options in Advanced Settings for both panels:

image
image553×1051 46.1 KB

17

I have the iPad and Omniverse Streaming Client application, but since I cannot access those panels and configure the app for CloudXR, I could not test streaming.

18

Let me contact one of our XR experts and see what we are missing here. Try the ipad streaming as well

19

Hello again!

I have tested with a more performant device, and it helped making progress. I am also able to link the device to the Omniverse scene, using OpenXR runtime (SteamVR as default).

For CloudXR, I have added the dependencies and settings under the .kit file given under “Modify your Kit App Extensions & Settings” section in this documentation. As shown in the image below, “CloudXR Runtime” option appears under “OpenXR Runtime” dropdown. However, the advanced settings are still the same.

image
image838×872 51.1 KB

When starting Tablet AR (or the others) with CloudXR runtime, I read the following logs in the console. It fails to activate OpenXR service startup.

 INFO [ipc_server_main_common] NVIDIA™ CloudXR™ Runtime 'v5.0.0-beta-rc1' starting up...
 INFO [create_device] Created device and COMPUTE queue with QUEUE_GLOBAL_PRIORITY_MEDIUM.
 INFO [vk_print_device_info] Device info:
        name: NVIDIA GeForce RTX 4080 Laptop GPU
        vendor: 0x10de
        product: 0x27e0
        deviceType: VK_PHYSICAL_DEVICE_TYPE_DISCRETE_GPU
        apiVersion: 1.3.280
        driverVersion: 0x8c178000
 INFO [vk_print_features_info] Features:
        timestamp_compute_and_graphics: true
        timestamp_period: 1.000000
        timestamp_valid_bits: 64
        timeline_semaphore: true
 INFO [vk_print_external_handles_info] Supported images:
        VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_WIN32_BIT:
                color import=true export=true
                depth import=true export=true
        VK_EXTERNAL_MEMORY_HANDLE_TYPE_D3D11_TEXTURE_BIT:
                color import=true export=false
                depth import=true export=false
 INFO [vk_print_external_handles_info] Supported fences:
        VK_EXTERNAL_FENCE_HANDLE_TYPE_OPAQUE_WIN32_BIT: true
 INFO [vk_print_external_handles_info] Supported semaphores:
        VK_EXTERNAL_SEMAPHORE_HANDLE_TYPE_D3D12_FENCE_BIT(binary): false
        VK_EXTERNAL_SEMAPHORE_HANDLE_TYPE_OPAQUE_WIN32_BIT(binary): true
        VK_EXTERNAL_SEMAPHORE_HANDLE_TYPE_D3D12_FENCE_BIT(timeline): false
        VK_EXTERNAL_SEMAPHORE_HANDLE_TYPE_OPAQUE_WIN32_BIT(timeline): true
ERROR [create] nvstInitializeLogger() failed: NVST_R_INVALID_OPERATION
ERROR [create] Server::create failed: Result::ERROR_STREAMSDK_LOG_INIT [C:\GitLab-Runner\builds\Ym-RxDzvN\0\cloudxr\cloudxr-openxr-runtime\src\nvidia\cxr\nv_runtime_system.cpp:167]
ERROR [createRuntimeSystem] RuntimeSystem::create failed: Result::ERROR_STREAMSDK_LOG_INIT [C:\GitLab-Runner\builds\Ym-RxDzvN\0\cloudxr\cloudxr-openxr-runtime\src\nvidia\cxr\nv_runtime_service.cpp:383]
 INFO [ipc_server_main_common] Server exiting: '0'
 INFO [print_end_user_failed_information] 
################################################################################
#                                                                              #
#           The NVIDIA™ CloudXR™ Runtime service has failed to start.          #
#                                                                              #
################################################################################
2025-05-19 14:32:57 [73,037ms] [Error] [omni.kit.xr.system.openxr.plugin] Service startup failed
2025-05-19 14:32:58 [74,122ms] [Error] [omni.kit.xr.ui.window.viewport.viewport_layer.viewport_layer] Failed to activate OpenXR: Service startup failed

Hope we are close to figuring this out :)

Thank you so very much!

20

Ok thanks for the log. I have re-read your whole thread he and shown our top engineer. He is telling me that CloudXR only works on an Apple Vision Pro or an iPad at the moment. Do you have one of those two things?