Mixed Reality Passthrough Leakage?

[axel.goris]

Hey! :)

I'm currently working on a mixed reality with passthrough demo and I'm having a small but annoying issue.
When I use vr_mixed_reality_video_enabled 1, even with alpha_blend and blend_masking at 0, it seems that the skybox still has some passthrough applied?
Here's the issue:

And it looks awfully similar to the current ceiling...

Here are my settings:


Is there an option that I'm missing and that I should disable? Or is it a bug?

[silent]

Hi Axel,

Great view! :)

We’ll need to reproduce similar behavior on our side first. I’ll pass this along to our QA team hopefully, they’ll be able to identify the root cause and suggest some settings or adjustments to apply.

Thanks.

[silent]

I also did a quick check with the C++ VR sample, and there are following lines when MR is enabled:

RenderEnvironmentPresetPtr preset = Render::getEnvironmentPreset(0);

preset->setHazeColor(vec4_black);
preset->setSkyIntensity(0.0f);

Probably that's you what you need to do when you enabling MR in your application as well.

Could you please check if this works?

P.S. This code (slightly modified) is from void MixedRealityMenuGui::prepare_environment(bool enabled) method, taken from MixedRealityMenuGui.cpp.

Thanks!

[axel.goris]

So I disabled my skybox (just a regular ObjectSky) and added your line which makes for quite the "Berserk" like sky!

That's what I expected from it, if the skyIntensity is 0.0, it should be all black, right? Or is there something else somewhere?


PS: I've been trying to use the VR sample but we have the Varjo controllers and they don't work with SteamVR tracking so I'm trying the inside out one but Unigine is not placing them in the right place which makes it near unusable... :(

Edited April 30, 2025 by axel.goris
Added a PS

[axel.goris]

Alright, I redownloaded the vr_sample to use it as default and the issue is also here!
I just opened the Mixed reality panel (with lots of trouble...) and... Tadaaaaaam, the sky is also leaking through?

[silent]

Probably you can fix it by adjusting the exposure compensation parameters. Try to use vr_mixed_reality_override_color_correction_mode 2.

Thanks!

[silent]

If your goal is simply to disable the cubemap overlap while keeping the sky rendering, you can achieve that by setting Cubemap Mode to "Disabled" in the VR Sample settings:2025-04-30 13-35-03.mov

In that case you can also undo the changes that disables the sky rendering that was in a previous post.

Thanks!

[axel.goris]

Wait, I'm not sure I understand what is that cubemap mode? Where does it come from/what does it do?
 

[silent]

Hi Axel,

Cubemap Mode is a Varjo-specific feature. The HMD can generate an environment cubemap from the camera feed, which the engine can then use for various purposes.

The primary goal is to improve realism in mixed reality by calculating ambient lighting and reflections based on the actual surroundings - helping the MR visuals more closely match the real-world camera feed. While it’s not always necessary, it can be useful in cases where accurate environmental reflection and lighting are important.

The Preset 0–2 options indicate that the cubemap will replace the environment textures in the corresponding Environment Presets. The Overlap mode attempts to blend the cubemap with the existing environment instead of overriding it completely.

Hope that makes sense! :)

[axel.goris]

Hey Silent!

Right right right. Yep, I'll just turn it off, I'd rather have non-accurate reflection but no passthrough at all through the sky ahah!

That's really cool to explain, thanks. A little something in the doc could help a bit, or even a simple link to varjo doc?

 

Anyway, awesome, have a great day, you fixed it!

Axel