Varjo VR and post processes

[Amerio.Stephane]

Hello,

We are using a Varjo XR3 with focal rendering, and post processes are not behaving as we are expecting. To be clear, during the post process pass of the focal renders, the screen UV used are relative to the focal sub-render and not to the global screen.

This leads to some very visible artifacts, as can be seen here when enabling the vignette mask:  (vr_mirror_mode 3, showing both left and right eyes)

Of course, we could disable this very specific vignette fx, but the problem would remain when using a custom post process, such as NVG filters:

 

For a specific evaluation, we also need to simulate binoculars. At first, I'm trying with a simple post process that would simply mask everything but a disk at the point of focus (material attached)post_scoped.mgraph

But this doesn't work as intended:

 

As you can see, the focal rendering pass uses its own UV instead of reusing the global UV, leading to 4 visible disks instead of just 2.

But even if this was fixed, it wouldn't give us the final result we are aiming for, because it would give the user this kind of view in the headset: 

which is plain wrong, as in reality they would just perceive one single disk aperture, as if the disk was centered in the focal area:

So, although it's a different issue than the post-process UV one, how would you really simulate binoculars with a limited (and configurable)  inside a VR headset such as the Varjo?

Thanks!

[bmyagkov]

Hello, Stephane!

Thank you very much for your detailed description and the accompanying visuals — they were extremely helpful. It took us a bit of time to reproduce the issues you reported but we now have a clearer understanding of the situation.

On 6/24/2025 at 5:05 PM, Amerio.Stephane said:

We are using a Varjo XR3 with focal rendering, and post processes are not behaving as we are expecting. To be clear, during the post process pass of the focal renders, the screen UV used are relative to the focal sub-render and not to the global screen.

This leads to some very visible artifacts, as can be seen here when enabling the vignette mask:  (vr_mirror_mode 3, showing both left and right eyes)

This does indeed appear to be a genuine bug. The issue (En-10160) has already been identified and resolved in our internal development branch. The fix will be included in the upcoming 2.20 update which is scheduled for release in the coming days.

To ensure we're on the same page could you please try setting the render_environment_haze_gradient parameter to 0 or 1 via the console and check whether this has any effect on the behavior you’re seeing?

On 6/24/2025 at 5:05 PM, Amerio.Stephane said:

Of course, we could disable this very specific vignette fx, but the problem would remain when using a custom post process, such as NVG filters:

Could you also please try disabling SSSS by setting render_screen_space_shadow_shaft_mode to 0? This may influence the observed behavior — although we’ve noticed that it can occasionally behave incorrectly as well. This issue (En-11768) has already been filed in our internal bug tracker and is scheduled to be addressed. Additionally, we generally recommend keeping this option disabled in VR scenarios for improved performance.

In parallel, we've also opened an internal task (En-11766) to expose API-level functionality for custom compositing in foveated rendering mode. This will allow users to have direct control over how the final image which should offer more flexibility for custom post-processing workflows.

On 6/24/2025 at 5:05 PM, Amerio.Stephane said:

As you can see, the focal rendering pass uses its own UV instead of reusing the global UV, leading to 4 visible disks instead of just 2.

Currently, when working in VR, applying a post-effect material in the material graph affects all viewports simultaneously. However, users do not have the ability to control which specific viewports the effect is applied to — which becomes especially important in setups involving foveated rendering.

To address this, we have proposed (En-11767) to introduce post-effect blending modes that define how the effect is applied with respect to the foveated and context viewports.

Proposed modes:

1. Disabled - Do not apply the post-effect at all.
2. Do not apply to foveated viewport - Apply the post-effect only to the context (non-foveated) viewport; ignore the foveated one.
3. Copy context to foveated viewport - Apply the effect to the foveated viewport by copying the processed result from the context viewport.
4. Post composite - Apply the post-effect after the foveated/context compositing pass is completed — i.e., at the final image stage.

As this is being considered as a feature request and will need to be addressed accordingly, we would first like to confirm whether these proposed modes align with your needs.

Thanks!

[Amerio.Stephane]

5 hours ago, bmyagkov said:

Currently, when working in VR, applying a post-effect material in the material graph affects all viewports simultaneously. However, users do not have the ability to control which specific viewports the effect is applied to — which becomes especially important in setups involving foveated rendering.

This is a complex matter, but in my opinion the post-effect should somehow be able to consider that all VR viewports are really part of a single 'meta-viewport'. As an exemple of implementation, let's imagine a post-effect that must draw some rulers centered in the focal region (like a white cross with measuring ticks). The post effect may consider the UV range as common to both left and right eyes and both left and right focal regions. In such a case, maybe the left eye would only 'see' UVx from 0 to 0.7, while right eye would 'see' 0.3 to 1, and left focal would be UVx 0.4 to 0.7 and right focal 0.3 to 0.6 (all numbers purely hypothetical here, for the sake of the reflection). Then the 'center' of the UV (0.5) would match with the center of both focal viewport.

(violet+red=foveated left, red+orange=foveated right)

Surely this would ensure an easy way (for the artist...) to design a post-effect that is consistently applied and is correctly seen by the VR user. Also, it would be seen the same whatever the VR headset, with or without foveated rendering.

About the suggested API: my main concern here would be that the final render will be different whether or not it's seen on a VR headset with or without foveated rendering, and we try to stay consistent whatever the headset. I can't see how this would help in our case (NVG binoculars), but you might see otherwise (is "Post composite" for this case?)

Also, at this stage, I still don't know how to get access to the foveated center UV coordinates in the material and the current VR viewport (left, right, foveated?) (UV which, with the current implementation, would be different for all 4 viewports; it would be the same {0.5,0.5} with my suggestion)

 

In the mean time, as this won't be available soon, would you have any other suggestion (post-process or not) to implement scoped binoculars (that appears as a single disc aperture to the end user in VR)? Thanks a lot!

[bmyagkov]

Hi Stéphane,

Thank you for providing the additional context — it's very helpful!

The general concept of a meta-viewport is indeed well understood and in fact already partially implemented in our internal development branch. Our current approach allows the foveated viewport to inherit information from the context viewport which brings us closer to supporting the kind of post-effect behavior you're aiming for.

Regarding the post-processing modes we previously suggested it looks like Mode 4 ("post-composed") aligns closely with your needs — this mode applies post-effects after all viewports are composited together which should allow for a consistent visual result across both eyes including foveated regions.

As for your concern: since this is a new feature there was no support for this kind of effect in earlier releases. With release 2.19 due to the limitations of the initial foveated rendering implementation achieving a unified and consistent post-effect like the one you're describing would be extremely difficult and most likely impossible.

However, with the upcoming 2.20 release we've reworked the internal architecture making this kind of functionality feasible. That said, a few additional adjustments are still required on our side to fully support this use case — as mentioned earlier — but they’re relatively minor.

We’d also like to better understand your timeline for this feature. The 2.20 release is scheduled to become available within a week and it includes the internal changes required to support the kind of unified post-processing you're aiming for.

Following our initial investigation we believe that adding post-effect blending modes is a relatively straightforward task. These improvements could potentially be delivered shortly after the 2.20 release, as part of a stability or feature update.

Additionally, we’re considering including sample setups to help illustrate usage and simplify integration on your side.

Let us know if aligning with the 2.20 release works for you — we’ll be happy to prioritize this accordingly.

Thanks!

[Amerio.Stephane]

Hi, as we need this for binoculars and NVG goggles quite soon (this week), we reverted to use a node attached to the VR head for the limited scoped effect, and will use a post-process for the green-vision and we'll deactivate all screen-based effect (apart from this specific post-process obviously). In the future, a single post process will be simpler to manage for both the scoped effect and the green-vision I guess, but even this is just a step for a more complex implementation (to be discussed later).

Thanks!

[bmyagkov]

Hello Stéphane,

Thank you for letting us know.

Yes, it is indeed possible that further discussion may be needed to clarify and finalize the required functionality as you envision. That said, all related tickets have already been created and will be addressed accordingly, should any additional adjustments be necessary.

Thank you again and please accept our apologies for any inconvenience this may have caused.