Unigine as Module - Correct Usage? (.NET/WPF Solution)

[Alexander.Komarov]

Hello, Unigine Team!

Recently we started migration from 2.7.3 to 2.21 and basically re-writing the whole thing. Part of the old project was the WPF integration, and it seems like the new way of thigs is quite a bit simpler in that direction (rather than the old SharpDX approach).

But in the process of establishing the new solution pipeline with the new version of engine we got in some difficulties (and has solved them so far) which we want to share with others and ask a few questions.

First of all, what we discovered:

1. There's almost zero documentation on how to integrate Unigine to the other projects. The existing examples and samples are cryptic at best and may be very misleading at worst. I.e. WPF integration sample does not showing you how to properly set up the project as a part of the solution, it rather shows very isolated code sample with few basic classes, never even loading a world (which a user must first put to the data folder - that in itself must be figured out somehow) to demonstrate the sample is working correctly. It would be so much better if a WPF sample would be a template to create a project from, but here we must move to the issue #2.

2. The Unigine C# project structure is absolutely not friendly to change of $(OutDir). If any other path is set besides bin/, the project refuses to run, and the engine logs are of little help - at best case, the user can get a hunch on missing data/ folder and the fact that it might be near bin/ (by default), but this is never described anywhere (yes, there is slight mention at https://developer.unigine.com/en/docs/latest/code/command_line?id=120. Who could have thought that the data/ path could be easily changed? Well, anyone, but where they would look at for that info?)

3. Unigine C# project uses sigular bin/ folder for both debug and release configurations, hence the 'mandatory' conditional AssemblyName in .csproj that wants to add %assembly_name%_64d  <- that 'd' suffix to debug artifacts. This makes building complex solutions where other projects may depend on core Unigine project way more harder, especially for CI/CD and publishing.

4. Unigine Docs are lackng information on the 'minimal' project setup - i.e. we want to use only rendering, input and maybe FBX Importer plugin, what are the files and folder structure to build and run such an example? Anyone who wants something like this is doomed to waste tons of time experimenting.

[ludoviksoso]

Hello, @Alexander.Komarov! Thanks for sharing your experience in such detail.

We'll be updating and expanding the documentation on integration and project structure (including based on your feedback)

Regarding data_path - the Engine resolves the path to the "data/" folder relative to the executable location by default. This is also covered in the File System article. Glad to hear you were able to figure it out.

Regarding WPF integration, we hope you've had a chance to look at the following articles covering the integration workflow:

• Integrating with 3rd-Party Software - overview of all integration approaches;
• Integrating with Frameworks - using CustomSystemProxy to embed Unigine into an external application;
• WPF Integration - the WPF sample walkthrough, including project setup and world loading

Thanks for the feedback and good luck with the migration! Feel free to share if you run into anything else.

[Alexander.Komarov]

Thanks for your response! I have read those samples, WPF is a particular one that we are interested in, and as an experienced developer I can say those samples are somewhat lacking of clarity and detail: in part, the WPF sample itself is not really a full sample, it just sits in the SDK subfolder, relying on bin\ directory of the SDK (which is both source AND product in the case of this sample), showing the example of CustomSystemProxy and the engine window, and not really explaining how to set up the integration from scratch. Hope we'll get extended documentation, thanks for your attention again.

[ludoviksoso]

Thanks for the follow-up! Your earlier remarks have been noted and will be addressed in the documentation updates. Just to clarify, the article WPF Integration includes a section "Integrating WPF into Your Own Project" that describes how to set up WPF integration in a standalone project created via SDK Browser, independent of the SDK's own bin/ folder. If that doesn't match your scenario, could you describe what setup you're aiming for? That would help us understand what needs to be improved.

[Alexander.Komarov]

4 hours ago, ludoviksoso said:

Just to clarify, the article WPF Integration includes a section "Integrating WPF into Your Own Project" that describes how to set up WPF integration

Thanks again, but that article does not cover many things a developer might want to do from that point:

1. Set up WPF integration as a separate project (this part is not that hard);

2. Embed a viewport to the panel (this one is tricky, still working on it);

3. Make engine start/restart to not re-create a whole window. Tried to implement, so far no success - SystemProxyjust want the window to be fresh at SystemProxyWpf.cs:151 - attempt to use already existing window breaks everything, because during the Init phase the engine is closing any windows that are added to windows_data dictionary earlier.

The last one I need to make the EngineWindow (ex. MainWindow from WPF sampe) (or, in future, a Viewport panel in any app window) to display a dummy splash for design-time in order to ger rid of annoying 'crash' (the red error in VS, because the engine obviously can not start from under the Designer logic), and for the user too, if the engine has not yet started.

Those are barely surface-level issues we met by barely starting to move our legacy project to the new Unigine veresion and .NET 8. Hope there will be more samples, templates, tutorials and documentation 🥰

upd:

>could you describe what setup you're aiming for?

In general, we're developing a suite of apps for a wide variety of tasks, many of which are involving viewing or editing a 3D-scene. Some apps should have a viewport as one of the panels or as a separate window, other apps might just launch the engine window only (say, a VR training session), and that window is transient (the engine might be restarted multiple times). To satisfy this set of requirements, we kinda want to have full control over the engine life cycle, as well as its viewports - preferrably, independent from the engine itself, so that the Viewports could be spawned independently from the engine Start event and to be connected when it started. Of course, the actual implementation might involve all sorts of tricks to produce seamless user experience, at the end of the day its we, the developers, need to jump all the hoops to make it work. So far, the perspectives looks promising.

Edited May 14 by Alexander.Komarov

[Alexander.Komarov]

Here is the detailed breakdown of what functionality we want to acheive using Unigine as a core of the solution.

1. Let's take UnigineApp.csproj as our main project for standalone Unigine app:

- $(OutDir) must be _bin\$(Configuration)\UnigineApp\ separately for each configration;

- It must copy all Unigine dependencies using MSBuild functionality, so the artifacts could be properly cleaned up for all projects down the dependency tree.

What we have made so far: 

- split SDK files in the project: sdk\bin\ and sdk\bin_d for Release and Debug; and copy them during Build using MSBuild target (Exec Command="robocopy...").

- implemented separate output paths per configuration

- implemented UnigineHandle class to properly resolve data_path from the text file (our data folder is in $(OutDir))

2. Let's take WpfUnigineApp.csproj as .NET8 WPF+WinForms project:

- It must be built to its own $(OutDir), in that case we must copy the UnigineApp's $(OutDir) content, otherwise we're getting only UnigineSharp_double_x64.dll copied as a dependency.

- It must be able to run the engine:

- - in a separate window,

- - in a panel of another window,

- - in the full screen mode.

3. Take the EngineView as WPF component that shows the viewport. It must be independent of engine start event:

- opening EngineView shows the splash screen/placeholder (this is done)

- starting the engine swaps the content of EngineView (this is done too)

The main issues we're having right now:

- WPF app does launches the engine separately from spawning the window, but it shows only engine splash and then black screen (no console is active, no log errors), while the main UnigineApp.exe (copied as a dependency) is running from the same folder and loads empty.world with dummy camera;

- WPF app does launches the engine separately from spawning the window, but it first closing the window we're spawned (with our splash screen), then shows Unigine window with loading splash, then creates our window again, with the viewport in it this time;

- by sudying SystemProxyWPF and MainWindow(EngineWindow) from the WPF sample its hard to conclude, how the viewport integration can be done in a seamless manner using not only a separate window, but also the embedded options (say, as a part of a grid in the main app window).

 

If you don't mind to help us, here's the copy of our playground solution setup, stripped of SDK files of course: https://disk.yandex.ru/d/DzbFGyvK7ts_mA

Any directions are welcomed 😄

[ludoviksoso]

Hello, Alexander! 
Thanks for sharing the project, it helped us understand the issues you're running into. Here are a few tips that should help.

1) Black screen / Engine.Iterate() not running:

The root cause: accessing Handle in the constructor forces HWND creation, but WindowsFormsHost re-parents the control and changes the HWND. The static _mainLoopWindow captures the old handle, so the == Handle check in OnPaint never passes and Engine.Iterate() is never called.
You can move _mainLoopWindow capture to the first OnPaint, where the HWND is stable. We also added keyboard input handling directly in the control (via OnKeyDown/OnKeyUp/OnKeyPress overrides) so the console works when the control has focus.

See the attached UnigineFormControl.cs that will allow you to get the Engine running in your WPF setup.

UnigineFormControl.cs
 

2) On the window disappearing and reappearing during Engine start:

This happens because the Engine calls createWindow() twice when the boot screen is enabled - first for the boot/splash window, then for the main window. In SystemProxyWPF.createWindow(), the boot branch hides your original window (EngineWindow.First.Hide()), creates a separate window for the engine's boot animation, and after it finishes, removes the boot window and creates the main window from EngineWindow.First - which makes the window reappear.

A couple of things to consider:

• Removing the Hide() call keeps your original window visible during boot, though the engine still creates a temporary second window for the boot animation
• Since you already have your own splash (EngineSplash), you can disable the engine's boot screen entirely in default.boot (<screen><enabled>0</enabled></screen>) - this reduces it to a single createWindow() call

For more details on boot screen configuration, see the Loading and Splash Screens documentation.

3) On Engine restart (Init -> Shutdown -> Init)

In earlier versions of the Engine, multiple Init calls per process were supported (Init -> Shutdown -> Init). This is no longer the case - the Engine can only be initialized once per process, and a second Engine.Init() after Shutdown() will result in a fatal error. This is a known limitation.

The recommended approach is:

• Initialize the Engine once at application startup
• Use world_load / world_quit console commands to load and unload worlds as needed
• Use memory_clear to free video memory between scenes
• Hide the Engine window when it's not needed, rather than shutting down the engine

[Alexander.Komarov]

Thanks a lot, ludoviksoso! The explanations are of help. Even though I have solved the black screen issue like 30 minutes after posting the message 😄

I will experiment with your corrections and report on results. Meanwhile, let me ask again on the topic of how to acheive the engine start without the window creation (imagine we have the app window and one of the panels is an EngineView with the splash screen, and when the engine starts, it just swaps the splash control for UnigineControl).

[ludoviksoso]

Take a look at the -main_window startup option.

With -main_window 1 (default), the engine calls createWindow() on your proxy during Init() - your proxy decides what to return. In the WPF sample it creates a separate window, but you can return the HWND of any existing control, including a panel in your application window. The engine renders there directly, including built-in GUI (console, profiler, etc.) You can also register additional viewports at any time via initExternalWindowBuffers - you already have this pattern in your ExternalWindow. The same API works for embedded panels, not just separate windows. However, the main window created via createWindow() is bound for the entire engine lifetime - you can't disconnect and reconnect it later.

If you need the engine to start without any window at all (e.g. to connect a viewport later), take a look at -main_window 0 together with -auto_quit 0. In this case createWindow() is not called, and all viewports are managed through initExternalWindowBuffers / shutdownExternalWindowBuffers. Note that the engine's built-in GUI (console, profiler) won't be available in these external windows. You can create a separate Gui instance for each viewport - the Qt Integration sample (source/samples/3rdparty/ViewportQt/ViewportWidget.cpp) shows this pattern (can be easily adapted for your scenario). This approach gives you full control over the viewport lifecycle - you can connect, disconnect, and reconnect viewports independently from the engine, which sounds like what you need for your suite of apps.

The rendering pattern for external windows is already in the WPF sample's ExternalWindow (and the Qt sample's ViewportWidget) - RenderState.SetViewport() sets the render area dimensions for the external window before Viewport.Render(camera).

[Alexander.Komarov]

Thanks for the tips, looking into it.

For now, I'm trying to skip the engine splash window and use my EngineView control and its host window as engine's main window. The point of falure is, with splash_screen 0 the engine is still calling createWindow() twice, and after that deletes the first handle by calling removeWindow, and then later fails to get a display id, since the window was removed and closed. How could I go around this issue to spawn engine view in an existing window?

I found if I block the removeWindow call and leave the current window with EngineView as registered in SystemProxy, it shows only the last frame of the splash screen, although the splash screen is disabled in default.boot:

<splash_screen>0</splash_screen>

...

<screen>
        <enabled>0</enabled>
        <texture/>
        <font/>
        <messages/>
</screen>

Edited June 5 by Alexander.Komarov

[Alexander.Komarov]

Hello again. Still having difficulties with EngineView. This time something strange happens - that black screen I'm getting after engine init appears to be a floating issue. I was thought I'm missing something, but once in a while the engine gives a picture of a loaded scene and even resizes the viewport correctly when I drag window borders. But unfortunatelly, it looks like the engine misreads the keyboard input - it has reacted couple times to few buttons (i.e. WASD) but unreliably and has quickly stopped doing so at all. Now I'm trying to make it load again, but having no success - getting blackscreened again each time. Here's updated version of the playground project: https://disk.yandex.ru/d/7pZQplf1GiFffw

As before, follow the instructions in sdk_readme.txt to restore SDK dll's and data/ folder.

UPD:

By fiddling around I figured out that it might be the breakpoint issue. I have disabled all breakpoints at init stage and it looks like the engine began to show the image again, but the input is still missing, even though the keys are getting captured on WPF side and translated to the Unigine.

UPD2:

I fixed the inputs by adding the missing events calls. Seems like everything works as intended now.

---

Another question:

I'm trying to init the engine with the particular "-main_window_size" (the goal is to just spawn the viewport on pre-defined position in the layout). Let's say, the EngineView's visual size is 1065x853. I'm reading that value and adding it to the init args as ["-main_window_size", "1065 853"], but createWindow(..) method of SystemProxyWPF is being called with completely unrelated numbers, in case of above numbers its 1331x1152. How can I work this around? I would have assumed there's some DPI stuff involved, but those pair of dimensions aren't even proportional to each other.

UPD3:

Found out 😄

I just disabled setWindowSize() call for embedded EngineView and it works!

Edited June 8 by Alexander.Komarov