Crowd Generation

[K.Wagrez]

Hello guys,

I'm currently exploring methods to be able to render hundreds, even thousands, of animated characters at once, with variations in terms of animation/mesh. Basically a crowd rendering pipeline. I've read through these posts ( Instanced Animation Rendering - Showcase - UNIGINE Developers Community and Instanced Animation - Animation - UNIGINE Developers Community) but since they are starting to get old, I wondered if any progress had been made on this subject.

I have looked on the unigine store, nothing yet. I may need to get my hands dirty and go the same way as @christian.wolf2 by making shaders for Vertex Animation Textures to render instances of my animated characters, but I wanted to make sure Unigine didn't have implemented since 2021 any tool to do just that.

So ? Nothing new under the sun to do crowd rendering in Unigine?

 

[silent]

Hi Kevin,

There is indeed no built-in crowd animation tools available yet out of the box.

Depending on how many skinned meshes you want to actually draw you can also try a naive approach without involving any custom shaders and animations baking.

Using the built-in mechanism of node update you can try to make calculations less intensive when objects are not visible:

• https://developer.unigine.com/en/docs/2.19.1/objects/objects/mesh_skinned/?autotranslate=en#optimize

Also you can split mesh skinned into a groups (let's say 500 skinned per group) and update them sequentially to reduce CPU / GPU cost. If that's still not providing enough performance you can switch to vertex baked animations that you referenced in your first post.

Quote

but since they are starting to get old

These techniques are still quite popular, so you can safely integrate them without much worrying. Maybe also contacting Christian directly would be much more efficient approach :)

Thanks!

[K.Wagrez]

I was targeting a use case with over 2000 visible skinned characters at once, crowded together (like in a concert, or waiting for a train) and relatively close to use. There isn't a lot of hacks I can use to reduce the cost in this context. I think baking the bone animation in a texture might be the better path.

By the way I saw in Art samples you had an example of using Vertex Animation (Vertex Animation Sample - Documentation - Unigine Developer). You mention using Blender to bake the textures for the skinned characters. Which tool did you use by curiosity ? I have been trying this one (Vertex Animation Texture Addon (VAT) — Blender Extensions), thinking this might be the one, but I've had some trouble using it with your sample.

[silent]

AFAIK, we used different plugin for Blender to bake this animation: https://github.com/JoshRBogart/unreal_tools?files=1

The approximate pipeline is as follows:

1. Simulate the object in Blender.
2. Convert the simulation into keyframe animation.
3. Trim the animation so that the object is in its original state on the first keyframe (this step is important).
4. Select the mesh and click “Process Anim Meshes.” The add-on will generate a new mesh with appropriate UV coordinates for vertex animation, along with two textures: displacement and normals. These textures need to be saved using the settings shown in the provided screenshot:
   
5. Export the mesh (make sure the mesh is named exactly export_mesh) to the engine and assign it a material with the newly generated textures.

Important Notes:

• The mesh must have the exact same vertex count; otherwise, the animation may become misaligned with the texture.
• If you encounter issues with the object's normals in the final version (such as it appearing too dark), one of the normal axes may be incorrectly inverted. This can be fixed using an Expression node in the material graph.

Since it's just a sample you can develop your own pipeline with animations baking and playback and use different tools for that.

[christian.wolf2]

Hi @K.Wagrez,

thanks for referencing me. In my post I had implemented my own approach of building that textures, because I wanted to test which one is better (vertex baked animation or bone baked instancing). It is not that difficult, because UNIGINE provides some easy functions to store vector data into an texture. As written in the post, you need to consider your ideal approach. Bone baked instancing provides you an more flexible solution, because for each animation frame you store for each bone their position. So you will get an smaller texture size (Number of pixels needed = Number of bones x Number of animation frames) or you can even store multiple animations into one texture. Vertex baked animation should be only considered for objects with small/medium sized vertices, but is faster due to the simple LUT approach.

For a custom implementation, I would recommend the following approach:

• Use an ObjectMeshSkinned and load it into your scene + Load and apply an animation to it
• Use the setLayerFrame() and updateSkinned() to iterate over your whole animation cycle. NOTE: Because the internal animation of UNIGINE can interpolate between two frames, make sure to use a fixed step size. You can later increase precision by increasing this step size, but this will of course increase your animation texture.
• Grab with getBoneTransform() for each bone the transform matrix and store it into the texture.
• Once done, create a new shader which uses the TextureArray-functionality and load your animation frame into it.
• Take a look at this post, you can find a perfect example of how to render multiple objects in one DIP. On CPU side, provide an instance ID, the instance world transform and the frame-time and pass it to the GPU. On the shader side use the vertex shader to multiply each vertex for each instance ID with the transform stored into your texture. Multiply it with the world transform to get the final result.

No "magic" involved and me, as a shader noob, even got this implementation done by myself. :) So no worries and in case of further questions, I can will try to provide you some help. On top of that you can go further by introducing animation blending (common games and engines uses 2 animations for blending at max) and LOD-support.

 

For further "optimization" of crowd rendering I highly recommend you this video of a GDC conference talk from Creative Assembly, the developers behind the total war series. The have done some great deep-dive behind the LOD/instance selection and job system for supporting large crowds.

[K.Wagrez]

Wow @christian.wolf2 thanks, I wasn't expecting such a quick and detailed answer :D. I was planning at some point to send you a message to enquire about your work in instanced rendering but you've beat me to it :). 

I will investigate, try to make something work, look at the resources from GDC and will come back here if I'm stuck or just to share my progress :).

[christian.wolf2]

No problem, happy to help you. My implementation was done with an older UNIGINE version and I didn`t had any time to update the source code but nothing crucial has changed on the first look. So in case of any issues I can re-check it for you.

Also, the GDC stuff is a little bit more advanced but I think it is highly interesting how "the big guys" are doing it :)