Build Tool output for xml files

[chris.dyer]

Hi,
I am trying to use the build tool to collate a set of assets based on the contents of Asset Pack Layer node, the idea would be that this set of assets can be consumed by a prebuilt runtime engine.

So I use a config that just includes the single node file, so that it and its dependent nodes/meshes/textures etc. will be collated in the output.

But I note that the output always seems to include some xml and xsd files (that we use for some internal data structure in our code) that are completely unrelated.

I don't seem to be able to stop them getting into the output even by putting them in the list with both source and runtime disabled. 

Is this expected that all xml files are copied to the output?

We are still on 2.18.1 but I notice in the later versions the package tool has been added to the toolchain, and there I can exclude the xml files from getting into a .ung file.
So I can manually exclude xml/xsd files when I export the package in a similar way, but it seems strange that there are there in the first place.

Cheers,

Chris
 

[silent]

Hi Chris,

Quote

I don't seem to be able to stop them getting into the output even by putting them in the list with both source and runtime disabled. 

Is this expected that all xml files are copied to the output?

This is the expected behavior of the build tool for all files that are "unregistered" or not recognized by the Editor. Since we cannot reliably determine whether such files are required in the final build, the tool assumes they may be necessary and copies them as a precaution. This applies not only to XML and XSD files, but to any other unrecognized file types as well.

If you are completely certain that you do not need any XML/XSD files, the fastest solution for now is simply to remove them manually after the export.

---

Starting with 2.19.x you can indeed exclude specific extension to be skipped from final build packing (but not in 2.18.1):

Thanks!

[chris.dyer]

Ah OK I understand, thanks Silent.

That mechanism makes sense for the intended use of the build tool.

I am probably trying to hijack the workflow a little bit here to create some runtime shareable asset packs (as opposed to the upackage which is more like an editor sharable package).

The idea was that we would have an editor workspace to manage some common assets that can be supplied to a run time engine. 
So was looking for a way to package up some of those assets for delivery (without any actual executables etc.)

Is there some other mechanism or something in the roadmap for just creating an asset package that I should be looking out for?

Cheers,
Chris

[silent]

Hi Chris,

The existing API should be sufficient to implement a custom export plugin that checks the selected node and collects all the necessary information about meshes, materials, and any other parameters you require. A similar task was discussed in the following forum thread:

Your team can incorporate this logic into an Editor plugin so you can use it whenever you want to.

Thank you!

[chris.dyer]

OK, that thread is definitely a similar desire, even if the situation is slightly different, it also seems like it was possibly before the build tool existed?

Is any of the work that the build tool is doing to find all the necessary files available via editor api? or would we be needing to recreate that logic (knowing what references to iterate over to find the files etc.)

I ask as the build tool is doing a bunch of what we want (just as part of another workflow with some extra bits)
And it would seem more sustainable if we could make use of the logic so that it would (hopefully) automatically work for any future unigine versions that might modify the dependencies and dependency walking logic.

Cheers,
Chris

 

[silent]

Hi Chris,

Quote

Is any of the work that the build tool is doing to find all the necessary files available via editor api?

The dependency search logic is not yet exposed through the Editor API. However, we can see that this functionality is in demand, so we will consider adding additional Editor API helpers for it in a future version (most likely after 2.21).

 

Quote

would we be needing to recreate that logic (knowing what references to iterate over to find the files etc.)

Yes, for now that's the only option available, I'm afraid. The runtimes parsing and gathering does not actually require any of the complex dependency logic that we made inside Editor for it's internal use. The most challenging part might be traversing and parsing the NodeReference, but aside from that, the process is fairly straightforward.

 

Quote

it also seems like it was possibly before the build tool existed?

Yes, all the necessary APIs for retrieving the required files have always been available, and they are not related to the Editor API.

Thanks!

[chris.dyer]

OK thanks Silent,

Yes, I think it would be a useful tool for being able to create DLC type assets packs to be able to give to a customer's runtime engine, so having some available helpers would be great.

Yes I think the part that would take some time for us would be the traversal (knowing what children to expect and follow like property parameters that reference files etc. and that we have covered them all) rather than the actual obtaining the file and collating them.

Thanks again for the info and speedy responses as always,
We have a completely separate tool but it has no understanding of the unigine dependencies at present (which is why I was wanting to look at these options)
We will see whether we want to try and continue using the build tool with some extra filtering or write an editor plugin or wait for a later release.

 

[silent]

I believe you will still need to develop your own tool, as there are certain cases we cannot address on our side, and they may depend heavily on the specifics of your project and how content is being prepared and shipped initially.

For example, if you have node1 that using the material1 and texture1 and somewhere else the same node but with different color (node2 -> material2 -> texture1). How to correctly export both of them to avoid further conflicts in runtime? Either you need to do some post-export polishing and somehow tell engine that node2 depends on node1 (either via custom metadata from DLC) or maybe export them as meta-package, including all node variations inside a single ung. This example is probably not directly relevant; it was created mainly to illustrate the idea.

As far as I understand, you are interested in packaging DLC content for your application, correct? Is it primarily models (such as cars, buildings, vegetation), or do you have additional requirements? Understanding this would help us determine what features the dependency API should include in the Editor, and possibly adjust the implementation accordingly.

If you could share a bit more about your needs, that would be greatly appreciated :)

Thank you!

[chris.dyer]

Yep that's fair, for some more context,

The idea is that we are giving user the runtime engine, and we are giving them some capability to build some simple worlds, using some runtime UIs.
This will involve placing some assets in the world at various locations.
We want to build this world building application once, and then supply localised asset packs to a customer that can be imported. 

As you say, the assets are mostly things like buildings, vegetation, and other scenery objects, but some are a bit more specialised dynamic meshes that can be controlled during placement.

So for each of these assets I was thinking I would have a node file to define them, so 2 nodes with same mesh but different materials with different textures, i would expect to have 2 node files.
I might even have a property with different texture or anim file reference values on the 2 nodes. 

Then I was thinking I would have an asset_layer.node
And in this node I would include either node reference or instances (more likely node refences) to each of those node files. 

Then for the packaging tool, I was expecting to just give it that asset_layer.node file, and have it resolve all the dependencies of those children.

It seemed like the build tool was handling that pretty neatly, and it would find both materials, and both textures and any referenced anim files.

So I was expecting this to more work off of a selected file (like the build tool is selecting file or world), not so much a selected node in the world (which I think is what the problem situation you are describing is about?)