2.21 migration

[Amerio.Stephane]

Hi, just some quick question/remarks for the 2.21 migration:

• why choose qt6.5, instead of the more recent qt6.10 or 6.11? any limitation or bug if we go with 6.11?
• utils/natvis/qt6_9.natvis, not 6.5 (no big deal)
• what is configs/default.global ? compared with default.user/default.config?
• editor.boot contains a path to C:/Users/cash-metall/ ...
• dis/cigiconnector::addOnReceivePacketCallback doc is not up to date, and missing from API migration info. add some warnings about the onReceivePacketCallback now renamed onProcessPacketCallback, which is now different from the new onReceive (good thing, but just make it more clear)
• cigi::setProcessPacket removed, no migration info
• no migration info for IG::NetworkRegistrator
• Entity::getDISIDFromEntityID removed, Entity::getDISTypeFromEntityType removed, no migration info
• Meteo::getMeanWindSpeed removed, no migration info (easy to fix, but still)
• no doc info on entity owner ID (not very clear how to use it, new doc also needed for custom connectors)
• This kind of code no longer compiles:
  void WorldLogic::on_recv_ig_control(IG::CIGI::CigiHostPacket* p);
  cigi->addOnProcessPacketCallback(IG::CIGI::CIGI_OPCODE_IG_CONTROL, MakeCallback(this, &WorldLogic::on_recv_ig_control));
• How to safely migrate ig->getConverter()->rotationToEulerIG() ?

There may be more later...

[silent]

Hi Stephane,

Thank you for the feedback! We will improve IG migration guide soon.

Quote

why choose qt6.5, instead of the more recent qt6.10 or 6.11? any limitation or bug if we go with 6.11?

This is due to our current minimum requirements for the Windows compiler. While Qt 6.5.x APIs can still be used with Visual Studio 2017, newer versions are not compatible. Since some of our customers continue to rely on VS 2017, we are unable to adopt more recent versions at this time (at least for 2.21). We hope this situation will improve in the future, allowing us to transition to newer Qt versions.

 

Quote

How to safely migrate ig->getConverter()->rotationToEulerIG() ?

You can use ig->getNEDConverter()->rotationToEulerTARGET();

 

Quote

what is configs/default.global ? compared with default.user/default.config?

That's a bit different type of config, but it's currently not being used. Mostly it's required for global material parameters changes (not yet finished in 2.21). You can temporarily ignore it in your CSV and do not commit. It's probably will be replaced with some other solution in the future.

 

Quote

This kind of code no longer compiles:
void WorldLogic::on_recv_ig_control(IG::CIGI::CigiHostPacket* p);
cigi->addOnProcessPacketCallback(IG::CIGI::CIGI_OPCODE_IG_CONTROL, MakeCallback(this, &WorldLogic::on_recv_ig_control));

Try to change callback signature to:

void WorldLogic::on_recv_ig_control(const IG::CIGI::CigiHostPacket* p);

 

Quote

utils/natvis/qt6_9.natvis, not 6.5 (no big deal)
editor.boot contains a path to C:/Users/cash-metall/ ...

Thanks, will be fixed in the upcoming hotfix.

[Amerio.Stephane]

Thanks. Any tips about Entity::getDISIDFromEntityID, Entity::getDISTypeFromEntityType ?

[silent]

A bit more information about 2.21 IG migration in the attached .MD file. It will be available in the documentation quite soon.

migration-guide-ig-2.21.md

[Amerio.Stephane]

Thanks! Can you confirm if the new callback

typedef CallbackBase2<bool &, CigiHostPacket *> CigiRecvCallback;

can allow us to accept and modify a received packet? (for example, to fix a buggy 3rd party emitter, including IGControl message)?

Can we also alter the HAT/HOTx responses? (-> to change the ground material code)

 

[silent]

Hi Stephane,

Yes, that's the whole point of new callbacks 🙂

 

AppSystemLogic.cpp AppSystemLogic.h

[Amerio.Stephane]

Great, thanks a lot! 🙂

[Amerio.Stephane]

Continuing our 2.21 migration...

In 2.20 and before, I used a WorldLogic:: init() to retrieve some world data, but it now looks like during the init() everything is not correctly set up.

int WorldLogic::init() {
	// this used to retrieve the sun object, but now it returns null
	auto sun = checked_ptr_cast<LightWorld>(ig->getSkyMap()->getSunNode());
	// this used to retrieve the default player, but now it is also null
	auto player = Game::getPlayer();
	...
}

What is going on? is it the expected behavior?

[silent]

Hi Stephane,

Don't think that this behavior is expected.

We are currently trying to reproduce this (in 2.20 first and then in 2.21).

Could you please send us full engine log file of this project? Maybe there are some hints and warnings that would tell us more about this issue.

Thank you!

[Amerio.Stephane]

Very easy to reproduce.

1. Create a new IG project VSC++ from template with new SDK browser 2.1.1 (12629), SDK 2.21 (2.21.0.1 not available for download as of Sunday 12/04)
2. replace AppWorldLogic.cpp with attached one AppWorldLogic.cppand rebuild release version
3. launch_release.bat
4. console -> database_load 1
5. see log.txtlog.txt

Did you change the order or init()?

Edited April 12 by Amerio.Stephane

[Amerio.Stephane]

We also got A LOT of various error messages after the migration in the editor, like:

Mesh::info_mesh(): wrong first magic 0x3431736d in "C:/......./vectors/building_Building_0_0.mesh" file

Mesh::info_mesh(): wrong first magic 0x3231736d in "C:/......./sphere.mesh" file

 

Material Graph loading error (file:"......./AirwayRoad.mgraph"): Invalid file version

Note: I made the migration as I usually do: create a new project, copy all previous data to the new project data/ and then launch editor to do the migration.

building_Building_0_0.mesh sphere.mesh AirwayRoad.mgraph

[silent]

Hi Stephane,

Quote

 SDK 2.21 (2.21.0.1 not available for download as of Sunday 12/04)

Should be available right now, thank you for pointing that out.

 

Quote

Very easy to reproduce.

Create a new IG project VSC++ from template with new SDK browser 2.1.1 (12629), SDK 2.21 (2.21.0.1 not available for download as of Sunday 12/04)
replace AppWorldLogic.cpp with attached one AppWorldLogic.cppand rebuild release version

Thanks for reproduction, we can confirm that we have the same output on our side.

In version 2.21, the initialization order of the IG and Weather components has indeed changed and now occurs during the post-world init stage. We have already made some internal adjustments (purely for testing purposes) to the initialization order of the IG and Weather components, moving it to the pre-world init stage and so far this does not appear to have introduced any issues, at least based on our smaller test projects.

More likely there would be another hotfix release a bit later in April, we can include this changes into the IG. If you need them earlier, please let me know and I will send you patch this week.

 

Quote

We also got A LOT of various error messages after the migration in the editor, like:

Mesh::info_mesh(): wrong first magic 0x3431736d in "C:/......./vectors/building_Building_0_0.mesh" file

Mesh::info_mesh(): wrong first magic 0x3231736d in "C:/......./sphere.mesh" file

Our QA team is already reviewing these files, thank you for providing them. We will be sure to keep you informed as soon as we have any findings.

[Amerio.Stephane]

57 minutes ago, silent said:

In version 2.21, the initialization order of the IG and Weather components has indeed changed

This would explain the null for the sun, but not for the Game::getPlayer(). Something else must have changed.

Anyway, no need for an express fix, next hotfix is okay; I can simply delay the variable init in the update loop for current tests. I'm much more concerned about invalid meshes and materials right now.

[silent]

Hi Stephane,

Thanks for the update. Regarding the migration procedure:

1. Is there any chance that you still have migration logs after you copied 2.20 files into 2.21 project and started Editor?
2. Do you still have these "wrong magic" errors after second Editor launch after the migration or they disappear?

[silent]

I think we've found the root cause. At least we can reproduce the case when no migration is happening after copying the old files. The root cause is the missing configs dir or old configs dir from 2.20 version.

After you created a new empty project (2.21.x) you need to run Editor just once to generate 2.21 configs directory. It's also important to not overwrite it with 2.20 configs after that. Or you can do the migration in two steps:

1. Create new 2.21 project, copy 2.20 configs to it;
2. Run Editor to upgrade configs (2.20 -> 2.21);
   • make sure that configs are all 2.21 (including .meta; if not - try to delete .meta files and restart the Editor);
   • maybe it also worth just skipping configs migration, generate default configs for 2.21 and perform a 2.20 content migration;
3. Close Editor;
4. Copy the rest of the content excluding the configs dir;
5. Run Editor again.

Looks like a bug to me, will see if we can address it in the upcoming hotfix2 update.

[Amerio.Stephane]

Sooooo... I redo the migration from start, just to be sure:

1. created a new 2.20 project, launch editor, close
2. copied our real project data to this 2.20 data, launch editor, check everything is ok, close
3. do the migration to 2.21.0.1 from the SDK Browser, launch editor, close, launch editor, close (multiple times just to make sure it stabilizes)
4. aaaand it's still broken: the GLB importer is definitely broken in 2.21

2.20

2.21

For obvious reason I can't share this GLB. But clearly the 2.21 import makes incorrect names, loses surfaces (probably because of the names), and puts meshes transform everywhere (also, probably because of the names mix-up). Here is the kind of messages I get in the console:

ObjectMeshStatic(Name:"aa5334b0014a001_--a_ds001.fbx", ID:524760083, IDFromFile:524760083, File:"XXXXXXXXXX/entity.world", Hierarchy:"aa5334b0014a001_--a_ds001.fbx")

Object::loadWorld(): can't find "Metal_gray_triplanar" surface in "XXXXXXXXXXX/h140/h140_EMS.node" file

(name says  fbx, but it really is inside a glb)

In this specific case, the surface was not correctly loaded and its content is not the correct one

2.20 

2.21 

 

 

[Amerio.Stephane]

Any feedback about this GLB corruption? any insight about the message "wrong first magic" for some meshes, and the issue with some mgraph?

[silent]

Hi Stephane,

2.21 engine works only with a single mesh file (ms15). 

sphere.mesh is having mesh format 12 (ms12, 2.18.x times), it appears that it was able to successfully avoid a lot of file format change migrations.

building_Building_0_0.mesh is having mesh format 14 (ms14, 2.20x times). 

.mgraph file is from version 2.20, so it can be migrated to 2.21.

I just copied these files in to separate directory and migrated with upgrade script manually: https://developer.unigine.com/en/docs/2.21/upgrade/migration_content/#manual_upgrade

Everything went smooth and all the meshes right now having ms15 format. mgraph is also having a correct version.

Regarding the broken gltf import -- so far we cannot reproduce this behavior. Does this GLTF file contains any animations or bones (skinned mesh)?

As an alternative option could you please try to do fully manual content migration from 2.20x first (as I did with attached files)? Maybe there will be better results overall.

Thanks!

migrated_.zip

[silent]

@Amerio.Stephane Could you please check if you will drop the .GLB file again to the scene from the Asset Browser - will it display normally and have correct surface names?

Or better: try to reimport this GLB file, it should fix itself after that operation. At least on our test stand with similar issues reimport fixed them all. 

Sorry, that's is valid for upcoming hotfix 2 only.

Thanks!

[Amerio.Stephane]

As suggested, I redo a manul upgrade with the script, and it seems indeed to have fixed most issue. Thanks!

I just did some new test with the glb. And indeed, dragging again the glb to the scene fix the incorrect position of all parts and names looks corrects, so that's good.

I also tried to reimport it, and something strange happens: I voluntarily changed the reimport materials option (to prevent overwriting existing one), and a new "materials" folder was created (good) but with a green zipped icon. Strange, as it's a plain folder on disk. Also, there is no way to rename it from Editor. Is this a new expected behavior?

I also observed some broken parts position with imported skinned meshes, but reimporting them seems to fix it. 

 

Edited Friday at 06:22 PM by Amerio.Stephane

[silent]

That's quite interesting.

Can you reproduce this reliably on any broken GLTF file? Or this behavior is affects only a single file? So far we were unable to get similar results with zipped folder display.