The-State-of-Model-Importing-Processes-i

Hello, everyone. Welcome to this talk. I'm Plavian Picon. I'm a technical product manager on the Content Pipeline team. My goal today is that since the last few versions of the engine, we have incrementally changed, like updated some of the file format, added support for some extensions. Same for the interchange framework, like our own newish import framework into the engine. And it's not necessarily something that we highlight a lot. So I thought it's a good time to do a status update on that, to see where we are at and where we are going. So my goal today is to give you an overview of where we are on 3D model supports into the engine, also how to import materials in the engine, then present what are the framework, like the import framework changed since last year. And then go a bit more in depth on where we are at for each format. So current status in the engine. So here are the commonly used formats. They are almost in order of the ones that people are the most importing into the engine still lately. So FBX, I guess most of you are using it. It's the most used one we see from our users. Still there, interoperable. You can transfer static mesh, greater mesh, animation, et cetera. It's widely used. But it's not really in development anymore. And if you're looking to do some scene management, like things you would like to do with USD, you're limited in terms of complexity of things you want to represent in scenes. GLTF is quite high, especially since we enable fab into the engines. It's more, it was designed to be a format you use into browsers to do on websites. So there's a few things that comes with the format that makes it a bit more lightweight, or you can make it lightweight so it runs well into browser. The cool thing is that it's open standard, it's extensible, there's an active community requesting new features and all of that. so it's quite moving forward on this point. But even though the Chronos is discussing about can we use this format to represent scenes, it's not really made for that. So something to consider. I put them there because they are still in use, but OBG, it's still used. It's simple, but limited. You don't do a lot with it. CAD here, I'm lying a bit because CAD is a lot of formats, so it's not really a file format per se, but just to give some pointers on this one, this one is really for enterprise customers. It's very specific use, so you get parametric surfaces, not necessarily meshes. You can have fairly complex scenes, but it's more going to be something related to what they want to do in CAD, so construction tree, manufacturing information. It's not really a scene as you would have in virtual production or in-game. And finally, USD, not really the new kid on the block anymore, but not really used so much in the game industry scene so far as I've seen. But it's very interoperable. Also, you get static meshes, animations, creator mesh, materials. You get extra variants. also all the scene composition that comes with a USD that is quite useful if you have complex scenes that you want to transfer between DCC and the engine. Also nice things, it's extensible through schemas and there's an active community. So it's a lively format that is moving forward. Sorry for this huge error, but I wanted to highlight where we are in general. So the key points here is if you look at our file formats here, we can see that even USD, that is the last one we added in VNG, it's already been six years. So everything is fairly stable, production-ready. USD, even though it's written as beta here, actually we are very close to getting it production-ready. So the key message is that you can reliably use these different formats and bring them into the engine to transfer data. What we have, for example, for FBX and USD is that right now we are moving from our legacy import framework to the interchange one. So some of the format are by You still have a legacy so you can fall back to the previous framework if you want But inside UE we are definitely pushing towards this The main change lately is the FBX. If you guys are working at least on 5.5 or more, you might have seen that the UI has changed because you get now the interchange UI. So that came in 5.5, and we did the same for the, if you do importing to level, I guess not many people do that, but if you do importing to level, you will notice this change in 5.7. And just to show the commitment, in UEFN we are also switching the underlying framework to use IntelChange. So even though I'm not asking you to change overnight, it's something to keep in mind. And if you didn't start to look into it, please start looking into it because we are pushing towards this, using this framework to do the imports. That's just to bring back the idea that you can reliably use different formats to bring your assets into the scene. So basically, I took some Quicksil files online and I re-exported some animation from our mannequin to the different file format from Blender and I imported it in the engine. So you get fairly similar results. Of course, Devils is in the details. If you really use some very specific features, you might have things that the format doesn't support. For example, GLTF doesn't support this particular thing we are doing in the engine. Also, you might have some... Like GLTF and USD, they come with a lot of extension and schemas. So when someone says, we support this format, that usually means they support the core and some subset of these extensions. So this is where you might have some things that do not go forward, depending if the exporter will export these features or even if our importer on our side would support this feature. But in general, for the common asset types, you should get something very similar with these formats. Material-wise, the key change coming in 5.7 is we are changing our legacy material system to use Substrate now. So basically you're not going to use the one you're used to, the fixed shading, unlit, opaque, and et cetera. And now you get a system that allows you to be more customization on doing your surface representation. I'm not an expert at all, but there's a very nice talk this afternoon by Nathaniel that is some of the lead on this feature. So if you want to know more about this, I would recommend going and seeing his talk this afternoon. So material-wise, I guess most of the game industry is still making materials in the engine, but I want to take a few minutes to cover if you import, there's ways to import material in a reliable way into the engine, or at least to get a decent representation also in the engine. So FBX, of course, material representation from Glambert is very simple. But if you start looking, GLTF, it's a physically-based representation of material. Same for USD. These ones, they transfer well between DCCs and V-Engine. So you can get very nice-looking materials, and also you can transfer them between different software if you're using these file formats. I want to point out also MaterialX file format that we support in the engine since 2022. This format allows you to describe surface representations. It comes with generic nodes to define surfaces, or you also have preset uber shaders like OpenPBR can be represented with MaterialX. The USD preview surface can also be represented in MaterialX. so this is quite cool because with what Substrate allows us to do now to generate custom surface representation it's tied well with what MaterialX is doing because it carries that so we can transfer more realistically things more reliably things between MaterialX description and what we have in Substrate another thing I would like to highlight is we are slowly moving away from generating by default materials, programmatically generating materials in the engine. So instead, the engine is going to use predefined materials, master materials, material instance we provide, and just fill the parameters based on what we read into the different files. In a way, it's good because we do not generate materials, like hundreds of materials when you import something, you get material instance so you save some performance there It because the materials are designed by tech artists they might be more easily to read for human people and understand how the material is actually set up and then modify it yourself if you want. And also to update, and we don't have to re-update touching the code. We just, if we need to do some tweaks, then we're just going to tweak the material directly. When you import with Substrate, most of this material, basically, you're going to generate the legacy material that then is going to be auto-converted so it works with your Substrate-enabled project. But one thing we've done is, for example, for MaterialX, we have two sets of materials. So one that we use if you have legacy turned on and one if Substrate is on. so that allows us to be to take more to be more realistic or closer to the description that it carries in the materialics and this is something we are going to slowly roll out to the other format so that instead of converting a legacy material to a substrate we are going to try to provide materials that are less than native substrate from the start and most of these things what you can do if you want is that these are defined in the settings. Like in the settings, you can specify, for example, if I'm importing a GLTF, the opaque material I'm going to use to create my opaque materials for GLTF files is this one. So you can duplicate this material, tweak it if you don't like it, and then in the project setting said, please use this material instead of this other one. And the engine is just going to put the material settings into the good properties when you import. That's just to quickly highlight. Here I imported some MaterialX files with legacy and converted to substrate, like auto-convert to substrate. And here I enable substrate from scratch with all the bells and whistles. So the main difference here is just to show that there's a, like you have a coating layer that has been added here that you do not get from legacy. Just to show that by default we have difference because we use different preset materials, depending if you use legacy or substrate. This slide is more what I want to show here is that the fact that we have substrate now that enables us to make custom surface representation. We work with X-Rite guys that make, if you're working in the automotive space, you know them. They make scanners to scan materials. And then to get them into the engine, the cool thing is that they have their own material representation for materials. So they could, with all help, make custom materials that are fitting their own surface representation. And then they use the interchange framework to make their custom translator. So they make their own translation with interchange so they can read their own file formats and then generate their own materials using Substrate. So leveraging both Interchange and Substrate, they could quickly set up a plugin where you can read their own custom material and generate Substrate material in the engine. Cool. Interchange. So what's new with Interchange? I will start with just an overview of what it is and what you can do with it. Basically, the main goal was to make it more customizable for people to be able to tweak the import process, change presets, add their own step when they import things in the engine. So very high level, what we do is that you want to import a model or PNG texture or material in the engine. So the engine, so the Interchange Framework, is going to select the good translation step to read the file for you. then you have processing step that we call pipelines here where basically you're going to take the payload that has been read do you want to apply different settings like it's a mesh do you want do you want to bake it do you want to combine all the static machine one do you want to generate normals do you want to enable nanites etc so this is going there and this step is going to inform basically the factories that are actually going to create your actors and assets into the into the seen. And so because it's more modular, it's easier to now to just come and modify one of them or create your own steps without touching the full pipeline. And you can factories and translators, you would do that with C++. And for the processing step that is usually where people start to work to adjust their custom data prep operation This one you can go and do some Blueprint also Python also some C there We have a tutorial online on the EDC platform where you have a project where you can have a look at how to do any of these steps, basically. So I would recommend people interested in Interchange to have a look at this one. It's a good starting point. So here I will go to how do you customize things with Interchange. So the very first level, because depending how deep you want to go with it, you have different ways to work with it. The first thing is customizing presets. So basically, because now we have these import steps called pipelines here, these things is basically some code that is going to perform the operation I was saying before. Do you want to build a night or not? Do you want to recompute normals or not? Do you want to combine the mesh or not? And all of these options, they are exposed so that people can see them in their dialogue and then modify it. And also, all of this is currently instanced as one asset. So you have one asset in your content folder that carries the behavior with the preset settings that you want to apply when you import a 3D model, for example. And this asset is what is going to really read and use to present the import UI to you when you import. So when you import, by default, we are going to look at this, look at the preset settings, visibility, like is this option visible or not, what is the default setting, and show that to the user. So what you can easily do is just, you could duplicate this asset, it's just an asset. And if you edit it, you can go into it, change the settings, like default settings. Do you want to hide some options? Do you want to lock some options so people cannot access them? And then in your project settings, you say, instead of using this one, that's the default, I want you to use mine. And then the UI is going to be updated according on what you set up there. So that's the first thing you would do to, I don't like the preset default settings I have in the engine for this option, this option when I'm importing my FBX. I will duplicate this one, tweak what I want here, and then now my UI is always showing what I selected. If you want to go a bit further, there's a way that I was showing before where you can make a blueprint pipeline. Basically, again, it's an asset you create. Inside, you have some function hook where you can come and say, I want to do some operation on like, change some properties on the factory that is going to create my assets or I want to change properties on my asset now it's created. And the settings here, they are again exposed into the asset so you can select default values, are they visible or not? And then you will add this to your project settings, intention settings, and then they will show up here and then people can actually edit the settings that you make visible for your extra step when you are importing. So I will be a bit slower on this slide. The current philosophy is that I make some pipelines to import my 3D model. These are my presets. this works well for GLTF and OBEG. But when I want to import GLTF, some presets are different. And not just presets are different, like we've seen before. I might need some extra importing operations to be done because for GLTF, I do something specific for materials. So what we have done is that we have this thing we call a stack where you have a default behavior that is your default operations you want to apply. And then you can pair, let's say file format is more per translator. you can specify, in case I'm importing a 3D model, if it's a GLTF, I want you to use these presets. If it's USD, I want you to use those presets. So you can do this how you would work to make different settings based on the file format you're importing. On top of that, this is quite generic concept of stack. What we recommend is that maybe in your studio, you have presets. Everyone is importing FBX files, for example, but some people are animators, some people are environment artists, so their presets might be different. Some people work in Blender, so maybe their presets are different. So what you would do if you use Interchange, you would duplicate stacks so that you have a stack for animators where you select your presets. If I'm an animator and I'm importing importing FBX, I would have this preset. GLTF, I would have those ones, et cetera, et cetera. And then we added, and then you would switch. You can manually switch here when you're importing. Like, ah, I'm an animator, so I want to select the animator stack or the blender stack. Or I don't know how you, depending how you want to call them and which concept you want to use. And one thing we have added in 5.7, or 5.6, I'm not sure. We added a group option, so you can specify which one you want to use depending on a group that's just a string, so it's just a name. So you could, for example, specify an animator group, an environment group, and then you can specify myself as a user, I belong to the animator group. So because I'm an animator group, by default, I'm going to use the animator stack, so I get the animator presets. presets. So I would let you guys look at the documentation and ask me questions if you want more details on that. But just for you to know that this is available in the Engine and you can create as many presets as you want. And there's ways for you to automatically define presets for certain type of people or certain type of files. So the interchange is growing and UI wise, we start to get a bit complex. So we start working to improve it or make it a bit more simpler. One first step, there's many things to do. One first step is to have this UI here that you can enable in UE, but we mainly did it for UFN for now, where the first thing you're going to see is we hide all the settings here and we are just going to read the file. It tells you there's static mesh, there's materials, there's textures in the file, which ones you want to import. And if it's enough for you, you can just select that. Of course, you can always go to the advanced settings if you want to tweak more options there. And again, all of this is driven by what you set up in your asset here. So you can already by yourself hide many options that you don't want people to see because they don't need to see them. So in general, Interchange is a mature framework now. We added support for level import lately. So when you do import into level, now for FBX you will go through Interchange. We added some re-import settings. So when you're re-importing a static mesh, for example, by default we are just going to change the payload. So just the mesh is going to change and you can reapply the existing settings. That would be what happens if you do apply no properties. But then you can, on reimport, you can tweak some, do some changes on your pipeline. Say, oh, now I don't want to combine the mesh anymore or I don't want to recompute the normals anymore. So then you can specify here, do I want to apply some of the settings that I've changed on reimport on my asset? Yes, no, basically this is what you can do now. And if you are doing import into level or reimport into level, then you have some options. Do you want to recreate the assets that you've deleted? Or do you want to force assets that are not in your new file anymore, etc.? And we also try to add features because the engine is growing on other systems. So we added support for level instance actors. So when you're importing a scene, you can specify now I don't want just actors in the scene. I want you to bundle everything in one level instance actor. And the last thing we have added for 5.7 is the support for Nanite Foliage. So through USD and a custom schemas, the interchange process can now generate skeletal mesh with Nanite Foliage into it. So details per format. So FBX, what I mentioned earlier is what the main change that you are going to notice if you have not switched to 5.5 is the UI is going to change to go to the interchange one. We try to keep the same settings. The goal is not to disrupt. So we kept most of the same settings between the two, but you will notice some renaming, there's some options that have been split into options, some default value that has changed, there's been some reorganization. So again I made on the EDC right now before we put that in the full documentation on the EDC there a tutorial link where I listed everything that has changed between legacy and interchange, so you can have a look. If you're struggling to find a feature, there's some documentation online that would allow you to do the transition. As I mentioned earlier, the FBX is not really moving anywhere anymore so we do not, for the format itself we are not going to add any new features most of the features are coming from the fact that through Interchange you can customize your import one thing we are doing though is we are evaluating I don't know if it's UFBX or MicroFBX library to read the FBX into the engine I know it's using other softwares it would be an alternative to the FBX SDK we are using to load FBX files. So far, what I've heard is we have a few other teams that have been using it, testing it, and we get some improvement in the import speed. So that might be one thing that would, if it's working well, that's something we are going to enable and that will come in the near future. GLTF-wise, so sorry I would just one thing I've forgotten on FBX so FBX right now you have a legacy and interchange but I still the two framework live side by side by default now we are pushing you to interchange but I can understand it can be disruptive for some of you I'm not saying that all of you have to accept it tomorrow if you if you have deadlines if you are struggling with the interchange or you don't have time to modify your pipelines, your automation or your process to use interchange, please, of course, you can disable the interchange framework and UI and just go back to legacy. So if it's what works for you, please do it. I will not force you to move to interchange. One thing to keep in mind is I've seen people on forums, they have issues with Interchange and they disable the Interchange plugin. So I would absolutely not recommend to do that because there are some formats like MaterialX and GLTF that do use Interchange. And so what I would recommend is there's a CVAR to disable Interchange. So you use the CVAR pair file format. So you can specify a pair of file format if you want to interchange for FBX or not, or USD or not. So that would be the way forward. Not disable the full interchange framework, just disable it for FBX if you're not happy with it. So GLTF-wise, that's why I went back. We use interchange, but there's no more legacy. The legacy was that as for those of you that know it, and it's been disabled since quite some time. So it's interchange only. the key point here is if you look at the very colorful array on the side is that we added a lot of extensions all of these are extensions Kronos extensions they have different levels of extensions there is one ratified by Kronos and then there is some that are made by people in the ecosystem so we try to focus on the official ratified Kronos extension first and we added a lot support of these on materials and other features since 5.0. So if you've not touched GLTF imports since a few versions of the engine, we did have a lot of improvement in what we support when we're importing GLTF. And the current status is we are quite happy where we are with GLTF supports of features in the engine. So right now, we are just monitoring what are the new extensions that are coming in, which one are getting ratified, decide what our external user, internal user are asking for. And based on that, we will decide to add more extensions. But right now, we are stable. And the last few things we've done is make sure we have parity between import and export. So there's still a few things about exporting more animations that needs to be done. But basically, that's all. USD-wise, for those of you that do not use a USD in the engine, the first thing to understand is that there are two ways to use USD when you work with an Unreal Engine. We have the typical, I want to import my asset on my level, I get assets, I get Actors, and I work from that. You also get a decent exporter to export your scenes or your assets in the engine. But in some cases you might want to use what we call the USD stage actor So the difference is that you load an actor and the engine is going to keep a link between the USD file you loaded and the assets and actors that are still created in the engine. And then you get also this UI that allows you to have more USD view on the file you've loaded. So you can have information on the prims. you have access to some metadata that we would lose if you import just a straightforward import in the engine. And you can also toggle some properties on the USD stage itself that then would update your scene in the engine. So if you're really doing some come and go between a DCC and the engine using USD, you would probably favor the USD stage actor. So if you're just on pipelines where you want to import, sorry, processes where you want to import assets and be happy with that, then in that case, you would probably use the USD importer. Another thing to keep in mind is that as we, again, as we are moving at the same time as these two different ways of working with USD, we are also moving to interchange. So right now, the stage actor here would use the normal legacy code. At some point, we'll move it to interchange, but that's not planned yet. But right now, if you just want to do asset import or scene import, you can choose if you want to use the legacy importer or the interchange importer. Again, that was going to be a C var in your engine. Another difference with FBX and GLTF is that FBX and GLTF, they come by default. So the interchange framework is enabled by default, and FBX and OBGGLTF comes out of the box. USD, because it's quite heavy and we try to stay away from activating plugins that you might not need on every project, they are all disabled by default. So if you want USD in the engine, you have to actively go and activate some of the plugins. USD core is going to be activated by USD importer over interchange open USD. I will do a quick call out now. We are working to get more details on what we support in the USD format into the documentation, but at the moment, it's not there yet. I will give you a link at the end. There's also a link to a learning page on the EDC where, basically, I have a day that listed everything they are doing into the engine, and I put it there. So if you're looking at is this feature supporting USD, there's a huge page documentation online where you can check, is this feature there? Is it in legacy? Is it in interchange? Is it in the USD stage actor, et cetera? So USD, what we have done since a year, one of the main work was a migration to interchange. So make sure we have an interchange plugin that would work. the same way that the legacy is doing. So the first step in 5.5 was parity with FBX, so make sure that we can import static mesh, skeleton mesh, animation, material, the basic assets. And then with 5.6 and 5.7, we went to add some features that were missing from 5.5 plus some more specific USD features into the interchange version of the importer. and there's still work to be done here so this is still something we are going to expand. The other main topics we are working on in USD where you might notice some improvements is we added some quality of light improvements so performances when reducing load times and the memory footprint especially when you're loading big scenes both on legacy and interchange We added some custom collapsing of primitives. So you can specify in the schema, when you're importing the USD file, do you want everything that is under this primitive to be parsed, loaded, and created as separate assets? Or you can specify this one and this one. I just want to bundle everything into a static mesh. So that allows you, before you start loading, to predefine the granularity of what you want to import into the engines and then save you the creations of hundreds of assets that in the end you might not have wanted. We also do, of course, continue to extend the feature coverage, so add more schemas that are supported into the format So the last one we added is the schema to import audio files that we added camera attributes When you imported cameras in the past there was only a few options settings on the cameras that would come in. Now we added support for a few more. Details of that you would find, for example, on the learning page I mentioned earlier. We've also added some fallback on the collision type. In USD, you can specify which primitives you want to do what in terms of collision. Do you want to generate collision? Do you want to use one that is in the USD file? But when there's nothing specified, the current behavior was not to generate collision, so now you have a way to specify if there's no... if the USD format doesn't describe what to do when there's no collision, when there's collision, please generate something for me. And again, the export is, if you want to export something from the engine, I would either use GLTF or USD. And so we keep it quite up to date also in terms of feature of a USD exporter. The last options we've added is, do you want to export source mesh or render data? That's an issue we had with Nanite, for example. so that gives you do you want basically to export the nanite mesh that is stored in the asset of the random mesh that is that you are seeing since we added the audio for import we also added it for export if some of you are working with a camera rig rail animation we can export that now and we can also export skeletal mesh as a geometry cache and finally on materials so what we've worked on is getting substrate materials for material that come in with material x we added some volume shader support for some other materials another thing we've done is support MaterialX in USD so when you when you USD file allows you to use different type of materials one of them being MaterialX and then there you have options you can some USD file will come with the MaterialX on the side so this was supported already but some software I think it's Solaris for example when you export a USD file the MaterialX is going to be embedded in the USD was not something we supported yet. So this is there now. And the future work for materials, we are going to keep working on the, because Substrate is new, but it might, it's going to evolve. So make sure that the current materials we've made in Substrate are getting updated as we move forward. Add Substrate version of materials, for example, for USD and GLTF also. And one big thing we are planning to look into is how do you export materials from the engine? Now that we have complex representation through substrate, getting them into a MaterialX file format seems a good idea. So that's one of the next things we are going to do. So I'm at the end of my talk. So there's a QR code to a page online that I'm going to update. So what you are going to find on this page is that for Interchange, I put the link to the sample project so you get a learning page where you have a description of what the project does, what are the different pipelines you can modify, how we set up the project, where are the translators, where are the factories. So if you want to start using Interchange and see the breadth of what you can do with it, I would recommend that. There's a link for the FBX that I mentioned earlier. to show what has changed between legacy and interchange in the dialogue UI. So you can quickly track which settings have changed. USD, I put back the documentation to put a link towards the general documentation and also again the link to the list of what we support in the format. And finally, for material, there's documentation to substrate. And right now, I only put a link to a presentation on the IXF XRite plugin I mentioned earlier, and a link to download their plugin if some of you are interested and have Pentora. And I'm going to add the links to the presentation from Nathaniel once it's put on YouTube. And that's all for me. Thank you very much. Thank you.