Hello, everyone.
Welcome to this talk.
I'm Plavian Picon.
I'm a technical product manager on the Content Pipeline team.
My goal today is that since the last few versions of the engine,
we have incrementally changed, like updated some of the file
format, added support for some extensions.
Same for the interchange framework,
like our own newish import framework into the engine.
And it's not necessarily something that we highlight a lot.
So I thought it's a good time to do a status update on that,
to see where we are at and where we are going.
So my goal today is to give you an overview of where
we are on 3D model supports into the engine,
also how to import materials in the engine,
then present what are the framework,
like the import framework changed since last year.
And then go a bit more in depth on where
we are at for each format.
So current status in the engine.
So here are the commonly used formats.
They are almost in order of the ones that people
are the most importing into the engine still lately.
So FBX, I guess most of you are using it.
It's the most used one we see from our users.
Still there, interoperable.
You can transfer static mesh, greater mesh, animation, et
cetera.
It's widely used.
But it's not really in development anymore.
And if you're looking to do some scene management, like
things you would like to do with USD, you're limited in terms
of complexity of things you want to represent in scenes.
GLTF is quite high, especially since we enable fab into the engines.
It's more, it was designed to be a format you use into browsers to do on websites.
So there's a few things that comes with the format that makes it a bit more lightweight,
or you can make it lightweight so it runs well into browser.
The cool thing is that it's open standard, it's extensible,
there's an active community requesting new features and all of that.
so it's quite moving forward on this point.
But even though the Chronos is discussing about
can we use this format to represent scenes,
it's not really made for that.
So something to consider.
I put them there because they are still in use,
but OBG, it's still used.
It's simple, but limited.
You don't do a lot with it.
CAD here, I'm lying a bit because CAD is a lot of formats,
so it's not really a file format per se,
but just to give some pointers on this one,
this one is really for enterprise customers.
It's very specific use,
so you get parametric surfaces,
not necessarily meshes.
You can have fairly complex scenes,
but it's more going to be something related
to what they want to do in CAD,
so construction tree, manufacturing information.
It's not really a scene as you would have
in virtual production or in-game.
And finally, USD, not really the new kid on the block anymore, but not really used so much in the game industry scene so far as I've seen.
But it's very interoperable.
Also, you get static meshes, animations, creator mesh, materials.
You get extra variants.
also all the scene composition that comes with a USD
that is quite useful if you have complex scenes
that you want to transfer between DCC and the engine.
Also nice things, it's extensible through schemas
and there's an active community.
So it's a lively format that is moving forward.
Sorry for this huge error,
but I wanted to highlight where we are in general.
So the key points here is if you look at our file formats here,
we can see that even USD, that is the last one we added in VNG,
it's already been six years.
So everything is fairly stable, production-ready.
USD, even though it's written as beta here,
actually we are very close to getting it production-ready.
So the key message is that you can reliably use these different formats
and bring them into the engine to transfer data.
What we have, for example, for FBX and USD
is that right now we are moving from our legacy import framework
to the interchange one.
So some of the format are by You still have a legacy so you can fall back to the previous framework if you want But inside UE we are definitely pushing towards this
The main change lately is the FBX.
If you guys are working at least on 5.5 or more,
you might have seen that the UI has changed because you get now the interchange UI.
So that came in 5.5, and we did the same for the, if you do importing to level,
I guess not many people do that, but if you do importing to level,
you will notice this change in 5.7.
And just to show the commitment,
in UEFN we are also switching the underlying framework
to use IntelChange.
So even though I'm not asking you to change overnight,
it's something to keep in mind.
And if you didn't start to look into it,
please start looking into it
because we are pushing towards this,
using this framework to do the imports.
That's just to bring back the idea
that you can reliably use different formats
to bring your assets into the scene.
So basically, I took some Quicksil files online
and I re-exported some animation from our mannequin
to the different file format from Blender
and I imported it in the engine.
So you get fairly similar results.
Of course, Devils is in the details.
If you really use some very specific features,
you might have things that the format doesn't support.
For example, GLTF doesn't support this particular thing
we are doing in the engine.
Also, you might have some...
Like GLTF and USD, they come with a lot of extension and schemas.
So when someone says, we support this format,
that usually means they support the core
and some subset of these extensions.
So this is where you might have some things
that do not go forward,
depending if the exporter will export these features
or even if our importer on our side would support this feature.
But in general, for the common asset types,
you should get something very similar with these formats.
Material-wise, the key change coming in 5.7
is we are changing our legacy material system to use Substrate now.
So basically you're not going to use the one you're used to,
the fixed shading, unlit, opaque, and et cetera.
And now you get a system that allows you to be more customization
on doing your surface representation.
I'm not an expert at all,
but there's a very nice talk this afternoon by Nathaniel
that is some of the lead on this feature.
So if you want to know more about this,
I would recommend going and seeing his talk this afternoon.
So material-wise, I guess most of the game industry
is still making materials in the engine,
but I want to take a few minutes to cover
if you import, there's ways to import material
in a reliable way into the engine,
or at least to get a decent representation also in the engine.
So FBX, of course, material representation from Glambert is very simple.
But if you start looking, GLTF, it's a physically-based representation of material.
Same for USD.
These ones, they transfer well between DCCs and V-Engine.
So you can get very nice-looking materials,
and also you can transfer them between different software
if you're using these file formats.
I want to point out also MaterialX file format
that we support in the engine since 2022.
This format allows you to describe surface representations.
It comes with generic nodes to define surfaces,
or you also have preset uber shaders like OpenPBR
can be represented with MaterialX.
The USD preview surface can also be represented in MaterialX.
so this is quite cool
because with what Substrate allows us to do now
to generate custom surface representation
it's tied well with what MaterialX is doing
because it carries that
so we can transfer more realistically things
more reliably things between MaterialX description
and what we have in Substrate
another thing I would like to highlight
is we are slowly moving away from generating by default materials,
programmatically generating materials in the engine.
So instead, the engine is going to use predefined materials,
master materials, material instance we provide,
and just fill the parameters based on what we read
into the different files.
In a way, it's good because we do not generate materials,
like hundreds of materials when you import something,
you get material instance so you save some performance there It because the materials are designed by tech artists they might be more easily to read for human people
and understand how the material is actually set up
and then modify it yourself if you want.
And also to update,
and we don't have to re-update touching the code.
We just, if we need to do some tweaks,
then we're just going to tweak the material directly.
When you import with Substrate,
most of this material, basically,
you're going to generate the legacy material
that then is going to be auto-converted
so it works with your Substrate-enabled project.
But one thing we've done is, for example, for MaterialX,
we have two sets of materials.
So one that we use if you have legacy turned on
and one if Substrate is on.
so that allows us to be
to take more
to be more realistic or closer to the description
that it carries in the materialics
and this is something we are going to slowly
roll out to the other format
so that instead of converting
a legacy material to a substrate
we are going to try to provide
materials that are
less than native substrate from the start
and most of these things
what you can do
if you want is that these are defined in the settings.
Like in the settings, you can specify, for example,
if I'm importing a GLTF, the opaque material I'm going to use
to create my opaque materials for GLTF files is this one.
So you can duplicate this material, tweak it if you don't like it,
and then in the project setting said,
please use this material instead of this other one.
And the engine is just going to put the material settings
into the good properties when you import.
That's just to quickly highlight.
Here I imported some MaterialX files with legacy
and converted to substrate, like auto-convert to substrate.
And here I enable substrate from scratch with all the bells and whistles.
So the main difference here is just to show that there's a,
like you have a coating layer that has been added here
that you do not get from legacy.
Just to show that by default we have difference
because we use different preset materials,
depending if you use legacy or substrate.
This slide is more what I want to show here
is that the fact that we have substrate now
that enables us to make custom surface representation.
We work with X-Rite guys that make,
if you're working in the automotive space, you know them.
They make scanners to scan materials.
And then to get them into the engine,
the cool thing is that they have their own material
representation for materials.
So they could, with all help, make custom materials that
are fitting their own surface representation.
And then they use the interchange framework
to make their custom translator.
So they make their own translation with interchange
so they can read their own file formats
and then generate their own materials using Substrate.
So leveraging both Interchange and Substrate,
they could quickly set up a plugin
where you can read their own custom material
and generate Substrate material in the engine.
Cool.
Interchange.
So what's new with Interchange?
I will start with just an overview of what it is
and what you can do with it.
Basically, the main goal was to make it more customizable for people
to be able to tweak the import process, change presets,
add their own step when they import things in the engine.
So very high level, what we do is that you want to import a model
or PNG texture or material in the engine.
So the engine, so the Interchange Framework,
is going to select the good translation step to read the file for you.
then you have processing step that we call pipelines here where basically you're going to
take the payload that has been read do you want to apply different settings like it's a mesh do you
want do you want to bake it do you want to combine all the static machine one do you want to generate
normals do you want to enable nanites etc so this is going there and this step is going to inform
basically the factories that are actually going to create your actors and assets into the into the
seen. And so
because it's more modular,
it's easier to now
to just come and modify one
of them or create your own steps without
touching the full pipeline.
And you can
factories and translators,
you would do that with C++. And for the
processing step that is usually where people start to work to adjust their custom data prep operation This one you can go and do some Blueprint also Python also some C there
We have a tutorial online on the EDC platform
where you have a project where you can have a look at how
to do any of these steps, basically.
So I would recommend people interested in
Interchange to have a look at this one.
It's a good starting point.
So here I will go to how do you customize things
with Interchange.
So the very first level, because depending
how deep you want to go with it, you
have different ways to work with it.
The first thing is customizing presets.
So basically, because now we have these import steps
called pipelines here, these things
is basically some code that is going to perform the operation
I was saying before.
Do you want to build a night or not?
Do you want to recompute normals or not?
Do you want to combine the mesh or not?
And all of these options, they are exposed
so that people can see them in their dialogue
and then modify it.
And also, all of this is currently instanced as one asset.
So you have one asset in your content folder that carries the behavior with the preset settings that you want to apply when you import a 3D model, for example.
And this asset is what is going to really read and use to present the import UI to you when you import.
So when you import, by default, we are going to look at this, look at the preset settings, visibility, like is this option visible or not, what is the default setting, and show that to the user.
So what you can easily do is just,
you could duplicate this asset, it's just an asset.
And if you edit it, you can go into it,
change the settings, like default settings.
Do you want to hide some options?
Do you want to lock some options
so people cannot access them?
And then in your project settings,
you say, instead of using this one, that's the default,
I want you to use mine.
And then the UI is going to be updated
according on what you set up there.
So that's the first thing you would do to,
I don't like the preset default settings I have in the engine
for this option, this option when I'm importing my FBX.
I will duplicate this one, tweak what I want here,
and then now my UI is always showing what I selected.
If you want to go a bit further,
there's a way that I was showing before
where you can make a blueprint pipeline.
Basically, again, it's an asset you create.
Inside, you have some function hook
where you can come and say,
I want to do some operation on like,
change some properties on the factory
that is going to create my assets
or I want to change properties on my asset
now it's created.
And the settings here,
they are again exposed into the asset
so you can select default values,
are they visible or not?
And then you will add this to your project settings,
intention settings,
and then they will show up here
and then people can actually edit the settings
that you make visible for your extra step
when you are importing.
So I will be a bit slower on this slide.
The current philosophy is that I make some pipelines
to import my 3D model.
These are my presets.
this works well for GLTF and OBEG.
But when I want to import GLTF,
some presets are different.
And not just presets are different,
like we've seen before.
I might need some extra importing operations to be done
because for GLTF,
I do something specific for materials.
So what we have done is that
we have this thing we call a stack
where you have a default behavior
that is your default operations you want to apply.
And then you can pair,
let's say file format is more per translator.
you can specify, in case I'm importing a 3D model,
if it's a GLTF, I want you to use these presets.
If it's USD, I want you to use those presets.
So you can do this how you would work
to make different settings based on the file format you're importing.
On top of that, this is quite generic concept of stack.
What we recommend is that maybe in your studio,
you have presets. Everyone is importing FBX files, for example,
but some people are animators, some people are environment artists,
so their presets might be different.
Some people work in Blender, so maybe their presets are different.
So what you would do if you use Interchange,
you would duplicate stacks so that you have a stack for animators
where you select your presets. If I'm an animator and I'm importing
importing FBX, I would have this preset.
GLTF, I would have those ones, et cetera, et cetera.
And then we added, and then you would switch.
You can manually switch here when you're importing.
Like, ah, I'm an animator, so I want
to select the animator stack or the blender stack.
Or I don't know how you, depending
how you want to call them and which concept you want to use.
And one thing we have added in 5.7, or 5.6, I'm not sure.
We added a group option, so you can specify which one you want to use
depending on a group that's just a string, so it's just a name.
So you could, for example, specify an animator group, an environment group,
and then you can specify myself as a user, I belong to the animator group.
So because I'm an animator group, by default, I'm going to use the animator stack,
so I get the animator presets.
presets.
So I would let you guys look at the documentation
and ask me questions if you want more details on that.
But just for you to know that this is available in the Engine
and you can create as many presets as you want.
And there's ways for you to automatically define
presets for certain type of people or certain type of files.
So the interchange is growing and UI wise, we start to get a bit complex.
So we start working to improve it or make it a bit more simpler.
One first step, there's many things to do.
One first step is to have this UI here that you can enable in UE,
but we mainly did it for UFN for now,
where the first thing you're going to see
is we hide all the settings here
and we are just going to read the file.
It tells you there's static mesh,
there's materials,
there's textures in the file,
which ones you want to import.
And if it's enough for you,
you can just select that.
Of course, you can always go to the advanced settings
if you want to tweak more options there.
And again, all of this is driven
by what you set up in your asset here.
So you can already by yourself
hide many options that you don't want people to see
because they don't need to see them.
So in general,
Interchange is a mature framework now.
We added support for
level import
lately. So when you do import into level,
now for FBX
you will go through Interchange.
We added some re-import
settings.
So when you're
re-importing a static mesh, for example, by default
we are just going to change the payload.
So just the mesh is going to change
and you can reapply the existing settings.
That would be what happens if you do apply no properties.
But then you can, on reimport,
you can tweak some, do some changes on your pipeline.
Say, oh, now I don't want to combine the mesh anymore
or I don't want to recompute the normals anymore.
So then you can specify here,
do I want to apply some of the settings
that I've changed on reimport on my asset?
Yes, no, basically this is what you can do now.
And if you are doing import into level
or reimport into level,
then you have some options.
Do you want to recreate the assets
that you've deleted?
Or do you want to force assets
that are not in your new file anymore, etc.?
And we also try to add features
because the engine is growing on other systems.
So we added support for level instance actors.
So when you're importing a scene, you can specify now
I don't want just actors in the scene.
I want you to bundle everything in one level instance actor.
And the last thing we have added for 5.7
is the support for Nanite Foliage.
So through USD and a custom schemas,
the interchange process can now generate
skeletal mesh with Nanite Foliage into it.
So details per format.
So FBX, what I mentioned earlier is what the main change that you are going to notice if you have not switched to 5.5 is the UI is going to change to go to the interchange one.
We try to keep the same settings.
The goal is not to disrupt.
So we kept most of the same settings between the two,
but you will notice some renaming,
there's some options that have been split into options,
some default value that has changed,
there's been some reorganization.
So again I made on the EDC right now before we put that in the full documentation on the EDC there a tutorial link
where I listed everything that has changed
between legacy and interchange,
so you can have a look.
If you're struggling to find a feature,
there's some documentation online
that would allow you to do the transition.
As I mentioned earlier,
the FBX is not really moving anywhere anymore
so we do not, for the format itself
we are not going to add any new features
most of the features are coming from the fact that
through Interchange you can customize your import
one thing we are doing though is we are evaluating
I don't know if it's UFBX or MicroFBX library
to read the FBX into the engine
I know it's using other softwares
it would be an alternative to the FBX SDK
we are using to load FBX files.
So far, what I've heard is we have a few other teams
that have been using it, testing it,
and we get some improvement in the import speed.
So that might be one thing that would,
if it's working well, that's something we are going to enable
and that will come in the near future.
GLTF-wise,
so sorry I would just one thing I've forgotten on FBX so FBX right now you have a legacy and
interchange but I still the two framework live side by side by default now we are pushing you
to interchange but I can understand it can be disruptive for some of you I'm not saying that
all of you have to accept it tomorrow if you if you have deadlines if you are struggling with
the interchange or you don't have time to modify your pipelines,
your automation or your process to use interchange,
please, of course, you can disable the interchange framework
and UI and just go back to legacy.
So if it's what works for you, please do it.
I will not force you to move to interchange.
One thing to keep in mind is I've seen people on forums,
they have issues with Interchange and they disable the Interchange plugin.
So I would absolutely not recommend to do that
because there are some formats like MaterialX and GLTF that do use Interchange.
And so what I would recommend is there's a CVAR to disable Interchange.
So you use the CVAR pair file format.
So you can specify a pair of file format if you want to interchange for FBX or not, or USD or not.
So that would be the way forward.
Not disable the full interchange framework, just disable it for FBX if you're not happy with it.
So GLTF-wise, that's why I went back.
We use interchange, but there's no more legacy.
The legacy was that as for those of you that know it, and it's been disabled since quite some time.
So it's interchange only.
the key point here is
if you look at the very colorful
array on the side
is that we added a lot
of extensions
all of these are extensions
Kronos extensions
they have different levels of extensions
there is one ratified by Kronos and then there is some
that are made by
people in the ecosystem
so we try to focus on the
official ratified Kronos extension first
and we added a lot
support of these on materials and other features since 5.0.
So if you've not touched GLTF imports since a few versions of the engine,
we did have a lot of improvement in what we support when we're importing GLTF.
And the current status is we are quite happy where we are
with GLTF supports of features in the engine.
So right now, we are just monitoring what are the new extensions that are coming in,
which one are getting ratified,
decide what our external user, internal user are asking for.
And based on that, we will decide to add more extensions.
But right now, we are stable.
And the last few things we've done
is make sure we have parity between import and export.
So there's still a few things about exporting more animations
that needs to be done.
But basically, that's all.
USD-wise, for those of you that do not use a USD in the engine,
the first thing to understand is that there are two ways to use USD
when you work with an Unreal Engine.
We have the typical, I want to import my asset on my level,
I get assets, I get Actors, and I work from that.
You also get a decent exporter to export your scenes or your assets in the engine.
But in some cases you might want to use what we call the USD stage actor So the difference is that you load an actor and the engine is going to keep a link between the USD file you loaded
and the assets and actors that are still created in the engine.
And then you get also this UI that allows you to have more USD view
on the file you've loaded.
So you can have information on the prims.
you have access to some metadata that we would lose
if you import just a straightforward import in the engine.
And you can also toggle some properties on the USD stage itself
that then would update your scene in the engine.
So if you're really doing some come and go between a DCC and the engine using USD,
you would probably favor the USD stage actor.
So if you're just on pipelines where you want to import,
sorry, processes where you want to import assets
and be happy with that,
then in that case, you would probably use the USD importer.
Another thing to keep in mind is that as we, again,
as we are moving at the same time
as these two different ways of working with USD,
we are also moving to interchange.
So right now, the stage actor here
would use the normal legacy code.
At some point, we'll move it to interchange,
but that's not planned yet.
But right now, if you just want to do asset import
or scene import, you can choose
if you want to use the legacy importer
or the interchange importer.
Again, that was going to be a C var in your engine.
Another difference with FBX and GLTF
is that FBX and GLTF, they come by default.
So the interchange framework is enabled by default, and FBX and OBGGLTF comes out of the box.
USD, because it's quite heavy and we try to stay away from activating plugins that you might not need on every project,
they are all disabled by default.
So if you want USD in the engine, you have to actively go and activate some of the plugins.
USD core is going to be activated by USD importer over interchange open USD.
I will do a quick call out now.
We are working to get more details on what we support in the USD format into the documentation,
but at the moment, it's not there yet.
I will give you a link at the end.
There's also a link to a learning page on the EDC where, basically, I have a day that listed everything they are doing into the engine,
and I put it there.
So if you're looking at is this feature supporting USD,
there's a huge page documentation online where you can check,
is this feature there?
Is it in legacy?
Is it in interchange?
Is it in the USD stage actor, et cetera?
So USD, what we have done since a year,
one of the main work was a migration to interchange.
So make sure we have an interchange plugin that would work.
the same way that the
legacy is doing. So the first
step in 5.5 was parity with
FBX, so make sure that we can import
static mesh, skeleton mesh,
animation, material, the basic
assets. And then with
5.6 and 5.7, we went
to add some
features that were missing from 5.5 plus
some more specific USD
features into
the interchange version of
the importer.
and there's still work to be done here
so this is still something we are going to expand.
The other main topics we are working on in USD
where you might notice some improvements
is we added some quality of light improvements
so performances when reducing load times
and the memory footprint
especially when you're loading big scenes
both on legacy and interchange
We added some custom collapsing of primitives.
So you can specify in the schema,
when you're importing the USD file,
do you want everything that is under this primitive
to be parsed, loaded, and created as separate assets?
Or you can specify this one and this one.
I just want to bundle everything into a static mesh.
So that allows you, before you start loading,
to predefine the granularity of what you want to import into the engines
and then save you the creations of hundreds of assets
that in the end you might not have wanted.
We also do, of course, continue to extend the feature coverage,
so add more schemas that are supported into the format So the last one we added is the schema to import audio files that we added camera attributes When you imported cameras in the past there was only a few options
settings on the cameras that would come in. Now we added support for a few more. Details
of that you would find, for example, on the learning page I mentioned earlier. We've also
added some fallback on the
collision type.
In USD, you can specify
which primitives you want
to do what in terms of collision.
Do you want to generate collision? Do you want to use one that
is in the USD file? But when
there's nothing specified,
the current behavior was not to generate
collision, so now you have a way to specify
if there's no... if the USD
format doesn't describe
what to do when there's
no collision,
when there's collision, please generate something for me.
And again, the export is,
if you want to export something from the engine,
I would either use GLTF or USD.
And so we keep it quite up to date also
in terms of feature of a USD exporter.
The last options we've added is,
do you want to export source mesh or render data?
That's an issue we had with Nanite, for example.
so that gives you
do you want basically to export the
nanite mesh that is stored in the asset
of the random mesh that is
that you are seeing
since we
added the audio for import we also added
it for export
if some of you are working
with a camera rig rail animation
we can export that now
and we can also export
skeletal mesh as a geometry cache
and finally on materials so what we've worked on is getting substrate materials for
material that come in with material x we added some volume shader support for some other materials
another thing we've done is
support
MaterialX in USD so when you
when you USD
file allows you to use different
type of materials one of
them being MaterialX and then
there you have options you can
some USD file will come with the
MaterialX on the side so this was supported
already but some software I think
it's Solaris for example when you
export a USD file
the MaterialX is going to be embedded in the USD
was not something we supported yet.
So this is there now.
And the future work for materials,
we are going to keep working on the,
because Substrate is new,
but it might, it's going to evolve.
So make sure that the current materials
we've made in Substrate
are getting updated as we move forward.
Add Substrate version of materials,
for example, for USD and GLTF also.
And one big thing we are planning to look into
is how do you export materials from the engine?
Now that we have complex representation through substrate,
getting them into a MaterialX file format
seems a good idea.
So that's one of the next things we are going to do.
So I'm at the end of my talk.
So there's a QR code to a page online
that I'm going to update.
So what you are going to find on this page
is that for Interchange,
I put the link to the sample project
so you get a learning page
where you have a description of what the project does,
what are the different pipelines you can modify,
how we set up the project,
where are the translators, where are the factories.
So if you want to start using Interchange
and see the breadth of what you can do with it,
I would recommend that.
There's a link for the FBX that I mentioned earlier.
to show what has changed between legacy and interchange in the dialogue UI.
So you can quickly track which settings have changed.
USD, I put back the documentation to put a link towards the general documentation
and also again the link to the list of what we support in the format.
And finally, for material, there's documentation to substrate.
And right now, I only put a link to a presentation on the IXF XRite plugin I mentioned earlier,
and a link to download their plugin if some of you are interested and have Pentora.
And I'm going to add the links to the presentation from Nathaniel once it's put on YouTube.
And that's all for me. Thank you very much.
Thank you.