Gildor's Forums

Author Topic: [TUTORIAL] How to "Uncook" assets (with some limitations for the moment)  (Read 16386 times)
Soulax
Newbie
*
Posts: 8


View Profile
[TUTORIAL] How to "Uncook" assets (with some limitations for the moment)
« on: June 01, 2021, 23:05 »

Hello everyone,

Today I am sharing a video tutorial, as requested. In this video I show you from A to Z how to uncook an asset that was cooked in engine version 4.22 (the game is Trials of Mana). I then export the uncooked asset in UE5 as an exemple (but it could be any other engine version).

Note : I didn't try a lot of stuff, but I assume if your Editor is able to open your cooked asset then the uncooking process should work Wink

This tutorial is in 4 parts :

Part 1 : Setting up the engine to make it able to load cooked asset inside the editor.

Part 2 : Setting up new c++ classes to add a new blueprint function "Decook Assets"

Part 3 : Make a new Widget Editor and use this blueprint

Part 4 : Use the widget to uncook an asset and then export it to UE5

I hope that will be a good start point for a lot of futur evolutions. Guys I encourage you to work on it if you want more support and share it with everyone if you get some nice results !

Best regards !

C++ Files : https://www.mediafire.com/file/cs9o84350d4ipt3/UNCOOKER.7z/file

« Last Edit: June 04, 2021, 15:41 by Soulax » Logged
Digika
Jr. Member
**
Posts: 44


View Profile
Re: [TUTORIAL] How to "Uncook" assets (with some limitations for the moment)
« Reply #1 on: June 07, 2021, 16:45 »

This is kinda pointless for a few reasons:

1. uModel32 already extracts meshes and textures and animations and this is what you showcased in your tutorial

2. No uncooking for any of the Maps, Datatables or Custom Tables or Blueprint structure overview (not code, just overview), etc

3. This also requires building your own UE4 version and this is a very complex process. It is not enough to download and install all CORRECT requirement for compilation and have environment set up correctly (all dependencies and SDKs, settings, env, variables and custom tools), which is enormous task prone to smallest errors that cascade and break eveything, you need to download about 4Gb of code then also hope their Setup.bat will even work in your system (if you read their forums it is one of the first issues that stops people here, TONS of threads) and then hope there will be no compilation or linker error, fixing which you can spend days and weeks, and THEN you also need to wait probably whole day to compile it all on average machine. When you can just use uModel to extract it all in few seconds.
« Last Edit: June 07, 2021, 16:47 by Digika » Logged
spiritovod
Global Moderator
Hero Member
*****
Posts: 2189


View Profile
Re: [TUTORIAL] How to "Uncook" assets (with some limitations for the moment)
« Reply #2 on: June 07, 2021, 19:18 »

@Digika: It's surely some kind of misunderstanding here.

1) This approach will allow you to save a lot of time in certain situations with bulk transfers involved, on exporting assets -> third-party software -> UE in particular. It may sound almost the same as UE -> assets -> UE scheme, suggested in the tutorial, but it's actually not. It will also allow you to skip some possible problems with converting assets. In addition, it will be very useful for someone who wants to use it as base for implementing automation of maps converting/transferring.

2) While you may be right about maps (though uncooking them is usually very game dependent), all cooked blueprints and datatables are literally custom data and the logic behind them are stored in the exe after building the game. So it's impossible by design to parse them automatically, you can only parse them manually, based on you assumptions. Parsing import table only (it's what you called "overview") is pretty useless without interpreting the rest - in the whole context - considering the nature and purpose of this tutorial.  

3) That's completely not true. By coincidence, yesterday I've decided to compile custom version of one of UE 4.25 tools (unrealpak), because of "wild oodle appeared! all devs are started to randomly use it" thing. While I never dealt with UE source, following the official guide, it took me around 2 hours, and most of that time was downloading dependencies (which is fully automated). Compiling the tool with custom plugin was as easy as: download 400MB branch -> click on bat -> wait for dependencies to download -> put plugin in respective place -> click on another bat to create VS solutions -> open main project in VS -> wait for VS to install all required SDK dependencies (automatically) -> click on build for required tool -> wait for 5 minutes -> done. Officially they suggest 10 to 40 minutes compilation time for the whole engine, but on my potato laptop it would take around 2-3 hours maximum.
I believe that most issues are coming from outdated/custom Windows and environments. Otherwise compiling UE on win platform is pretty much hassle-free and automated, if you're not touching core dependencies. Also, because of the way it's compiling and storing intermediate components into "intermediate" folders, you can simply reset some stuff in case of problems.  

Considering all that, your post looks more like "it's not what I wanted" kind of post without addressing actual issues of this method - like huge UE distribution (I got 40GB in total, after compiling one component from 69) or something.  
« Last Edit: June 07, 2021, 19:30 by spiritovod » Logged
Digika
Jr. Member
**
Posts: 44


View Profile
Re: [TUTORIAL] How to "Uncook" assets (with some limitations for the moment)
« Reply #3 on: June 07, 2021, 19:31 »

There is no misunderstanding.

1) This approach will allow you to save a lot of time in certain situations with bulk transfers involved, on exporting assets -> third-party software -> UE in particular. It may sound almost the same as UE -> assets -> UE scheme, suggested in the tutorial, but it's actually not. It will also allow you to skip some possible problems with converting assets. In addition, it will be very useful for someone who wants to use it as base for implementing automation of maps converting/transferring.

But uModel32 allows bulk extraction of all supported assets?

Quote
all cooked blueprints and datatables are literally custom data and the logic behind them are stored in the exe after building the game. So it's impossible by design to parse them automatically, you can only parse them manually, based on you assumptions.  
Wrong.
All blueprint structure is standard and outer layer can be deserialized. All DataTables or stringtables also have standard structure and can be deserialized regardless of the fame. There are minor differences between versions for some properties, of course, but that is a given

Quote
3) That's completely not true. By coincidence, yesterday I've decided to compile custom version of one of UE 4.25 tools (unrealpak), because of "wild oodle appeared! all devs are started to randomly use it" thing. While I never dealt with UE source, following the official guide, it took me around 2 hours, and most of that time was downloading dependencies (which is fully automated). Compiling the tool with custom plugin was as easy as: download 400MB branch -> click on bat -> wait for dependencies to download -> put plugin in respective place -> click on another bat to create VS solutions -> open main project in VS -> wait for VS to install all required SDK dependencies (automatically) -> click on build for required tool -> wait for 5 minutes -> done. Officially they suggest 10 to 40 minutes compilation time for the whole engine, but on my potato laptop it would take around 2-3 hours maximum.
I believe that most issues are coming from outdated/custom Windows and environments. Otherwise compiling UE on win platform is pretty much hassle-free and automated, if you're not touching core dependencies. Also, because of the way it's compiling and storing intermediate components into "intermediate" folders, you can simply reset some stuff in case of problems.

It is true. if I ask you to build me a custom version of 4.26.2 with changes from OP your first thought would be "bugger off" because of the amount effort it takes. Sure, if you ALREADY have build development set up it is a no brainer but that is the thing - first, you need to set it up and it is madness.
You also completely ignore the whole context - I was comparing this approach against the one uModel provides. If you are already doing engine modification and other custom stuff and need sources my argument is not applicable, I'm talking about user-side of things.

Quote
Considering all that, your post looks more like "it's not what I wanted" kind of post
...Yes, and? You are saying this like it is a bad thing. It is not what I wanted and I stated my personal opinion and reasons as to why. If you dont like the feedback you can learn to ignore it or skidaddle, I dont really give a crap, I'm not here to only make posts you or someone else will like.

« Last Edit: June 07, 2021, 19:33 by Digika » Logged
GDL
Jr. Member
**
Posts: 74


View Profile
Re: [TUTORIAL] How to "Uncook" assets (with some limitations for the moment)
« Reply #4 on: June 07, 2021, 19:48 »

Hello everyone,

Today I am sharing a video tutorial, as requested. In this video I show you from A to Z how to uncook an asset that was cooked in engine version 4.22 (the game is Trials of Mana). I then export the uncooked asset in UE5 as an exemple (but it could be any other engine version).

Note : I didn't try a lot of stuff, but I assume if your Editor is able to open your cooked asset then the uncooking process should work Wink

This tutorial is in 4 parts :

Part 1 : Setting up the engine to make it able to load cooked asset inside the editor.

Part 2 : Setting up new c++ classes to add a new blueprint function "Decook Assets"

Part 3 : Make a new Widget Editor and use this blueprint

Part 4 : Use the widget to uncook an asset and then export it to UE5

I hope that will be a good start point for a lot of futur evolutions. Guys I encourage you to work on it if you want more support and share it with everyone if you get some nice results !

Best regards !

C++ Files : https://www.mediafire.com/file/cs9o84350d4ipt3/UNCOOKER.7z/file


Nice!

Hopefully this can be modified to work with assets from older UE4 versions, specially to get accurate body part scales that are calculated after the fact making the ingame mesh look different than the extracted mesh
Logged
spiritovod
Global Moderator
Hero Member
*****
Posts: 2189


View Profile
Re: [TUTORIAL] How to "Uncook" assets (with some limitations for the moment)
« Reply #5 on: June 07, 2021, 20:05 »

@Digika: So you like to read between the lines. You didn't even try to understand what I was talking about, just arguing because it seems you like to argue. And you even failed to understand the purpose of that method - obviously it's not for very beginners, but also not for professional devs - though it's still pretty automated, you don't need to setup dependencies or environments, because supplied engine solutions and VS will do it for you.

Also, look at all those universal blueprints / datatables uncookers around, which are not game dependent! Oh wait... Again, in engine-to-engine scheme (which is what this tutorial is about) blueprints and datatables are out of the board - because even if you can deduct some basic stuff from cooked versions of them, the core of custom logic is stored in the exe and they're not transferable even partially.

I wonder, if the topic was called differently, like "How to transfer cooked model and animation assets between different engine versions without using umodel", would you complain as much as you do now.
« Last Edit: June 07, 2021, 20:29 by spiritovod » Logged
Gildor
Administrator
Hero Member
*****
Posts: 7971



View Profile WWW
Re: [TUTORIAL] How to "Uncook" assets (with some limitations for the moment)
« Reply #6 on: June 08, 2021, 09:48 »

50 cents from me.

It is possible to use Marketplace (Epic Games Store) version of the engine for building custom code. For instance, at one of my job's projects I use it to build a plugin which can work as project-side and engine-dide plugin. No engine build is required, and even more I must use the Marketplace engine for being able to distribute plugin for it.

UModel is NOT doing things explained in this topic. Using the code provided here you may load some of asset types directly into UE4, what is desired by large percent of the people. You can't load it into Blender (what's desired by other part). However you may export assets from UE4, the most interesting thing is saving meshes and animations into Fbx format, so you'll have morphs, curves etc.

Blueprints and materials can't be uncooked for more or less modern UE4, because the data is entirely stripped from assets. For instance, materials will not have anything from the node graph stored - only the compiled shader bytecode is in uasset file. I believe the same is true for blueprints.
Logged
Digika
Jr. Member
**
Posts: 44


View Profile
Re: [TUTORIAL] How to "Uncook" assets (with some limitations for the moment)
« Reply #7 on: June 08, 2021, 12:57 »

Blueprints and materials can't be uncooked for more or less modern UE4, because the data is entirely stripped from assets. For instance, materials will not have anything from the node graph stored - only the compiled shader bytecode is in uasset file. I believe the same is true for blueprints.
You can get a general structure like function names, reference and the way they relate to each other, it is generic info that is not tied to specific game (unless they rewrote blueprint source code)
Logged
Almie
Newbie
*
Posts: 2


View Profile
Re: [TUTORIAL] How to "Uncook" assets (with some limitations for the moment)
« Reply #8 on: June 13, 2021, 06:51 »

This is very impressive! I will be trying this for sure, assuming I can figure out how to compile a custom UE4 build Grin It would be super cool to see other asset types being supported in the future, especially materials! I don't know if it's possible, but that's something I would be most interested in.
Logged
Digika
Jr. Member
**
Posts: 44


View Profile
Re: [TUTORIAL] How to "Uncook" assets (with some limitations for the moment)
« Reply #9 on: June 18, 2021, 21:40 »

If you know someone who compiled 4.26.2 with these changes and uploaded it - please share the link, some people asking for it for modding for morphs
Logged
ksimpson1986
Newbie
*
Posts: 2


View Profile
Re: [TUTORIAL] How to "Uncook" assets (with some limitations for the moment)
« Reply #10 on: July 14, 2021, 23:17 »

Hey! this is awesome and I got it to work on 4.23.1 as well on the new Spongebob remake. since this video is a month and a half old, have you managed to figure out how to view static meshes as well? I can only seem to view skeletal meshes and that's it. Static meshes will throw out some kind of "LOD.DistanceFieldData" error. Just curious if you've figured out anything new!
Logged
Digika
Jr. Member
**
Posts: 44


View Profile
Re: [TUTORIAL] How to "Uncook" assets (with some limitations for the moment)
« Reply #11 on: July 18, 2021, 02:15 »

Code:
if (lodData.SkinWeightVertexBuffer.GetNumVertices() > 0)
{
if (lodData.SkinWeightVertexBuffer.HasExtraBoneInfluences())
{
const TSkinWeightInfo<true>* info = lodData.SkinWeightVertexBuffer.GetSkinWeightPtr<true>(idx);
FMemory::Memcpy(vtx.InfluenceBones, info->InfluenceBones, TSkinWeightInfo<true>::NumInfluences);
FMemory::Memcpy(vtx.InfluenceWeights, info->InfluenceWeights, TSkinWeightInfo<true>::NumInfluences);
} else
{
const TSkinWeightInfo<false>* info = lodData.SkinWeightVertexBuffer.GetSkinWeightPtr<false>(idx);
FMemory::Memcpy(vtx.InfluenceBones, info->InfluenceBones, TSkinWeightInfo<false>::NumInfluences);
FMemory::Memcpy(vtx.InfluenceWeights, info->InfluenceWeights, TSkinWeightInfo<false>::NumInfluences);
}
}

No longer compiles in 4.25, HasExtraBoneInfluences does not even exist
Logged
spiritovod
Global Moderator
Hero Member
*****
Posts: 2189


View Profile
Re: [TUTORIAL] How to "Uncook" assets (with some limitations for the moment)
« Reply #12 on: July 18, 2021, 17:59 »

They changed SkinWeightVertexBuffer in latest versions, so I suppose this part should look more like this now, but didn't test it:
Code:
if (lodData.SkinWeightVertexBuffer.GetNumVertices() > 0)
{
FSkinWeightInfo info = lodData.SkinWeightVertexBuffer.GetVertexSkinWeights(idx);
FMemory::Memcpy(vtx.InfluenceBones, info->InfluenceBones, sizeof(info->InfluenceBones));
FMemory::Memcpy(vtx.InfluenceWeights, info->InfluenceWeights, sizeof(info->InfluenceWeights);
}
« Last Edit: July 18, 2021, 18:04 by spiritovod » Logged
Digika
Jr. Member
**
Posts: 44


View Profile
Re: [TUTORIAL] How to "Uncook" assets (with some limitations for the moment)
« Reply #13 on: July 18, 2021, 20:03 »

Nope, that is definitely not a solution. First of all, you can use pointer memeber dereference -> on a reference type and second that condition was there for a reason. There is plenty of other errors anyway, someone knowledgeable should take alook at some point
Logged
swjames
Newbie
*
Posts: 3


View Profile
Re: [TUTORIAL] How to "Uncook" assets (with some limitations for the moment)
« Reply #14 on: August 20, 2021, 13:28 »

Thanks for sharing. Would this method work for just viewing the material graph in ue 4.21? I don't really need the uncooked ver.

EDIT : Just to follow up. The material graph open fine in editor. The nodes all show up but are all disconnected. The biggest problem is that almost all the material function is empty, so it's not really usable. Game is Granblue Versus btw.
« Last Edit: August 21, 2021, 04:49 by swjames » Logged
Jump to:  

Powered by SMF | SMF © 2006-2009, Simple Machines LLC
Leviathan design by Bloc | XHTML | CSS