October 16, 2018, 17:19
bigger smaller reset 800px Wide width Full width Reset * *

Gildor's Forums

  Homepage Facebook Donate
Welcome, Guest. Please login or register.
Did you miss your activation email?


Login with username, password and session length
« previous next »
Pages: [1] Print
Author Topic: The Last Remnant Texture2D extraction  (Read 154 times)
cfarl
Newbie
*
Posts: 7


View Profile
« on: September 29, 2018, 00:29 »

Hi, Gildor!

First of all, great work you have done in UModel and others tools!

Well, I'm working in a brazilian translation of The Last Remnant. I did some tools to extract and inject text in upk files, but currently I'm in a figth with some upk files trying extract textures from them to edit and replace.

I used UModel to open, visualize and extract the texture tga file from the upk. Here my upk:

https://drive.google.com/open?id=1M-ZwBeQzuq0HbWfMnIDYEA8fjxC1hRcT

So, I used your Unreal Package Extractor to extract the .Texture2D file from this upk (from xbox360). Here is this file:

https://drive.google.com/open?id=1g6hnbk5F3m0lDxnLvOUMgq4_0_lir4na

It seems there is inside the .Texture2D a LZO file, but I had no sucess in extracting it.

So... Here I'm, asking for your help.  

What can I do do extract the .tga file from the .Texture2D? Or, how can I get the .tga from the LZO in the .Texture2D file?

Thanks in advance,

cfarl
« Last Edit: October 02, 2018, 17:30 by cfarl » Logged
Gildor
Administrator
Hero Member
*****
Posts: 6843



View Profile WWW
« Reply #1 on: September 29, 2018, 09:43 »

Hi.
Yes, UE3 textures has 2nd level of compression. I can't help you with editing.
BTW I'll move this thread to the more correct place, and you may read other threads there - may be they'll help.
Logged
cfarl
Newbie
*
Posts: 7


View Profile
« Reply #2 on: September 29, 2018, 16:48 »

Thanks for tour replay, Gildor!

I looked in some topics, but I cannot find anything that could help in this compression issue.

So, in this texture2D file there is a double compression?

Lzo is one of them, right? How can I find the other? There is a tool to compress/desconpress this kind of stuff?

There is some documents you can share about this?

It would be nice if someone here could help me in solving this problem.

Thanks!
Logged
Gildor
Administrator
Hero Member
*****
Posts: 6843



View Profile WWW
« Reply #3 on: September 29, 2018, 17:07 »

1. texture data has compression
2. then upk file has compression layer too

Usually only one of them it used at time. "Decompressor" tool from this site removes upk compression, but it is not possible to remove texture data compression.

You may extract texture data in DDS format (if it uses DXT1-5 internally). However putting compressed data back is very very tricky - some people did that. I never modified games myself (outside of my interest area).

You may also check some links from umodel wiki - I don't remember if I posted them on the forum or not, so just in case ...
https://github.com/gildor2/UModel/wiki
Logged
cfarl
Newbie
*
Posts: 7


View Profile
« Reply #4 on: October 02, 2018, 18:10 »

Well, I did some improvements...

After looking at this page http://wiki.tesnexus.com/index.php/Savegame_file_format_-_XCOM:EU_2012#Data_Structures I found that there is FCompressedChunk in my Texture2D file.

So, I extracted this part of the Texture2D file, that begins with the magic 9e 2a 83 c1:

https://pasteboard.co/HGAOU21.jpg 

Now, using the FCompressedChunk record:

FCompressedChunk
   - FCompressedChunkHeader     // 16 bytes
   - FCompressedBlock
       - FCompressedBlockHeader // 8 bytes
       - Raw compressed data    // variable size

I got:

// FCompressedChunkHeader - 16 bytes
9E 2A 83 C1    4 bytes UE3 package signature (magic)
00 02 00 00    4 bytes Block size
00 00 0b 36    4 bytes Compressed size
00 00 80 00    4 bytes Uncompressed size

// FCompressedBlockHeader - 8 bytes
00 00 0b 36    4 bytes Compressed size
00 00 80 00    4 bytes Uncompressed size

The uncompressed image is a PF_DXT5, width 256, height 32
In fact, I have 0x8000 = 32.768 = 256 x 32 (uncompressed image size) * 4 (as each pixel is RGB and a alpha channel)

// Raw compressed data
From compressed size, 0x0b36 = 2.870 bytes

In the previous image, I selected 2.870 bytes from the end of the Texture2D file. They are the bytes in blue.

So, this is what I have until now.

But... There is 8 unknow bytes just between FCompressedBlockHeader and Raw compressed data: 02 01 00 00 , 00 00 60 00. Do you know what could be this data?

And... the raw data is LZO1X_1 compressed. What I have to do to extract this data? I have to take it and add a lzo header to extract with lzop or another lzo extractor?

Your help is appreciated. Thanks!

  




Logged
Gildor
Administrator
Hero Member
*****
Posts: 6843



View Profile WWW
« Reply #5 on: October 02, 2018, 18:14 »

It's hard for me to operate "offsets" and "data structures". These terms are usually not applicable to Unreal data.
Please take a look at UTexture2D::LoadBulkTexture(), this function performs loading of compressed texture. As a final action it calls FByteBulkData::SerializeData().
Logged
cfarl
Newbie
*
Posts: 7


View Profile
« Reply #6 on: October 02, 2018, 19:59 »

New advances!

There is no 8 unknow bytes. This bytes are part of the raw data.

So, using the following bms script I could extract the compressed data:

endian big
comtype lzo1x
get magic long
get lixo long
get ZSIZE long
get SIZE long

print "Value of magic: %magic|x%"
print "Compressed size: %ZSIZE|x%"
print "Uncompressed size: %SIZE|x%"

get trash long
get trash long

clog "extracted.txt" 0x18 ZSIZE SIZE

Result:

offset filesize filename
--------------------------------------
Value of magic: 0x9e2a83c1
Compressed size: 0x00000b36
Uncompressed size: 0x00008000
00000018 32768 extracted.txt

- 1 files found in 0 seconds
coverage file 0 99% 2894 2902 . offset 00000018
Logged
cfarl
Newbie
*
Posts: 7


View Profile
« Reply #7 on: October 02, 2018, 20:05 »

Now... I have this extracted data....

It seems to be the DDS data of my 256 x 32 DXT5 image.

https://drive.google.com/open?id=1U8WU5x37iyfZgfY80EUEObCH7LmHvrJZ

Can somebody help me to transform this data to open in some image viewer?
Logged
cfarl
Newbie
*
Posts: 7


View Profile
« Reply #8 on: October 02, 2018, 20:48 »

ok!

In the extracted file, from xbox, you have to do unswizzle and byte swap, and add a dds header.

If you are extracting from pc, just put a dds header.

That's all!  Wink

Thanks for your help, Gildor!
Logged
Gildor
Administrator
Hero Member
*****
Posts: 6843



View Profile WWW
« Reply #9 on: October 03, 2018, 00:19 »

If you need pure format descriptions, check this page (it has many links)
https://github.com/gildor2/UModel/wiki
Logged
Pages: [1] Print 
« previous next »
Jump to:  

Powered by SMF | SMF © 2006-2009, Simple Machines LLC
Leviathan design by Bloc | XHTML | CSS