Compression

Suuk

Newbie

Posts: 5

Compression

« on: May 16, 2009, 18:53 »

Hello guys

Well we know that more and more Ut3 Engine games are procompressed with different compression schemes like gzip,lzo or lzx and we have still no general decompresser for those containers because the structure can vary from game to game.

Anyway why not start to get a few compression informations together.
I start now with Lzo. The lzo algorithm is always lzo1x .The LZO segments have evertime the magic ending keys 11 00 00.
The Lzo stream beginning is headerless. Decoding possible by hand but takes ages because you have many lzo segments in a file and every game has a little different container structure. Maybe a lzo scanner is possible - scan first for the ending key and then via brureforce for the correct beginning (theory).

Here are lzo files from 5 different games + lzo start/end offsets
http://www.megaupload.com/?d=AZSQ1PK6

Have also added a Gears of war 2 file to the pack - Gears of war 2 use lzx - dont know how we can decompress this one.

Any other hints?


	Logged

Gildor

Administrator
Hero Member

Posts: 7971

Re: Compression

« Reply #1 on: May 16, 2009, 20:58 »

Umodel can decompress packages with all UE3 compression schemes: ZLib, LZO and LZX (XBox360) without searching for signatures or other hacks - my program knows package format


	Logged

Suuk

Newbie

Posts: 5

Re: Compression

« Reply #2 on: May 16, 2009, 22:31 »

Hey great but can you add other formats to your tool too like audio files. If you are able to decompress all those files then you could develop an extractor for ut3 engine games in general. I think that would be really great.

Anyway my post is wrong here - better you move it to miscellaneous


« Last Edit: May 16, 2009, 22:36 by Suuk »	Logged

Gildor

Administrator
Hero Member

Posts: 7971

Re: Compression

« Reply #3 on: May 16, 2009, 23:08 »

UE3 uses a few audio formats: ogg, fmod, xaudio (xbox360) ... possibly others. It is not complex to extract OGG files, but what should I do with other formats? Xaudio can be decoded with XDK tools, but these tools are not public.
I have created a small tool named "unreal package extractor" (it has no own page, posted here). I'm periodically update this tool to support new package formats. It's not complex to extract ogg files from the data generated by this tool.


	Logged

Suuk

Newbie

Posts: 5

Re: Compression

« Reply #4 on: May 17, 2009, 16:09 »

Wow thx a lot for this extractor - have never known that the ut3 engine packs are completely extractable.
Really a great work

Yeah the main problem are these crap compression schemes. File extracting is not a problem for me -
By the way can you add support for stranglehold? - Its an older Ut3 engine game but it use also Lzo (x360 version), dont know how the pc version is compressed. A file is also inside the megaupload pack.


	Logged

Gildor

Administrator
Hero Member

Posts: 7971

Re: Compression

« Reply #5 on: May 19, 2009, 12:24 »

Package compression is used for the following reasons:

faster file loading (it's faster to decompress file than load uncompressed file from DVD)
smaller file sizes - more content on a single media

There is no reason to compress files just to prevent data extraction.


	Logged

Suuk

Newbie

Posts: 5

Re: Compression

« Reply #6 on: May 21, 2009, 18:22 »

Yep right but why then headerless? - anyway thx for your work


« Last Edit: May 22, 2009, 02:54 by Suuk »	Logged

Gildor

Administrator
Hero Member

Posts: 7971

Re: Compression

« Reply #7 on: May 21, 2009, 22:29 »

What you mean - "headerless"?


	Logged

Suuk

Newbie

Posts: 5

Re: Compression

« Reply #8 on: May 22, 2009, 02:59 »

My mistake gildor. Have thought that Lzo has some kind of a beginning ID or a header like gzip but it looks like that lzo is in general headerless.


	Logged

Pages: [1]

Gildor's Forums

Gildor's Forums