Gildor's Forums

Author Topic: Lost Ark  (Read 41239 times)
1212aaa
Newbie
*
Posts: 24


View Profile
Lost Ark
« on: August 04, 2016, 06:18 »



http://lostark.co.kr/
Download : http://www.lostark.co.kr/download

Special UModel build: see this message
« Last Edit: March 12, 2022, 10:06 by Gildor » Logged
h4x0r
Newbie
*
Posts: 24



View Profile WWW
Re: Lost Ark CBT
« Reply #1 on: December 22, 2018, 04:22 »

Some information about packages.

Engine version
Ver: 868/16 Engine: 12097 Game: 80004C

In 90% of cases after the magics id 0xC1832A9E data blocks are encrypted by AES. Only 4096 bytes in blocks are encrypted. Key for decryption is: V1ZEG1PL34V77SQW39A9I4VUW34T6L15

Compression flag (offset 0x6D) always equals 0x44 (this is LZ4) > The game defines it so
Code:
CompressionFlag = (CompressionFlag & 0xF); // = 4

I tried to add support in UModel and other utilities such as extract / decompress. Added next lines in code:

Code:
appDecryptAES(CompressedBuffer, 4096, "V1ZEG1PL34V77SQW39A9I4VUW34T6L15", 32);
Flags = (Flags & 0xF);
if(Flags == 4)
{
int newLen = LZ4_decompress_safe((const char*)CompressedBuffer, (char*)UncompressedBuffer, CompressedSize, UncompressedSize);
if (newLen <= 0)
appError("LZ4_decompress_safe returned %d\n", newLen);
if (newLen != UncompressedSize) appError("lz4 len mismatch: %d != %d", newLen, UncompressedSize);
return newLen;
}

Here is a log when decompressing a small file after compiling:

Code:
Detected game root Z:/Games/Smilegate/Games/LOSTARK/EFGame/ReleasePC (no recurse)
Found 12 game files (0 skipped) at path "Z:/Games/Smilegate/Games/LOSTARK/EFGame/ReleasePC"
Loading package: 2VD9SGDRPE2H7TOC9BLMAP.u Ver: 868/16 Engine: 12097 Names: 543 Exports: 769 Imports: 58 Game: 80004C
2VD9SGDRPE2H7TOC9BLMAP.u: uncompressed size 123962
******** 2VD9SGDRPE2H7TOC9BLMAP.u ********
*** WARNING: wrong size of 2VD9SGDRPE2H7TOC9BLMAP.u: differs in 4 bytes

In fact, it was unpacked without problems, but it refuses to work correctly with other files like upk. Someone else's code for me is like a dark forest. So it would be great if Gildor added support from himself. I did not manage to do this to the end. The main packages loader in EFEngine.dll (library is not encrypted), if this interests the author of course Smiley

Here samples and executable (just in case made a dump and the main executable): Mirror 1 or Mirror 2
« Last Edit: December 22, 2018, 16:45 by h4x0r » Logged
Gildor
Administrator
Hero Member
*****
Posts: 7962



View Profile WWW
Re: Lost Ark CBT
« Reply #2 on: December 22, 2018, 09:21 »

Thanks for description.

... it would be great if Gildor added support from himself.
I'm afraid this won't happen. I'm too busy with other things - family, work, and when I'll have time for umodel I'll spend time on something else, not about the "Game A" or "Game B", but rather on something general.
Logged
spiritovod
Hero Member
*****
Posts: 1053


View Profile
Re: Lost Ark
« Reply #3 on: April 24, 2021, 21:09 »

It seems non-asian versions of the game didn't change since that post (except the encryption key). I've managed to implement changes, explained by h4x0r (Ekey), but it seems the upk structure changed a lot since CBT1, so I can parse import table at best in the latest game version. Unless someone is willing to provide decrypted CBT1 client for comparison, I doubt something else can be done.

Also, the game is suffering from floating bug in stable lz4 versions prior to 1.9.3, when lz4_decompress_safe could fail due to non-standard literals at the end of compressed chunks (expected 5 zero bytes, but could be 6 upon compression). Updating to 1.9.3 resolved most of decompression issues.


* Clipboard04.jpg (132.2 KB, 1627x1142 - viewed 604 times.)
« Last Edit: April 24, 2021, 21:14 by spiritovod » Logged
CriticalError
Full Member
***
Posts: 171



View Profile
Re: Lost Ark
« Reply #4 on: June 22, 2021, 06:30 »

Hello guys somebody can unpack upk from the latest version posted? it not work and trying figure out guide with LAUTool_0.2 but this is for wem files no for upk or lpk so any news of tool to unpack that awesome game?
Logged
spiritovod
Hero Member
*****
Posts: 1053


View Profile
Re: Lost Ark
« Reply #5 on: June 22, 2021, 18:00 »

Decrypted archives will be of no good anyway. Aside from changed properties, the game still has that issues with decompression, which I've mentioned in my post above. And because of this or modified chunks structure buffer positioning is also broken - you can build up a buffer with 1, 2 chunks and read from it, and then it may through you somewhere in the middle of actual file instead of buffer - i.e. it's not even possible to navigate within properly decrypted and decompress chunks. I was able to load static mesh from 1archive-1mesh though...
Logged
Brouznouf
Newbie
*
Posts: 4


View Profile
Re: Lost Ark
« Reply #6 on: September 07, 2021, 17:33 »

I manage to extract most of the textures of the Korean version by using what h4x0r and spiritovod pointed out, and also applying other fixes.

There seems to be no problem with lz4 decompression (but i don't use the same librairies), however compressed chunk structure in header has an extra uint 32 variable, if it equals 1 then the chunk should be decompressed using AES + LZ4 like h4x0r said, if it equals 0 it means there is no compression nor crypted data.

Also the int property of object happens to be on a 12 size, rather than 4 (certainly a 64 bit integer and a flags to know between signed / unsigned / ...), there may also be an extra 8 bytes on byte property.

For object decompression inside the file, often the flags is '0x80' which equals to LZX in ueviewer where it's in fact LZ4 in Lost Ark, and there is no aes crypted buffer there.

FYI i have extracted this in a 2019 version of lost ark so there may be update since then, but i don't think so, also if someone has key for the west or RU version i may verifiy those findings in current version.

« Last Edit: September 07, 2021, 17:37 by Brouznouf » Logged
spiritovod
Hero Member
*****
Posts: 1053


View Profile
Re: Lost Ark
« Reply #7 on: September 07, 2021, 21:19 »

@Brouznouf: Thanks for the info, it makes sense now why navigation inside buffer was broken like that. Last time I've checked ru version (it was easier to obtain), the key was "EAI643MZ91DG7J5K34A3RRD9FJ8QU6M2" - feel free to check it with the latest updates.
Logged
Brouznouf
Newbie
*
Posts: 4


View Profile
Re: Lost Ark
« Reply #8 on: September 08, 2021, 01:17 »

Thanks, it still works the same way with the RU version, i'm able to export the majority of textures, there is still some bugs with somes fils, like some name table with bad data inside, but maybe it's due to my decompress library.

I have also updated ueviewer to test some mesh, it still crash on a lot of files (don't know why, it instantly exit), but it works for some of them, here is a patch if somebody wants to try it out: https://pastebin.com/4n8g5hWy
Logged
Brouznouf
Newbie
*
Posts: 4


View Profile
Re: Lost Ark
« Reply #9 on: September 08, 2021, 22:54 »

With this patch UEviewer can certainly extract some sk or sm, but not all as there is still bugs and i don't have a lot of knowledge on upk format and game resources.

The "tool" (more a garbage script) that i wrote to extract some textures (map level) that i need to build an interactive map, and does not do anything else.

However with those findings i think there is all the necessary materials to write a tool that extract everything from lost ark if someone want to give a go.
« Last Edit: September 08, 2021, 22:56 by Brouznouf » Logged
jericho777
Newbie
*
Posts: 17


View Profile
Re: Lost Ark
« Reply #10 on: September 09, 2021, 22:01 »

hmm its weird, my kr client wont even extract any file even texture. which patch ueviewer are you using?
Logged
spiritovod
Hero Member
*****
Posts: 1053


View Profile
Re: Lost Ark
« Reply #11 on: September 11, 2021, 05:05 »

Thanks to Brouznouf internal navigation was fixed, so here is specific build for the game (download, link updated) - you need to use game specific override option at startup screen. Skeletal/static meshes and textures should work properly, animations are not supported due to ACL compression, some materials are not supported as well (you don't need to disable them though, but in some cases you need to disable animations).
 
There are three builds for ru (la[.]mail[.]ru), steam and jp (lostark[.]pmang[.]jp) clients in the provided archive now. Other versions are not supported at the moment. It's recommended to rename ReleasePC\LOD9*.upk to textures.tfc for proper support of effects textures.

Update: Specific build updated with support for crunch compression, so most textures should work even in latest game version. Note that it's more like test build and it may eventually crash here and there, especially when used on PCs with low amount of RAM.

Update 2: Small fix added to specific build for better memory handling.

Update 3: All crashes related to textures should be fixed in the updated build (fix2 version).

Update 4: Separate build for steam client added to the archive. Some animations may occasionally work, but most of them are using ACL compression as well.

Update 5: Both builds are updated to avoid crash on some embedded tfc textures.

Update 6: Separate build for jp client added to the archive.


* Clipboard10.jpg (111.46 KB, 2539x1232 - viewed 814 times.)

* Clipboard11.jpg (124.11 KB, 2552x1243 - viewed 1021 times.)
« Last Edit: July 11, 2022, 12:05 by spiritovod » Logged
GDL
Newbie
*
Posts: 34


View Profile
Re: Lost Ark
« Reply #12 on: September 12, 2021, 19:25 »

I downloaded the latest RU client from the official page and most of the models and textures show up like this, not sure what's wrong.  Undecided



* lost.jpg (56.58 KB, 802x652 - viewed 328 times.)
Logged
Ehlyon
Full Member
***
Posts: 153



View Profile
Re: Lost Ark
« Reply #13 on: September 12, 2021, 19:27 »

I downloaded the latest RU client from the official page and most of the models and textures show up like this, not sure what's wrong.  Undecided

I guess the answer lies here.

Please note that the build is developed for ru client, so it may be incompatible with clients from other regions. Also, it's valid for versions prior to august 2021 update, where they've started to use crunch compression for textures, unsupported by umodel.
Logged
spiritovod
Hero Member
*****
Posts: 1053


View Profile
Re: Lost Ark
« Reply #14 on: September 13, 2021, 06:53 »

Thanks to Gildor's advices specific build updated with support for crunch compression for textures. Though due to poor memory management in the initial implementation it may crash here and there eventually - so please do not report hard crashes (like access violation errors). Also, some textures may not work as expected, I'm aware of it as well. It's just all this stuff is very time consuming to test.

About animations - they're using ACL compression now, which is not supported even for UE4 games, so...
Logged
Jump to:  

Powered by SMF | SMF © 2006-2009, Simple Machines LLC
Leviathan design by Bloc | XHTML | CSS