March 28, 2024, 21:55
bigger smaller reset     1020px Wide width Full width Reset   * *

Gildor's Forums

  Homepage Facebook Read news on Twitter Youtube channel Github page
Welcome, Guest. Please login or register.
Did you miss your activation email?

« previous next »
Print
Author Topic: Lost Ark  (Read 63088 times)
1212aaa
Newbie
*
Posts: 24


View Profile
« on: August 04, 2016, 06:18 »



http://lostark.co.kr/
Download : http://www.lostark.co.kr/download

Special UModel build: see this message
« Last Edit: March 12, 2022, 10:06 by Gildor » Logged
h4x0r
Newbie
*
Posts: 24



View Profile WWW
« Reply #1 on: December 22, 2018, 04:22 »

Some information about packages.

Engine version
Ver: 868/16 Engine: 12097 Game: 80004C

In 90% of cases after the magics id 0xC1832A9E data blocks are encrypted by AES. Only 4096 bytes in blocks are encrypted. Key for decryption is: V1ZEG1PL34V77SQW39A9I4VUW34T6L15

Compression flag (offset 0x6D) always equals 0x44 (this is LZ4) > The game defines it so
Code:
CompressionFlag = (CompressionFlag & 0xF); // = 4

I tried to add support in UModel and other utilities such as extract / decompress. Added next lines in code:

Code:
appDecryptAES(CompressedBuffer, 4096, "V1ZEG1PL34V77SQW39A9I4VUW34T6L15", 32);
Flags = (Flags & 0xF);
if(Flags == 4)
{
int newLen = LZ4_decompress_safe((const char*)CompressedBuffer, (char*)UncompressedBuffer, CompressedSize, UncompressedSize);
if (newLen <= 0)
appError("LZ4_decompress_safe returned %d\n", newLen);
if (newLen != UncompressedSize) appError("lz4 len mismatch: %d != %d", newLen, UncompressedSize);
return newLen;
}

Here is a log when decompressing a small file after compiling:

Code:
Detected game root Z:/Games/Smilegate/Games/LOSTARK/EFGame/ReleasePC (no recurse)
Found 12 game files (0 skipped) at path "Z:/Games/Smilegate/Games/LOSTARK/EFGame/ReleasePC"
Loading package: 2VD9SGDRPE2H7TOC9BLMAP.u Ver: 868/16 Engine: 12097 Names: 543 Exports: 769 Imports: 58 Game: 80004C
2VD9SGDRPE2H7TOC9BLMAP.u: uncompressed size 123962
******** 2VD9SGDRPE2H7TOC9BLMAP.u ********
*** WARNING: wrong size of 2VD9SGDRPE2H7TOC9BLMAP.u: differs in 4 bytes

In fact, it was unpacked without problems, but it refuses to work correctly with other files like upk. Someone else's code for me is like a dark forest. So it would be great if Gildor added support from himself. I did not manage to do this to the end. The main packages loader in EFEngine.dll (library is not encrypted), if this interests the author of course Smiley

Here samples and executable (just in case made a dump and the main executable): Mirror 1 or Mirror 2
« Last Edit: December 22, 2018, 16:45 by h4x0r » Logged
Gildor
Administrator
Hero Member
*****
Posts: 7978



View Profile WWW
« Reply #2 on: December 22, 2018, 09:21 »

Thanks for description.

... it would be great if Gildor added support from himself.
I'm afraid this won't happen. I'm too busy with other things - family, work, and when I'll have time for umodel I'll spend time on something else, not about the "Game A" or "Game B", but rather on something general.
Logged
spiritovod
Global Moderator
Hero Member
*****
Posts: 1901


View Profile
« Reply #3 on: April 24, 2021, 21:09 »

It seems non-asian versions of the game didn't change since that post (except the encryption key). I've managed to implement changes, explained by h4x0r (Ekey), but it seems the upk structure changed a lot since CBT1, so I can parse import table at best in the latest game version. Unless someone is willing to provide decrypted CBT1 client for comparison, I doubt something else can be done.

Also, the game is suffering from floating bug in stable lz4 versions prior to 1.9.3, when lz4_decompress_safe could fail due to non-standard literals at the end of compressed chunks (expected 5 zero bytes, but could be 6 upon compression). Updating to 1.9.3 resolved most of decompression issues.


* Clipboard04.jpg (132.2 KB, 1627x1142 - viewed 1027 times.)
« Last Edit: April 24, 2021, 21:14 by spiritovod » Logged
CriticalError
Full Member
***
Posts: 172



View Profile
« Reply #4 on: June 22, 2021, 06:30 »

Hello guys somebody can unpack upk from the latest version posted? it not work and trying figure out guide with LAUTool_0.2 but this is for wem files no for upk or lpk so any news of tool to unpack that awesome game?
Logged
spiritovod
Global Moderator
Hero Member
*****
Posts: 1901


View Profile
« Reply #5 on: June 22, 2021, 18:00 »

Decrypted archives will be of no good anyway. Aside from changed properties, the game still has that issues with decompression, which I've mentioned in my post above. And because of this or modified chunks structure buffer positioning is also broken - you can build up a buffer with 1, 2 chunks and read from it, and then it may through you somewhere in the middle of actual file instead of buffer - i.e. it's not even possible to navigate within properly decrypted and decompress chunks. I was able to load static mesh from 1archive-1mesh though...
Logged
Brouznouf
Newbie
*
Posts: 4


View Profile
« Reply #6 on: September 07, 2021, 17:33 »

I manage to extract most of the textures of the Korean version by using what h4x0r and spiritovod pointed out, and also applying other fixes.

There seems to be no problem with lz4 decompression (but i don't use the same librairies), however compressed chunk structure in header has an extra uint 32 variable, if it equals 1 then the chunk should be decompressed using AES + LZ4 like h4x0r said, if it equals 0 it means there is no compression nor crypted data.

Also the int property of object happens to be on a 12 size, rather than 4 (certainly a 64 bit integer and a flags to know between signed / unsigned / ...), there may also be an extra 8 bytes on byte property.

For object decompression inside the file, often the flags is '0x80' which equals to LZX in ueviewer where it's in fact LZ4 in Lost Ark, and there is no aes crypted buffer there.

FYI i have extracted this in a 2019 version of lost ark so there may be update since then, but i don't think so, also if someone has key for the west or RU version i may verifiy those findings in current version.

« Last Edit: September 07, 2021, 17:37 by Brouznouf » Logged
spiritovod
Global Moderator
Hero Member
*****
Posts: 1901


View Profile
« Reply #7 on: September 07, 2021, 21:19 »

@Brouznouf: Thanks for the info, it makes sense now why navigation inside buffer was broken like that. Last time I've checked ru version (it was easier to obtain), the key was "EAI643MZ91DG7J5K34A3RRD9FJ8QU6M2" - feel free to check it with the latest updates.
Logged
Brouznouf
Newbie
*
Posts: 4


View Profile
« Reply #8 on: September 08, 2021, 01:17 »

Thanks, it still works the same way with the RU version, i'm able to export the majority of textures, there is still some bugs with somes fils, like some name table with bad data inside, but maybe it's due to my decompress library.

I have also updated ueviewer to test some mesh, it still crash on a lot of files (don't know why, it instantly exit), but it works for some of them, here is a patch if somebody wants to try it out: https://pastebin.com/4n8g5hWy
Logged
Brouznouf
Newbie
*
Posts: 4


View Profile
« Reply #9 on: September 08, 2021, 22:54 »

With this patch UEviewer can certainly extract some sk or sm, but not all as there is still bugs and i don't have a lot of knowledge on upk format and game resources.

The "tool" (more a garbage script) that i wrote to extract some textures (map level) that i need to build an interactive map, and does not do anything else.

However with those findings i think there is all the necessary materials to write a tool that extract everything from lost ark if someone want to give a go.
« Last Edit: September 08, 2021, 22:56 by Brouznouf » Logged
jericho777
Newbie
*
Posts: 20


View Profile
« Reply #10 on: September 09, 2021, 22:01 »

hmm its weird, my kr client wont even extract any file even texture. which patch ueviewer are you using?
Logged
spiritovod
Global Moderator
Hero Member
*****
Posts: 1901


View Profile
« Reply #11 on: September 11, 2021, 05:05 »

Thanks to Brouznouf internal navigation was fixed, so here are specific build for the game (download, link updated) - you need to use game specific override option and choose respective region at startup screen. If you're a newcomer, check this post before launching the build. Skeletal/static meshes and textures should work properly, animations are supported only in latest specific build.
 
Current build supports steam, ru (la[.]mail[.]ru), jp (lostark[.]pmang[.]jp), tw (lostark[.]mangot5[.]com), kr (lostark[.]game[.]onstove[.]com) and cn (lostark[.]qq[.]com) clients. It's recommended to rename ReleasePC\LOD9*.upk to textures.tfc for proper support of some textures. Quickbms script for lpk and tool for audio are available here. Solution for correct packages naming is available in this post.

Update: Specific build updated with support for crunch compression, so most textures should work even in latest game version.
Update 2: Small fix added to specific build for better memory handling.
Update 3: All crashes related to textures should be fixed in the updated build.
Update 4: Separate build for steam client added to the archive. Some animations may occasionally work, but most of them are using ACL compression as well.
Update 5: Both builds are updated to avoid crash on some embedded tfc textures.
Update 6: Separate build for jp client added to the archive.

Update 7: ACL animations are now supported in v2 builds. Though there may be some issues with non-acl animations.
Update 8: A few fixes added to improve animations support. Now empty animations without actual data are properly skipped.
Update 9: Some improvements for animations added. Now properties like rate and frame count are taken directly from tracks properties for ACL animations and artificially calculated for non-acl ones.

Update 10: Builds are now merged into single build, you can set required region at startup screen (at bottom right). Support for tw version is added as well.
Update 11: Support for kr version is added, it would also work for some early versions from 2018-2019.
Update 12: Support for cn version is added. The build is now 64-bit version and you need to use SDL2_64.dll from official umodel.
Update 13: Export for morphs is added (for gltf output only).
Update 14: Fix for some textures is added. Now fallback is used for cases when texture is flagged as crunched, while it's actually not.


* Clipboard10.jpg (111.46 KB, 2539x1232 - viewed 1682 times.)

* Clipboard11.jpg (124.11 KB, 2552x1243 - viewed 2242 times.)
« Last Edit: December 11, 2023, 22:43 by spiritovod » Logged
GDL
Jr. Member
**
Posts: 64


View Profile
« Reply #12 on: September 12, 2021, 19:25 »

I downloaded the latest RU client from the official page and most of the models and textures show up like this, not sure what's wrong.  Undecided



* lost.jpg (56.58 KB, 802x652 - viewed 697 times.)
Logged
Ehlyon
Full Member
***
Posts: 186



View Profile
« Reply #13 on: September 12, 2021, 19:27 »

I downloaded the latest RU client from the official page and most of the models and textures show up like this, not sure what's wrong.  Undecided

I guess the answer lies here.

Please note that the build is developed for ru client, so it may be incompatible with clients from other regions. Also, it's valid for versions prior to august 2021 update, where they've started to use crunch compression for textures, unsupported by umodel.
Logged
spiritovod
Global Moderator
Hero Member
*****
Posts: 1901


View Profile
« Reply #14 on: September 13, 2021, 06:53 »

Thanks to Gildor's advices specific build updated with support for crunch compression for textures. Though due to poor memory management in the initial implementation it may crash here and there eventually - so please do not report hard crashes (like access violation errors). Also, some textures may not work as expected, I'm aware of it as well. It's just all this stuff is very time consuming to test.

About animations - they're using ACL compression now, which is not supported even for UE4 games, so...
Logged
Print 
« previous next »
Jump to:  

Powered by SMF | SMF © 2006-2009, Simple Machines LLC
Leviathan design by Bloc | XHTML | CSS