Gildor's Forums

Splinter cell conviction support umodel? ???

Unlikely right out of the box, Conviction uses a heavily modified UE2.5 engine called LEAD that ubisoft designed themselves.

Gildor will most likely have to add support if he can be bothered ;)

No I have a copy of it and none of the files are extracting. I believe Gildor could possibly get everything except animations to work.

I hope so!

I don't want to burst your bubbles, but I saw the demo a while back, and I say no way... It doesn't use Epic package format, and it massively different. Maybe you might have luck with someone else. Gildor only deals with unreal package formats. I doubt the game is different than the demo. What can I say UBI is not my favorite developer for these reasons. :-X

..:: ESR911 ::..

But when you start the game out unreal technology.

That doesn't mean anything. It only means that they are licensees of Epic's unreal and that this game uses the Epic's unreal engine. They (UBI) can then change everything any way they want with the engine.

I am 99.99% sure it's not going to be supported by umodel, because if it was even possible it would require a massive commitment of Gildor's time for just one game, and it isn't worth it. Gildor only has so much time. Also there have been donations, but It is likely that extraction of assets of this game may not even be possible for Gildor at any price.

Again this is my opinion. If you feel so inclined you are welcome to provide the files Gildor needs, and he can weigh in.

..:: ESR911 ::..

Ubisoft almost always modifies the Unreal engine to suit their needs. Very rarely do they keep the packages in the standard format, or not modify the general compression in some way or another.

So in short, don't hold your breath, it might be possible, it might not.

Quiet Man, I do not create any disengaged, it was just curiosity.
Of all the ways sorry if I offended you!
A greeting and thanks!

Convictions files for Gildor. If Gildor can do it, these are the files he needs.

http://www.sendspace.com/file/k45dat

..:: ESR911 ::..

This modern Ubisoft game uses heavily modified Unreal Engine 2 with custom package format.

Translate this!
Its a good news or bad?
 ::)

I will repeat my message in another words:
Good news: this is an Unreal Engine game.
Bad news: it uses heavily modified engine, its package format is either heavily modified or completely different
;D

Sorry! Why so serious!? ;D

What? he's just telling you that it can't be done or it be a waste of time to do. You should of probably understood that from his first post  :P

I wanna asking Gildor about this game, maybe have some progress, because it's very close to double agent? Models in the game a very-very quality.

No. Progress was stopped a few months ago. I can decompress ass/umd files, but these files really a some kind of file container - it contains other files inside. Umodel has no "virtual file system" to process packages nested inside other files.

You are wrong (but to be correct, I have not seen double agent packages - it looks like that packages were placed in archives too - that's why you have not seen them).

Sorry I am so late to this game! You are correct.
We will release our research with this soon!

Where you able to fully decompress the ZLIB data in Conviction? If so, I might be able to figure out their library system - but I could not figure out how to decompress the data (only was successful with one small chunk of .umd file which contained .ini file)

Conviction uses Microsoft compression library for XBox360 (XMemCompress, wrapper around LZX), not zlib. It even uses big-endian data storage. I don't know why they're used XBox library for PC, but it's a fact. They should link PC game executable with XDK ...

Question...when you decompressed this data was there Unreal file headers anywhere to be found throughout the container file?
Also, what method did you use to decompress the files?

Yes.
It uses ZLib and LZX ("XMem").

Hopefully I am not asking too much, but can you please send me an uncompressed archive file?
I would like to look through it to see if it matches other similar Ubisoft archive file containers.

Please remind me in a 3-4 days.

Will do. Thanks :)

Just a friendly reminder. :)

Sent you a PM with the file link. Sorry for delay - I have no free time at all :(

Thanks - I just downloaded it!

I know what it's like to not have free time, I'm short on time too! If only there were more hours in the day ... :P

I will look at this archive soon. .UMD is a common extension Ubisoft has been using for UMOD similar file storage since Chaos Theory. I believe Xbox and Playstation .UMD files have been different for Chaos Theory (I have to check). Double Agent uses LIN localization.

I will look at this and see if I see either formats or something new (though from a quick glance I didn't see any Unreal headers). Maybe when you have time you can explain to me step by step how you exported this content from Conivction.

Hey Gildor,

Okay I had a look at the file. A few questions:

-Where do you see Unreal headers? I searched through the file for declaration of Unreal and engine version and didn't see any consistency. What did you mean when you said you saw Unreal file headers?

-I see what seems to be a layout of files used in a modified UMOD format declared at the beginning of the file, but the contents of the rest of the package seem to still be compressed. It's modified UMD format from what Ubisoft normally has done (in the past).
Can you please provide me with exact method/tools used to extract the contents from .ass/.umd files? This way I can extract all packages without needing to bother you.  :)

I would especially like to extract "loc-int.umd" files as they should be plain English and prove that the extraction/deflate method is correct. I was able to extract some contents from it before, but only a few bytes (correctly).

Would be helpful if you could provide me with this information. :)
Also, Just a thought -  I noticed that the file you sent me was smaller (by about 16MB) from .ass/.umd file. If it was compressed before, shouldn't the extract file be larger??

Hi
Probably I've made a mistake. All contents is encrypted (but most probably with a simple xor or similar).
I can send you a tool which will save decompressed umd. But currently this tool is slightly buggy - it will crash on some files.I've stopped Conviction reversing when I've found that ass/umd is an archive file.
Original 01_Market.umd is 31,299,592 bytes long. Decompressed is 107Mb. Rar (say: recompressed) file is 26 Mb. What's wrong?

-In response to your tool, Please and thank you. :) I can extract archives from previous .UMD game archives uses by Ubisoft and it does in fact contain original Unreal files. Though conviction .umd does in fact appear to be encrypted/compressed further.

-oops. I thought your extracted .umd file was BOTH .umd AND .ass files. I am now assuming that .ass is container for map files (as only map names contain .ass and .umd).

If you can send me the tool, I will analyze the archives further.  :P

Sent you a PM with a file and information.

I have reversed UMD file format. Facts:

• it is really uses simple encryption - it uses single XOR for all bytes in the file
• it is not an archive, UMD is a recorded file system access
  
• UMD contains only those parts of files which are read when loading some level

Benefits of such system:

• seek-free level loading, mostly for loading from the CD/DVD media (much faster than loading from the original game packages)
• UDM file contains only required file parts, all unused resources are not included into the file
  
  Recovering data from such files is very complex task, but probably I will try to do it later. This should be done in a few steps:
  1) decompress UMD files
  2) produce unreal packages from the decompressed UMD files
  3) process packages to find holes in data (resources which are not included into the game) and patch export table to hide them (to prevent umodel from crash)
  4) use umodel on the resulting packages

That makes sense. They probably just stripped out the data from the packages after the fact once they figured out what data is not used at runtime?
Did you find the encryption key in the .exe?

Would it be possible for you to send me the method (or compiled version of your tool) that unencrypted the UMD data files?
I would like to anaylze them. The game appears to have the ability to run with original data support - non umod)
I would like to see this game become mod-able.

Your help is greatly appreciated.  :)

No. UMD consists of parts of the files in pseudo-random order.
No. But I've found it in the 1st try.
It is not possible to post method - file format is too complex.
Game engine supports UMD transparently. Higher level systems are even know nothing about UMD. It works with resources exactly in the same way as with packages.

It is not possible to post method - file format is too complex.

Can I have your updated tool that un-encrypts so that I can investigate the file format myself? :)

Can you provide me with what the encryption key is? Is it "Conviction"??  ;)

later
No. It is hex B7 :) All bytes in the file were xor-ed with the same key.

Information update.
I have finished reversing umd/ass format. It looks like .ass contains textures (similar to .tfc files). This file has no headers inside - only pure data.

I wonder how the game knows what texture is where without header/name table? Or are you just referring to no Áƒ*žd.Z. ?
Also, the .umd contained the .unr map file as well?


?? I am confused... the key was not B7? ...????
I am itching to get to see the archive format after decompression/ un-encrypted  ;)

Just wanted to check in on a status update.

There is no updates yet. I've been busy with my job.

Just wanted to bump this. I've been really busy and haven't spent much time with the format.

Hopefully you've had more luck than me.  :P

I have no time for tasks of such complexity :-\

Hi Gildor, Sorry for the bump but it seems that you were the only person on the internet who was interested in reversing Conviction. I am a huge fan of conviction and mainly like to modify textures and such. I just wanted to know if you managed to reverse any files on conviction? If so if you had a tool that could help me do such task?
Many Thanks :)

I probably didn't make it clear what I was trying to do, Im not so much getting the 3D models, but the raw textures in DDS? to make higher resolution. I heard that there were some config files encrypted that changed the behaviour of the game and the AI and that I would love to do.

Thanks again. :) :)

Hi Lenners. I'd rather speak of Conviction modding as of impossible task.

That's no good :( Anyway why do you think they 'encrypted' the files with some impossible-to-break encryption? Most games have the files in some zip-like container that just involves renaming the extension compared to Conviction. All i wanted to so was to extract the files, decompress all the games files.

I think textures are stored in .ass files. I reversed .umd file format, but know nothing about .ass.

Reading around, it appears that .ASS and .UMD are similar formats but I can't say much about that as I dont have much experience in this type of thing. I used offzip on Conviction.ass and it came back with a .dat which I have no idea how to use.

I'd give my right kidney just to for a method of extracting any of the resource files. It all seems so complex. The only mention of .ASS files anywhere is 'ASS I/O' a file compression software for compressing 'multiple' files together. I fired that up but there was no reversing option. This game is also similar to Ubisofts 'I Am Alive', both run on Unreal 2.5 and share the same folder hierarchy and file formats. On 'XeNTAx' (game research forum) people were having just as much trouble decompressing the UMD and ASS files. They were doing it for localization translation. I have no idea what Ubisofts problem is but they sure like there files encrypted  :(

UMD and ASS is some alternative to UE3's cooking process. That's quite easy to implement way to make seek-free level loading without rewritting full UE2 code.
All Ubisoft's UE games has enough modifications in comparison to previous game (for example, Splinter Cell 4 compared to Splinter Cell 3) so I'd rather suppose formats are quite different. UMD is a wrapper around Unreal's package format. It consists of file chunks, some directory (which helped me to restore original game packages), and they're compressed with XBox360's LZX library (even for PC version!). Encryption is very simple - just XOR'ed all the data with single value.
I've provided my decoder for UMD files to one of Brasilian's localizers and as I know he successfully localaized Splinter Cell: Conviction. But replacing texts and replacing textures are quite different tasks, especially for this game.
As I said, encryption is very simple. Everything else done for optimization.

It may be a big ask, but would you by any chance still have the UMD decoding tool? I could try running it on the UMDs which I assume have textures and configurations within. I have learnt from 'I Am Alive's file structure that the ass files contain mostly mapping data and some textures so I am assuming that the rest lay within the UMDS.
Thank you for all your help Gildor :) :)

P.S Im wanting access to the games configuration file to fix up some FPS bugs when playing multiplayer. When on matchmaking with a friend it locks FPS at 30 which is not playable but thats another story. :)  ;D

UMDs has Texture objects inside, but their mipmaps are stored somewhere else (I think inside .ass). When my tool restores utx files all textures has zeros instead of actual data.

After doing some more research it would appear that some textures for relevant models might be inside the UTX file format.

It wasnt so much the MipMaps I was looking for but the colour maps and specular maps that I wanted to change. At the moment I am using TexMod to do this for me but that requires a pirated version of the game when I own it. If UMD and ASS are similar it would seem they renamed the file extension to identify what both files contain rather than having seperately named files.

All of this optimization you mentioned must of been done for the XBox360's hardware. As this game was ported im guessing they left it all in as there was no advantage in 're-writing' the filesystem for PC.

If I can successfully extract any textures or models (or config files), (I am learning 3DS MAX) I would at least owe you a beer next time im in Russia :)  :P

Texture metadata are inside UTX, but texture data itself - not. They're stored separately.
They're not similar. They can't be similar.
Loading time optimization is important on all hardware.
I don't drink beer :)

OK, Thanks for all your help, it would seem impossible to achieve what I wanted so I will move on.  ::)

Aside from all the texture and model, where would things like game assets and configuration files be at? I have been reading the Conviction.ini and it mentions Engine.Console and Developer commands such as toggles for god mode and AI commands yet none of them seem to be bound. The game itself uses input key translation and has a list of keys and their relative translated listed but none of them seem to work. If the only files in the 'UMDS' directory are indeed UMD and ASS which contain their respective files where else could game assets be stored? I have been reading over 'iservealot's previous posts on the ubisoft forums and he mentions that the files would be encrypted, which to me makes little to no sense.

Tweaking with the main config about how to game should boot just doesn't seem to change.


Where there are mentions of debugs I can't find any relevant information in the other configs so they must be stored elsewhere.

Most configs are stored in UMDs.

You mentioned something about yourself being able to decode UMDS for localization? I would imagine that localization takes place with text files as do general configs?
Thanks,
Lenners

I mentioned that someone used my tool to do the localization. I never digged into game modifications area by myself, it's not interesting for me. By the way, when I restored original game files from UMDs I obtained .ini and .int files as well (don't remember exactly, did it more than 2 years ago).

That is very good news. I wasn't sure where the configs would be located. I would assume they would be location in Conviction.UMD because the others are map related (although there may be map-specific configs in those, im not too sure).  The other UMDS are for localization.
If I had a method and/or tool, I would test PEC-Main.umd, Menu.umd and conviction.umd for what I was after. Then again the one tool would decode all of the UMDS I would assume.
Many thanks,
Lenners

Would anyone be in possession of said tool?

I'd prefer to not share this tool for some reasons.

Ok I understand that. Is there a method of doing it instead? No matter how time consuming? It would mean the world to me if I could do this :)

Sorry?

Is there a way of decompressing the games original files without using your tool at all? You're the only one on the internet who seems to know or have interest in things like this.
Thanks,
Lenners

I don't know any alternatives. By the way, my interest to Conviction was stopped when I've found the complexity to extract anything from this game.

I guess your secret tool is the only thing that would do it. ANYWAY, i guess it is time for me to really move on. Patiently wait until blacklist comes out.
Thanks for all your help, Gildor.
Lenners

Hi,

Could you use QuickBMS to make an extractor? It can handle xbox360 format.
http://aluigi.altervista.org/quickbms.htm

Or explain how the UMD header works? I think I've found decompression method (01,02) and maybe where LZX data roughly is. But it's still confusing.

Thanks



edit1:
Successful decompress of loc-int.umd. QuickBMS script attached.

Don't understand header file though (locint-0.bin).

[вложение удалено Администратором]

I found a couple of things in regards to this
.umd extractor but the links are down http://www.se7ensins.com/forums/threads/umd-editor-rerelease.908/
apparently the older .umd files might be similar to umod files but I have had no luck with the UMOD extractor
http://www.oldunreal.com/cgi-bin/yabb2/YaBB.pl?num=1317591752

and then there is this guy's tool
http://forums.xbox-scene.com/index.php?showtopic=504469

all the links are dead, but if we could get the people to post the files again there might be hope that Gildor can figure this out

I believe Convictions has own UMD format, not the same like in older Ubisoft's games.

yeah I don't have my doubts that they are different, but is it possible that those extractors could be of some use to you? I saw earlier that you managed to open up the .umd/.ass files so you clearly have an understanding of this newer file.

I know UMD format for Convictions.

QuickBMS scripts created for SCC.
- Have to run 1 file at a time
- table.bms = some header file
- data.bms = actual UMD data unpacked

Doesn't work on every file but many. Still need to xor the data using gildor's key 0xB7.


xor start:
01_Market.umd = 0x20000
C05_Antonov.umd = 0x20000
Conviction.umd = 0x60000
loc-int.umd = not needed
menu.umd = 0x20000
PEC-Main.umd = 0x20000
T08_SecretLabs.umd = 0x80000


edit:
QuickBMS extractor fails on some files like dynamicxbox.umd. Because Ubisoft mixes LZX + Zlib.
Just letting people know for future.

Thanks gildor


dynamicxbox.umd
(0001)  1A4 - 593C  =  0 - 10000  (10000)  <-- ms lzx
..
(0011)  2EBF6 - 313F8  =  A0000 - B0000  (10000)  <-- ms lzx
(0012)  313F8 - 2043DA  =  B0000 - 2D0000  (220000) <-- this is Zlib compress, not ms lzx
(rest is ms lzx)


[вложение удалено Администратором]

Thank you for the scripts! Shame Gildor won't let his tool loose on the web, but I see his reasons. Any modding possibilities here?

I'm interested in translating the game. I managed to extract "loc-int.umd", using @spinbot script "scc_conviction_data.bms". After that I used I Am Alive "iaa_unpacker.exe" to extract INT files from "dump_scc_data.bin". Now the problem is that I do not know how to repack modified files back to a "loc-int.umd". If anyone could help me I would be very grateful.

just  Extract to Data/Localization without zipping back up and make sure  src/system/loc_int.umd is deleted

This is the method I used back when I translated "I Am Alive", but this time it doesn't work this way. The game just hangs.

hmm conviction's file structure might be different then... really odd as they are using the same engine... could also be different builds I will look into this a bit more. Also try posting in this forum http://forum.xentax.com/viewtopic.php?f=10&t=8497&start=30 or start a new thread in regards to the unpacker. Also have you been able to extract anything from scc_data.bin from the conviction.umd My iaa unpacker says that the file is not supported yet. Maybe take a look at the logs for the game to see what causes the hang?

Actually, I posted here - http://forum.xentax.com/viewtopic.php?f=10&t=4317&p=87738#p87738 and have some progress. I managed to get the text and inject it back. But now the problem is, that it seems some part of in-game text is missing. I'm talking about the in-game dialogues, during gameplay. I'm not sure what exactly the problem is, since if I delete my loc-rus.umd (I'm using russian file, since it is in cyrillic and is bigger) file the the game starts in english, but without subtitles, since I made it to play Language=rus in the Configuration.ini. So the text must be there, in that file, but I cannot find it. All I get in-game (while playing without my loc-rus.umd) is "....! ...?" in the place where subtitles shoud be. In other words, now I just need to find where in-game dialogues are, and I'm done with the job. :)

Edit: I just saw, that @swuforce has edited his comment, saying that the in-game text is in src\system\UMDS\01-11_xy.umd files.

How yo extract file menu.umd? Where I have to apply the XOR offset?

benzin how did you unpack the .bin file when i used iaa_unpacker on that same file  it said that the file format is not supported yet

Try this tool.
unumd.zip

I'd recommend move any localization issues to this board:
http://www.gildor.org/smf/index.php/board,22.0.html

Any questions about UnUMD - here.

 :D  Will give this a try as soon as I get home from work! Thank you Gildor!   As for XOR check the thread benzin linked to and look at the previous posts here by spinbot he explains how to use xor.

I saw but did not understand how to do. Can you do that for me?


This tool and spinbot's script perform the same function? And then extract the files ini?

 I did try the UnUMD tool and I have some issues.
1. when set the tool to the conviction directory and tell it to "unpack" files still come out as UMDs
2. when i use "out" I get this
buf=20000 file600F237 dir=11 F=82
compr tables: skip=883 uncompTblSize=BE7 comSize=883
PRocessing: T08_SecretLabs.umd
Disabled!      

You should use -unpack option to remove compression from UMD. This tool already was successfully used in some Convictions localizations, so I suppose it's enough. Non-unpack behavior (extraction of utx, ukx etc) was disabled because it's useless without umodel's support.

Hi
I've been trying to extract .utx files but no luck yet.Here's what i've understood so far :
off 16 :name count
off 20 :name offset
off 24 :import or offset count ?????
off 44 :filesize
I can't figure out export or import table.I can't find serial offset or serial size in the table.
Can you help me a bit please?
Thanks.

Unpacked UMD consists of mixed package chunks, so it's not possible to extract package without figuring out complete UMD format.

cool! I did use -unpack but it is good to know that it puts out file chunks. 

Does unumd work with I am alive .umd files?It's the same engine.

It makes me happy that people still want to toy with SCC. lol

If you guys need help let me know, but it seems you've got it.
Also, remember that SCC can run without UMD once you extract all of the packages to their original Unreal format.

Nice information. But unumd tool I provided has disabled package extraction feature.

I would use iaa_unpacker http://forum.xentax.com/viewtopic.php?f=10&t=8497&start=30   look for a post by Vash

Gildor what do you mean by unused resources?

For example, some textures which are placed into utx file by designer but which are not used in game levels - such textures will not be stored in umd file, and after extraction with my tool there will be "holes" in package files (filled with zeros), and umodel will crash on them.

It is possible to patch the table to allow for reading of only data that is present in package.

For the record, the game doesn't care about the missing data in Unreal packages since the data it seeks is in fact present. It's the left over data not used at runtime that is ripped out. If we had the editor, it would crash on loading these packages, just like UMODEL would without patching the import/export table.

I would be interested to see if someone can figure out how the .bin animation files can be restored back into the .ukx animation package, since it seems that they are ripped out from there and dynamically loaded back in at runtime.

Also, I would be interested if anyone can get the command line or log to display for Conviction or Blacklist.

I have been curious about the dev console for some time. It has been absent since pandora tomorrow I think.

How extract Dynamic*.umd files? (from conviction and blacklist)

It doesn't work for most of the files.
It can decompress them but no extraction.

I am in the same boat, benzin somehow managed to extract files using xor and the bms scripts but I have had no luck and he won't answer pm's  >:(

Does anybody want unumd tool with possibility to extract source (ukx, utx etc) files? Note: you won't not be able to open these files in umodel or any other software, and you'll not be able to put these (changed) files back to UMD. So it's rather useless functionality.

Gildor the extract functionality would be amazing

Uploaded UnUmd with unlocked extraction function. Hope it would be helpful to anybody.

 ;D Gildor, I appreciate your help so much

Where the download link?

http://www.gildor.org/smf/index.php/topic,458.msg15196.html#msg15196

thx, now try to add splitscreen in pc. I use hex editor for mod exe file and get it - http://www.youtube.com/watch?v=r7U_t8tKH6Q
But game does not define a second gamepad. Maybe some info in ini files.

what if you try to change the file Convictionuserpc.ini and convictionuser.ini and change [netplayer] local=true

don't work, i try it

try this maybe

https://code.google.com/p/x360ce/downloads/detail?name=x360ce.App-2.1.2.191.zip

download the newest toca edit xbox360 editor and put the files: xinputtest, dinput8.dll, xinput1_3.dll and x360ce in your game directory

next, you open the x360ce file in notepad, and you change [PAD 1] to [PAD 2] , this will make your gamepad player 2, in this file you can configure your controls for your controller. To check if your controller works, open the xinputtest file and look for >controller 2< if it says connected it works.

Yes, and it also tried. But still does not work

hmm, any luck so far?

I've just uploaded new UnUMD version. It has some changes:
1) you may disable XOR decryption when using -unpack option - use "-noxor" switch for that.
2) you may specify .umd file in command line when you want to decompress single .umd file instead of all files found in -path folder.

I've got a report that Splinter Cell: Blacklist UMD files could be extracted with UnUMD tool.

So does this mean when I extract the packages, I can get the models or no?

No, you can't. Umodel doesn't have Convictions support.

for models Use Ninja Ripper or 3D Ripper DX it will take some extra steps, but you will be able to get the models out.

my apologies for the bump, but has anyone in here had luck getting the game to load the extracted packages as opposed to the umds  ???

hei
I just unpacked all umd... -unpack
and returned to the game directory, but the game does start.....
what problem?

Pls

I think this is my problem...
Can someone explain this to me step by step??
can someone explain to me how to use QuickBMS???
this method is for PC or XBOX??? i nedd to PC....

Use UnUMD it is far easier and with less errors, you can find it on the prior pages. In order for the game to start you need to extract every single umd file and place it into the main directory. I do not have access to my PC right now I will get back with the exact location when I get home. However, I have had no luck getting the game running, I PMd a guy who managed to get his working but he is not responding.

Hmmm