Gildor's Forums

Author Topic: Retargeting extracted animations  (Read 2632 times)
fuegofuego2
Newbie
*
Posts: 16


When life turns its back on you... grab its ass!!


View Profile
Retargeting extracted animations
« on: September 05, 2019, 19:07 »

Couldn't find more information about this problem


Am using Unreal 4.22, created a 3rt person example.

I extracted from Mortal Combat using UE Viewer + ActorXImporter some FBX animations from Mortal Kombat.
Here a link with some of the FBX to reproduce the exact same scenario.
https://mega.nz/#!iW4TnYyb

Then i proceed to do the test

So I import the fbx with no skeleton



I create a RETARGETING file for both the MANNEKIN skeleton and another for the 4A_FrontEnd_Loop SKELETON



Then i try creating a new DUPLICATE ANIM ASSETS AND RETARGETS

I select the manekin

and get back this message


Then i try with the other skeleton instead of creating a new DUPLICATE ANIM ASSETS AND RETARGETS

I select the MORTAL KOMBAT skeleton

And it crashes the unreal.

Am I the first person to ever happen this result?

What am expecting.. Is to take "MY SKELETON" and make new ANIMATING SEQUENCES with this MORTAL KOMBAT ANIMATIONS usin my SKELETON CHARACTERS

Does anybody see if am missing some puzzle pieces? couse i can't see it right now  huh huh huh huh thanks!
« Last Edit: September 05, 2019, 20:21 by Gildor » Logged

TrumpetPro
Newbie
*
Posts: 25


View Profile
Re: Retargeting extracted animations
« Reply #1 on: March 11, 2021, 11:51 »

Just do exactly what the error message says, just set the preview meshes for each skeleton. Usually you just have to go into the skeleton asset and hit "apply to skeleton" on the panel to the right, but that was answered in the very first Google result as well.

Note that this forum is for umodel, not UE4 editor features.
Logged
Gildor
Administrator
Hero Member
*****
Posts: 7973



View Profile WWW
Re: Retargeting extracted animations
« Reply #2 on: March 11, 2021, 12:06 »

Interestingly, I'm working on retargeting now. But inside UModel, not in UE4. Preparing the test project ATM for being able to compare things how they looks in UE4 and in UModel.
Logged
RedBear
Full Member
***
Posts: 226


View Profile
Re: Retargeting extracted animations
« Reply #3 on: March 11, 2021, 18:56 »

Interestingly, I'm working on retargeting now. But inside UModel, not in UE4. Preparing the test project ATM for being able to compare things how they looks in UE4 and in UModel.
Good news. Long-awaited feature as for me.

UPD: Still, what is interesting to me is that we will be able to retargeting skeleton animations from one game to another one?
For example, from Mortal Kombat 11 to Fortnite or vice versa.
« Last Edit: March 11, 2021, 19:21 by RedBear » Logged
Gildor
Administrator
Hero Member
*****
Posts: 7973



View Profile WWW
Re: Retargeting extracted animations
« Reply #4 on: March 11, 2021, 20:21 »

Probably yes, with use of "Rig" and "retarget AnimBlueprint" things in the engine. Don't ask me for details.
Logged
RedBear
Full Member
***
Posts: 226


View Profile
Re: Retargeting extracted animations
« Reply #5 on: March 12, 2021, 09:08 »

Actually I was asking about retargeting animations in UModel, not in Unreal Engine 4.  grin
Logged
Gildor
Administrator
Hero Member
*****
Posts: 7973



View Profile WWW
Re: Retargeting extracted animations
« Reply #6 on: March 12, 2021, 10:42 »

Absolutely, retargeting animations in Unreal Engine 4 is very easy to do, I do it all the time Cheesy
Don't you know how to retarget animations without duplicating animation sequences? I've watched lots of documentations and videos over past time, tried to do the retargeting myself. I'm trying to replicate the way how Fortnite works. What Fortnite has: there're body and head meshes. Both has own unique skeleton. There's a large number of animations, made for another skeleton. In some way, engine combines them all together, and plays the animation made for that another skeleton. It does retargeting at runtime, compensating differences between body and animation skeletons.

I'm doing the same thing in UModel, but there are unclear things, some bugs. So, I decided to prepare a test project, compare behavior in UModel and UE4, and debug it with full UE4 source code. I'm using Paragon female character (from marketplace) and standard Mannequin, trying to play mannequin's animations on female.

When I'm assigning Rig to animation mesh (mannequin) and to target mesh (female), animation is still unplayable there - editor doesn't let to select animation sequence for the body mesh, when it has a different skeleton - no matter the same Rig is used. When I'm doing "retarget Anim Blueprint" or "retarget anim assets", there's a window with options appears, and no matter what I set there - execution of the action will always make the duplicated animation sequences, which belongs to the body's skeleton. This is not the way Fortnite works - there are no duplicates in the game (I do not bother about Fortnite much, but I want to reproduce the way UE4 works, so ALL games will receive the fix).

Some (relatively old) Epic's video shows that "modular characters" are using CopyPoseFromMesh blueprint function, and it doesn't clarify how do they use it. I've read its code,, and I think that retargeting won't work this way - it seems that code simply copies bone positions to the target mesh, without doing any retargeting adjustments.

I really need some help with this, or may be the sample project, where the same animation sequence will be playable on characters with different skeletons.
Logged
Stiffy360
Newbie
*
Posts: 12


View Profile
Re: Retargeting extracted animations
« Reply #7 on: March 19, 2021, 06:02 »

I'm not familiar with fortnite's method, but are the skeletons the "same" in that they share the same hierarchy and bone names? What I've seen some games do is essentially force the model onto a skeleton that doesn't fit it, then use an additive animation to force the bones into the correct shape then use ik to align important stuff such as hands and legs. I'd imagine that's what that "CopyPoseFromMesh" does.
« Last Edit: March 19, 2021, 08:42 by Gildor » Logged
Gildor
Administrator
Hero Member
*****
Posts: 7973



View Profile WWW
Re: Retargeting extracted animations
« Reply #8 on: March 19, 2021, 08:44 »

Each character mesh part in Fortnite has its own skeleton.
CopyPoseFromMesh seems does pure copying of bone matrices, without doing anything else.
Logged
Gildor
Administrator
Hero Member
*****
Posts: 7973



View Profile WWW
Re: Retargeting extracted animations
« Reply #9 on: March 19, 2021, 16:53 »

Yes, I know these things, thank you. The problem is that skeletons has different bone positions, proportions and so on. Currently I'm thinking on proper way of supporting bone scales, probably this is the root of the issue. The problem is that it will require huge changes in UModel's animation system, plus file formats used for export are not suitable for bone scales.
Logged
Gildor
Administrator
Hero Member
*****
Posts: 7973



View Profile WWW
Re: Retargeting extracted animations
« Reply #10 on: March 23, 2021, 13:49 »

I think I found how animation on different skeleton with retargeting works in Fortnite.

Some information first.

1. All player models consists of multiple parts (aka "modular characters").
2. Each part is using its own skeleton, except probably head, which is using Fortnite_Base_Head_Export_Skeleton.
3. All animations are bound to Fortnite_M_Avg_Player_Skeleton, which differs from mesh skeletons.
4. For being able to play animation on a mesh, both mesh and animation should be bound to the same skeleton, there's no way to play animation on a different one - engine will simply reject that (I've analyzed C++ code, so I can confirm).
5. Skeleton object is used only for animation. Each mesh has its own kind of skeleton, called "reference skeleton", which is actually a set of bones and their positions for that mesh.

So, there's no way to play player animation on the player part without doing special tricks. According to Epic's videos, there's an invisible "skeleton" mesh which is used for animation playback. And then the pose is copied to all character's parts, so they are animated together. I did find these invisible meshes, there's a partial list of them:
Code:
/Game/Characters/Player/Male/Male_Avg_Base/M_LRG_Base_Skeleton.uasset
/Game/Characters/Player/Male/Male_Avg_Base/M_Med_Base_Skeleton.uasset
/Game/Characters/Player/Female/Medium/Base/SK_M_Female_Base_Skeleton.uasset
/Game/Characters/Player/Female/Large/Base/SK_L_Female_Base_Skeleton.uasset

All of these meshes are bound to the same skeleton as animation - Fortnite_M_Avg_Player_Skeleton, so they are compatible with animations. Game code adds one of these meshes to the scene, and makes it invisible. Animation code plays the track, and performs retargeting to the "reference skeleton" with different proportions. Then, code does copy of the bone positions to all character part meshes, without doing any additional adjustments.

Note that these invisible skeleton meshes doesn't have any face bone, so it seems face animation is entirely ignored.

So, regarding the support of animation retargeting in UE Viewer. It already works, and seems works as it should. The problem is with face bones. UE Viewer can't know that some bones should be ignored, because it doesn't know that there should be some "intermediate" invisible mesh which should be used for animation. There's no standard data structures for storing that information, Fortnite definitely has either structures, or a blueprint which does the linkage. This is not an engine core feature, so I simply can't do the full support of the animation, avoiding any bugs.
« Last Edit: March 23, 2021, 14:03 by Gildor » Logged
RedBear
Full Member
***
Posts: 226


View Profile
Re: Retargeting extracted animations
« Reply #11 on: March 23, 2021, 18:35 »

This means that we should not wait for correct animation retargeting for games such as My Hero One's Justice 2 (https://www.gildor.org/smf/index.php/topic,7010.msg36154.html#msg36154)? Since animation retargeting for these games is not working correctly at the moment.  :'(
Logged
Gildor
Administrator
Hero Member
*****
Posts: 7973



View Profile WWW
Re: Retargeting extracted animations
« Reply #12 on: March 23, 2021, 20:38 »

This might be a scale issue, which I probably fixed over last week in local (experimental) build. I need sample files for being able to test it myself.
Logged
RedBear
Full Member
***
Posts: 226


View Profile
Re: Retargeting extracted animations
« Reply #13 on: March 25, 2021, 18:51 »

This might be a scale issue, which I probably fixed over last week in local (experimental) build. I need sample files for being able to test it myself.
I extracted game assets using QuickBMS and uploaded them to my cloud: https://cloud.mail.ru/public/hJqB/hxE4FJdv9
I tested animation retargeting with UModel (build 1527) but the problem persists (сhanging the retargeting mode doesn't help):

Logged
RedBear
Full Member
***
Posts: 226


View Profile
Re: Retargeting extracted animations
« Reply #14 on: March 29, 2021, 10:49 »

Hello Gildor.
So have you tested retargeting with the files of this game on your local (experimental) build? How does it work for you?
Logged
Jump to:  

Powered by SMF | SMF © 2006-2009, Simple Machines LLC
Leviathan design by Bloc | XHTML | CSS