Rokoko mo-cap suit streaming directly into Unreal Engine via live link on OSF on-set render engine. OK so first off please don’t be too hard on this demo output, as we are under virus lockdown and the poor Rokoko suit is surrounded by more tech in our bunker that must be messing havoc with calibration, so there’s a lot of room here to get better calibration, rigging and solving data, but under the circumstances we are mostly interested in the pipeline set up and how it runs on our virtual production machines.
So how do you connect the Rokoko motion-capture suit to Unreal Engine.
Setting up the Rokoko suit, first we connected the Rokoko suit to the same machine that UE4 was running on, later we’ll break this out onto a dedicated machine, but for now we just wanted to get the pipeline up and working. The suit connects over the wireless network to the UE render engine and into Rokoko Studio where we sign the suit a personal profile or the performer. We then start streaming the data into UE by selecting the Unreal Engine option in the Rokoko Studios Live Tab, this is a feature only available to Rokoko Pro Licence users.
Moving over to Unreal, there is a small amount of blue-prints to set up. As you can see in the below blue-prints screen grab. In the Smart Suite body pose node this is where you will set your bone maps and tick the relative to start option. This sets the zero in Rokoko Studio to wherever you set your character in Unreal. Another thing to mention is that Rokoko only works with T pose characters. In our case, we are not really animators so we use quick and easy models exported from iClone, if you want to do the same to get your pipeline up and running just remember to export the iClone model with the T pose option selected.
After setting up the character blue-prints pull out a Smart Suit Controller and a Smart Suit Receiver, from the Modes Panel onto your stage. In the Smart Suit Controller , under the Smart Suit Tab, select auto-connect and assign the suit name, which can be found back in Rokoko Studio, here…
Back in UE, under the Smart Suit Receiver tab, change the streaming data port to the connecting Rokoko Studio port (the default is 14043). And hit play.
Our focus is the tech driving these pipelines, we test break, fix and work out what works best for virtual production on-set. In our previous test we had all apps running on one render engine iClone, Rokoko Studio and Unreal. This was no issue for our Onset render engines in regards of power as all held a steady 100fps in unreal and the maximum of 60 in iClone, the issues came from the memory usage and dedication between software’s. iClones stream of data would become noisy when it runs in the background this wouldn’t be a issue if we was to stream from a designated engine.
So now starts our optimization work. That’s why in this test we had just two applications running, Rokoko Studio and Unreal Engine, on one machine. The result on the same spec render engine was massively better, the data fidelity of the Rokoko Live Link to Unreal was so much faster and the movement was almost good enough for real-time production. But we feel it’s not there yet.
So next up we will move Rokoko Studio to its own OSF render engine and run Unreal Engine on it’s own render engine too, so it seems that the best set up is to have a PC render engine for each application and function in your pipeline, we’ll prove this to be the case in the coming weeks. Also in later tests we’ll show you the facial capture set up too.