This week we’ve been testing Unreal Engines multi-user framework over VPN to enable remote collaboration between UE artists and developers. For these first tests we used UE4.24 and UE4.25 versions running on OSF computers over the StormCloud VPN service that runs on the Microsoft Azure infrastructure. From a network perspective StormCloud is designed to connect many thousands of remote VPN users in Unreal Engine.
UPDATE: The team at Epic soon pointed out to us that the main problems we go into in this post are due to the fact that we’d not set up any solid version control. Now being technical film folk we’d not really encountered source control before outside of VFX, but be sure to check out this second post, which brings you up to date with our user experience. Thanks to the Epic UE team for putting us on the right path.
Back to setting up your first multi-user session for virtual production.
Getting your session running
To get started you distribute an identical level to each user, you boot up your VPN, the host opens the project, starts a named multi-user server, everyone else then opens their session browser, and on seeing the server listed, they all jump in.
When you get a multi-user server up and running for the first time you’ll be seeing the avatar of your co-workers zip around. Collaboration soon becomes addictive and you’ll start working together. This is where multi-user feels like something new, a new way to co-create, especially if you jump in a VR headset. One of us jumped into our VIVE Pro, fully immersed, they became ‘fully in’ exploring the scene textures and details, while watching the rest of us flying around, moving objects. Multiple minds in one virtual space.
Then things got tricky
When we started out, joining into a starter content level and a new server everything worked fine, but it’s when your levels grows and you begin building your co-creation together, everything becomes a little tricky to keep track off. Especially when trying to keep track of your project files, and keeping them all up to date and identical, this is when things for us became frustrating, at least for a couple of hours until we worked it out.
When it’s all up and working, multi-user works well. But we found ourselves at one stage struggling with the project file synchronisation when user log back on to a server session. If the host has been doing any work on the server instance with other users logged off, when the user logs back on the files have to sync, the session. When syncing multi-user sessions is when we encountered problems, with UE crashing, this went on for some time, we questioned StormCloud but the VPN was up and stats were good, but then we had an idea.
We are still unclear on how the session stores and transmits session states, but, we found that going sessions again worked if the host deleted the old server and session, and started a whole new multi-user server and session, for us to join. All still using the same project files, we just started a new server and session. It worked!, but we are not sure what we lost and we have more testing to do to understand how project files and session instances save, one ting is for sure, using multi-user is a skill to be learned and mastered.
Also annoying was sending large project files to users local machines, so that you all have the same project file to start with. Over domestic internet this was slow (ours project file was only 3GB). So we’d recommend you be sure you are happy with the project you are to distribute, and also set aside time to get set up, as its a pain having to wait around waiting for the project file to get to everyone.
Overall verdict, we had a promising start with lots of smiles and laughs, and this somehow feels like the future of where we’ll all be building something new together, when it’s up and running, it works like a dream. If you’d like to register for beta test accounts for StormCloud VPN you can here.
Another day of testing and we’ve now got a firm hand on how multi-user file synchronisation works. Most important thing to know about is something we’ve called the session game state, the host is always the master of the game state and all other users need to have this game state passed to them so that they can be up-to date in the session and all see the same thing. We have even had to nudge all assets on the host machine to force the host to send its current game state over the VPN to the remote users in the session, otherwise the remote users may not see the same as the host.
It’s this game state that can sometimes get a little lost in translation as you log in and log out, edit your levels with users offline and you will need to re-share your project, if you find out that you can see things differently from different machines, its probably time to all get off the server, redistribute the hosts saved project, and then all jump back in.
Find out how we solved our frustrations with version control.