Vive Virtual SteadyCam in Unreal Engine 4.24
Recently we’ve been testing and developing with the HTC VIVE tracker. The tracker has gone above and beyond our initial expectations, although the tracker is best used in a fully virtual production use case. The HTC Vive trackers are priced at an affordable £110 allowing anyone to pick one up and start developing.
Although the tracker impressed us we did struggle with the high sensitivity. To start we built our own VirtualCam rig to shoot with, this gives the Vcam extra weight to smoothen out physical manoeuvres but this still didn’t solve our jittering issue with the trackers sensitivity. We decided a virtual camera needs a virtual stabiliser so this what we got.
Below you can see the blueprint used to stabilise the tracking data and moves.
To walk you through exactly what the blueprint is showing it starts at the ID of the tracker which can change depending on the order things are turned on. Once the tracker is found the data is broke up into a break Vector(position) and Rotator(orientation) this breaks the X,Y,Z and Roll, Pitch, Yaw down turns them into text and create a new rotator and orientation value to the nearest decimal. The actual smoothing is done within the “Weighted Moving Average Vector/ Rotator” nodes. The nodes receive a current and previous sample through the inputs and a weighted value. The Return values are then sent out to a “SET” node for both the current vector sample and rotator, whilst also going to a “Set Relative Location And Rotation” node this will set the positioning of the tracker. The “Set Relative Location And Rotation” node also has a “Cine Camera” component connected, this will be where the tracking is applied to and provide the view for the tracker.
To activate the Vive SteadyCam simply drag out the pawn actor and play the scene. Unfortunately the tracking isn’t view-able in editor the game has to be in play mode to receive data as no live link plugin is available as of yet. This makes the framing of shots and editing scenes on the fly difficult. The tracker does also fall down on its reliability as loses connection to the engine and its transforms can bug out and become inverted.
Although this blueprint works perfectly when using take recorder as the only source needing to be recorded is the pawn. Because the pawn has a Cine camera component to it the take reorder automatically records the cameras viewport as well as its transform, this makes rendering out shots easier.
Thanks to Hussein H Abdallah for the blueprint support, its great to see such an open and developing virtual production community.