Live Streaming the Australian Open in 360 Degree Video

Introduction

Eastwood, in partnership with StreamShark approached Tennis Australia with a proposal to live stream the 2016 Australian Open in full 360 degree, 4K video. This would deliver a live, immersive 360 experience from centre court (Rod Laver Arena) during the Australian Open Tennis Grand Slam.

 

The Client

The Australian Open is one of the world’s largest tennis grand slams, attracting the top tennis players, along with many millions of viewers across the globe. With a total of $44 million in prize money up for grabs, it makes for a highly competitive event.

 

The Proposal

The proposal brought forward was to live stream a 360-degree video feed directly from Rod Laver Arena (the main tennis court). As Tennis Australia is always on the hunt for new ways to uniquely engage with their user base, we thought this proposal was perfect for them.

An immersive fully 360-degree video solution allows viewers who missed out on tickets, or were unable to attend, the option to view the game in real time from court side. This court side position was usually restricted to umpires, media photographers and of course the ball boys and girls, so it gave a very unique perspective of the game.

Visitors to the Australian Open general area would be able to view the stream from the ‘Australian Open 360 Experience’ booth, which had Samsung Galaxy S6 phones mounted in custom made head sets designed by Eastwood.

 

The Challenges

The project involved combining two technologies, live streaming video and 360-degree video, into a seamless experience. The live stream would need to run for 12 hours a day uninterrupted, for 16 days straight. Access to the broadcasting equipment was limited to the very initial setup, and any ongoing hardware management would need to be conducted by Tennis Australia staff, or via remote management tools.

The current form of 360° capture is heavily focused around pre-recording footage, running the video processing software, then uploading it to the internet for consumption at a later date. As this was to be a live event, we didn’t have the luxury of pre-recording video, everything had to be done in real time.

We were limited by the hardware. As mentioned, the majority of 360° camera equipment is focused on recording video to memory cards. Finding the right recording equipment for a live event was essential.

Our solution needed to deliver a consistent experience across Android phones and desktop computers. Viewing 360-degree video requires a unique, customised video player that can work in the hand as well as when viewed on a VR headset. Desktop users need to be able to pan around with the cursor, and Android viewers needed to be able to intuitively view the stream through a Google Cardboard or equivalent headset.

 

The Hardware & Software Setup

To give some background on how 360° cameras work, generally a 360° camera has multiple wide angle camera lenses, with overlapping fields of view. Once the video is recorded, it needs to be processed and have all the video recordings blended and stitched together into one. The video is stitched into what is known as a ‘Equirectangular projection’, which is the same method the world map is displayed on a flat plane.

Over the course of 2 months we experimented with a range of 360° cameras, however the issue we kept encountering was that the output formats from the cameras couldn’t be handled by our video player. One example being the Ricoh Theta S, which could only output two fisheyes. (Update: The Ricoh Theta S now supports live streaming with a full equirectangular stitch, albeit at a low 1280×640 resolution)

Previous to this, we had hoped to have a very portable setup, with a hardware video encoder and a 360° camera, but we had come to the realisation that it wasn’t possible at this point in time to get the quality we were aiming for.

Over the course of testing we had come across a real time video stitching solution called ‘Vahana VR’ made by an aptly named company called ‘VideoStitch’. Unfortunately all our existing cameras and video capture cards weren’t compatible with this software.

For our final 360° capture rig, we settled on the following equipment:

• 1x Freedom360 Broadcaster camera mount.
• 6x GroPro Hero4 Black cameras.
• 6x HDMI cables, 6 USB cables.
• 1x 6 port powered USB hub.
• 1x High-spec Windows 10 PC.
• 1x VideoStitch Vahana VR software.

 

The Video Delivery Setup

StreamShark specialises in delivering highly reliable and available live streams to the internet, so this aspect of the setup was pretty straight forward.

Our standard live streaming event setup uses Wowza for the video transcoding, which we then hook into StreamShark’s global content delivery network. As we had two target devices, Samsung Galaxy S6 and desktop computers, we needed multiple video qualities which we transcoded on the fly.

The Samsung Galaxy S6 couldn’t handle the raw source output from the cameras, so we lowered the bitrate and resolution to a level which the phone could play smoothly. For the desktop version we had multiple qualities available, as well as the direct source.

 

The Video Player

360° video playback is not as easily implemented as traditional video. Viewers need to be able to interact with the video and change the point of view. On a desktop computer this involves using the mouse cursor to pan around. However when a viewer uses a head mounted display in combination with their phone, or a dedicated headset such as an Oculus Rift, it requires motion control to give a fully immersive experience.

For this purpose we developed our own 360 degree video player, which is available on GitHub here. Unlike the majority of 360° video players out there, ours doesn’t require Flash, which means it works across a much larger variety of devices. At StreamShark, we focus on “Responsive 360/VR” solutions. With our 360° player – there is no need to wait for head mounted displays (HMDs) such as the Oculus Rift or the HTC VIve to become mainstream. You can start delivering great 360/VR content now. The player scales well for all form factors & input, from a standalone phone or tablet (where your viewport can move by leveraging the in-built sensors/gyroscopes contained in billions of phones out there), to intermediate headsets such as Google Cardboard or GearVR, all the way to professional desktop headsets such as the Oculus Rift.

 

Key Learnings

GoPros overheat very easily! They will shut down when their internal temperature reaches 51°C. This temperature is surprisingly easy to reach when you have 6 GoPros strapped together, in direct sunlight on a 34°C day. To avoid this the GoPros needed to be shutdown before major matches to allow them sufficient time to cool. In the future we will come up with a better cooling solution.

Real time video stitching is still very much in its infancy. In order to stream an audio feed from the GoPro microphones we needed a beta version of Vahana VR, or else the stream would crash. It’s also very important to make sure you position the video stitching lines in areas that aren’t the focus of attention, as stitching isn’t 100% perfect at this stage.

Finally finding the right camera stand is a much harder task than imagined. The 360-degree mount had quite a small area where you affix a pole/tripod. For our setup we used a monopod, however we used it upside down as the diameter of the base was small enough to allow a flush connection. For the base we used a basic ball tripod, with additional counter weights.

 

The Outcome

In the end we successfully streamed the whole event for 16 consecutive days, for 12 hours each day. The client, Tennis Australia, was extremely happy with how everything turned out and every one of the approximately 8,000 people who visited the Australian Open 360 Experience installation left with a smile on their faces!

 

 
If you’re interested in live streaming, or 360 degree live streaming then feel free to send us a message. We love talking about 360 degree live streaming!