FascinatE Network and Delivery
The transmission of the FascinatE layered scene representation represents a major challenge for a delivery network, essentially in terms of bandwidth and processing requirements. As an example, the live delivery of the current FascinatE audio/video test material requires an uncompressed data rate of around 16Gbps. FascinatE aims to deliver immersive video services to a large range of terminals from high end audio/video set-ups with fibre connectivity, to low-powered mobile devices. To deliver an immersive and interactive media experience to any device in a scalable manner, the project has focused so far on the development of Audio/Video Proxies. Their role is to perform some of the media processing tasks on behalf of a terminal so as to reduce the processing and bandwidth requirements for the terminal hardware. The following two prototypes were demonstrated at IBC 2011:
1. Network Proxy for Real-Time Video Navigation:
In this demonstrator we focused on a specific case where the proxy is able to process in real-time end user requests for navigating a very high resolution video panorama (Figure 1). The proxy has access to a 7k x 2k video panorama and sends a reframed and compressed video to each client device at an appropriate resolution and bitrate. In this demonstrator, the end-user can directly navigate the 7k panoramic video using a tablet or a mobile phone equipped with a touchscreen. The user commands are translated into a stream of 2D translation and zooming commands that are sent upstream to the proxy. The corresponding reframing, rescaling and coding processes are executed in real-time for each client device. The proxy then delivers a compressed video stream containing the requested views, which only requires a standard decoding step before display. With this approach, Ultra-HD content can be watched interactively in a natural manner, even on a low-power and small-display device.
2. Spatial segmented delivery of immersive media:
Spatial segmentation is used as a method to efficiently deliver parts of the 7k x 2k video panorama to devices which are not capable of displaying the entire resolution at once, such as smart phones and tablets. The general concept behind spatial segmentation is to spatially split each video frame into several tiles. The video frames corresponding to the various tiles are encoded independently and stored separately as a new video stream, or spatial segment. We focused in this demonstrator on the case where the A/V proxy only requests a subset of segments, based on the ROI selected by the user(s) for which it performs the spatial segment re-assembly. In the prototype, spatial segments are transported using a protocol similar to HTTP adaptive streaming. The spatial segments are then reassembled by the proxy. The navigation can be controlled on a mobile device, such as a tablet or smart phone. Functionality is further increased by using multiple resolution layers, which allow for smoother zooming (Figure 2).