Person Tracking & Production Scripting
At IBC 2011 we presented a demo for person detection and tracking in high-resolution panoramic video streams, obtained from a panoramic camera stitching video streams from 6 HD resolution tiles. The tracking algorithm has to detect and track persons over six static and rectified HD image-sequences from the OmniCam. Instead of using the ultra-high definition image, each video tile is separately analysed by different workstations to enable real-time analysis. The AV content analysis uses a CUDA accelerated feature point tracker, a blob detector, a CUDA HOG person detector, which are used for region tracking in each of the tiles before fusing the results for the entire panorama. The results of the person and blob detector for each image of the different image sequences yield the regions of detected persons for further processing. Furthermore, person IDs are linked to the appropriate combined regions with their corresponding feature points. The tracking system is demonstrated on a single PC with appropriate graphics board, processing a full HD stream. Results are shown in the figure below.
Production Scripting Engine
The Production Scripting Engine (PSE) is responsible for decision making on content selection. The key feature is to automatically select a suitable area within the OmniCam panorama image, in addition to cuts between different broadcast cameras. Selection behaviour is based on pragmatic (cover most interesting actions) and cinematographic (ensure basic aesthetic principles) rules. In some cases, this is not fully automatic but involves a human-in-the-loop, a production team member deciding between prepared options. The PSE is a distributed component with at least one instance at the production site and one at the terminal end. The output of the PSE is called “script”, which consists of a combination of content selection options and decisions, renderer instructions, user interface options etc. Scripts are passed to subsequent PSE components from the production site towards the terminal, where final instructions are given to a device-specific renderer.