in Blog

November 17, 2022

Computer vision in the entertainment sector. How to use AI to enhance the visual experience


Edwin Lisowski

CSO & Co-Founder

Reading time:

4 minutes

Computer vision, a field of study that explores the ways of teaching computers to see the world like us, has become crucial in multiple sectors thanks to high-tech solutions for image recognition and processing, pattern recognition, visual search, object recognition, scene reconstruction, and more. But these terms that we all get used to by now scratch the surface of AI possibilities. There is so much more beyond, and the creative sector eagerly explores them to enhance the experience it provides.

Computer vision finds applications across various business areas, such as top-notch surveillance systems, automotive, and quality control processes. Still, it enhances and transforms the media and entertainment industry, including TV and film production, interactive media, games, events, sports, advertising, and gamified customer experience. And we are not talking just about smart glasses or augmented reality.

The AI or – to be more precise – machine and deep learning models reshape the creative process inside out, not by excluding human labor, but by freeing their creative energy to handle more, well, creative tasks while delivering an immersive visual experience.

That was the goal of the entertainment giant, widely known for organizing the most iconic events in sports and entertainment. The US-based company was looking for ways to enhance the visual experience in the venue about to open very soon. The new venue, packed with top-notch hi-tech solutions, would be a place where visual effects would be on a higher, previously unattainable level.

How AI can boost visual experience in the entertainment sector?

The company was looking for a partner to work on an experimental R&D project with no benchmarks and ready-to-use solutions to base on. The main task was to create an automated video stitching platform that seamlessly combines multiple viewpoints into one coherent panorama.

AI in the entertainment sector

Manually editing and sewing the video together was insanely ineffective, given the number of frames that must be processed and included. Increasing the level of automation was a natural choice, but – on the other hand – implementing a fully automated solution that works perfectly for every possible footage was impossible, given the uniqueness of particular footage. The editors had to be able to control materials and manually indicate elements requiring intervention.

Addepto’s approach

The pure idea of stitching images wasn’t new, and there are already solutions. Yet their quality wasn’t enough for the company, which was aware of the need for a custom-made solution. Since money was not an issue, it decided to find an AI-focus talent able to explore new possibilities, and Addepto turned out to be the one who won the audition.

project's approach, computer vision in the entertainment sector

Addepto AI Team already had an idea of how to handle this project, and – first – they divided it into three main tasks:

  • stitching video images coming from cameras networks
  • segmentation of objects pointed out by users
  • image inpainting, which means reconstructing missing regions in an image

However, finding the right “engine” to handle these tasks with the highest possible performance was difficult. The input video footage was extremely high quality as the camera array consisted of cameras creating Hollywood-quality pictures, which in Data Science means one: the massive amount of data to be processed.

The project was highly challenging and exciting. The company was also aware of the AI R&D project besides it takes time and resources and needs a solid amount of training data to make the model efficient. We were provided with everything we needed.

                                              – Bartłomiej Śmietanka, Data Scientist, Addepto

Then, Addepto AI Team dived into a search through research papers, and recent advances in the field, trying to find meaningful tips on how it can be done. The natural direction was Machine and Deep Learning modules accompanied by a classical computer vision approach. The secret sauce was to cherry-pick the best-of-breed elements, connecting them into a seamlessly working platform and building a user-friendly interface.

Addepto AI Team reached for Machine and Deep Learning models enriched by classical algorithms typical for Computer Vision and statistical algorithmics. They combined them into a seamlessly working platform with a user-friendly interface.

The solution developed by the Addepto team will be embedded in a massive IT infrastructure where the fusion of technologies with the space will create a unique cinematic experience during live events.



Computer Vision