An even more common demand is the removal of unwelcome background objects, something that might previously have required a round-trip to Resolve’s built-in Fusion VFX package. “We have an object recognition AI where you can use magic mask – it knows the difference between the horse and the rider. The classic example is a logo on a shirt; just go in with object removal and it goes away. That’s your wire removal tools, too.” IT REMOVES ALL OF THE ORIGINAL AUDIO AND BUILDS AN ARTIFICIAL SIMULATION OF WHAT’S SPOKEN
SIFT AND SORT With Resolve already boasting Fairlight digital audio tools, Blackmagic has put AI to work in sound post-production, too. “At NAB, we announced audio recognition that searches through a set of clips in a bin and sifts them depending on whether they’re dialogue or other content,” Heffernan explains, “which would be a pretty mundane task otherwise.” The apparent simplicity of that kind of task belies the sheer time spent on it, given almost every project will need to do it. Meanwhile, the option to clean up less-than-ideal original recordings is a rarer ask, though something the world has long pined for. AI has made inroads there also. “Within the audio package we did for neural engine, you can take the voice sample of someone on a noisy street and it will give you a completely isolated voice output.” Much as background noise separation has been tried before using conventional techniques, Heffernan describes how the AI approach works very differently. “Inside the voice isolation is an interesting example – it doesn’t remove the noise and give you the speaker’s voice. It removes all of the original audio and builds an artificial simulation of what’s been spoken.” Some things Resolve’s AI will do are entirely new. The most commonly used things like the clip searching and object recognition could have been done by hand – in fact have been done by hand for years, but only with an impractical amount of effort. “The use of AI is to speed up tasks,” Heffernan finishes. “Our philosophy around AI is that the tool can’t replace the creative. It has to support the creative, to give you speed and precision.” THE AUTOMATION ERA Applying that sort of thinking to the other end of production is an attractive idea to anyone who has ever wanted to televise a football match but lacked the financial wherewithal to do so. Julien Bergeaud has worked in sports media for more than 20 years with companies including Eurosport and Discovery
– and is currently vice president of sales in Europe for Pixellot, a company which aims to bring automation to live production. The company’s work centres around a camera, albeit one which is about as tightly integrated with software as a device could ever be. Bergeaud describes the system as “the camera, which is hardware, that goes alongside software. That’s an algorithm built with AI which has learned a sport. It’s capable of automatically producing that sport as if there’s human intervention – but there isn’t. Thanks to the AI, the camera is capable of doing everything a conventional broadcast is capable of doing – zooming in, out and following the action.” Making that a reality requires a carefully implemented camera head, with three lenses which create a 180° view of the sports field combined into a single 12K-wide image. Then, claims Bergeaud: “We learn the sport. We watch the sports and we teach the machine what’s happening on the pitch and why it’s important. Now we have 17 different sports. People think systems like ours are simply tracking the ball. There was a game in Scotland where a system got confused and tracked the head of a bald man. This is not us. What we do is machine learning, deep learning; we learn the sport.” There are benefits for the teams taking part, too, as Bergeaud explains. “The
ON TRACK Pixellot’s Show S3 is the future of automated video streaming tech
second part is all the analytics. The AI is capable of recording everything happening on pitch – all the components of the game, for a single player or for a team. The machine does it automatically and it’s fantastic for elite teams. In France, we
have already worked on Ligue 1 and now we are
feedmagazine.tv
Powered by FlippingBook