![]() As with most things in life, it is easy to take a wrong turn or ‘bark up the wrong tree’. If the error is high then via backpropagation it is reduced, this declining error rate is mapped visually in Nuke and is called gradient descent. This rippling back is called backpropagation. To reduce the errors, it ripples back through the network improvements to its’ approach, changing the weights of nodes inside its neural network. The network is called a Deep Network as it has layers. In maths terms, this is directly related to the reduction of errors in the ML network. What makes the computer so powerful is its ability to try things a million different ways and if it starts getting better results it continues with that line of inference. It also does not view the whole frame as one thing, it works in patches. It infers best inside the space not extrapolating from the examples. ![]() The further you ask the computer to infer outside the examples of training data space, the worse the results. ![]() It works on statistical inference and while we clearly see a person or a car, the computer never does, it just sees pixels and it tries to reduce errors so it’s output aligns mathematically with the training data. The first thing to understand is that the computer has no understanding of the image, it appears to, but it does not. The output seems so logical and sensible that when it does not work, (and it most certainly can fail), such failures seem to make almost no sense. It is easy to anthropomorphize the actions and imagine the computer ‘sees’ the frame as we do. To really master ML it is worth understanding how this works and what is really happening under the hood. The system then infers what has been done and applies it to a clip. To use Cop圜at is relatively easy, sample frames of roto or beauty work are provided as before and after frames to Nuke. Importantly there are also ML solutions that are classified as Unsupervised and Reinforcement Learning, more on those below. This is because it is part of a class of ML called supervised learning. Not all AI or ML requires training data examples, but the Cop圜at node does. Beauty work here removing the actor’s beard Supervised Learning There is now little doubt that while such AI tools will not replace artists, those who fail to understand them may fall away as a new generation of complex AI solutions are deployed. The approach of providing training material to a ML node which then infers a result is truly revolutionary and exceeds even the current hype surrounding the general AI buzz in the press. What makes it so exciting is that ML represents a new way to solve problems not just a new tool or node in Nuke. There is no doubt that ML brings to Visual Effects a whole new world of solutions to visual effects problems. Dan Ring, Head of Research at the Foundry. “We do believe that the MSRN is a magic network, it solves a huge variety of challenges, and it does it well,” comments Dr. As with the ML-Server, the core ML tool is a Multi-Scale Recurrent Network (MSRN). Primary amongst these is the Cop圜at node. ![]() With Nuke v13 the Foundry’s AIR team now offers native nodes inside Nuke. Two years ago, fxguide published a story on the Foundry’s open-source ML-Server client/server system that enabled rapid prototyping, experimentation and development of ML models on a separate server, with the aim of introducing a way to have ML tools in Nuke, but developed in parallel. The reduction of blur can be seen here split screen inside NUKE The ML networks for these nodes can be refined using Cop圜at to create even higher-quality shots or studio-specific versions in addition to their primary use for resizing footage and removing motion blur. Upscale and Deblur – two new tools for common compositing tasks were developed using the ML methodology behind Cop圜at and open-source ML-Server.Inference- is the node that runs the neural networks produced by Copy Cat, applying the model to your image sequence or another sequence.This artist-focused shot-specific approach enables the creation of high-quality, bespoke models relatively quickly within Nuke without custom training environments, complex network permissions, or sending data to the cloud. Cop圜at – an artist can create an effect on a small number of frames in a sequence and train a network to replicate this effect with the Cop圜at node.The key components of the ML toolset include: Research team (AIR), it enables artists to create bespoke effects with applications of the toolset including upres, removing motion blur, tracking marker removal, beauty work, garbage matting, and more. The ML Toolset was developed by Foundry’s A.I. Nuke 13 includes Machine Learning (ML), a flexible machine learning toolset.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |