Gadget Show - AI deepfake
The Gadget Show approached Solarflare Studio with the task of being able to transfer John Bentley's face onto footage of a singing guitarist, so that it would appear like he was able to both play the guitar and sing, and in doing so win a head to head music battle.
SOLUTION
DeepFake is a technique for human image synthesis based on artificial intelligence. This involves a multi stage approach, firstly by using machine learning to track the key 68 facial landmarks and using that data to generate an alignment file for both faces.Then using a machine learning technique known as Generative Adversarial Networks, we were able to set 2 AI networks competing against each other. One AI generating forgeries, and the other AI attempting to detect those forgeries.Through the competition of the 2 networks, the forgeries become more convincing over time.
RESULT
We completed over 285,000 iterations, allowing the AI to match facial landmarks and expressions and be able to effectively swap faces. This created a full 3 minute video, with John Bentley's face appearing to sing and play the guitar to the classic 'House of The Rising Sun' track.The final video was projected onto the stage at a music venue, where John Bentley was competing against a singer songwriter. The Gadget Show episode was shown on Channel 5 on Friday 22nd November, and marks the 4th project that members of the Solarlfare team have worked with The Gadget Show..
The first stage that we needed to go through, was getting the computer to understand the 2 faces that we would be using for the Face Swap. This was achieved by using Machine Learning to step through frame by frame and identify where the key 68 facial landmarks are, and save those into an alignment file. These alignment files are essential for training and are also used during the final converting process.
Using the alignment files for each of the learnt faces, the computer then set about the task of trying to train itself. This is achieved by using a machine learning technique known as Generative Adversarial Networks. Essentially where you compete 2 artificial intelligence networks against each other. One AI is responsible for generating a forgery of both faces, and the other AI is trying to detect the forgery. After each iteration, the AI improves and the forgeries are slightly more belivebale. So after hundreds of thousands of iterations the face swaps are more convincing.
