Idles - Grace

British rock band IDLES have unveiled a surprise collaboration, bringing together the video from Coldplay’s classic song ‘Yellow’ with IDLES’ new single ‘Grace’ Commissioner John Moule approached production studio Joyrider with the project - posing the challenge of taking the renowned music promo (released in 2000) and making it look like Chris Martin was singing IDLES’ new track. In turn, Joyrider approached Stone Dogs to work alongside director Jonathan Irwin, harnessing our flame compositing and finishing expertise to help realise this amazing challenge. In a cyclic turn of events Brian Carbin, senior VFX artist and co-founder of Stone Dogs had worked on the original ‘Yellow’ video. Stonedogs VFX team of Brian Carbin, Dave Kiddie, Rufus Blackwell and Danny Coster were so deeply immersed in the process that they learned the differences between the nasolabial crease, philtrum, cupid’s bow, upper lip, lateral commissure, vermillion border and the lower lip - even finding themselves on occasion singing the lyrics whilst looking at a mirror in order to mimic the phenoms needed for convincing animated lip sync. Jonathan Irwin, director and AI technical lead, Joyrider, led weeks of research and development and refining machine-learning approaches saying: “To create the training set, we fed every frame of the original video into DeepFaceLab and ran for five million iterations, approximately one and a half months of processing time, running 24/7. At each stage of improvement, there would be backwards and forwards of new AI training runs and flame VFX work, tracking, extensions, clean up, more tracking, test composites, more AI, more clean up etc.” Wanting to make the video look more like Chris did over 20 years ago, the team were able to take the production to the next level and film the “real” Chris Martin in the band’s studio, singing the song at double speed (50fps) with three 4K cameras. The video was then played back and slowed down, adding an additional layer of subtlety to match the original plate. The filmed material was then Deepfaked further, helping to deliver a convincing AI performance of Chris Martin. In edit, there was a further challenge of how to cut the original one shot ‘Yellow’ promo into a shorter IDLES track, without losing the opening sea at night setting or the bright walking away end scene. Careful timing adjustments were needed to find the best fit for the IDLES lyrics to match Chris’ delivery as the footage is all in one take. A time/speed ramp effect was applied to a section of the film to bring the ‘Yellow’ and ‘Grace’ promos in sync. This speed effect was in keeping with the original creative narrative of time advancing. Additional time warp effects were positioned over certain points of the video, to take the viewer away from the original Coldplay ‘Yellow’ video and into an IDLES-esque world. Grade was undertaken by Stone Dogs’ colourist Mark Meadows, who seamlessly blended in the new deep fake faces with the original footage, matching flesh tones and grain without affecting the original promo. The video begins with the beach and sky being somewhat dark until sunlight rises at around the video’s midpoint, therefore the new AI derived face had to match the changing colour grade.