If you’ve been following the BBC’s seven-part television fantasy Jonathan Strange & Mr Norrell, running in both the UK and USA, you’ll have no doubt been blown away by the often cinematic visual effects that have brought Susanna Clarke’s tale to magical life.
Set in a parallel history which features the likes of Wellington and Napoleon, it concerns the revival of magic to Georgian England and the men and women who both cause and are affected by its return.
Over almost two years, Milk VFX created 1,000 shots to deliver all the visual effects in all the TV drama. This mammoth task, more akin to a film production, encompassed everything from animated statues to elemental forces at the battle of Waterloo.
The company has just given us exclusive access to host the final VFX breakdown of video (above) for the last two episodes in the series, so we thought it was a fine time to look over some of the other fantastic work by Milk on this marvellous magical series.
Making magic real
Milk’s primary task was to make magic utterly believable by bringing to life the natural elements used by the magicians. Milk created an impressive body of effects work throughout the series, bringing to life stone, sand, water and mud as well as a flock of ravens.
Working closely with director Toby Haynes, the boutique studio designed and made standalone VFX sequences in each episode.
The Milk team spent several invaluable months doing R&D prior to and during the first months of the shoot for the key sequences. Milk collaborated closely with Haynes and producer Nick Hirschkorn prior to pre-production and throughout the process with the production designer and edit team.
This was in addition to the traditional work associated with a period drama, such as paint outs/clean up, set extensions and matte paintings, to set the scene and add scope and scale in this alternative Georgian world.
Opening shot of York Minster
The opening wide shot in Episode 1, when Mr Segundus leaves his lodgings in the Shambles in York, is the first time we enter the world of Jonathan Strange and Mr Norrell.
As an important scene setting shot, it sets the tone for the entire series evoking a heightened reality with a hint of the magical and dark fairytale world.
In order to overcome the logistical challenges of shooting in the Shambles, an alternative York Street was shot and recreated by Milk. This involved adding the entire left hand side of the street with overhanging buildings.
To accommodate the parallax in this crane shot, Milk created a complex hybrid environment shot with fully modelled, textured and rendered 3D buildings in the foreground, 2.5D environment in the mid-ground (achieved by projecting matte paintings onto geometry in Nuke) and a flat 2D matte painting in the background.
As the Milk team refined the shot, Toby Haynes wanted to include hints of the magical world, to portray Georgian England based in a heightened magical reality.
To achieve this Milk subtly bowed the buildings bringing a hint of 'Tim Burton style' into the world. The flat matte painting in the background with ‘god rays’ add a tonal hint of the magical world the viewer is entering.
The irony of the shot is that York Minster is actually behind the crane, owing to the shooting restrictions in the real Shambles
Moving statues in York Minster
The key VFX sequence in Episode 1 is Mr Norrell demonstrating to the York Society of Magic and therefore the world, his practical use of magic. He does this by bringing the stone statues in York Minster to life
Milk created 30 shots for the statue sequence.
"The challenge in animating stone objects is ensuring that they don’t look made of rubber," said Milk CEO Will Cohen. "It was also important to ensure that they looked like realistic carved stone, in keeping with the style of sculpture for the period."
"Through R&D we made use of cracks and dust to maintain the feeling of stone moving. It also became apparent that the cracks in statues faces could be used to enhance their performance. Photographic reference of the actual statues was collected on location in York Minster along with some artistic license to make fresh assets to put in situ – such as the Bishop or the gargoyles."
Milk created and animated 4 different types of statues, small statues on the top of columns, the seven kings, the musicians and the Bishop.
The Bishop was played by Bertie Carvel, the actor who plays Jonathan Strange in the series.
Toby Haynes invited him to perform the part of the Bishop in a green chroma key suit so that the actor playing opposite the Bishop had a real person to interact with for his motion captured performance – this was then used as reference for the animation of the Bishop character.
VFX of Jonathan Strange & Mr Norrell Episode 1
Milk created the fleet of rain ships conjured by Mr Norrell in the Port of Brest to hoodwink the French Navy, at the opening of episode two.
"This fabulous concept in the book and script was a challenge to realise," said Will Cohen. "The ships needed to be made of rain but effective enough to fool the French Navy sufficiently for them to get in a boat to investigate closely. We received basic sketches from the production, which we turned into concepts. The look we designed was best achieved by backlighting the ships so that they were silhouetted in the sequence."
The sequence was shot in a pond on location in Yorkshire against five 20m x 20m green screens. Milk replaced everything in shot except the actors in the boats, which travel out from the shore to investigate the fleet. The rain ships had to therefore stand up to close scrutiny.
First the Milk team replaced the pond water with sea and made it feel vast. They put raindrops on to the sea in order to create the feeling of being in the middle of a vast storm, which was done with a mixture of 2D and 3D. Our 3D team supplied ocean, rain and different variations of smoke elements.
The sea extension, waves and rain rippling were created in Houdini and subsequently rendered in Maya using Arnold. The rain, mist and rainships’ fog elements were created and rendered using Houdini. A deep compositing workflow was used to ensure the correct and efficient layering of complex elements together.
Concept artwork was supplied by the production. Milk then built full CG ships with galleon sails. They needed to feel like real ships from a distance but up close had to look like water – as if created from both the ocean and rain.
An organic fluid look was achieved with swirling effects in 3D and swirling displacement effects and interactive rain created in Houdini. Water effects, rain effects, smoke, mist and lightning were added by Milk’s effects team to help create the atmosphere. One shot had to track the actor’s hand as he reaches out to touch the rain ship, and the effects team added an interactive rain splash as he does so.
Milk created the heavy clouds and sky in 2D. The team were given various shot elements which they composited together to evoke a dark gloomy effect with a painterly look. The composition of the shot assisted this, with the horizon line occupying one third of the screen and the sky and rain, two thirds.
Digital matte paintings – London
Mr Norrell journeys to London and to the Houses of Parliament in Westminster Square. In order to tell the story of his journey and accurately set the scene in Georgian London, the camera passes through a London archway.
It gives a glimpse of the Bank of England and travels down Threadneedle Street, where we get an alternative view of the Bank and into Westminster Square.
“To achieve this and bring to life the bustle of busy Georgian streets we created matte paintings and 2D crowd replication,” said Will Cohen. “Matte paintings and set extensions also helped set the scene for the many of the characters entrance and exits into buildings across multiple angles in Hanover St and Soho Square.”
Milk worked in close collaboration with the Production Designer David Rogers – who is an expert on Georgian London - to ensure that details like windows and even the size of bricks were historically accurate.
VFX of Jonathan Strange & Mr Norrell Episode 2
Milk created the dark fairy world of Lost Hope which appears in several sequences. Haynes’ early idea was to create a dark, macabre world devoid of life, based on millennia of war debris stacked in layers on a battlefield.
Everything about the shot needed to evoke a sense of death and chaos: A bleak world based in a battlefield that has been used hundreds of times over millennia. Milk’s Grant Bonser created the concept designs.
Milk created the Road to Lost Hope environment in full 3D, from concept to final shot. The establishing shot - when The Gentleman takes Stephen Black through the woods towards Lost Hope - was created with a long camera-tracking move. Key elements were built and instanced for the environment including armour, skulls, layers and layers of 3D models, all piled on top of each other.
When the parallax lines up the image of a skull within, Lost Hope gradually comes into view.
Rolling mist, smoke, and volumetric lighting were added to create atmosphere, and the only real element in this shot was the actors and some trees.
The shot is used multiple times throughout the series, planned so that we would simply have to track and re-render each subsequent version,"revealed Cohen. "Multiple stand-ins were used for key assets such as towers, trees, one stand in for skeletons and debris, rotating and creating new versions of it. Sim cloth was created for flags. All 3D trees were modelled in Mudbox. We created tree extensions and two high-resolution trees that resembled gnarly creatures."
In Episode 3, The British army camp next to a windmill while searching for stolen artillery.
The windmill base was built and shot on location and Milk created a CG windmill from the base upwards. The team built a 3D model including some dynamic animation into the sails as the windmill starts to turn.
Milk also built CG interior mechanisms of the windmill, which are triggered by Jonathan Strange’s magic.
Milk also created a beautiful matte painting for the entire background in every shot showing the windmill, including the tree line behind it.
To add scale to the end of the sequence the team added army tents, a Spanish town and flames and smoke rising from the burning windmill.
VFX of Jonathan Strange & Mr Norrell Episodes 3 and 4
The Battle of Waterloo
Episode 5 opens to a spectacular sixty-second swooping aerial shot of Waterloo, panning over a smoke-covered battlefield filled with thousands of soldiers and cannon-fire to reveal the full scale of the war.
Milk worked closely with director Toby Haynes to design the sequence, pre-visualise the shot and get the camera move absolutely right in order to transition from full 3D to live action.
“Rather than just seeing twenty extras in the scene, with the main battle happening off-camera we wanted to get the full horror of the fighting,” said Will Cohen. “Milk has done similar scale shots for films such as Insurgent, so this represents a crossover to what can be done in high-end television in 2015. The shot was created over a three-month period.”
Inspired by battle scenes in Sergei Bondarchuk’s Napoleon – in which 40,00 extras were used – the Milk team began building the Waterloo sequence, using the film as a point of reference for terrain and textures.
The Milk team studied historical reference maps of the battle and Google maps of the area to ensure the terrain geography and formation of the soldiers were historically correct.
Once the camera move and length of the shot were locked down, the team worked on the choreography of the shot by blocking the soldiers’ actions using simplistic rigs.
Milk used Maya plug-in Golaem Crowd to create 50,000 soldiers in the battle scene. This was based on four different types of soldier identifiable by their uniforms and two types of soldiers on horseback. The team also created a variety of props including cannons, trees, bushes and houses.
All the props were researched to ensure historical accuracy. Each cannon for example, had five soldiers manning it. The production’s costume department provided the soldiers’ uniform references and Milk photo scanned each one.
In addition to the 50,000 digital extras, Milk created cannon fire and the crowd fighting and running.
“We used Golaem Ragdoll for the soldiers being hit by cannonballs,” said Cohen. “We created a library of canon explosions with projection of a cannon ball, done in Maya and Houdini. We also created glinting water for the puddles and created footprints for every person on the battlefield."
"Smoke, mud and atmosphere elements were simulated in Houdini and rendered in Maya. The clouds at the beginning of the shot were rendered in Mantra. Once we had created these assets and interaction we sought Toby’s input."
The Waterloo scenes are augmented by a number of environment extensions and matte paintings, including the final shot in which the camera cranes up to reveal a wide shot of the post battle carnage.
Throughout the series Milk’s brief was to bring to life a very visceral and naturalistic form of magic. Towards the end of the Hougoumont sequence in Episode 5, Jonathan Strange creates a giant mud hand to crush an attacking French soldier to death.
"The brief for the mud hand was to create a large scale, realistic hand made of mud that rises up from the ground and crushes a soldier to death," said Cohen. "Milk’s effects team began by matching the mud hand to the terrain at the location where the sequence was shot. Once we saw the plate we knew that the mud needed to be quite fluid, gooey mud."
During an R&D period the team were able to explore a variety of techniques to produce different mud qualities.
"One method involved allowing the hand to break apart dynamically, depending on which areas of the animated mesh articulated the most," explained Cohen. "This produced natural looking results but gave the impression of a dry, brittle material. The method that we settled upon was more heavily based on particle fluid simulations."
"The animated mesh was deformed in such a way as to give the impression of flowing mud, with viscous fluid particles being emitted from the bumpiest areas to produce falling chunks and streams. Extra displacement and colour variation was added at render time to create more detail and introduce a rain interaction effect."
Milk’s animation team built a hand asset, which lifts up and crushes the French soldier to death. The actor was filmed on a wire and the hand animation team had to accommodate his position with him leaning slightly forwards.
Milk also created and enhanced the rain that is falling throughout the sequence, as well as creating a dramatic water funnel effect and animated killer vines.
VFX of Jonathan Strange & Mr Norrell Episode 5
“All the Mirrors of the World”
Episode four marks the first time Jonathan Strange works out how to travel using mirrors into the world of the Raven king and the King’s Roads. To bring the journey to life Milk created the 3D environment for the world of the King’s Roads.
Milk created the concept with a brief from Toby Haynes for a look and style based on the work of the artist Escher – it needed to look endless, vast and run-down and also untouched for 300 years.
The scenes were shot at Kirkstall Abbey in Leeds.
This is also the first time we see people travelling through mirrors and Milk spent some time designing how this would work to establish the look. It was designed by the lead 2D artist Luka Leskovsek as a 2D effect using Nuke.
Towers and roses
Several VFX shots surround the mysterious Black Tower, which features in both episodes 6 and 7.
“In the book it’s left to your imagination, it’s like eternal night, so just having darkness wasn’t going to be terribly interesting in shadow,” said Cohen. “So we came up with this black tornado, an effect that was completed in Houdini."
In episode 6, Jonathan Strange also engages in a duel against The Gentleman (Marc Warren) in which they use leaves.
“The Gentleman summons all the leaves around him and there’s a powerful duel going on between them,” explained Nicolas Hernandez. “That was done in Maya with particles and instanced geometry.”
Another challenging shot involved a rose that emerges from Lady Pole’s (Alice Englert) mouth to show that she has been bewitched.
“There was a spell on her which makes her spout what people think is gibberish but it’s actually the history of fairytales," explained Cohen. "Tracking that rose was a real challenge - Lady Pole was wriggling her head around in horror on set, and you’ve got to track a rose and thorns emerging from her mouth in a realistic way!”
Swarming ravens and subsumed Gentlemen
The final episode propels events to a dark and dramatic conclusion. One of the major effects Milk created to support this was swarming and flocking ravens, which transform from books into birds, then later turn into The Raven King (Niall Greig Fulton).
“We used the crowd tools in Golaem Crowd for flocking,” Milk’s Nicolas Hernandez explained. “We created a complicated flight cycle of a CG raven, and used Golaem to cache the animation. We had to do a lot of match-moving and other work to get shadows working on the sets and correct integration.”
In addition to landmark-type visual effects work, Milk’s work in Jonathan Strange & Mr Norrell also featured large amounts of invisible shots – from set extensions, to clean-ups, and shots that solved important story points without impacting highly on the show’s budget.
For example, a shot of Strange and Norrell conjuring a rain portal was required. “The book describes a ‘bonsai cloud’,” explains Cohen, “and it starts raining in the story and this causes the opening of a portal to walk through. Very quickly we asked ourselves how we could tell this story without building a 3D rain cloud?”
“We figured we could put a layer of rain over the camera looking back at them, and they will run towards the camera and we’ll cut to the reverse and they’ll disappear. That’s the level of simplification we went to because we wanted to focus on the sequence in which The Gentleman meets his end by being subsumed by a tree.”
This occurs when after causing various forms of havoc during the series, the malevolent Gentleman meets his fate when he is swallowed by tree in Lost Hope. This involved the actor miming his demise, adding procedurally growing tendrils and also a digi-double of the actor for part of the shot.