Maya & Nuke tutorial: Composite a CG character into video footage seamlessly

The filmmakers behind the beautiful quirky short film Gloam reveal how they composited a CG creature into a real-world wood.

Intro


In this masterclass, filmmakers David Elwell and Gareth Hughes will show you how to recreate a shot from their short film Gloam, where a CG creature walks through a real-world wood. The beautiful film follows a horned wooden creature whose encounter with some fireflies leads to some strange experiences, leading up to a, well, you should watch the film above to find out.

We’ll begin by tracking our camera footage in The Pixel Farm's PFTrack, importing the match-moved camera into Maya and finally compositing our CG character into the live-action plate using The Foundry's Nuke

This tutorial aims to give you an understanding of a compositing workflow that you can use to create your own CG and live-action projects.

In the project files, you’ll find the clean plate (background footage) and rendered creature as image sequences.

Time to complete 

2 days

Tools

Nuke, Maya, PFTrack  

Download

Files for this tutorial are downloadable from here.

Step 1

STEP 1

We’ll start by tracking our footage in PFTrack. Create a new project and import your footage. Locate it using the File Browser, and then drag it across to the Tree View. Load it into the Cinema panel. You may prefer to use a compressed image sequence instead of the original Mov file, which can be memory intensive to cache.


Step 2

STEP 2

Before we begin tracking, check your Clip and Camera Preset settings are correct. In the Camera Preset tab, ensure the Film Back width matches that of the camera – we used a Canon 600D, which was 22.3mm.

Advertisement. Article continues below

Step 3

STEP 3

Right-click on your footage in the Tree View and add an Auto Track node. Increase the Candidate Number to 250, Target Number to 150 and Failure Threshold to 0.900. A good matchmove usually requires around 100 trackers, so by setting the target number higher, we can lose a few and still maintain a good solve. 

Keep the Search mode as Better Speed. This is less accurate than Optical Flow, but with this being a simple and smooth shot it should work well. We also compensated for this by increasing the failure threshold. Hit Auto-Track.


Step 4

STEP 4

Delete any faraway trackers or ones created by two intersecting objects as these can cause errors in the solve. Add a Camera Solver node to the Tree View. Knowing the focal length of the lens can help us get a more accurate solve – ours was 50mm. Hit Solve All.

To tidy up the track, go to the Errors tab. The thick white line indicates the average error. Anything below 1 is considered acceptable. Select Trim, adjust the line down to a value of 1 and then click on Refine All.


Step 5

STEP 5

Add an Orient Scene node to the Tree View. Using the Marquee tool, select a cluster of trackers along the path that we want the creature to walk along. Click on Set Origin in the Orientation tab, and put Set Plane to X-Z. Change the Edit mode to adjust the grid so it’s square to the camera.

In the Trackers tab, select two trackers that are adjacent along the Z-axis, and set the distance to 15. Our character is roughly 15 Maya units wide, so matching the path width to this will ensure our scene is scaled correctly. You may need to bump the grid size back up after this.

Advertisement. Article continues below

Step 6

STEP 6

It’s time to test our track. Add a Test Object node to the Tree View. Uncheck Re-centre and add a Tall Marker to the scene. Change the Interaction mode to Scale and make the Marker bigger (as we’ve unchecked Re-centre, the Marker will scale upwards from the base). 

Change the interaction to Place at Selected Feature, and position your Marker. Then duplicate a few more markers and place around the scene. Hide the trackers, and ground and play through the footage to see how well your markers are sticking. If they appear to be static in the scene, you have a perfect track.


Step 7

STEP 7

Finally, add an Export node. Keep the format as FBX, give your track a filename and export the scene.

It’s now time to head over to Maya.


Step 8

STEP 8

You may need to set up Maya to accept FBX files. Go to the Plugin Manager (Windows > Settings/Preferences > Plug-in Manager) and make sure fbxmaya.bundle is loaded.

Import (Cmd/Ctrl + I) your FBX file, loading in the camera and trackers from PFTrack. Select the camera, and lock the transform/rotate attributes so you don’t accidentally move it. You will also have to increase the Maya timeline to accommodate the frame range of the camera.

Advertisement. Article continues below

Step 9

STEP 9

At this point you would create your animation, rendering it out in the imported camera. We’ve provided you with the master beauty, occlusion and motion vector passes of the creature that we’ll use in the next section.


Step 10

STEP 10

Open Nuke. Select the Node Graph, then hit R in the Node Graph to create Read nodes to bring in the clean plate (your original footage) and the rendered CG you want to composite onto it.

Using a Merge node (M), connect the A pipe to the creature and the B pipe to the plate. This is our most basic composite. With the Merge node selected, press 1 to load its result into the Viewer.


Step 11

STEP 11

Add another Merge node and connect your occlusion pass, changing the operation to Multiply.

Advertisement. Article continues below

Step 12

STEP 12

Press S. In the Project Settings, set the full size format to that of our footage. Hit tab in the Node Graph and add a ShuffleCopy node to take the info from the motion pass and put in the footage’s Forward channel so you can use the info later for tricks such as motion blur.

As we rendered using V-Ray, our motion pass will require grading in order to work correctly. Add a Grade node (G) between the Shuffle Copy and motion vector pass, adjusting the blackpoint to 0.5.


Step 13

STEP 13

After the Merge node between the ShuffleCopy and occlusion pass, add a Grade node sandwiched between an Unpremult and a Premult node (so the transparency is preserved). This ensures our anti-aliased edges are adjusted correctly to give smooth, realistic edging around our character.

Using the Color Picker, hold Cmd/Ctrl + Shift and drag-select in the viewer, selecting a mid-tone range from the creature for the whitepoint and a mid-tone range from the plate for the gain. This will help shift the creature’s colour space into that of the footage.


Step 14

STEP 14

Still in the Grade node, we can make further adjustments to help blend the creature with the plate. Increase the Offset so the darkest areas of the creature match the plate. You can also play with the Mix level to adjust the overall effect the grade has on the original render.

Advertisement. Article continues below

Step 15

STEP 15

Motion blur can be a subtle yet vital effect to help sell a final composite. To add some here, first add a Vector Blur node. Change the UV Channel to forward, Multiply to 0.5, Offset to -0.5 and the Method also to forward, so that the motion blur goes in the right direction. Add another Blur node and set the size to 1. This will ensure any areas not affected by the motion blur don’t remain super sharp. 


Step 16

STEP 16

We need to rotoscope the trees in the footage’s foreground so the creative appears to walk behind them. Add a RotoPaint node and plug the Merge node’s mask pipe into it. Set the Merge mask to be inverted.

Using the Bezier tool, draw around the first tree our creature passes behind. Move along a few frames (using the right arrow) and realign the mask. You will have to adjust individual anchor points too. 

Repeat this process for the rest of the sequence and for the other trees. You can also adjust the mask feather in the RotoPaint node Shape tab.


Step 17

STEP 17

By now you should have a good composite working. At this point we took our semi-finished shot over to After Effects and applied a final stylised grade using Magic Bullet Looks. Also, the addition of grain and chromatic aberration adds subtle artefacts to your composite, giving it a more realistic finish. We then cropped the sequence into an Anamorphic ratio to give it that cinematic look.

Advertisement. Article continues below