Photoshop tutorial: Use layer masks for seamless compositing of photos in Photoshop

Give your photo illustrations extra pop using advanced compositing techniques, layer masking and custom brushes in Photoshop.

Intro


It’s almost impossible to look at this image and not feel perked up: perhaps it’s the vibrant colour palette, the ingenuity of the original concept, and the polished effect that Vince Fraser achieves.

See also: 86 Best Photoshop tutorials

In this tutorial, you’ll learn how to create an astonishing image where the character appears to be gently breaking apart, using simple but effective techniques in Photoshop.

One of Photoshop’s great capabilities that is most often overlooked is the layer mask: when used correctly, layer masks can help you blend images seamlessly, giving a professional finish. You’ll learn how to combine layer masks with brushes and adjustment layers to add in subtle elements that complement the image.

Software used

Adobe Photoshop

Time to complete

2 - 3 hours

Project Files

Please visit the desktop site to download the project files.

Step 1

STEP 1

Download the free sunflower field image, and remove the sky using the Pen tool. Open Malta sky.jpg and copy and paste in the sunflowers. Resize and reposition them accordingly, and increase the canvas size so it’s twice as broad as before. Add a colour balance adjustment layer (Layer > Adjustment Layer > Color Balance) with the settings shown here.

See also: 86 Best Photoshop tutorials

Merge all layers (Cmd/Ctrl + E) then duplicate this, flip the duplicate horizontally, and align so that the layers meet at the centre. Merge all the layers again.


Step 2

STEP 2

Open sunflower model.jpg. This image was kindly provided by Abi Oshodi, and you should use it for this tutorial only. Make a selection around the portrait using the Pen tool (P) or Magic Wand (W). Zoom in to about 200% to get a clearer view.

See also: 86 Best Photoshop tutorials

Advertisement. Article continues below

Step 3

STEP 3

Once you’ve made the selection, right-click (Ctrl + click) on the layer and rasterize it (Layer > Rasterize), then select it using the Magic Wand.

See also: 86 Best Photoshop tutorials

With the selection active, click on the original layer with the woman cutout, and paste the selection of the model onto the background image. Position the model; you may need to scale her up or down slightly (Edit > Transform > Scale).


Step 4

STEP 4

Rename the layer with the model shot ‘summer model’, then add a layer mask (Layer > Layer Mask > Reveal All).  Now grab the Brush tool (B) and select a leaf brush from the default library settings (click on the brush icon in the top bar, then on the triangle in a circle to bring up a menu. Select Presets, then select Brushes from the Preset Type drop-down menu).

See also: 86 Best Photoshop tutorials

Select a leaf brush, and the following settings: diameter – 90 pixels; opacity – 100%; Flow – 100%. In the Brush Presets menu, ensure that Shape Dynamics and Smoothing are both ticked, set the Size Jitter to 50% and Roundness Jitter to 50%, and select Pen Pressure from the Control dropdown menu. Vary the diameter of the brush between 30 and 90 pixels to vary the pattern.


Step 5

STEP 5

To add more depth, we’re going to give the impression parts of the model’s headdress are breaking away in the distance, by painting in the layer mask area using the leaf brush. Make random single strokes cutting away parts of the headdress to the left of the model; use the screenshot for reference.

See also: 86 Best Photoshop tutorials

Remember to click on the layer mask symbol so that it’s active and set the colour of the mask brush to black so that strokes are clearly visible in the layer mask.

Advertisement. Article continues below

Step 6

STEP 6

Next, duplicate the background layer and add a Gaussian blur: select Filter > Blur > Gaussian Blur, and enter a radius value of 35.4 pixels. Add a layer mask (Layer > Layer mask > Reveal All) and using a large soft brush (200-300 pixels diameter, varying the opacity between 20% and 40%) paint into the active mask area using a black brush.

See also: 86 Best Photoshop tutorials

I’ve highlighted a rough guide in this screenshot to show the areas that should be left white using the Lasso tool (L).


Step 7

STEP 7

It’s time to add some more brush effects: select a large, soft, dark red brush, then double-click on the foreground colour swatch at the bottom of the tools palette and enter the value #53080d in the colour picker.

See also: 86 Best Photoshop tutorials

Now change the brush diameter to around 300 pixels with 20% opacity, and make some random marks behind the left side of the model where the headdress is breaking away. Next, add a Gaussian blur (see step 5) with a radius of 8.9 pixels, with the blending mode set to Overlay.


Step 8

STEP 8

Create a new layer, and repeat the process from step 7 with the same brush, making random spray marks – this time concentrate on just behind the right-hand side of the model. Keep the brush’s opacity low – around 20% - and slowly build up the colour.

See also: 86 Best Photoshop tutorials

Set the blending mode to Overlay. Repeat this twice more on new layers, with a white brush layer’s blending mode set to Screen, rather than Overlay. Make sure these layers are all below the ‘summer model’ layer.

Advertisement. Article continues below

Step 9

STEP 9

Create a new layer, and using the same leaf brush as before, start painting leaves, choosing an orange (I used #ff8f07). Again, take care to vary the brush size and opacity levels – use the larger brush sizes nearest to the headdress, and smaller ones further into the distance. Then add a Gaussian blur to the flowers (see step 5), with a radius of 10.1 pixels, setting the blending mode to Screen.

See also: 86 Best Photoshop tutorials

Again, make sure this layer is below the ‘summer model’ layer.


Step 10

STEP 10

Return to the original ‘summer model’ layer, then right-click (Ctrl + click) on the layer mask and select Disable Layer Mask. This will show you the full headdress, as before. Using the rectangular Marquee tool (M), click and drag a selection similar to that shown here. Copy and paste your selection into a new layer. You can now enable the layer mask for the ‘summer model’ layer that we disabled at the start of the step.

See also: 86 Best Photoshop tutorials


Step 11

STEP 11

Using the Distort tool (Edit > Transform > Distort), stretch the new layer until it resembles this screenshot by pulling at the vertices. Remember to cover the model’s body slightly, as this will be important in the next step.

See also: 86 Best Photoshop tutorials

Advertisement. Article continues below

Step 12

STEP 12

Now we’re going to give the impression that parts of the model’s headdress are breaking away and becoming leaves. Repeat the same technique described in step 4: add a layer mask (Layer > Layer Mask > Hide All) to the layer, and paint in the mask area using the same leaf brush. Vary the brush size between 20 and 70 pixels.

See also: 86 Best Photoshop tutorials

Again, don’t forget to use a white brush in order to see anything in the mask area. You may need to experiment a little with the brush size to get the precise effect. Match the parameters in this screenshot – you can also add jitter to the size and angle. Adding scattering also breaks up the regularity.

It’s crucial that you don’t interfere with the actual layer – only work on the mask.


Step 13

STEP 13

Using the same principle as in step 12, sample a portion of hair from a different part of the headdress and copy and paste it into a new layer. Stretch it using the Distort tool (see step 10), stretching it into a rectangular shape to cover a larger area. We add another layer mask to the layer (see step 11) but this time hide everything. With the mask selected, use the same leaf brush as before and paint strokes, varying the size as discussed in step 12.

See also: 86 Best Photoshop tutorials

Repeat this process with a new layer but leave the layer mode at 100%. Tweak the settings until you’re satisfied. Adding more layers will give the illustration extra depth.


Step 14

STEP 14

Create a new layer and start painting using your leaf brush (I used #6c061c), again varying the brush size and opacity level as before. Then apply a Gaussian blur with a radius of around 10.1 pixels to the leaves, setting the blur’s blending mode to Overlay. Again, make sure this layer is below the ‘summer model’ layer.

See also: 86 Best Photoshop tutorials

Advertisement. Article continues below

Step 15

STEP 15

This is the point where you tweak your image to make it exactly to your liking. Add some adjustment layers to create highlights, shadows and colour tweaks. Here are three different adjustments, with the brushed areas highlighted in the mask area. In the first, I adjusted the levels (Cmd/Ctrl + L) of an area of the model’s shoulder to create a deeper shadow.

See also: 86 Best Photoshop tutorials


Step 16

STEP 16

In the second, I tweaked the output levels of some of the highlights surrounding the model to make her glow more.

See also: 86 Best Photoshop tutorials


Step 17

STEP 17

In the third, I adjusted the hue and saturation to make the image redder.

See also: 86 Best Photoshop tutorials

Advertisement. Article continues below