Creating and Editing Vector Masks in Photoshop

Whereas layer masks let you create soft-edged masks, vector masks create hard-edged masks defined by shapes created by a vector path on a layer. Vector-based shapes produce clean, smooth, and well-defined edges that are never jagged. And you can size and transform vector shapes without ever degrading the appearance of the element. You create a vector mask when you create a shape with any one of the shape tools. You can also create a vector mask when you convert type to a shape (Layer-->Type-->Convert to Shape).

Adding a vector mask to a layer
To add a vector mask to layer, follow these steps:
1. Select the layer in the Layers palette and choose Layer➪Add Vector Mask➪Reveal All or Hide All.
I describe Reveal All and Hide All in the “Creating layer masks” section.
2. On the vector mask, create a path with the Pen tool or grab any Shape tool and create a shape.
I selected the Custom Shape tool and dragged a sunburst shape on my vector mask. Notice how everything outside of the path is hidden. Like layer masks, you can add vector masks only to layers, not backgrounds. If necessary, simply convert your background to a layer by double-clicking background in the Layers palette.
3. If your vector mask is satisfactory, save your file and then close it.
Another way to add a vector mask is to select your desired layer, draw a work path with the Pen tool or one of the shape tools, and then select Layer-->Add Vector Mask-->Current Path.

Managing vector masks
Here are a few more vector mask tips. You can perform the following tasks:
- Edit a vector mask path: Use the pen tools and the Direct Selection tool.
- Add multiple shapes or paths to the existing vector mask: All you need to do is just drag another shape with any of the shape tools. Or add another path with the Pen tool.
- Remove a vector mask from a layer: Drag the thumbnail to the trash can icon in the Layers palette or choose Layer-->Delete Vector Mask.
- Disable (temporarily hide) or enable a vector mask: Shift+click the vector mask thumbnail or choose Layer-->Enable (or Disable) Vector Mask.
- Rasterize a vector mask: Rasterizing (or turning the mask into a pixel-based image) converts the vector mask into a layer mask. Choose Layer-->Rasterize-->Vector Mask.
- Apply layer styles to vector shapes: This is a quick and easy way to create buttons for a Web page or a custom logo, as shown in Figure 3-5. Just select the layer, not the vector mask, and choose Layer-->Layer Style. Select your style of choice.

Creating Channel Masks
Photoshop’s channel masks are probably the most time-consuming masks to use because they require a lot of manual labor. Not heavy lifting, mind you, but work with the tools and commands in Photoshop. But, don’t get me wrong, it’s time well spent. Channel masks can usually accurately select what the other Photoshop tools can only dream about — wisps of hair, tufts of fur, a ficus benjamina tree with 9,574 leaves. There are lots of ways to create a channel mask, but I’m here to offer you one that works most of the time. To create a channel mask, follow these steps:

1. Analyze your existing channels to find a suitable candidate to use to create a duplicate channel.
This is usually the channel with the most contrast between what you want and don’t want. For instance, in my example, the Red channel provided the most contrast between the tree and lanterns, which I wanted to mask, and the background, which I didn’t. After you duplicate the channel, it then becomes an alpha channel.
2. Choose Image-->Adjustments-->Levels.
Using the Histogram and the sliders in the Levels dialog box, increase the contrast between the element(s) you want and don’t want selected.
3. Choose a tool, such as the Brush, and paint and edit the alpha channel to refine the mask.
My mask was pretty clean to start with, so it didn’t require much editing.
4. When you complete the mask, click the Load Channel as Selection icon (the dotted circle icon on the far left) at the bottom of the Channels palette. Then click your composite channel at the top of the list of channels.
This loads your mask as a selection, giving you that familiar selection outline. You can also use one of my favorite keyboard shortcuts: Ctrl+click (Ô+click on the Mac) directly on the alpha channel to load the mask as a selection. Your selection is now ready to go.
5. You can leave it within the original image or drag and drop it onto another image.
If you’ve done a good job, nobody will be the wiser that the two images never met in real life.