vc022: Projection Mapping the Piano Corner

Video #22 of my #100videoclips project explores projection mapping on existing objects, in this case, my piano corner. I wanted to see how many small individual objects I could do projection mapping on. This is the result.

The Inspiration

My main inspiration is probably Jack Conte and his projection mapping videos, such as this one. He doesn’t project on household items, but I like how his technique is so transparent.

What I Did

These basic steps require a camera and projector, Photoshop, AfterEffects, and Resolume Arena. I try to give enough detail to figure out what is going on, although some of the steps may be incomplete.

  1. Prepare your scene to be projected on.
    1. In my case, that involved adding a few more white objects to the scene, or covering some things with white so they would make a better projection surface.
  2. Take a photo of your scene, from a position that is as close as possible to where your projector will be.
    1. In a perfect world, your camera lens would be the same focal length as the lens on your projector. However, you may not be able to manage it.
    2. If the lens focal lengths are very different, but if your projected surface is relatively two dimensional, it won’t make much difference. The more three dimensional your projection surfaces are, the more problems will be caused by different focal lengths because of the angle of view.
  3. Resize your photo (in Photoshop) to the native resolution of your projector, e.g. 1920 x 1080 pixels.
  4. Import the photo into After Effects for the initial projection mapping.
  5. Drag the photo onto the New Composition icon to create a new comp.
  6. Change the comp settings (Cmd+K) so the comp is the right length, based on your audio, e.g. 2 minutes.
    1. You may also want to change the fps (frames per second) of the comp. 30 fps is a nice rate because it is easy to synchronize music that is 120 bpm.
  7. Drag the comp timeline bar to be the full length of the comp.
  8. Lock the timeline for the photo (click on the lock).
  9. For your projection, you will often want either a video or just a solid surface that will change colors and have other effects applied to it.
    1. If you want a video to be projected onto one of your surfaces, add the video as a layer to the comp.
    2. If you just want a solid surface of changing colors, create a new solid layer, giving it a color other than black or white.
    3. Rename each layer after you create it so it easy to identify (right click on the name, choose Rename at bottom of the list).
  10. To create a mask for the video or solid surface to conform to the shape and size of your projection surface:
    1. Select but hide the video or solid layer.
    2. Zoom into photo (scroll wheel on mouse), and use the hand tool (H) to move around to the spot that has the area you want to mask.
  11. Mask the object with the pen tool (G), drawing around the object you want to receive the projection.
    1. Click with the pen tool around the edges of the object.
    2. To draw a curve, click and hold, then drag in a direction until you can see which way the curve is being formed.
    3. Hold down the space bar to move around the object (Hand tool) if you are zoomed in too far to see all the edges.
    4. Keep clicking until you complete the mask back at its starting point.
    5. If you break the mask, use the select tool (V), click outside the mask, then click the Pen tool (G) again and click on an end of the pen line.
    6. Use the selection tool (V) to click on individual points and move them or adjust the bezier curve by grabbing the bezier handles.
  12. To reveal only the inside of the mask, open up the layer in the comp, open up the mask, and click on Inverted.
  13. If you need to conform a video or layer to an irregular but basically rectangular shape, use Distort, Corner Pin effect.
  14. To loop a clip, right click on the clip in the project window, Interpret footage, Main, then specify the number of times to loop the footage.
  15. Once you have all of your project surfaces masked, you may want to add a black layer under all of them, so the non-projected area will be as black as possible.
  16. Once all of your projection surfaces are masked, render the composition, import it into Resolume, and project the video through a projector that is set up approximately where your camera was when you took the picture.
  17. In Resolume, use the Advanced Screen Setup to refine the projection mapping onto your surface.

What I Learned

  • For the actual projection, an adjustable projector stand (which I didn’t have) would be extremely helpful– truly adjustable to any reasonable height and angle. I had to pile up books on furniture, and never got it quite right.
    • My projector has a camera-standard (1/4″ 20) threaded mounting hole on the bottom, so in theory it could be mounted to a tripod. Unfortunately, there isn’t enough room around the hole to mount a tripod plate, so it didn’t really help.
  • For multiple items in a scene, doing individual slices in Resolume for each item is helpful.
  • If you are projection mapping small sections of a larger scene, you may need same ambient or supplemental general light to light the scene.
    • You need to balance being able to see the overall scene and maximizing the contrast of the projected portions.
    • Too bright means the projections are harder to see and are less impressive; too dark and you can’t see the scene.