Dynamic Paper Cutouts
Posted by Lindsay Ruthven – December 14, 2009While stop motion can be a fantastic medium, the process is too slow for many productions. Even for simple elements, like wrinkled paper, the time it takes to do things physically is often untenable; for the Microsoft PowerPivot online advertising campaign, Vectorform required lightening quick turnaround on visual elements and animation revisions, often on an hour by hour basis.
This is how the paper cutout effects were designed and generated dynamically using Adobe After Effects.
Animation with alpha
For our example video, character animation was handled in After Effects. Having broken down the script and subsequent voice acting into paragraphs, the lipsync was animated in Papagayo (see Lipsyncing with Papagayo for more on this important aspect of the high-speed workflow) and the elements were animated in individual compositions for each clip.
In other cases, the animation may be a 3D rendering, from another application, or hand drawn. The biggest concern here is that the elements must have full alpha, with no backgrounds or half-transparent portions (antialiasing is good, but there may be issues with large patches of partial transparency once we start playing with alpha boundaries).
Paper textures
Depending on the desired effect, this can take all sorts of forms. For the Microsoft PowerPivot video, a sequence of crumpled paper photos were used along with frame blending in After Effects. This was slowed down (again, using frame blending) till the desired look was found. The final paper texture animation should be the same size and length as the other comps.
Using directional R/G/B lighting when photographing the crumpled paper could make it possible to develop false geometric distortions in After Effects, gaining a little more realism in how the imagery follows the shape of the paper. This tutorial, however, only covers the use of plain-lit paper textures.
Effects
The animation and paper compositions should be added to a new comp of the same size, and an adjustment layer and plain white solid should be created.
adjustment layer
Two expression slider controls allow us to globally control the strength of the displacement effects (to be applied to the animation and white solid layers), while the drop shadow performs a very basic role; add a little bit of lighting to an otherwise flat paper effect.
animation
Because After Effects’ Displacement Map effect is unidirectional, two are applied to help give a little more depth and variation. The displacement source for each effect should be set to the paper texture layer. Option-clicking on the animation badge (the stopwatch icon) lets us add expressions to the Max Horizontal and Max Vertical displacement values.
For the first Displacement Map effect, apply this expression to both Max Horizontal and Max Vertical:
thisComp.layer("Adjustment Layer 1").effect("Displacement Slider")("Slider")
The second Displacement Map effect needs this expression applied to both Max Horizontal and Max Vertical:
thisComp.layer("Adjustment Layer 1").effect("Displacement Slider 2")("Slider")
Lastly, a Curves adjustment is added to help the colours and values pop. Adding all the shading, textures, and distortions can start to muddy and dull the animation, so curves are used throughout to brighten and enhance the imagery.
Set the layer Transparency Mode to Multiply to apply it to the underlying layers.
white solid
This is the most complicated of the layers, with two main purposes; to expand and create the torn paper edges, and add depth to the underlying paper textures.
The Set Channels effect is important, as we use this to grab the alpha channel from the animation layer. Select layer 2 in the source layer 4 input, all the other settings can be left at their defaults.
Use Simple Choker to expand the alpha, creating the extra white space needed for the torn paper edge.
Just like the animation layer, add two Displacement Map layers with inputs set to the paper texture, and the following expressions applied to Max Horizontal and Max Vertical displacement values.
thisComp.layer("Adjustment Layer 1").effect("Displacement Slider")("Slider")
thisComp.layer("Adjustment Layer 1").effect("Displacement Slider 2")("Slider")
To finish the tearing of the paper edge, apply a Turbulent Displace effect and tune to taste. The amount, size, and complexity will all depend on the scale of the compositions and the desired style. Because the previous displacements are based on the animated paper, and the turbulence is based off of the new alpha shape, the effect will change automatically based on the strength and speed of the paper animation. Animating the evolution is possible, but typically adds far too much action and confusion.
After Effects’ Emboss is rudimentary at best, but once applied, we can add Channel Blur to soften the results into a usable effect (taking care not to blur the alpha). Matching the emboss relief value with the R/G/B channel blurs tends to work best. The direction is left at 0 degrees, giving a slightly concave appearance. This helps create the appearance of a slight page curl along the edges.
Lastly, a Curves adjustment is added to brighten up the output and improve the final composited tonality. Setting the layer transparency mode to Soft Light allows the emboss effect to adjust the underlying paper texture.
paper texture
Set the Track Matte to use the above layer’s Alpha, and the look is almost complete. In the example project, Levels were used to brighten the paper texture. This depends, of course, on the paper textures used and how dark the resulting paper crumple needs to be.
Final Compositing
Once completed, the composition is ready to be used in the final edit. For graphics that border the edge of the composition, there may be some artifacts left over from the multiple displacements. To fix this, either increase the size of the pre-comps to slightly larger than your output, or scale up the comp in the final video to cut off edges.
Watch the final PowerPivot video on YouTube or Vimeo to rate it or leave a comment!
Vectorform Applications at the German Chamber of Commerce and Industry
Posted by Lindsay Ruthven – December 9, 2009News from Vectorform Germany. “Neue Arbeiten” (“new kinds of work”) was the event theme of the IHK and Microsoft Event in Bochum. At this event there was a presentation of the MS Surface and the 24’’ and a 46’’ Multi-touch Citron Displays. The special focus of this event was how modern ways of cooperation and communication can sustainably change companies. Vectorform provided various solutions out of different market segments on MS Surface and the Multi-touch displays and revealed the future potentials that lie within the use of Windows 7 and Multi-touch applications.
Dipl.-Kfm. Gerd Pieper, President IHK Bochum, Achim Berg CEO Microsoft Germany
Tillmann Neinhaus, General ManagerIHK Bochum (from left to right)
Achim Berg CEO Microsoft Germany, Dipl.-Kfm. Gerd Pieper, President IHK Bochum, Tillmann Neinhaus, General ManagerIHK Bochum (from left to right)
Lipsyncing with Papagayo
Posted by Lindsay Ruthven – December 7, 2009Using freely available tools, lipsyncing a character in Adobe After Effects doesn’t have to be a pain. Papagayo is an open source app for breaking a script down into phonemes and syncing the pieces to an audio track, while LipSync is an OS X dashboard widget that helps translate the resulting moho switch files into After Effects keyframes.
The workflow was developed at Vectorform as part of the production pipeline for Microsoft’s PowerPivot online advertising campaign (to see how the torn effects were created, check out Dynamic Paper Cutouts).
Composition Setup
Papagayo uses the Preston-Blair phoneme set, so we’ll need to set up a pre-comp in After Effects with a single phoneme each frame, with (at minimum) the following, and in this order:
AI, E, FV, L, MBP, O, U, WQ, etc, rest
For the Microsoft PowerPivot project, a separate pre-comp was created for both happy and sad faces, and the comps switched out depending on the desired mood. However, the details are entirely up to you; a more extensive comp with a phoneme at every second and mood variations on every frame would also work, though it would require a frame offset setup to control the mouth expression. Fairly easy with basic knowledge of expressions, but not covered by this tutorial.
Audio prep
The final audio edit for this project was split into paragraphic chunks to make syncing easier. It’s all about breaking the process down into manageable pieces. Each piece was exported as it’s own audio file, and compressed using Quicktime Pro into a .wav file using LinearPCM set to output high quality stereo. Mono should work as well (some lipsync applications such as Yolo and JLipSync can work better with Mono), so it’s up to your preference.
Play audio clip » Play
Lipsync
Open up Papagayo and import the .wav audio clip. In OS X, there’s a bug with the Frames Per Second input field – you can’t type numerals. To get around this, open up TextEdit and type your desired fps, copy it to the clipboard, then back in Papgayo, paste it into the field using command+v. It’s important to do this before you start editing phoneme timing, as changing the value will completely reset script pieces to the default positions. If you have trouble with Papagayo not recognizing the correct length of your audio (for example, preventing script pieces from being dragged to the end of the audio), try pasting in a different FPS value, then the correct one again. This should force the app to refresh, correcting the audio settings.
In the Spoken Text field, enter the script. It’ll be helpful to enter a carriage return at natural breaks or pauses, as Papagayo will treat each line as an individual chunk of text. Numerals should be spelled out, otherwise the automatic phoneme dictionary will fail to recognize them.
“It all started with my second cup of coffee. Eh, no, my 3rd… it’d been a long Friday, when an email from the boss attacked.”
Should be written as:
“It all started with my second cup of coffee
Eh, no, my third
it’d been a long Friday
when an email from the boss attacked”
Once you have the script ready to go, select a language from the several listed under Phonetic Breakdown to automatically generate phonemes. If the phoneme translation engine encounters anything it doesn’t recognize, you’ll be prompted to enter your own phonetical spelling. It helps to be familiar with the Preston-Blair phoneme set and the specific mouth shapes your final animation will use, as phoneme shapes rarely line up with how a word is spelled or pronounced, but how an individual speaker says it.
Starting with the last phrase block, line up the beginning and end with the correct section of audio. You can either press the play button (which starts from the beginning), or scrub through manually (clicking anywhere in the audio window and dragging back and forth). Give yourself enough frames at the start and end of each clip so as to allow for a little wiggle room. Rearranging the sentences later will undo any fine-tuning you’ve completed in the word and phoneme levels, so getting each level worked out in the proper order helps prevent lost work.
Once the sentence fragments are timed correctly, go through and align each word. You can double click any block to play the corresponding audio, and right click any word to edit the phonemes currently used. With judicious use of phonemes, you may not even have to manually adjust them, but inevitably you’ll need to tweak a few phoneme positions as well.
Typically, a phoneme should start a frame before the corresponding sound. Punctuated phonemes such as “P” or “B” can often start several frames or more ahead of the sound, and consonants sometimes need an extra frame or two to register with the viewer (flashing a consonant for just one, maybe two frames is often not enough time). An “M” sound may seem insignificant, but the pent up pressure is essential to selling the vocal animation. Depending on the recorded audio and your character illustrations, “AI” and “E” may need to be interchanged.
For the example audio, Papagayo inserts a lot of “AI” phoneme shapes where “E” would be more appropriate, and vice-versa. Edit the phonemes to match what the audio sounds like, not what the words actually are.
That said, there are innumerable articles, books, classes, and degrees in animation, and I am no character animator myself. This is a starting point, but hardly an introduction to the rich art of animation!
Once the alignment is polished and the preview animation in Papagayo meets satisfaction, you can export the data as a Moho switch file (.dat). This essentially defines a simple list of frames and the associated phoneme shape changes.
Conversion
Using the Lipsync widget for OS X, this is the easiest part. The defaults are set up for Lightwave, so once installed, change the format to AE frames (or seconds, depending on the pre-comp setup), and you’re ready to go.
To convert, simply click and start dragging the .dat lipsync file while opening Dashboard (usually the f12 or f4 key). Drag it into the “drop MOHO file here” zone and let go. It’ll be automatically converted into Time Remap keyframes, and the results both copied to your clipboard and saved to your hard drive (placed alongside the original .dat file).
Final output
Back in Adobe After Effects, open the main animation comp, move the playhead to the begining of the comp (or more specifically, to the beginning of the audio clip) and select the phoneme pre-comp. Paste the keyframes copied by LipSync (command+v), and Time Remap will automatically be applied to the layer. Select all of the keyframes (easiest way to do that is to click the Time Remap title), right-click, and choose Toggle Hold Keyframe, otherwise the phoneme shapes will bounce back and forth, instead of holding till the next one is set.
Make sure it’s in sync with the audio, and you should be all done. Here’s the final video, completed by Vectorform for the Microsoft PowerPivot online advertising campaign.
Watch the final PowerPivot video on YouTube or Vimeo to rate it or leave a comment!