Tutorial: Basic Audio Visualizer with Quartz Composer


This tutorial will help you set the foundation patches needed to create audio reactive visualizations with Quartz Composer in Leopard. The visualizations can be used with iTunes or any other software conforming to Apple’s Music Visualization Protocol. If you prefer to use audio input from the system mic or line-in, you will need make changes as said at the end of this tutorial. This is necessary if you want the effects to be mixed using a VJ software. The popular VJ software VDMX however offers a simpler way to do this - which is the Audio Analysis plugin found in VDMX being used to maniplate Quartz Composer parameters. You can also download the tutorial project from the link at the bottom. Please note that this composition does not create much eye candy by itself - it is just a base which can be used with various other rendering patches in Quartz Composer.

Audio Extraction and Preparation

  1. Start with a blank project. If you choose the visualizer template instead of the blank project, you will need to delete more patches to get started. So just start with a blank and follow.
  2. Go to Editor -> Edit Protocol Conformance. Choose Music Visualizer and click “Done”.
  3. You will now see some default patches in the editor window. Keep the patches that say “Required” and delete the optional ones. The required patches would be the Audio Spectrum patch and the Audio Peak patch.
  4. Add a Clear patch. You do not have to modify it or connect it to anything. The important thing is to keep the Clear patch on Layer 1. This can be done by clicking the number on the right top corner of the patch and selecting “Layer 1″
  5. Add 3 Structure Index Member patches. A Structure Index Member patch is used to extract one element from a “structure” output (or an array output) - such as all the audio frequencies and their respective output levels published by the Spectrum patch. Each Structure Index Member patch will be used to extract a single band from the Spectrum patch’s output. So, one of these 3 patches will be used for a bass frequency, one for a mid level frequency and the last one for a high range frequency (kick, snares and hi-hats).  Rename the patches “Low”, “Mid” and “High” for easy identification.
  6. Connect the output of the Audio Spectrum to the Structure property of “Low”, “Mid” and “High”. The output can be connected to multiple patches.
  7. Modify the Index properties of “Low”, “Mid” and “High” as 0, 4 and 11 respectively. These numbers represent the frequency band. The Spectrum outputs 16 independent bands as follows:
    • 0 - 30Hz, 200Hz.1 - 800Hz
    • 2 - 1.15KHz
    • 3 - 1.6KHz
    • 4 - 1.75KHz
    • 5 - 2.6-3KHz
    • 6 - 5.15KHz
    • 7 - 7.2KHz
    • 8 - 9.6KHz
    • 9 - 12KHz
    • 10 - 14KHz
    • 11 - 16.5KHz
    • 12 - 18.8KHz
    • 13 - ?
    • 14 - ?
    • 15 - ?
  8. Add 4 Smooth patches. Rename them “Volume Smooth”, “Low Smooth”, “Mid Smooth”, and “High Smooth”.
  9. Connect the Value property of the “Volume Smooth” to the Output property of the Audio Peak patch.
  10. Connect the Value properties of the “Low Smooth”, “Mid Smooth” and “High Smooth” patches to the Member properties of “Low”, “Mid” and “High” Spectrum Index Member patches respectively.
  11. Set the Increasing Duration and Decreasing Duration of “Volume Smooth”, “Low Smooth”, “Mid Smooth” and “High Smooth” as 0.01, 0.1, 0.05 and 0.01 respectively. These may need to be modified later to increase or decrease the intensity of the effects created by the respective frequency band. Lower values will result in faster changing effects and higher values will result in slower changing effects.
  12. Add 4 Math patches. Rename them “Volume Amplifier”, “Low Amplifier”, “Mid Amplifier” and “High Amplifier”.
  13. Set the Operation of all the Math patches to “Multiply” and set the Operand values as “2″. These may have to be adjusted later for your liking to increase or decrease the intensity of the effects created by each frequency band. You need these because the actual output values of the spectrum bands are quite low and may not be significant enough to make changes to rendered objects and effects.
  14. Connect the Smoothed Value property of each of the Smooth patch to the Initial Value property of the respective Math (Amplifier) patch.
  15. Audio extraction and pre-processing parts are now done.

Basic Visualization Example

To see how the audio extraction setup above works, continue with the following steps.

  1. Add 3 Sphere patches
  2. Connect the Z Position of the 3 Sphere patches like this: one to the “Low Amplifier” Resulting Value, one to the “Mid Amplifier” and finally the last one to the “High Amplifier”.
  3. Set the X Position of the 3 Spheres as “-0.2″, “0″, and “0.2″. This is to arrange the 3 spheres nicely in a row.
  4. Save the composition.
  5. Move the .qtz file to /Library/Compositions.
  6. Restart iTunes and load the visualization which will now be available in the list of iTunes visualizers.

One sphere will react to the kick drum and bass, one will react to the snares, toms or vocals and the last one will react to hi-hats and crashes.

Important Notes

When you have the visualization enabled in iTunes you will also be able to see it with Quartz Composer and make changes. The changes will be visible immediately in the Quartz Composer’s output window, which is a great feature, but iTunes will have to be restarted for the changes to be visible. The Smooth patches and Math (Amplifier) parameters in this example are optimized for electronic music. Adjust them while you monitor. Also, you will notice that output of the “Volume Amplifier” patch is not used with the spheres. But they work the same way as the rest of the Amplifiers and can be used in your other compositions. As a final note, this setup can be combined into a macro and used with other rendering patches available in Quartz Composer. I hope to publish more ‘prettier’ compositions using this example as a base.

Alternative Audio Input

If you want to use the system input (system mic or line-in) as an audio source instead of iTunes, you will need to add an Audio Input patch and remove the Audio Spectrum and Audio Peak patches. You can then connect the Spectrum output property of the Audio Input patch to the Smooth patches and connect the Audio Input’s Volume Peak output to the “Volume Smooth” patch. You will need to increase the amplifiers’ operand values, and remember that the audio output values provided by the Audio Input patch will also depend on the volume level of the audio source set in your system preferences.

Download

Feel free to download and use this composition. Any suggestions are welcome. AudioReactiveEffects.qtz.zip

Posted November 20th, 2008 in category quartz composer.