Recently we’ve been in contact with Alioscopy, a French company that develops auto-stereoscopic 3D displays that can produce a sensation of depth without needing cumbersome eyewear and they asked us if we could bring live VJing to their 3D screens.

The problem is, even if you are only mixing footage that is compatible with 3D projection you are still missing live visualizers which are, most of the time, a very important part of the mix and which obviously can’t be pre-encoded for 3D projection..

Alioscopy’s 3D display range currently includes 24″, 42″ and 47″ screens integrating a technology that is based on standard Full HD LCD displays but equipped with an array of lenticular lenses. Each lens overlaps 8 different points of view and casts different images onto each eye, allowing for one only to be seen through the lens from any given angle.

Thus to recreate this feeling of depth, a 3D scene needs to be rendered from 8 different points of view and combined to one auto-stereoscopic image. To achieve this, we developed an easy to use auto-stereoscopic Quartz Composition template wherein you can transform your original Quartz Composer composition to a obtain real 3D sensation on an Alioscopy screen.

Integrating support for Alioscopy in GrandVJ through a Quartz Composition template was a logical step since GrandVJ now fully supports Quartz since version 1.2. – and of course, the same integration will also be supported in MediaMaster – But the big advantage of a Quartz template is that it’s fully customizable by the user; you can fine-tune the depth effect exactly the way you want!

When edited in Quartz Composer, the template appears as a layer above the original composition. Descend the patch hierarchy, paste your scene, adapt the auto-stereoscopic parameters depending on your scene and you’re done. The 3D-enabled composition can be used and mixed as any other Quartz Composition in GrandVJ but of course, it must be output to an Alioscopy screen to see the magic happen.

Now we just have to decide how we are going to go to the market with this new development but you should read more about this very soon..

Quartz Composer is a free graphics and imaging tool on the Mac platform which is available as an optional install that can be found on your Mac OS X DVD since version 10.4.

It allows very easy creation of complex audio-reactive animations or effects which can now be played in GrandVJ (read the previous article about GrandVJ and Quartz Composer). As I’m writing these lines, the last version of Quartz-Composer is 4.0 on Mac OS 10.6 (Snow Leopard).

In this article we will demonstrate how to quickly create a bass-reactive Zoom/Blur effect like the one you can see below:

Important: This tutorial requires that you have some basic knowledge of Quartz Composer: if you have never used it, you really should first go through the “Getting Started”. You will also find some helpful readings at the Apple Developer Connection web site about Quartz Composer.

To create our effect we will simply edit Quartz-Composer’s default template named “Image Filter” to make it audio reactive. Basically, we will connect bass levels of the computer’s audio input signal to the “Amount” parameter of our Zoom/Blur filter. (Read more below..)

• Start the Quartz-Composer.app which is located in /Developer/Applications/

• From the menus, choose (File –> New from Template –>) Image Filter :
  

• Here’s the patch that will open:

• In the menus choose Editor –> Edit Protocol Conformance

• We will now tell the ArKaos software what inputs must be transmitted to the Composition. The ArKaos engine can send visuals or sound for processing by the Composition but only if he knows that these feeds will be accepted.
  To enable this you must check “Music Visualizer” and click “Done”

• You’re back to the Composition and several input modules have been added on the left:

• Remove every unnecessary input modules, so they won’t be listed as input parameters in GrandVJ‘s effect parameter panel:

• Quartz composer can hide blocks inside blocks, it’s called macros. It’s very powerful and in this case it holds the most important part of our composition: the processing part, which will affect the visual before copying it to the output.
  Double click on the “Process Image” macro to open it, Quartz Composer details each block contained in the macro:

• We want to connect the audio volume to the “Amount” input of the “Zoom Blur” filter contained in this macro. To do that we need to make the “Amount” parameter available in the macro block so that it can be accessed from the top level, it’s called publishing an input.
  Publish the “Amount” input as in this screenshot:

• When it’s done, we can go back at the top level to add some processing.
  In the top bar, click the “Edit Parent” button to go back the parent patch:

• Now we will add a processing block to create a value from the sound input.
  From the Library, add an “Audio Processor” module:

• You’re back to the patch with the “Audio Processor” module added. The magical thing about Quartz Composer is that you can connect the modules in many different ways, when you add a module you just have to tell Quartz Composer how to feed the inputs and connect the outputs.

• First we want to process the audio received from the ArKaos engine. To do that we connect together the “Audio Spectrum” and the “Audio Processor” module:

• Now the audio processor will create three values from the audio we are feeding, Lows, Mids and Highs. For this effect we just want to use the lower value which is the one reflecting the bass volume. So we connect the Lows output from the “Audio Processor” module to the “Amount” input we just published in the “Process the Image” macro:

At this point we need to get an audio signal so we can test our effect within Quartz Composer. For this purpose we will add an “Audio Input” that we will disconnect in the end so GrandVJ can input its own audio signal into the “Audio Processor” module.

• From the Library, add an “Audio Input“ module..

• ..and connect its “Audio Spectrum” output to the input from “Audio Processor”

Now we can see the image reacting to the audio input signal, but the effect is definitely too low, so we need to boost the audio signal values. A simple way to do that is to add a new processing module, specifically a math module:

• From the Library, add a “Math Module”..

• ..and configure it to operator Multiply, set Operand #1 to value 100:

• Connect the Lows output from “Audio Processor” to “Multiply” then connect “Multiply” to the “Amount” input from “Process the Image”

• You can now disconnect the “Audio Input” from the “Audio Spectrum” so that, once this composition is loaded in GrandVJ, the software will automatically send the audio input signal directly to “Audio Spectrum”.

• Finally save the composition as “MyBassZoomBlur.qtz” in /Users/MyAccount/Library/Application Support/ArKaos/GrandVJ/Quartz Composer/Effects/

Now is time to start GrandVJ and find your hard work rewarded by playing with your very own brand new effect. You will find it located in the Effects panel under the “Quartz-Composer (User)” effect category.

Notice how GrandVJ has made available the X and Y effect parameters from the Composition in the Effect parameter panel so you can use them to control the behavior of your effect in real time.

Now, that wasn’t hard, was it? If you want to go further, a good start will be to create new effects based on this Composition by replacing the “Zoom Blur” effect by another filter.

Don’t hesitate to share your creations on the forums :)

We have optimized the Quartz Composer integration in GrandVJ 1.2 so that the compostions are rendered natively through the fastest available interface.

Finally, when using a composition as an effect the application will scan the published inputs and map them as effect parameters, if the format is compatible with the engine. GrandVJ can automatically map parameters that are float values, indexes, colors and booleans and there’s one visual input of the composition that receive the cell visual.

Now let’s have a quick look at how you can use Quartz compositions with GrandVJ..

Using compositions as sources

The quick way to use a composition as source is to import it just like you do with any movie, by drag and drop on a cell or by reaching your .qtz through the file browser. When doing so, GrandVJ will save the path to the .qtz file in your project as it does for any other visual.

For your convenience, GrandVJ installs a set of source compositions in “/Users/<username>/Library/Application Support/GrandVJ/Quartz Composer/Visuals”, these are listed in GrandVJ under the “Quartz-Composer (User)” section of the sources browser so that they are always available when you create a new project.

The idea is that you can also add your favorite source compositions there so that you always have them ready when you work on your projects.

Playing compositions as source does not allow modifying the variables from within GrandVJ.

Path for user compostions as Sources	User compositions as Sources

Using compositions as effects

System compositions as Effects

For effects, GrandVJ will add all the compositions from your system that are supported. It does it by automatically scanning the system folder “/System/Library/Compositions”. These compositions will appear in the application under the “Quartz-Composer (System)” section of the effects browser. You can use them as starting point to develop new effects. You can also add your own compositions as effects, just drop them in the local directory “/Users/<username>/Library/Application Support/GrandVJ/Quartz Composer/Effects”. They will appear in the “Quartz-Composer (User)” section of the effect browser.

Path for user compostions as Effects	User compositions as Effects

About the parameters

As a quick example, let’s have a look at the effect parameters from the system composition Image Hose. With the Quartz Composer application (found in Developer > Applications – you need to have installed Apple’s developer tools of course), open the file “/System/Library/Compositions/Image Hose.qtz” (ore better, open a copy) here’s what you should see:

Image Hose composition

Open the inspector (Editor > Show Inspector) and from the menu, look at the Published Inputs & Outputs, here’s what you have:

Image Hose composition parameters

In GrandVJ, the first 4 parameters that can be mapped automatically will be displayed. In this case it will be PrimaryColor, SecondaryColor, Pace and Size:

Quartz composition parameters in GrandVJ

As you can see, GrandVJ will only display the meaningful part of the parameter name and remove all _ProtocolInput strings from parameter names – this allows to easily display the full parameter name in the panel – it will also make things a little nicer by capitalizing the first letter of the parameter if necessary.

There are many resources on the web where you can find information and help about creating your own Quartz compositions, you can start by looking at Apple’s Developer Connection.

Note: you can download GrandVJ 1.2 beta 2 from our forums

As promised, here’s the first in a series of tutorials showing how to take advantage of the Flash capabilities within ArKaos GrandVJ. Note that it will also work for NuVJ and MediaMaster since the variables are the same.

In this tutorial I’ll explain how to tap into the EQ data to create some simple audio-reactive animations. We’ll be using some basic ActionScript 2.0 for the inexperienced Flash programmer. I won’t go into much detail on the how the code works but it should be enough of an introduction to allow some experimentation in creating new content for your VJ arsenal.

You can download the final .fla and .swf files here or follow along the video tutorial below.

The genlocking we refer to is the action of locking the frequency of a media to a reference signal or clock (check out the related wikipedia article for a complete explanation).

Let’s examine quickly the process of displaying frames. Roughly, 3 steps must be acheived:

1. get video frames from the disk
2. upload them to the graphic card
3. do the composition / blending of the layers and  present them to the display

Now let’s describe how these steps are achieved without genlock and with genlock:

Without genlock:

At each new monitor frame, the software finds if a new movie frame needs to be processed. If so, it starts working sequentially through the 3 phases. The problem is that the time necessary for a codec to read the data from the disk and convert it into a decompressed frame is not constant and might, in some cases, even be quite lengthy. Mutliply this by 8 (once for each layer), add composition and effecting to it and you end up with a process that is highly variable, and might sometimes take more time than a monitor frame (typically 1/60th second), which creates fluidity problems.

Let’s illustrate this process with a graphic to make it easier to understand; in a classical real-time video processing application this will work like this:

The above graph shows an application trying to play a video loop encoded at 30 fps on a monitor running at 60 fps. In a perfect world the application should present each frames of the movie exactly twice.

There are 2 problems with this “traditional” way of doing the video processing:

• The time base of the computer is different from the time base of the graphic card. This means that even an extremely powerful computer will display video with small hiccups once in a while because the time you use to compute movie frame is not exactly synchronized with the time it’s going to be shown, introducing a jitter effect.
• When the software start working to display a new frame it has only in this example 1/60th of second to read a video frame, upload it to the graphic card and display it. The available time depends on the monitor fps and not on the video source fps. Which means that, the higher the fps of the monitor, the more stress you give to your system.

With genlock:

The processing done by MediaMaster 1.1 is much more efficient. Three different engine accuracy modes are now available to the user in the preferences dialog: “Minimal” was the default mode in MediaMaster 1.0, “Frame Blending” another new option in version 1.1 which we will discuss later and “Buffered”, which is the subject of this article.

In Buffered mode, the software keeps a predictive buffer that stacks frames that will be needed in the future. As soon as a frame has been decoded, it evaluates what are the next frames that are going to be needed later and queues their decoding directly. This process is running in the background and is separated from the monitor lock needed to present frames without tearing. Whenever the software needs a frame to display, it takes it from the buffer that was prepared upfront.

Also, in buffered mode the content syncs with the monitor frequency rather than with the computer clock. As far as I know, very few other media players are capable of doing something as subtle as this.

So, if we keep the same timing as in the first example but adapt our graphic according to the way the buffered mode is working, here is the result:

Because the movie clock is genlocked to the monitor and because the software always have minimum one frame in advance, the movie plays with a perfect regularity and timing : 1 1 2 2 3 3 4 4 … each frame is played exactly twice.

Why is it so cool?

If you are still not too sure why it’s nice to be genlocked at 60 fps here is a real time recording of the output from MediaMaster running 2 layers (captured with Fraps). The lower layer with the speakers runs at 30 fps while the top layer is a pattern that runs at 60 fps, this one is part of our test loops, it’s a ramp that allows to see very easily if the system is perfectly genlocked.

Most importantly, when applying effects to your content these effects will be rendered at 60 fps so with a perfect synchronization, your eyes will have the feeling that everything is crisp and fluid.

In order to see the videos in this article you need to have QuickTime installed. Of course, ensure your monitor frequency is set to 60 fps and the video below should play perfectly smooth. It’s an original screen capture of the MediaMaster output in 60 fps, it has just been resized to fit in this article.

Now in order for you to be able to see the difference here is the same loop at 30 fps:

Let’s continue to lower down the fps, here’s the 20 fps version:

And finally the 15 fps version:

Try for yourself!

If you are curious, you can test this for yourself, don’t hesitate to stress MediaMaster 1.1 when it’s out but also why not compare and test with any other software video mixer.. For your convenience, you can download our 60 fps test loops as QuickTime movie files further below.

For best results you should let them loop for a while. In a VJ application you can simply add them as the latest layer of your composition with an addition blending and you will see if your system is powerful enough and your software correctly coded.

You can download MediaMaster from our web site.

Scott [amoeba], a long time ArKaos user (since 1998!!) has just published hot news on his web site theestateovcreation. If you don’t know [amoeba] yet, it’s really time to check it out and discover his very unique style.

[amoeba]‘s creation/mixing process involves a lot of different tools such as After Effects or Ableton Live and various techniques like typography and animation generally mixed an synced together whit GrandVJ.

In the video below he’s been using Ableton MIDI patterns to control and trigger micro QuickTime movies that he created with After Effects edits and constructivist motion graphics with synched audio through ArKaos software.

Fore more inspiring VJ work by [amoeba] check out the rest of this post..

3 of 10 music videos and a/v set for Politica world tour

amoeba v politica – electronic police state on Vimeo

4 burlesque films for Enamore

Creative music videos for various other clients

July 21, 2009
5:00 pm to 8:00 pm
Rich Media Institute
525 Venezia Ave.,
Venice, CA, 90291

How to create interactive Flash animations for GrandVJ?
Learn with the pros and raffle a free license!

ArKaos is pleased to sponsor La-Va.org’s Flash Video Mixing Workshop: an afternoon of lectures, live demonstrations, and discussions of programming techniques, platform quirks, and methods of creating interactive Flash visuals in a live setting.

Flash animation has become a weapon of choice in a VJ’s arsenal due to its small file size, flexible scalability, and (most importantly) the potential for interactive programmatic elements.

Ben Guerrette (VJ ECIN – creator of Trigger Motion), will demonstrate developing audio reactive Flash visuals for use with Arkaos GrandVJ. Ben has worked with ArKaos to integrate Flash into GrandVJ and the Numark NuVJ in addition to providing stock Flash visuals included with both releases.

Eric Medine (MkUltra) will be demonstrating the interactive and dynamic text capabilities of Flash as well as MIDI-reactive Flash animation.

Following the lectures and technical demonstrations will be a raffle for a licensed version of Arkaos GrandVJ and Resolume v3, and a live Vj battle between MkUltra and ECIN.

Check out the announcement on the Lav-Va.org Blog for more info!

About LaVa (Los Angeles Video Artists)

La-Va (Los Angeles Video Artists) is a loosely organized communtiy of video artists working primarily in the field of live video mixing and improvised visuals. The goal of the LaVa community is to promote and nurture the movement of video mixing and improvisational video art in the greater Los Angeles area.

This is due to the fact that on a PC, the internal QuickTime decoding routines are not capable of multi-threading, which makes it impossible to decode frames in parallel. So even if one layer can run properly using QuickTime libraries, the more layer you pile up, the more the decoding bottleneck will show up.

On Mac OSX, QuickTime is now mostly multi-threaded decoding so it’s not such an issue but we recommend to try to use codec supported by our FFMPEG player in any case.

Important: make sure you understand we refer to the internal QuickTime player and NOT to QuickTime files in general. Some QuickTime files (for example, files based on mpeg/mpeg2/mpeg4/PhotoJPEG/Motion JPEG/H.264 codec) are played through the FFMPEG player.

To know what internal player is used for what file, use the information shown in the visual library:

Visuals library

In this case, both movies are QuickTime files. The first one is read through QuickTime because the Cinepak Codec is not handled by FFMPEG.It will play but it is not optimal.

The second is read through FFMPEG because it’s using H.264 codec, it is therefore better for performances.

As a part of their creation process they make a very advanced use of ArKaos software’s blend and mask modes to map visuals on onto architecture or objects.

They have once again caught our attention with one of their latest installations, a stunning visual show in Breda’s cathedral. It’s a beautiful mix of painting projection, light sculpture, perfect mapping and great music made by a team of very experienced professionals..

But enough words, have a look at the video; it’s really worth watching to the end!

Credits goes to Yannick Jacquet (Legoman), Joanie Lemercier (crustea), Romain Tardy (Aalto) and Olivier Ratsi (Emovie) from ANTIVJ. Original music is by Thomas Vaquié, played on organ by Gerard Maters and the lighting design is by Giacinto Caponio.

Finally, we couldn’t resist adding another video to this post; the TV report from Dutch television where you can see the ANTIVJ team interviewed while setting up the show and explaining their background and how they evolved to this concept (the presentation is in Dutch but the interviews are in English). It also briefly shows some of ANTIVJ’s other very interesting realizations.

For more, be sure to check out Crustea’s YouTube Channel, ANTIVJ on Vimeo and the ANTIVJ Blog.

It features  Interviews and Footage from The Light Surgeons, D-Fuse, Hexstatic, Vj Anyone, Addictive TV, Vector Meldrew and Fatamorgana.

About V.I.A.

With over six years experience in the live events industry, the V.I.A – (Visually Impaired Artists) cultivate a range of digital disciplines for the mutual love of live performance.

With professional backgrounds in the Broadcasting, Film and Live events industries their live performances bring together a mix of Animation, Motion Graphics, 8mm, Film & Video, Video feedback, live camera work and Lighting Design.