In this article we’ll explain a little bit more how you can use Syphon in combination with MediaMaster and GrandVJ, then we’ll give you some inspiration from those who have already started using it..

About Syphon

As explained on the official Syphon web page: “Syphon is an open source Mac OS X technology that allows applications to share frames – full frame rate video or stills – with one another in realtime. Now you can leverage the expressive power of a plethora of tools to mix, mash, edit, sample, texture-map, synthesize, and present your imagery using the best tool for each part of the job. Syphon gives you flexibility to break out of single-app solutions and mix creative applications to suit your needs.”

Practically, a Syphon client application receives video frames from one or more syphon server application(s), and because the Syphon-compatible applications share texture resources within the graphical card, there’s almost no latency between the server and the client applications.

GrandVJ or MediaMaster as Syphon server

Prior versions of GrandVJ and MediaMaster were already able to act as Syphon server, but it required loading the Quartz-Composer Server patch on a dedicated layer.

GrandVJ 1.6 and MediaMaster 2.2.5 are now native Syphon server applications. The usage is pretty straight forward since there’s no configuration at all: the software will start sharing its output frames as soon as it detects a Syphon client application.

GrandVJ output in Syphon Simple Client

GrandVJ or MediaMaster as Syphon client

GrandVJ and MediaMaster can also act as Syphon clients thanks to the Quartz-Composer Client plugin and patch provided by the guys behind Syphon. You can get these from the Syphon Implementations page (grab the file named “Syphon For Quartz Composer” of course. It’s currently in beta but that’s ok). The installation procedure is detailled in “Read Me.rtf” file you get in the .dmg but basically you first need to install the plugin before you can use the patch.

So, once you installed the plugin, grab the Quartz-Composer patch (.qtz file) and either import it in your MediaMaster media library or drag & drop it on a bank in GrandVJ so you can use it with our software.

Syphon Simple Server in GrandVJ

Note that, by default, the patch will display the frames of the first registered Syphon server application. If you want to use a server that’s not the first, or to to mix between several servers, you will have to get your hands in the dirt and launch the Quartz-Composer application (that you get with Xcode from Apple) to modify the “index” value in the patch (the default value zero being the first server)..

Modifying the “index” value in Quartz-Composer

Oh, and if you don’t want to fiddle around with Quartz-Composer you can simply download the alternative patches we’ve modified for you (yeah, we’re like that..).

Remark:
You cannot use GrandVJ or MediaMaster as Syphon client if you have not first started a Syphon server, if you try to do so you will notice that MediaMaster / GrandVJ creates an infinite visual feedback, its output frames being shared with the layer that runs the Syphon layer!

If you intend to use GrandVJ or MediaMaster as a client, you should either start the other Syphon server application first, or you can use one of the alternative Quartz-Composer patches mentioned earlier in this article.

Some inspiration from those who have already started using it

All this seems a little too abstract for you? No problem, let’s see concretely what you can achieve with this new feature..

Koen de Puysseleir – Tiësto Club Life tour

It all started with Koen de Puysseleir, Creative Director for Tiësto and owner of Light in Motion, a Belgian design and show control company. Koen already worked with MediaMaster for the previous Tiësto tour (the Kaleïdoscope Tour) and this time he got challenged with some difficult video setups involving split screens and side screens for Tiësto Club Life performances. He needed a solution to output MediaMaster to MadMapper and thus contacted us to inquire about how to achieve it.

We immediately understood that something cool could be done here and so we started working on developing this feature and then sent Koen a custom version of MediaMaster with Syphon support for him to test.. Below is a video from the show at the Home Depot Center in Carson, you can read the complete story here on our web site.

http://www.youtube.com/watch?v=HfGpTW8h36I

Koen’s laptop running MediaMaster and MadMapper

Momo the Monster – The Glitch Mob Live show

Visual Artist Momo the Monster has been using MediaMaster Express as a Syphon client to drive LED panels on The Glitch Mob Live shows this summer, read the full story at CreateDigitalMotion

Charlie Mars recording mixes in GrandVJ

France VJ champion Charlie Mars explains (in French) how he is using GrandVJ and Syphon Recorder to record his mixes.

http://www.youtube.com/watch?v=D5J-eqx1kEI

Conclusion

As you can see, Syphon support in MediaMaster or GrandVJ really “gives you flexibility to break out of single-app solutions and mix creative applications to suit your needs” just like they say on the Syphon web site!

Ok, ok.. I can already hear some of you complaining it’s again a Mac-only solution.. But don’t worry Windows users, we’re working on a similar solution for you too..

Thank you Syphon creators, interoperability is the future!

Recently we spotted the video below on Twitter, it’s produced by live audio-visual artist Ryan Noise and Unique Squared and we found it to be a great introduction to the use of GrandVJ in connection with Ableton Live. Note that Ryan connects GrandVJ and Ableton through Rewire here, but the latest GrandVJ 1.5 comes with a VST plugin which allows not only to send MIDI to GrandVJ but also sound, so that you can use it to trigger sound-reactive Generators or Flash animations.

http://www.youtube.com/watch?v=zGHruKnCWwE

Read on for Ryan Noise Interview..

For years Ryan has used GrandVJ as his live video mixing solution. When asked, “Why did you decide to go with Ableton Live, Akai’s APC40 and GrandVJ in your setup?”, Ryan replied: “I decided on Ableton as my main host for it’s over all audio manipulation, MIDI functionality, low CPU load and simple programmability.

I’m using the APC40 as a tactile control unit, this gives me the ability to control live video and remix audio by applying filters and parameters to the assignable knobs and buttons on one customized surface.

Lastly for video, I use GrandVJ because of its ease of use, MIDI and OSC mapping abilities, multi-camera inputs, stellar effects, and dead-simple customization.

GrandVJ has been the back bone of my visual performance for years. The visual possibilities with this software are limitless and I always find new was to surprise even myself.”

http://www.youtube.com/watch?v=OzU8AjCeAZw

The settings can be modified in the “Geometric Correction” dialog which can be accessed via the “Geometric Correction” button near the soft edge span dimensions on the video tab of the preferences dialog.

The number and arrangement of the outputs is determined by the soft edge span dimensions.

Geometric Correction

With geometric correction it is possible to correct output deformations via a correction grid.

Three main areas can be distinguished: the screen arrangement, the toolbar and the correction grid.  Via the clickable screen output arrangement, at the top left of the dialog, it is possible to switch between the corrections grids of the different screen outputs.

A normal output is represented by a blue rectangle and the overall correction grid is represented by an orange rectangle. With this overall correction grid it is possible to apply a correction over all displays at once. This especially meaningful when displaying using multiple outputs to project on one curved screen.

With the drop-down list on the toolbar the geometric correction type can be chosen. 4 different correction types are available, namely:

1. No Correction: If you don’t want to apply geometric correction to this screen. In this way MediaMaster knows that it can bypass the geometric correction process for this screen.
2. Linear: A linearly interpolated grid for straight corrections.
3. Curved: For smooth geometric corrections such as a curved screen.
4. Nurb: Another method for smooth geometric corrections.

The correction grid is modified by moving one of its control points. The control points can be moved by the mouse cursor or by arrow keys for per pixel precision. Control points can be added with the appropriate  buttons on the toolbar and via the keyboard shortcuts (F1-F4). Via the reset button the grid can be set to its original state.

The magnetic border makes the screen border ‘sticky’, thus making it easier to correctly position a control point to the border of the screen.

The following shortcuts are defined for the geometric grid window:

Soft Edge per Output

With the custom soft edge it is possible to adapt the soft edge parameters per edge of an output.

Two controls can be found in the “Custom Soft Edge” panel, namely the check-box “Use Custom Soft Edge” and the button “Set Default Values”.
The check-box will determine if MediaMaster will use the standard overall soft edge or the custom soft edge per side.
The “Set Default Values” button will copy the curve and fade curve parameters of the overall soft edge settings, plus the soft edge setup, to the custom soft edge settings.

Note that using the custom soft-edge will disable the master fixture’s soft-edge.

In the soft edge tab it possible to adapt and enable the curve and width parameters per side of an output.

Now with new geometric correction and custom soft edge in MediaMaster Pro, it has never been so easy to project video on non-planar surfaces.

In this tutorial we will demonstrate how to add your own Quartz-Composer Effects in MediaMaster.

Remark:

Check out the article “How to: Create audio-reactive effects for GrandVJ using Quartz-Composer” for more details about the creation of Quartz-Composer effects. You will also find lots of resources and how-to’s for Quartz Composer on the web or in the Reference Library from the Mac Dev Center.

Simple Mode (MediaMaster Express and Pro) and Fixture Mode (MediaMaster Pro)

It’s easy to add Quartz effects in MediaMaster 2.0 when in Simple Mode, but in Fixture Mode effect presets are defined within the layer fixture and these fixture files shouldn’t be modified in any circumstances. As a consequence, to make use of new effects we have to add a new custom layer fixture that will include new effect presets..

In order to do so, we will first copy some new files to the user’s local MediaMaster resources, where the MediaMaster Library is located:

1. Download the following file “MediaMasterQuartzComposerEffectsResources.zip“, then copy it to “/Users/MyAccount/Library/Application Support/ArKaos/MediaMaster/”.
2. Decompress the zip file by double clicking on it. Two new directories should have been added:
   • “MMFixtures/Layer/” contains a custom layer fixture file.
   • “Quartz-Composer/Effects/” contains two new Quartz-Composer effect files: “MyBassZoomBlur.qtz” and “MyKaleidoscope.qtz”

At this point, those new Quartz effects will be available from the menus in MediaMaster 2.0 when in Simple Mode, continue reading further to learn how to use them with MediaMaster in Fixture Mode.

Adding a new custom layer fixture

Open the file “MMFixtures/Layer/MyCustomLayerFixture.xml” using any text editing software, we will look at how it is made in order to understand how to add new effect presets in a custom fixture. This file is a slightly modified version of the original MediaMaster Pro “Layer Fixture 1.1″, it contains some extra effect presets.

This files defines the DMX channel assignments and the effect presets libraries that we’ll focus on.

The first line of the file defines the name of the fixture, as it will appear in MediaMaster Pro preferences dialog. This name must be different from the other fixtures.

At the very bottom of the XML document, fixture effect library 11 contains four new effect presets.

You may have noticed that the Quartz-Composer effect “MyKaleidoscope.qtz” appears in three presets with different parameters.

An effect preset is basically defined by an index, a name, an effect and a set of effect parameters. An effect can be used in several different presets and even different libraries with different parameters setup.

In a preset, each effect parameter can be either a fixed value or a variable that corresponds to one of the four effect controllers available in MediaMaster Pro.

Based on this small example, add your own Quartz-Composer effects and create your custom fixtures to extend the possibilities of MediaMaster.

Now let’s try this new fixture

Start MediaMaster Pro, and in “Preferences –> DMX –> Patch” choose “My Custom Layer Fixture” as the “Layers fixture”.

Back in the main window, on any layer, select a visual, push the dimmer to the max to start it, and finally select an Effect between 1 and 4 in Effect Library 11.

Now it’s your turn to create your very own Quartz-Composer effects and use them to leverage the visual quality of your show, be creative!

In a perfect world, we would have zero frame delay but unfortunately this is not possible with current computer hardware; the simple fact that a computer is processing video already adds one frame of input delay, that frame is needed for the computer to store the complete image before having it processed by the video card. Then you can add another frame of output delay, caused by the video card processing the image. Before or after that, each device that is added to the pipeline – such as a camera, a mixer, a LED controller or a projector – will add some delay into the processing. In the end, if the delay becomes too big you will notice that the sound and the video are out of sync, for example you will hear the voice of the speaker before his lips start moving on the screen.

If we cannot tell what the latency will be with your setup, we can however provide you with a point of comparison by measuring how many frames are needed to process a live input on an ideal setup: MediaMaster running on one of our A10 Media Servers that are specially built and configured to get the most out of our software.

Here is how we proceeded: we’ve setup a MacBook Pro with DVI/HDMI output connected to the Blackmagic Intensity Pro input card from the A10 Media Server which was set to an input resolution of 720P 60Hz. Then we started a frame counter movie on the MacBook Pro and launched the live input from MediaMaster full screen on the A10 Media Server. Finally, we put the monitors next to each other and took a picture so we can see the difference between the frame counter on both screens.

As you can see in the movie, when we took the picture the MacBook Pro was at frame count 24429 and the output of the A10 Media Server was at 24424: there is a 5 frames delay between the counter from the laptop and the counter from the server. Since our whole setup was running at 60Hz, this means that the processing delay equals 83 milliseconds (1 second divided by 60 frames = 16.6 milliseconds per frame, 5 times 16.6 milliseconds equals 83 milliseconds).

Quartz Composer is a free graphics and imaging tool on the Mac platform which is available as an optional install that can be found on your Mac OS X DVD since version 10.4.

It allows very easy creation of complex audio-reactive animations or effects which can now be played in GrandVJ (read the previous article about GrandVJ and Quartz Composer). As I’m writing these lines, the last version of Quartz-Composer is 4.0 on Mac OS 10.6 (Snow Leopard).

In this article we will demonstrate how to quickly create a bass-reactive Zoom/Blur effect like the one you can see below:

Important: This tutorial requires that you have some basic knowledge of Quartz Composer: if you have never used it, you really should first go through the “Getting Started”. You will also find some helpful readings at the Apple Developer Connection web site about Quartz Composer.

To create our effect we will simply edit Quartz-Composer’s default template named “Image Filter” to make it audio reactive. Basically, we will connect bass levels of the computer’s audio input signal to the “Amount” parameter of our Zoom/Blur filter. (Read more below..)

• Start the Quartz-Composer.app which is located in /Developer/Applications/

• From the menus, choose (File –> New from Template –>) Image Filter :
  

• Here’s the patch that will open:

• In the menus choose Editor –> Edit Protocol Conformance

• We will now tell the ArKaos software what inputs must be transmitted to the Composition. The ArKaos engine can send visuals or sound for processing by the Composition but only if he knows that these feeds will be accepted.
  To enable this you must check “Music Visualizer” and click “Done”

• You’re back to the Composition and several input modules have been added on the left:

• Remove every unnecessary input modules, so they won’t be listed as input parameters in GrandVJ‘s effect parameter panel:

• Quartz composer can hide blocks inside blocks, it’s called macros. It’s very powerful and in this case it holds the most important part of our composition: the processing part, which will affect the visual before copying it to the output.
  Double click on the “Process Image” macro to open it, Quartz Composer details each block contained in the macro:

• We want to connect the audio volume to the “Amount” input of the “Zoom Blur” filter contained in this macro. To do that we need to make the “Amount” parameter available in the macro block so that it can be accessed from the top level, it’s called publishing an input.
  Publish the “Amount” input as in this screenshot:

• When it’s done, we can go back at the top level to add some processing.
  In the top bar, click the “Edit Parent” button to go back the parent patch:

• Now we will add a processing block to create a value from the sound input.
  From the Library, add an “Audio Processor” module:

• You’re back to the patch with the “Audio Processor” module added. The magical thing about Quartz Composer is that you can connect the modules in many different ways, when you add a module you just have to tell Quartz Composer how to feed the inputs and connect the outputs.

• First we want to process the audio received from the ArKaos engine. To do that we connect together the “Audio Spectrum” and the “Audio Processor” module:

• Now the audio processor will create three values from the audio we are feeding, Lows, Mids and Highs. For this effect we just want to use the lower value which is the one reflecting the bass volume. So we connect the Lows output from the “Audio Processor” module to the “Amount” input we just published in the “Process the Image” macro:

At this point we need to get an audio signal so we can test our effect within Quartz Composer. For this purpose we will add an “Audio Input” that we will disconnect in the end so GrandVJ can input its own audio signal into the “Audio Processor” module.

• From the Library, add an “Audio Input“ module..

• ..and connect its “Audio Spectrum” output to the input from “Audio Processor”

Now we can see the image reacting to the audio input signal, but the effect is definitely too low, so we need to boost the audio signal values. A simple way to do that is to add a new processing module, specifically a math module:

• From the Library, add a “Math Module”..

• ..and configure it to operator Multiply, set Operand #1 to value 100:

• Connect the Lows output from “Audio Processor” to “Multiply” then connect “Multiply” to the “Amount” input from “Process the Image”

• You can now disconnect the “Audio Input” from the “Audio Spectrum” so that, once this composition is loaded in GrandVJ, the software will automatically send the audio input signal directly to “Audio Spectrum”.

• Finally save the composition as “MyBassZoomBlur.qtz” in /Users/MyAccount/Library/Application Support/ArKaos/GrandVJ/Quartz Composer/Effects/

Now is time to start GrandVJ and find your hard work rewarded by playing with your very own brand new effect. You will find it located in the Effects panel under the “Quartz-Composer (User)” effect category.

Notice how GrandVJ has made available the X and Y effect parameters from the Composition in the Effect parameter panel so you can use them to control the behavior of your effect in real time.

Now, that wasn’t hard, was it? If you want to go further, a good start will be to create new effects based on this Composition by replacing the “Zoom Blur” effect by another filter.

Don’t hesitate to share your creations on the forums :)

One thing that appears obvious to us after more than 10 years of working with the various solutions that can be used on Mac and Windows to decompress video is that not all solutions are equal.

We believe that our users should be able to play almost any kind of media with the ArKaos software and at the same time we do everything a team of developer can to get the best performance on the platforms supported.

Practically the ArKaos software can play media files through the following decoders: QuickTime, FFmpeg, the Flash plugin, Quartz Composer and Windows Media Player. Currently (December 2009) we don’t allow the user to decide through which solution a media will be decoded, instead we try to make the best decision based on our many years of experience and tests. In the case where several video codecs (such as Mpeg4) can be played through different decoders like QuickTime, FFmpeg and Windows Media Player, we always prefer, no matter the platform, to first try to handle those codecs with FFmpeg as we have always had the best results with FFmpeg during our tests, and here’s why..

The other solutions..

Of course, on the Mac we could stick to QuickTime.. But sadly it appears that QuickTime has become almost obsolete today and is just used to decompress the codecs needed by the iPods. Most codecs are not properly multi threaded in QuickTime and this is a big problem for us. With GrandVJ or MediaMaster taking advantage of multi core machines by using each core separately to decompress the video layers you are mixing, it’s too bad to miss that great improvement by playing videos that absolutely need the QuickTime support.

On the PC we could be using Windows Media Player, it is a wonderful technology but it’s architecture is completely incompatible with the sophisticated engine we use to allow frame blending and frame prediction in the latest version of GrandVJ and MediaMaster.

Flash animations and Quartz Compositions, of course, are unique in the sense that they can only be read by their own dedicated player.

As you now understand, once you open a media file in ArKaos your content can go through various roads and it’s important to consider that. To get the most out of GrandVJ or MediaMaster, benefit from the advantages of multi threading and perform frame blending you should make sure that your content is always played through FFmpeg.

How does ArKaos process the movie file?

On the Windows platform it’s pretty easy because the software first tries to play a file with the FFmpeg player and then falls back to the others (Windows Media Player or QuickTime) if needed.

But things get harder with the Mac platform because some files that could be played by FFmpeg may be rejected and fall back to QuickTime just because they have not been saved the correctly. Many QuickTime files have been created in a way that make them compatible with Mac OS X but also it’s ancestors Mac OS 9 and 8 and 7 and… Those files make use of what is called a Resource Fork.

To make it short, when Apple introduced the Macintosh years ago they decided that a single file would in fact be two files: the data file itself and an optional resource file. Many software have been using that Resource Fork to store additional data and information. This is now a problem for many .mov files that were created in the past and even still today with some applications that were born under the Mac OS 9 era and still generate QuickTime .mov files that have a Resource Fork.

This is quiet an issue for us because when the FFmpeg decoder can’t handle those files and makes ArKaos fall back to QuickTime, you end with QuickTime’s very bad multi core support even for very common codecs such as Photo JPEG. Another bad side effect of this Resource Fork is that most of the time when you will copy a movie file that has a Resource Fork to a Windows or Linux machine it will simply not be readable anymore because that tiny bit of information contained in the Resource Fork is not read on other platforms although it is vital to play the content.

How to get rid of the Resource Fork?

Thankfully it’s very easy to remove that annoying Resource Fork from a QuickTime file. You can simply open the file in today’s QuickTime player and do a “Save As”, QuickTime will reprocess the file correctly. The process is called “flattening” a movie file, it is very fast and will not re-compress your movie, this is very important to avoid re-compressing when possible.

Open your file in QuickTime and go to File, Save As..

.. and enter a new name

Of course, you will only need to do that if your file was created on a Mac, the free version of the QuickTime player is enough to do this and it works on all Mac OS X versions up to Snow Leopard with the latest QuickTime player. Now if you have lots of movie files to convert, you can of course look for some professional software that can queue and “flatten” a bunch of files from a directory.

FYI, here is a QuickTime file that has a Resource Fork (moov), inspected with a programming utility called Resorcerer:

And here is the same file that has been flattened:

How do I know what decoder is used?

In GrandVJ it’s very easy to see what player is used by the ArKaos engine when processing a file; simply hover your mouse over the cell that contains the video and the contextual help appears.

In this one, the video file with a resource fork will be played through QuickTime:

In this one, the same video flattened with the QuickTime player will go through the FFmpeg path:

(The same information is also available in GrandVJ for each movie listed in the visual library and in MediaMaster you can get that information from the file library window.)

Conclusion

During the first days of the release of GrandVJ 1.2 beta 3 with the new audio engine (which relies on FFmpeg) some users have complained that some old .mov files were not playing audio correctly with the beta version, this was most of the time caused by the Resource Fork and flattening the files fixed the problem. As a bonus, it will make your files cross platform.

This should give you some lights on what is happening when you drop a file on GrandVJ or MediaMaster, as you can see a lot of work is being done under the hood :)

Here’s my second tutorial showing how to take advantage of the Flash capabilities within ArKaos GrandVJ. This one covers the creation of dynamic text visuals. I’ll go explain how to tap into the “statement” variable within Flash so you can create your own text based clips.

I’ll also touch on the “duration” and “position” variables, which if used properly, allow playback speed adjustment of code based animations. We’ll be using some basic ActionScript 2.0 that’s a little more advanced than the previous tutorial.. Note that it will also work for NuVJ and MediaMaster since the variables are the same.

You can download the final .fla and .swf files here or follow along the video tutorial below.

In mixer mode, you could always assign a black image and assign it to layer 8 but people using the synth mode (or if you simply don’t wanna waste an additional layer) have to use some tricks to acheive it.

The key is to use the master brightness and contrast on the master:  If you pull the brightness to zero and the contrast to the maximum, all output will be blackened.

So we want to go from the “normal state”:

To the “black out state”:

Using a single controller or key.

In this tutorial, we’ll suppose you want to use a midi controller to go gradually from one to the other, providing a master fade functionality.

Let’s start by mapping the brightness control. Right click it and choose “Midi mapping…”. This will bring the MIDI mapping dialog:

Click on “learn” and start moving the knob you want to assign to the master fade out. By default, it will control the brightness to it’s full extent (0-100%) over the controller range. However, our “normal state” needs the brightness to be at 50 percent so we’ll tweak the range so that the Max. Value is 50.

Close the dialog. Now moving your controller will move the brightness slider from 0 to the default position.

Now let’s tweak the Contrast. Right click it and select again “Midi mapping…”

Click “Learn” again and move the same controller you assigned to the Brightness. GrandVJ allows you to map the same controller to multiple functionalities, so that not an issue.

Now you want that when the brightness is full (the controller is to its maximum value), the contrast is to zero and that when the brightness is zero, the contrast is full. That means you want to invert the behaviour of the contrast. To do so, it’s very easy, simply put 100.0 as Min. Value and 0.0 as Max. Value.

Now press ok on the dialog and you will have a dedicated control to fade the master output.

Of course, you can also assign this to a note event or a key and you will have a black out switch instead of a gradual fade…

As promised, here’s the first in a series of tutorials showing how to take advantage of the Flash capabilities within ArKaos GrandVJ. Note that it will also work for NuVJ and MediaMaster since the variables are the same.

In this tutorial I’ll explain how to tap into the EQ data to create some simple audio-reactive animations. We’ll be using some basic ActionScript 2.0 for the inexperienced Flash programmer. I won’t go into much detail on the how the code works but it should be enough of an introduction to allow some experimentation in creating new content for your VJ arsenal.

You can download the final .fla and .swf files here or follow along the video tutorial below.