One thing that appears obvious to us after more than 10 years of working with the various solutions that can be used on Mac and Windows to decompress video is that not all solutions are equal.

We believe that our users should be able to play almost any kind of media with the ArKaos software and at the same time we do everything a team of developer can to get the best performance on the platforms supported.

Practically the ArKaos software can play media files through the following decoders: QuickTime, FFmpeg, the Flash plugin, Quartz Composer and Windows Media Player. Currently (December 2009) we don’t allow the user to decide through which solution a media will be decoded, instead we try to make the best decision based on our many years of experience and tests. In the case where several video codecs (such as Mpeg4) can be played through different decoders like QuickTime, FFmpeg and Windows Media Player, we always prefer, no matter the platform, to first try to handle those codecs with FFmpeg as we have always had the best results with FFmpeg during our tests, and here’s why..

The other solutions..

Of course, on the Mac we could stick to QuickTime.. But sadly it appears that QuickTime has become almost obsolete today and is just used to decompress the codecs needed by the iPods. Most codecs are not properly multi threaded in QuickTime and this is a big problem for us. With GrandVJ or MediaMaster taking advantage of multi core machines by using each core separately to decompress the video layers you are mixing, it’s too bad to miss that great improvement by playing videos that absolutely need the QuickTime support.

On the PC we could be using Windows Media Player, it is a wonderful technology but it’s architecture is completely incompatible with the sophisticated engine we use to allow frame blending and frame prediction in the latest version of GrandVJ and MediaMaster.

Flash animations and Quartz Compositions, of course, are unique in the sense that they can only be read by their own dedicated player.

As you now understand, once you open a media file in ArKaos your content can go through various roads and it’s important to consider that. To get the most out of GrandVJ or MediaMaster, benefit from the advantages of multi threading and perform frame blending you should make sure that your content is always played through FFmpeg.

How does ArKaos process the movie file?

On the Windows platform it’s pretty easy because the software first tries to play a file with the FFmpeg player and then falls back to the others (Windows Media Player or QuickTime) if needed.

But things get harder with the Mac platform because some files that could be played by FFmpeg may be rejected and fall back to QuickTime just because they have not been saved the correctly. Many QuickTime files have been created in a way that make them compatible with Mac OS X but also it’s ancestors Mac OS 9 and 8 and 7 and… Those files make use of what is called a Resource Fork.

To make it short, when Apple introduced the Macintosh years ago they decided that a single file would in fact be two files: the data file itself and an optional resource file. Many software have been using that Resource Fork to store additional data and information. This is now a problem for many .mov files that were created in the past and even still today with some applications that were born under the Mac OS 9 era and still generate QuickTime .mov files that have a Resource Fork.

This is quiet an issue for us because when the FFmpeg decoder can’t handle those files and makes ArKaos fall back to QuickTime, you end with QuickTime’s very bad multi core support even for very common codecs such as Photo JPEG. Another bad side effect of this Resource Fork is that most of the time when you will copy a movie file that has a Resource Fork to a Windows or Linux machine it will simply not be readable anymore because that tiny bit of information contained in the Resource Fork is not read on other platforms although it is vital to play the content.

How to get rid of the Resource Fork?

Thankfully it’s very easy to remove that annoying Resource Fork from a QuickTime file. You can simply open the file in today’s QuickTime player and do a “Save As”, QuickTime will reprocess the file correctly. The process is called “flattening” a movie file, it is very fast and will not re-compress your movie, this is very important to avoid re-compressing when possible.

Open your file in QuickTime and go to File, Save As..

.. and enter a new name

Of course, you will only need to do that if your file was created on a Mac, the free version of the QuickTime player is enough to do this and it works on all Mac OS X versions up to Snow Leopard with the latest QuickTime player. Now if you have lots of movie files to convert, you can of course look for some professional software that can queue and “flatten” a bunch of files from a directory.

FYI, here is a QuickTime file that has a Resource Fork (moov), inspected with a programming utility called Resorcerer:

And here is the same file that has been flattened:

How do I know what decoder is used?

In GrandVJ it’s very easy to see what player is used by the ArKaos engine when processing a file; simply hover your mouse over the cell that contains the video and the contextual help appears.

In this one, the video file with a resource fork will be played through QuickTime:

In this one, the same video flattened with the QuickTime player will go through the FFmpeg path:

(The same information is also available in GrandVJ for each movie listed in the visual library and in MediaMaster you can get that information from the file library window.)

Conclusion

During the first days of the release of GrandVJ 1.2 beta 3 with the new audio engine (which relies on FFmpeg) some users have complained that some old .mov files were not playing audio correctly with the beta version, this was most of the time caused by the Resource Fork and flattening the files fixed the problem. As a bonus, it will make your files cross platform.

This should give you some lights on what is happening when you drop a file on GrandVJ or MediaMaster, as you can see a lot of work is being done under the hood :)

When you are in the show industry and try to find out the best way to compress your content it’s still a little bit of black magic. While some codec are able to compress video very well they are heavy to handle for the machines, even more, the codec that does the best job at compressing while keeping a good quality such as H264 are very bad when you need to scratch your media.

The best codec for users that need to interact a lot with the content should allow to play forward and backward easily and should allow to jump into the content quickly.

Recently, I’ve heard about experiments with a file format designed by graphic card manufacturers and based on texture compression DXT1 DXT3 or DXT5 and I was wondering how hard it would be to add support for such a format to QuickTime through a new codec. Being programing for the QuickTime API since it’s version 1.0 beta I was considering the challenge fun and interesting.

I started by resuscitating an old sample code from 1999 found on the Apple web site and after a few hours of coding and testing I had an ArKaos codec generating video files that could be played back by QuickTime.

ArKaos codec in the QuickTime player

Then I jumped on the DXT texture compression problem and chose to use the squish library to handle the job. A few more hours later and my experimental codec was working with texture compression.

The codec when used by QuickTime at that point was not efficient because the texture decompression was done on the CPU, so there’s no advantage of using the GPU to handle that texture format.

To measure the speed up I used an experimental player I am working on, and patched ffmpeg to handle my new codec. This allowed me to leave the texture data untouched and pass it to the GPU via the OpenGL extensions that handle texture compression.

When this was done I could start enjoying the show and make some performance tests; in the table below the file size is in MB and CPU load is taken from running top in the console:

If you are curious about frame quality here is a crop from an original frame in Apple intermediate :

Original frame - Apple intermediate (click for full size)

Here is the same cropped frame using Dxt1 with texture compression:

HD frame - Dxt1 compressed (click for full size)

And here is the same cropped frame with Dxt5 compression:

Same frame with Dxt5 compression (click for full size)

So the results of these tests are that pure Dxt1 is the best format when it comes to CPU load, Dxt5 is better if you focus on quality but it generates files that are twice bigger. Actually, using Dxt1 I could have played 7 HD loops at the same time with my laptop … I mean, if hard drive’s data rate would’ve allowed it!

The problem with pure Dxt5 or even Dxt1 files is that they are huge, about 4 times bigger than photo jpeg or H264 encoded files. So my workaround was to use data compression on the CPU, since it’s almost unused!

To conclude, it looks like using Dxt1 with texture compression is the best solution to optimize both CPU and disk usage with a file size that is almost the same as Photo jpeg and a little bit bigger than H264. And the codec is still twice faster than any other codec I tested when using the CPU to decompress the texture data from disk.

On my venerable old MacBook pro I can’t smoothly play a loop in Full HD and even at 1280*720 my machine is still struggling. Thanks to Dxt1 with texture compression I can play two HD layers and three 720p layers.

Now I know you guys would like to play with this new toy so here’s a .dmg file with the test player I have used, along with a set of video loops. Beware that this dmg file is for Mac OS X x86 only (no Power PC): and here’s the link

You can be sure that we will bundle that new experimental codec in the next update of GrandVJ and MediaMaster, so everyone can enjoy it!