Quality analysis

CineForm 444 Quality Analysis Versus HDCam SR

Originally posted April 4, 2007 – PDF Version

Still image extracted from sequence of 'Magic Hour' StEM footage used in tests

Still image extracted from sequence of "Magic Hour" StEM footage used in tests

HDCAM-SR has achieved an important position in the market, being used routinely for on-set capture and final mastering of feature film projects. The quality of HDCam SR is quite high and has repeatedly proven itself worthy of both. So as CineForm introduces its 12-bit CineForm 444 format it is important to test visual fidelity comparatively against HDCam SR.

Test Methodology

In designing the comparative quality tests, two sets of uncompressed dual-link HD-SDI sources were used:

1. “Magic Hour” sequence of StEM (Standard Evaluation Material) commissioned by DCI (Digital Cinema Initiative) for purposes of testing the quality of compression codecs.
2. Viper camera shot in dual-link Filmstream mode (courtesy of Thomson).

In both cases the uncompressed source served as the reference for performing PSNR (Peak Signal-to-Noise Ratio) tests which measure, in decibels (dB) how close the compressed images (after reconstruction) match the uncompressed source images.  The higher the number the better the result.

In this write-up, we present the visual quality performance results obtained from analyzing the StEM footage for both HDCam SR and CineForm 444.  Results from the Viper Filmstream tests are very similar, and are publicly available as a series of posts on the blog of David Newman at http://cineform.blogspot.com/.


The figure above defines the hardware configuration for the quality tests using the StEM material.

Our source material was 580 frames of the “Magic Hour” sequence from the Standard test Evaluation Material (StEM) which is used for DCI compliance testing. The footage is quite demanding (intentionally) on compression, with a lot of film grain, complex motion and significant detail within each frame. Before we performed the tests, using After Effects 7, we resized the StEM sequence to 10-bit 1920 x 1080 RGB (from its original 4096 x 1714 16-bit TIFF files) for compatibility with dual-link HD-SDI.  The resized StEM footage was stored on a workstation that included a Blackmagic Decklink dual-link HD-SDI card for content delivery.

Content was then played out from the uncompressed workstation into both the SRW1 (SQ-440 Mbps mode) and the Wafian HR-2.  The Wafian HR-2 recorded the StEM footage using two different quality settings defined for CineForm 444:

Filmscan 1 and Keying 1

In software, from the original uncompressed StEM footage, we later created CineForm content using two additional CineForm 444 modes:

Filmscan 2 and Keying 2

After recording onto the SRW1, its material was then played back out and re-captured on the uncompressed workstation so we now had a “before” and “after” sequence for PSNR analysis.

As a matter of background, the CineForm designation of “Filmscan” is used because these modes are designed to accurately reproduce the grain characteristics of 35mm and 16mm film scans.  The “Keying” designation is used to define internal quantization settings for the codec that are more optimum for recorded material that is intended for keying.  The differences between the “Keying” and non-”Keying” images will show up later in the analysis results.  Finishing the analogy, the “Filmscan 1 / Keying 1″ designations are the modes CineForm normally recommends for our customers doing film or digital cinema projects.  The “Filmscan 2 / Keying 2″ names do offer additional measurable visual fidelity, but the files are about 40% larger, and any differences are not visually apparent.

Test Result Summary

(Click image to see larger version)

(Click image to see larger version)

At left is the summary PSNR results.  Overall PSNR results are calculated using the ITU-709 luminance conversion algorithm for signals containing individual channels of of R, G, and B. The higher the numbers, the more accurately the compressed signal approaches the uncompressed source signal.

The red line is the HDCam SR PSNR performance resulting from the SQ-440 Mbps mode.  The variation in its PSNR is characteristic of fixed bitrate algorithms which adjust quality up and down based on varying scene complexity.

The other lines on the chart show the PSNR for the four different CineForm 444 recording modes.  Notice the PSNR for each remains relatively constant as is characteristic of variable bitrate (constant quality) algorithms.

CineForm 444 in Filmscan 1 mode exceeds the quality of SQ-440 Mbps by about 3dB.  CineForm 444 in Filmscan 2 mode exceeds the quality of SQ-440 Mbps by about 5 dB.

Individual Charts of Test Results

HDCam SRW1 - PSNR of individual RGB channels
HDCam SRW1 – PSNR of individual RGB channels

Above is the PSNR graph for the individual R, G, and B channels from the SRW1, and below are the same graphs for the individual channels of the four separate recordings of CineForm 444.  To the right of each graph are the statistics for (calculated) luminance plus the individual R, G, and B channels.

Some interesting observations about the PSNR of the individual CineForm graphs are worth noting.  In each of the sets of two charts, you’ll notice that the green channel in the leftmost chart (of the two) remains the same as the “keying” chart to its immediate right.  The difference between the two is that the red and blue channels rise about 4dB in the keying mode as part of the algorithm differences optimized for keying.  In the non-keying modes the algorithm assigns different weights to red and blue based on the characteristics of the human visual system. 

CineForm 444 - - Individual channels of RGB for Filmscan 1 (L) and Keying 1 (R).

CineForm 444 - - Individual channels of RGB for Filmscan 1 (L) and Keying 1 (R).

CineForm 444 - - Individual channels of RGB for Filmscan 2 (L) and Keying 2 (R).

CineForm 444 - - Individual channels of RGB for Filmscan 2 (L) and Keying 2 (R).


Final Observations

The implications of the test results are straightforward.  Recording direct-to-disk using CineForm 444 as an alternative to HDCam SR will not compromise the fidelity of the recorded material.  Specifically, these results show that recorded fidelity of CineForm 444 is higher than the same images recorded to tape.  Although tape remains the dominate recording solution in the market, as the production world moves towards IT solutions, it is important to realize that disk-based compression can deliver necessary visual fidelity.

Some other workflow benefits of are worth noting:

•  An entire two hour feature can be mastered to a single 350GB hard drive for long term archive which offers longer shelf life than tape
•  Lossless duplication for storage redundancy and data migration is simple
•  Delivery for film-out or mastering to other delivery formats is straightforward because the source material is always online
•  Drives now cost less than the equivalent storage on tape(s).