Canon 5DMK3 clean HDMI output - first impressions (+ download uncompressed files)

Hi all,

Yesterday Canon released their so sought-after update for the Canon 5DMK3.
To be honest, for me, the cross type AF is not as interesting as the clean HDMI output.

So I've recorded initial test footage for some testing:

Youtube:


Download uncompressed:
http://www.solidfiles.com/d/579e4310bc/

Youtube:


Download uncompressed:
http://www.solidfiles.com/d/5a88f92606/

(The files are huge so the ones I've uploaded are very short)

Equipment:
Camera: Canon 5D Mark 3
Profile: Neutral - 0,-4,-2,0
Lens: 24-105L F4 @ F8
Shutter speed: 1/160
Recording device: BlackMagic Shuttle

Brief review:
Canon declares that the output is 8bpc@4:2:2, which is a little bit disappointing especially when the HDMI standard supports 10bpc and I almost sure that in Canon cameras the video processing pipe works at least in 12bpc.
I decided to go beyond Canon's declares and check if I can force it to push 10bpc, in the following way:
- BlackMagic Shuttle will always record at 10bpc; if the input is 8bpc, the Shuttle bit shifts the values (multiplies by 4: 0->0, 1->4 ... 255->1020).
- I've extracted a single YUV frame and warped it in 16bpc (w/o bit shift).
- In hex editor you get, for example, this (in this case it is the U channel):

And you can see that each value (little endian) divides by 4 without fraction.
A801 = 424 -> 424/4 = 106
9C01 = 412 -> 412/4 = 103
etc...

So yes, the conclusion is that the output of the HDMI in the 5D is 8bpc only.
Think how would it be if it was 10bps... In short, the following operation theoretically would give an image without the ugly banding:

The question that should be asked now is: Does it really give 4:2:2 chroma sub-sampling?

Well, I don't know in this moment. Canon may perform scale-up algorithm other than nearest-neighbor (x2 width) and you will never find cloned pixels (although you can see a lot of typical duplicates in the hex viewer, but it is not an indicator), especially in the so soft Canon's 3x3 sensor binning with flat picture style.

For for this moment, lets believe it is pure 4:2:2, and maybe in the future posts I'll do some more in-depth test that will approve or refute the claim :)

That's all for today,

Mark.

Why should we so appreciate All-I video in cameras?

WE SHOULDN'T.

(This article will discuss top level topic because digging to details will be pointless.)
Many manufacturers come with new encoder feature for their camera line. They call it All-I, iFrame & etc.
To explain it, I’ll start with the available encoder frame types:
I (Intra) = Intra frame that draws the image from scratch and compress it. Equivalent to JPEG image.
P (Predictive) = Inter frame that subtracts itself from previous “I” frame and the compression is applied on the residual. “Forward prediction”
B (Bi-directional) = Inter frame that can subtract itself both from previous “I” or “P” frames (forward prediction) and from the following “I” or “P” frames (backward prediction). The compression is applied on the residual.
("B" and "P" frames can also change their behavior in macro-block level to "skip" and "intra", if the encoder decides that it will save more bits)

There are 3 main arrangements for GOPs (group of pictures - the distance between 2 “I” frames): I-only, IP, IBP.

I-Only (All-I)

In this case every frame is unique and encoded separately without any temporal prediction. Actually it is like Motion JPEG (MJPEG). The most primitive encoder.

IP

Between 2 "I" frames will be a variable number of "P" frames. In cameras the distance between the "I"s is usually 15 frames (for 30 fps footage). Smart software encoders usually rely on scene changes to push "I" frames.

IBP

Between "I" and "P" frames or 2 "P" frames will appear a constant number of "B" frames. Cameras usually use 2 "B" frames. Software encoders will usually push from 1 to 5 "B" frames. The increase in "B" frames increases the complexity of both the encoder and the decoder exponentially.

After understanding the above, I won't be explaining why IP and IBP has extreme superiority over I-only - I will show you an example.

The video below was captured absolutely uncompressed. I've compressed it using X264 with "Camera like settings" (with most of the limitations that hardware has). Video's motion: continuous horizontal pan. The frame I've decided to grab is frame no.94 because it is a "B" type frame in IbbP GOP, i.e. the most compressed + the bit rate control got steady.

(CLICK TO ENLARGE)

Original images (Click to download):

• IBBP - 18Mbps
• IBBP - 42Mbps
• All-I - 72Mbps

Full Side-by-Side videos (Click to download):

• All-I (72Mbps) vs IBBP (18Mbps) 53MB
• All-I (72Mbps) vs IBBP (42Mbps) 58MB

So why are the manufacturers so proud of 20 years regression in compression technology?

1. Good for editing purposes - well this is true. I-only is very light for today's workstations and enables you a real time playback of multi layered 1080p60 content. This is exactly why DNxHD and ProRes exist! Does it worth 4 times bigger files for the same quality? Personally, my workflow is to encode all the files to DNxHD 36mbps (terrible quality), edit them and then, when exporting, link all the timeline to the native files.
2. They claim that IP and IBP may cause temporal artifacts (such as ghosting and trails). It is because they have crappy H.264 encoders (H.264 isn't a constant encoder - it very depends on how it is calibrated and which tools are in use) and instead of improving their H.264 encoder - they choose the cheap and easy way.

For more experienced users I would suggest to encode master videos using X264 in IP, ~90Mbps with less deblocking, High profile with 4:2:2 (Instead of DNxHD/ProRes). The result is almost lossless! 

In summary,

Don't be afraid of H.264 IP/IBP and don't surrender to I-only. We should demand improvement in cameras' encoder. 

Mark.