Proposal to lower target video bitrates (again)

[lisamelton]

Proposal to lower target video bitrates (again)

The 0.6 release of the video_transcoding Gem contained perhaps the most significant change to the transcode-video tool since the deprecated --big option became the default behavior. Not only was the ratecontrol system enhanced to produce higher quality, the target bitrates for 480i and 720p content were lowered to limit the higher bitrates produced by that higher quality.

For the upcoming 0.7 release, I propose to lower those video bitrate targets again, and not just for 480i and 720p content. The targets for 2160p (4K/UHD) content would also be lowered. However, I'm not proposing any change to bitrate targets for 1080p.

Also different for 0.7 would be lowering the targets when using the --small or --small-video options.

Default target video bitrate

Input resolution	Version 0.5.x	Version 0.6	Proposed 0.7
2160p or 4K/UHD video	16000 Kbps	(same)	12000 Kbps
1080p or Blu-ray video	8000 Kbps	(same)	(same)
720p	6000 Kbps	5000 Kbps	4000 Kbps
480i, 576p or DVD video	3000 Kbps	2500 Kbps	2000 Kbps

Target video bitrate with --small or --small-video

Input resolution	Version 0.5.x	Version 0.6	Proposed 0.7
2160p or 4K/UHD video	10000 Kbps	(same)	8000 Kbps
1080p or Blu-ray video	5000 Kbps	(same)	(same)
720p	4000 Kbps	(same)	3000 Kbps
480i, 576p or DVD video	2000 Kbps	(same)	1600 Kbps

OK, what's the rationale for doing this?

First, the new ratecontrol system introduced in version 0.6 with a constant rate factor (CRF) quality target of 1 does much better at putting pressure on x264 to hit that target bitrate than the old quality target of 16 ever did. So, bitrates are higher now. Weirdly, 480i and 720p content is suddenly too good. And that seems very wasteful of space.

Second, these proposed targets seem to be more in line with what Apple, Netflix, warez (you know what I mean), etc. are doing now as opposed to what they were doing when the project started.

This makes sense to me and my tests so far indicate that quality is still good with lower target video bitrates.

BTW, you can try any of these targets yourself by setting vbv-maxrate directly like this:

transcode-video --encoder-option vbv-maxrate=4000 "/path/to/Movie.mkv"

So, I'm asking for your feedback. Let me know what you think. Soon. Because I'm liable to make the change as early as next week.

Thanks.

[J-Swift]

I've got nothing to add to your proposal, I just want to say I really appreciate the work you do on this tool, and what a great maintainer you are in general.

In the limited time I've spent trying to learn about the nuances of encoding, I've realized I don't have time to learn about the nuances of encoding. Having this available has enabled me to completely offload the actual video encoding, and instead focus my time and energy on the rest of the automated processing pipeline. It means a lot that I can fully trust that the tool you put out is going to 'do the right thing' and give me the best shot at optimizing any video I throw at it. And I've not had to spend any time debugging issues due to anything your tool has been responsible for.

So, keep it up and thanks again!

[lisamelton]

@J-Swift Wow! Such praise will just go to my head. :) Seriously, thank you so much! People like yourself are exactly why I keep doing this. (Well, you and the fact that I'm completely nuts.)

[bpharriss]

Don,
What effect do you expect your changes to have on the file sizes of DVD-sourced video? I know the answer is probably "It depends", but I just wanted to make sure.

I took 9 movies that I had previously encoded with 0.5.x and the --big flag, and ran them through transcode-video again with 0.6, using default options. I then ran them through again, using --encoder-option vbv-maxrate=2000.

Visual exam: I wouldn't call my eyesight super-sharp by any stretch of the imagination, but in doing a quick review of the video files generated, the 0.6 files look better than the 0.5.x files, and I couldn't really tell the difference between the files generated with 0.6 defaults and 0.6 with --encoder-option vbv-maxrate=2000. So my vote is to go ahead and make that change moving forward for DVD-type sources, IMVHO.

What I wasn't sure about was if the file sizes should then move consistently as well. In my 9 test files, 5 of them got smaller between 0.5.x and 0.6, while 4 of them got larger. In the case of Spectre, it grew by 50%.
When comparing 0.5.x with 0.6 and --encoder-option vbv-maxrate=2000, 6 of the files were smaller, and 3 were larger, but by a much smaller percentage.

Here are the movies I ran the tests on (please don't judge) and the different file sizes that resulted, in GB.

Filename	0.5.x with --big	0.6 default options	0.6 with vbv-maxrate=2000
101 Dalmatians (1961)	1.6	1.45	1.17
Alvin & Chipmunks Road Chimp (2015)	1.75	1.93	1.62
Ghostbusters (1984)	2.4	2.34	1.96
Ghostbusters 2 (1989)	2.45	2.4	2.01
Hotel Transylvania 2 (2015)	1.44	1.82	1.52
Spectre (2015)	2.03	3.19	2.66
Spider-Man (2002)	2.79	2.66	2.23
The Good Dinosaur (2015)	2	1.98	1.66
The Peanuts Movie (2015)	1.41	1.77	1.48

This this about what you expected to happen?

And, if I may be so bold: I echo @J-Swift sentiments above.

[lambdan]

I'm just re-posting here what I replied to Don on Twitter: I'm onboard.
I couldn't tell any difference between any of the transcodes in terms of quality but I really like the file size reduction:

I'm sure the file size reduction isn't gonna be as good on every transcode (and they might even become bigger as the person above me has noted) but if just a majority of them are smaller that's alot of freed up space.

[lisamelton]

@bpharriss This is great data! Thanks do doing all these tests and collating the results so nicely!

(And don't apologize for your title selection. I have most of the same movies. :)

Yes, this is what I expected, i.e. that the size of some DVD transcodings will actually be larger between 0.5.x and 0.6. That's due to the new ratecontrol system moving bitrates closer to the targets. Which, of course, is why I want to lower the targets for 0.7. Now, that won't lower all sizes below 0.5.x levels, but the average size will be slightly lower if multiple videos are considered together.

Hopefully, they will look as good, if not better, too. Which I think is what you're seeing.

And thank you for the kind praise! Much appreciated, sir.

[lisamelton]

@lambdan Thanks for posting this in the GitHub issue! I was just going to do that after responding to you on Twitter. :)

And thanks for doing all these tests!

Yes, the sizes will vary somewhat depending on complexity, grain, etc., but overall they should be slightly smaller if several are considered together.

I'm glad you like the visual results, too, with a vote of confidence. The new ratecontrol system is finally making some of my old, grainy black and white full frame movies look halfway decent.

[lisamelton]

Update: After more testing by myself and others (thank you all), I'm much more confident in my proposed video bitrate target changes for 480p and 720p content.

However, I'm now doubting the wisdom of changing the 2160p (4K) targets. This is because no matter how low I set the target, I can't get the bitrate below 10000 Kbps for any of my, admittedly few, 2160p samples using the new ratecontrol system.

Since I don't even document that transcode-video supports 2160p and nobody has complained about any bitrate bloat yet, I'll probably just leave these untouched for version 0.7.

[lisamelton]

OK, version 0.7.0 of transcode-video has been released! A bit early, I suppose. :) As always, just...

gem update video_transcoding

...to get the latest version.

Thanks again for all your help!