Beamr compares their HEVC encoder to x265

By Tom Vaughan

Recently, a competitor (Beamr) published a blog post comparing their HEVC encoder to x265. They claimed that their HEVC encoder is faster, AND it produces better video quality.

Of course, we’re used to people comparing other HEVC encoders to x265.

x265 is available under an open source license, and therefore it is widely available for any competitor to use in such comparison tests.
x265 is by far the most widely known and widely used HEVC encoder, and so other companies want to build their brand by comparing to x265.
But most importantly, they know that x265 is the “gold standard” of HEVC encoders. It’s the benchmark that all others must try to beat.

So, did Beamr’s encoder really beat x265? Or are their claims just a bunch of marketing bluster, that isn’t backed up by the facts?

This latest test was conducted in a way that mostly followed the encoder comparison guidelines we recently published. But there were many flaws.

The bit rates produced were not identical. On average the Beamr encodes had 6% higher bit rates. In one case, the Beamr encode had a bit rate that was 18% higher. This was due to the fact that the test video sequences were very short, and one-pass encoding was used (giving the encoder less time to “dial in” to the target bit rate). This problem could have been avoided by using 2 pass encoding, which can achieve a more accurate bit rate while leveling quality across the encode.
All of the test parameters were hand-picked by Beamr. The test content, the test hardware, the settings used, including the bit rate, the number of hardware threads used and the x265 presets. This is known as “cherry picking” (the fallacy of incomplete evidence). We wonder how many tests were run other different conditions, in order to determine the tests that showed the Beamr encoder under the most favorable light.
- Beamr chose to use x265’s ultrafast, medium and veryslow presets. They claimed a big speed advantage against x265’s veryslow preset. Of course, as the name implies, we designed our veryslow preset to be very slow. It is focused on achieving the highest possible quality, and no compromises are made that would improve performance. The next preset, slower, is twice as fast, but has nearly identical encoding efficiency. Why didn’t Beamr compare their encoder to x265 with the slower preset? [Because they would have lost on a speed comparison, as well as on quality.]
- Beamr only chose video sequences with relatively low motion and detail. Clips like Bar Scene, Dinner Scene, and Wind And Nature are exceptionally easy to encode. Why no sports, or other high detail + high motion content? Perhaps it’s because Beamr relies on using Tiles, which cannot use inter-prediction across tile boundaries. x265 uses Wavefront Parallel Processing, which is more efficient.
- Beamr chose only 4K content, at 7 Mbps bit rates. Why didn’t they show compare the 2 encoders for 1080P, 720P and lower picture sizes? Why not compare the encoders at a wider range of bit rates?
- Beamr chose a 4 year old hardware architecture (Xeon E5 v2, code named “Ivy Bridge”) to run their comparison test on. These processors don’t support AVX2 instructions, which newer Xeon generations (Haswell, Broadwell, Purley) do. We wonder how the Beamr encoder compares to x265 on modern machines.

Beamr claims a speed advantage over x265 under these carefully selected conditions. Most commercial companies use a preset like “slow” or “slower”, but rarely use our veryslow preset for their high quality offline encoding. For real-time encoding, we have a more advanced encoding library called UHDkit that can run multiple encoding instances in parallel to achieve high performance on a many-core server, allowing for higher quality settings than x265’s ultrafast preset. On modern Haswell, Broadwell or Purley generation Xeon powered servers, customers who have tried competing HEVC encoders tell us that UHDkit outperforms the competition under a range of scenarios.

Beamr made the bitstreams available, so that anyone could compare the quality of the video produced. You should download the videos and compare for yourself. Beamr claimed that the visual quality of their encodes was clearly superior, but we are confused by this claim. Perhaps their definition of higher visual quality is different from ours. Perhaps by “higher quality”, they mean “softer, with less detail.” If you prefer video with more detail and accuracy, x265 is the clear winner.

To compare video, you can’t look at still images – you need to actually watch the video at full speed. We have a special video comparison tool (UHDcode Pro Player) that we make available to customers and partners which can play 2 streams simultaneously, letting you hide or reveal more or less of each stream. This makes it easy to see which encode is better. But the screen shots below are fairly representative of the difference in detail you will see in the competing encodes. Take a look at the texture on all of the surfaces (the road, the buildings, the water). Take a look a the sharpness of the detail on every object. Everyone we’ve talked to so far agrees that x265 produced the better video. That matches the feedback we get from our customers and prospective customers that have compared x265 to Beamr, and all of our other competitors. We consistently win these customer shoot-outs, based on the quality and performance of x265 and our premium encoding library, UHDkit.

On the Beamr blog, they’ve posted some screen shots, which you can download to see for yourself how much more detail the x265 encodes have. Here are some additional examples…

[one_half]x265: Ritual Dance 4K Medium Frame 412

[/one_half]