Make sure to visit us at our kiosk in the Intel booth at 5.B65, where we will be demonstrating enhancements to x265 that improve the throughput of encoding for ABR streaming by over 2X by leveraging Machine Learning on the CPU. We continue to innovate in this space to bring ground-breaking improvement in performance and quality by leveraging our expertise in video compression algorithms, machine learning technology, and microarchitecture-aware optimizations that enabled us to use Intel AVX512 instructions while encoding.
Adaptive Multi-Resolution Encoding in x265
x265’s ability to leverage AI to accelerate encoding Adaptive Bitrate Streaming (ABR) has been presented in the paper titled “Adaptive Multi-Resolution Encoding Scheme for ABR Streaming” that is to appear in the proceedings of IEEE International Conference on Image Processing (ICIP), 2018. Adaptive streaming enables dynamic adaptation to changes in network conditions by encoding video content in multiple bitrates and resolutions. Multi-pass x265 encodes exploit the structural redundancy across multiple resolutions by sharing analysis data in the order of increasing resolution, thereby reducing the computational burden significantly.
Static and dynamic refinement features were conceptualized for the Adaptive streaming topology in x265. As the first step, the input video sequence is scaled down to the lowest resolution of the bitrate ladder and is encoded. Coding metadata such as CU quadtree structure, PU predictions, coding modes and motion vectors (MVs) are stored during this first pass encode at CTU level abstraction. The subsequent higher-resolution encodes invoke either the static or the dynamic refinement algorithm. The figure below depicts this architecture for a three pass system. Theoretically, this can be extended to N passes.
The static refinement algorithm may further be classified as intra-picture and inter-picture refinement wherein the information from the previous pass is used at different levels of granularity thereby giving the current encode a head start. The encoder intelligently refines the analysis data based on certain heuristics and later uses it in the Rate Distortion Optimization (RDO) process. By tweaking the degree of information that is being reused, the user can control the performance vs quality balance of the encode.
The dynamic refinement algorithm applies Bayesian classification on the information saved in the previous pass and identifies patterns between the complexity of the content and the refinement level being used. Based on these patterns, the dynamic refinement algorithm switches between the static refinement levels. This will allow the encoder to find the optimal balance between performance and quality while taking the content complexity into account, making it more efficient than the static refinement methods.
Compared to conventional single-resolution approaches, the computational complexity of higher resolution encodes is drastically reduced with the use of the AI-based adaptive multi-resolution technique. The adaptive multi-resolution scheme is able to achieve a whopping speedup of about 2.5X, with a negligible drop in the quality. The above table from the research paper compares the speed vs quality of static and dynamic refinement levels. Since the AI tools used in x265 are not based on CNN models, x265 is able to leverage the advances in CPU technology to achieve these improvements.
Performance and Energy Consumption Analysis of x265
By Dieison Silveira (Ph.D. Student, PGMICRO/UFRGS)
& Vignesh V Menon
A detailed energy consumption analysis, accounting for the consumption components of CPU, cache memories and main memory, for all x265 presets executing in a multi-core system was presented in the paper titled ‘Performance and Energy Consumption Analysis of the x265 Video Encoder’ published in 2017 25th European Signal Processing Conference (EUSIPCO). The Holy Grail of preset options in x265 is the optimized trade-off between encoding speed and compression efficiency. This work, on the other hand, presents a detailed performance and energy consumption analysis of the x265 video encoder, considering the energy consumption related to CPU, main memory, and caches. The figure below depicts the energy efficiency considering the energy consumption per encoded byte.
According to the paper, ultrafast preset has a bitrate 45% higher than the placebo preset. The results also showed that the system energy consumption increases 45x, from ultrafast preset to the placebo preset. This happens because slower presets tests more encoding options and are more computationally intensive, spending more energy. The paper recommends the user to choose the presets depending on the video resolution, memory, and target platform. Some of the key noteworthy findings from the paper:
- Fast, medium and slow presets present the best bitrate/energy trade-offs. In this way, if the device has bitrate and energy constraints these three presets are good options.
- For energy-constrained devices, such as battery-powered embedded systems, a better solution is the use of fastest presets, which have the lowest energy consumption.
- For high-resolution videos such as UHD 4K and 8K, slower presets are recommended, since these presets achieve best compression ratio.
The figure above presents the behavior of bitrate and energy consumption for all presets. Please find the full paper here.
We encourage everyone working on x265 or using x265 as a tool to evaluate any video quality metric, pre and post processing tools to write to us about their observations and recommendations to improve x265. We welcome to share their findings on some visual quality or performance improvement algorithms which could be beneficial if implemented in x265.
Disclaimer: The graphs presented in the blog are from the research paper.
Detailed white-paper on our results from AVX512 acceleration
By Pradeep Ramachandran
A month or so back, we published a white-paper that gave high-level results on the encoder gains that we saw from our recently implemented AVX512 kernels in x265. We have now published a more detailed paper that tears the results down into more detail, including per-kernel gains for AVX512 over AVX2. The paper can be accessed from here. We hope you find the detailed analysis more informative to understand the reasons behind the results that you see on your hardware.
Do share your findings with us. As you have probably already noticed, AVX512 is part of the recently-released v2.8 of x265!
x265 Version 2.8 Released
x265 version 2.8 with cool new features including AVX512 accelerating, dynamic refinement (that enables optimized encoding for ABR streaming), support for measuring VMAF has been released! More details in our release notes at http://x265.readthedocs.io/en/stable/releasenotes.html#version-2-8
x265 turns 5!
As it turns out, the x265 project turned 5 a couple of months back; our first commits from the HM encoder date back to March, 2013. And being the geeks that we are, we didn’t even realize it!
Many thanks to all those who have enabled x265 accomplish all that it has over 5 years! We continue to innovate inside x265 to improve both quailty, and performance and look forward to celebrating our 10th anniversary with you.
Cheers,
Team-x265.
Thanks for a great NAB 2018
By Pradeep Ramachandran
Now that the dust has settled, it is time to thank all the contributors for enabling a great showing by x265 at NAB 2018. We showed-off our new ML-accelerated content adaptive encoding for ABR, AVX-512 acceleration, and the recently added support for HDR10+/HLG at NAB. We received great feedback on what people would like to see in the coming releases, and will be working hard to continue to innovate in that space. We’ve also formed a committee to guide the future development of x265, and other open source media codecs that we will blog about more in the coming weeks; read this article for an initial idea of what this is about.
Until our next showing, don’t hesitate to reach out to us on the developer mailing list, doom9 (), or Facebook to talk.
And finally, avx512 in x265!
By Pradeep Ramachandran
Finally, the acceleration that we’ve all been waiting for is here! We’ve been working extensively with Intel for the last few months to use Intel Advanced Vector Extensions 512 (AVX-512) to accelerate x265. After much effort, we’re delighted to share that we’ve been able to accelerate 4K HDR encoding in main10 profile by over 15% for high-quality offline encoding. Checkout this white-paper on the Intel site for more information.
The patches will be pushed to the default branch soon. Let us know the results of your tests – you know where to find us!
Excited about AV1 closing doors, but…
By Pradeep Ramachandran
After what seems to have been a long delay, AV1 finally froze its bitstream last week! Like many folks in the industry, we have been waiting for this moment for a long time to see what a truly ‘royalty-free’ codec can can bring in terms of tools to the encoding space.
A few months ago, we stopped looking further into how AV1’s tools compare to that of HEVC due to this peer-review paper published in a leading journal from leading researchers in the field of video coding. That paper reported that the HM encoder provide average bitrate savings of 30% relative to AV1 with an encoding speed that was 25X that of the AV1 encoder; the informed would know that x265’s veryslow is very comparable in encoder efficiency to the HM encoder. While optimizations could bridge the gap in speed, bridging the gap in encoder efficiency would be hard unless some fundamental improvements (that are not covered by the HEVC-patents, mind you) are made. Now that the bitstream is frozen, we will be digging to see what new tools have been brought to the table that were not included in this comparison in the hope to answer the question “Is AV1 fundamentally better than HEVC as a standard?”. Stay tuned here to hear more, or share your thoughts on doom9/x265-devel mailing list.
And of course, there is the issue of patent royalties and licensing, but we will leave that up to the lawyers to deliberate and decide; lets talk more tech here!
NAB 2018, of course we will be there!
By Pradeep Ramachandran
MulticoreWare, the developers of x265, will be available to discuss all things media at NAB 2018 in Las Vegas from April 9th – 12th in their booth at SU-14708. Swing by to talk about the soon-to-publish AVX512 acceleration, content adaptive optimizations for ABR encoding with x265, or if you just want a selfie with the creators of the world’s most popular HEVC encoder.
See you in Vegas!
PS: Make sure to mention this blog post if you stop by to stand a chance to win some open-source memorabilia!