3 Reasons Why AV1 Encoding is not Ready for Real-Time Streaming

AV1 is a new video codec with a very promising –and this cannot be emphasized enough– future.

The buzz around AV1 centers upon its ability to more effectively compress video. This is really exciting for large VOD based companies like Netflix as AV1 can reduce bandwidth requirements by 30-50% beyond currently employed codecs. Being able to stream the same content while paying for less data is important for VOD and OTT applications.

However, for live video use-cases concerned with latency (i.e. everything not VOD), bandwidth is only a part of the equation. This brings us to our main point:

AV1 cannot currently support real-time live streaming. Read on to find out the reasons why.

1. Encoder Speed

Originally announced in 2015, the AV1 codec is still maturing. Though it has gained a foothold in the video streaming infrastructure it is still working on establishing a real footprint and has ways to go before widespread adoption.

While there are performant AV1 decoders in the works such as dav1d, the issue lies with encoding speed.

Currently, all the encoders that implement AV1 cannot process the video quick enough. For any live stream, upon capturing the video, the codec must be written via an encoder before it can be streamed out. A test conducted by Graphics and Video Lab Media Group, showed AV1 encoding times 2,500 to 3,000 times slower than competitors.

No matter how well you compress the video, if it takes too long to actually perform that compression task, the efficiency gained by reducing the bitrate will be far outweighed by the slow encoding speed, and thus latency it adds to the stream.

Without a fast encoder, AV1 support is pointless for any real-time applications.

Part of the reason it is proving difficult to write an efficient real-time encoder is due to the way AV1 compresses video. Essentially, the AV1 algorithm simplifies, organizes and reduces the amount of data necessary to deliver a video stream. Specifically it works around simplifying the data at the RTP payload format level. The RTP payload format contains the necessary parameters of the media encoding of the video, such as signal sampling rate, frame size and timing.

AV1 uses the concept of predefined scalability structures. These are a set of commonly used picture prediction structures that can be referenced simply via a template. This reduces overhead, as highly redundant information can be predefined and sent once.

Furthermore, each video input frame contains two layers; a spatial layer which defines the frame rate and resolution along with a quality layer which allows for the same resolution at different quality settings.  AV1 supports quality layers as spatial layers without any resolution changes which further simplifies the amount of data needed. All of this maintains high quality imagery while also reducing the size of the video and thus the bandwidth required to send it.

After reading all of this, you may be thinking “If AV1 makes the whole encoding process much simpler, shouldn’t it be easier to work with it? Why does it take such a long time to encode?”.

As with many other things in the tech world, simplifying one end of the equation can result in more complex concerns on the other side. Despite having a more efficient compression algorithm, the actual writing of the format takes more work. The AV1 encoder implementation will have to do a whole lot more gathering the pre-encoding data than an equivalent AVC, HEVC, or VP9 encoder.

We do think that eventually someone will write a very fast implementation (SocioNext is already making good headway for example). Of course, computers will continue to increase in processor speed which speeds everything up. Lastly, once fast AV1 hardware encoders (more on those later) go into the chip level, that will benefit the pre-encoding data gathering process. Once those factors are realized, a real-time AV1 encoder will be much more viable.

2. Almost Zero Software Encoders

As mentioned above, not only are current software encoders for AV1 too slow, but also too few.

While desktop browsers such as Firefox, Google, and Opera currently support the decoding of AV1 it will be a while before they can write an efficient software-based encoder. Either that or they will have to wait until AV1 hardware encoders show-up in people’s laptops and phones which is also going to take a really long time. For now there is no browser support at all.

Bitmovin built a software encoder and according to their own tests:

The encoding runs with about 1.93 fpm (frames per minute) which would translate to 0.032 fps (frames per second). If you want to achieve real-time with 24 fps you would need at least 746 times the computing power on a single machine, which is not very practical in a real-world scenario. Clearly we need another approach to encode with reasonable speeds, especially when it comes to live-streaming.

3. No Production Ready Hardware Encoders

As a new protocol, it’s going to be a while before groups like Haivision or Videon add AV1 support into their hardware encoders and eventually release the product. Alternatively, H.264 encoders enjoy nearly ubiquitous support on hardware chips and can perform the encoding process in 10 or 15 milliseconds.

However, efficient hardware based encoders are currently in development by groups like SocioNext. They made headlines when they revealed the first real-time encoder with AWS which they demonstrated at NAB 2019. Although this is obviously exciting news, there is a long distance between creating a demo and releasing a production ready product.

What’s Next for AV1?

As a forward-facing tech company, Red5 Pro acknowledges that AV1 will one day be a very useful technology especially as it will make high quality video delivery much more efficient then current options available. So we are paying close attention to these efforts and looking to take advantage of them.

Not only are we paying attention to developments but we are actively running experiments with partners in the streaming industry. AV1 isn’t a viable option right now, but it certainly will be. Keep an eye out here for exciting announcements about AV1 support for Red5 Pro in the future.

Like many things in the tech field, the latest developments promising improvements will take some time to actually deliver those improvements. For example,  early implementations of 4K TVs didn’t really have access to 4K streams. Similarly, AV1 encoding is a really cool piece of technology without practical applications for low latency AV1 streams.

We look forward to adding it to our toolset once it’s fully ready to go.

Any questions? Want to find out more about our tech roadmap or current offerings? Send an email to info@red5pro or schedule a call.