The Structure of HEVC Video

• Download source - 2.3 MB

Introduction

This tip is about HEVC(h265) video bitstream syntax. In this tip, you will find information about the main elements of the HEVC video. The places of some aspects of the original video (fps, size, aspect ratio) in encoded HEVC bitstream will be shown. Only high-level elements are reviewed. It can be interesting for beginners in HEVC and video compression in general.

To analyze the HEVC video, we will use HEVCESBrowser. HEVCESBrowser is the open source tool to display syntax elements of the encoded HEVC files.

Main HEVC Syntax Elements

HEVC bitstream is an ordered sequence of the syntax elements. Each syntax element is placed into a logical packet called a NAL (network abstraction layer) Unit. There are 64 different NAL Unit types. They can be grouped into 10 classes:

1. VPS - Video parameter set
2. SPS - Sequence parameter set
3. PPS - Picture parameter set
4. Slice (different types)
5. AUD - Access unit delimiter
6. EOS - End of sequence
7. EOB - End of bitstream
8. FD - Filler data
9. SEI - Supplemental enhancement information
10. Reserved and unspecified

VPS, SPS and PPS contain general video parameters. They provide a robust mechanism for conveying data that are essential to the decoding process. They can be either a part of bitstream or can be stored separately.

Slice NAL unit contains data from encoded video frame. It can contain full frame or its part. Each slice can be decoded independently, that is, without using information from any other slice. Thereby, slices can be used as a tool to support parallel encoding/decoding.
There are following slice types:

1. I-slice - slice with only intra prediction
2. P-slice - slice with inter prediction from one I or P slices
3. B-slice - slice with inter prediction from two I or P slices

There is one special slice type called IDR-slice. There are no references from slices after IDR-slice to slices before it.

AUD can be used for signaling about start of video frame.

FD can be used for bitrate smoothing.

SEI provides support for different types of metadata. It includes picture timing, color space information, etc.

This image is the conventional structure of HEVC bitstream. It starts with VPS, SPS and PPS elements. IDR slice follows them. Then there is P slice, after which there is a group of B slices, then again P, then B, etc.

Some Aspects of Original Video in Encoded HEVC Bitstream

Used further files: the_structure_of_hevc_video.zip

Frame per second value

Information about FPS is placed in sps:vui:num_units_in_tick, sps:vui:time_scale and vps:num_units_in_tick, vps:time_scale. FPS in SPS and VPS can be present together, one of them or neither.

FPS = sps:vui:time_scale / sps:vui:num_units_in_tick = vps:time_scale / vps:num_units_in_tick

For this file FPS = 24000 / 1001 = 23,97

Size of picture

Size of picture is stored in sps:pic_width_in_luma_samples and sps:pic_height_in_luma_samples.

In this example, size of original video frame is 1920x800.

Aspect ratio

Aspect ratio can be present as predefined constant or as a pair of horizontal and vertical values. Aspect ratio value is placed into sps::vui::aspect_ratio_idc. If sps::vui::aspect_ratio_idc==EXTENDED_SAR then sar_width and sar_height are used.

Predefined value is used in foreman_cif_sar2.hevc. For this file sps::vui::aspect_ratio_idc=2. Aspect ratio is 12:11.

The pair of sizes is used in foreman_cif_128-117.hevc.

The aspect ration of the original file is 128:117.

References

• HEVCESBrowser [https://github.com/virinext/hevcesbrowser]