This page describes the FFmpeg repository at a high level: its purpose, the libraries it provides, the command-line tools built from them, and how all components relate to each other. For detailed documentation of each subsystem, follow the links to the relevant wiki pages throughout this document.
FFmpeg is a cross-platform multimedia framework for decoding, encoding, transcoding, muxing, demuxing, streaming, filtering, and playing virtually any media format. The repository contains:
libav*, libsw*) providing reusable APIs.ffmpeg, ffprobe, ffplay) built on top of those libraries.configure + Makefiles).This page is a map of the whole codebase. Deeper treatment of each area is in child pages — see page 2 for the build system, page 3 for the core libraries, page 4 for command-line tools, page 5 for codec implementations, page 6 for container formats, page 7 for hardware acceleration, page 8 for filters, page 9 for network protocols, and page 10 for testing.
The top-level directories map closely to library boundaries:
| Directory | Contents |
|---|---|
libavutil/ | Foundational utilities shared by all other libraries. |
libavcodec/ | Codec (encoder/decoder) implementations and infrastructure. |
libavformat/ | Container format muxers and demuxers. |
libavfilter/ | Audio/video filter graph engine and filter implementations. |
libswscale/ | Image scaling and pixel format conversion. |
libswresample/ | Audio resampling and format conversion. |
libavdevice/ | Input/output device abstractions (capture, display). |
fftools/ | Source for the ffmpeg, ffprobe, and ffplay binaries. |
doc/ | Texinfo documentation and API change log. |
tests/ | FATE test harness and reference data. |
tools/ | Miscellaneous utilities (e.g., graph2dot). |
Sources: configure128-145 libavcodec/Makefile1-2 libavutil/Makefile1-2
The libraries form a layered dependency graph with libavutil at the foundation. Lower layers have no upward dependencies. The build system (configure script) generates conditional compilation flags (CONFIG_* macros) that control which components are included in each library.
Figure: Core library architecture and dependencies
Sources: configure1-60 libavcodec/Makefile1-75 libavcodec/version.h28-44 libavutil/version.h81-91
The foundational layer. Every other library depends on it. It provides shared data structures, memory management, and hardware context abstractions.
Key public headers and their primary types:
| Header | Key types / functions |
|---|---|
frame.h | AVFrame, av_frame_alloc, av_frame_ref, av_frame_unref |
packet.h | AVPacket, av_packet_alloc, av_packet_ref |
buffer.h | AVBufferRef, av_buffer_alloc, av_buffer_ref |
hwcontext.h | AVHWDeviceContext, AVHWFramesContext |
pixfmt.h | AVPixelFormat, pixel format enumerations |
channel_layout.h | AVChannelLayout, av_channel_layout_* API |
opt.h | AVOption, av_opt_set, av_opt_get |
log.h | av_log, AVClass |
rational.h | AVRational, av_rescale_q |
Sources: libavutil/version.h81-91 libavutil/frame.h39-40 libavutil/buffer.h17-18 libavutil/hwcontext.h44-45
Provides all codec implementations and the encode/decode API. It handles bitstream parsing, entropy coding, and motion compensation.
Key types:
AVCodecContext — per-stream codec state, holds all encoding/decoding parameters.AVCodec / FFCodec — public-facing and internal codec descriptor libavcodec/allcodecs.c35-50AVCodecParameters — codec parameters independent of a codec instance.AVCodecID — enumeration identifying every codec libavcodec/codec_id.h1-100The send/receive API is the primary interface:
avcodec_send_packet → avcodec_receive_frame.avcodec_send_frame → avcodec_receive_packet.All built-in codecs are registered through libavcodec/allcodecs.c. Recent additions include AV_CODEC_ID_PCM_DVDA doc/APIchanges5-6 and AV_CODEC_ID_JPEGXS doc/APIchanges146-147
Sources: libavcodec/version.h32-43 libavcodec/Makefile1-80 libavcodec/allcodecs.c32-38 libavcodec/codec_id.h5-6
Handles container format I/O — muxing and demuxing. It supports a vast range of formats from legacy AVI to modern ISOBMFF (MP4) and streaming protocols like DASH and HLS.
Key types:
AVFormatContext — top-level context for an open media file or stream.AVStream — represents one elementary stream within a container.AVInputFormat — registered demuxer descriptor.AVOutputFormat — registered muxer descriptor.Recent updates include support for AVStreamGroupLayeredVideo and Dolby Vision metadata in stream groups doc/APIchanges58-61
Sources: doc/APIchanges122-127 libavformat/riff.c1-50 doc/APIchanges58-61
A directed-graph filter engine. Filters are connected into an AVFilterGraph. Data flows as AVFrame objects through AVFilterLink connections between AVFilterContext nodes.
Sources: libavfilter/version.h1-26 libavfilter/Makefile1-26
Scales video frames and converts between pixel formats. Recent updates include a new graph-based architecture for complex scaling pipelines involving SwsGraph and SwsContext backends doc/APIchanges52-54 It now includes support for hardware format testing via sws_test_hw_format() doc/APIchanges119-120
Sources: Changelog59 doc/APIchanges52-54 doc/APIchanges105-107
Resamples audio and converts between sample formats and channel layouts. It handles dithering and custom channel remapping. It now accepts custom order channel layouts as input Changelog142-143
Sources: Changelog142-143 libswresample/version.h1-26
Three programs live under fftools/:
| Binary | Source entry point | Purpose |
|---|---|---|
ffmpeg | fftools/ffmpeg.c | Universal media converter / transcoder. |
ffprobe | fftools/ffprobe.c | Media file analysis and metadata extraction. |
ffplay | fftools/ffplay.c | Interactive media player. |
ffmpeg constructs a full transcoding pipeline at startup. Since FFmpeg 7.0, each stage (demuxing, decoding, filtering, encoding, and muxing) runs in parallel Changelog154-155
Figure: Transcoding pipeline data flow
Sources: Changelog154-155 MAINTAINERS33-41
FFmpeg supports a massive variety of codecs, many of which are registered in libavcodec/allcodecs.c.
| Category | Examples |
|---|---|
| Modern Standards | H.264, HEVC, VVC/H.266, AV1, VP9 libavcodec/allcodecs.c151-165 |
| Professional | ProRes, DNxHD, CineForm, DPX libavcodec/allcodecs.c95-98 |
| Lossless | FFV1, HuffYUV, APNG libavcodec/allcodecs.c119-121 |
| Legacy | MPEG-1/2, H.261, H.263, Cinepak libavcodec/allcodecs.c143-146 |
| Category | Examples |
|---|---|
| Standard | AAC (native/libfdk), AC-3, MP3, Opus, Vorbis |
| Lossless | FLAC, ALAC, WavPack |
| Specialized | ADPCM (IMA, MS, EA, etc.) libavcodec/adpcm.c1-20 |
| Experimental | xHE-AAC, MPEG-H 3D Audio Changelog42-56 |
Sources: libavcodec/allcodecs.c35-185 Changelog1-100 libavcodec/adpcm.c1-20
FFmpeg provides a unified hardware acceleration framework. Recent development has focused heavily on Vulkan and D3D12.
| Backend | Capability | Implementation Examples |
|---|---|---|
| Vulkan | Decode (H.264, HEVC, AV1), Encode, Filters | ff_ffv1_vulkan_encoder libavcodec/allcodecs.c120 vf_v360_vulkan Changelog19 |
| D3D12VA | Decode, Encode (H.264, HEVC, AV1) | ff_h264_d3d12va_encoder Changelog43 vf_scale_d3d12 Changelog50 |
| AMF | Encode, Filters | amfenc.c, vf_vpp_amf Changelog16-31 |
Sources: libavcodec/Makefile77-102 Changelog13-32 libavcodec/allcodecs.c49-120 doc/APIchanges14-15
The build system is driven by a configure script that detects the environment and enables/disables components based on user flags and available external libraries.
Key configure options:
--enable-gpl / --enable-nonfree: Licensing control configure97-103--disable-everything: Minimal build starting point configure148--list-decoders / --list-encoders: Discovery of available components configure67-68Sources: configure1-168
The project is maintained by a global community. Quality is ensured through automated testing and fuzzing.
Quality is ensured through:
target_dec_fuzzer.c .forgejo/CODEOWNERS221Sources: MAINTAINERS1-100 configure114 .forgejo/CODEOWNERS221 MAINTAINERS67
Refresh this wiki
This wiki was recently refreshed. Please wait 2 days to refresh again.