\input texinfo @c -*- texinfo -*- @documentencoding UTF-8 @settitle Developer Documentation @titlepage @center @titlefont{Developer Documentation} @end titlepage @top @contents @chapter Introduction This text is concerned with the development @emph{of} FFmpeg itself. Information on using the FFmpeg libraries in other programs can be found elsewhere, e.g. in: @itemize @bullet @item the installed header files @item @url{http://ffmpeg.org/doxygen/trunk/index.html, the Doxygen documentation} generated from the headers @item the examples under @file{doc/examples} @end itemize For more detailed legal information about the use of FFmpeg in external programs read the @file{LICENSE} file in the source tree and consult @url{https://ffmpeg.org/legal.html}. If you modify FFmpeg code for your own use case, you are highly encouraged to @emph{submit your changes back to us}, using this document as a guide. There are both pragmatic and ideological reasons to do so: @itemize @bullet @item Maintaining external changes to keep up with upstream development is time-consuming and error-prone. With your code in the main tree, it will be maintained by FFmpeg developers. @item FFmpeg developers include leading experts in the field who can find bugs or design flaws in your code. @item By supporting the project you find useful you ensure it continues to be maintained and developed. @end itemize All proposed code changes should be submitted for review to @url{mailto:ffmpeg-devel@@ffmpeg.org, the development mailing list}, as described in more detail in the @ref{Submitting patches} chapter. The code should comply with the @ref{Development Policy} and follow the @ref{Coding Rules}. The developer making the commit and the author are responsible for their changes and should try to fix issues their commit causes. @anchor{Coding Rules} @chapter Coding Rules @section Language FFmpeg is mainly programmed in the ISO C99 language, extended with: @itemize @bullet @item Atomic operations from C11 @file{stdatomic.h}. They are emulated on architectures/compilers that do not support them, so all FFmpeg-internal code may use atomics without any extra checks. However, @file{stdatomic.h} must not be included in public headers, so they stay C99-compatible. @end itemize Compiler-specific extensions may be used with good reason, but must not be depended on, i.e. the code must still compile and work with compilers lacking the extension. The following C99 features must not be used anywhere in the codebase: @itemize @bullet @item variable-length arrays; @item complex numbers; @item mixed statements and declarations. @end itemize @subsection SIMD/DSP @anchor{SIMD/DSP} As modern compilers are unable to generate efficient SIMD or other performance-critical DSP code from plain C, handwritten assembly is used. Usually such code is isolated in a separate function. Then the standard approach is writing multiple versions of this function – a plain C one that works everywhere and may also be useful for debugging, and potentially multiple architecture-specific optimized implementations. Initialization code then chooses the best available version at runtime and loads it into a function pointer; the function in question is then always called through this pointer. The specific syntax used for writing assembly is: @itemize @bullet @item NASM on x86; @item GAS on ARM. @end itemize A unit testing framework for assembly called @code{checkasm} lives under @file{tests/checkasm}. All new assembly should come with @code{checkasm} tests; adding tests for existing assembly that lacks them is also strongly encouraged. @subsection Other languages Other languages than C may be used in special cases: @itemize @bullet @item Compiler intrinsics or inline assembly when the code in question cannot be written in the standard way described in the @ref{SIMD/DSP} section. This typically applies to code that needs to be inlined. @item Objective-C where required for interacting with macOS-specific interfaces. @end itemize @section Code formatting conventions There are the following guidelines regarding the indentation in files: @itemize @bullet @item Indent size is 4. @item The TAB character is forbidden outside of Makefiles as is any form of trailing whitespace. Commits containing either will be rejected by the git repository. @item You should try to limit your code lines to 80 characters; however, do so if and only if this improves readability. @item K&R coding style is used. @end itemize The presentation is one inspired by 'indent -i4 -kr -nut'. @subsection Vim configuration In order to configure Vim to follow FFmpeg formatting conventions, paste the following snippet into your @file{.vimrc}: @example " indentation rules for FFmpeg: 4 spaces, no tabs set expandtab set shiftwidth=4 set softtabstop=4 set cindent set cinoptions=(0 " Allow tabs in Makefiles. autocmd FileType make,automake set noexpandtab shiftwidth=8 softtabstop=8 " Trailing whitespace and tabs are forbidden, so highlight them. highlight ForbiddenWhitespace ctermbg=red guibg=red match ForbiddenWhitespace /\s\+$\|\t/ " Do not highlight spaces at the end of line while typing on that line. autocmd InsertEnter * match ForbiddenWhitespace /\t\|\s\+\%#\@@ 5 lines) chunk of code, then either do NOT change the indentation of the inner part within (do not move it to the right)! or do so in a separate commit @subheading Credit the author of the patch. Make sure the author of the commit is set correctly. (see git commit --author) If you apply a patch, send an answer to ffmpeg-devel (or wherever you got the patch from) saying that you applied the patch. @subheading Always wait long enough before pushing changes Do NOT commit to code actively maintained by others without permission. Send a patch to ffmpeg-devel. If no one answers within a reasonable time-frame (12h for build failures and security fixes, 3 days small changes, 1 week for big patches) then commit your patch if you think it is OK. Also note, the maintainer can simply ask for more time to review! @section Code @subheading Warnings for correct code may be disabled if there is no other option. Compiler warnings indicate potential bugs or code with bad style. If a type of warning always points to correct and clean code, that warning should be disabled, not the code changed. Thus the remaining warnings can either be bugs or correct code. If it is a bug, the bug has to be fixed. If it is not, the code should be changed to not generate a warning unless that causes a slowdown or obfuscates the code. @section Library public interfaces Every library in FFmpeg provides a set of public APIs in its installed headers, which are those listed in the variable @code{HEADERS} in that library's @file{Makefile}. All identifiers defined in those headers (except for those explicitly documented otherwise), and corresponding symbols exported from compiled shared or static libraries are considered public interfaces and must comply with the API and ABI compatibility rules described in this section. Public APIs must be backward compatible within a given major version. I.e. any valid user code that compiles and works with a given library version must still compile and work with any later version, as long as the major version number is unchanged. "Valid user code" here means code that is calling our APIs in a documented and/or intended manner and is not relying on any undefined behavior. Incrementing the major version may break backward compatibility, but only to the extent described in @ref{Major version bumps}. We also guarantee backward ABI compatibility for shared and static libraries. I.e. it should be possible to replace a shared or static build of our library with a build of any later version (re-linking the user binary in the static case) without breaking any valid user binaries, as long as the major version number remains unchanged. @subsection Adding new interfaces Any new public identifiers in installed headers are considered new API - this includes new functions, structs, macros, enum values, typedefs, new fields in existing structs, new installed headers, etc. Consider the following guidelines when adding new APIs. @subsubheading Motivation While new APIs can be added relatively easily, changing or removing them is much harder due to abovementioned compatibility requirements. You should then consider carefully whether the functionality you are adding really needs to be exposed to our callers as new public API. Your new API should have at least one well-established use case outside of the library that cannot be easily achieved with existing APIs. Every library in FFmpeg also has a defined scope - your new API must fit within it. @subsubheading Replacing existing APIs If your new API is replacing an existing one, it should be strictly superior to it, so that the advantages of using the new API outweight the cost to the callers of changing their code. After adding the new API you should then deprecate the old one and schedule it for removal, as described in @ref{Removing interfaces}. If you deem an existing API deficient and want to fix it, the preferred approach in most cases is to add a differently-named replacement and deprecate the existing API rather than modify it. It is important to make the changes visible to our callers (e.g. through compile- or run-time deprecation warnings) and make it clear how to transition to the new API (e.g. in the Doxygen documentation or on the wiki). @subsubheading API design The FFmpeg libraries are used by a variety of callers to perform a wide range of multimedia-related processing tasks. You should therefore - within reason - try to design your new API for the broadest feasible set of use cases and avoid unnecessarily limiting it to a specific type of callers (e.g. just media playback or just transcoding). @subsubheading Consistency Check whether similar APIs already exist in FFmpeg. If they do, try to model your new addition on them to achieve better overall consistency. The naming of your new identifiers should follow the @ref{Naming conventions} and be aligned with other similar APIs, if applicable. @subsubheading Extensibility You should also consider how your API might be extended in the future in a backward-compatible way. If you are adding a new struct @code{AVFoo}, the standard approach is requiring the caller to always allocate it through a constructor function, typically named @code{av_foo_alloc()}. This way new fields may be added to the end of the struct without breaking ABI compatibility. Typically you will also want a destructor - @code{av_foo_free(AVFoo**)} that frees the indirectly supplied object (and its contents, if applicable) and writes @code{NULL} to the supplied pointer, thus eliminating the potential dangling pointer in the caller's memory. If you are adding new functions, consider whether it might be desirable to tweak their behavior in the future - you may want to add a flags argument, even though it would be unused initially. @subsubheading Documentation All new APIs must be documented as Doxygen-formatted comments above the identifiers you add to the public headers. You should also briefly mention the change in @file{doc/APIchanges}. @subsubheading Bump the version Backward-incompatible API or ABI changes require incrementing (bumping) the major version number, as described in @ref{Major version bumps}. Major bumps are significant events that happen on a schedule - so if your change strictly requires one you should add it under @code{#if} preprocesor guards that disable it until the next major bump happens. New APIs that can be added without breaking API or ABI compatibility require bumping the minor version number. Incrementing the third (micro) version component means a noteworthy binary compatible change (e.g. encoder bug fix that matters for the decoder). The third component always starts at 100 to distinguish FFmpeg from Libav. @anchor{Removing interfaces} @subsection Removing interfaces Due to abovementioned compatibility guarantees, removing APIs is an involved process that should only be undertaken with good reason. Typically a deficient, restrictive, or otherwise inadequate API is replaced by a superior one, though it does at times happen that we remove an API without any replacement (e.g. when the feature it provides is deemed not worth the maintenance effort, out of scope of the project, fundamentally flawed, etc.). The removal has two steps - first the API is deprecated and scheduled for removal, but remains present and functional. The second step is actually removing the API - this is described in @ref{Major version bumps}. To deprecate an API you should signal to our users that they should stop using it. E.g. if you intend to remove struct members or functions, you should mark them with @code{attribute_deprecated}. When this cannot be done, it may be possible to detect the use of the deprecated API at runtime and print a warning (though take care not to print it too often). You should also document the deprecation (and the replacement, if applicable) in the relevant Doxygen documentation block. Finally, you should define a deprecation guard along the lines of @code{#define FF_API_ (LIBAVBAR_VERSION_MAJOR < XX)} (where XX is the major version in which the API will be removed) in @file{libavbar/version_major.h} (@file{version.h} in case of @code{libavutil}). Then wrap all uses of the deprecated API in @code{#if FF_API_ .... #endif}, so that the code will automatically get disabled once the major version reaches XX. You can also use @code{FF_DISABLE_DEPRECATION_WARNINGS} and @code{FF_ENABLE_DEPRECATION_WARNINGS} to suppress compiler deprecation warnings inside these guards. You should test that the code compiles and works with the guard macro evaluating to both true and false. @anchor{Major version bumps} @subsection Major version bumps A major version bump signifies an API and/or ABI compatibility break. To reduce the negative effects on our callers, who are required to adapt their code, backward-incompatible changes during a major bump should be limited to: @itemize @bullet @item Removing previously deprecated APIs. @item Performing ABI- but not API-breaking changes, like reordering struct contents. @end itemize @section Documentation/Other @subheading Subscribe to the ffmpeg-devel mailing list. It is important to be subscribed to the @uref{https://lists.ffmpeg.org/mailman/listinfo/ffmpeg-devel, ffmpeg-devel} mailing list. Almost any non-trivial patch is to be sent there for review. Other developers may have comments about your contribution. We expect you see those comments, and to improve it if requested. (N.B. Experienced committers have other channels, and may sometimes skip review for trivial fixes.) Also, discussion here about bug fixes and FFmpeg improvements by other developers may be helpful information for you. Finally, by being a list subscriber, your contribution will be posted immediately to the list, without the moderation hold which messages from non-subscribers experience. However, it is more important to the project that we receive your patch than that you be subscribed to the ffmpeg-devel list. If you have a patch, and don't want to subscribe and discuss the patch, then please do send it to the list anyway. @subheading Subscribe to the ffmpeg-cvslog mailing list. Diffs of all commits are sent to the @uref{https://lists.ffmpeg.org/mailman/listinfo/ffmpeg-cvslog, ffmpeg-cvslog} mailing list. Some developers read this list to review all code base changes from all sources. Subscribing to this list is not mandatory. @subheading Keep the documentation up to date. Update the documentation if you change behavior or add features. If you are unsure how best to do this, send a patch to ffmpeg-devel, the documentation maintainer(s) will review and commit your stuff. @subheading Important discussions should be accessible to all. Try to keep important discussions and requests (also) on the public developer mailing list, so that all developers can benefit from them. @subheading Check your entries in MAINTAINERS. Make sure that no parts of the codebase that you maintain are missing from the @file{MAINTAINERS} file. If something that you want to maintain is missing add it with your name after it. If at some point you no longer want to maintain some code, then please help in finding a new maintainer and also don't forget to update the @file{MAINTAINERS} file. We think our rules are not too hard. If you have comments, contact us. @anchor{Submitting patches} @chapter Submitting patches First, read the @ref{Coding Rules} above if you did not yet, in particular the rules regarding patch submission. When you submit your patch, please use @code{git format-patch} or @code{git send-email}. We cannot read other diffs :-). Also please do not submit a patch which contains several unrelated changes. Split it into separate, self-contained pieces. This does not mean splitting file by file. Instead, make the patch as small as possible while still keeping it as a logical unit that contains an individual change, even if it spans multiple files. This makes reviewing your patches much easier for us and greatly increases your chances of getting your patch applied. Use the patcheck tool of FFmpeg to check your patch. The tool is located in the tools directory. Run the @ref{Regression tests} before submitting a patch in order to verify it does not cause unexpected problems. It also helps quite a bit if you tell us what the patch does (for example 'replaces lrint by lrintf'), and why (for example '*BSD isn't C99 compliant and has no lrint()') Also please if you send several patches, send each patch as a separate mail, do not attach several unrelated patches to the same mail. Patches should be posted to the @uref{https://lists.ffmpeg.org/mailman/listinfo/ffmpeg-devel, ffmpeg-devel} mailing list. Use @code{git send-email} when possible since it will properly send patches without requiring extra care. If you cannot, then send patches as base64-encoded attachments, so your patch is not trashed during transmission. Also ensure the correct mime type is used (text/x-diff or text/x-patch or at least text/plain) and that only one patch is inline or attached per mail. You can check @url{https://patchwork.ffmpeg.org}, if your patch does not show up, its mime type likely was wrong. @subheading Sending patches from email clients Using @code{git send-email} might not be desirable for everyone. The following trick allows to send patches via email clients in a safe way. It has been tested with Outlook and Thunderbird (with X-Unsent extension) and might work with other applications. Create your patch like this: @verbatim git format-patch -s -o "outputfolder" --add-header "X-Unsent: 1" --suffix .eml --to ffmpeg-devel@ffmpeg.org -1 1a2b3c4d @end verbatim Now you'll just need to open the eml file with the email application and execute 'Send'. @subheading Reviews Your patch will be reviewed on the mailing list. You will likely be asked to make some changes and are expected to send in an improved version that incorporates the requests from the review. This process may go through several iterations. Once your patch is deemed good enough, some developer will pick it up and commit it to the official FFmpeg tree. Give us a few days to react. But if some time passes without reaction, send a reminder by email. Your patch should eventually be dealt with. @chapter New codecs or formats checklist @enumerate @item Did you use av_cold for codec initialization and close functions? @item Did you add a long_name under NULL_IF_CONFIG_SMALL to the AVCodec or AVInputFormat/AVOutputFormat struct? @item Did you bump the minor version number (and reset the micro version number) in @file{libavcodec/version.h} or @file{libavformat/version.h}? @item Did you register it in @file{allcodecs.c} or @file{allformats.c}? @item Did you add the AVCodecID to @file{avcodec.h}? When adding new codec IDs, also add an entry to the codec descriptor list in @file{libavcodec/codec_desc.c}. @item If it has a FourCC, did you add it to @file{libavformat/riff.c}, even if it is only a decoder? @item Did you add a rule to compile the appropriate files in the Makefile? Remember to do this even if you're just adding a format to a file that is already being compiled by some other rule, like a raw demuxer. @item Did you add an entry to the table of supported formats or codecs in @file{doc/general.texi}? @item Did you add an entry in the Changelog? @item If it depends on a parser or a library, did you add that dependency in configure? @item Did you @code{git add} the appropriate files before committing? @item Did you make sure it compiles standalone, i.e. with @code{configure --disable-everything --enable-decoder=foo} (or @code{--enable-demuxer} or whatever your component is)? @end enumerate @chapter Patch submission checklist @enumerate @item Does @code{make fate} pass with the patch applied? @item Was the patch generated with git format-patch or send-email? @item Did you sign-off your patch? (@code{git commit -s}) See @uref{https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/plain/Documentation/process/submitting-patches.rst, Sign your work} for the meaning of @dfn{sign-off}. @item Did you provide a clear git commit log message? @item Is the patch against latest FFmpeg git master branch? @item Are you subscribed to ffmpeg-devel? (the list is subscribers only due to spam) @item Have you checked that the changes are minimal, so that the same cannot be achieved with a smaller patch and/or simpler final code? @item If the change is to speed critical code, did you benchmark it? @item If you did any benchmarks, did you provide them in the mail? @item Have you checked that the patch does not introduce buffer overflows or other security issues? @item Did you test your decoder or demuxer against damaged data? If no, see tools/trasher, the noise bitstream filter, and @uref{http://caca.zoy.org/wiki/zzuf, zzuf}. Your decoder or demuxer should not crash, end in a (near) infinite loop, or allocate ridiculous amounts of memory when fed damaged data. @item Did you test your decoder or demuxer against sample files? Samples may be obtained at @url{https://samples.ffmpeg.org}. @item Does the patch not mix functional and cosmetic changes? @item Did you add tabs or trailing whitespace to the code? Both are forbidden. @item Is the patch attached to the email you send? @item Is the mime type of the patch correct? It should be text/x-diff or text/x-patch or at least text/plain and not application/octet-stream. @item If the patch fixes a bug, did you provide a verbose analysis of the bug? @item If the patch fixes a bug, did you provide enough information, including a sample, so the bug can be reproduced and the fix can be verified? Note please do not attach samples >100k to mails but rather provide a URL, you can upload to @url{https://streams.videolan.org/upload/}. @item Did you provide a verbose summary about what the patch does change? @item Did you provide a verbose explanation why it changes things like it does? @item Did you provide a verbose summary of the user visible advantages and disadvantages if the patch is applied? @item Did you provide an example so we can verify the new feature added by the patch easily? @item If you added a new file, did you insert a license header? It should be taken from FFmpeg, not randomly copied and pasted from somewhere else. @item You should maintain alphabetical order in alphabetically ordered lists as long as doing so does not break API/ABI compatibility. @item Lines with similar content should be aligned vertically when doing so improves readability. @item Consider adding a regression test for your code. All new modules should be covered by tests. That includes demuxers, muxers, decoders, encoders filters, bitstream filters, parsers. If its not possible to do that, add an explanation why to your patchset, its ok to not test if theres a reason. @item If you added YASM code please check that things still work with --disable-yasm. @item Test your code with valgrind and or Address Sanitizer to ensure it's free of leaks, out of array accesses, etc. @end enumerate @chapter Patch review process All patches posted to ffmpeg-devel will be reviewed, unless they contain a clear note that the patch is not for the git master branch. Reviews and comments will be posted as replies to the patch on the mailing list. The patch submitter then has to take care of every comment, that can be by resubmitting a changed patch or by discussion. Resubmitted patches will themselves be reviewed like any other patch. If at some point a patch passes review with no comments then it is approved, that can for simple and small patches happen immediately while large patches will generally have to be changed and reviewed many times before they are approved. After a patch is approved it will be committed to the repository. We will review all submitted patches, but sometimes we are quite busy so especially for large patches this can take several weeks. If you feel that the review process is too slow and you are willing to try to take over maintainership of the area of code you change then just clone git master and maintain the area of code there. We will merge each area from where its best maintained. When resubmitting patches, please do not make any significant changes not related to the comments received during review. Such patches will be rejected. Instead, submit significant changes or new features as separate patches. Everyone is welcome to review patches. Also if you are waiting for your patch to be reviewed, please consider helping to review other patches, that is a great way to get everyone's patches reviewed sooner. @anchor{Regression tests} @chapter Regression tests Before submitting a patch (or committing to the repository), you should at least test that you did not break anything. Running 'make fate' accomplishes this, please see @url{fate.html} for details. [Of course, some patches may change the results of the regression tests. In this case, the reference results of the regression tests shall be modified accordingly]. @section Adding files to the fate-suite dataset If you need a sample uploaded send a mail to samples-request. When there is no muxer or encoder available to generate test media for a specific test then the media has to be included in the fate-suite. First please make sure that the sample file is as small as possible to test the respective decoder or demuxer sufficiently. Large files increase network bandwidth and disk space requirements. Once you have a working fate test and fate sample, provide in the commit message or introductory message for the patch series that you post to the ffmpeg-devel mailing list, a direct link to download the sample media. @section Visualizing Test Coverage The FFmpeg build system allows visualizing the test coverage in an easy manner with the coverage tools @code{gcov}/@code{lcov}. This involves the following steps: @enumerate @item Configure to compile with instrumentation enabled: @code{configure --toolchain=gcov}. @item Run your test case, either manually or via FATE. This can be either the full FATE regression suite, or any arbitrary invocation of any front-end tool provided by FFmpeg, in any combination. @item Run @code{make lcov} to generate coverage data in HTML format. @item View @code{lcov/index.html} in your preferred HTML viewer. @end enumerate You can use the command @code{make lcov-reset} to reset the coverage measurements. You will need to rerun @code{make lcov} after running a new test. @section Using Valgrind The configure script provides a shortcut for using valgrind to spot bugs related to memory handling. Just add the option @code{--toolchain=valgrind-memcheck} or @code{--toolchain=valgrind-massif} to your configure line, and reasonable defaults will be set for running FATE under the supervision of either the @strong{memcheck} or the @strong{massif} tool of the valgrind suite. In case you need finer control over how valgrind is invoked, use the @code{--target-exec='valgrind } option in your configure line instead. @anchor{Release process} @chapter Release process FFmpeg maintains a set of @strong{release branches}, which are the recommended deliverable for system integrators and distributors (such as Linux distributions, etc.). At regular times, a @strong{release manager} prepares, tests and publishes tarballs on the @url{https://ffmpeg.org} website. There are two kinds of releases: @enumerate @item @strong{Major releases} always include the latest and greatest features and functionality. @item @strong{Point releases} are cut from @strong{release} branches, which are named @code{release/X}, with @code{X} being the release version number. @end enumerate Note that we promise to our users that shared libraries from any FFmpeg release never break programs that have been @strong{compiled} against previous versions of @strong{the same release series} in any case! However, from time to time, we do make API changes that require adaptations in applications. Such changes are only allowed in (new) major releases and require further steps such as bumping library version numbers and/or adjustments to the symbol versioning file. Please discuss such changes on the @strong{ffmpeg-devel} mailing list in time to allow forward planning. @anchor{Criteria for Point Releases} @section Criteria for Point Releases Changes that match the following criteria are valid candidates for inclusion into a point release: @enumerate @item Fixes a security issue, preferably identified by a @strong{CVE number} issued by @url{http://cve.mitre.org/}. @item Fixes a documented bug in @url{https://trac.ffmpeg.org}. @item Improves the included documentation. @item Retains both source code and binary compatibility with previous point releases of the same release branch. @end enumerate The order for checking the rules is (1 OR 2 OR 3) AND 4. @section Release Checklist The release process involves the following steps: @enumerate @item Ensure that the @file{RELEASE} file contains the version number for the upcoming release. @item Add the release at @url{https://trac.ffmpeg.org/admin/ticket/versions}. @item Announce the intent to do a release to the mailing list. @item Make sure all relevant security fixes have been backported. See @url{https://ffmpeg.org/security.html}. @item Ensure that the FATE regression suite still passes in the release branch on at least @strong{i386} and @strong{amd64} (cf. @ref{Regression tests}). @item Prepare the release tarballs in @code{bz2} and @code{gz} formats, and supplementing files that contain @code{gpg} signatures @item Publish the tarballs at @url{https://ffmpeg.org/releases}. Create and push an annotated tag in the form @code{nX}, with @code{X} containing the version number. @item Propose and send a patch to the @strong{ffmpeg-devel} mailing list with a news entry for the website. @item Publish the news entry. @item Send an announcement to the mailing list. @end enumerate @bye