This filter accepts all the dnn networks which do image processing.
Currently, frame with formats rgb24 and bgr24 are supported. Other
formats such as gray and YUV will be supported next. The dnn network
can accept data in float32 or uint8 format. And the dnn network can
change frame size.
The following is a python script to halve the value of the first
channel of the pixel. It demos how to setup and execute dnn model
with python+tensorflow. It also generates .pb file which will be
used by ffmpeg.
import tensorflow as tf
import numpy as np
import imageio
in_img = imageio.imread('in.bmp')
in_img = in_img.astype(np.float32)/255.0
in_data = in_img[np.newaxis, :]
filter_data = np.array([0.5, 0, 0, 0, 1., 0, 0, 0, 1.]).reshape(1,1,3,3).astype(np.float32)
filter = tf.Variable(filter_data)
x = tf.placeholder(tf.float32, shape=[1, None, None, 3], name='dnn_in')
y = tf.nn.conv2d(x, filter, strides=[1, 1, 1, 1], padding='VALID', name='dnn_out')
sess=tf.Session()
sess.run(tf.global_variables_initializer())
output = sess.run(y, feed_dict={x: in_data})
graph_def = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ['dnn_out'])
tf.train.write_graph(graph_def, '.', 'halve_first_channel.pb', as_text=False)
output = output * 255.0
output = output.astype(np.uint8)
imageio.imsave("out.bmp", np.squeeze(output))
To do the same thing with ffmpeg:
- generate halve_first_channel.pb with the above script
- generate halve_first_channel.model with tools/python/convert.py
- try with following commands
./ffmpeg -i input.jpg -vf dnn_processing=model=halve_first_channel.model:input=dnn_in:output=dnn_out:fmt=rgb24:dnn_backend=native -y out.native.png
./ffmpeg -i input.jpg -vf dnn_processing=model=halve_first_channel.pb:input=dnn_in:output=dnn_out:fmt=rgb24:dnn_backend=tensorflow -y out.tf.png
Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
1. must enable low_power mode since just VDENC can be supported by iHD
driver right now
2. Coding option1 and extra_data are not supported by MSDK
3. IVF header will be inserted in MSDK by default, but it is not needed
for FFmpeg, so disable it.
Signed-off-by: Zhong Li <zhongli_dev@126.com>
Support for VDPAU accelerated VP9 decoding was added with libvdpau-1.3.
Support for the same in ffmpeg is added with this patch. Profiles
related to VDPAU VP9 can be found in latest vdpau.h present in
libvdpau-1.3. DRC clips are not supported yet due to
http://trac.ffmpeg.org/ticket/8068
Add VP9 VDPAU to list of hwaccels and supported formats
Added file vdpau_vp9.c and Modified configure to add VDPAU VP9 support.
Mapped VP9 profiles to VDPAU VP9 profiles. Populated the codec specific
params that need to be passed to VDPAU.
Signed-off-by: Philip Langdale <philipl@overt.org>
Due to the recent addition of Vulkan support to AMF, we require more
recent headers that include the new structures, which have been
available since AMF 1.4.9 released in September 2018.
Fixes Ticket #8125
Use the command ./configure with/without --disable-v4l2-m2m test.
Reviewed-by: Aman Gupta <aman@tmm1.net>
Signed-off-by: Jun Zhao <barryjzhao@tencent.com>
When compiling natively on an RPI where libomxil-bellagio-dev
was also installed, `check_headers OMX_Core.h` succeeded and
the -isystem compiler flag was never added to the build.
For non-native builds, the error message now mentions the
raspberrypi/firmware repository where the RPI specific
headers are available.
Signed-off-by: Aman Gupta <aman@tmm1.net>
When ffmpeg was streaming, multiple clients were only supported by using a
multicast destination address. An alternative was to stream to a server which
re-distributes the content. This commit adds ZeroMQ as a protocol, which allows
multiple clients to connect to a single ffmpeg instance.
Signed-off-by: Marton Balint <cus@passwd.hu>
The current code in libavfilter/af_sofalizer.c requires
mysofa_neighborhood_init_withstepdefine function, which only appeared
in libmysofa 0.7. Use this function in configure script to bail out
early if a too old libmysofa is found in the system instead of failing
at compile time.
Used a technique similar to lavc/tdsc.c for invoking the MJPEG decoder.
This commit adds support for:
- DNG tiles
- DNG tile huffman lossless JPEG decoding
- DNG 8-bpp ("packed" as dcraw calls it) decoding
- DNG color scaling [1]
- LinearizationTable tag
- BlackLevel tag
[1]: As specified in the DNG Specification - Chapter 5
Signed-off-by: Nick Renieris <velocityra@gmail.com>
Many ffmpeg + rpi compilation guides on the internet recommend
using `./configure --enable-omx --enable-omx-rpi`. This fails
to find the IL OMX headers on device because the omx require_headers
check happens first before the add_cflags in omx_rpi.
A workaround is to use `./configure --enable-omx-rpi` only, since
omx_rpi already implies omx. But because many users expect to use
existing scripts and commands, we swap the order here so omx_rpi
special cases are applied first.
In the past this wasn't an issue because users noticed the OMX_Core.h
missing error and installed libomxil-bellagio-dev. But since
76c82843cc, the rpi specific headers from /opt/vc/include/IL
are required.
Signed-off-by: Aman Gupta <aman@tmm1.net>
MSYS2 converts paths to MinGW-based applications from unix to
pseudo-windows paths on execution time.
Since there was no space between '-include' and the path, MSYS2 doesn't
detect the path properly.
Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
This avoids using the CUDA SDK at all; instead, we provide a minimal
reimplementation of the basic functionality that lavfi actually uses.
It generates very similar code to what NVCC produces.
The header contains no implementation code derived from the SDK.
The function and type declarations are derived from the SDK only to the
extent required to build a compatible implementation. This is generally
accepted to qualify as fair use.
Because this option does not require the proprietary SDK, it does not require
the "--enable-nonfree" flag in configure.
Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
Loongson 3A4000 and 2k1000 has supported MSA2.0.
This patch optimized SAD_UB2_UH,UNPCK_R_SH_SW,UNPCK_SB_SH and UNPCK_SH_SW with MSA2.0 instruction.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Get rid of pr dependency and write the columns strictly
alphabetical without page size considerations (POSIX
specifies 66 lines as default).
Setting the page size via pr's -l option was considered,
but as there is issue #5680 which wants to avoid pr
mainly because it's not in busybox, we chose to replace
pr instead.
Before pr would attempt to write pages, thus if a page
boundary was reached, the output looked confusing as one
couldn't see there was a new page and the alphabetical
order was disrupted when scanning down one of the columns.
This change is based on a shell implementation submitted
before by Yejun.
Possible differences to the current version using pr:
1. pr implementations should truncate items to not overflow columns;
depending on how it's done not truncating shall be better IMHO.
2. pr implementations might balance columns differently;
we use minimum number of lines and might end up not
using all columns or might have lesser entries in the
last column(s)
3. we use spaces only for padding the columns; at least the GNU pr
version on my system also by default stuffs in tabs in addition
to a single space in between columns. I don't see that this
behaviour is demanded by POSIX, though I might be very well
overlooking things. Anyway for our use case I can't see a need
for having the additional tabs, or why it would be better compared
to padding with spaces only.
Fixes output for sizes with width < column width, too.
Fixes remaining part of ticket #5680
Contributor: Guo, Yejun <yejun.guo@intel.com>
This patch is based on https://trac.ffmpeg.org/ticket/5680 provided by
Kylie McClain <somasis@exherbo.org> at Wed, 29 Jun 2016 16:37:20 -0400,
and have some changes.
contributor: Kylie McClain <somasis@exherbo.org>
contributor: avih <avihpit@yahoo.com>
Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Adding the support to build FFMPEG with HW accelerated decode and encode on PPC64
little endian architecture.
Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
* commit 'c0bd865ad60da31282c5d8e1000c98366249c31e':
configure: Add -D_POSIX_C_SOURCE=200112 -D_XOPEN_SOURCE=600 for mingw as well
Merged-by: James Almer <jamrial@gmail.com>
Mingw headers have got header inline implementations of localtime_r
and gmtime_r, but only visible if certain posix thread safe functions
have been requested.
This is a preparatory step for improving the detection of those
functions.
Signed-off-by: Martin Storsjö <martin@martin.st>
Currectly just standard header path can be found,
check_type/struct will fail if vaapi is installed somewhere else.
Move them followed "check_pkg_config"
Reviewed-by: Mark Thompson <sw@jkqxz.net>
Reviewed-by: Timo Rothenpieler <timo@rothenpieler.org>
Signed-off-by: Zhong Li <zhong.li@intel.com>
Bump the minimum required version to the first one with the logger API callback.
Reviewed-by: Vittorio Giovara <vittorio.giovara@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>
libvpx can be compiled with the VP8 decoder and encoder disabled, and
there's no reason to force their presence if the user only wants VP9.
Signed-off-by: James Almer <jamrial@gmail.com>
autorotate is enabled by default in ffmpeg so the rotation filters
are required and will be attempted for insertion without the user's
knowledge if an input stream has rotation side-data.
With all of our existing users of cuda_sdk switched over to ffnvcodec,
we could remove cuda_sdk completely and say that we should no longer
add code that requires the full sdk, and rather insist that such code
only use ffnvcodec.
As discussed previously, the use of nvcc from the sdk is still
supported with a distinct option.
Signed-off-by: Philip Langdale <philipl@overt.org>
Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
This change switches the vf_thumbnail_cuda filter from using the
full cuda sdk to using the ffnvcodec headers and loader.
Most of the change is a direct mapping, but I also switched from
using texture references to using texture objects. This is supposed
to be the preferred way of using textures, and the texture object API
is the one I added to ffnvcodec.
Signed-off-by: Philip Langdale <philipl@overt.org>
Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
This change switches the vf_scale_cuda filter from using the
full cuda sdk to using the ffnvcodec headers and loader.
Most of the change is a direct mapping, but I also switched from
using texture references to using texture objects. This is supposed
to be the preferred way of using textures, and the texture object API
is the one I added to ffnvcodec.
Signed-off-by: Philip Langdale <philipl@overt.org>
Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
This change switches the vf_thumbnail_cuda filter from using the
full cuda sdk to using the ffnvcodec headers and loader.
Signed-off-by: Philip Langdale <philipl@overt.org>
Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
The use of nvcc to compile cuda kernels is distinct from the use of
cuda sdk libraries and linking against those libraries. We have
previously not bothered to distinguish these two cases because all
the filters that used cuda kernels also used the sdk. In the following
changes, I'm going to remove the sdk dependency from those filters,
but we need a way to ensure that nvcc is present and functioning, and
also a way to explicitly disable its use so that the filters are not
built.
Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
If we enable a component but a dependant library is disabled, then the enabled
component gets silently disabled. Warning about disabled explicitly enabled components
allows configure to show the missing dependencies and if --fatal-warnings is
used it can also fail if the user wants it so.
For example if libdav1d is not availble ./configure --enable-decoder=libdav1d
succeeds but the libdav1d decoder is not be enabled. After the patch configure
will warn about this:
WARNING: Disabled libdav1d_decoder because not all dependencies are satisfied: libdav1d
Signed-off-by: Marton Balint <cus@passwd.hu>
* Outputs ASS lines with basic coloring and font scaling for each
given region.
* Sets the default style to the resolution of the subtitle plane
(for example, 960x540 / 36pt font for profile A).
* Has options to:
* Disable ruby text (which is coded as regions which have
half-height text in libaribb24).
Enabled by default as without positioning ruby text only
confuses as it is usually coded in the beginning of the decoded
subtitle line.
* Set the working directory, in which libaribb24 will read
configuration as well as into which it may save broadcast extra
symbols as PNG.
Unset by default.
The unconventional library check can be explained by the library's
current master branch being licensed as LGPLv3, but at the time of
writing the latest official release is still licensed under GPLv3.
Thus, one either has to wait for the following release, or enable
GPLv3.
DXVA2 may be enabled even when every relevant module is disabled,
which would result in the dependency generator not including its
extralibs to avcodec.
Fixes ticket #7642.
Signed-off-by: James Almer <jamrial@gmail.com>
The color fields were moved to another struct, and a way to propagate
timestamps and other input metadata was introduced, so the packet
fifo can be removed.
Add support for 12bit streams, an option to disable film grain, and
read the profile from the sequence header referenced by the ouput
picture instead of guessing based on output pix_fmt.
Signed-off-by: James Almer <jamrial@gmail.com>
Also add SIMD which works on lines because it is faster then calculating it on
8x8 blocks using pixelutils.
Signed-off-by: Marton Balint <cus@passwd.hu>
This is a cuda implementation of yadif, which gives us a way to
do deinterlacing when using the nvdec hwaccel. In that scenario
we don't have access to the nvidia deinterlacer.
Simple parser to set keyframes, frame type, structure, width, height, and pixel
format, plus stream profile and level.
Reviewed-by: Mark Thompson <sw@jkqxz.net>
Signed-off-by: James Almer <jamrial@gmail.com>
Also bump the API version requirement to 10.9.5, because on olders versions
there were some reports of crashes using the undocumented, yet available
BMDDeckLinkDeviceHandle.
Signed-off-by: Marton Balint <cus@passwd.hu>
Set the minimum version to 0.35.0 (libva 1.3.0) and remove redundant
configure tests. This also allows the proprietary libmfx fork of libva,
which always shows the version number 0.99.0 (independent of the actual
version).
Hook in libklvanc and use it for output of EIA-708 captions over
SDI. The bulk of this patch is just general support for ancillary
data for the Decklink SDI module - the real work for construction
of the EIA-708 CDP and VANC line construction is done by libklvanc.
Libklvanc can be found at: https://github.com/stoth68000/libklvanc
Updated to reflect feedback from Marton Balint <cus@passwd.hu>,
Carl Eugen Hoyos <ceffmpeg@gmail.com>, Aaron Levinson
<alevinsn_dev@levland.net>, and Moritz Barsnick <barsnick@gmx.net>
Signed-off-by: Devin Heitmueller <dheitmueller@ltnglobal.com>
Signed-off-by: Marton Balint <cus@passwd.hu>
* commit '39f3b6f3fc2b46b405b680cce3599f1b370e342d':
configure: Move add_fooflags() helper functions into canonical order
Merged-by: James Almer <jamrial@gmail.com>
* commit '5691c746cf62e69806aae1baf0a6e8252d519444':
configure: Group toolchain parameter mangling functions together
Merged-by: James Almer <jamrial@gmail.com>
* commit '25c2a27c9ec0150210d75ee5ac8ed1bfa14c1a56':
configure: Make require_cc() and require_cpp_condition() functions consistent
Merged-by: James Almer <jamrial@gmail.com>
Also make sure we set the URL context max packet size accordingly.
Based on a patch by Tudor Suciu <tudor.suciu@gmail.com>
Signed-off-by: Marton Balint <cus@passwd.hu>
- Allow to add deps in any order rather than "in linking order".
- Expand deps chains as required rather than just once.
- Validate that there are no cycles.
- Validate that [after expansion] deps are limited to other fflibs.
- Remove expectation for a specific output order of unique().
Previously when adding items to <fflib>_deps, developers were
required to add them in linking order. This can be awkward and
bug-prone, especially when a list is not empty, e.g. when adding
conditional deps.
It also implicitly expected unique() to keep the last instance of
recurring items such that these lists maintain their linking order
after removing duplicate items.
This patch mainly allows to add deps in any order by keeping just
one master list in linking order, and then reordering all the
<fflib>_deps lists to align with the master list order.
This master list is LIBRARY_LIST itself, where otherwise its order
doesn't matter.
The patch also removes a limit where these deps lists were expanded
only once. This could have resulted in incomplete expanded lists,
or forcing devs to add already-deducable deps to avoid this issue.
Note: it is possible to deduce the master list order automatically
from the deps lists, but in this case it's probably not worth the
added complexity, even if minor. Maintaining one list should be OK.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
x4 - x25 faster.
check_deps() recursively enables/disables components, and its loop is
iterated nearly 6000 times. It's particularly slow in bash - currently
consuming more than 50% of configure runtime, and about 20% with other
shells.
This commit applies few local optimizations, most effective first:
- Use $1 $2 ... instead of pushvar/popvar, and same at enable_deep*
- Abort early in one notable case - empty deps, to avoid costly no-op.
- Smaller changes which do add up:
- Handle ${cfg}_checking locally instead of via enable[d]/disable
- ${cfg}_checking: test done before inprogress - x2 faster in 50%+
- one eval instead of several at the empty-deps early abort path.
- The "actual work" part is unmodified - just its surroundings.
Biggest speedups (relative and absolute) are observed with bash.
Tested-by: Michael Niedermayer <michael@niedermayer.cc>
Tested-by: Helmut K. C. Tessarek <tessarek@evermeet.cx>
Tested-by: Dave Yeo <daveryeo@telus.net>
Tested-by: Reino Wijnsma <rwijnsma@xs4all.nl>
Signed-off-by: James Almer <jamrial@gmail.com>
x4 - x10 faster.
Inside print_enabled components, the filter_list case invokes sed
about 350 times to parse the same source file and extract different
info for each arg. This is never instant, and on systems where fork is
slow (notably MSYS2/Cygwin on windows) it takes many seconds.
Change it to use sed once on the source file and set env vars with the
parse results, then use these results inside the loop.
Additionally, the cases of indev_list and outdev_list are very
infrequent, but nevertheless they're faster, and arguably cleaner, with
shell parameter substitutions than with command substitutions.
Tested-by: Michael Niedermayer <michael@niedermayer.cc>
Tested-by: Helmut K. C. Tessarek <tessarek@evermeet.cx>
Tested-by: Dave Yeo <daveryeo@telus.net>
Tested-by: Reino Wijnsma <rwijnsma@xs4all.nl>
Signed-off-by: James Almer <jamrial@gmail.com>
x50 - x200 faster.
Currently configure spends 50-70% of its runtime inside a single
function: flatten_extralibs[_wrapper] - which does string processing.
During its run, nearly 20K command substitutions (subshells) are used,
including its callees unique() and resolve(), which is the reason
for its lengthy run.
This commit avoids all subshells during its execution, speeding it up
by about two orders of magnitude, and reducing the overall configure
runtime by 50-70% .
resolve() is rewritten to avoid subshells, and in unique() and
flatten_extralibs() we "inline" the filter[_out] functionality.
Note that logically, "unique" functionality has more than one possible
output (depending on which of the recurring items is kept). As it
turns out, other parts expect the last recurring item to be kept
(which was the original behavior of uniqie()). This patch preservs
its output order.
Tested-by: Michael Niedermayer <michael@niedermayer.cc>
Tested-by: Helmut K. C. Tessarek <tessarek@evermeet.cx>
Tested-by: Dave Yeo <daveryeo@telus.net>
Tested-by: Reino Wijnsma <rwijnsma@xs4all.nl>
Signed-off-by: James Almer <jamrial@gmail.com>
Some containers, like Matroska, may propagate key frames with no Sequence
Header OBU since it's provided in extradata instead.
With this change, the Sequence Header will be appended to the packet data
before calling aom_codec_decode().
Signed-off-by: James Almer <jamrial@gmail.com>
Also remove the superfluous aandcttables dependency from all the modules
that only need it because of mpegvideoenc
Fixes ticket #7333
Signed-off-by: James Almer <jamrial@gmail.com>
And add it to the CONFIGURABLE_COMPONENTS list in Makefile. This way, changes
to the new file will be tracked and the usual warning to suggest re-running
configure will be shown.
Reviewed-by: Rostislav Pehlivanov <atomnuker@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>
aom_codec_get_global_headers() is not implemented as of libaom 1.0.0 for AV1, so
we're forced to extract the relevant header OBUs from the first packet and propagate
them as packet side data.
Signed-off-by: James Almer <jamrial@gmail.com>
Lensfun is a library that applies lens correction to an image using a
database of cameras/lenses (you provide the camera and lens models, and
it uses the corresponding database entry's parameters to apply lens
correction). It is licensed under LGPL3.
The lensfun filter utilizes the lensfun library to apply lens
correction to videos as well as images.
This filter was created out of necessity since I wanted to apply lens
correction to a video and the lenscorrection filter did not work for me.
While this filter requires little info from the user to apply lens
correction, the flaw is that lensfun is intended to be used on indvidual
images. When used on a video, the parameters such as focal length is
constant, so lens correction may fail on videos where the camera's focal
length changes (zooming in or out via zoom lens). To use this filter
correctly on videos where such parameters change, timeline editing may
be used since this filter supports it.
Note that valgrind shows a small memory leak which is not from this
filter but from the lensfun library (memory is allocated when loading
the lensfun database but it somehow isn't deallocated even during
cleanup; it is briefly created in the init function of the filter, and
destroyed before the init function returns). This may have been fixed by
the latest commit in the lensfun repository; the current latest release
of lensfun is almost 3 years ago.
Bi-Linear interpolation is used by default as lanczos interpolation
shows more artifacts in the corrected image in my tests.
The lanczos interpolation is derived from lenstool's implementation of
lanczos interpolation. Lenstool is an app within the lensfun repository
which is licensed under GPL3.
v2 of this patch fixes license notice in libavfilter/vf_lensfun.c
v3 of this patch fixes code style and dependency to gplv3 (thanks to
Paul B Mahol for pointing out the mentioned issues).
v4 of this patch fixes more code style issues that were missed in
v3.
v5 of this patch adds line breaks to some of the documentation in
doc/filters.texi (thanks to Gyan Doshi for pointing out the issue).
v6 of this patch fixes more problems (thanks to Moritz Barsnick for
pointing them out).
v7 of this patch fixes use of sqrt() (changed to sqrtf(); thanks to
Moritz Barsnick for pointing this out). Also should be rebased off of
latest master branch commits at this point.
Signed-off-by: Stephen Seo <seo.disparate@gmail.com>