RepoMirrors/ffmpeg
1
0
Fork 0
mirror of https://git.ffmpeg.org/ffmpeg.git synced 2025-01-05 23:00:02 +00:00
Code Issues Projects Releases Wiki Activity
FFmpeg git repo
95,662 Commits 37 Branches 400 Tags 252 MiB
C 90.3%
Assembly 7.8%
Makefile 1.3%
C++ 0.2%
Objective-C 0.2%
Other 0.1%
4d980a8ceb
Go to file
Guo, Yejun 4d980a8ceb avfilter/vf_dnn_processing: add a generic filter for image proccessing with dnn networks 
This filter accepts all the dnn networks which do image processing.
Currently, frame with formats rgb24 and bgr24 are supported. Other
formats such as gray and YUV will be supported next. The dnn network
can accept data in float32 or uint8 format. And the dnn network can
change frame size.

The following is a python script to halve the value of the first
channel of the pixel. It demos how to setup and execute dnn model
with python+tensorflow. It also generates .pb file which will be
used by ffmpeg.

import tensorflow as tf
import numpy as np
import imageio
in_img = imageio.imread('in.bmp')
in_img = in_img.astype(np.float32)/255.0
in_data = in_img[np.newaxis, :]
filter_data = np.array([0.5, 0, 0, 0, 1., 0, 0, 0, 1.]).reshape(1,1,3,3).astype(np.float32)
filter = tf.Variable(filter_data)
x = tf.placeholder(tf.float32, shape=[1, None, None, 3], name='dnn_in')
y = tf.nn.conv2d(x, filter, strides=[1, 1, 1, 1], padding='VALID', name='dnn_out')
sess=tf.Session()
sess.run(tf.global_variables_initializer())
output = sess.run(y, feed_dict={x: in_data})
graph_def = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ['dnn_out'])
tf.train.write_graph(graph_def, '.', 'halve_first_channel.pb', as_text=False)
output = output * 255.0
output = output.astype(np.uint8)
imageio.imsave("out.bmp", np.squeeze(output))

To do the same thing with ffmpeg:
- generate halve_first_channel.pb with the above script
- generate halve_first_channel.model with tools/python/convert.py
- try with following commands
  ./ffmpeg -i input.jpg -vf dnn_processing=model=halve_first_channel.model:input=dnn_in:output=dnn_out:fmt=rgb24:dnn_backend=native -y out.native.png
  ./ffmpeg -i input.jpg -vf dnn_processing=model=halve_first_channel.pb:input=dnn_in:output=dnn_out:fmt=rgb24:dnn_backend=tensorflow -y out.tf.png

Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
2019-11-07 15:46:00 -03:00
compat compat/cuda: Change inclusion guards 2019-08-05 12:07:09 +02:00
doc avfilter/vf_dnn_processing: add a generic filter for image proccessing with dnn networks 2019-11-07 15:46:00 -03:00
ffbuild build: add support for building CUDA files with clang 2019-08-04 19:08:08 +02:00
fftools fftools/ffmpeg_opt: Fix mixed declarations and code 2019-11-06 20:38:03 +01:00
libavcodec avcodec/extract_extradata_bsf: fix typo in comments 2019-11-06 20:38:03 +01:00
libavdevice avdevice/v4l2: Remove av_assert0 when format not supported 2019-11-06 20:38:03 +01:00
libavfilter avfilter/vf_dnn_processing: add a generic filter for image proccessing with dnn networks 2019-11-07 15:46:00 -03:00
libavformat avformat/nutenc: Do not pass NULL to memcmp() in get_needed_flags() 2019-11-05 21:21:59 +01:00
libavresample
libavutil avcodec/mips: msa optimizations for vc1dsp 2019-10-30 18:09:00 +01:00
libpostproc Bump minor versions again on master to keep 4.2 versions separate from master 2019-07-21 18:36:31 +02:00
libswresample swresample/audioconvert: fix invalid left shift for 64bit sample format 2019-09-26 16:22:47 +02:00
libswscale swscale/swscale_unscaled: fix gbrap10be md5 different on big endian system 2019-11-01 14:43:16 +01:00
presets
tests FATE: add a test for freeezedetect 2019-10-30 18:09:00 +01:00
tools tools/enum_options: replace the deprecated API 2019-11-04 23:27:50 +08:00
.gitattributes
.gitignore tools/python: add script to convert TensorFlow model (.pb) to native model (.model) 2019-07-01 10:23:47 -03:00
.travis.yml Merge commit '899ee03088d55152a48830df0899887f055da1de' 2019-03-14 15:53:16 -03:00
Changelog lavc/qsvenc: enable vp9 encoder 2019-11-03 16:45:35 +08:00
configure avfilter/vf_dnn_processing: add a generic filter for image proccessing with dnn networks 2019-11-07 15:46:00 -03:00
CONTRIBUTING.md
COPYING.GPLv2
COPYING.GPLv3
COPYING.LGPLv2.1
COPYING.LGPLv3
CREDITS
INSTALL.md INSTALL.md: Fix Markdown formatting 2019-01-31 10:29:16 -09:00
LICENSE.md LICENSE: Add missing libraries that need --enable-version3. 2019-08-12 02:25:39 +02:00
MAINTAINERS MAINTAINERS: add myself to OMX 2019-08-23 17:07:05 -07:00
Makefile tools: Add fuzzer for demuxers 2019-05-31 22:44:30 +02:00
README.md
RELEASE RELEASE: Update value for post 4.2 branch 2019-07-21 18:37:32 +02:00

README.md

FFmpeg README

FFmpeg is a collection of libraries and tools to process multimedia content such as audio, video, subtitles and related metadata.

Libraries

  • libavcodec provides implementation of a wider range of codecs.
  • libavformat implements streaming protocols, container formats and basic I/O access.
  • libavutil includes hashers, decompressors and miscellaneous utility functions.
  • libavfilter provides a mean to alter decoded Audio and Video through chain of filters.
  • libavdevice provides an abstraction to access capture and playback devices.
  • libswresample implements audio mixing and resampling routines.
  • libswscale implements color conversion and scaling routines.

Tools

  • ffmpeg is a command line toolbox to manipulate, convert and stream multimedia content.
  • ffplay is a minimalistic multimedia player.
  • ffprobe is a simple analysis tool to inspect multimedia content.
  • Additional small tools such as aviocat, ismindex and qt-faststart.

Documentation

The offline documentation is available in the doc/ directory.

The online documentation is available in the main website and in the wiki.

Examples

Coding examples are available in the doc/examples directory.

License

FFmpeg codebase is mainly LGPL-licensed with optional components licensed under GPL. Please refer to the LICENSE file for detailed information.

Contributing

Patches should be submitted to the ffmpeg-devel mailing list using git format-patch or git send-email. Github pull requests should be avoided because they are not part of our review process and will be ignored.