RepoMirrors
/
ffmpeg
mirror of https://git.ffmpeg.org/ffmpeg.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
99,111 Commits 36 Branches 392 Tags 311 MiB
Commit Graph

64 Commits

Author SHA1 Message Date
Andreas Rheinhardt 9beaf536fe dnn/dnn_backend_native_layer_conv2d: Fix allocation size 
Found via ASAN with the dnn-layer-conv2d FATE-test.

Reviewed-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
 2020-09-09 14:58:26 +02:00
Xu Jun 3c7cad69f2 dnn_backend_native_layer_conv2d.c:Add mutithread function 
Use pthread to multithread dnn_execute_layer_conv2d.
Can be tested with command "./ffmpeg_g -i input.png -vf \
format=yuvj420p,dnn_processing=dnn_backend=native:model= \
espcn.model:input=x:output=y:options=conv2d_threads=23 \
 -y sr_native.jpg -benchmark"

before patch: utime=11.238s stime=0.005s rtime=11.248s
after patch:  utime=20.817s stime=0.047s rtime=1.051s
on my 3900X 12c24t @4.2GHz

About the increase of utime, it's because that CPU HyperThreading
technology makes logical cores twice of physical cores while cpu's
counting performance improves less than double. And utime sums
all cpu's logical cores' runtime. As a result, using threads num
near cpu's logical core's number will double utime, while reduce
rtime less than half for HyperThreading CPUs.

Signed-off-by: Xu Jun <xujunzz@sjtu.edu.cn>
Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
 2020-09-09 14:24:36 +08:00
Xu Jun 235e01f5a0 dnn_backend_native.c: parse options in native backend 
Signed-off-by: Xu Jun <xujunzz@sjtu.edu.cn>
Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
 2020-09-09 14:24:36 +08:00
Ting Fu 4a11a6f4cc dnn/tensorflow: add log error message 
Signed-off-by: Ting Fu <ting.fu@intel.com>
 2020-08-31 13:12:10 +08:00
Ting Fu 74358ff4a4 dnn/openvino: add log error message 
Signed-off-by: Ting Fu <ting.fu@intel.com>
 2020-08-31 13:12:10 +08:00
Ting Fu c8ba0daf8d dnn/native: add log error message 
Signed-off-by: Ting Fu <ting.fu@intel.com>
 2020-08-25 13:03:46 +08:00
Ting Fu 230cf9d185 dnn/native: unify error return to DNN_ERROR 
Unify all error return as DNN_ERROR, in order to cease model executing
when return error in ff_dnn_execute_model_native layer_func.pf_exec

Signed-off-by: Ting Fu <ting.fu@intel.com>
 2020-08-25 13:03:46 +08:00
Guo, Yejun 0f7a99e37a dnn: move output name from DNNModel.set_input_output to DNNModule.execute_model 
currently, output is set both at DNNModel.set_input_output and
DNNModule.execute_model, it makes sense that the output name is
provided at model inference time so all the output info is set
at a single place.

and so DNNModel.set_input_output is renamed to DNNModel.set_input

Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
 2020-08-25 09:02:59 +08:00
Mingyu Yin 3477feb643 dnn_backend_native_layer_mathbinary: add floormod support 
Signed-off-by: Mingyu Yin <mingyu.yin@intel.com>
 2020-08-24 09:09:11 +08:00
Mingyu Yin 37ef1acedb dnn_backend_native_layer_mathbinary: change to function pointer 
Signed-off-by: Mingyu Yin <mingyu.yin@intel.com>
 2020-08-24 09:09:11 +08:00
Andreas Rheinhardt 128e6df1cd dnn_backend_native_layer_avgpool: Fix invalid assignment, use av_assert 
dnn_execute_layer_avg_pool() contains the following line:

assert(avgpool_params->padding_method = VALID);

This statement contains an assignment where obviously a comparison was
intended. Furthermore, *avgpool_params is const, so that the attempted
assignment leads to a compilation failure if asserts are enabled
(i.e. if DEBUG is defined which leads libavutil/internal.h to not define
NDEBUG). Moreover, the enumeration constant VALID actually has the value 0,
so that the assert would be triggered if a compiler compiles this with
asserts enabled. Finally, the statement uses assert() directly instead
of av_assert*().

All these errors have been fixed.

Thanks to ubitux for providing a FATE-box [1] where DEBUG is defined.

[1]: http://fate.ffmpeg.org/history.cgi?slot=x86_64-archlinux-gcc-ddebug

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Guo, Yejun <yejun.guo@intel.com>
 2020-08-21 22:12:39 +08:00
Ting Fu a6e830ae7f dnn/native: rename struct ConvolutionalNetwork to NativeModel 
Signed-off-by: Ting Fu <ting.fu@intel.com>
Reviewed-by: Guo, Yejun <yejun.guo@intel.com>
 2020-08-21 10:39:00 +08:00
Guo, Yejun 3c05c8a15f dnn_backend_tf.c: fix build issue for tensorflow backend 
Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
 2020-08-14 08:59:39 +08:00
Guo, Yejun 0a51abe8ab dnn: add backend options when load the model 
different backend might need different options for a better performance,
so, add the parameter into dnn interface, as a preparation.

Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
 2020-08-12 15:43:40 +08:00
Mingyu Yin 4ed6bca4ae dnn_backend_native_layer_mathunary: add round support 
Signed-off-by: Mingyu Yin <mingyu.yin@intel.com>
Reviewed-by: Guo, Yejun <yejun.guo@intel.com>
 2020-08-12 10:30:46 +08:00
Ting Fu 91efc41a69 dnn/native: add native support for avg_pool 
Not support pooling strides in channel dimension yet.

Signed-off-by: Ting Fu <ting.fu@intel.com>
Reviewed-by: Guo, Yejun <yejun.guo@intel.com>
 2020-08-10 16:37:39 +08:00
Mingyu Yin fab00b0ae0 dnn_backend_native_layer_mathunary: add floor support 
It can be tested with the model generated with below python script:

import tensorflow as tf
import os
import numpy as np
import imageio
from tensorflow.python.framework import graph_util
name = 'floor'

pb_file_path = os.getcwd()
if not os.path.exists(pb_file_path+'/{}_savemodel/'.format(name)):
    os.mkdir(pb_file_path+'/{}_savemodel/'.format(name))

with tf.Session(graph=tf.Graph()) as sess:
    in_img = imageio.imread('detection.jpg')
    in_img = in_img.astype(np.float32)
    in_data = in_img[np.newaxis, :]
    input_x = tf.placeholder(tf.float32, shape=[1, None, None, 3], name='dnn_in')
    y_ = tf.math.floor(input_x*255)/255
    y = tf.identity(y_, name='dnn_out')
    sess.run(tf.global_variables_initializer())
    constant_graph = graph_util.convert_variables_to_constants(sess, sess.graph_def, ['dnn_out'])

    with tf.gfile.FastGFile(pb_file_path+'/{}_savemodel/model.pb'.format(name), mode='wb') as f:
        f.write(constant_graph.SerializeToString())

    print("model.pb generated, please in ffmpeg path use\n \n \
    python tools/python/convert.py {}_savemodel/model.pb --outdir={}_savemodel/ \n \nto generate model.model\n".format(name,name))

    output = sess.run(y, feed_dict={ input_x: in_data})
    imageio.imsave("out.jpg", np.squeeze(output))

    print("To verify, please ffmpeg path use\n \n \
    ./ffmpeg -i detection.jpg -vf format=rgb24,dnn_processing=model={}_savemodel/model.pb:input=dnn_in:output=dnn_out:dnn_backend=tensorflow -f framemd5 {}_savemodel/tensorflow_out.md5\n  \
    or\n \
    ./ffmpeg -i detection.jpg -vf format=rgb24,dnn_processing=model={}_savemodel/model.pb:input=dnn_in:output=dnn_out:dnn_backend=tensorflow {}_savemodel/out_tensorflow.jpg\n \nto generate output result of tensorflow model\n".format(name, name, name, name))

    print("To verify, please ffmpeg path use\n \n \
    ./ffmpeg -i detection.jpg -vf format=rgb24,dnn_processing=model={}_savemodel/model.model:input=dnn_in:output=dnn_out:dnn_backend=native -f framemd5 {}_savemodel/native_out.md5\n  \
    or \n \
    ./ffmpeg -i detection.jpg -vf format=rgb24,dnn_processing=model={}_savemodel/model.model:input=dnn_in:output=dnn_out:dnn_backend=native {}_savemodel/out_native.jpg\n \nto generate output result of native model\n".format(name, name, name, name))

Signed-off-by: Mingyu Yin <mingyu.yin@intel.com>
 2020-08-07 10:34:22 +08:00
Mingyu Yin 9fbdd5454b dnn_backend_native_layer_mathunary: add ceil support 
It can be tested with the model generated with below python script:

import tensorflow as tf
import os
import numpy as np
import imageio
from tensorflow.python.framework import graph_util
name = 'ceil'

pb_file_path = os.getcwd()
if not os.path.exists(pb_file_path+'/{}_savemodel/'.format(name)):
    os.mkdir(pb_file_path+'/{}_savemodel/'.format(name))

with tf.Session(graph=tf.Graph()) as sess:
    in_img = imageio.imread('detection.jpg')
    in_img = in_img.astype(np.float32)
    in_data = in_img[np.newaxis, :]
    input_x = tf.placeholder(tf.float32, shape=[1, None, None, 3], name='dnn_in')
    y = tf.math.ceil( input_x, name='dnn_out')
    sess.run(tf.global_variables_initializer())
    constant_graph = graph_util.convert_variables_to_constants(sess, sess.graph_def, ['dnn_out'])

    with tf.gfile.FastGFile(pb_file_path+'/{}_savemodel/model.pb'.format(name), mode='wb') as f:
        f.write(constant_graph.SerializeToString())

    print("model.pb generated, please in ffmpeg path use\n \n \
    python tools/python/convert.py ceil_savemodel/model.pb --outdir=ceil_savemodel/ \n \n \
    to generate model.model\n")

    output = sess.run(y, feed_dict={ input_x: in_data})
    imageio.imsave("out.jpg", np.squeeze(output))

    print("To verify, please ffmpeg path use\n \n \
    ./ffmpeg -i detection.jpg -vf format=rgb24,dnn_processing=model=ceil_savemodel/model.pb:input=dnn_in:output=dnn_out:dnn_backend=tensorflow -f framemd5 ceil_savemodel/tensorflow_out.md5\n \n \
    to generate output result of tensorflow model\n")

    print("To verify, please ffmpeg path use\n \n \
    ./ffmpeg -i detection.jpg -vf format=rgb24,dnn_processing=model=ceil_savemodel/model.model:input=dnn_in:output=dnn_out:dnn_backend=native -f framemd5 ceil_savemodel/native_out.md5\n \n \
    to generate output result of native model\n")

Signed-off-by: Mingyu Yin <mingyu.yin@intel.com>
Reviewed-by: Guo, Yejun <yejun.guo@intel.com>
 2020-08-04 19:56:54 +08:00
Reimar Döffinger 584f396132 dnn_backend_native: Add overflow check for length calculation. 
We should not silently allocate an incorrect sized buffer.
Fixes trac issue #8718.

Signed-off-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Reviewed-by: Guo, Yejun <yejun.guo@intel.com>
 2020-07-06 20:22:30 +08:00
Ting Fu c0cdeea0ee dnn_backend_native_layer_mathunary: add atanh support 
It can be tested with the model generated with below python script:

import tensorflow as tf
import numpy as np
import imageio

in_img = imageio.imread('input.jpeg')
in_img = in_img.astype(np.float32)/255.0
in_data = in_img[np.newaxis, :]

x = tf.placeholder(tf.float32, shape=[1, None, None, 3], name='dnn_in')

please uncomment the part you want to test

x_sinh_1 = tf.sinh(x)
x_out = tf.divide(x_sinh_1, 1.176) # sinh(1.0)

x_cosh_1 = tf.cosh(x)
x_out = tf.divide(x_cosh_1, 1.55) # cosh(1.0)

x_tanh_1 = tf.tanh(x)
x__out = tf.divide(x_tanh_1, 0.77) # tanh(1.0)

x_asinh_1 = tf.asinh(x)
x_out = tf.divide(x_asinh_1, 0.89) # asinh(1.0/1.1)

x_acosh_1 = tf.add(x, 1.1)
x_acosh_2 = tf.acosh(x_acosh_1) # accept (1, inf)
x_out = tf.divide(x_acosh_2, 1.4) # acosh(2.1)

x_atanh_1 = tf.divide(x, 1.1)
x_atanh_2 = tf.atanh(x_atanh_1) # accept (-1, 1)
x_out = tf.divide(x_atanh_2, 1.55) # atanhh(1.0/1.1)

y = tf.identity(x_out, name='dnn_out') #please only preserve the x_out you want to test

sess=tf.Session()
sess.run(tf.global_variables_initializer())

graph_def = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ['dnn_out'])
tf.train.write_graph(graph_def, '.', 'image_process.pb', as_text=False)

print("image_process.pb generated, please use \
path_to_ffmpeg/tools/python/convert.py to generate image_process.model\n")

output = sess.run(y, feed_dict={x: in_data})
imageio.imsave("out.jpg", np.squeeze(output))

Signed-off-by: Ting Fu <ting.fu@intel.com>
 2020-07-06 12:45:14 +08:00
Ting Fu cd2e3a864d dnn_backend_native_layer_mathunary: add acosh support 
Signed-off-by: Ting Fu <ting.fu@intel.com>
 2020-07-06 12:45:14 +08:00
Ting Fu 9d14b38d9d dnn_backend_native_layer_mathunary: add asinh support 
Signed-off-by: Ting Fu <ting.fu@intel.com>
 2020-07-06 12:45:14 +08:00
Ting Fu ea71e731f4 dnn_backend_native_layer_mathunary: add tanh support 
Signed-off-by: Ting Fu <ting.fu@intel.com>
 2020-07-06 12:45:14 +08:00
Ting Fu 62fc7e3035 dnn_backend_native_layer_mathunary: add cosh support 
Signed-off-by: Ting Fu <ting.fu@intel.com>
 2020-07-06 12:45:14 +08:00
Ting Fu 91b4037101 dnn_backend_native_layer_mathunary: add sinh support 
Signed-off-by: Ting Fu <ting.fu@intel.com>
 2020-07-06 12:45:14 +08:00
Guo, Yejun ff37ebaf30 dnn: add openvino as one of dnn backend 
OpenVINO is a Deep Learning Deployment Toolkit at
https://github.com/openvinotoolkit/openvino, it supports CPU, GPU
and heterogeneous plugins to accelerate deep learning inferencing.

Please refer to https://github.com/openvinotoolkit/openvino/blob/master/build-instruction.md
to build openvino (c library is built at the same time). Please add
option -DENABLE_MKL_DNN=ON for cmake to enable CPU path. The header
files and libraries are installed to /usr/local/deployment_tools/inference_engine/
with default options on my system.

To build FFmpeg with openvion, take my system as an example, run with:
$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/deployment_tools/inference_engine/lib/intel64/:/usr/local/deployment_tools/inference_engine/external/tbb/lib/
$ ../ffmpeg/configure --enable-libopenvino --extra-cflags=-I/usr/local/deployment_tools/inference_engine/include/ --extra-ldflags=-L/usr/local/deployment_tools/inference_engine/lib/intel64
$ make

Here are the features provided by OpenVINO inference engine:
- support more DNN model formats
It supports TensorFlow, Caffe, ONNX, MXNet and Kaldi by converting them
into OpenVINO format with a python script. And torth model
can be first converted into ONNX and then to OpenVINO format.

see the script at https://github.com/openvinotoolkit/openvino/tree/master/model-optimizer/mo.py
which also does some optimization at model level.

- optimize at inference stage
It optimizes for X86 CPUs with SSE, AVX etc.

It also optimizes based on OpenCL for Intel GPUs.
(only Intel GPU supported becuase Intel OpenCL extension is used for optimization)

Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
 2020-07-02 09:36:34 +08:00
Ting Fu 13f5613e68 dnn_backend_native_layer_mathunary: add atan support 
It can be tested with the model generated with below python script:

import tensorflow as tf
import numpy as np
import imageio

in_img = imageio.imread('input.jpeg')
in_img = in_img.astype(np.float32)/255.0
in_data = in_img[np.newaxis, :]

x = tf.placeholder(tf.float32, shape=[1, None, None, 3], name='dnn_in')
x1 = tf.atan(x)
x2 = tf.divide(x1, 3.1416/4) # pi/4
y = tf.identity(x2, name='dnn_out')

sess=tf.Session()
sess.run(tf.global_variables_initializer())

graph_def = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ['dnn_out'])
tf.train.write_graph(graph_def, '.', 'image_process.pb', as_text=False)

print("image_process.pb generated, please use \
path_to_ffmpeg/tools/python/convert.py to generate image_process.model\n")

output = sess.run(y, feed_dict={x: in_data})
imageio.imsave("out.jpg", np.squeeze(output))

Signed-off-by: Ting Fu <ting.fu@intel.com>
Signed-off-by: Guo Yejun <yejun.guo@intel.com>
 2020-06-25 08:41:50 +08:00
Ting Fu 461485feac dnn_backend_native_layer_mathunary: add acos support 
It can be tested with the model generated with below python script:

import tensorflow as tf
import numpy as np
import imageio

in_img = imageio.imread('input.jpeg')
in_img = in_img.astype(np.float32)/255.0
in_data = in_img[np.newaxis, :]

x = tf.placeholder(tf.float32, shape=[1, None, None, 3], name='dnn_in')
x1 = tf.acos(x)
x2 = tf.divide(x1, 3.1416/2) # pi/2
y = tf.identity(x2, name='dnn_out')

sess=tf.Session()
sess.run(tf.global_variables_initializer())

graph_def = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ['dnn_out'])
tf.train.write_graph(graph_def, '.', 'image_process.pb', as_text=False)

print("image_process.pb generated, please use \
path_to_ffmpeg/tools/python/convert.py to generate image_process.model\n")

output = sess.run(y, feed_dict={x: in_data})
imageio.imsave("out.jpg", np.squeeze(output))

Signed-off-by: Ting Fu <ting.fu@intel.com>
Signed-off-by: Guo Yejun <yejun.guo@intel.com>
 2020-06-25 08:41:50 +08:00
Ting Fu 486c0c419d dnn_backend_native_layer_mathunary: add asin support 
It can be tested with the model generated with below python script:

import tensorflow as tf
import numpy as np
import imageio

in_img = imageio.imread('input.jpeg')
in_img = in_img.astype(np.float32)/255.0
in_data = in_img[np.newaxis, :]

x = tf.placeholder(tf.float32, shape=[1, None, None, 3], name='dnn_in')
x1 = tf.asin(x)
x2 = tf.divide(x1, 3.1416/2) # pi/2
y = tf.identity(x2, name='dnn_out')

sess=tf.Session()
sess.run(tf.global_variables_initializer())

graph_def = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ['dnn_out'])
tf.train.write_graph(graph_def, '.', 'image_process.pb', as_text=False)

print("image_process.pb generated, please use \
path_to_ffmpeg/tools/python/convert.py to generate image_process.model\n")

output = sess.run(y, feed_dict={x: in_data})
imageio.imsave("out.jpg", np.squeeze(output))

Signed-off-by: Ting Fu <ting.fu@intel.com>
Signed-off-by: Guo Yejun <yejun.guo@intel.com>
 2020-06-25 08:41:50 +08:00
Guo Yejun 0b3bd001ac dnn_backend_native: check operand index 
it fixed the issue in https://trac.ffmpeg.org/ticket/8716
 2020-06-17 13:42:52 +08:00
Guo Yejun fc932195ab dnn_backend_native.c: refine code for fail case 2020-06-17 13:42:52 +08:00
Ting Fu 22d0860c13 dnn_backend_native_layer_mathunary: add tan support 
It can be tested with the model generated with below python scripy

import tensorflow as tf
import numpy as np
import imageio

in_img = imageio.imread('input.jpeg')
in_img = in_img.astype(np.float32)/255.0
in_data = in_img[np.newaxis, :]

x = tf.placeholder(tf.float32, shape=[1, None, None, 3], name='dnn_in')
x1 = tf.multiply(x, 0.78)
x2 = tf.tan(x1)
y = tf.identity(x2, name='dnn_out')

sess=tf.Session()
sess.run(tf.global_variables_initializer())

graph_def = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ['dnn_out'])
tf.train.write_graph(graph_def, '.', 'image_process.pb', as_text=False)

print("image_process.pb generated, please use \
path_to_ffmpeg/tools/python/convert.py to generate image_process.model\n")

output = sess.run(y, feed_dict={x: in_data})
imageio.imsave("out.jpg", np.squeeze(output))

Signed-off-by: Ting Fu <ting.fu@intel.com>
Signed-off-by: Guo Yejun <yejun.guo@intel.com>
 2020-06-11 11:10:51 +08:00
Ting Fu 88fb494f42 dnn_backend_native_layer_mathunary: add cos support 
It can be tested with the model generated with below python scripy

import tensorflow as tf
import numpy as np
import imageio

in_img = imageio.imread('input.jpeg')
in_img = in_img.astype(np.float32)/255.0
in_data = in_img[np.newaxis, :]

x = tf.placeholder(tf.float32, shape=[1, None, None, 3], name='dnn_in')
x1 = tf.multiply(x, 1.5)
x2 = tf.cos(x1)
y = tf.identity(x2, name='dnn_out')

sess=tf.Session()
sess.run(tf.global_variables_initializer())

graph_def = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ['dnn_out'])
tf.train.write_graph(graph_def, '.', 'image_process.pb', as_text=False)

print("image_process.pb generated, please use \
path_to_ffmpeg/tools/python/convert.py to generate image_process.model\n")

output = sess.run(y, feed_dict={x: in_data})
imageio.imsave("out.jpg", np.squeeze(output))

Signed-off-by: Ting Fu <ting.fu@intel.com>
Signed-off-by: Guo Yejun <yejun.guo@intel.com>
 2020-06-11 11:10:51 +08:00
Ting Fu 0b6d3f0d83 dnn_backend_native_layer_mathunary: add sin support 
It can be tested with the model file generated with below python scripy:

import tensorflow as tf
import numpy as np
import imageio

in_img = imageio.imread('input.jpeg')
in_img = in_img.astype(np.float32)/255.0
in_data = in_img[np.newaxis, :]

x = tf.placeholder(tf.float32, shape=[1, None, None, 3], name='dnn_in')
x1 = tf.multiply(x, 3.14)
x2 = tf.sin(x1)
y = tf.identity(x2, name='dnn_out')

sess=tf.Session()
sess.run(tf.global_variables_initializer())

graph_def = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ['dnn_out'])
tf.train.write_graph(graph_def, '.', 'image_process.pb', as_text=False)

print("image_process.pb generated, please use \
path_to_ffmpeg/tools/python/convert.py to generate image_process.model\n")

output = sess.run(y, feed_dict={x: in_data})
imageio.imsave("out.jpg", np.squeeze(output))

Signed-off-by: Ting Fu <ting.fu@intel.com>
Signed-off-by: Guo Yejun <yejun.guo@intel.com>
 2020-06-11 11:10:51 +08:00
Wu Zhiwen b6d7c4c1d4 dnn/native: fix typo for definition of DOT_INTERMEDIATE 
Signed-off-by: Wu Zhiwen <zhiwen.wu@intel.com>
Reviewed-by: Guo Yejun <yejun.guo@intel.com>
 2020-06-03 09:57:22 +08:00
Ting Fu f73cc61bf5 dnn_backend_native_layer_mathunary: add abs support 
more math unary operations will be added here

It can be tested with the model file generated with below python scripy:

import tensorflow as tf
import numpy as np
import imageio

in_img = imageio.imread('input.jpeg')
in_img = in_img.astype(np.float32)/255.0
in_data = in_img[np.newaxis, :]

x = tf.placeholder(tf.float32, shape=[1, None, None, 3], name='dnn_in')
x1 = tf.subtract(x, 0.5)
x2 = tf.abs(x1)
y = tf.identity(x2, name='dnn_out')

sess=tf.Session()
sess.run(tf.global_variables_initializer())

graph_def = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ['dnn_out'])
tf.train.write_graph(graph_def, '.', 'image_process.pb', as_text=False)

print("image_process.pb generated, please use \
path_to_ffmpeg/tools/python/convert.py to generate image_process.model\n")

output = sess.run(y, feed_dict={x: in_data})
imageio.imsave("out.jpg", np.squeeze(output))

Signed-off-by: Ting Fu <ting.fu@intel.com>
Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
 2020-05-28 11:04:21 +08:00
Guo, Yejun 71e28c5422 dnn/native: add native support for minimum 
it can be tested with model file generated with below python script:
import tensorflow as tf
import numpy as np
import imageio

in_img = imageio.imread('input.jpg')
in_img = in_img.astype(np.float32)/255.0
in_data = in_img[np.newaxis, :]

x = tf.placeholder(tf.float32, shape=[1, None, None, 3], name='dnn_in')
x1 = tf.minimum(0.7, x)
x2 = tf.maximum(x1, 0.4)
y = tf.identity(x2, name='dnn_out')

sess=tf.Session()
sess.run(tf.global_variables_initializer())

graph_def = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ['dnn_out'])
tf.train.write_graph(graph_def, '.', 'image_process.pb', as_text=False)

print("image_process.pb generated, please use \
path_to_ffmpeg/tools/python/convert.py to generate image_process.model\n")

output = sess.run(y, feed_dict={x: in_data})
imageio.imsave("out.jpg", np.squeeze(output))

Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
 2020-05-08 15:22:27 +08:00
Guo, Yejun 8ce9d88f93 dnn/native: add native support for divide 
it can be tested with model file generated with below python script:
import tensorflow as tf
import numpy as np
import imageio

in_img = imageio.imread('input.jpg')
in_img = in_img.astype(np.float32)/255.0
in_data = in_img[np.newaxis, :]

x = tf.placeholder(tf.float32, shape=[1, None, None, 3], name='dnn_in')
z1 = 2 / x
z2 = 1 / z1
z3 = z2 / 0.25 + 0.3
z4 = z3 - x * 1.5 - 0.3
y = tf.identity(z4, name='dnn_out')

sess=tf.Session()
sess.run(tf.global_variables_initializer())

graph_def = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ['dnn_out'])
tf.train.write_graph(graph_def, '.', 'image_process.pb', as_text=False)

print("image_process.pb generated, please use \
path_to_ffmpeg/tools/python/convert.py to generate image_process.model\n")

output = sess.run(y, feed_dict={x: in_data})
imageio.imsave("out.jpg", np.squeeze(output))

Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
 2020-04-22 13:15:00 +08:00
Guo, Yejun ef79408e97 dnn/native: add native support for 'mul' 
it can be tested with model file generated from above python script:

import tensorflow as tf
import numpy as np
import imageio

in_img = imageio.imread('input.jpg')
in_img = in_img.astype(np.float32)/255.0
in_data = in_img[np.newaxis, :]

x = tf.placeholder(tf.float32, shape=[1, None, None, 3], name='dnn_in')
z1 = 0.5 + 0.3 * x
z2 = z1 * 4
z3 = z2 - x - 2.0
y = tf.identity(z3, name='dnn_out')

sess=tf.Session()
sess.run(tf.global_variables_initializer())

graph_def = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ['dnn_out'])
tf.train.write_graph(graph_def, '.', 'image_process.pb', as_text=False)

print("image_process.pb generated, please use \
path_to_ffmpeg/tools/python/convert.py to generate image_process.model\n")

output = sess.run(y, feed_dict={x: in_data})
imageio.imsave("out.jpg", np.squeeze(output))

Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
 2020-04-22 13:14:47 +08:00
Guo, Yejun 6aa7e07e7c dnn/native: add native support for 'add' 
It can be tested with the model file generated with below python script:

import tensorflow as tf
import numpy as np
import imageio

in_img = imageio.imread('input.jpg')
in_img = in_img.astype(np.float32)/255.0
in_data = in_img[np.newaxis, :]

x = tf.placeholder(tf.float32, shape=[1, None, None, 3], name='dnn_in')
z1 = 0.039 + x
z2 = x + 0.042
z3 = z1 + z2
z4 = z3 - 0.381
z5 = z4 - x
y = tf.math.maximum(z5, 0.0, name='dnn_out')

sess=tf.Session()
sess.run(tf.global_variables_initializer())

graph_def = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ['dnn_out'])
tf.train.write_graph(graph_def, '.', 'image_process.pb', as_text=False)

print("image_process.pb generated, please use \
path_to_ffmpeg/tools/python/convert.py to generate image_process.model\n")

output = sess.run(y, feed_dict={x: in_data})
imageio.imsave("out.jpg", np.squeeze(output))

Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
 2020-04-22 13:14:30 +08:00
Guo, Yejun ffa1561608 dnn_backend_native_layer_mathbinary: add sub support 
more math binary operations will be added here

Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
 2020-04-07 11:04:34 +08:00
Carl Eugen Hoyos 61dcaf5fb7 lavf, lavfi: Remove uses of sizeof(char). 
The C standard requires sizeof(char) == 1.
 2020-04-04 23:21:14 +02:00
Guo, Yejun f4b3c0e55c avfilter/dnn: add a new interface to query dnn model's input info 
to support dnn networks more general, we need to know the input info
of the dnn model.

background:
The data type of dnn model's input could be float32, uint8 or fp16, etc.
And the w/h of input image could be fixed or variable.

Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
 2019-10-30 11:07:06 -03:00
Guo, Yejun e1b45b8596 avfilter/dnn: get the data type of network output from dnn execution result 
so,  we can make a filter more general to accept different network
models, by adding a data type convertion after getting data from network.

After we add dt field into struct DNNData, it becomes the same as
DNNInputData, so merge them with one struct: DNNData.

Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
 2019-10-30 11:00:41 -03:00
Guo, Yejun dff39ea9f0 dnn: add tf.nn.conv2d support for native model 
Unlike other tf.*.conv2d layers, tf.nn.conv2d does not create many
nodes (within a scope) in the graph, it just acts like other layers.
tf.nn.conv2d only creates one node in the graph, and no internal
nodes such as 'kernel' are created.

The format of native model file is also changed, a flag named
has_bias is added, so change the version number.

Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
 2019-10-30 10:31:55 -03:00
Guo, Yejun 2558e62713 avfilter/dnn: unify the layer load function in native mode 
Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
 2019-10-15 18:56:54 -03:00
Guo, Yejun 3fd5ac7e92 avfilter/dnn: unify the layer execution function in native mode 
Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
 2019-10-15 18:56:25 -03:00
Guo, Yejun b78dc27bba avfilter/dnn: add DLT prefix for enum DNNLayerType to avoid potential conflicts 
and also change CONV to DLT_CONV2D for better description

Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
 2019-10-15 16:35:39 -03:00
Guo, Yejun 8f13a557ca libavfilter/dnn: support multiple outputs for native mode 
Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
 2019-09-20 14:51:57 -03:00
Guo, Yejun 75ca94f3cf libavfilter/dnn/dnn_backend_native: find the input operand according to input name 
Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
 2019-09-20 14:51:50 -03:00