video: rewrite filtering glue code
Get rid of the old vf.c code. Replace it with a generic filtering
framework, which can potentially handle more than just --vf. At least
reimplementing --af with this code is planned.
This changes some --vf semantics (including runtime behavior and the
"vf" command). The most important ones are listed in interface-changes.
vf_convert.c is renamed to f_swscale.c. It is now an internal filter
that can not be inserted by the user manually.
f_lavfi.c is a refactor of player/lavfi.c. The latter will be removed
once --lavfi-complex is reimplemented on top of f_lavfi.c. (which is
conceptually easy, but a big mess due to the data flow changes).
The existing filters are all changed heavily. The data flow of the new
filter framework is different. Especially EOF handling changes - EOF is
now a "frame" rather than a state, and must be passed through exactly
once.
Another major thing is that all filters must support dynamic format
changes. The filter reconfig() function goes away. (This sounds complex,
but since all filters need to handle EOF draining anyway, they can use
the same code, and it removes the mess with reconfig() having to predict
the output format, which completely breaks with libavfilter anyway.)
In addition, there is no automatic format negotiation or conversion.
libavfilter's primitive and insufficient API simply doesn't allow us to
do this in a reasonable way. Instead, filters can use f_autoconvert as
sub-filter, and tell it which formats they support. This filter will in
turn add actual conversion filters, such as f_swscale, to perform
necessary format changes.
vf_vapoursynth.c uses the same basic principle of operation as before,
but with worryingly different details in data flow. Still appears to
work.
The hardware deint filters (vf_vavpp.c, vf_d3d11vpp.c, vf_vdpaupp.c) are
heavily changed. Fortunately, they all used refqueue.c, which is for
sharing the data flow logic (especially for managing future/past
surfaces and such). It turns out it can be used to factor out most of
the data flow. Some of these filters accepted software input. Instead of
having ad-hoc upload code in each filter, surface upload is now
delegated to f_autoconvert, which can use f_hwupload to perform this.
Exporting VO capabilities is still a big mess (mp_stream_info stuff).
The D3D11 code drops the redundant image formats, and all code uses the
hw_subfmt (sw_format in FFmpeg) instead. Although that too seems to be a
big mess for now.
f_async_queue is unused.
2018-01-16 10:53:44 +00:00
|
|
|
#pragma once
|
|
|
|
|
|
|
|
#include "options/m_option.h"
|
|
|
|
#include "video/mp_image.h"
|
|
|
|
|
|
|
|
#include "filter.h"
|
|
|
|
|
|
|
|
enum mp_output_chain_type {
|
|
|
|
MP_OUTPUT_CHAIN_VIDEO = 1, // --vf
|
2018-01-18 13:44:20 +00:00
|
|
|
MP_OUTPUT_CHAIN_AUDIO, // --af
|
video: rewrite filtering glue code
Get rid of the old vf.c code. Replace it with a generic filtering
framework, which can potentially handle more than just --vf. At least
reimplementing --af with this code is planned.
This changes some --vf semantics (including runtime behavior and the
"vf" command). The most important ones are listed in interface-changes.
vf_convert.c is renamed to f_swscale.c. It is now an internal filter
that can not be inserted by the user manually.
f_lavfi.c is a refactor of player/lavfi.c. The latter will be removed
once --lavfi-complex is reimplemented on top of f_lavfi.c. (which is
conceptually easy, but a big mess due to the data flow changes).
The existing filters are all changed heavily. The data flow of the new
filter framework is different. Especially EOF handling changes - EOF is
now a "frame" rather than a state, and must be passed through exactly
once.
Another major thing is that all filters must support dynamic format
changes. The filter reconfig() function goes away. (This sounds complex,
but since all filters need to handle EOF draining anyway, they can use
the same code, and it removes the mess with reconfig() having to predict
the output format, which completely breaks with libavfilter anyway.)
In addition, there is no automatic format negotiation or conversion.
libavfilter's primitive and insufficient API simply doesn't allow us to
do this in a reasonable way. Instead, filters can use f_autoconvert as
sub-filter, and tell it which formats they support. This filter will in
turn add actual conversion filters, such as f_swscale, to perform
necessary format changes.
vf_vapoursynth.c uses the same basic principle of operation as before,
but with worryingly different details in data flow. Still appears to
work.
The hardware deint filters (vf_vavpp.c, vf_d3d11vpp.c, vf_vdpaupp.c) are
heavily changed. Fortunately, they all used refqueue.c, which is for
sharing the data flow logic (especially for managing future/past
surfaces and such). It turns out it can be used to factor out most of
the data flow. Some of these filters accepted software input. Instead of
having ad-hoc upload code in each filter, surface upload is now
delegated to f_autoconvert, which can use f_hwupload to perform this.
Exporting VO capabilities is still a big mess (mp_stream_info stuff).
The D3D11 code drops the redundant image formats, and all code uses the
hw_subfmt (sw_format in FFmpeg) instead. Although that too seems to be a
big mess for now.
f_async_queue is unused.
2018-01-16 10:53:44 +00:00
|
|
|
};
|
|
|
|
|
|
|
|
// A classic single-media filter chain, which reflects --vf and --af.
|
|
|
|
// It manages the user-specified filter chain, and VO/AO output conversions.
|
|
|
|
// Also handles some automatic filtering (auto rotation and such).
|
|
|
|
struct mp_output_chain {
|
|
|
|
// This filter will have 1 input (from decoder) and 1 output (to VO/AO).
|
|
|
|
struct mp_filter *f;
|
|
|
|
|
|
|
|
bool got_output_eof;
|
|
|
|
|
|
|
|
// The filter chain output could not be converted to any format the output
|
|
|
|
// supports.
|
|
|
|
bool failed_output_conversion;
|
|
|
|
|
|
|
|
// Set if any formats in the chain changed. The user can reset the flag.
|
|
|
|
// For implementing change notifications out input/output_params.
|
|
|
|
bool reconfig_happened;
|
|
|
|
|
|
|
|
// --- for type==MP_OUTPUT_CHAIN_VIDEO
|
|
|
|
struct mp_image_params input_params;
|
|
|
|
struct mp_image_params output_params;
|
|
|
|
double container_fps;
|
video: make decoder wrapper a filter
Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes
a source filter. vd.h mostly disappears, because mp_filter takes care of
the dataflow, but its remains are in struct mp_decoder_fns.
One goal is to simplify dataflow by letting the filter framework handle
it (or more accurately, using its conventions). One result is that the
decode calls disappear from video.c, because we simply connect the
decoder wrapper and the filter chain with mp_pin_connect().
Another goal is to eventually remove the code duplication between the
audio and video paths for this. This commit prepares for this by trying
to make f_decoder_wrapper.c extensible, so it can be used for audio as
well later.
Decoder framedropping changes a bit. It doesn't seem to be worse than
before, and it's an obscure feature, so I'm content with its new state.
Some special code that was apparently meant to avoid dropping too many
frames in a row is removed, though.
I'm not sure how the source code tree should be organized. For one,
video/decode/vd_lavc.c is the only file in its directory, which is a bit
annoying.
2018-01-28 09:08:45 +00:00
|
|
|
void (*update_subtitles)(void *ctx, double pts);
|
|
|
|
void *update_subtitles_ctx;
|
2018-01-18 13:44:20 +00:00
|
|
|
|
|
|
|
// --- for type==MP_OUTPUT_CHAIN_AUDIO
|
|
|
|
struct mp_aframe *input_aformat;
|
|
|
|
struct mp_aframe *output_aformat;
|
|
|
|
// If true, there was a format change. output_aformat might have changed,
|
|
|
|
// and the implementation drained the filter chain and unset the internal ao
|
|
|
|
// reference. The API user needs to call mp_output_chain_set_ao() again.
|
|
|
|
// Until this is done, the filter chain will not output new data.
|
|
|
|
bool ao_needs_update;
|
video: rewrite filtering glue code
Get rid of the old vf.c code. Replace it with a generic filtering
framework, which can potentially handle more than just --vf. At least
reimplementing --af with this code is planned.
This changes some --vf semantics (including runtime behavior and the
"vf" command). The most important ones are listed in interface-changes.
vf_convert.c is renamed to f_swscale.c. It is now an internal filter
that can not be inserted by the user manually.
f_lavfi.c is a refactor of player/lavfi.c. The latter will be removed
once --lavfi-complex is reimplemented on top of f_lavfi.c. (which is
conceptually easy, but a big mess due to the data flow changes).
The existing filters are all changed heavily. The data flow of the new
filter framework is different. Especially EOF handling changes - EOF is
now a "frame" rather than a state, and must be passed through exactly
once.
Another major thing is that all filters must support dynamic format
changes. The filter reconfig() function goes away. (This sounds complex,
but since all filters need to handle EOF draining anyway, they can use
the same code, and it removes the mess with reconfig() having to predict
the output format, which completely breaks with libavfilter anyway.)
In addition, there is no automatic format negotiation or conversion.
libavfilter's primitive and insufficient API simply doesn't allow us to
do this in a reasonable way. Instead, filters can use f_autoconvert as
sub-filter, and tell it which formats they support. This filter will in
turn add actual conversion filters, such as f_swscale, to perform
necessary format changes.
vf_vapoursynth.c uses the same basic principle of operation as before,
but with worryingly different details in data flow. Still appears to
work.
The hardware deint filters (vf_vavpp.c, vf_d3d11vpp.c, vf_vdpaupp.c) are
heavily changed. Fortunately, they all used refqueue.c, which is for
sharing the data flow logic (especially for managing future/past
surfaces and such). It turns out it can be used to factor out most of
the data flow. Some of these filters accepted software input. Instead of
having ad-hoc upload code in each filter, surface upload is now
delegated to f_autoconvert, which can use f_hwupload to perform this.
Exporting VO capabilities is still a big mess (mp_stream_info stuff).
The D3D11 code drops the redundant image formats, and all code uses the
hw_subfmt (sw_format in FFmpeg) instead. Although that too seems to be a
big mess for now.
f_async_queue is unused.
2018-01-16 10:53:44 +00:00
|
|
|
};
|
|
|
|
|
|
|
|
// (free by freeing mp_output_chain.f)
|
|
|
|
struct mp_output_chain *mp_output_chain_create(struct mp_filter *parent,
|
|
|
|
enum mp_output_chain_type type);
|
|
|
|
|
|
|
|
// Set the VO, which will be used to determine basic capabilities like format
|
|
|
|
// and rotation support, and to init hardware filtering things.
|
|
|
|
// For type==MP_OUTPUT_CHAIN_VIDEO only.
|
|
|
|
struct vo;
|
|
|
|
void mp_output_chain_set_vo(struct mp_output_chain *p, struct vo *vo);
|
|
|
|
|
2018-01-18 13:44:20 +00:00
|
|
|
// Set the AO. The AO format will be used to determine the filter chain output.
|
|
|
|
// The API user may be asked to update the AO midstream if ao_needs_update is
|
|
|
|
// set.
|
|
|
|
// For type==MP_OUTPUT_CHAIN_AUDIO only.
|
|
|
|
struct ao;
|
|
|
|
void mp_output_chain_set_ao(struct mp_output_chain *p, struct ao *ao);
|
|
|
|
|
video: rewrite filtering glue code
Get rid of the old vf.c code. Replace it with a generic filtering
framework, which can potentially handle more than just --vf. At least
reimplementing --af with this code is planned.
This changes some --vf semantics (including runtime behavior and the
"vf" command). The most important ones are listed in interface-changes.
vf_convert.c is renamed to f_swscale.c. It is now an internal filter
that can not be inserted by the user manually.
f_lavfi.c is a refactor of player/lavfi.c. The latter will be removed
once --lavfi-complex is reimplemented on top of f_lavfi.c. (which is
conceptually easy, but a big mess due to the data flow changes).
The existing filters are all changed heavily. The data flow of the new
filter framework is different. Especially EOF handling changes - EOF is
now a "frame" rather than a state, and must be passed through exactly
once.
Another major thing is that all filters must support dynamic format
changes. The filter reconfig() function goes away. (This sounds complex,
but since all filters need to handle EOF draining anyway, they can use
the same code, and it removes the mess with reconfig() having to predict
the output format, which completely breaks with libavfilter anyway.)
In addition, there is no automatic format negotiation or conversion.
libavfilter's primitive and insufficient API simply doesn't allow us to
do this in a reasonable way. Instead, filters can use f_autoconvert as
sub-filter, and tell it which formats they support. This filter will in
turn add actual conversion filters, such as f_swscale, to perform
necessary format changes.
vf_vapoursynth.c uses the same basic principle of operation as before,
but with worryingly different details in data flow. Still appears to
work.
The hardware deint filters (vf_vavpp.c, vf_d3d11vpp.c, vf_vdpaupp.c) are
heavily changed. Fortunately, they all used refqueue.c, which is for
sharing the data flow logic (especially for managing future/past
surfaces and such). It turns out it can be used to factor out most of
the data flow. Some of these filters accepted software input. Instead of
having ad-hoc upload code in each filter, surface upload is now
delegated to f_autoconvert, which can use f_hwupload to perform this.
Exporting VO capabilities is still a big mess (mp_stream_info stuff).
The D3D11 code drops the redundant image formats, and all code uses the
hw_subfmt (sw_format in FFmpeg) instead. Although that too seems to be a
big mess for now.
f_async_queue is unused.
2018-01-16 10:53:44 +00:00
|
|
|
// Send a command to the filter with the target label.
|
|
|
|
bool mp_output_chain_command(struct mp_output_chain *p, const char *target,
|
|
|
|
struct mp_filter_command *cmd);
|
|
|
|
|
|
|
|
// Perform a seek reset _and_ reset all filter failure states, so that future
|
|
|
|
// filtering continues normally.
|
|
|
|
void mp_output_chain_reset_harder(struct mp_output_chain *p);
|
|
|
|
|
|
|
|
// Try to exchange the filter list. If creation of any filter fails, roll
|
|
|
|
// back the changes, and return false.
|
|
|
|
struct m_obj_settings;
|
|
|
|
bool mp_output_chain_update_filters(struct mp_output_chain *p,
|
|
|
|
struct m_obj_settings *list);
|
|
|
|
|
2018-01-18 13:44:20 +00:00
|
|
|
// Desired audio speed, with resample being strict resampling.
|
|
|
|
void mp_output_chain_set_audio_speed(struct mp_output_chain *p,
|
2020-05-23 02:04:46 +00:00
|
|
|
double speed, double resample, double drop);
|
2018-01-18 13:44:20 +00:00
|
|
|
|
2018-04-07 12:38:54 +00:00
|
|
|
// Total delay incurred by the filter chain, as measured by the recent filtered
|
2018-01-18 13:44:20 +00:00
|
|
|
// frames. The intention is that this sums the measured delays for each filter,
|
|
|
|
// so if a filter is removed, the caller can estimate how much audio is missing
|
|
|
|
// due to the change.
|
|
|
|
// Makes sense for audio only.
|
|
|
|
double mp_output_get_measured_total_delay(struct mp_output_chain *p);
|