RepoMirrors
/
mpv
mirror of https://github.com/mpv-player/mpv
1
0
Fork 0
Code Issues Releases Wiki Activity
mpv/video/zimg.c

712 lines
23 KiB
C
Raw Normal View History

video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
/*
* This file is part of mpv.
*
* mpv is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation; either
* version 2.1 of the License, or (at your option) any later version.
*
* mpv is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with mpv. If not, see <http://www.gnu.org/licenses/>.
*/
Use mp_log2() instead of av_log2()
2019-10-31 12:17:18 +00:00
#include <math.h>
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
zimg: add slice threading and use it by default This probably makes it much faster (I wouldn't know, I didn't run any benchmarks ). Seems to work as well (although I'm not sure, it's not like I'd perform rigorous tests). The scale_zimg test seems to mysteriously treat color in fully transparent alpha differently, which makes no sense, and isn't visible (but makes the test fail). I can't be bothered with investigating this more. What do you do with failing tests? Correct, you disable them. Or rather, you disable whatever appears to cause them to fail, which is the threading in this case. This change follows mostly the tile_example.cpp. The slice size uses a minimum of 64, which was suggested by the zimg author. Some of this commit is a bit inelegant and weird, such as recomputing the scale factor for every slice, or the way slice_h is managed. Too lazy to make this more elegant. zimg git had a regressio around active_region (which is needed by the slicing), which was fixed in commit 83071706b2e6bc634. Apparently, the bug was never released, so just add a warning to the manpage.
2020-07-14 20:52:27 +00:00
#include <libavutil/cpu.h>
zimg: add support for big endian input and output One of the extremely annoying dumb things in ffmpeg is that most pixel formats are available as little endian and big endian variants. (The sane way would be having native endian formats only.) Usually, most of the real codecs use native formats only, while non-native formats are used by fringe raw codecs only. But the PNG encoders and decoders unfortunately use big endian formats, and since PNG it such a popular format, this causes problems for us. In particular, the current zimg wrapper will refuse to work (and mpv will fall back to sws) when writing non-8 bit PNGs. So add non-native endian support to zimg. This is done in a fairly "generic" way (which means lots of potential for bugs). If input is a "regular" format (and just byte-swapped), the rest happens automatically, which happens to cover all interesting formats. Some things could be more efficient; for example, unpacking is done on the data before it's passed to the unpacker. You could make endian swapping part of the actual unpacking process, which might be slightly faster. You could avoid copying twice in some cases (such as when there's no actual repacker, or if alignment needs to be corrected). But I don't really care. It's reasonably fast for the normal case. Not entirely sure whether this is correct. Some (but not many) formats are covered by the tests, some I tested manually. Some I can't even test, because libswscale doesn't support them (like nv20*).
2020-04-12 20:14:55 +00:00
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
#include "common/common.h"
#include "common/msg.h"
#include "csputils.h"
zimg: add slice threading and use it by default This probably makes it much faster (I wouldn't know, I didn't run any benchmarks ). Seems to work as well (although I'm not sure, it's not like I'd perform rigorous tests). The scale_zimg test seems to mysteriously treat color in fully transparent alpha differently, which makes no sense, and isn't visible (but makes the test fail). I can't be bothered with investigating this more. What do you do with failing tests? Correct, you disable them. Or rather, you disable whatever appears to cause them to fail, which is the threading in this case. This change follows mostly the tile_example.cpp. The slice size uses a minimum of 64, which was suggested by the zimg author. Some of this commit is a bit inelegant and weird, such as recomputing the scale factor for every slice, or the way slice_h is managed. Too lazy to make this more elegant. zimg git had a regressio around active_region (which is needed by the slicing), which was fixed in commit 83071706b2e6bc634. Apparently, the bug was never released, so just add a warning to the manpage.
2020-07-14 20:52:27 +00:00
#include "misc/thread_pool.h"
#include "misc/thread_tools.h"
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
#include "options/m_config.h"
#include "options/m_option.h"
video: separate repacking code from zimg and make it independent For whatever purpose. If anything, this makes the zimg wrapper cleaner. The added tests are not particular exhaustive, but nice to have. This also makes the scale_zimg.c test pretty useless, because it only tests repacking (going through the zimg wrapper). In theory, the repack_tests things could also be used on scalers, but I guess it doesn't matter. Some things are added over the previous zimg wrapper code. For example, some fringe formats can now be expanded to 8 bit per component for convenience.
2020-05-09 15:56:44 +00:00
#include "repack.h"
zimg: add support for some RGB fringe formats This covers 8 and 16 bit packed RGB formats. It doesn't really help with any actual use-cases, other than giving the finger to libswscale. One problem is with different color depths. For example, rgb565 provides 1 bit more resolution to the green channel. zimg can only dither to a uniform depth. I tried dithering to the highest depth and shifting away 1 bit for the lower channels, but that looked ugly (or I messed up somewhere), so instead it dithers to the lowest depth, and adjusts the value range if needed. Testing with bgr4_byte (extreme case with 1/2/1 depths), it looks more "grainy" (ordered dithering artifacts) than libswscale, but it also looks cleaner and smoother. It doesn't have libswscale's weird red-shift. So I call it a success. Big endian formats need to be handled explicitly; the generic big endian swapper code assumes byte-aligned components. Unpacking is done with shifts and 3 LUTs. This is symmetric to the packer. Using a generated palette might be better, but I preferred to keep the symmetry, and not having to mess with a generated palette and the pal8 code. This uses FFmepg pixfmts constants directly. I would have preferred keeping zimg completely separate. But neither do I want to add an IMGFMT alias for every of these formats, nor do I want to extend our imgfmt code such that it can provide a complete description of each packed RGB format (similar to FFmpeg pixdesc). It also appears that FFmpeg pixdesc as well as the FFmpeg pixfmt doxygen have an error regarding RGB8: the R/B bit depths are swapped. libswscale appears to be handling them differently. Not completely sure, as this is the only packed format case with R/B havuing different depths (instead of G, the middle component, where things are symmetric).
2020-04-13 00:36:46 +00:00
#include "video/fmt-conversion.h"
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
#include "video/img_format.h"
#include "zimg.h"
static_assert(MP_IMAGE_BYTE_ALIGN >= ZIMG_ALIGN, "");
zimg: add alpha support libzimg recently added direct alpha support and new API for it. (The API change is rather minimal, and it turns out we can easily support old and new zimg versions.) This does not support _all_ alpha formats. For example, gray + alpha is not supported yet, because my stupid design in the zimg wrapper would require a planar gray + alpha format, while ffmpeg provides only a packed one.
2020-02-09 18:16:54 +00:00
#define HAVE_ZIMG_ALPHA (ZIMG_API_VERSION >= ZIMG_MAKE_API_VERSION(2, 4))
sws_utils, zimg: destroy vo_x11 and vo_drm performance Raise swscale and zimg default parameters. This restores screenshot quality settings (maybe) unset in the commit before. Also expose some more libswscale and zimg options. Since these options are also used for VOs like x11 and drm, this will make x11/drm/etc. much slower. For compensation, provide a profile that sets the old option values: sw-fast. I'm also enabling zimg here, just as an experiment. The core problem is that we have a single set of command line options which control the settings used for most swscale/zimg uses. This was done in the previous commit. It cannot differentiate between the VOs, which need to be realtime and may accept/require lower quality options, and things like screenshots or vo_image, which can be slower, but should not sacrifice quality by default. Should this have two sets of options or something similar to do the right thing depending on the code which calls libswscale? Maybe. Or should I just ignore the problem, make it someone else's problem (users who want to use software conversion VOs), provide a sub-optimal solution, and call it a day? Definitely, sounds good, pushing to master, goodbye.
2019-10-31 15:45:28 +00:00
static const struct m_opt_choice_alternatives mp_zimg_scalers[] = {
{"point", ZIMG_RESIZE_POINT},
{"bilinear", ZIMG_RESIZE_BILINEAR},
{"bicubic", ZIMG_RESIZE_BICUBIC},
{"spline16", ZIMG_RESIZE_SPLINE16},
{"spline36", ZIMG_RESIZE_SPLINE36},
{"lanczos", ZIMG_RESIZE_LANCZOS},
{0}
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
};
sws_utils: allow setting zimg options directly One could wonder, why not just use the zimg wrapper directly?
2020-05-09 15:56:21 +00:00
const struct zimg_opts zimg_opts_defaults = {
.scaler = ZIMG_RESIZE_LANCZOS,
.scaler_params = {NAN, NAN},
.scaler_chroma_params = {NAN, NAN},
.scaler_chroma = ZIMG_RESIZE_BILINEAR,
.dither = ZIMG_DITHER_RANDOM,
.fast = 1,
};
options: change option macros and all option declarations Change all OPT_* macros such that they don't define the entire m_option initializer, and instead expand only to a part of it, which sets certain fields. This requires changing almost every option declaration, because they all use these macros. A declaration now always starts with {"name", ... followed by designated initializers only (possibly wrapped in macros). The OPT_* macros now initialize the .offset and .type fields only, sometimes also .priv and others. I think this change makes the option macros less tricky. The old code had to stuff everything into macro arguments (and attempted to allow setting arbitrary fields by letting the user pass designated initializers in the vararg parts). Some of this was made messy due to C99 and C11 not allowing 0-sized varargs with ',' removal. It's also possible that this change is pointless, other than cosmetic preferences. Not too happy about some things. For example, the OPT_CHOICE() indentation I applied looks a bit ugly. Much of this change was done with regex search&replace, but some places required manual editing. In particular, code in "obscure" areas (which I didn't include in compilation) might be broken now. In wayland_common.c the author of some option declarations confused the flags parameter with the default value (though the default value was also properly set below). I fixed this with this change.
2020-03-14 20:28:01 +00:00
#define OPT_PARAM(var) OPT_DOUBLE(var), .flags = M_OPT_DEFAULT_NAN
sws_utils, zimg: destroy vo_x11 and vo_drm performance Raise swscale and zimg default parameters. This restores screenshot quality settings (maybe) unset in the commit before. Also expose some more libswscale and zimg options. Since these options are also used for VOs like x11 and drm, this will make x11/drm/etc. much slower. For compensation, provide a profile that sets the old option values: sw-fast. I'm also enabling zimg here, just as an experiment. The core problem is that we have a single set of command line options which control the settings used for most swscale/zimg uses. This was done in the previous commit. It cannot differentiate between the VOs, which need to be realtime and may accept/require lower quality options, and things like screenshots or vo_image, which can be slower, but should not sacrifice quality by default. Should this have two sets of options or something similar to do the right thing depending on the code which calls libswscale? Maybe. Or should I just ignore the problem, make it someone else's problem (users who want to use software conversion VOs), provide a sub-optimal solution, and call it a day? Definitely, sounds good, pushing to master, goodbye.
2019-10-31 15:45:28 +00:00
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
#define OPT_BASE_STRUCT struct zimg_opts
const struct m_sub_options zimg_conf = {
.opts = (struct m_option[]) {
options: change option macros and all option declarations Change all OPT_* macros such that they don't define the entire m_option initializer, and instead expand only to a part of it, which sets certain fields. This requires changing almost every option declaration, because they all use these macros. A declaration now always starts with {"name", ... followed by designated initializers only (possibly wrapped in macros). The OPT_* macros now initialize the .offset and .type fields only, sometimes also .priv and others. I think this change makes the option macros less tricky. The old code had to stuff everything into macro arguments (and attempted to allow setting arbitrary fields by letting the user pass designated initializers in the vararg parts). Some of this was made messy due to C99 and C11 not allowing 0-sized varargs with ',' removal. It's also possible that this change is pointless, other than cosmetic preferences. Not too happy about some things. For example, the OPT_CHOICE() indentation I applied looks a bit ugly. Much of this change was done with regex search&replace, but some places required manual editing. In particular, code in "obscure" areas (which I didn't include in compilation) might be broken now. In wayland_common.c the author of some option declarations confused the flags parameter with the default value (though the default value was also properly set below). I fixed this with this change.
2020-03-14 20:28:01 +00:00
{"scaler", OPT_CHOICE_C(scaler, mp_zimg_scalers)},
{"scaler-param-a", OPT_PARAM(scaler_params[0])},
{"scaler-param-b", OPT_PARAM(scaler_params[1])},
{"scaler-chroma", OPT_CHOICE_C(scaler_chroma, mp_zimg_scalers)},
{"scaler-chroma-param-a", OPT_PARAM(scaler_chroma_params[0])},
{"scaler-chroma-param-b", OPT_PARAM(scaler_chroma_params[1])},
{"dither", OPT_CHOICE(dither,
{"no", ZIMG_DITHER_NONE},
{"ordered", ZIMG_DITHER_ORDERED},
{"random", ZIMG_DITHER_RANDOM},
{"error-diffusion", ZIMG_DITHER_ERROR_DIFFUSION})},
{"fast", OPT_FLAG(fast)},
zimg: add slice threading and use it by default This probably makes it much faster (I wouldn't know, I didn't run any benchmarks ). Seems to work as well (although I'm not sure, it's not like I'd perform rigorous tests). The scale_zimg test seems to mysteriously treat color in fully transparent alpha differently, which makes no sense, and isn't visible (but makes the test fail). I can't be bothered with investigating this more. What do you do with failing tests? Correct, you disable them. Or rather, you disable whatever appears to cause them to fail, which is the threading in this case. This change follows mostly the tile_example.cpp. The slice size uses a minimum of 64, which was suggested by the zimg author. Some of this commit is a bit inelegant and weird, such as recomputing the scale factor for every slice, or the way slice_h is managed. Too lazy to make this more elegant. zimg git had a regressio around active_region (which is needed by the slicing), which was fixed in commit 83071706b2e6bc634. Apparently, the bug was never released, so just add a warning to the manpage.
2020-07-14 20:52:27 +00:00
{"threads", OPT_CHOICE(threads, {"auto", 0}), M_RANGE(1, 64)},
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
{0}
},
.size = sizeof(struct zimg_opts),
sws_utils: allow setting zimg options directly One could wonder, why not just use the zimg wrapper directly?
2020-05-09 15:56:21 +00:00
.defaults = &zimg_opts_defaults,
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
};
zimg: refactor (move around fields) The intention is to add slice-threading to the wrapper. For that purpose, move all zimg related state to a new struct mp_zimg_state. There is now an array of instances of this struct. The intention is to have one instance for each thread. As of this commit, this is hardcoded to 1 thread; the following commit extends this.
2020-07-14 18:48:48 +00:00
struct mp_zimg_state {
zimg_filter_graph *graph;
void *tmp;
void *tmp_alloc;
struct mp_zimg_repack *src;
struct mp_zimg_repack *dst;
zimg: add slice threading and use it by default This probably makes it much faster (I wouldn't know, I didn't run any benchmarks ). Seems to work as well (although I'm not sure, it's not like I'd perform rigorous tests). The scale_zimg test seems to mysteriously treat color in fully transparent alpha differently, which makes no sense, and isn't visible (but makes the test fail). I can't be bothered with investigating this more. What do you do with failing tests? Correct, you disable them. Or rather, you disable whatever appears to cause them to fail, which is the threading in this case. This change follows mostly the tile_example.cpp. The slice size uses a minimum of 64, which was suggested by the zimg author. Some of this commit is a bit inelegant and weird, such as recomputing the scale factor for every slice, or the way slice_h is managed. Too lazy to make this more elegant. zimg git had a regressio around active_region (which is needed by the slicing), which was fixed in commit 83071706b2e6bc634. Apparently, the bug was never released, so just add a warning to the manpage.
2020-07-14 20:52:27 +00:00
int slice_y, slice_h; // y start position, height of target slice
double scale_y;
struct mp_waiter thread_waiter;
zimg: refactor (move around fields) The intention is to add slice-threading to the wrapper. For that purpose, move all zimg related state to a new struct mp_zimg_state. There is now an array of instances of this struct. The intention is to have one instance for each thread. As of this commit, this is hardcoded to 1 thread; the following commit extends this.
2020-07-14 18:48:48 +00:00
};
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
struct mp_zimg_repack {
bool pack; // if false, this is for unpacking
zimg: add support for big endian input and output One of the extremely annoying dumb things in ffmpeg is that most pixel formats are available as little endian and big endian variants. (The sane way would be having native endian formats only.) Usually, most of the real codecs use native formats only, while non-native formats are used by fringe raw codecs only. But the PNG encoders and decoders unfortunately use big endian formats, and since PNG it such a popular format, this causes problems for us. In particular, the current zimg wrapper will refuse to work (and mpv will fall back to sws) when writing non-8 bit PNGs. So add non-native endian support to zimg. This is done in a fairly "generic" way (which means lots of potential for bugs). If input is a "regular" format (and just byte-swapped), the rest happens automatically, which happens to cover all interesting formats. Some things could be more efficient; for example, unpacking is done on the data before it's passed to the unpacker. You could make endian swapping part of the actual unpacking process, which might be slightly faster. You could avoid copying twice in some cases (such as when there's no actual repacker, or if alignment needs to be corrected). But I don't really care. It's reasonably fast for the normal case. Not entirely sure whether this is correct. Some (but not many) formats are covered by the tests, some I tested manually. Some I can't even test, because libswscale doesn't support them (like nv20*).
2020-04-12 20:14:55 +00:00
struct mp_image_params fmt; // original mp format (possibly packed format,
// swapped endian)
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
int zimgfmt; // zimg equivalent unpacked format
zimg: rename zplanes field This was a confusing name, because 1. there's also a z_planes[] field, and 2. it was not specific to zimg indexes. Possibly there used to be an idea involved about supporting alpha to non-alpha formats by discarding the alpha plane, but zimg does this now (and zimg will correctly blend the alpha component too).
2020-02-10 16:59:13 +00:00
int num_planes; // number of planes involved
zimg: fix some confusion about plane permutation We reorder the planes between mpv and zimg conventions. It turns out the code still confused when which convention was used. So the way it actually works is that the _only_ place where zimg order is used is the zimg_image_buffer.plane[] array. plane_aligned[] and zmask[] were accessed incorrectly, although I guess it rarely had a reason to fail (plane reordering is mostly for RGB, which has planes of all the same size). Adjust some comments accordingly too.
2020-02-10 16:45:20 +00:00
unsigned zmask[4]; // zmask[mp_index] = zimg mask (using mp index!)
int z_planes[4]; // z_planes[zimg_index] = mp_index (or -1)
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
video: separate repacking code from zimg and make it independent For whatever purpose. If anything, this makes the zimg wrapper cleaner. The added tests are not particular exhaustive, but nice to have. This also makes the scale_zimg.c test pretty useless, because it only tests repacking (going through the zimg wrapper). In theory, the repack_tests things could also be used on scalers, but I guess it doesn't matter. Some things are added over the previous zimg wrapper code. For example, some fringe formats can now be expanded to 8 bit per component for convenience.
2020-05-09 15:56:44 +00:00
struct mp_repack *repack; // converting to/from planar
zimg: add support for some RGB fringe formats This covers 8 and 16 bit packed RGB formats. It doesn't really help with any actual use-cases, other than giving the finger to libswscale. One problem is with different color depths. For example, rgb565 provides 1 bit more resolution to the green channel. zimg can only dither to a uniform depth. I tried dithering to the highest depth and shifting away 1 bit for the lower channels, but that looked ugly (or I messed up somewhere), so instead it dithers to the lowest depth, and adjusts the value range if needed. Testing with bgr4_byte (extreme case with 1/2/1 depths), it looks more "grainy" (ordered dithering artifacts) than libswscale, but it also looks cleaner and smoother. It doesn't have libswscale's weird red-shift. So I call it a success. Big endian formats need to be handled explicitly; the generic big endian swapper code assumes byte-aligned components. Unpacking is done with shifts and 3 LUTs. This is symmetric to the packer. Using a generated palette might be better, but I preferred to keep the symmetry, and not having to mess with a generated palette and the pal8 code. This uses FFmepg pixfmts constants directly. I would have preferred keeping zimg completely separate. But neither do I want to add an IMGFMT alias for every of these formats, nor do I want to extend our imgfmt code such that it can provide a complete description of each packed RGB format (similar to FFmpeg pixdesc). It also appears that FFmpeg pixdesc as well as the FFmpeg pixfmt doxygen have an error regarding RGB8: the R/B bit depths are swapped. libswscale appears to be handling them differently. Not completely sure, as this is the only packed format case with R/B havuing different depths (instead of G, the middle component, where things are symmetric).
2020-04-13 00:36:46 +00:00
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
// Temporary memory for slice-wise repacking. This may be set even if repack
// is not set (then it may be used to avoid alignment issues). This has
// about one slice worth of data.
struct mp_image *tmp;
zimg: refactor (move around fields) The intention is to add slice-threading to the wrapper. For that purpose, move all zimg related state to a new struct mp_zimg_state. There is now an array of instances of this struct. The intention is to have one instance for each thread. As of this commit, this is hardcoded to 1 thread; the following commit extends this.
2020-07-14 18:48:48 +00:00
// Temporary memory for zimg buffer.
zimg_image_buffer zbuf;
zimg: add slice threading and use it by default This probably makes it much faster (I wouldn't know, I didn't run any benchmarks ). Seems to work as well (although I'm not sure, it's not like I'd perform rigorous tests). The scale_zimg test seems to mysteriously treat color in fully transparent alpha differently, which makes no sense, and isn't visible (but makes the test fail). I can't be bothered with investigating this more. What do you do with failing tests? Correct, you disable them. Or rather, you disable whatever appears to cause them to fail, which is the threading in this case. This change follows mostly the tile_example.cpp. The slice size uses a minimum of 64, which was suggested by the zimg author. Some of this commit is a bit inelegant and weird, such as recomputing the scale factor for every slice, or the way slice_h is managed. Too lazy to make this more elegant. zimg git had a regressio around active_region (which is needed by the slicing), which was fixed in commit 83071706b2e6bc634. Apparently, the bug was never released, so just add a warning to the manpage.
2020-07-14 20:52:27 +00:00
struct mp_image cropped_tmp;
zimg: refactor (move around fields) The intention is to add slice-threading to the wrapper. For that purpose, move all zimg related state to a new struct mp_zimg_state. There is now an array of instances of this struct. The intention is to have one instance for each thread. As of this commit, this is hardcoded to 1 thread; the following commit extends this.
2020-07-14 18:48:48 +00:00
zimg: correct output to odd (chroma un-aligned) sizes As suggested by the zimg author: active_region is not supported on outputs (and the API returns an error), so instead scale to the "full" surface, but adjust the source rectangle such that the cropped output image happens to cover the correct region. Does this even work? Since Balmer Peak doesn't work, I can't really say, but it seems to look correct.
2020-02-12 17:01:22 +00:00
int real_w, real_h; // aligned size
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
};
sws_utils: shuffle around some shit Purpose uncertain. I guess it's slightly better, maybe. The move of the sws/zimg options from VO opts (vo_opt_list) to the top-level option list is tricky. VO opts have some helper code in vo.c, that sends VOCTRL_SET_PANSCAN to the VO on every VO opts change. That's because updating certain VO options used to be this way (and not just the panscan option). This isn't needed anymore for sws/zimg options, so explicitly move them away.
2019-10-31 14:18:57 +00:00
static void mp_zimg_update_from_cmdline(struct mp_zimg_context *ctx)
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
{
sws_utils: shuffle around some shit Purpose uncertain. I guess it's slightly better, maybe. The move of the sws/zimg options from VO opts (vo_opt_list) to the top-level option list is tricky. VO opts have some helper code in vo.c, that sends VOCTRL_SET_PANSCAN to the VO on every VO opts change. That's because updating certain VO options used to be this way (and not just the panscan option). This isn't needed anymore for sws/zimg options, so explicitly move them away.
2019-10-31 14:18:57 +00:00
m_config_cache_update(ctx->opts_cache);
struct zimg_opts *opts = ctx->opts_cache->opts;
sws_utils, zimg: destroy vo_x11 and vo_drm performance Raise swscale and zimg default parameters. This restores screenshot quality settings (maybe) unset in the commit before. Also expose some more libswscale and zimg options. Since these options are also used for VOs like x11 and drm, this will make x11/drm/etc. much slower. For compensation, provide a profile that sets the old option values: sw-fast. I'm also enabling zimg here, just as an experiment. The core problem is that we have a single set of command line options which control the settings used for most swscale/zimg uses. This was done in the previous commit. It cannot differentiate between the VOs, which need to be realtime and may accept/require lower quality options, and things like screenshots or vo_image, which can be slower, but should not sacrifice quality by default. Should this have two sets of options or something similar to do the right thing depending on the code which calls libswscale? Maybe. Or should I just ignore the problem, make it someone else's problem (users who want to use software conversion VOs), provide a sub-optimal solution, and call it a day? Definitely, sounds good, pushing to master, goodbye.
2019-10-31 15:45:28 +00:00
ctx->opts = *opts;
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
}
static zimg_chroma_location_e mp_to_z_chroma(enum mp_chroma_location cl)
{
switch (cl) {
csputils: add MP_CHROMA_TOPLEFT This is commonly used by UHD/HDR sources, and mpv hilariously ignores it up until now, just blindly mapping it to MP_CHROMA_AUTO without even so much as a warning message. It would be justified to add all the other chroma locations as well, but I'm lazy and just wanted to quickly fix this bug.
2020-12-02 00:34:42 +00:00
case MP_CHROMA_TOPLEFT: return ZIMG_CHROMA_TOP_LEFT;
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
case MP_CHROMA_LEFT: return ZIMG_CHROMA_LEFT;
case MP_CHROMA_CENTER: return ZIMG_CHROMA_CENTER;
default: return ZIMG_CHROMA_LEFT;
}
}
static zimg_matrix_coefficients_e mp_to_z_matrix(enum mp_csp csp)
{
switch (csp) {
case MP_CSP_BT_601: return ZIMG_MATRIX_BT470_BG;
case MP_CSP_BT_709: return ZIMG_MATRIX_BT709;
case MP_CSP_SMPTE_240M: return ZIMG_MATRIX_ST240_M;
case MP_CSP_BT_2020_NC: return ZIMG_MATRIX_BT2020_NCL;
case MP_CSP_BT_2020_C: return ZIMG_MATRIX_BT2020_CL;
case MP_CSP_RGB: return ZIMG_MATRIX_RGB;
zimg: add some more colorspace mappings As suggested by the zimg author. This is mostly related to XYZ support. It's unclear whether this works. Using the only XYZ test sample we know, and the next commits to consume the pixfmt, it looks wrong.
2019-10-20 13:26:18 +00:00
case MP_CSP_XYZ: return ZIMG_MATRIX_RGB;
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
case MP_CSP_YCGCO: return ZIMG_MATRIX_YCGCO;
default: return ZIMG_MATRIX_BT709;
}
}
static zimg_transfer_characteristics_e mp_to_z_trc(enum mp_csp_trc trc)
{
switch (trc) {
case MP_CSP_TRC_BT_1886: return ZIMG_TRANSFER_BT709;
case MP_CSP_TRC_SRGB: return ZIMG_TRANSFER_IEC_61966_2_1;
case MP_CSP_TRC_LINEAR: return ZIMG_TRANSFER_LINEAR;
case MP_CSP_TRC_GAMMA22: return ZIMG_TRANSFER_BT470_M;
case MP_CSP_TRC_GAMMA28: return ZIMG_TRANSFER_BT470_BG;
case MP_CSP_TRC_PQ: return ZIMG_TRANSFER_ST2084;
case MP_CSP_TRC_HLG: return ZIMG_TRANSFER_ARIB_B67;
case MP_CSP_TRC_GAMMA18: // ?
case MP_CSP_TRC_GAMMA20:
case MP_CSP_TRC_GAMMA24:
case MP_CSP_TRC_GAMMA26:
case MP_CSP_TRC_PRO_PHOTO:
case MP_CSP_TRC_V_LOG:
case MP_CSP_TRC_S_LOG1:
case MP_CSP_TRC_S_LOG2: // ?
default: return ZIMG_TRANSFER_BT709;
}
}
static zimg_color_primaries_e mp_to_z_prim(enum mp_csp_prim prim)
{
switch (prim) {
case MP_CSP_PRIM_BT_601_525:return ZIMG_PRIMARIES_ST170_M;
case MP_CSP_PRIM_BT_601_625:return ZIMG_PRIMARIES_BT470_BG;
case MP_CSP_PRIM_BT_709: return ZIMG_PRIMARIES_BT709;
case MP_CSP_PRIM_BT_2020: return ZIMG_PRIMARIES_BT2020;
case MP_CSP_PRIM_BT_470M: return ZIMG_PRIMARIES_BT470_M;
zimg: add some more colorspace mappings As suggested by the zimg author. This is mostly related to XYZ support. It's unclear whether this works. Using the only XYZ test sample we know, and the next commits to consume the pixfmt, it looks wrong.
2019-10-20 13:26:18 +00:00
case MP_CSP_PRIM_CIE_1931: return ZIMG_PRIMARIES_ST428;
case MP_CSP_PRIM_DCI_P3: return ZIMG_PRIMARIES_ST431_2;
case MP_CSP_PRIM_DISPLAY_P3:return ZIMG_PRIMARIES_ST432_1;
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
case MP_CSP_PRIM_APPLE: // ?
case MP_CSP_PRIM_ADOBE:
case MP_CSP_PRIM_PRO_PHOTO:
case MP_CSP_PRIM_V_GAMUT:
case MP_CSP_PRIM_S_GAMUT: // ?
default: return ZIMG_PRIMARIES_BT709;
}
}
static void destroy_zimg(struct mp_zimg_context *ctx)
{
zimg: refactor (move around fields) The intention is to add slice-threading to the wrapper. For that purpose, move all zimg related state to a new struct mp_zimg_state. There is now an array of instances of this struct. The intention is to have one instance for each thread. As of this commit, this is hardcoded to 1 thread; the following commit extends this.
2020-07-14 18:48:48 +00:00
for (int n = 0; n < ctx->num_states; n++) {
struct mp_zimg_state *st = ctx->states[n];
talloc_free(st->tmp_alloc);
zimg_filter_graph_free(st->graph);
TA_FREEP(&st->src);
TA_FREEP(&st->dst);
talloc_free(st);
}
ctx->num_states = 0;
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
}
static void free_mp_zimg(void *p)
{
struct mp_zimg_context *ctx = p;
destroy_zimg(ctx);
zimg: add slice threading and use it by default This probably makes it much faster (I wouldn't know, I didn't run any benchmarks ). Seems to work as well (although I'm not sure, it's not like I'd perform rigorous tests). The scale_zimg test seems to mysteriously treat color in fully transparent alpha differently, which makes no sense, and isn't visible (but makes the test fail). I can't be bothered with investigating this more. What do you do with failing tests? Correct, you disable them. Or rather, you disable whatever appears to cause them to fail, which is the threading in this case. This change follows mostly the tile_example.cpp. The slice size uses a minimum of 64, which was suggested by the zimg author. Some of this commit is a bit inelegant and weird, such as recomputing the scale factor for every slice, or the way slice_h is managed. Too lazy to make this more elegant. zimg git had a regressio around active_region (which is needed by the slicing), which was fixed in commit 83071706b2e6bc634. Apparently, the bug was never released, so just add a warning to the manpage.
2020-07-14 20:52:27 +00:00
TA_FREEP(&ctx->tp);
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
}
struct mp_zimg_context *mp_zimg_alloc(void)
{
struct mp_zimg_context *ctx = talloc_ptrtype(NULL, ctx);
*ctx = (struct mp_zimg_context) {
.log = mp_null_log,
};
sws_utils, zimg: destroy vo_x11 and vo_drm performance Raise swscale and zimg default parameters. This restores screenshot quality settings (maybe) unset in the commit before. Also expose some more libswscale and zimg options. Since these options are also used for VOs like x11 and drm, this will make x11/drm/etc. much slower. For compensation, provide a profile that sets the old option values: sw-fast. I'm also enabling zimg here, just as an experiment. The core problem is that we have a single set of command line options which control the settings used for most swscale/zimg uses. This was done in the previous commit. It cannot differentiate between the VOs, which need to be realtime and may accept/require lower quality options, and things like screenshots or vo_image, which can be slower, but should not sacrifice quality by default. Should this have two sets of options or something similar to do the right thing depending on the code which calls libswscale? Maybe. Or should I just ignore the problem, make it someone else's problem (users who want to use software conversion VOs), provide a sub-optimal solution, and call it a day? Definitely, sounds good, pushing to master, goodbye.
2019-10-31 15:45:28 +00:00
ctx->opts = *(struct zimg_opts *)zimg_conf.defaults;
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
talloc_set_destructor(ctx, free_mp_zimg);
return ctx;
}
sws_utils: shuffle around some shit Purpose uncertain. I guess it's slightly better, maybe. The move of the sws/zimg options from VO opts (vo_opt_list) to the top-level option list is tricky. VO opts have some helper code in vo.c, that sends VOCTRL_SET_PANSCAN to the VO on every VO opts change. That's because updating certain VO options used to be this way (and not just the panscan option). This isn't needed anymore for sws/zimg options, so explicitly move them away.
2019-10-31 14:18:57 +00:00
void mp_zimg_enable_cmdline_opts(struct mp_zimg_context *ctx,
struct mpv_global *g)
{
if (ctx->opts_cache)
return;
ctx->opts_cache = m_config_cache_alloc(ctx, g, &zimg_conf);
destroy_zimg(ctx); // force update
mp_zimg_update_from_cmdline(ctx); // first update
}
zimg: add support for big endian input and output One of the extremely annoying dumb things in ffmpeg is that most pixel formats are available as little endian and big endian variants. (The sane way would be having native endian formats only.) Usually, most of the real codecs use native formats only, while non-native formats are used by fringe raw codecs only. But the PNG encoders and decoders unfortunately use big endian formats, and since PNG it such a popular format, this causes problems for us. In particular, the current zimg wrapper will refuse to work (and mpv will fall back to sws) when writing non-8 bit PNGs. So add non-native endian support to zimg. This is done in a fairly "generic" way (which means lots of potential for bugs). If input is a "regular" format (and just byte-swapped), the rest happens automatically, which happens to cover all interesting formats. Some things could be more efficient; for example, unpacking is done on the data before it's passed to the unpacker. You could make endian swapping part of the actual unpacking process, which might be slightly faster. You could avoid copying twice in some cases (such as when there's no actual repacker, or if alignment needs to be corrected). But I don't really care. It's reasonably fast for the normal case. Not entirely sure whether this is correct. Some (but not many) formats are covered by the tests, some I tested manually. Some I can't even test, because libswscale doesn't support them (like nv20*).
2020-04-12 20:14:55 +00:00
static int repack_entrypoint(void *user, unsigned i, unsigned x0, unsigned x1)
{
struct mp_zimg_repack *r = user;
video: separate repacking code from zimg and make it independent For whatever purpose. If anything, this makes the zimg wrapper cleaner. The added tests are not particular exhaustive, but nice to have. This also makes the scale_zimg.c test pretty useless, because it only tests repacking (going through the zimg wrapper). In theory, the repack_tests things could also be used on scalers, but I guess it doesn't matter. Some things are added over the previous zimg wrapper code. For example, some fringe formats can now be expanded to 8 bit per component for convenience.
2020-05-09 15:56:44 +00:00
// If reading is not aligned, just read slightly more data.
if (!r->pack)
x0 &= ~(unsigned)(mp_repack_get_align_x(r->repack) - 1);
zimg: add support for big endian input and output One of the extremely annoying dumb things in ffmpeg is that most pixel formats are available as little endian and big endian variants. (The sane way would be having native endian formats only.) Usually, most of the real codecs use native formats only, while non-native formats are used by fringe raw codecs only. But the PNG encoders and decoders unfortunately use big endian formats, and since PNG it such a popular format, this causes problems for us. In particular, the current zimg wrapper will refuse to work (and mpv will fall back to sws) when writing non-8 bit PNGs. So add non-native endian support to zimg. This is done in a fairly "generic" way (which means lots of potential for bugs). If input is a "regular" format (and just byte-swapped), the rest happens automatically, which happens to cover all interesting formats. Some things could be more efficient; for example, unpacking is done on the data before it's passed to the unpacker. You could make endian swapping part of the actual unpacking process, which might be slightly faster. You could avoid copying twice in some cases (such as when there's no actual repacker, or if alignment needs to be corrected). But I don't really care. It's reasonably fast for the normal case. Not entirely sure whether this is correct. Some (but not many) formats are covered by the tests, some I tested manually. Some I can't even test, because libswscale doesn't support them (like nv20*).
2020-04-12 20:14:55 +00:00
video: separate repacking code from zimg and make it independent For whatever purpose. If anything, this makes the zimg wrapper cleaner. The added tests are not particular exhaustive, but nice to have. This also makes the scale_zimg.c test pretty useless, because it only tests repacking (going through the zimg wrapper). In theory, the repack_tests things could also be used on scalers, but I guess it doesn't matter. Some things are added over the previous zimg wrapper code. For example, some fringe formats can now be expanded to 8 bit per component for convenience.
2020-05-09 15:56:44 +00:00
// mp_repack requirements and zimg guarantees.
assert(!(i & (mp_repack_get_align_y(r->repack) - 1)));
assert(!(x0 & (mp_repack_get_align_x(r->repack) - 1)));
unsigned i_src = i & (r->pack ? r->zmask[0] : ZIMG_BUFFER_MAX);
unsigned i_dst = i & (r->pack ? ZIMG_BUFFER_MAX : r->zmask[0]);
zimg: add support for big endian input and output One of the extremely annoying dumb things in ffmpeg is that most pixel formats are available as little endian and big endian variants. (The sane way would be having native endian formats only.) Usually, most of the real codecs use native formats only, while non-native formats are used by fringe raw codecs only. But the PNG encoders and decoders unfortunately use big endian formats, and since PNG it such a popular format, this causes problems for us. In particular, the current zimg wrapper will refuse to work (and mpv will fall back to sws) when writing non-8 bit PNGs. So add non-native endian support to zimg. This is done in a fairly "generic" way (which means lots of potential for bugs). If input is a "regular" format (and just byte-swapped), the rest happens automatically, which happens to cover all interesting formats. Some things could be more efficient; for example, unpacking is done on the data before it's passed to the unpacker. You could make endian swapping part of the actual unpacking process, which might be slightly faster. You could avoid copying twice in some cases (such as when there's no actual repacker, or if alignment needs to be corrected). But I don't really care. It's reasonably fast for the normal case. Not entirely sure whether this is correct. Some (but not many) formats are covered by the tests, some I tested manually. Some I can't even test, because libswscale doesn't support them (like nv20*).
2020-04-12 20:14:55 +00:00
video: separate repacking code from zimg and make it independent For whatever purpose. If anything, this makes the zimg wrapper cleaner. The added tests are not particular exhaustive, but nice to have. This also makes the scale_zimg.c test pretty useless, because it only tests repacking (going through the zimg wrapper). In theory, the repack_tests things could also be used on scalers, but I guess it doesn't matter. Some things are added over the previous zimg wrapper code. For example, some fringe formats can now be expanded to 8 bit per component for convenience.
2020-05-09 15:56:44 +00:00
repack_line(r->repack, x0, i_dst, x0, i_src, x1 - x0);
zimg: add support for big endian input and output One of the extremely annoying dumb things in ffmpeg is that most pixel formats are available as little endian and big endian variants. (The sane way would be having native endian formats only.) Usually, most of the real codecs use native formats only, while non-native formats are used by fringe raw codecs only. But the PNG encoders and decoders unfortunately use big endian formats, and since PNG it such a popular format, this causes problems for us. In particular, the current zimg wrapper will refuse to work (and mpv will fall back to sws) when writing non-8 bit PNGs. So add non-native endian support to zimg. This is done in a fairly "generic" way (which means lots of potential for bugs). If input is a "regular" format (and just byte-swapped), the rest happens automatically, which happens to cover all interesting formats. Some things could be more efficient; for example, unpacking is done on the data before it's passed to the unpacker. You could make endian swapping part of the actual unpacking process, which might be slightly faster. You could avoid copying twice in some cases (such as when there's no actual repacker, or if alignment needs to be corrected). But I don't really care. It's reasonably fast for the normal case. Not entirely sure whether this is correct. Some (but not many) formats are covered by the tests, some I tested manually. Some I can't even test, because libswscale doesn't support them (like nv20*).
2020-04-12 20:14:55 +00:00
return 0;
}
zimg: add slice threading and use it by default This probably makes it much faster (I wouldn't know, I didn't run any benchmarks ). Seems to work as well (although I'm not sure, it's not like I'd perform rigorous tests). The scale_zimg test seems to mysteriously treat color in fully transparent alpha differently, which makes no sense, and isn't visible (but makes the test fail). I can't be bothered with investigating this more. What do you do with failing tests? Correct, you disable them. Or rather, you disable whatever appears to cause them to fail, which is the threading in this case. This change follows mostly the tile_example.cpp. The slice size uses a minimum of 64, which was suggested by the zimg author. Some of this commit is a bit inelegant and weird, such as recomputing the scale factor for every slice, or the way slice_h is managed. Too lazy to make this more elegant. zimg git had a regressio around active_region (which is needed by the slicing), which was fixed in commit 83071706b2e6bc634. Apparently, the bug was never released, so just add a warning to the manpage.
2020-07-14 20:52:27 +00:00
static bool wrap_buffer(struct mp_zimg_state *st, struct mp_zimg_repack *r,
struct mp_image *a_mpi)
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
{
zimg: refactor (move around fields) The intention is to add slice-threading to the wrapper. For that purpose, move all zimg related state to a new struct mp_zimg_state. There is now an array of instances of this struct. The intention is to have one instance for each thread. As of this commit, this is hardcoded to 1 thread; the following commit extends this.
2020-07-14 18:48:48 +00:00
zimg_image_buffer *buf = &r->zbuf;
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
*buf = (zimg_image_buffer){ZIMG_API_VERSION};
zimg: add slice threading and use it by default This probably makes it much faster (I wouldn't know, I didn't run any benchmarks ). Seems to work as well (although I'm not sure, it's not like I'd perform rigorous tests). The scale_zimg test seems to mysteriously treat color in fully transparent alpha differently, which makes no sense, and isn't visible (but makes the test fail). I can't be bothered with investigating this more. What do you do with failing tests? Correct, you disable them. Or rather, you disable whatever appears to cause them to fail, which is the threading in this case. This change follows mostly the tile_example.cpp. The slice size uses a minimum of 64, which was suggested by the zimg author. Some of this commit is a bit inelegant and weird, such as recomputing the scale factor for every slice, or the way slice_h is managed. Too lazy to make this more elegant. zimg git had a regressio around active_region (which is needed by the slicing), which was fixed in commit 83071706b2e6bc634. Apparently, the bug was never released, so just add a warning to the manpage.
2020-07-14 20:52:27 +00:00
struct mp_image *mpi = a_mpi;
if (r->pack) {
mpi = &r->cropped_tmp;
*mpi = *a_mpi;
mp_image_crop(mpi, 0, st->slice_y, mpi->w, st->slice_y + st->slice_h);
}
video: separate repacking code from zimg and make it independent For whatever purpose. If anything, this makes the zimg wrapper cleaner. The added tests are not particular exhaustive, but nice to have. This also makes the scale_zimg.c test pretty useless, because it only tests repacking (going through the zimg wrapper). In theory, the repack_tests things could also be used on scalers, but I guess it doesn't matter. Some things are added over the previous zimg wrapper code. For example, some fringe formats can now be expanded to 8 bit per component for convenience.
2020-05-09 15:56:44 +00:00
bool direct[MP_MAX_PLANES] = {0};
for (int p = 0; p < mpi->num_planes; p++) {
// If alignment is good, try to avoid copy.
direct[p] = !((uintptr_t)mpi->planes[p] % ZIMG_ALIGN) &&
!(mpi->stride[p] % ZIMG_ALIGN);
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
}
video: separate repacking code from zimg and make it independent For whatever purpose. If anything, this makes the zimg wrapper cleaner. The added tests are not particular exhaustive, but nice to have. This also makes the scale_zimg.c test pretty useless, because it only tests repacking (going through the zimg wrapper). In theory, the repack_tests things could also be used on scalers, but I guess it doesn't matter. Some things are added over the previous zimg wrapper code. For example, some fringe formats can now be expanded to 8 bit per component for convenience.
2020-05-09 15:56:44 +00:00
if (!repack_config_buffers(r->repack, 0, r->pack ? mpi : r->tmp,
0, r->pack ? r->tmp : mpi, direct))
return false;
zimg: support gray/alpha conversion The special thing about this format is 1. mpv assigns the component ID 4 to alpha, and component IDs 2 and 3 are not present, which causes some messy details. 2. zimg always wants the alpha plane as plane 3, and plane 1 and 2 are not present, while FFmpeg/mpv put the alpha plane as plane 1. In theory, 2. could be avoided, since FFmpeg actually doesn't have a any 2 plane formats (alpha is either packed, or plane 3). But having to skip "empty" planes would break expectations. zplanes is not equivalent to the mpv plane count (actually it was always used this way), while zimg does not really have a plane count, but does, in this case, only use plane 0 and 3, while 2 and 3 are unused and unset. z_planes[] (not zplanes) is now always valid for all 4 array entries (because it uses zimg indexes), but a -1 entry means it's an unused plane. I wonder if these conventions taken by mpv/zimg are not just causing extra work. Maybe component IDs should just be indexes by the "natural" order (e.g. R-G-B-A, Y-U-V-A, Y-A), and alpha should be represented as a field that specifies the component ID for it, or just strictly assume that 2/4 component formats always use the last component for alpha.
2020-02-10 16:57:01 +00:00
for (int n = 0; n < MP_ARRAY_SIZE(buf->plane); n++) {
zimg: fix some confusion about plane permutation We reorder the planes between mpv and zimg conventions. It turns out the code still confused when which convention was used. So the way it actually works is that the _only_ place where zimg order is used is the zimg_image_buffer.plane[] array. plane_aligned[] and zmask[] were accessed incorrectly, although I guess it rarely had a reason to fail (plane reordering is mostly for RGB, which has planes of all the same size). Adjust some comments accordingly too.
2020-02-10 16:45:20 +00:00
// Note: this is really the only place we have to care about plane
// permutation (zimg_image_buffer may have a different plane order
// than the shadow mpi like r->tmp). We never use the zimg indexes
// in other places.
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
int mplane = r->z_planes[n];
zimg: support gray/alpha conversion The special thing about this format is 1. mpv assigns the component ID 4 to alpha, and component IDs 2 and 3 are not present, which causes some messy details. 2. zimg always wants the alpha plane as plane 3, and plane 1 and 2 are not present, while FFmpeg/mpv put the alpha plane as plane 1. In theory, 2. could be avoided, since FFmpeg actually doesn't have a any 2 plane formats (alpha is either packed, or plane 3). But having to skip "empty" planes would break expectations. zplanes is not equivalent to the mpv plane count (actually it was always used this way), while zimg does not really have a plane count, but does, in this case, only use plane 0 and 3, while 2 and 3 are unused and unset. z_planes[] (not zplanes) is now always valid for all 4 array entries (because it uses zimg indexes), but a -1 entry means it's an unused plane. I wonder if these conventions taken by mpv/zimg are not just causing extra work. Maybe component IDs should just be indexes by the "natural" order (e.g. R-G-B-A, Y-U-V-A, Y-A), and alpha should be represented as a field that specifies the component ID for it, or just strictly assume that 2/4 component formats always use the last component for alpha.
2020-02-10 16:57:01 +00:00
if (mplane < 0)
continue;
zimg: pass through Y plane when repacking nv12 Normally, the Y plane can just be passed directly to zimg, and only the chroma plane needs to be (de)interleaved. It still needs a copy if the Y pointer is not aligned, though. (Whether this is actually a problem depends on the CPU and probably zimg's compiler.) This requires deciding per plane whether the plane should go through the repack buffer or not. This logic is active in non-nv12 cases, because not doing so would require extra code (maybe 2 lines or so). repack_align is now always called, even if it's planar->planar with all input aligned, but it won't actually do anything in that case. The assumption is that zimg won't change behavior if you pass a callback that does nothing versus passing NULL as callback.
2019-11-02 01:18:24 +00:00
video: separate repacking code from zimg and make it independent For whatever purpose. If anything, this makes the zimg wrapper cleaner. The added tests are not particular exhaustive, but nice to have. This also makes the scale_zimg.c test pretty useless, because it only tests repacking (going through the zimg wrapper). In theory, the repack_tests things could also be used on scalers, but I guess it doesn't matter. Some things are added over the previous zimg wrapper code. For example, some fringe formats can now be expanded to 8 bit per component for convenience.
2020-05-09 15:56:44 +00:00
struct mp_image *tmpi = direct[mplane] ? mpi : r->tmp;
zimg: pass through Y plane when repacking nv12 Normally, the Y plane can just be passed directly to zimg, and only the chroma plane needs to be (de)interleaved. It still needs a copy if the Y pointer is not aligned, though. (Whether this is actually a problem depends on the CPU and probably zimg's compiler.) This requires deciding per plane whether the plane should go through the repack buffer or not. This logic is active in non-nv12 cases, because not doing so would require extra code (maybe 2 lines or so). repack_align is now always called, even if it's planar->planar with all input aligned, but it won't actually do anything in that case. The assumption is that zimg won't change behavior if you pass a callback that does nothing versus passing NULL as callback.
2019-11-02 01:18:24 +00:00
buf->plane[n].data = tmpi->planes[mplane];
buf->plane[n].stride = tmpi->stride[mplane];
video: separate repacking code from zimg and make it independent For whatever purpose. If anything, this makes the zimg wrapper cleaner. The added tests are not particular exhaustive, but nice to have. This also makes the scale_zimg.c test pretty useless, because it only tests repacking (going through the zimg wrapper). In theory, the repack_tests things could also be used on scalers, but I guess it doesn't matter. Some things are added over the previous zimg wrapper code. For example, some fringe formats can now be expanded to 8 bit per component for convenience.
2020-05-09 15:56:44 +00:00
buf->plane[n].mask = direct[mplane] ? ZIMG_BUFFER_MAX : r->zmask[mplane];
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
}
video: separate repacking code from zimg and make it independent For whatever purpose. If anything, this makes the zimg wrapper cleaner. The added tests are not particular exhaustive, but nice to have. This also makes the scale_zimg.c test pretty useless, because it only tests repacking (going through the zimg wrapper). In theory, the repack_tests things could also be used on scalers, but I guess it doesn't matter. Some things are added over the previous zimg wrapper code. For example, some fringe formats can now be expanded to 8 bit per component for convenience.
2020-05-09 15:56:44 +00:00
return true;
zimg: slightly cleanup some mpv format handling nonsense Move lookup GBRP or planar gray/alpha formats to separate functions in some cases. Make setup_regular_rgb_packer() not use a 4:4:4 YUV format to "pass" RGB. This was used as a "trick" to avoid the stupid GBRP plane permutation, but it confused severely, so get rid of it. Just do the reordering, even if the zimg wrapper itself will reorder it back (which is so stupid that I used the other approach at first). The comment saying IMGFMT_420P was bogus of course; typically it was IMGFMT_444P.
2020-04-23 11:10:41 +00:00
}
zimg: refactor (move around fields) The intention is to add slice-threading to the wrapper. For that purpose, move all zimg related state to a new struct mp_zimg_state. There is now an array of instances of this struct. The intention is to have one instance for each thread. As of this commit, this is hardcoded to 1 thread; the following commit extends this.
2020-07-14 18:48:48 +00:00
// (ctx and st can be NULL for probing.)
video: separate repacking code from zimg and make it independent For whatever purpose. If anything, this makes the zimg wrapper cleaner. The added tests are not particular exhaustive, but nice to have. This also makes the scale_zimg.c test pretty useless, because it only tests repacking (going through the zimg wrapper). In theory, the repack_tests things could also be used on scalers, but I guess it doesn't matter. Some things are added over the previous zimg wrapper code. For example, some fringe formats can now be expanded to 8 bit per component for convenience.
2020-05-09 15:56:44 +00:00
static bool setup_format(zimg_image_format *zfmt, struct mp_zimg_repack *r,
bool pack, struct mp_image_params *user_fmt,
zimg: refactor (move around fields) The intention is to add slice-threading to the wrapper. For that purpose, move all zimg related state to a new struct mp_zimg_state. There is now an array of instances of this struct. The intention is to have one instance for each thread. As of this commit, this is hardcoded to 1 thread; the following commit extends this.
2020-07-14 18:48:48 +00:00
struct mp_zimg_context *ctx,
struct mp_zimg_state *st)
zimg: add support for some RGB fringe formats This covers 8 and 16 bit packed RGB formats. It doesn't really help with any actual use-cases, other than giving the finger to libswscale. One problem is with different color depths. For example, rgb565 provides 1 bit more resolution to the green channel. zimg can only dither to a uniform depth. I tried dithering to the highest depth and shifting away 1 bit for the lower channels, but that looked ugly (or I messed up somewhere), so instead it dithers to the lowest depth, and adjusts the value range if needed. Testing with bgr4_byte (extreme case with 1/2/1 depths), it looks more "grainy" (ordered dithering artifacts) than libswscale, but it also looks cleaner and smoother. It doesn't have libswscale's weird red-shift. So I call it a success. Big endian formats need to be handled explicitly; the generic big endian swapper code assumes byte-aligned components. Unpacking is done with shifts and 3 LUTs. This is symmetric to the packer. Using a generated palette might be better, but I preferred to keep the symmetry, and not having to mess with a generated palette and the pal8 code. This uses FFmepg pixfmts constants directly. I would have preferred keeping zimg completely separate. But neither do I want to add an IMGFMT alias for every of these formats, nor do I want to extend our imgfmt code such that it can provide a complete description of each packed RGB format (similar to FFmpeg pixdesc). It also appears that FFmpeg pixdesc as well as the FFmpeg pixfmt doxygen have an error regarding RGB8: the R/B bit depths are swapped. libswscale appears to be handling them differently. Not completely sure, as this is the only packed format case with R/B havuing different depths (instead of G, the middle component, where things are symmetric).
2020-04-13 00:36:46 +00:00
{
video: separate repacking code from zimg and make it independent For whatever purpose. If anything, this makes the zimg wrapper cleaner. The added tests are not particular exhaustive, but nice to have. This also makes the scale_zimg.c test pretty useless, because it only tests repacking (going through the zimg wrapper). In theory, the repack_tests things could also be used on scalers, but I guess it doesn't matter. Some things are added over the previous zimg wrapper code. For example, some fringe formats can now be expanded to 8 bit per component for convenience.
2020-05-09 15:56:44 +00:00
r->fmt = *user_fmt;
r->pack = pack;
zimg: add support for some RGB fringe formats This covers 8 and 16 bit packed RGB formats. It doesn't really help with any actual use-cases, other than giving the finger to libswscale. One problem is with different color depths. For example, rgb565 provides 1 bit more resolution to the green channel. zimg can only dither to a uniform depth. I tried dithering to the highest depth and shifting away 1 bit for the lower channels, but that looked ugly (or I messed up somewhere), so instead it dithers to the lowest depth, and adjusts the value range if needed. Testing with bgr4_byte (extreme case with 1/2/1 depths), it looks more "grainy" (ordered dithering artifacts) than libswscale, but it also looks cleaner and smoother. It doesn't have libswscale's weird red-shift. So I call it a success. Big endian formats need to be handled explicitly; the generic big endian swapper code assumes byte-aligned components. Unpacking is done with shifts and 3 LUTs. This is symmetric to the packer. Using a generated palette might be better, but I preferred to keep the symmetry, and not having to mess with a generated palette and the pal8 code. This uses FFmepg pixfmts constants directly. I would have preferred keeping zimg completely separate. But neither do I want to add an IMGFMT alias for every of these formats, nor do I want to extend our imgfmt code such that it can provide a complete description of each packed RGB format (similar to FFmpeg pixdesc). It also appears that FFmpeg pixdesc as well as the FFmpeg pixfmt doxygen have an error regarding RGB8: the R/B bit depths are swapped. libswscale appears to be handling them differently. Not completely sure, as this is the only packed format case with R/B havuing different depths (instead of G, the middle component, where things are symmetric).
2020-04-13 00:36:46 +00:00
video: separate repacking code from zimg and make it independent For whatever purpose. If anything, this makes the zimg wrapper cleaner. The added tests are not particular exhaustive, but nice to have. This also makes the scale_zimg.c test pretty useless, because it only tests repacking (going through the zimg wrapper). In theory, the repack_tests things could also be used on scalers, but I guess it doesn't matter. Some things are added over the previous zimg wrapper code. For example, some fringe formats can now be expanded to 8 bit per component for convenience.
2020-05-09 15:56:44 +00:00
zimg_image_format_default(zfmt, ZIMG_API_VERSION);
zimg: add support for some RGB fringe formats This covers 8 and 16 bit packed RGB formats. It doesn't really help with any actual use-cases, other than giving the finger to libswscale. One problem is with different color depths. For example, rgb565 provides 1 bit more resolution to the green channel. zimg can only dither to a uniform depth. I tried dithering to the highest depth and shifting away 1 bit for the lower channels, but that looked ugly (or I messed up somewhere), so instead it dithers to the lowest depth, and adjusts the value range if needed. Testing with bgr4_byte (extreme case with 1/2/1 depths), it looks more "grainy" (ordered dithering artifacts) than libswscale, but it also looks cleaner and smoother. It doesn't have libswscale's weird red-shift. So I call it a success. Big endian formats need to be handled explicitly; the generic big endian swapper code assumes byte-aligned components. Unpacking is done with shifts and 3 LUTs. This is symmetric to the packer. Using a generated palette might be better, but I preferred to keep the symmetry, and not having to mess with a generated palette and the pal8 code. This uses FFmepg pixfmts constants directly. I would have preferred keeping zimg completely separate. But neither do I want to add an IMGFMT alias for every of these formats, nor do I want to extend our imgfmt code such that it can provide a complete description of each packed RGB format (similar to FFmpeg pixdesc). It also appears that FFmpeg pixdesc as well as the FFmpeg pixfmt doxygen have an error regarding RGB8: the R/B bit depths are swapped. libswscale appears to be handling them differently. Not completely sure, as this is the only packed format case with R/B havuing different depths (instead of G, the middle component, where things are symmetric).
2020-04-13 00:36:46 +00:00
video: separate repacking code from zimg and make it independent For whatever purpose. If anything, this makes the zimg wrapper cleaner. The added tests are not particular exhaustive, but nice to have. This also makes the scale_zimg.c test pretty useless, because it only tests repacking (going through the zimg wrapper). In theory, the repack_tests things could also be used on scalers, but I guess it doesn't matter. Some things are added over the previous zimg wrapper code. For example, some fringe formats can now be expanded to 8 bit per component for convenience.
2020-05-09 15:56:44 +00:00
int rp_flags = 0;
zimg: add support for some RGB fringe formats This covers 8 and 16 bit packed RGB formats. It doesn't really help with any actual use-cases, other than giving the finger to libswscale. One problem is with different color depths. For example, rgb565 provides 1 bit more resolution to the green channel. zimg can only dither to a uniform depth. I tried dithering to the highest depth and shifting away 1 bit for the lower channels, but that looked ugly (or I messed up somewhere), so instead it dithers to the lowest depth, and adjusts the value range if needed. Testing with bgr4_byte (extreme case with 1/2/1 depths), it looks more "grainy" (ordered dithering artifacts) than libswscale, but it also looks cleaner and smoother. It doesn't have libswscale's weird red-shift. So I call it a success. Big endian formats need to be handled explicitly; the generic big endian swapper code assumes byte-aligned components. Unpacking is done with shifts and 3 LUTs. This is symmetric to the packer. Using a generated palette might be better, but I preferred to keep the symmetry, and not having to mess with a generated palette and the pal8 code. This uses FFmepg pixfmts constants directly. I would have preferred keeping zimg completely separate. But neither do I want to add an IMGFMT alias for every of these formats, nor do I want to extend our imgfmt code such that it can provide a complete description of each packed RGB format (similar to FFmpeg pixdesc). It also appears that FFmpeg pixdesc as well as the FFmpeg pixfmt doxygen have an error regarding RGB8: the R/B bit depths are swapped. libswscale appears to be handling them differently. Not completely sure, as this is the only packed format case with R/B havuing different depths (instead of G, the middle component, where things are symmetric).
2020-04-13 00:36:46 +00:00
video: separate repacking code from zimg and make it independent For whatever purpose. If anything, this makes the zimg wrapper cleaner. The added tests are not particular exhaustive, but nice to have. This also makes the scale_zimg.c test pretty useless, because it only tests repacking (going through the zimg wrapper). In theory, the repack_tests things could also be used on scalers, but I guess it doesn't matter. Some things are added over the previous zimg wrapper code. For example, some fringe formats can now be expanded to 8 bit per component for convenience.
2020-05-09 15:56:44 +00:00
// For e.g. RGB565, go to lowest depth on pack for less weird dithering.
zimg: get rid of special "override" fields for low depth RGB/gray This makes it use the previously added fringe image formats. What is the purpose of this change? Who knows.
2020-04-23 11:20:46 +00:00
if (r->pack) {
video: separate repacking code from zimg and make it independent For whatever purpose. If anything, this makes the zimg wrapper cleaner. The added tests are not particular exhaustive, but nice to have. This also makes the scale_zimg.c test pretty useless, because it only tests repacking (going through the zimg wrapper). In theory, the repack_tests things could also be used on scalers, but I guess it doesn't matter. Some things are added over the previous zimg wrapper code. For example, some fringe formats can now be expanded to 8 bit per component for convenience.
2020-05-09 15:56:44 +00:00
rp_flags |= REPACK_CREATE_ROUND_DOWN;
zimg: add support for 1 bit per pixel formats Again worthless, slow, and only for libswscale parity. With this, we support all formats libswscale supports, except bayer input, and rgb4/bgr4 output. We even support some formats libswscale doesn't. It's possible that the zimg wrapper isn't always as fast as libswscale. But there is optimization potential: the inner repack loops are self-contained enough that they could be reasonably be implemented in assembler (probably), and doing everything slice-wise should reduce the overhead of the separate pack/unpack stages.
2020-04-13 18:42:34 +00:00
} else {
video: separate repacking code from zimg and make it independent For whatever purpose. If anything, this makes the zimg wrapper cleaner. The added tests are not particular exhaustive, but nice to have. This also makes the scale_zimg.c test pretty useless, because it only tests repacking (going through the zimg wrapper). In theory, the repack_tests things could also be used on scalers, but I guess it doesn't matter. Some things are added over the previous zimg wrapper code. For example, some fringe formats can now be expanded to 8 bit per component for convenience.
2020-05-09 15:56:44 +00:00
rp_flags |= REPACK_CREATE_EXPAND_8BIT;
zimg: support RGB30 output This may be used later elsewhere.
2019-10-20 17:39:59 +00:00
}
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
video: separate repacking code from zimg and make it independent For whatever purpose. If anything, this makes the zimg wrapper cleaner. The added tests are not particular exhaustive, but nice to have. This also makes the scale_zimg.c test pretty useless, because it only tests repacking (going through the zimg wrapper). In theory, the repack_tests things could also be used on scalers, but I guess it doesn't matter. Some things are added over the previous zimg wrapper code. For example, some fringe formats can now be expanded to 8 bit per component for convenience.
2020-05-09 15:56:44 +00:00
r->repack = mp_repack_create_planar(r->fmt.imgfmt, r->pack, rp_flags);
if (!r->repack)
return false;
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
video: separate repacking code from zimg and make it independent For whatever purpose. If anything, this makes the zimg wrapper cleaner. The added tests are not particular exhaustive, but nice to have. This also makes the scale_zimg.c test pretty useless, because it only tests repacking (going through the zimg wrapper). In theory, the repack_tests things could also be used on scalers, but I guess it doesn't matter. Some things are added over the previous zimg wrapper code. For example, some fringe formats can now be expanded to 8 bit per component for convenience.
2020-05-09 15:56:44 +00:00
int align_x = mp_repack_get_align_x(r->repack);
zinmg: stop using GBRP for RGB Instead, use a YUV planar format. It doesn't matter, since we use the format only internally and for "management" purposes. We're only interested in the physical layout, not what colorspace FFmpeg "forcibly" associates with it. Also get rid of using the old and slightly sketchy mp_imgfmt_find() function. Yep, the IMGFMT_RGB30 now "constructs" the planar format, instead of using a pixfmt constant. Slightly inconvenient, tricky, and fragile, but I like it, so bugger off. This whole thing gets rid of some of the strange plane permutations that were needed earlier.
2019-11-02 17:11:02 +00:00
video: separate repacking code from zimg and make it independent For whatever purpose. If anything, this makes the zimg wrapper cleaner. The added tests are not particular exhaustive, but nice to have. This also makes the scale_zimg.c test pretty useless, because it only tests repacking (going through the zimg wrapper). In theory, the repack_tests things could also be used on scalers, but I guess it doesn't matter. Some things are added over the previous zimg wrapper code. For example, some fringe formats can now be expanded to 8 bit per component for convenience.
2020-05-09 15:56:44 +00:00
r->zimgfmt = r->pack ? mp_repack_get_format_src(r->repack)
: mp_repack_get_format_dst(r->repack);
zimg: support 3 component 16 bit pixel unpacking Works for RGB (e.g. rgb48le) and XYZ. It's unsure whether XYZ is really correctly converted.
2019-10-20 14:14:36 +00:00
video: separate repacking code from zimg and make it independent For whatever purpose. If anything, this makes the zimg wrapper cleaner. The added tests are not particular exhaustive, but nice to have. This also makes the scale_zimg.c test pretty useless, because it only tests repacking (going through the zimg wrapper). In theory, the repack_tests things could also be used on scalers, but I guess it doesn't matter. Some things are added over the previous zimg wrapper code. For example, some fringe formats can now be expanded to 8 bit per component for convenience.
2020-05-09 15:56:44 +00:00
if (ctx) {
talloc_steal(r, r->repack);
} else {
TA_FREEP(&r->repack);
zimg: support 3 component 16 bit pixel unpacking Works for RGB (e.g. rgb48le) and XYZ. It's unsure whether XYZ is really correctly converted.
2019-10-20 14:14:36 +00:00
}
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
struct mp_image_params fmt = r->fmt;
mp_image_params_guess_csp(&fmt);
struct mp_regular_imgfmt desc;
if (!mp_get_regular_imgfmt(&desc, r->zimgfmt))
return false;
video: separate repacking code from zimg and make it independent For whatever purpose. If anything, this makes the zimg wrapper cleaner. The added tests are not particular exhaustive, but nice to have. This also makes the scale_zimg.c test pretty useless, because it only tests repacking (going through the zimg wrapper). In theory, the repack_tests things could also be used on scalers, but I guess it doesn't matter. Some things are added over the previous zimg wrapper code. For example, some fringe formats can now be expanded to 8 bit per component for convenience.
2020-05-09 15:56:44 +00:00
// Relies on zimg callbacks reading on 64 byte alignment.
if (!MP_IS_POWER_OF_2(align_x) || align_x > 64 / desc.component_size)
return false;
zimg: add alpha support libzimg recently added direct alpha support and new API for it. (The API change is rather minimal, and it turns out we can easily support old and new zimg versions.) This does not support _all_ alpha formats. For example, gray + alpha is not supported yet, because my stupid design in the zimg wrapper would require a planar gray + alpha format, while ffmpeg provides only a packed one.
2020-02-09 18:16:54 +00:00
// no weird stuff
video: change chroma_w/chroma_h fields to use shift instead of size When I added mp_regular_imgfmt, I made the chroma subsampling use the actual chroma division factor, instead of a shift (log2 of the actual value). I had some ideas about how this was (probably?) more intuitive and general. But nothing ever uses non-power of 2 subsampling (except jpeg in rare cases apparently, because the world is a bad place). Change the fields back to use shifts and rename them to avoid mistakes.
2020-04-21 21:11:23 +00:00
if (desc.num_planes > 4)
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
return false;
zimg: support gray/alpha conversion The special thing about this format is 1. mpv assigns the component ID 4 to alpha, and component IDs 2 and 3 are not present, which causes some messy details. 2. zimg always wants the alpha plane as plane 3, and plane 1 and 2 are not present, while FFmpeg/mpv put the alpha plane as plane 1. In theory, 2. could be avoided, since FFmpeg actually doesn't have a any 2 plane formats (alpha is either packed, or plane 3). But having to skip "empty" planes would break expectations. zplanes is not equivalent to the mpv plane count (actually it was always used this way), while zimg does not really have a plane count, but does, in this case, only use plane 0 and 3, while 2 and 3 are unused and unset. z_planes[] (not zplanes) is now always valid for all 4 array entries (because it uses zimg indexes), but a -1 entry means it's an unused plane. I wonder if these conventions taken by mpv/zimg are not just causing extra work. Maybe component IDs should just be indexes by the "natural" order (e.g. R-G-B-A, Y-U-V-A, Y-A), and alpha should be represented as a field that specifies the component ID for it, or just strictly assume that 2/4 component formats always use the last component for alpha.
2020-02-10 16:57:01 +00:00
for (int n = 0; n < 4; n++)
r->z_planes[n] = -1;
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
for (int n = 0; n < desc.num_planes; n++) {
if (desc.planes[n].num_components != 1)
return false;
int c = desc.planes[n].components[0];
zimg: add alpha support libzimg recently added direct alpha support and new API for it. (The API change is rather minimal, and it turns out we can easily support old and new zimg versions.) This does not support _all_ alpha formats. For example, gray + alpha is not supported yet, because my stupid design in the zimg wrapper would require a planar gray + alpha format, while ffmpeg provides only a packed one.
2020-02-09 18:16:54 +00:00
if (c < 1 || c > 4)
return false;
if (c < 4) {
// Unfortunately, ffmpeg prefers GBR order for planar RGB, while zimg
// is sane. This makes it necessary to determine and fix the order.
r->z_planes[c - 1] = n;
} else {
r->z_planes[3] = n; // alpha, always plane 4 in zimg
#if HAVE_ZIMG_ALPHA
video: add alpha type metadata This is mostly for testing. It adds passing through the metadata through the video chain. The metadata can be manipulated with vf_format. Support for zimg alpha conversion (if built with zimg after it gained alpha support) is implemented. Support premultiplied input in vo_gpu. Some things still seem to be buggy.
2020-04-24 12:41:50 +00:00
zfmt->alpha = fmt.alpha == MP_ALPHA_PREMUL
? ZIMG_ALPHA_PREMULTIPLIED : ZIMG_ALPHA_STRAIGHT;
zimg: add alpha support libzimg recently added direct alpha support and new API for it. (The API change is rather minimal, and it turns out we can easily support old and new zimg versions.) This does not support _all_ alpha formats. For example, gray + alpha is not supported yet, because my stupid design in the zimg wrapper would require a planar gray + alpha format, while ffmpeg provides only a packed one.
2020-02-09 18:16:54 +00:00
#else
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
return false;
zimg: add alpha support libzimg recently added direct alpha support and new API for it. (The API change is rather minimal, and it turns out we can easily support old and new zimg versions.) This does not support _all_ alpha formats. For example, gray + alpha is not supported yet, because my stupid design in the zimg wrapper would require a planar gray + alpha format, while ffmpeg provides only a packed one.
2020-02-09 18:16:54 +00:00
#endif
}
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
}
zimg: rename zplanes field This was a confusing name, because 1. there's also a z_planes[] field, and 2. it was not specific to zimg indexes. Possibly there used to be an idea involved about supporting alpha to non-alpha formats by discarding the alpha plane, but zimg does this now (and zimg will correctly blend the alpha component too).
2020-02-10 16:59:13 +00:00
r->num_planes = desc.num_planes;
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
zimg: add slice threading and use it by default This probably makes it much faster (I wouldn't know, I didn't run any benchmarks ). Seems to work as well (although I'm not sure, it's not like I'd perform rigorous tests). The scale_zimg test seems to mysteriously treat color in fully transparent alpha differently, which makes no sense, and isn't visible (but makes the test fail). I can't be bothered with investigating this more. What do you do with failing tests? Correct, you disable them. Or rather, you disable whatever appears to cause them to fail, which is the threading in this case. This change follows mostly the tile_example.cpp. The slice size uses a minimum of 64, which was suggested by the zimg author. Some of this commit is a bit inelegant and weird, such as recomputing the scale factor for every slice, or the way slice_h is managed. Too lazy to make this more elegant. zimg git had a regressio around active_region (which is needed by the slicing), which was fixed in commit 83071706b2e6bc634. Apparently, the bug was never released, so just add a warning to the manpage.
2020-07-14 20:52:27 +00:00
// Take care of input/output size, including slicing.
zimg: correct output to odd (chroma un-aligned) sizes As suggested by the zimg author: active_region is not supported on outputs (and the API returns an error), so instead scale to the "full" surface, but adjust the source rectangle such that the cropped output image happens to cover the correct region. Does this even work? Since Balmer Peak doesn't work, I can't really say, but it seems to look correct.
2020-02-12 17:01:22 +00:00
// Note: formats with subsampled chroma may have odd width or height in
// mpv and FFmpeg. This is because the width/height is actually a cropping
zimg: support subsampled chroma with non-aligned image sizes
2019-11-02 17:49:17 +00:00
// rectangle. Reconstruct the image allocation size and set the cropping.
video: change chroma_w/chroma_h fields to use shift instead of size When I added mp_regular_imgfmt, I made the chroma subsampling use the actual chroma division factor, instead of a shift (log2 of the actual value). I had some ideas about how this was (probably?) more intuitive and general. But nothing ever uses non-power of 2 subsampling (except jpeg in rare cases apparently, because the world is a bad place). Change the fields back to use shifts and rename them to avoid mistakes.
2020-04-21 21:11:23 +00:00
zfmt->width = r->real_w = MP_ALIGN_UP(fmt.w, 1 << desc.chroma_xs);
zfmt->height = r->real_h = MP_ALIGN_UP(fmt.h, 1 << desc.chroma_ys);
zimg: add slice threading and use it by default This probably makes it much faster (I wouldn't know, I didn't run any benchmarks ). Seems to work as well (although I'm not sure, it's not like I'd perform rigorous tests). The scale_zimg test seems to mysteriously treat color in fully transparent alpha differently, which makes no sense, and isn't visible (but makes the test fail). I can't be bothered with investigating this more. What do you do with failing tests? Correct, you disable them. Or rather, you disable whatever appears to cause them to fail, which is the threading in this case. This change follows mostly the tile_example.cpp. The slice size uses a minimum of 64, which was suggested by the zimg author. Some of this commit is a bit inelegant and weird, such as recomputing the scale factor for every slice, or the way slice_h is managed. Too lazy to make this more elegant. zimg git had a regressio around active_region (which is needed by the slicing), which was fixed in commit 83071706b2e6bc634. Apparently, the bug was never released, so just add a warning to the manpage.
2020-07-14 20:52:27 +00:00
if (st) {
if (r->pack) {
zfmt->height = r->real_h = st->slice_h =
MPMIN(st->slice_y + st->slice_h, r->real_h) - st->slice_y;
assert(MP_IS_ALIGNED(r->real_h, 1 << desc.chroma_ys));
} else {
// Relies on st->dst being initialized first.
struct mp_zimg_repack *dst = st->dst;
zfmt->active_region.width = dst->real_w * (double)fmt.w / dst->fmt.w;
zfmt->active_region.height = dst->real_h * st->scale_y;
zfmt->active_region.top = st->slice_y * st->scale_y;
}
zimg: correct output to odd (chroma un-aligned) sizes As suggested by the zimg author: active_region is not supported on outputs (and the API returns an error), so instead scale to the "full" surface, but adjust the source rectangle such that the cropped output image happens to cover the correct region. Does this even work? Since Balmer Peak doesn't work, I can't really say, but it seems to look correct.
2020-02-12 17:01:22 +00:00
}
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
video: change chroma_w/chroma_h fields to use shift instead of size When I added mp_regular_imgfmt, I made the chroma subsampling use the actual chroma division factor, instead of a shift (log2 of the actual value). I had some ideas about how this was (probably?) more intuitive and general. But nothing ever uses non-power of 2 subsampling (except jpeg in rare cases apparently, because the world is a bad place). Change the fields back to use shifts and rename them to avoid mistakes.
2020-04-21 21:11:23 +00:00
zfmt->subsample_w = desc.chroma_xs;
zfmt->subsample_h = desc.chroma_ys;
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
zfmt->color_family = ZIMG_COLOR_YUV;
zimg: support gray/alpha conversion The special thing about this format is 1. mpv assigns the component ID 4 to alpha, and component IDs 2 and 3 are not present, which causes some messy details. 2. zimg always wants the alpha plane as plane 3, and plane 1 and 2 are not present, while FFmpeg/mpv put the alpha plane as plane 1. In theory, 2. could be avoided, since FFmpeg actually doesn't have a any 2 plane formats (alpha is either packed, or plane 3). But having to skip "empty" planes would break expectations. zplanes is not equivalent to the mpv plane count (actually it was always used this way), while zimg does not really have a plane count, but does, in this case, only use plane 0 and 3, while 2 and 3 are unused and unset. z_planes[] (not zplanes) is now always valid for all 4 array entries (because it uses zimg indexes), but a -1 entry means it's an unused plane. I wonder if these conventions taken by mpv/zimg are not just causing extra work. Maybe component IDs should just be indexes by the "natural" order (e.g. R-G-B-A, Y-U-V-A, Y-A), and alpha should be represented as a field that specifies the component ID for it, or just strictly assume that 2/4 component formats always use the last component for alpha.
2020-02-10 16:57:01 +00:00
if (desc.num_planes <= 2) {
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
zfmt->color_family = ZIMG_COLOR_GREY;
zinmg: stop using GBRP for RGB Instead, use a YUV planar format. It doesn't matter, since we use the format only internally and for "management" purposes. We're only interested in the physical layout, not what colorspace FFmpeg "forcibly" associates with it. Also get rid of using the old and slightly sketchy mp_imgfmt_find() function. Yep, the IMGFMT_RGB30 now "constructs" the planar format, instead of using a pixfmt constant. Slightly inconvenient, tricky, and fragile, but I like it, so bugger off. This whole thing gets rid of some of the strange plane permutations that were needed earlier.
2019-11-02 17:11:02 +00:00
} else if (fmt.color.space == MP_CSP_RGB || fmt.color.space == MP_CSP_XYZ) {
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
zfmt->color_family = ZIMG_COLOR_RGB;
}
if (desc.component_type == MP_COMPONENT_TYPE_UINT &&
desc.component_size == 1)
{
zfmt->pixel_type = ZIMG_PIXEL_BYTE;
} else if (desc.component_type == MP_COMPONENT_TYPE_UINT &&
desc.component_size == 2)
{
zfmt->pixel_type = ZIMG_PIXEL_WORD;
} else if (desc.component_type == MP_COMPONENT_TYPE_FLOAT &&
desc.component_size == 2)
{
zfmt->pixel_type = ZIMG_PIXEL_HALF;
} else if (desc.component_type == MP_COMPONENT_TYPE_FLOAT &&
desc.component_size == 4)
{
zfmt->pixel_type = ZIMG_PIXEL_FLOAT;
} else {
return false;
}
// (Formats like P010 are basically reported as P016.)
zfmt->depth = desc.component_size * 8 + MPMIN(0, desc.component_pad);
zfmt->pixel_range = fmt.color.levels == MP_CSP_LEVELS_PC ?
ZIMG_RANGE_FULL : ZIMG_RANGE_LIMITED;
zfmt->matrix_coefficients = mp_to_z_matrix(fmt.color.space);
zfmt->transfer_characteristics = mp_to_z_trc(fmt.color.gamma);
zfmt->color_primaries = mp_to_z_prim(fmt.color.primaries);
zfmt->chroma_location = mp_to_z_chroma(fmt.chroma_location);
sws_utils, zimg: destroy vo_x11 and vo_drm performance Raise swscale and zimg default parameters. This restores screenshot quality settings (maybe) unset in the commit before. Also expose some more libswscale and zimg options. Since these options are also used for VOs like x11 and drm, this will make x11/drm/etc. much slower. For compensation, provide a profile that sets the old option values: sw-fast. I'm also enabling zimg here, just as an experiment. The core problem is that we have a single set of command line options which control the settings used for most swscale/zimg uses. This was done in the previous commit. It cannot differentiate between the VOs, which need to be realtime and may accept/require lower quality options, and things like screenshots or vo_image, which can be slower, but should not sacrifice quality by default. Should this have two sets of options or something similar to do the right thing depending on the code which calls libswscale? Maybe. Or should I just ignore the problem, make it someone else's problem (users who want to use software conversion VOs), provide a sub-optimal solution, and call it a day? Definitely, sounds good, pushing to master, goodbye.
2019-10-31 15:45:28 +00:00
if (ctx && ctx->opts.fast) {
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
// mpv's default for RGB output slows down zimg significantly.
if (zfmt->transfer_characteristics == ZIMG_TRANSFER_IEC_61966_2_1 &&
zfmt->color_family == ZIMG_COLOR_RGB)
zfmt->transfer_characteristics = ZIMG_TRANSFER_BT709;
}
zimg: get rid of special "override" fields for low depth RGB/gray This makes it use the previously added fringe image formats. What is the purpose of this change? Who knows.
2020-04-23 11:20:46 +00:00
// mpv treats _some_ gray formats as RGB; zimg doesn't like this.
if (zfmt->color_family == ZIMG_COLOR_GREY &&
zfmt->matrix_coefficients == ZIMG_MATRIX_RGB)
zfmt->matrix_coefficients = ZIMG_MATRIX_BT470_BG;
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
return true;
}
zimg: refactor (move around fields) The intention is to add slice-threading to the wrapper. For that purpose, move all zimg related state to a new struct mp_zimg_state. There is now an array of instances of this struct. The intention is to have one instance for each thread. As of this commit, this is hardcoded to 1 thread; the following commit extends this.
2020-07-14 18:48:48 +00:00
static bool allocate_buffer(struct mp_zimg_state *st, struct mp_zimg_repack *r)
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
{
unsigned lines = 0;
int err;
if (r->pack) {
zimg: refactor (move around fields) The intention is to add slice-threading to the wrapper. For that purpose, move all zimg related state to a new struct mp_zimg_state. There is now an array of instances of this struct. The intention is to have one instance for each thread. As of this commit, this is hardcoded to 1 thread; the following commit extends this.
2020-07-14 18:48:48 +00:00
err = zimg_filter_graph_get_output_buffering(st->graph, &lines);
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
} else {
zimg: refactor (move around fields) The intention is to add slice-threading to the wrapper. For that purpose, move all zimg related state to a new struct mp_zimg_state. There is now an array of instances of this struct. The intention is to have one instance for each thread. As of this commit, this is hardcoded to 1 thread; the following commit extends this.
2020-07-14 18:48:48 +00:00
err = zimg_filter_graph_get_input_buffering(st->graph, &lines);
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
}
if (err)
return false;
zimg: fix out of bounds memory accesses due to broken zmask We've set all planes to the same zmask. But for subsampled chroma, the zmask obviously needs to be smaller. This could lead to out of bounds memory read and write accesses. Move the align repacker to a single function, since this is now more convenient.
2019-11-01 23:48:04 +00:00
r->zmask[0] = zimg_select_buffer_mask(lines);
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
// Either ZIMG_BUFFER_MAX, or a power-of-2 slice buffer.
zimg: fix out of bounds memory accesses due to broken zmask We've set all planes to the same zmask. But for subsampled chroma, the zmask obviously needs to be smaller. This could lead to out of bounds memory read and write accesses. Move the align repacker to a single function, since this is now more convenient.
2019-11-01 23:48:04 +00:00
assert(r->zmask[0] == ZIMG_BUFFER_MAX || MP_IS_POWER_OF_2(r->zmask[0] + 1));
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
zimg: add slice threading and use it by default This probably makes it much faster (I wouldn't know, I didn't run any benchmarks ). Seems to work as well (although I'm not sure, it's not like I'd perform rigorous tests). The scale_zimg test seems to mysteriously treat color in fully transparent alpha differently, which makes no sense, and isn't visible (but makes the test fail). I can't be bothered with investigating this more. What do you do with failing tests? Correct, you disable them. Or rather, you disable whatever appears to cause them to fail, which is the threading in this case. This change follows mostly the tile_example.cpp. The slice size uses a minimum of 64, which was suggested by the zimg author. Some of this commit is a bit inelegant and weird, such as recomputing the scale factor for every slice, or the way slice_h is managed. Too lazy to make this more elegant. zimg git had a regressio around active_region (which is needed by the slicing), which was fixed in commit 83071706b2e6bc634. Apparently, the bug was never released, so just add a warning to the manpage.
2020-07-14 20:52:27 +00:00
int h = r->zmask[0] == ZIMG_BUFFER_MAX ? r->real_h : r->zmask[0] + 1;
if (h >= r->real_h) {
h = r->real_h;
zimg: fix out of bounds memory accesses due to broken zmask We've set all planes to the same zmask. But for subsampled chroma, the zmask obviously needs to be smaller. This could lead to out of bounds memory read and write accesses. Move the align repacker to a single function, since this is now more convenient.
2019-11-01 23:48:04 +00:00
r->zmask[0] = ZIMG_BUFFER_MAX;
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
}
zimg: add slice threading and use it by default This probably makes it much faster (I wouldn't know, I didn't run any benchmarks ). Seems to work as well (although I'm not sure, it's not like I'd perform rigorous tests). The scale_zimg test seems to mysteriously treat color in fully transparent alpha differently, which makes no sense, and isn't visible (but makes the test fail). I can't be bothered with investigating this more. What do you do with failing tests? Correct, you disable them. Or rather, you disable whatever appears to cause them to fail, which is the threading in this case. This change follows mostly the tile_example.cpp. The slice size uses a minimum of 64, which was suggested by the zimg author. Some of this commit is a bit inelegant and weird, such as recomputing the scale factor for every slice, or the way slice_h is managed. Too lazy to make this more elegant. zimg git had a regressio around active_region (which is needed by the slicing), which was fixed in commit 83071706b2e6bc634. Apparently, the bug was never released, so just add a warning to the manpage.
2020-07-14 20:52:27 +00:00
r->tmp = mp_image_alloc(r->zimgfmt, r->real_w, h);
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
talloc_steal(r, r->tmp);
zimg: add support for big endian input and output One of the extremely annoying dumb things in ffmpeg is that most pixel formats are available as little endian and big endian variants. (The sane way would be having native endian formats only.) Usually, most of the real codecs use native formats only, while non-native formats are used by fringe raw codecs only. But the PNG encoders and decoders unfortunately use big endian formats, and since PNG it such a popular format, this causes problems for us. In particular, the current zimg wrapper will refuse to work (and mpv will fall back to sws) when writing non-8 bit PNGs. So add non-native endian support to zimg. This is done in a fairly "generic" way (which means lots of potential for bugs). If input is a "regular" format (and just byte-swapped), the rest happens automatically, which happens to cover all interesting formats. Some things could be more efficient; for example, unpacking is done on the data before it's passed to the unpacker. You could make endian swapping part of the actual unpacking process, which might be slightly faster. You could avoid copying twice in some cases (such as when there's no actual repacker, or if alignment needs to be corrected). But I don't really care. It's reasonably fast for the normal case. Not entirely sure whether this is correct. Some (but not many) formats are covered by the tests, some I tested manually. Some I can't even test, because libswscale doesn't support them (like nv20*).
2020-04-12 20:14:55 +00:00
if (!r->tmp)
return false;
video: separate repacking code from zimg and make it independent For whatever purpose. If anything, this makes the zimg wrapper cleaner. The added tests are not particular exhaustive, but nice to have. This also makes the scale_zimg.c test pretty useless, because it only tests repacking (going through the zimg wrapper). In theory, the repack_tests things could also be used on scalers, but I guess it doesn't matter. Some things are added over the previous zimg wrapper code. For example, some fringe formats can now be expanded to 8 bit per component for convenience.
2020-05-09 15:56:44 +00:00
// Note: although zimg doesn't require that the chroma plane's zmask is
// divided by the full size zmask, the repack callback requires it,
// since mp_repack can handle only proper slices.
zimg: add support for big endian input and output One of the extremely annoying dumb things in ffmpeg is that most pixel formats are available as little endian and big endian variants. (The sane way would be having native endian formats only.) Usually, most of the real codecs use native formats only, while non-native formats are used by fringe raw codecs only. But the PNG encoders and decoders unfortunately use big endian formats, and since PNG it such a popular format, this causes problems for us. In particular, the current zimg wrapper will refuse to work (and mpv will fall back to sws) when writing non-8 bit PNGs. So add non-native endian support to zimg. This is done in a fairly "generic" way (which means lots of potential for bugs). If input is a "regular" format (and just byte-swapped), the rest happens automatically, which happens to cover all interesting formats. Some things could be more efficient; for example, unpacking is done on the data before it's passed to the unpacker. You could make endian swapping part of the actual unpacking process, which might be slightly faster. You could avoid copying twice in some cases (such as when there's no actual repacker, or if alignment needs to be corrected). But I don't really care. It's reasonably fast for the normal case. Not entirely sure whether this is correct. Some (but not many) formats are covered by the tests, some I tested manually. Some I can't even test, because libswscale doesn't support them (like nv20*).
2020-04-12 20:14:55 +00:00
for (int n = 1; n < r->tmp->fmt.num_planes; n++) {
r->zmask[n] = r->zmask[0];
if (r->zmask[0] != ZIMG_BUFFER_MAX)
r->zmask[n] = r->zmask[n] >> r->tmp->fmt.ys[n];
zimg: fix out of bounds memory accesses due to broken zmask We've set all planes to the same zmask. But for subsampled chroma, the zmask obviously needs to be smaller. This could lead to out of bounds memory read and write accesses. Move the align repacker to a single function, since this is now more convenient.
2019-11-01 23:48:04 +00:00
}
zimg: add support for big endian input and output One of the extremely annoying dumb things in ffmpeg is that most pixel formats are available as little endian and big endian variants. (The sane way would be having native endian formats only.) Usually, most of the real codecs use native formats only, while non-native formats are used by fringe raw codecs only. But the PNG encoders and decoders unfortunately use big endian formats, and since PNG it such a popular format, this causes problems for us. In particular, the current zimg wrapper will refuse to work (and mpv will fall back to sws) when writing non-8 bit PNGs. So add non-native endian support to zimg. This is done in a fairly "generic" way (which means lots of potential for bugs). If input is a "regular" format (and just byte-swapped), the rest happens automatically, which happens to cover all interesting formats. Some things could be more efficient; for example, unpacking is done on the data before it's passed to the unpacker. You could make endian swapping part of the actual unpacking process, which might be slightly faster. You could avoid copying twice in some cases (such as when there's no actual repacker, or if alignment needs to be corrected). But I don't really care. It's reasonably fast for the normal case. Not entirely sure whether this is correct. Some (but not many) formats are covered by the tests, some I tested manually. Some I can't even test, because libswscale doesn't support them (like nv20*).
2020-04-12 20:14:55 +00:00
return true;
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
}
zimg: refactor (move around fields) The intention is to add slice-threading to the wrapper. For that purpose, move all zimg related state to a new struct mp_zimg_state. There is now an array of instances of this struct. The intention is to have one instance for each thread. As of this commit, this is hardcoded to 1 thread; the following commit extends this.
2020-07-14 18:48:48 +00:00
static bool mp_zimg_state_init(struct mp_zimg_context *ctx,
zimg: add slice threading and use it by default This probably makes it much faster (I wouldn't know, I didn't run any benchmarks ). Seems to work as well (although I'm not sure, it's not like I'd perform rigorous tests). The scale_zimg test seems to mysteriously treat color in fully transparent alpha differently, which makes no sense, and isn't visible (but makes the test fail). I can't be bothered with investigating this more. What do you do with failing tests? Correct, you disable them. Or rather, you disable whatever appears to cause them to fail, which is the threading in this case. This change follows mostly the tile_example.cpp. The slice size uses a minimum of 64, which was suggested by the zimg author. Some of this commit is a bit inelegant and weird, such as recomputing the scale factor for every slice, or the way slice_h is managed. Too lazy to make this more elegant. zimg git had a regressio around active_region (which is needed by the slicing), which was fixed in commit 83071706b2e6bc634. Apparently, the bug was never released, so just add a warning to the manpage.
2020-07-14 20:52:27 +00:00
struct mp_zimg_state *st,
int slice_y, int slice_h)
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
{
sws_utils, zimg: destroy vo_x11 and vo_drm performance Raise swscale and zimg default parameters. This restores screenshot quality settings (maybe) unset in the commit before. Also expose some more libswscale and zimg options. Since these options are also used for VOs like x11 and drm, this will make x11/drm/etc. much slower. For compensation, provide a profile that sets the old option values: sw-fast. I'm also enabling zimg here, just as an experiment. The core problem is that we have a single set of command line options which control the settings used for most swscale/zimg uses. This was done in the previous commit. It cannot differentiate between the VOs, which need to be realtime and may accept/require lower quality options, and things like screenshots or vo_image, which can be slower, but should not sacrifice quality by default. Should this have two sets of options or something similar to do the right thing depending on the code which calls libswscale? Maybe. Or should I just ignore the problem, make it someone else's problem (users who want to use software conversion VOs), provide a sub-optimal solution, and call it a day? Definitely, sounds good, pushing to master, goodbye.
2019-10-31 15:45:28 +00:00
struct zimg_opts *opts = &ctx->opts;
zimg: refactor (move around fields) The intention is to add slice-threading to the wrapper. For that purpose, move all zimg related state to a new struct mp_zimg_state. There is now an array of instances of this struct. The intention is to have one instance for each thread. As of this commit, this is hardcoded to 1 thread; the following commit extends this.
2020-07-14 18:48:48 +00:00
st->src = talloc_zero(NULL, struct mp_zimg_repack);
st->dst = talloc_zero(NULL, struct mp_zimg_repack);
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
zimg: add slice threading and use it by default This probably makes it much faster (I wouldn't know, I didn't run any benchmarks ). Seems to work as well (although I'm not sure, it's not like I'd perform rigorous tests). The scale_zimg test seems to mysteriously treat color in fully transparent alpha differently, which makes no sense, and isn't visible (but makes the test fail). I can't be bothered with investigating this more. What do you do with failing tests? Correct, you disable them. Or rather, you disable whatever appears to cause them to fail, which is the threading in this case. This change follows mostly the tile_example.cpp. The slice size uses a minimum of 64, which was suggested by the zimg author. Some of this commit is a bit inelegant and weird, such as recomputing the scale factor for every slice, or the way slice_h is managed. Too lazy to make this more elegant. zimg git had a regressio around active_region (which is needed by the slicing), which was fixed in commit 83071706b2e6bc634. Apparently, the bug was never released, so just add a warning to the manpage.
2020-07-14 20:52:27 +00:00
st->scale_y = ctx->src.h / (double)ctx->dst.h;
st->slice_y = slice_y;
st->slice_h = slice_h;
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
zimg_image_format src_fmt, dst_fmt;
zimg: refactor (move around fields) The intention is to add slice-threading to the wrapper. For that purpose, move all zimg related state to a new struct mp_zimg_state. There is now an array of instances of this struct. The intention is to have one instance for each thread. As of this commit, this is hardcoded to 1 thread; the following commit extends this.
2020-07-14 18:48:48 +00:00
// Note: do dst first, because src uses fields from dst.
if (!setup_format(&dst_fmt, st->dst, true, &ctx->dst, ctx, st) ||
!setup_format(&src_fmt, st->src, false, &ctx->src, ctx, st))
return false;
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
zimg_graph_builder_params params;
zimg_graph_builder_params_default(&params, ZIMG_API_VERSION);
sws_utils, zimg: destroy vo_x11 and vo_drm performance Raise swscale and zimg default parameters. This restores screenshot quality settings (maybe) unset in the commit before. Also expose some more libswscale and zimg options. Since these options are also used for VOs like x11 and drm, this will make x11/drm/etc. much slower. For compensation, provide a profile that sets the old option values: sw-fast. I'm also enabling zimg here, just as an experiment. The core problem is that we have a single set of command line options which control the settings used for most swscale/zimg uses. This was done in the previous commit. It cannot differentiate between the VOs, which need to be realtime and may accept/require lower quality options, and things like screenshots or vo_image, which can be slower, but should not sacrifice quality by default. Should this have two sets of options or something similar to do the right thing depending on the code which calls libswscale? Maybe. Or should I just ignore the problem, make it someone else's problem (users who want to use software conversion VOs), provide a sub-optimal solution, and call it a day? Definitely, sounds good, pushing to master, goodbye.
2019-10-31 15:45:28 +00:00
params.resample_filter = opts->scaler;
params.filter_param_a = opts->scaler_params[0];
params.filter_param_b = opts->scaler_params[1];
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
sws_utils, zimg: destroy vo_x11 and vo_drm performance Raise swscale and zimg default parameters. This restores screenshot quality settings (maybe) unset in the commit before. Also expose some more libswscale and zimg options. Since these options are also used for VOs like x11 and drm, this will make x11/drm/etc. much slower. For compensation, provide a profile that sets the old option values: sw-fast. I'm also enabling zimg here, just as an experiment. The core problem is that we have a single set of command line options which control the settings used for most swscale/zimg uses. This was done in the previous commit. It cannot differentiate between the VOs, which need to be realtime and may accept/require lower quality options, and things like screenshots or vo_image, which can be slower, but should not sacrifice quality by default. Should this have two sets of options or something similar to do the right thing depending on the code which calls libswscale? Maybe. Or should I just ignore the problem, make it someone else's problem (users who want to use software conversion VOs), provide a sub-optimal solution, and call it a day? Definitely, sounds good, pushing to master, goodbye.
2019-10-31 15:45:28 +00:00
params.resample_filter_uv = opts->scaler_chroma;
params.filter_param_a_uv = opts->scaler_chroma_params[0];
params.filter_param_b_uv = opts->scaler_chroma_params[1];
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
sws_utils, zimg: destroy vo_x11 and vo_drm performance Raise swscale and zimg default parameters. This restores screenshot quality settings (maybe) unset in the commit before. Also expose some more libswscale and zimg options. Since these options are also used for VOs like x11 and drm, this will make x11/drm/etc. much slower. For compensation, provide a profile that sets the old option values: sw-fast. I'm also enabling zimg here, just as an experiment. The core problem is that we have a single set of command line options which control the settings used for most swscale/zimg uses. This was done in the previous commit. It cannot differentiate between the VOs, which need to be realtime and may accept/require lower quality options, and things like screenshots or vo_image, which can be slower, but should not sacrifice quality by default. Should this have two sets of options or something similar to do the right thing depending on the code which calls libswscale? Maybe. Or should I just ignore the problem, make it someone else's problem (users who want to use software conversion VOs), provide a sub-optimal solution, and call it a day? Definitely, sounds good, pushing to master, goodbye.
2019-10-31 15:45:28 +00:00
params.dither_type = opts->dither;
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
params.cpu_type = ZIMG_CPU_AUTO_64B;
sws_utils, zimg: destroy vo_x11 and vo_drm performance Raise swscale and zimg default parameters. This restores screenshot quality settings (maybe) unset in the commit before. Also expose some more libswscale and zimg options. Since these options are also used for VOs like x11 and drm, this will make x11/drm/etc. much slower. For compensation, provide a profile that sets the old option values: sw-fast. I'm also enabling zimg here, just as an experiment. The core problem is that we have a single set of command line options which control the settings used for most swscale/zimg uses. This was done in the previous commit. It cannot differentiate between the VOs, which need to be realtime and may accept/require lower quality options, and things like screenshots or vo_image, which can be slower, but should not sacrifice quality by default. Should this have two sets of options or something similar to do the right thing depending on the code which calls libswscale? Maybe. Or should I just ignore the problem, make it someone else's problem (users who want to use software conversion VOs), provide a sub-optimal solution, and call it a day? Definitely, sounds good, pushing to master, goodbye.
2019-10-31 15:45:28 +00:00
if (opts->fast)
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
params.allow_approximate_gamma = 1;
if (ctx->src.color.sig_peak > 0)
params.nominal_peak_luminance = ctx->src.color.sig_peak;
zimg: refactor (move around fields) The intention is to add slice-threading to the wrapper. For that purpose, move all zimg related state to a new struct mp_zimg_state. There is now an array of instances of this struct. The intention is to have one instance for each thread. As of this commit, this is hardcoded to 1 thread; the following commit extends this.
2020-07-14 18:48:48 +00:00
st->graph = zimg_filter_graph_build(&src_fmt, &dst_fmt, &params);
if (!st->graph) {
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
char err[128] = {0};
zimg_get_last_error(err, sizeof(err) - 1);
MP_ERR(ctx, "zimg_filter_graph_build: %s \n", err);
zimg: refactor (move around fields) The intention is to add slice-threading to the wrapper. For that purpose, move all zimg related state to a new struct mp_zimg_state. There is now an array of instances of this struct. The intention is to have one instance for each thread. As of this commit, this is hardcoded to 1 thread; the following commit extends this.
2020-07-14 18:48:48 +00:00
return false;
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
}
size_t tmp_size;
zimg: refactor (move around fields) The intention is to add slice-threading to the wrapper. For that purpose, move all zimg related state to a new struct mp_zimg_state. There is now an array of instances of this struct. The intention is to have one instance for each thread. As of this commit, this is hardcoded to 1 thread; the following commit extends this.
2020-07-14 18:48:48 +00:00
if (!zimg_filter_graph_get_tmp_size(st->graph, &tmp_size)) {
zimg: remove C11 aligned_alloc() requirement It's not available on Windows because MinGW is fucking horrible and Microsoft are fucking assholes.
2020-04-30 22:55:13 +00:00
tmp_size = MP_ALIGN_UP(tmp_size, ZIMG_ALIGN) + ZIMG_ALIGN;
zimg: refactor (move around fields) The intention is to add slice-threading to the wrapper. For that purpose, move all zimg related state to a new struct mp_zimg_state. There is now an array of instances of this struct. The intention is to have one instance for each thread. As of this commit, this is hardcoded to 1 thread; the following commit extends this.
2020-07-14 18:48:48 +00:00
st->tmp_alloc = ta_alloc_size(NULL, tmp_size);
if (st->tmp_alloc)
st->tmp = (void *)MP_ALIGN_UP((uintptr_t)st->tmp_alloc, ZIMG_ALIGN);
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
}
zimg: refactor (move around fields) The intention is to add slice-threading to the wrapper. For that purpose, move all zimg related state to a new struct mp_zimg_state. There is now an array of instances of this struct. The intention is to have one instance for each thread. As of this commit, this is hardcoded to 1 thread; the following commit extends this.
2020-07-14 18:48:48 +00:00
if (!st->tmp_alloc)
return false;
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
zimg: refactor (move around fields) The intention is to add slice-threading to the wrapper. For that purpose, move all zimg related state to a new struct mp_zimg_state. There is now an array of instances of this struct. The intention is to have one instance for each thread. As of this commit, this is hardcoded to 1 thread; the following commit extends this.
2020-07-14 18:48:48 +00:00
if (!allocate_buffer(st, st->src) || !allocate_buffer(st, st->dst))
return false;
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
return true;
zimg: refactor (move around fields) The intention is to add slice-threading to the wrapper. For that purpose, move all zimg related state to a new struct mp_zimg_state. There is now an array of instances of this struct. The intention is to have one instance for each thread. As of this commit, this is hardcoded to 1 thread; the following commit extends this.
2020-07-14 18:48:48 +00:00
}
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
zimg: refactor (move around fields) The intention is to add slice-threading to the wrapper. For that purpose, move all zimg related state to a new struct mp_zimg_state. There is now an array of instances of this struct. The intention is to have one instance for each thread. As of this commit, this is hardcoded to 1 thread; the following commit extends this.
2020-07-14 18:48:48 +00:00
bool mp_zimg_config(struct mp_zimg_context *ctx)
{
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
destroy_zimg(ctx);
zimg: refactor (move around fields) The intention is to add slice-threading to the wrapper. For that purpose, move all zimg related state to a new struct mp_zimg_state. There is now an array of instances of this struct. The intention is to have one instance for each thread. As of this commit, this is hardcoded to 1 thread; the following commit extends this.
2020-07-14 18:48:48 +00:00
if (ctx->opts_cache)
mp_zimg_update_from_cmdline(ctx);
zimg: add slice threading and use it by default This probably makes it much faster (I wouldn't know, I didn't run any benchmarks ). Seems to work as well (although I'm not sure, it's not like I'd perform rigorous tests). The scale_zimg test seems to mysteriously treat color in fully transparent alpha differently, which makes no sense, and isn't visible (but makes the test fail). I can't be bothered with investigating this more. What do you do with failing tests? Correct, you disable them. Or rather, you disable whatever appears to cause them to fail, which is the threading in this case. This change follows mostly the tile_example.cpp. The slice size uses a minimum of 64, which was suggested by the zimg author. Some of this commit is a bit inelegant and weird, such as recomputing the scale factor for every slice, or the way slice_h is managed. Too lazy to make this more elegant. zimg git had a regressio around active_region (which is needed by the slicing), which was fixed in commit 83071706b2e6bc634. Apparently, the bug was never released, so just add a warning to the manpage.
2020-07-14 20:52:27 +00:00
int slices = ctx->opts.threads;
if (slices < 1)
slices = av_cpu_count();
slices = MPCLAMP(slices, 1, 64);
struct mp_imgfmt_desc dstfmt = mp_imgfmt_get_desc(ctx->dst.imgfmt);
if (!dstfmt.align_y)
goto fail;
int full_h = MP_ALIGN_UP(ctx->dst.h, dstfmt.align_y);
int slice_h = (full_h + slices - 1) / slices;
slice_h = MP_ALIGN_UP(slice_h, dstfmt.align_y);
slice_h = MP_ALIGN_UP(slice_h, 64); // for dithering and minimum slice size
slices = (full_h + slice_h - 1) / slice_h;
int threads = slices - 1;
if (threads != ctx->current_thread_count) {
// Just destroy and recreate all - dumb and costly, but rarely happens.
TA_FREEP(&ctx->tp);
ctx->current_thread_count = 0;
if (threads) {
MP_VERBOSE(ctx, "using %d threads for scaling\n", threads);
ctx->tp = mp_thread_pool_create(NULL, threads, threads, threads);
if (!ctx->tp)
goto fail;
ctx->current_thread_count = threads;
}
}
zimg: refactor (move around fields) The intention is to add slice-threading to the wrapper. For that purpose, move all zimg related state to a new struct mp_zimg_state. There is now an array of instances of this struct. The intention is to have one instance for each thread. As of this commit, this is hardcoded to 1 thread; the following commit extends this.
2020-07-14 18:48:48 +00:00
zimg: add slice threading and use it by default This probably makes it much faster (I wouldn't know, I didn't run any benchmarks ). Seems to work as well (although I'm not sure, it's not like I'd perform rigorous tests). The scale_zimg test seems to mysteriously treat color in fully transparent alpha differently, which makes no sense, and isn't visible (but makes the test fail). I can't be bothered with investigating this more. What do you do with failing tests? Correct, you disable them. Or rather, you disable whatever appears to cause them to fail, which is the threading in this case. This change follows mostly the tile_example.cpp. The slice size uses a minimum of 64, which was suggested by the zimg author. Some of this commit is a bit inelegant and weird, such as recomputing the scale factor for every slice, or the way slice_h is managed. Too lazy to make this more elegant. zimg git had a regressio around active_region (which is needed by the slicing), which was fixed in commit 83071706b2e6bc634. Apparently, the bug was never released, so just add a warning to the manpage.
2020-07-14 20:52:27 +00:00
for (int n = 0; n < slices; n++) {
struct mp_zimg_state *st = talloc_zero(NULL, struct mp_zimg_state);
MP_TARRAY_APPEND(ctx, ctx->states, ctx->num_states, st);
if (!mp_zimg_state_init(ctx, st, n * slice_h, slice_h))
goto fail;
zimg: refactor (move around fields) The intention is to add slice-threading to the wrapper. For that purpose, move all zimg related state to a new struct mp_zimg_state. There is now an array of instances of this struct. The intention is to have one instance for each thread. As of this commit, this is hardcoded to 1 thread; the following commit extends this.
2020-07-14 18:48:48 +00:00
}
zimg: add slice threading and use it by default This probably makes it much faster (I wouldn't know, I didn't run any benchmarks ). Seems to work as well (although I'm not sure, it's not like I'd perform rigorous tests). The scale_zimg test seems to mysteriously treat color in fully transparent alpha differently, which makes no sense, and isn't visible (but makes the test fail). I can't be bothered with investigating this more. What do you do with failing tests? Correct, you disable them. Or rather, you disable whatever appears to cause them to fail, which is the threading in this case. This change follows mostly the tile_example.cpp. The slice size uses a minimum of 64, which was suggested by the zimg author. Some of this commit is a bit inelegant and weird, such as recomputing the scale factor for every slice, or the way slice_h is managed. Too lazy to make this more elegant. zimg git had a regressio around active_region (which is needed by the slicing), which was fixed in commit 83071706b2e6bc634. Apparently, the bug was never released, so just add a warning to the manpage.
2020-07-14 20:52:27 +00:00
assert(ctx->num_states == slices);
zimg: refactor (move around fields) The intention is to add slice-threading to the wrapper. For that purpose, move all zimg related state to a new struct mp_zimg_state. There is now an array of instances of this struct. The intention is to have one instance for each thread. As of this commit, this is hardcoded to 1 thread; the following commit extends this.
2020-07-14 18:48:48 +00:00
return true;
zimg: add slice threading and use it by default This probably makes it much faster (I wouldn't know, I didn't run any benchmarks ). Seems to work as well (although I'm not sure, it's not like I'd perform rigorous tests). The scale_zimg test seems to mysteriously treat color in fully transparent alpha differently, which makes no sense, and isn't visible (but makes the test fail). I can't be bothered with investigating this more. What do you do with failing tests? Correct, you disable them. Or rather, you disable whatever appears to cause them to fail, which is the threading in this case. This change follows mostly the tile_example.cpp. The slice size uses a minimum of 64, which was suggested by the zimg author. Some of this commit is a bit inelegant and weird, such as recomputing the scale factor for every slice, or the way slice_h is managed. Too lazy to make this more elegant. zimg git had a regressio around active_region (which is needed by the slicing), which was fixed in commit 83071706b2e6bc634. Apparently, the bug was never released, so just add a warning to the manpage.
2020-07-14 20:52:27 +00:00
fail:
destroy_zimg(ctx);
return false;
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
}
bool mp_zimg_config_image_params(struct mp_zimg_context *ctx)
{
zimg: refactor (move around fields) The intention is to add slice-threading to the wrapper. For that purpose, move all zimg related state to a new struct mp_zimg_state. There is now an array of instances of this struct. The intention is to have one instance for each thread. As of this commit, this is hardcoded to 1 thread; the following commit extends this.
2020-07-14 18:48:48 +00:00
if (ctx->num_states) {
// All states are the same, so checking only one of them is sufficient.
struct mp_zimg_state *st = ctx->states[0];
if (st->src && mp_image_params_equal(&ctx->src, &st->src->fmt) &&
st->dst && mp_image_params_equal(&ctx->dst, &st->dst->fmt) &&
(!ctx->opts_cache || !m_config_cache_update(ctx->opts_cache)) &&
st->graph)
return true;
}
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
return mp_zimg_config(ctx);
}
zimg: refactor (move around fields) The intention is to add slice-threading to the wrapper. For that purpose, move all zimg related state to a new struct mp_zimg_state. There is now an array of instances of this struct. The intention is to have one instance for each thread. As of this commit, this is hardcoded to 1 thread; the following commit extends this.
2020-07-14 18:48:48 +00:00
static void do_convert(struct mp_zimg_state *st)
{
assert(st->graph);
// An annoyance.
zimg_image_buffer *zsrc = &st->src->zbuf;
zimg_image_buffer_const zsrc_c = {ZIMG_API_VERSION};
for (int n = 0; n < MP_ARRAY_SIZE(zsrc_c.plane); n++) {
zsrc_c.plane[n].data = zsrc->plane[n].data;
zsrc_c.plane[n].stride = zsrc->plane[n].stride;
zsrc_c.plane[n].mask = zsrc->plane[n].mask;
}
// (The API promises to succeed if no user callbacks fail, so no need
// to check the return value.)
zimg_filter_graph_process(st->graph, &zsrc_c, &st->dst->zbuf, st->tmp,
repack_entrypoint, st->src,
repack_entrypoint, st->dst);
}
zimg: add slice threading and use it by default This probably makes it much faster (I wouldn't know, I didn't run any benchmarks ). Seems to work as well (although I'm not sure, it's not like I'd perform rigorous tests). The scale_zimg test seems to mysteriously treat color in fully transparent alpha differently, which makes no sense, and isn't visible (but makes the test fail). I can't be bothered with investigating this more. What do you do with failing tests? Correct, you disable them. Or rather, you disable whatever appears to cause them to fail, which is the threading in this case. This change follows mostly the tile_example.cpp. The slice size uses a minimum of 64, which was suggested by the zimg author. Some of this commit is a bit inelegant and weird, such as recomputing the scale factor for every slice, or the way slice_h is managed. Too lazy to make this more elegant. zimg git had a regressio around active_region (which is needed by the slicing), which was fixed in commit 83071706b2e6bc634. Apparently, the bug was never released, so just add a warning to the manpage.
2020-07-14 20:52:27 +00:00
static void do_convert_thread(void *ptr)
{
struct mp_zimg_state *st = ptr;
do_convert(st);
mp_waiter_wakeup(&st->thread_waiter, 0);
}
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
bool mp_zimg_convert(struct mp_zimg_context *ctx, struct mp_image *dst,
struct mp_image *src)
{
ctx->src = src->params;
ctx->dst = dst->params;
if (!mp_zimg_config_image_params(ctx)) {
MP_ERR(ctx, "zimg initialization failed.\n");
return false;
}
zimg: refactor (move around fields) The intention is to add slice-threading to the wrapper. For that purpose, move all zimg related state to a new struct mp_zimg_state. There is now an array of instances of this struct. The intention is to have one instance for each thread. As of this commit, this is hardcoded to 1 thread; the following commit extends this.
2020-07-14 18:48:48 +00:00
for (int n = 0; n < ctx->num_states; n++) {
struct mp_zimg_state *st = ctx->states[n];
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
zimg: add slice threading and use it by default This probably makes it much faster (I wouldn't know, I didn't run any benchmarks ). Seems to work as well (although I'm not sure, it's not like I'd perform rigorous tests). The scale_zimg test seems to mysteriously treat color in fully transparent alpha differently, which makes no sense, and isn't visible (but makes the test fail). I can't be bothered with investigating this more. What do you do with failing tests? Correct, you disable them. Or rather, you disable whatever appears to cause them to fail, which is the threading in this case. This change follows mostly the tile_example.cpp. The slice size uses a minimum of 64, which was suggested by the zimg author. Some of this commit is a bit inelegant and weird, such as recomputing the scale factor for every slice, or the way slice_h is managed. Too lazy to make this more elegant. zimg git had a regressio around active_region (which is needed by the slicing), which was fixed in commit 83071706b2e6bc634. Apparently, the bug was never released, so just add a warning to the manpage.
2020-07-14 20:52:27 +00:00
if (!wrap_buffer(st, st->src, src) || !wrap_buffer(st, st->dst, dst)) {
zimg: refactor (move around fields) The intention is to add slice-threading to the wrapper. For that purpose, move all zimg related state to a new struct mp_zimg_state. There is now an array of instances of this struct. The intention is to have one instance for each thread. As of this commit, this is hardcoded to 1 thread; the following commit extends this.
2020-07-14 18:48:48 +00:00
MP_ERR(ctx, "zimg repacker initialization failed.\n");
return false;
}
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
}
zimg: add slice threading and use it by default This probably makes it much faster (I wouldn't know, I didn't run any benchmarks ). Seems to work as well (although I'm not sure, it's not like I'd perform rigorous tests). The scale_zimg test seems to mysteriously treat color in fully transparent alpha differently, which makes no sense, and isn't visible (but makes the test fail). I can't be bothered with investigating this more. What do you do with failing tests? Correct, you disable them. Or rather, you disable whatever appears to cause them to fail, which is the threading in this case. This change follows mostly the tile_example.cpp. The slice size uses a minimum of 64, which was suggested by the zimg author. Some of this commit is a bit inelegant and weird, such as recomputing the scale factor for every slice, or the way slice_h is managed. Too lazy to make this more elegant. zimg git had a regressio around active_region (which is needed by the slicing), which was fixed in commit 83071706b2e6bc634. Apparently, the bug was never released, so just add a warning to the manpage.
2020-07-14 20:52:27 +00:00
for (int n = 1; n < ctx->num_states; n++) {
struct mp_zimg_state *st = ctx->states[n];
st->thread_waiter = (struct mp_waiter)MP_WAITER_INITIALIZER;
bool r = mp_thread_pool_run(ctx->tp, do_convert_thread, st);
// This is guaranteed by the API; and unrolling would be inconvenient.
assert(r);
}
zimg: refactor (move around fields) The intention is to add slice-threading to the wrapper. For that purpose, move all zimg related state to a new struct mp_zimg_state. There is now an array of instances of this struct. The intention is to have one instance for each thread. As of this commit, this is hardcoded to 1 thread; the following commit extends this.
2020-07-14 18:48:48 +00:00
do_convert(ctx->states[0]);
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
zimg: add slice threading and use it by default This probably makes it much faster (I wouldn't know, I didn't run any benchmarks ). Seems to work as well (although I'm not sure, it's not like I'd perform rigorous tests). The scale_zimg test seems to mysteriously treat color in fully transparent alpha differently, which makes no sense, and isn't visible (but makes the test fail). I can't be bothered with investigating this more. What do you do with failing tests? Correct, you disable them. Or rather, you disable whatever appears to cause them to fail, which is the threading in this case. This change follows mostly the tile_example.cpp. The slice size uses a minimum of 64, which was suggested by the zimg author. Some of this commit is a bit inelegant and weird, such as recomputing the scale factor for every slice, or the way slice_h is managed. Too lazy to make this more elegant. zimg git had a regressio around active_region (which is needed by the slicing), which was fixed in commit 83071706b2e6bc634. Apparently, the bug was never released, so just add a warning to the manpage.
2020-07-14 20:52:27 +00:00
for (int n = 1; n < ctx->num_states; n++) {
struct mp_zimg_state *st = ctx->states[n];
mp_waiter_wait(&st->thread_waiter);
}
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
return true;
}
static bool supports_format(int imgfmt, bool out)
{
zimg: add support for big endian input and output One of the extremely annoying dumb things in ffmpeg is that most pixel formats are available as little endian and big endian variants. (The sane way would be having native endian formats only.) Usually, most of the real codecs use native formats only, while non-native formats are used by fringe raw codecs only. But the PNG encoders and decoders unfortunately use big endian formats, and since PNG it such a popular format, this causes problems for us. In particular, the current zimg wrapper will refuse to work (and mpv will fall back to sws) when writing non-8 bit PNGs. So add non-native endian support to zimg. This is done in a fairly "generic" way (which means lots of potential for bugs). If input is a "regular" format (and just byte-swapped), the rest happens automatically, which happens to cover all interesting formats. Some things could be more efficient; for example, unpacking is done on the data before it's passed to the unpacker. You could make endian swapping part of the actual unpacking process, which might be slightly faster. You could avoid copying twice in some cases (such as when there's no actual repacker, or if alignment needs to be corrected). But I don't really care. It's reasonably fast for the normal case. Not entirely sure whether this is correct. Some (but not many) formats are covered by the tests, some I tested manually. Some I can't even test, because libswscale doesn't support them (like nv20*).
2020-04-12 20:14:55 +00:00
struct mp_image_params fmt = {.imgfmt = imgfmt};
struct mp_zimg_repack t;
zimg_image_format zfmt;
zimg: refactor (move around fields) The intention is to add slice-threading to the wrapper. For that purpose, move all zimg related state to a new struct mp_zimg_state. There is now an array of instances of this struct. The intention is to have one instance for each thread. As of this commit, this is hardcoded to 1 thread; the following commit extends this.
2020-07-14 18:48:48 +00:00
return setup_format(&zfmt, &t, out, &fmt, NULL, NULL);
video: add zimg wrapper This provides a very similar API to sws_utils.h, which can be used to convert and scale from one mp_image to another. This commit adds only the code, but does not use it anywhere. The code is quite preliminary and barely tested. It supports only a few pixel formats, and will return failure for many others. (Unlike libswscale, which tries to support anything that FFmpeg knows.) zimg itself accepts only planar formats. Supporting other formats requires manual packing/unpacking. (Compared to libswscale, the zimg API is generally lower level, but allows for more flexibility.) Only BGR0 output was actually tested. It appears to work.
2019-10-19 23:54:45 +00:00
}
bool mp_zimg_supports_in_format(int imgfmt)
{
return supports_format(imgfmt, false);
}
bool mp_zimg_supports_out_format(int imgfmt)
{
return supports_format(imgfmt, true);
}