client API, lua: add new API for setting OSD overlays

Lua scripting has an undocumented mp.set_osd_ass() function, which is
used by osc.lua and console.lua. Apparently, 3rd party scripts also use
this. It's probably time to make this a public API.

The Lua implementation just bypassed the libmpv API. To make it usable
by any type of client, turn it into a command, "osd-overlay".

There's already a "overlay-add". Ignore it (although the manpage admits
guiltiness). I don't really want to deal with that old command. Its main
problem is that it uses global IDs, while I'd like to avoid that scripts
mess with each others overlays (whether that is accidentally or
intentionally). Maybe "overlay-add" can eventually be merged into
"osd-overlay", but I'm too lazy to do that now.

Scripting now uses the commands. There is a helper to manage OSD
overlays. The helper is very "thin"; I only want to force script authors
to use the ID allocation, which may help with putting multiple scripts
into a single .lua file without causing conflicts (basically, avoiding
singletons within a script's environment). The old set_osd_ass() is
emulated with the new API.

The JS scripting wrapper also provides a set_osd_ass() function, which
calls internal mpv API. Comment that part (to keep it compiling), but
I'm leaving it to @avih to finish the change.

DOCS/man/input.rst

@@ -937,6 +937,86 @@ Input Commands that are Possibly Subject to Change
     Remove an overlay added with ``overlay-add`` and the same ID. Does nothing
     if no overlay with this ID exists.
 
+``osd-overlay``
+    Add/update/remove an OSD overlay.
+
+    (Although this sounds similar to ``overlay-add``, ``osd-overlay`` is for
+    text overlays, while ``overlay-add`` is for bitmaps. Maybe ``overlay-add``
+    will be merged into ``osd-overlay`` to remove this oddity.)
+
+    You can use this to add text overlays in ASS format. ASS has advanced
+    positioning and rendering tags, which can be used to render almost any kind
+    of vector graphics.
+
+    This command accepts the following parameters:
+
+    ``id``
+        Arbitrary integer that identifies the overlay. Multiple overlays can be
+        added by calling this command with different ``id`` parameters. Calling
+        this command with the same ``id`` replaces the previously set overlay.
+
+        There is a separate namespace for each libmpv client (i.e. IPC
+        connection, script), so IDs can be made up and assigned by the API user
+        without conflicting with other API users.
+
+        If the libmpv client is destroyed, all overlays associated with it are
+        also deleted. In particular, connecting via ``--input-ipc-server``,
+        adding an overlay, and disconnecting will remove the overlay immediately
+        again.
+
+    ``format``
+        String that gives the type of the overlay. Accepts the following values:
+
+        ``ass-events``
+            The ``data`` parameter is a string. The string is split on the
+            newline character. Every line is turned into the ``Text`` part of
+            a ``Dialogue`` ASS event. Timing is unused (but behavior of timing
+            dependent ASS tags may change in future mpv versions).
+
+            Note that it's better to put multiple lines into ``data``, instead
+            of adding multiple OSD overlays.
+
+            This provides 2 ASS ``Styles``. ``OSD`` contains the text style as
+            defined by the current ``--osd-...`` options. ``Default`` is
+            similar, and contains style that ``OSD`` would have if all options
+            were set to the default.
+
+            In addition, the ``res_x`` and ``res_y`` options specify the value
+            of the ASS ``PlayResX`` and ``PlayResY`` header fields. If ``res_y``
+            is set to 0, ``PlayResY`` is initialized to an arbitrary default
+            value (but note that the default for this command is 720, not 0).
+            If ``res_x`` is set to 0, ``PlayResX`` is set based on ``res_y``
+            such that a virtual ASS pixel has a square pixel aspect ratio.
+
+        ``none``
+            Special value that causes the overlay to be removed. Most parameters
+            other than ``id`` and ``format`` are mostly ignored.
+
+    ``data``
+        String defining the overlay contents according to the ``format``
+        parameter.
+
+    ``res_x``, ``res_y``
+        Used if ``format`` is set to ``ass-events`` (see description there).
+        Optional, defaults to 0/720.
+
+    ``z``
+        The Z order of the overlay. Optional, defaults to 0.
+
+        Note that Z order between different overlays of different formats is
+        static, and cannot be changed (currently, this means that bitmap
+        overlays added by ``overlay-add`` are always on top of the ASS overlays
+        added by ``osd-overlay``). In addition, the builtin OSD components are
+        always below any of the custom OSD. (This includes subtitles of any kind
+        as well as text rendered by ``show-text``.)
+
+        It's possible that future mpv versions will randomly change how Z order
+        between different OSD formats and builtin OSD is handled.
+
+    Note: always use named arguments (``mpv_command_node()``). Scripts should
+    use the ``mp.create_osd_overlay()`` helper instead of invoking this command
+    directly.
+
 ``script-message [<arg1> [<arg2> [...]]]``
     Send a message to all clients, and pass it the following list of arguments.
     What this message means, how many arguments it takes, and what the arguments

DOCS/man/lua.rst

@@ -536,6 +536,41 @@ are useful only in special situations.
     Undo a previous registration with ``mp.register_script_message``. Does
     nothing if the ``name`` wasn't registered.
 
+``mp.create_osd_overlay(format)``
+    Create an OSD overlay. This is a very thin wrapper around the ``osd-overlay``
+    command. The function returns a table, which mostly contains fields that
+    will be passed to ``osd-overlay``. The ``format`` parameter is used to
+    initialize the ``format`` field. The ``data`` field contains the text to
+    be used as overlay. For details, see the ``osd-overlay`` command.
+
+    In addition, it provides the following methods:
+
+    ``update()``
+        Commit the OSD overlay to the screen, or in other words, run the
+        ``osd-overlay`` command with the current fields of the overlay table.
+
+    ``remove()``
+        Remove the overlay from the screen. A ``update()`` call will add it
+        again.
+
+    Example:
+
+    ::
+
+        ov = mp.create_osd_overlay("ass-events")
+        ov.data = "{\\an5}{\\b1}hello world!"
+        ov:update()
+
+    The advantage of using this wrapper (as opposed to running ``osd-overlay``
+    directly) is that the ``id`` field is allocated automatically.
+
+``mp.get_osd_size()``
+    Returns a tuple of ``osd_width, osd_height, osd_par``. The first two give
+    the size of the OSD in pixels (for video ouputs like ``--vo=xv``, this may
+    be "scaled" pixels). The third is the display pixel aspect ratio.
+
+    May return invalid/nonsense values if OSD is not initialized yet.
+
 mp.msg functions
 ----------------
 

player/client.c

@@ -457,7 +457,7 @@ static void mp_destroy_client(mpv_handle *ctx, bool terminate)
     // causes a crash, block until all asynchronous requests were served.
     mpv_wait_async_requests(ctx);
 
-    osd_set_external(mpctx->osd, ctx, 0, 0, NULL);
+    osd_set_external_remove_owner(mpctx->osd, ctx);
     mp_input_remove_sections_by_owner(mpctx->input, ctx->name);
 
     pthread_mutex_lock(&clients->lock);

player/command.c

@@ -4085,6 +4085,25 @@ static void overlay_uninit(struct MPContext *mpctx)
         mp_image_unrefp(&cmd->overlay_osd[n].packed);
 }
 
+static void cmd_osd_overlay(void *p)
+{
+    struct mp_cmd_ctx *cmd = p;
+    struct MPContext *mpctx = cmd->mpctx;
+
+    struct osd_external_ass ov = {
+        .owner  = cmd->cmd->sender,
+        .id     = cmd->args[0].v.i64,
+        .format = cmd->args[1].v.i,
+        .data   = cmd->args[2].v.s,
+        .res_x  = cmd->args[3].v.i,
+        .res_y  = cmd->args[4].v.i,
+        .z      = cmd->args[5].v.i,
+    };
+
+    osd_set_external(mpctx->osd, &ov);
+    mp_wakeup_core(mpctx);
+}
+
 static struct track *find_track_with_url(struct MPContext *mpctx, int type,
                                          const char *url)
 {
@@ -5877,6 +5896,19 @@ const struct mp_cmd_def mp_cmds[] = {
                                         OPT_INT("stride", v.i, 0), }},
     { "overlay-remove", cmd_overlay_remove, { OPT_INT("id", v.i, 0) } },
 
+    { "osd-overlay", cmd_osd_overlay,
+        {
+            OPT_INT64("id", v.i64, 0),
+            OPT_CHOICE("format", v.i, 0, ({"none", 0},
+                                          {"ass-events", 1})),
+            OPT_STRING("data", v.s, 0),
+            OPT_INT("res_x", v.i, 0, OPTDEF_INT(0)),
+            OPT_INT("res_y", v.i, 0, OPTDEF_INT(720)),
+            OPT_INT("z", v.i, 0, OPTDEF_INT(0)),
+        },
+        .is_noisy = true,
+    },
+
     { "write-watch-later-config", cmd_write_watch_later_config },
 
     { "hook-add", cmd_hook_add, { OPT_STRING("arg0", v.s, 0),

player/javascript.c

@@ -738,7 +738,7 @@ static void script_set_osd_ass(js_State *J)
     int res_x = jsL_checkint(J, 1);
     int res_y = jsL_checkint(J, 2);
     const char *text = js_tostring(J, 3);
-    osd_set_external(ctx->mpctx->osd, ctx->client, res_x, res_y, (char *)text);
+    //osd_set_external(ctx->mpctx->osd, ctx->client, res_x, res_y, (char *)text);
     mp_wakeup_core(ctx->mpctx);
     push_success(J);
 }

player/lua.c

@@ -998,31 +998,6 @@ static int script_raw_abort_async_command(lua_State *L)
     return 0;
 }
 
-static int script_set_osd_ass(lua_State *L)
-{
-    struct script_ctx *ctx = get_ctx(L);
-    int res_x = luaL_checkinteger(L, 1);
-    int res_y = luaL_checkinteger(L, 2);
-    const char *text = luaL_checkstring(L, 3);
-    if (!text[0])
-        text = " "; // force external OSD initialization
-    osd_set_external(ctx->mpctx->osd, ctx->client, res_x, res_y, (char *)text);
-    mp_wakeup_core(ctx->mpctx);
-    return 0;
-}
-
-static int script_get_osd_size(lua_State *L)
-{
-    struct MPContext *mpctx = get_mpctx(L);
-    struct mp_osd_res vo_res = osd_get_vo_res(mpctx->osd);
-    double aspect = 1.0 * vo_res.w / MPMAX(vo_res.h, 1) /
-                    (vo_res.display_par ? vo_res.display_par : 1);
-    lua_pushnumber(L, vo_res.w);
-    lua_pushnumber(L, vo_res.h);
-    lua_pushnumber(L, aspect);
-    return 3;
-}
-
 static int script_get_osd_margins(lua_State *L)
 {
     struct MPContext *mpctx = get_mpctx(L);
@@ -1275,8 +1250,6 @@ static const struct fn_entry main_fns[] = {
     FN_ENTRY(set_property_native),
     FN_ENTRY(raw_observe_property),
     FN_ENTRY(raw_unobserve_property),
-    FN_ENTRY(set_osd_ass),
-    FN_ENTRY(get_osd_size),
     FN_ENTRY(get_osd_margins),
     FN_ENTRY(get_mouse_pos),
     FN_ENTRY(get_time),

player/lua/defaults.lua

@@ -589,6 +589,65 @@ function mp.abort_async_command(t)
     end
 end
 
+local overlay_mt = {}
+overlay_mt.__index = overlay_mt
+local overlay_new_id = 0
+
+function mp.create_osd_overlay(format)
+    overlay_new_id = overlay_new_id + 1
+    local overlay = {
+        format = format,
+        id = overlay_new_id,
+        data = "",
+        res_x = 0,
+        res_y = 720,
+    }
+    setmetatable(overlay, overlay_mt)
+    return overlay
+end
+
+function overlay_mt.update(ov)
+    local cmd = {}
+    for k, v in pairs(ov) do
+        cmd[k] = v
+    end
+    cmd.name = "osd-overlay"
+    mp.command_native(cmd)
+end
+
+function overlay_mt.remove(ov)
+    mp.command_native {
+        name = "osd-overlay",
+        id = ov.id,
+        format = "none",
+        data = "",
+    }
+end
+
+-- legacy API
+function mp.set_osd_ass(res_x, res_y, data)
+    if not mp._legacy_overlay then
+        mp._legacy_overlay = mp.create_osd_overlay("ass-events")
+    end
+    mp._legacy_overlay.res_x = res_x
+    mp._legacy_overlay.res_y = res_y
+    mp._legacy_overlay.data = data
+    mp._legacy_overlay:update()
+end
+
+function mp.get_osd_size()
+    local w = mp.get_property_number("osd-width", 0)
+    local h = mp.get_property_number("osd-height", 0)
+    local par = mp.get_property_number("osd-par", 0)
+    if par == 0 then
+        par = 1
+    end
+
+    local aspect = 1.0 * w / math.max(h) / par
+    return w, h, aspect
+end
+
+
 local mp_utils = package.loaded["mp.utils"]
 
 function mp_utils.format_table(t, set)

sub/osd.h

@@ -202,9 +202,18 @@ struct mp_osd_res osd_get_vo_res(struct osd_state *osd);
 void osd_rescale_bitmaps(struct sub_bitmaps *imgs, int frame_w, int frame_h,
                          struct mp_osd_res res, double compensate_par);
 
+struct osd_external_ass {
+    void *owner; // unique pointer (NULL is also allowed)
+    int64_t id;
+    int format;
+    char *data;
+    int res_x, res_y;
+    int z;
+};
+
 // defined in osd_libass.c and osd_dummy.c
-void osd_set_external(struct osd_state *osd, void *id, int res_x, int res_y,
+void osd_set_external(struct osd_state *osd, struct osd_external_ass *ov);
-                      char *text);
+void osd_set_external_remove_owner(struct osd_state *osd, void *owner);
 void osd_get_text_size(struct osd_state *osd, int *out_screen_h, int *out_font_h);
 void osd_get_function_sym(char *buffer, size_t buffer_size, int osd_function);
 

sub/osd_dummy.c

@@ -24,8 +24,11 @@ void osd_object_get_bitmaps(struct osd_state *osd, struct osd_object *obj,
     *out_imgs = (struct sub_bitmaps) {0};
 }
 
-void osd_set_external(struct osd_state *osd, void *id, int res_x, int res_y,
+void osd_set_external(struct osd_state *osd, struct osd_external_ass *ov)
-                      char *text)
+{
+}
+
+void osd_set_external_remove_owner(struct osd_state *osd, void *owner)
 {
 }
 

sub/osd_libass.c

@@ -80,8 +80,8 @@ static void destroy_ass_renderer(struct ass_state *ass)
 
 static void destroy_external(struct osd_external *ext)
 {
-    talloc_free(ext->text);
     destroy_ass_renderer(&ext->ass);
+    talloc_free(ext);
 }
 
 void osd_destroy_backend(struct osd_state *osd)
@@ -90,7 +90,7 @@ void osd_destroy_backend(struct osd_state *osd)
         struct osd_object *obj = osd->objs[n];
         destroy_ass_renderer(&obj->ass);
         for (int i = 0; i < obj->num_externals; i++)
-            destroy_external(&obj->externals[i]);
+            destroy_external(obj->externals[i]);
         obj->num_externals = 0;
     }
 }
@@ -470,11 +470,9 @@ static void update_osd(struct osd_state *osd, struct osd_object *obj)
 static void update_external(struct osd_state *osd, struct osd_object *obj,
                             struct osd_external *ext)
 {
-    bstr t = bstr0(ext->text);
+    bstr t = bstr0(ext->ov.data);
-    if (!t.len)
+    ext->ass.res_x = ext->ov.res_x;
-        return;
+    ext->ass.res_y = ext->ov.res_y;
-    ext->ass.res_x = ext->res_x;
-    ext->ass.res_y = ext->res_y;
     create_ass_track(osd, obj, &ext->ass);
 
     clear_ass(&ext->ass);
@@ -497,29 +495,42 @@ static void update_external(struct osd_state *osd, struct osd_object *obj,
     }
 }
 
-void osd_set_external(struct osd_state *osd, void *id, int res_x, int res_y,
+static int cmp_zorder(const void *pa, const void *pb)
-                      char *text)
+{
+    const struct osd_external *a = *(struct osd_external **)pa;
+    const struct osd_external *b = *(struct osd_external **)pb;
+    return a->ov.z == b->ov.z ? 0 : (a->ov.z > b->ov.z ? 1 : -1);
+}
+
+void osd_set_external(struct osd_state *osd, struct osd_external_ass *ov)
 {
     pthread_mutex_lock(&osd->lock);
     struct osd_object *obj = osd->objs[OSDTYPE_EXTERNAL];
-    struct osd_external *entry = 0;
+    bool zorder_changed = false;
+    int index = -1;
+
     for (int n = 0; n < obj->num_externals; n++) {
-        if (obj->externals[n].id == id) {
+        struct osd_external *e = obj->externals[n];
-            entry = &obj->externals[n];
+        if (e->ov.id == ov->id && e->ov.owner == ov->owner) {
+            index = n;
             break;
         }
     }
-    if (!entry && !text)
-        goto done;
 
-    if (!entry) {
+    if (index < 0) {
-        struct osd_external new = { .id = id };
+        if (!ov->format)
+            goto done;
+        struct osd_external *new = talloc_zero(NULL, struct osd_external);
+        new->ov.owner = ov->owner;
+        new->ov.id = ov->id;
         MP_TARRAY_APPEND(obj, obj->externals, obj->num_externals, new);
-        entry = &obj->externals[obj->num_externals - 1];
+        index = obj->num_externals - 1;
+        zorder_changed = true;
     }
 
-    if (!text) {
+    struct osd_external *entry = obj->externals[index];
-        int index = entry - &obj->externals[0];
+
+    if (!ov->format) {
         destroy_external(entry);
         MP_TARRAY_REMOVE_AT(obj->externals, obj->num_externals, index);
         obj->changed = true;
@@ -527,22 +538,46 @@ void osd_set_external(struct osd_state *osd, void *id, int res_x, int res_y,
         goto done;
     }
 
-    if (!entry->text || strcmp(entry->text, text) != 0 ||
+    entry->ov.format = ov->format;
-        entry->res_x != res_x || entry->res_y != res_y)
+    if (!entry->ov.data)
-    {
+        entry->ov.data = talloc_strdup(entry, "");
-        talloc_free(entry->text);
+    entry->ov.data[0] = '\0'; // reuse memory allocation
-        entry->text = talloc_strdup(NULL, text);
+    entry->ov.data = talloc_strdup_append(entry->ov.data, ov->data);
-        entry->res_x = res_x;
+    entry->ov.res_x = ov->res_x;
-        entry->res_y = res_y;
+    entry->ov.res_y = ov->res_y;
-        update_external(osd, obj, entry);
+    zorder_changed |= entry->ov.z != ov->z;
-        obj->changed = true;
+    entry->ov.z = ov->z;
-        osd->want_redraw_notification = true;
+
+    update_external(osd, obj, entry);
+
+    obj->changed = true;
+    osd->want_redraw_notification = true;
+
+    if (zorder_changed) {
+        qsort(obj->externals, obj->num_externals, sizeof(obj->externals[0]),
+              cmp_zorder);
     }
 
 done:
     pthread_mutex_unlock(&osd->lock);
 }
 
+void osd_set_external_remove_owner(struct osd_state *osd, void *owner)
+{
+    pthread_mutex_lock(&osd->lock);
+    struct osd_object *obj = osd->objs[OSDTYPE_EXTERNAL];
+    for (int n = obj->num_externals - 1; n >= 0; n--) {
+        struct osd_external *e = obj->externals[n];
+        if (e->ov.owner == owner) {
+            destroy_external(e);
+            MP_TARRAY_REMOVE_AT(obj->externals, obj->num_externals, n);
+            obj->changed = true;
+            osd->want_redraw_notification = true;
+        }
+    }
+    pthread_mutex_unlock(&osd->lock);
+}
+
 static void append_ass(struct ass_state *ass, struct mp_osd_res *res,
                        ASS_Image **img_list, bool *changed)
 {
@@ -574,8 +609,8 @@ void osd_object_get_bitmaps(struct osd_state *osd, struct osd_object *obj,
 
     append_ass(&obj->ass, &obj->vo_res, &obj->ass_imgs[0], &obj->changed);
     for (int n = 0; n < obj->num_externals; n++) {
-        append_ass(&obj->externals[n].ass, &obj->vo_res, &obj->ass_imgs[n + 1],
+        append_ass(&obj->externals[n]->ass, &obj->vo_res,
-                   &obj->changed);
+                   &obj->ass_imgs[n + 1], &obj->changed);
     }
 
     mp_ass_packer_pack(obj->ass_packer, obj->ass_imgs, obj->num_externals + 1,

sub/osd_state.h

@@ -38,7 +38,7 @@ struct osd_object {
     struct dec_sub *sub;
 
     // OSDTYPE_EXTERNAL
-    struct osd_external *externals;
+    struct osd_external **externals;
     int num_externals;
 
     // OSDTYPE_EXTERNAL2
@@ -56,9 +56,7 @@ struct osd_object {
 };
 
 struct osd_external {
-    void *id;
+    struct osd_external_ass ov;
-    char *text;
-    int res_x, res_y;
     struct ass_state ass;
 };