Even for successful calls, utils.format_json returns nil as the second
return value. This doesn't have any purpose and is not documented, and
it is inconvenient for passing JSON to script-message, because
mp.commandv('script-message', 'foo', utils.format_json(...))
errors because mp.commandv receives nil as the fourth argument.
This commit makes format_json return only one value in case of success
to fix this.
This only existed as essentially a workaround for meson's behavior and
to maintain compatibility with the waf build. Since waf put everything
in a generated subdirectory, we had to put make a subdirectory called
"generated" in the source for meson so stuff could go to the right
place. Well now we don't need to do that anymore. Move the meson.build
files around so they go in the appropriate place in the subdirectory of
the source tree and change the paths of the headers accordingly. A
couple of important things to note.
1. mpv.com now gets made in build/player/mpv.com (necessary because of
a meson limitation)
2. The macos icon generation path is shortened to
TOOLS/osxbundle/icon.icns.inc.
Normally there was no issue, but when the code converted a deeply
nested table into an mpv node - it didn't ensure the stack has room.
Lua doesn't check stack overflow when invoking lua_push* functions,
and leaves this responsibility to the (c) user via lua_checkstack.
Normally that's not an issue because when a lua (or autofree) function
is called, it's guaranteed at least LUA_MINSTACK (20) pushes.
However, pushnode and makenode are recursive, and each iteration can
add few values at the stack (which are popped when the recursion
unwinds), so checkstack must be used on (recursive) entry.
pushnode already checked the stack, makenode did not.
This commit checks the stack at makenode as well. The value of 6
(stack places to reserve) is with some room to spare, and in pratice
each iteration needs 2-3 at most (pushnode also leaves room).
Example which could previously corrupt the stack:
utils.format_json({d1={d2={<8 more times>}}}
This uses makenode to convert the lua table into an mpv node which
the json writer uses as input, and if the depth is 10 or more then
corruption could occur. mp.command_native is also affected, as well as
any other mp/utils command which takes a lua table as input.
While at it, fix the error string which pushnode used (luaL_checkstack
uses the provided string with "Stack overflow (%s)", so the user
message only needs to be additional info).
The speedup is due to moving big part of the autofree runtime overhead
(new lua c closure) to happen once on init at af_pushcclosure, instead
of on every call. This also allows supporting upvalues trivially, even
if mpv doesn't use it currently, and is more consistent with lua APIs.
While x2 infrastructure speedup is meaningful - and similar between
lua 5.1/5.2/jit, in practice it's a much smaller improvement because
the autofree overhead is typically small compared to the "real" work
(the actual functionality, and talloc+free which is needed anyway).
So with this commit, fast functions improve more in percentage.
E.g. utils.parse_json("0") is relatively fast and is now about 25%
faster, while a slower call like mp.command_native("ignore") is now
about 10% faster than before. If we had mp.noop() which does nothing
(but still "needs" alloc/free) - it would now be about 50% faster.
Overall, it's a mild performance improvements, the API is now more
consistent with lua, and without increasing code size or complexity.
We now have at least 3 scripting APIs which are trivial wrappers
around properties: mp.get_mouse_pos, utils.getcwd, utils.getpid.
After some discussion on IRC it was decided that it's easier for us to
maintain them as trivial wrappers than to deprecate them and inflict
pain on users and script authors, so currently no plan to deprecate.
mp.get_mouse_pos() is undocumented and is no longer required - the
property can be used officially by any client now, however, osc.lua
uses it, and also some user scripts learnt to rely on it, so we keep
it - as a trivial wrapper around the new mouse-pos property.
lua/js utils.get_env_list() uses `environ' which was ANSI, thus
it broke any unicode names/values.
mpv already has an internal utf8_environ for win32, but it's used
only at the getenv(..) wrapper and not exposed in itself, and also it
has lazy initialization - on first getenv() call.
Now `environ' maps to a function which ensures initialization while
keeping it an l-value (like posix expects).
The cost of this fuglyness is that files should include osdep/io.h
(which now declares environ as extern) rather than declaring it
themselves, or else the build will break on mingw.
This was a vague idea how to handle passing byte arrays from Lua to the
mpv command interface in a somewhat reasonable way. The idea was
cancelled, but leave the Lua part of it, since it might get useful
later, and prevents passing (and silently cutting off) such byte
strings.
Barely tested, let's say not tested at all.
This is taken from a somewhat older proof-of-concept script. The basic
idea, and most of the implementation, is still the same. The way the
profiles are actually defined changed.
I still feel bad about this being a Lua script, and running user
expressions as Lua code in a vaguely defined environment, but I guess as
far as balance of effort/maintenance/results goes, this is fine.
It's a bit bloated (the Lua scripting state is at least 150KB or so in
total), so in order to enable this by default, I decided it should
unload itself by default if no auto-profiles are used. (And currently,
it does not actually rescan the profile list if a new config file is
loaded some time later, so the script would do nothing anyway if no auto
profiles were defined.)
This still requires defining inverse profiles for "unapplying" a
profile. Also this is still somewhat racy. Both will probably be
alleviated to some degree in the future.
Bill Gates did not only create COVID, he's also responsible for the
world's worst OS, where you have to literally jump through hoops of fire
to open files with Unicode file names. Lua did not care to implement any
jumping, so it's our turn to jump.
Untested (on win32).
Fixes: #7701
This is the proper fix for 1e7802. Turns out the solution is dead
simple: we can still set the allocator with lua_getallocf /
lua_setalloc.
This commit makes memory accounting work on luajit as well.
This is a stopgap measure. In theory we could maybe poll the memory
usage on luajit, but for now, simply reverting this part of fd3caa26
makes Lua work again. (And we can still collect cpu usage metrics)
Proper solution pending (tm)
Add an infrastructure for collecting performance-related data, use it in
some places. Add rendering of them to stats.lua.
There were two main goals: minimal impact on the normal code and normal
playback. So all these stats_* function calls either happen only during
initialization, or return immediately if no stats collection is going
on. That's why it does this lazily adding of stats entries etc. (a first
iteration made each stats entry an API thing, instead of just a single
stats_ctx, but I thought that was getting too intrusive in the "normal"
code, even if everything gets worse inside of stats.c).
You could get most of this information from various profilers (including
the extremely primitive --dump-stats thing in mpv), but this makes it
easier to see the most important information at once (at least in
theory), partially because we know best about the context of various
things.
Not very happy with this. It's all pretty primitive and dumb. At this
point I just wanted to get over with it, without necessarily having to
revisit it later, but with having my stupid statistics.
Somehow the code feels terrible. There are a lot of meh decisions in
there that could be better or worse (but mostly could be better), and it
just sucks but it's also trivial and uninteresting and does the job. I
guess I hate programming. It's so tedious and the result is always shit.
Anyway, enjoy.
There's no more need to call mp_lua_PITA to get the ctx, and the
autofree prototype is now enforced at the C level at compile time.
Also remove the redundant talloc_free_children at these functions
since it's now freed right after the function completes.
Also, rename auto_free_node to steal_node_allocations to be more
explicit and to avoid confusion with the autofree terminology.
Advantages of this approach:
- All the resources are released right after the function ends
regardless if it threw an error or not, without having to wait
for GC.
- Simpler code.
- Simpler lua setup which most likely uses less memory allocation and
as a result should be quicker, though it wasn't measured.
This commit adds the autofree wrapper and uses it where mp_lua_PITA
was used. It's not yet enforced at the C level, there are still
redundant talloc_free_children leftovers, and there are few more
places which could also use autofree. The next commits will address
those.
Lua changed behavior for this specific event. I considered the change
minor enough that it would not need to go through deprecation, but
someone hit it immediately and ask on the -dev channel.
It's probably better to restore the behavior. But mark it as deprecated,
since it's problematic (mismatch with the C API). Unfortunately, no
automatic warning is possible. (Or maybe it is, by playing sophisticated
Lua tricks such as setting a metatable and overriding indexing, but
let's not.)
Move some parts that can be generic to the client API code. It turns out
lua.c doesn't need anything special.
This adds the "id" field. I think this was actually missing from the
JSON IPC code (i.e. it's a very recent regression that is fixed with
this commit).
Both Lua and the JSON IPC code need to convert the mpv_event struct (and
everything it points to) to Lua tables or JSON.
I was getting sick of having to make the same changes to Lua and IPC. Do
what has been done everywhere else, and let the core handle this by
going through mpv_node (which is supposed to serve both Lua tables and
JSON, and potentially other scripting language backends). Expose it as
new libmpv API function.
The new API is still a bit "rough" and support for other event types
might be added in the future.
This silently adds support for the playlist_entry_id fields to both Lua
and JSON IPC.
There is a small API change for Lua; I don't think this matters, so I
didn't care about compatibility. The new code in client.c is mashed up
from the Lua and the IPC code. The manpage additions are moved from the
Lua docs, and made slightly more "general".
Some danger for unintended regressions both in Lua and IPC. Also damn
these node functions suck, expect crashes due to UB.
Not sure why this became more code instead of less compared to before
(according to the diff stat), even though some code duplication across
Lua and IPC was removed. Software development sucks.
It appears Lua's package paths try to load .lua files from the current
working directory. Not only that, but also shared libraries.
WHAT THE FUCK IS WHOEVER IS RESPONSIBLE FOR THIS FUCKING DOING?
mpv isn't setting this package path; currently it's only extending it.
In any sane world, this wouldn't be a default. Most programs use
essentially random working directories and don't change it.
I cannot comprehend what bullshit about "convenience" or whatever made
them do something this broken and dangerous. Thousands of programs using
Lua out there will try to randomly load random code from random
directories.
In mpv's case, this is so security relevant, because mpv is normally
used from the command line, and you will most likely actually change
into your media directory or whatever with the shell, and play a file
from there. No, you don't want to load a (probably downloaded) shared
library from this directory if a script try to load a system lib with
the same name or so.
I'm not sure why LUA_PATH_DEFAULT in luaconf.h (both upstream and the
Debian version) put "./?.lua" at the end, but in any case, trying to
load a module that doesn't exist nicely lists all package paths in
order, and confirms it tries to load files from the working directory
first (anyone can try this). Even if it didn't, this would be
problematic at best.
Note that scripts are _not_ sandboxed. They're allowed to load system
libraries, which is also why we want to keep the non-idiotic parts of
the package paths.
Attempt to fix this by filtering out relative paths. This is a bit
fragile and not very great for something security related, but probably
the best we can do without having to make assumptions about the target
system file system layout. Also, someone else can fix this for Windows.
Also replace ":" with ";" (for the extra path). On a side note, this
extra path addition is just in this function out of laziness, since
I'd rather not have 2 functions with edit the package path.
mpv in default configuration (i.e. no external scripts) is probably not
affected. All builtin scripts only "require" preloaded modules, which,
in a stroke of genius by the Lua developers, are highest priority in the
load order. Otherwise, enjoy your semi-remote code execution bug.
Completely unrelated this, I'm open for scripting languages and
especially implementations which are all around better than Lua, and are
suited for low footprint embedding.
The intention is to provide a slightly nicer way to distribute scripts.
For example, you could put multiple source files into the directory, and
then import them from the actual script file (this is still
unimplemented).
At first I wanted to require a config file (because you need to know at
least which scripting backend it should use). This wouldn't have been
too hard (could have reused/abused the mpv config file parsing
mechanism, and I already had working code that was just 2 function
calls). But probably better to do this without new config files, because
it might become a pain in the distant future.
So this just probes for "main.lua", "main.js", etc., until an existing
file is found.
Another important change is that this skips all directory entries whose
name starts with ".". This automatically excludes the "." and ".."
special directories, and is probably useful to exclude random crap that
might be lying around in the directory (such as editor temporary files,
or OSX, in its usual hrmful, annoying, and idiotic modus operandi,
sharting all over any directories opened by "Finder").
Although the changelog mentions the docs, they're added only in a later
commit.
See previous commit.
A nice side-effect is that mp.get_osd_margins() is not a special
Lua-only thing anymore. I didn't test whether this function still works
as expected, though.
Lua scripting has an undocumented mp.set_osd_ass() function, which is
used by osc.lua and console.lua. Apparently, 3rd party scripts also use
this. It's probably time to make this a public API.
The Lua implementation just bypassed the libmpv API. To make it usable
by any type of client, turn it into a command, "osd-overlay".
There's already a "overlay-add". Ignore it (although the manpage admits
guiltiness). I don't really want to deal with that old command. Its main
problem is that it uses global IDs, while I'd like to avoid that scripts
mess with each others overlays (whether that is accidentally or
intentionally). Maybe "overlay-add" can eventually be merged into
"osd-overlay", but I'm too lazy to do that now.
Scripting now uses the commands. There is a helper to manage OSD
overlays. The helper is very "thin"; I only want to force script authors
to use the ID allocation, which may help with putting multiple scripts
into a single .lua file without causing conflicts (basically, avoiding
singletons within a script's environment). The old set_osd_ass() is
emulated with the new API.
The JS scripting wrapper also provides a set_osd_ass() function, which
calls internal mpv API. Comment that part (to keep it compiling), but
I'm leaving it to @avih to finish the change.
Merged from mpv-repl git repo commit 5ea2bf64f9c239f0326b02. Some
changes were made on top of it:
- Tabs were converted to 4 spaces indentation (plus some manual
indentation fixes in some places).
- All user-visible mentions of "repl" were renamed to "console".
- The README was converted to a manpage (with heavy changes, some
additions taken from stats.rst; rossy converted the key bindings
table to RST).
- The method to change the default key binding was changed.
- Change minor detail about "font" default value setting (not a
functional change).
- Integrate into the player as builtin script, including an option to
prevent loading it.
Above changes and commit message done by wm4.
Signed-off-by: wm4 <wm4@nowhere>
This will just make it not work if mpv_request_log_messages() gets
extended to accept more names. Pass the argument without checking.
To keep the behavior the same (for whatever reasons, probably not
important), still raise an error if the libmpv API function returns an
error that the argument was bad.
(The check_loglevel() function is still used when the script _emits_ log
messages, which is different, and for which there is no API anyway.)
The "run" command is old. I'm not sure why the separate Lua
implementation was added. But maybe it as because the "run" command used
to be limited to a small number of arguments. This limit has been
removed a while ago. In any case, the old implementation is not needed
anymore.
Guido Cella
Kacper Michajłow
Dudemanguy
cvzi
Alexander Seiler
rcombs
Cœur
sfan5
Avi Halachmi (:avih)
wm4
Niklas Haas
Sai Ke WANG
James Ross-Gowan