mars/docu/screener.help

201 lines
7.1 KiB
Plaintext
Raw Normal View History

2018-04-23 13:25:32 +00:00
\begin{verbatim}
./screener.sh: Run _unattended_ processes in screen sessions.
Useful for MASS automation, running hundreds of unattended
commands in parallel.
HINT: for running more than ~500 sessions in parallel, you might need
some system tuning (e.g. rlimits, kernel patches etc) for creating
a huge number of file descritor / sockets / etc.
ADVANTAGE: You may attach to individual screens, kill them, or continue
some waiting commands.
Synopsis:
./screener.sh --help [--verbose]
./screener.sh list-running
./screener.sh list-waiting
2018-09-26 13:48:23 +00:00
./screener.sh list-interrupted
./screener.sh list-illegal
./screener.sh list-timeouted
2018-04-23 13:25:32 +00:00
./screener.sh list-failed
./screener.sh list-critical
./screener.sh list-serious
./screener.sh list-done
./screener.sh list
2018-09-26 13:48:23 +00:00
./screener.sh list-archive
2018-04-23 13:25:32 +00:00
./screener.sh list-screens
./screener.sh run <file.csv> [<condition_list>]
./screener.sh start <screen_id> <cmd> <args...>
./screener.sh [<options>] <operation> <screen_id>
Inquiry operations:
./screener.sh list-screens
Equivalent to screen -ls
./screener.sh list-<type>
Show a list of currently running, waiting (for continuation), failed,
and done/completed screen sessions.
./screener.sh list
First show a list of currently running screens, then
for each <type> a list of (old) failed / completed / sessions
(and so on).
./screener.sh status <screen_id>
Like list-*, but filter <sceen_id> and dont report timestamps.
./screener.sh show <screen_id>
Show the last logfile of <screen_id> at standard output.
./screener.sh less <screen_id>
Show the last logfile of <screen_id> using "less -r".
MASS starting of screen sessions:
./screener.sh run <file.csv> <condition_list>
Commands are launched in screen sessions via "./screener.sh start" commands,
unless the same <screen_id> is already running,
or is in some error state, or is already done (see below).
The commands are given by a column with CSV header name
containing "command", or by the first column.
The <screen_id> needs to be given by a column with CSV header
name matching "screen_id|resource".
The number and type of commands to launch can be reduced via
any combination of the following filter conditions:
--max=<number>
Limit the number of _new_ sessions additionally started this time.
--<column_name>==<value>
Only select lines where an arbitrary CSV column (given by its
CSV header name in C identifier syntax) has the given value.
--<column_name>!=<value>
Only select lines where the colum has _not_ the given value.
--<column_name>=~<bash_regex>
Only select lines where the bash regular expression matches
at the given column.
--max-per=<number>
Limit the number per _distinct_ value of the column denoted by
the _next_ filter condition.
Example: ./screener.sh run test.csv --dry-run --max-per=2 --dst_network=~.
would launch only 2 Football processes per destination network.
Hint: filter conditions can be easily checked by giving --dry-run.
Start / restart / kill / continue screen sessions:
./screener.sh start <screen_id> <cmd> <args...>
Start a new screen session, running arbitrary <cmd> and <args...>
inside.
./screener.sh restart <screen_id>
Works only when the last command for <screen_id> failed.
This will restart the old <cmd> and its <args...> as before.
Use only when you want to repeat the same command once again.
./screener.sh kill <screen_id>
Terminate the running screen session forcibly.
./screener.sh continue
./screener.sh continue <screen_id> [<screen_id_list>]
./screener.sh continue <number>
Useful for MASS automation of processes involving critical sections
such as customer downtime.
When giving a numerical <number> argument, up to that number
of sessions are resumed (ordered by age).
When no further arugment is given, _all_ currently waiting sessions
are continued.
When --auto-attach is given, it will sequentially resume the
sessions to be continued. By default, unless --force_attach is set,
it uses "screen -r" skipping those sessions which are already
attached to somebody else.
This feature works only with prepared scripts which are creating
an empty flagfile
/home/schoebel/mars/mars-migration.git/screener-logdir-testing/running/$screen_id.waiting
whenever they want to wait for manual intervention (for whatever reason).
Afterwards, the script must be polling this flagfile for removal.
This screener operation simply removes the flagfile, such that
the script will then continue afterwards.
Example: look into ./football.sh
and search for occurrences of substring "call_hook start_wait".
./screener.sh wakeup
./screener.sh wakeup <screen_id> [<screen_id_list>]
./screener.sh wakeup <number>
Similar to continue, but refers to delayed commands waiting for
a timeout. This can be used to individually shorten the timeout
period.
Example: Football cleanup operations may be artificially delayed
before doing "lvremove", to keep some sort of 'backup' for a
limited time. When your project is under time pressure, these
delays may be hindering.
Use this for premature ending of such artificial delays.
./screener.sh up <...>
Do both continue and wakeup.
./screener.sh auto <...>
Equivalent to ./screener.sh --auto-attach up <...>
Remember that only session without current attachment will be
attached to.
Attach to a running session:
./screener.sh attach <screen_id>
This is equivalent to screen -x $screen_id
./screener.sh resume <screen_id>
This is equivalent to screen -r $screen_id
Communication:
./screener.sh notify <screen_id> <txt>
May be called from external scripts to send emails etc.
Locking (only when supported by <cmd>):
./screener.sh lock
./screener.sh unlock
./screener.sh lock <screen_id>
./screener.sh unlock <screen_id>
Cleanup / bookkeeping:
./screener.sh clear-critical <screen_id>
./screener.sh clear-serious <screen_id>
2018-09-26 13:48:23 +00:00
./screener.sh clear-interrupted <screen_id>
./screener.sh clear-illegal <screen_id>
./screener.sh clear-timeouted <screen_id>
2018-04-23 13:25:32 +00:00
./screener.sh clear-failed <screen_id>
Mark the status as "done" and move the logfile away.
./screener.sh purge [<days>]
This will remove all old logfiles which are older than
<days>. By default, the variable $screener_log_purge_period
will be used, which is currently set to '30'.
./screener.sh cron
You should call this regulary from a user cron job, in order
to purge old logfiles, or to detect hanging sessions, or to
automatically send pending emails, etc.
Options:
--variable
--variable=$value
These must come first, in order to prevent mixup with
options of <cmd> <args...>.
Allows overriding of any internal shell variable.
--help --verbose
Show all overridable shell variables, also for plugins.
PLUGIN screener-email
Generic plugin for sending emails (or SMS via gateways)
upon status changes, such as script failures.
\end{verbatim}