Commit Graph

222 Commits

Author SHA1 Message Date
Willy Tarreau
00dd44f67f MINOR: activity: add a "memory" entry to "profiling"
This adds the necessary flags to permit run-time enabling/disabling of
memory profiling. For now this is disabled.

A few words were added to the management doc about it and recalling that
this is limited to certain OSes.
2021-05-05 18:55:02 +02:00
Willy Tarreau
bb51c44d64 MINOR: map/acl: make "add map/acl" support an optional version number
By passing a version number to "add map/acl", it becomes possible to
atomically replace maps and ACLs. The principle is that a new version
number is first retrieved by calling"prepare map/acl", and this version
number is used with "add map" and "add acl". Newly added entries then
remain invisible to the matching mechanism but are visible in "show
map/acl" when the version number is specified, or may be cleard with
"clear map/acl". Finally when the insertion is complete, a
"commit map/acl" command must be issued, and the version is atomically
updated so that there is no intermediate state with incomplete entries.
2021-04-30 15:36:31 +02:00
Willy Tarreau
7a562ca809 MINOR: map/acl: add the "commit map/acl" CLI command
The command is used to atomically replace a map/acl with the pending
contents of the designated version. The new version must have been
allocated by "prepare map/acl" prior to this. At the moment it is not
possible to force the version when adding new entries, so this may only
be used to atomically clear an ACL/map.
2021-04-30 15:36:31 +02:00
Willy Tarreau
97218ce3a9 MINOR: map/acl: add the "prepare map/acl" CLI command
This command allocates a new version for the map/acl, that will be usable
later to prepare the addition of new values to atomically replace existing
ones. Technically speaking the operation consists in atomically incrementing
the next version. There's no "undo" operation here, if a version is not
committed, it will automatically be trashed when committing a newer version.
2021-04-30 15:36:31 +02:00
Willy Tarreau
ff3feeb5cf MINOR: map/acl: add the possibility to specify the version in "clear map/acl"
This will ease maintenance of versionned maps by allowing to clear old or
failed updates instead of the current version. Nothing was done to allow
clearing everyhing, though if there was a need for this, implementing "@all"
or something equivalent wouldn't require more than 3 lines of code.
2021-04-30 15:36:31 +02:00
Willy Tarreau
95f753e403 MINOR: map/acl: add the possibility to specify the version in "show map/acl"
The maps and ACLs internally all have two versions, the "current" one,
which is the one being matched against, and the "next" one, the one being
filled during an atomic replacement. Till now the "show" commands only used
to show the current one but it can be convenient to be able to show other
ones as well, so let's add the ability to do this with "show map" and
"show acl". The method used here consists in passing the version number
as "@<ver>" before the map/acl name or ID. It would have been better after
it but that could create confusion with keys already using such a format.
2021-04-30 15:36:31 +02:00
Willy Tarreau
4000ff0448 DOC: management: move "set var" to the proper place
Commit b8bd1ee89 ("MEDIUM: cli: add a new experimental "set var" command")
added "get var" and "set var" but "set var" was misplaced in the doc,
breaking the alphabetic ordering.
2021-04-30 15:36:31 +02:00
Amaury Denoyelle
eafd701dc5 MINOR: server: fix doc/trace on lb algo for dynamic server creation
The text mentionned that only backends with consistent hash method were
supported for dynamic servers. In fact, it is only required that the lb
algorith is dynamic.
2021-04-29 14:59:42 +02:00
Ilya Shipitsin
b2be9a1ea9 CLEANUP: assorted typo fixes in the code and comments
This is 22nd iteration of typo fixes
2021-04-26 10:42:58 +02:00
Amaury Denoyelle
e558043e13 MINOR: server: implement delete server cli command
Implement a new CLI command 'del server'. It can be used to removed a
dynamically added server. Only servers in maintenance mode can be
removed, and without pending/active/idle connection on it.

Add a new reg-test for this feature. The scenario of the reg-test need
to first add a dynamic server. It is then deleted and a client is used
to ensure that the server is non joinable.

The management doc is updated with the new command 'del server'.
2021-04-21 11:00:31 +02:00
Remi Tricot-Le Breton
b5f0fac765 DOC: ssl: Certificate hot update works on server certificates
The CLI's "set ssl cert" command also works on backend certificates
(see GitHub issue #427).

It does not need to be backported.
2021-04-15 17:52:08 +02:00
Remi Tricot-Le Breton
3445909a63 DOC: ssl: Certificate hot update only works on fronted certificates
The CLI's "set ssl cert" command only works on frontend certificates but
the documentation did not specify this limitations yet.

This patch can be backported to all stable branches.
2021-04-15 17:52:08 +02:00
Amaury Denoyelle
7b01a8dbdd MINOR: global: define diagnostic mode of execution
Define MODE_DIAG which is used to run haproxy in diagnostic mode. This
mode is used to output extra warnings about possible configuration
blunder or sub-optimal usage. It can be activated with argument '-dD'.

A new output function ha_diag_warning is implemented reserved for
diagnostic output. It serves to standardize the format of diagnostic
messages.

A macro HA_DIAG_WARN_COND is also available to automatically check if
diagnostic mode is on before executing the diagnostic check.
2021-04-01 18:03:37 +02:00
Willy Tarreau
b8bd1ee893 MEDIUM: cli: add a new experimental "set var" command
set var <name> <expression>
  Allows to set or overwrite the process-wide variable 'name' with the result
  of expression <expression>. Only process-wide variables may be used, so the
  name must begin with 'proc.' otherwise no variable will be set. The
  <expression> may only involve "internal" sample fetch keywords and converters
  even though the most likely useful ones will be str('something') or int().
  Note that the command line parser doesn't know about quotes, so any space in
  the expression must be preceeded by a backslash. This command requires levels
  "operator" or "admin". This command is only supported on a CLI connection
  running in experimental mode (see "experimental-mode on").

Just like for "set-var" in the global section, the command uses a temporary
dummy proxy to create a temporary "set-var(name)" rule to assign the value.

The reg test was updated to verify that an updated global variable is properly
reflected in subsequent HTTP responses.
2021-03-26 16:57:43 +01:00
Willy Tarreau
c35eb38f1d MINOR: vars/cli: add a "get var" CLI command to retrieve global variables
Process-wide variables can now be displayed from the CLI using "get var"
followed by the variable name. They must all start with "proc." otherwise
they will not be found. The output is very similar to the one of the
debug converter, with a type and value being reported for the embedded
sample.

This command is limited to clients with the level "operator" or higher,
since it can possibly expose traffic-related data.
2021-03-26 16:52:13 +01:00
Ilya Shipitsin
ba13f16aa2 CLEANUP: assorted typo fixes in the code and comments
This is 21st iteration of typo fixes
2021-03-20 09:28:58 +01:00
Amaury Denoyelle
304672320e MINOR: server: support keyword proto in 'add server' cli
Allow to specify the mux proto for a dynamic server. It must be
compatible with the backend mode to be accepted. The reg-tests has been
extended for this error case.
2021-03-18 16:22:10 +01:00
Amaury Denoyelle
fc465a54fd MINOR: server: enable standard options for dynamic servers
Enable a subset of server options to be used as keywords on the CLI
command 'add server'. These options are safe and can be applied
flawlessly for a dynamic server.
2021-03-18 16:22:10 +01:00
Amaury Denoyelle
f99f77a500 MEDIUM: server: implement 'add server' cli command
Add a new cli command 'add server'. This command is used to create a new
server at runtime attached on an existing backend. The syntax is the
following one :

$ add server <be_name>/<sv_name> [<kws>...]

This command is only available through experimental mode for the moment.

Currently, no server keywords are supported. They will be activated
individually when deemed properly functional and safe.

Another limitation is put on the backend load-balancing algorithm. The
algorithm must use consistent hashing to guarantee a minimal
reallocation of existing connections on the new server insertion.
2021-03-18 15:52:07 +01:00
Amaury Denoyelle
18487fb532 MINOR: cli: implement experimental-mode
Experimental mode is similar to expert-mode. It can be used to access to
features still in development.
2021-03-18 15:37:05 +01:00
Willy Tarreau
49962b58d0 MINOR: peers/cli: do not dump the peers dictionaries by default on "show peers"
The "show peers" output has become huge due to the dictionaries making it
less readable. Now this feature has reached a certain level of maturity
which doesn't warrant to dump it all the time, given that it was essentially
needed by developers. Let's make it optional, and disabled by default, only
when "show peers dict" is requested. The default output reminds about the
command. The output has been divided by 5 :

  $ socat - /tmp/sock1  <<< "show peers dict" | wc -l
  125
  $ socat - /tmp/sock1  <<< "show peers" | wc -l
  26

It could be useful to backport this to recent stable versions.
2021-02-12 17:00:52 +01:00
William Dauchy
d1a7b85a40 MEDIUM: server: support {check,agent}_addr, agent_port in server state
logical followup from cli commands addition, so that the state server
file stays compatible with the changes made at runtime; use previously
added helper to load server attributes.

also alloc a specific chunk to avoid mixing with other called functions
using it

Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-12 16:04:52 +01:00
William Dauchy
7cabc06da6 MEDIUM: cli: add agent-port command
this patch allows to set agent port at runtime. In order to align with
both `addr` and `check-addr` commands, also add the possibility to
optionnaly set port on `agent-addr` command. This led to a small
refactor in order to use the same function for both `agent-addr` and
`agent-port` commands.

Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-12 16:04:52 +01:00
William Dauchy
b456e1f389 MEDIUM: cli: add check-addr command
this patch allows to set server health check address at runtime. In
order to align with `addr` command, also allow to set port optionnaly.
This led to a small refactor in order to use the same function for both
`check-addr` and `check-port` commands.
for `check-port`, we however don't permit the change anymore if checks
are not enabled on the server.

This command becomes more and more useful for people having a consul
like architecture:
- the backend server is located on a container with its own IP
- the health checks are done the consul instance located on the host
  with the host IP

Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-12 16:04:52 +01:00
William Dauchy
fe03e7d045 MEDIUM: server: adding support for check_port in server state
We can currently change the check-port using the cli command `set server
check-port` but there is a consistency issue when using server state.
This patch aims to fix this problem but will be also a good preparation
work to get rid of checkport flag, so we are able to know when checkport
was set by config.

I am fully aware this is not making github #953 moving forward, I
however think this might be acceptable while waiting for a proper
solution and resolve consistency problem faced with port settings.

Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-04 10:46:52 +01:00
Willy Tarreau
7eff06e162 MINOR: activity: add a new "show tasks" command to list currently active tasks
This finally adds the long-awaited solution to inspect the run queues
and figure what is eating the CPU or causing latencies. We can even see
the experienced latencies when profiling is enabled. Example on a
saturated process:

> show tasks
Running tasks: 14983 (4 threads)
  function                     places     %    lat_tot   lat_avg
  process_stream                 4948   33.0   5.840m    70.82ms
  h1_io_cb                       2535   16.9      -         -
  main+0x9e670                   2508   16.7   2.930m    70.10ms
  ssl_sock_io_cb                 2499   16.6      -         -
  si_cs_io_cb                    2493   16.6      -         -
2021-01-29 12:12:28 +01:00
Willy Tarreau
cfa7101d59 MINOR: activity: flush scheduler stats on "set profiling tasks on"
If a user enables profiling by hand, it makes sense to reset the stats
counters to provide fresh new measurements. Therefore it's worth using
this as the standard method to reset counters.
2021-01-29 12:10:33 +01:00
Willy Tarreau
1bd67e9b03 MINOR: activity: also report collected tasks stats in "show profiling"
"show profiling" will now dump the stats collected by the scheduler if
profiling was previously enabled. This will immediately make it obvious
what functions are responsible for others' high latencies or which ones
are suffering from others, and should help spot issues like undesired
wakeups.

Example:

Per-task CPU profiling              : on      # set profiling tasks {on|auto|off}
Tasks activity:
  function                      calls   cpu_tot   cpu_avg   lat_tot   lat_avg
  si_cs_io_cb                 5569479   23.37s    4.196us      -         -
  h1_io_cb                    5558654   13.60s    2.446us      -         -
  process_stream               250841   1.476s    5.882us   3.499s    13.95us
  main+0x9e670                    198      -         -      5.526ms   27.91us
  task_run_applet                  17   1.509ms   88.77us   205.8us   12.11us
  srv_cleanup_idle_connections     12   44.51us   3.708us   25.71us   2.142us
  main+0x158c80                     9   48.72us   5.413us      -         -
  srv_cleanup_toremove_connections  5   165.1us   33.02us   123.6us   24.72us
2021-01-29 12:10:33 +01:00
Willy Tarreau
87ef323971 DOC: management: fix "show resolvers" alphabetical ordering
Not sure why it was located between "show ssl" and "show table"...
This should be backported.
2021-01-29 12:10:33 +01:00
Willy Tarreau
8050efeacb MINOR: cli: give the show_fd helpers the ability to report a suspicious entry
Now the show_fd helpers at the transport and mux levels return an integer
which indicates whether or not the inspected entry looks suspicious. When
an entry is reported as suspicious, "show fd" will suffix it with an
exclamation mark ('!') in the dump, that is supposed to help detecting
them.

For now, helpers were adjusted to adapt to the new API but none of them
reports any suspicious entry yet.
2021-01-21 08:58:15 +01:00
Thayne McCombs
cdbcca9995 DOC: fix some spelling issues over multiple files
This is from the output of codespell and may be backported.
2021-01-08 14:53:47 +01:00
Ilya Shipitsin
2272d8aeea DOC: assorted typo fixes in the documentation
This is another round of cleanups in various docs
2020-12-21 11:24:56 +01:00
Remi Tricot-Le Breton
e3e1e5f34b MINOR: cache: Dump secondary entries in "show cache"
The duplicated entries (in case of vary) were not taken into account by
the "show cache" command. They are now dumped too.
A new "vary" column is added to the output. It contains the complete
seocndary key (in hex format).
2020-12-10 15:59:49 +01:00
Phil Scherer
b931f964e1 DOC/MINOR: Fix formatting in Management Guide
section numbering used '9.2)' instead of '9.2.'.
2020-12-03 06:40:51 +01:00
William Dauchy
f63704488e MEDIUM: cli/ssl: configure ssl on server at runtime
in the context of a progressive backend migration, we want to be able to
activate SSL on outgoing connections to the server at runtime without
reloading.
This patch adds a `set server ssl` command; in order to allow that:

- add `srv_use_ssl` to `show servers state` command for compatibility,
  also update associated parsing
- when using default-server ssl setting, and `no-ssl` on server line,
  init SSL ctx without activating it
- when triggering ssl API, de/activate SSL connections as requested
- clean ongoing connections as it is done for addr/port changes, without
  checking prior server state

example config:

backend be_foo
  default-server ssl
  server srv0 127.0.0.1:6011 weight 1 no-ssl

show servers state:

  5 be_foo 1 srv0 127.0.0.1 2 0 1 1 15 1 0 4 0 0 0 0 - 6011 - -1

where srv0 can switch to ssl later during the runtime:

  set server be_foo/srv0 ssl on

  5 be_foo 1 srv0 127.0.0.1 2 0 1 1 15 1 0 4 0 0 0 0 - 6011 - 1

Also update existing tests and create a new one.

Signed-off-by: William Dauchy <wdauchy@gmail.com>
2020-11-18 17:22:28 +01:00
Willy Tarreau
1db5579bf8 [RELEASE] Released version 2.4-dev0
Released version 2.4-dev0 with the following main changes :
    - MINOR: version: it's development again.
    - DOC: mention in INSTALL that it's development again
2020-11-05 17:20:35 +01:00
Daniel Corbett
c40edacbda DOC: Add dns as an available domain to show stat
Within management.txt, proxy was listed as the only available option. "dns"
is now supported so let's add that. This change also updates the command to list
the available options <dns|proxy> for "domain" as previously it only specified
<domain>, which could be confusing as a user may think this field accepts
dynamic options when it actually requires a specific keyword.
2020-11-05 16:46:24 +01:00
Willy Tarreau
bd71510024 MINOR: stats: report server's user-configured weight next to effective weight
The "weight" column on the stats page is somewhat confusing when using
slowstart becaue it reports the effective weight, without being really
explicit about it. In some situations the user-configured weight is more
relevant (especially with long slowstarts where it's important to know
if the configured weight is correct).

This adds a new uweight stat which reports a server's user-configured
weight, and in a backend it receives the sum of all servers' uweights.
In addition it adds the mention of "effective" in a few descriptions
for the "weight" column (help and doc).

As a result, the list of servers in a backend is now always scanned
when dumping the stats. But this is not a problem given that these
servers are already scanned anyway and for way heavier processing.
2020-10-23 22:47:30 +02:00
Willy Tarreau
698097b54c DOC: management: apply the "show stat" modifiers to "show stat", not "show info"
By mistake I added the "up" then "maint" output modifiers to the "show info"
block instead of the "show stat" one in the two previous commits 65141ffc4
("MINOR: stats: support the "up" output modifier for "show stat"") and
3e3203670 ("MINOR: stats: also support a "no-maint" show stat modifier").

No backport is needed.
2020-10-23 20:22:33 +02:00
Willy Tarreau
3e32036701 MINOR: stats: also support a "no-maint" show stat modifier
"no-maint" is a bit similar to "up" except that it will only hide
servers that are in maintenance (or disabled in the configuration), and
not those that are enabled but failed a check. One benefit here is to
significantly reduce the output of the "show stat" command when using
large server-templates containing entries that are not yet provisioned.

Note that the prometheus exporter also has such an option which does
the exact same.
2020-10-23 18:11:24 +02:00
Willy Tarreau
65141ffc4f MINOR: stats: support the "up" output modifier for "show stat"
We already had it on the HTTP interface but it was not accessible on the
CLI. It can be very convenient to hide servers which are down, do not
resolve, or are in maintenance.
2020-10-23 18:11:24 +02:00
Willy Tarreau
ccf429960b MEDIUM: config: remove the deprecated and dangerous global "debug" directive
This one was scheduled for removal in 2.3 since 2.2-dev3 by commit
1b85785bc ("MINOR: config: mark global.debug as deprecated"). Let's
remove it now. It remains totally possible to use -d on the command
line though.
2020-10-09 19:18:45 +02:00
Pierre Cheynier
08eb718494 DOC: Add missing stats fields in the management doc
Added latest fields: idle_conn_cur, safe_conn_cur, used_conn_cur, need_conn_est
2020-10-09 09:56:37 +02:00
Amaury Denoyelle
fbd0bc98fe MINOR: dns/stats: integrate dns counters in stats
Use the new stats module API to integrate the dns counters in the
standard stats. This is done in order to avoid code duplication, keep
the code related to cli out of dns and use the full possibility of the
stats function, allowing to print dns stats in csv or json format.
2020-10-05 12:02:14 +02:00
Amaury Denoyelle
50660a894d MEDIUM: stats: add delimiter for static proxy stats on csv
Use the character '-' to mark the end of static statistics on proxy
domain. After this marker, the order of the fields is not guaranteed and
should be parsed with care.
2020-10-05 12:02:14 +02:00
Amaury Denoyelle
072f97eddf MINOR: stats: define the concept of domain for statistics
The domain option will be used to have statistics attached to other
objects than proxies/listeners/servers. At the moment, only the PROXY
domain is available.

Add an argument 'domain' on the 'show stats' cli command to specify the
domain. Only 'domain proxy' is available now. If not specified, proxy
will be considered the default domain.

For HTML output, only proxy statistics will be displayed.
2020-10-05 12:02:14 +02:00
William Lallemand
f9ff3ece83 DOC: ssl: new "cert bundle" behavior
Update the documentation with the new bundle behavior which does not use
the same OpenSSL certificate store anymore but loads the PEM separately
as multiple "crt" were specified.

It should fix issue #872.
2020-10-02 18:02:16 +02:00
William Dauchy
25407965fd DOC: crt: advise to move away from cert bundle
especially when starting to use `new ssl cert` runtime API, it might
become a bit confusing for users to mix bundle and single cert,
especially when it comes to use the commit command:
e.g.:
- start the process with `crt` loading a bundle
- use `set ssl cert my_cert.pem.ecdsa`: API detects it as a replacement
  of a bundle.
- `commit` has to be done on the bundle: `commit ssl cert my_cert.pem`

however:
- add a new cert: `new ssl cert my_cert.pem.rsa`: added as a single
  certificate
- `commit` has to be done on the certificate: `commit ssl cert
  my_cert.pem.rsa`

this should resolve github issue #872

this should probably be backported in >= v2.2 in order to encourage
people to move away from bundle certificates loading.

Signed-off-by: William Dauchy <w.dauchy@criteo.com>
2020-09-27 21:54:10 +02:00
Jackie Tapia
749f74c622 DOC: Use gender neutral language
This patch updates the documentation files and code comments to avoid
the use of gender specific phrasing in favor of "they" or "it".
2020-07-26 22:35:43 +02:00
Willy Tarreau
33205c23a7 [RELEASE] Released version 2.3-dev0
Released version 2.3-dev0 with the following main changes :
    - exact copy of 2.2.0
2020-07-07 16:35:28 +02:00
Willy Tarreau
69f591e3b0 MINOR: cli/proxy: add a new "show servers conn" command
This command reuses the existing "show servers state" to also dump the
state of active and idle connections. The main use is to serve as a
debugging tool to troubleshot connection reuse issues.
2020-07-01 10:32:54 +02:00
Willy Tarreau
c6e7a1b8e9 MINOR: cli: make "show sess" stop at the last known session
"show sess" and particularly "show sess all" can be very slow when dumping
lots of information, and while dumping, new sessions might appear, making
the output really endless. When threads are used, this causes a double
problem:
  - all threads are paused during the dump, so an overly long dump degrades
    the quality of service ;

  - since all threads are paused, more events get postponed, possibly
    resulting in more streams to be dumped on next invocation of the dump
    function.

This patch addresses this long-lasting issue by doing something simple:
the CLI's stream is moved at the end of the steams list, serving as an
identifiable marker to end the dump, because all entries past it were
added after the command was entered. As a result, the CLI's stream always
appears as the last one.

It may make sense to backport this to stable branches where dumping live
streams is difficult as well.
2020-06-28 01:41:56 +02:00
William Lallemand
c184d87558 DOC: ssl: update the documentation of "commit ssl cert"
Update the documentation of "commit ssl cert" in management.txt to
explain the behavior with new certificates.
2020-06-26 15:39:57 +02:00
Willy Tarreau
3eb10b8e98 MINOR: init: add -dW and "zero-warning" to reject configs with warnings
Since some systems switched to service managers which hide all warnings
by default, some users are not aware of some possibly important warnings
and get caught too late with errors that could have been detected earlier.

This patch adds a new global keyword, "zero-warning" and an equivalent
command-line option "-dW" to refuse to start in case any warning is
detected. It is recommended to use these with configurations that are
managed by humans in order to catch mistakes very early.
2020-04-15 16:42:39 +02:00
Willy Tarreau
bebd212064 MINOR: init: report in "haproxy -c" whether there were warnings or not
This helps quickly checking if the config produces any warning. For
this we reuse the "warned" bit field to add a new WARN_ANY bit that is
set by ha_warning(). The rest of the bit field was also cleaned from
unused bits.
2020-04-15 16:42:00 +02:00
William Lallemand
419e6349f6 MINOR: ssl/cli: 'del ssl cert' deletes a certificate
Delete a certificate store from HAProxy and free its memory. The
certificate must be unused and removed from any crt-list or directory.
The deletion doesn't work with a certificate referenced directly with
the "crt" directive in the configuration.
2020-04-08 12:08:03 +02:00
William Lallemand
c69f02d0f0 MINOR: ssl/cli: replace dump/show ssl crt-list by '-n' option
The dump and show ssl crt-list commands does the same thing, they dump
the content of a crt-list, but the 'show' displays an ID in the first
column. Delete the 'dump' command so it is replaced by the 'show' one.
The old 'show' command is replaced by an '-n' option to dump the ID.
And the ID which was a pointer is replaced by a line number and placed
after colons in the filename.

Example:
  $ echo "show ssl crt-list -n kikyo.crt-list" | socat /tmp/sock1 -
  # kikyo.crt-list
  kikyo.pem.rsa:1 secure.domain.tld
  kikyo.pem.ecdsa:2 secure.domain.tld
2020-04-06 19:33:33 +02:00
William Lallemand
0a9b9414f0 MINOR: ssl/cli: 'del ssl crt-list' delete an entry
Delete an entry in a crt-list, this is done by iterating over the
ckch_inst in the crtlist_entry. For each ckch_inst the bind_conf lock is
held during the deletion of the sni_ctx in the SNI trees. Everything
is free'd.

If there is several entries with the same certificate, a line number
must be provided to chose with entry delete.
2020-04-06 19:33:28 +02:00
William Lallemand
accac23aff DOC: management: add the new crt-list CLI commands
Add the new crt-lists command in the management manual:
"new ssl cert", "add ssl crt-list", "dump ssl crt-list", "show ssl
crt-list".
2020-04-02 17:50:02 +02:00
Ilya Shipitsin
2a950d02a9 DOC: assorted typo fixes in the documentation and Makefile
This is another round of cleanups in various docs and comments in the
Makefile.
2020-03-06 10:49:55 +01:00
Adis Nezirovic
1a693fc2fd MEDIUM: cli: Allow multiple filter entries for "show table"
For complex stick tables with many entries/columns, it can be beneficial
to filter using multiple criteria. The maximum number of filter entries
can be controlled by defining STKTABLE_FILTER_LEN during build time.

This patch can be backported to older releases.
2020-01-22 14:33:17 +01:00
Christopher Faulet
0159ee4032 MINOR: stats: Report internal errors in the proxies/listeners/servers stats
The stats field ST_F_EINT has been added to report internal errors encountered
per proxy, per listener and per server. It appears in the CLI export and on the
HTML stats page.
2020-01-20 15:18:45 +01:00
William Lallemand
d4f946c469 MINOR: ssl/cli: 'show ssl cert' give information on the certificates
Implement the 'show ssl cert' command on the CLI which list the frontend
certificates. With a certificate name in parameter it will show more
details.
2019-12-18 18:16:34 +01:00
William Lallemand
6ab08b3fd4 DOC: ssl/cli: set/commit/abort ssl cert
Document the "set/commit/abort ssl cert" CLI commands in management.txt.

Must be backported in 2.1.
2019-11-29 16:53:08 +01:00
Willy Tarreau
2e077f8d53 [RELEASE] Released version 2.2-dev0
Released version 2.2-dev0 with the following main changes :
    - exact copy of 2.1.0
2019-11-25 20:36:16 +01:00
Christopher Faulet
2ac2574409 DOC: Add missing stats fields in the management manual
Following fields was missing : srv_icur, src_ilim, qtime_max, ctime_max,
rtime_max and ttime_max.
2019-11-15 14:24:06 +01:00
Willy Tarreau
7297429fa5 DOC: management: fix typo on "cache_lookups" stats output
The trailing "s" was missing.
2019-11-08 07:29:34 +01:00
Willy Tarreau
b24ab22ac0 MINOR: debug: make most debug CLI commands accessible in expert mode
Instead of relying on DEBUG_DEV for most debugging commands, which is
limiting, let's condition them to expert mode. Only one ("debug dev exec")
remains conditionned to DEBUG_DEV because it can have a security implication
on the system. The commands are not listed unless "expert-mode on" was first
entered on the CLI :

 > expert-mode on
 > help
   debug dev close <fd>        : close this file descriptor
   debug dev delay [ms]        : sleep this long
   debug dev exec  [cmd] ...   : show this command's output
   debug dev exit  [code]      : immediately exit the process
   debug dev hex   <addr> [len]: dump a memory area
   debug dev log   [msg] ...   : send this msg to global logs
   debug dev loop  [ms]        : loop this long
   debug dev panic             : immediately trigger a panic
   debug dev stream ...        : show/manipulate stream flags
   debug dev tkill [thr] [sig] : send signal to thread

 > debug dev stream
 Usage: debug dev stream { <obj> <op> <value> | wake }*
      <obj>   = {strm | strm.f | sif.f | sif.s | sif.x | sib.f | sib.s | sib.x |
                 txn.f | req.f | req.r | req.w | res.f | res.r | res.w}
      <op>    = {'' (show) | '=' (assign) | '^' (xor) | '+' (or) | '-' (andnot)}
      <value> = 'now' | 64-bit dec/hex integer (0x prefix supported)
      'wake' wakes the stream asssigned to 'strm' (default: current)
2019-10-24 18:38:00 +02:00
Willy Tarreau
abb9f9b057 MINOR: cli: add an expert mode to hide dangerous commands
Some commands like the debug ones are not enabled by default but can be
useful on some production environments. In order to avoid the temptation
of using them incorrectly, let's introduce an "expert" mode for a CLI
connection, which allows some commands to appear and be used. It is
enabled by command "expert-mode on" which is not listed by default.
2019-10-24 18:38:00 +02:00
Willy Tarreau
6b19b142e8 MINOR: stats: make "show stat" and "show info"
Now "show info" supports "desc" after the default and "typed" formats,
and "show stat" supports this after the typed format. In both cases
this appends the description for the represented metric between double
quotes. The same could be done for JSON output but would possibly require
to update the schema first.
2019-10-10 11:30:07 +02:00
Christopher Faulet
78c4306165 DOC: Fix documentation about the cli command to get resolver stats
In the management guide, this command was still referenced as "show stat
resolvers" instead of "show resolvers". The cli command was fixed by the commit
ff88efbd7 ("BUG/MINOR: dns: Fix CLI keyword declaration").

This patch fixes the issue #296. It can be backported as fas as 1.7.
2019-09-27 10:45:48 +02:00
Willy Tarreau
1d181e489c MEDIUM: ring: implement a wait mode for watchers
Now it is possible for a reader to subscribe and wait for new events
sent to a ring buffer. When new events are written to a ring buffer,
the applets that are subscribed are woken up to display new events.
For now we only support this with the CLI applet called by "show events"
since the I/O handler is indeed a CLI I/O handler. But it's not
complicated to add other mechanisms to consume events and forward them
to external log servers for example. The wait mode is enabled by adding
"-w" after "show events <sink>". An extra "-n" was added to directly
seek to new events only.
2019-08-30 11:58:58 +02:00
Willy Tarreau
370a694879 MINOR: trace: change the detail_level to per-source verbosity
The detail level initially based on syslog levels is not used, while
something related is missing, trace verbosity, to indicate whether or
not we want to call the decoding callback and what level of decoding
we want (raw captures etc). Let's change the field to "verbosity" for
this. A verbosity of zero means that the decoding callback is not
called, and all other levels are handled by this callback and are
source-specific. The source is now prompted to list the levels that
are proposed to the user. When the source doesn't define anything,
"quiet" and "default" are available.
2019-08-29 17:11:25 +02:00
Willy Tarreau
09fb0df6fd MINOR: trace: prepend the function name for developer level traces
Working on adding traces to mux-h2 revealed that the function names are
manually copied a lot in developer traces. The reason is that they are
not preprocessor macros and as such cannot be concatenated. Let's
slightly adjust the trace() function call to take a function name just
after the file:line argument. This argument is only added for the
TRACE_DEVEL and 3 new TRACE_ENTER, TRACE_LEAVE, and TRACE_POINT macros
and left NULL for others. This way the function name is only reported
for traces aimed at the developers. The pretty-print callback was also
extended to benefit from this. This will also significantly shrink the
data segment as the "entering" and "leaving" strings will now be merged.

One technical point worth mentioning is that the function name is *not*
passed as an ist to the inline function because it's not considered as
a builtin constant by the compiler, and would lead to strlen() being
run on it from all call places before calling the inline function. Thus
instead we pass the const char * (that the compiler knows where to find)
and it's the __trace() function that converts it to an ist for internal
consumption and for the pretty-print callback. Doing this avoids losing
5-10% peak performance.
2019-08-29 17:09:13 +02:00
Willy Tarreau
2ea549bc43 MINOR: trace: change the "payload" level to "data" and move it
The "payload" trace level was ambigous because its initial purpose was
to be able to dump received data. But it doesn't make sense to force to
report data transfers just to be able to report state changes. For
example, all snd_buf()/rcv_buf() operations coming from the application
layer should be tagged at this level. So here we move this payload level
above the state transitions and rename it to avoid the ambiguity making
one think it's only about request/response payload. Now it clearly is
about any data transfer and is thus just below the developer level. The
help messages on the CLI and the doc were slightly reworded to help
remove this ambiguity.
2019-08-29 10:46:11 +02:00
Willy Tarreau
9f830d7408 MINOR: sink: implement "show events" to show supported sinks and dump the rings
The new "show events" CLI keyword lists supported event sinks. When
passed a buffer-type sink it completely dumps it.

no drops at all during attachment even at 8 millon evts/s.
still missing the attachment limit though.
2019-08-27 17:14:19 +02:00
n9@users.noreply.github.com
25a1c8e453 DOC: fixed typo in management.txt
replaced fot -> for
added two periods
2019-08-23 11:35:58 +02:00
Willy Tarreau
f909c91e8a DOC: management: document the "trace" and "show trace" commands
At the moment the subsystem is still not complete and the various modules
do not yet produce traces (some dirty experimental code for H2 exists) but
this aims at easing a broad adoption.

Among the missing elements, we can enumerate the lack of configuration
of the sinks (e.g. it's still not possible to change their output format
nor enable/disable timestamps) and since timestamps are not availalbe in
the sinks, they are not collected nor passed by the traces.
2019-08-22 20:23:08 +02:00
Jérôme Magnin
34ebb5cbab DOC: management: document cache_hits and cache_lookups in the CSV format
Counters for cache_hits and cache_lookups were added with commit
a1214a50 ("MINOR: cache: report the number of cache lookups and cache
hits") but not documented in management.txt.
2019-07-17 14:11:38 +02:00
Jérôme Magnin
708eb88845 DOC: management: document reuse and connect counters in the CSV format
Counters for connect and reuse were added in the stats with commit
f1573848 ("MINOR: backend: count the number of connect and reuse
per server and per backend") but not documented the CSV format in
management.txt
2019-07-17 09:40:46 +02:00
Willy Tarreau
9dc6b97429 [RELEASE] Released version 2.1-dev0
Released version 2.1-dev0 with the following main changes :
    - exact copy of 2.0.0
2019-06-16 21:49:47 +02:00
William Lallemand
1dc6963086 MINOR: mworker: add the HAProxy version in "show proc"
Displays the HAProxy version so you can compare the version of old
processes and new ones.
2019-06-12 19:19:57 +02:00
Michael Prokop
4438c6061d DOC: fix typos
s/accidently/accidentally/
s/any ot these messages/any of theses messages/
s/catched/caught/
s/completly/completely/
s/convertor/converter/
s/desribing/describing/
s/developper/developer/
s/eventhough/even though/
s/exectution/execution/
s/functionnality/functionality/
s/If it receive a/If it receives a/
s/In can even/It can even/
s/informations/information/
s/it will be remove /it will be removed /
s/langage/language/
s/mentionned/mentioned/
s/negociated/negotiated/
s/Optionnaly/Optionally/
s/ouputs/outputs/
s/outweights/outweighs/
s/ressources/resources/
2019-05-25 07:34:24 +02:00
Willy Tarreau
e6a02fa65a MINOR: threads: add a "stuck" flag to the thread_info struct
This flag is constantly cleared by the scheduler and will be set by the
watchdog timer to detect stuck threads. It is also set by the "show
threads" command so that it is easy to spot if the situation has evolved
between two subsequent calls : if the first "show threads" shows no stuck
thread and the second one shows such a stuck thread, it indicates that
this thread didn't manage to make any forward progress since the previous
call, which is extremely suspicious.
2019-05-22 11:50:48 +02:00
Willy Tarreau
6bdf3e9b11 MINOR: debug/cli: add some debugging commands for developers
When haproxy is built with DEBUG_DEV, the following commands are added
to the CLI :

  debug dev close <fd>        : close this file descriptor
  debug dev delay [ms]        : sleep this long
  debug dev exec  [cmd] ...   : show this command's output
  debug dev exit  [code]      : immediately exit the process
  debug dev hex   <addr> [len]: dump a memory area
  debug dev log   [msg] ...   : send this msg to global logs
  debug dev loop  [ms]        : loop this long
  debug dev panic             : immediately trigger a panic
  debug dev tkill [thr] [sig] : send signal to thread

These are essentially aimed at helping developers trigger certain
conditions and are expected to be complemented over time.
2019-05-20 16:59:30 +02:00
Willy Tarreau
c7091d89ae MEDIUM: debug/threads: implement an advanced thread dump system
The current "show threads" command was too limited as it was not possible
to dump other threads' detailed states (e.g. their tasks). This patch
goes further by using thread signals so that each thread can dump its
own state in turn into a shared buffer provided by the caller. Threads
are synchronized using a mechanism very similar to the rendez-vous point
and using this method, each thread can safely dump any of its contents
and the caller can finally report the aggregated ones from the buffer.

It is important to keep in mind that the list of signal-safe functions
is limited, so we take care of only using chunk_printf() to write to a
pre-allocated buffer.

This mechanism is enabled by USE_THREAD_DUMP and is enabled by default
on Linux 2.6.28+. On other platforms it falls back to the previous
solution using the loop and the less precise dump.
2019-05-17 17:16:20 +02:00
Willy Tarreau
4e2b646d60 MINOR: cli/debug: add a thread dump function
The new function ha_thread_dump() will dump debugging info about all known
threads. The current thread will contain a bit more info. The long-term goal
is to make it possible to use it in signal handlers to improve the accuracy
of some dumps.

The function dumps its output into the trash so as it was trivial to add,
a new "show threads" command appeared on the CLI.
2019-05-16 18:06:45 +02:00
Willy Tarreau
4c35693eb0 DOC: management: place "show activity" at the right place
This time it seems to be me who had trouble with alphabetical ordering.
This should be backported to 1.9.
2019-05-16 18:02:03 +02:00
Willy Tarreau
d2d3348acb MINOR: activity: enable automatic profiling turn on/off
Instead of having to manually turn task profiling on/off in the
configuration, by default it will work in "auto" mode, which
automatically turns on on any thread experiencing sustained loop
latencies over one millisecond averaged over the last 1024 samples.

This may happen with configs using lots of regex (thing map_reg for
example, which is the lazy way to convert Apache's rewrite rules but
must not be abused), and such high latencies affect all the process
and the problem is most often intermittent (e.g. hitting a map which
is only used for certain host names).

Thus now by default, with profiling set to "auto", it remains off all
the time until something bad happens. This also helps better focus on
the issues when looking at the logs as well as in "show sess" output.
It automatically turns off when the average loop latency over the last
1024 calls goes below 990 microseconds (which typically takes a while
when in idle).

This patch could be backported to stable versions after a bit more
exposure, as it definitely improves observability and the ability to
quickly spot the culprit. In this case, previous patch ("MINOR:
activity: make the profiling status per thread and not global") must
also be taken.
2019-04-25 17:26:46 +02:00
Emeric Brun
0bbec0fa34 MINOR: peers: adds counters on show peers about tasks calls.
This patch adds a counter of calls on the orchestator peers task
and a counter on the tasks linked to applet i/o handler for
each peer.

Those two counters are useful to detect if a peer sync is active
or frozen.

This patch is related to the commit:
  "MINOR: peers: Add a new command to the CLI for peers."
and should be backported with it.
2019-04-18 18:24:25 +02:00
Frédéric Lécaille
21dde5053a DOC: update for "show peers" CLI command.
Add the documentation for the new "show peers" CLI command which comes with
this commit "MINOR: peers: Add a new command to the CLI for peers.".
2019-04-16 09:58:40 +02:00
Willy Tarreau
3f36448e17 DOC: update management.txt to reflect that threads are used by default
It was still mentioned "single-threaded" there. It was also the opportunity
to mention that multiple threads are started by default.
2019-02-27 15:01:46 +01:00
Emeric Brun
9e7547740c MINOR: ssl: add support of aes256 bits ticket keys on file and cli.
Openssl switched from aes128 to aes256 since may 2016  to compute
tls ticket secrets used by default. But Haproxy still handled only
128 bits keys for both tls key file and CLI.

This patch permit the user to set aes256 keys throught CLI or
the key file (80 bytes encoded in base64) in the same way that
aes128 keys were handled (48 bytes encoded in base64):
- first 16 bytes for the key name
- next 16/32 bytes for aes 128/256 key bits key
- last 16/32 bytes for hmac 128/256 bits

Both sizes are now supported (but keys from same file must be
of the same size and can but updated via CLI only using a key of
the same size).

Note: This feature need the fix "dec func ignores padding for output
size checking."
2019-01-14 19:32:58 +01:00
Willy Tarreau
fba74ea7b0 [RELEASE] Released version 2.0-dev0
Released version 2.0-dev0 with the following main changes :
    - BUG/MAJOR: connections: Close the connection before freeing it.
    - REGTEST: Require the option LUA to run lua tests
    - REGTEST: script: Process script arguments before everything else
    - REGTEST: script: Evaluate the varnishtest command to allow quoted parameters
    - REGTEST: script: Add the option --clean to remove previous log direcotries
    - REGTEST: script: Add the option --debug to show logs on standard ouput
    - REGTEST: script: Add the option --keep-logs to keep all log directories
    - REGTEST: script: Add the option --use-htx to enable the HTX in regtests
    - REGTEST: script: Print only errors in the results report
    - REGTEST: Add option to use HTX prefixed by the macro 'no-htx'
    - REGTEST: Make reg-tests target support argument.
    - REGTEST: Fix a typo about barrier type.
    - REGTEST: Be less Linux specific with a syslog regex.
    - REGTEST: Missing enclosing quotes for ${tmpdir} macro.
    - REGTEST: Exclude freebsd target for some reg tests.
    - BUG/MEDIUM: h2: Don't forget to quit the sending_list if SUB_CALL_UNSUBSCRIBE.
    - BUG/MEDIUM: mux-h2: Don't forget to quit the send list on error reports
    - BUG/MEDIUM: dns: Don't prevent reading the last byte of the payload in dns_validate_response()
    - BUG/MEDIUM: dns: overflowed dns name start position causing invalid dns error
    - BUG/MINOR: compression/htx: Don't compress responses with unknown body length
    - BUG/MINOR: compression/htx: Don't add the last block of data if it is empty
    - MEDIUM: mux_h1: Implement h1_show_fd.
    - REGTEST: script: Add support of alternatives in requited options list
    - REGTEST: Add a basic test for the compression
    - BUG/MEDIUM: mux-h2: don't needlessly wake up the demux on short frames
    - REGTEST: A basic test for "http-buffer-request"
    - BUG/MEDIUM: server: Also copy "check-sni" for server templates.
    - MINOR: ssl: Add ssl_sock_set_alpn().
    - MEDIUM: checks: Add check-alpn.
2018-12-22 11:20:35 +01:00
William Lallemand
a57b7e33ef MINOR: cli: implements 'reload' on master CLI
The reload command reload the haproxy master like it is done with a kill
-USR2 on the master process.
2018-12-15 13:33:49 +01:00
Willy Tarreau
52880f97b9 MINOR: mworker/cli: indicate in the master prompt when a reload failed
If a reload was issued to the master process and failed, it is critical
that the admin sees it because it means that the saved configuration
does not work anymore and might not be usable after a full restart. For
this reason in this case we modify the "master" prompt to explicitly
indicate that a reload failed.
2018-12-15 13:30:03 +01:00
William Lallemand
b7ea141cbb MEDIUM: cli: handle CLI level from the master CLI
Handle the CLI level in the master CLI. In order to do this, the master
CLI stores the level in the stream. Each command are prefixed by a
"user" or "operator" command before they are forwarded to the target
CLI.

The level can be configured in the haproxy program arguments with the
level keyword: -S /tmp/sock,level,admin -S /tmp/sock2,level,user.
2018-12-13 09:45:16 +01:00
William Lallemand
67a234f399 MEDIUM: cli: show and change CLI permissions
Implement "show cli level" which show the level of the current CLI
session.

Implement "operator" and "user" which lower the permissions of the
current CLI session.
2018-12-13 09:45:16 +01:00
William Lallemand
256bf0d37b MINOR: cli: change 'show proc' output of old processes
Change the output of the relative pid for the old processes, displays
"[was: X]" instead of just "X" which was confusing if you want to
connect to the CLI of an old PID.
2018-12-12 13:54:17 +01:00
William Lallemand
142db37043 DOC: master CLI documentation in management.txt
Documentation which explains the basic usage of the master CLI.
2018-12-12 11:47:24 +01:00