RepoMirrors/haproxy
1
0
Fork 0
mirror of http://git.haproxy.org/git/haproxy.git/ synced 2025-01-12 08:49:29 +00:00
Code Issues Packages Projects Releases Wiki Activity
10eed8ed03
haproxy/addons/promex/service-prometheus.c

1577 lines
67 KiB
C
Raw Normal View History

CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
/*
* Promex is a Prometheus exporter for HAProxy
*
* It is highly inspired by the official Prometheus exporter.
* See: https://github.com/prometheus/haproxy_exporter
*
* Copyright 2019 Christopher Faulet <cfaulet@haproxy.com>
*
* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License
* as published by the Free Software Foundation; either version
* 2 of the License, or (at your option) any later version.
*
*/
REORG: include: move action.h to haproxy/action{,-t}.h List.h was missing for LIST_ADDQ(). A few unneeded includes of action.h were removed from certain files. This one still relies on applet.h and stick-table.h.
2020-06-04 08:15:32 +00:00
#include <haproxy/action-t.h>
CLEANUP: include: tree-wide alphabetical sort of include files This patch fixes all the leftovers from the include cleanup campaign. There were not that many (~400 entries in ~150 files) but it was definitely worth doing it as it revealed a few duplicates.
2020-06-09 07:07:15 +00:00
#include <haproxy/api.h>
REORG: move applet.h to haproxy/applet{,-t}.h The type file was slightly tidied. The cli-specific APPCTX_CLI_ST1_* flag definitions were moved to cli.h. The type file was adjusted to include buf-t.h and not the huge buf.h. A few call places were fixed because they did not need this include.
2020-06-04 17:42:41 +00:00
#include <haproxy/applet.h>
REORG: include: move backend.h to haproxy/backend{,-t}.h The files remained mostly unchanged since they were OK. However, half of the users didn't need to include them, and about as many actually needed to have it and used to find functions like srv_currently_usable() through a long chain that broke when moving the file.
2020-06-04 20:50:02 +00:00
#include <haproxy/backend.h>
REORG: include: move cfgparse.h to haproxy/cfgparse.h There's no point splitting the file in two since only cfgparse uses the types defined there. A few call places were updated and cleaned up. All of them were in C files which register keywords. There is nothing left in common/ now so this directory must not be used anymore.
2020-06-04 22:00:29 +00:00
#include <haproxy/cfgparse.h>
MAJOR: contrib/prometheus-exporter: move health check status to labels this patch is a breaking change between v2.3 and v2.4: we move from using gauge value for health check states to labels values. The diff is quite small thanks to the preparation work from Christopher to allow more flexibility in labels, see commit 5a2f938732126f43bbf4cea5482c01552b0d0314 ("MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones") this is a follow up of commit c6464591a365bfcf509b322bdaa4d608c9395d75 ("MAJOR: contrib/prometheus-exporter: move ftd/bkd/srv states to labels"). The main goal being to be better aligned with prometheus use cases in terms of queries. More specifically to health checks, Pierre C. mentioned the possible quirks he had to put in place in order to make use of those metrics through prometheus: <aggregator_function> by(proxy, check_status) (count_values by(proxy, instance) ("check_status", haproxy_server_check_status)) I am perfectly aware this introduces a lot more metrics but I don't see how we can improve the usability without it. The main issue remains in the cardinality of the states which are > 20. Prometheus recommends to stay below a cardinality of 10 for a given metric but I consider our case very specific, because highly linked to the level of precision haproxy exposes. Even before this patch I saw several large production setup (a few hundreds of MB in output) which are making use of the scope parameter to simply ignore the server metrics, so that the scrapping can be faster, and memory consumed on client side not too high. So I believe we should eventually continue in that direction and offer more granularity of filtering of the output. That being said it is already possible to filter out the data on prometheus client side. this is related to github issue #1029 Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-01 12:11:51 +00:00
#include <haproxy/check.h>
REORG: include: move frontend.h to haproxy/frontend.h There was no type file for this one, it only contains frontend_accept().
2020-06-04 09:23:07 +00:00
#include <haproxy/frontend.h>
REORG: include: split global.h into haproxy/global{,-t}.h global.h was one of the messiest files, it has accumulated tons of implicit dependencies and declares many globals that make almost all other file include it. It managed to silence a dependency loop between server.h and proxy.h by being well placed to pre-define the required structs, forcing struct proxy and struct server to be forward-declared in a significant number of files. It was split in to, one which is the global struct definition and the few macros and flags, and the rest containing the functions prototypes. The UNIX_MAX_PATH definition was moved to compat.h.
2020-06-04 15:05:57 +00:00
#include <haproxy/global.h>
REORG: include: split common/http.h into haproxy/http{,-t}.h So the enums and structs were placed into http-t.h and the functions into http.h. This revealed that several files were dependeng on http.h but not including it, as it was silently inherited via other files.
2020-06-02 17:11:26 +00:00
#include <haproxy/http.h>
BUILD: tree-wide: add missing http_ana.h from many places At least 6 files make use of s->txn without including http_ana which defines it. They used to get it from other includes.
2021-10-06 16:56:42 +00:00
#include <haproxy/http_ana.h>
REORG: include: move http_htx.h to haproxy/http_htx{,-t}.h A few includes had to be added, namely list-t.h in the type file and types/proxy.h in the proto file. actions.h was including http-htx.h but didn't need it so it was dropped.
2020-06-04 07:08:41 +00:00
#include <haproxy/http_htx.h>
REORG: include: split common/htx.h into haproxy/htx{,-t}.h Most of the file was a large set of HTX elements manipulation functions and few types, so splitting them allowed to further reduce dependencies and shrink the build time. Doing so revealed that a few files (h2.c, mux_pt.c) needed haproxy/buf.h and were previously getting it through htx.h. They were fixed.
2020-06-03 06:44:35 +00:00
#include <haproxy/htx.h>
REORG: include: move the error reporting functions to from log.h to errors.h Most of the files dealing with error reports have to include log.h in order to access ha_alert(), ha_warning() etc. But while these functions don't depend on anything, log.h depends on a lot of stuff because it deals with log-formats and samples. As a result it's impossible not to embark long dependencies when using ha_warning() or qfprintf(). This patch moves these low-level functions to errors.h, which already defines the error codes used at the same places. About half of the users of log.h could be adjusted, sometimes revealing other issues such as missing tools.h. Interestingly the total preprocessed size shrunk by 4%.
2020-06-05 15:27:29 +00:00
#include <haproxy/list.h>
REORG: include: move listener.h to haproxy/listener{,-t}.h stdlib and list were missing from listener.h, otherwise it was OK.
2020-06-04 12:58:24 +00:00
#include <haproxy/listener.h>
REORG: include: move the error reporting functions to from log.h to errors.h Most of the files dealing with error reports have to include log.h in order to access ha_alert(), ha_warning() etc. But while these functions don't depend on anything, log.h depends on a lot of stuff because it deals with log-formats and samples. As a result it's impossible not to embark long dependencies when using ha_warning() or qfprintf(). This patch moves these low-level functions to errors.h, which already defines the error codes used at the same places. About half of the users of log.h could be adjusted, sometimes revealing other issues such as missing tools.h. Interestingly the total preprocessed size shrunk by 4%.
2020-06-05 15:27:29 +00:00
#include <haproxy/log.h>
REORG: include: move proxy.h to haproxy/proxy{,-t}.h This one is particularly difficult to split because it provides all the functions used to manipulate a proxy state and to retrieve names or IDs for error reporting, and as such, it was included in 73 files (down to 68 after cleanup). It would deserve a small cleanup though the cut points are not obvious at the moment given the number of structs involved in the struct proxy itself.
2020-06-04 20:29:18 +00:00
#include <haproxy/proxy.h>
REORG: include: move sample.h to haproxy/sample{,-t}.h This one is particularly tricky to move because everyone uses it and it depends on a lot of other types. For example it cannot include arg-t.h and must absolutely only rely on forward declarations to avoid dependency loops between vars -> sample_data -> arg. In order to address this one, it would be nice to split the sample_data part out of sample.h.
2020-06-04 13:33:47 +00:00
#include <haproxy/sample.h>
REORG: include: move server.h to haproxy/server{,-t}.h extern struct dict server_name_dict was moved from the type file to the main file. A handful of inlined functions were moved at the bottom of the file. Call places were updated to use server-t.h when relevant, or to simply drop the entry when not needed.
2020-06-04 21:20:13 +00:00
#include <haproxy/server.h>
REORG: include: move stats.h to haproxy/stats{,-t}.h Just some minor reordering, and the usual cleanup of call places for those which didn't need it. We don't include the whole tools.h into stats-t anymore but just tools-t.h.
2020-06-04 17:58:55 +00:00
#include <haproxy/stats.h>
REORG: include: move stream.h to haproxy/stream{,-t}.h This one was not easy because it was embarking many includes with it, which other files would automatically find. At least global.h, arg.h and tools.h were identified. 93 total locations were identified, 8 additional includes had to be added. In the rare files where it was possible to finalize the sorting of includes by adjusting only one or two extra lines, it was done. But all files would need to be rechecked and cleaned up now. It was the last set of files in types/ and proto/ and these directories must not be reused anymore.
2020-06-04 21:46:14 +00:00
#include <haproxy/stream.h>
REORG: include: move stream_interface.h to haproxy/stream_interface{,-t}.h Almost no changes, removed stdlib and added buf-t and connection-t to the types to avoid a warning.
2020-06-04 18:45:39 +00:00
#include <haproxy/stream_interface.h>
REORG: include: move task.h to haproxy/task{,-t}.h The TASK_IS_TASKLET() macro was moved to the proto file instead of the type one. The proto part was a bit reordered to remove a number of ugly forward declaration of static inline functions. About a tens of C and H files had their dependency dropped since they were not using anything from task.h.
2020-06-04 15:25:40 +00:00
#include <haproxy/task.h>
BUILD: promex: service-prometheus.c needs tools.h It calls url_decode() and memprintf() but used to inherit them through others.
2021-05-08 10:58:12 +00:00
#include <haproxy/tools.h>
MINOR: contrib/prometheus-exporter: export build_info commit c55a626217a7e676e1cc ("MINOR: contrib/prometheus-exporter: Add missing global and per-server metrics") is renaming two metrics between v2.2 and v2.3: server_idle_connections_current server_idle_connections_limit It is breaking some tools which are making use of those metrics while supporting several haproxy versions. This build_info will permit tools which make use of metrics to be able to match the haproxy version and change the list of expected metrics. This was possible using the haproxy stats socket but not with prometheus export. This patch follows prometheus best pratices to export specific software informations. It is adding a new field `build_info` so we can extend it to other parameters if needed in the future. example output: # HELP haproxy_process_build_info HAProxy build info. # TYPE haproxy_process_build_info gauge haproxy_process_build_info{version="2.4-dev5-2e1a3f-5"} 1 Even though it is not a bugfix, this patch will make more sense when backported up to >= 2.0 Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-08 12:18:06 +00:00
#include <haproxy/version.h>
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
/* Prometheus exporter applet states (appctx->st0) */
enum {
PROMEX_ST_INIT = 0, /* initialized */
PROMEX_ST_HEAD, /* send headers before dump */
PROMEX_ST_DUMP, /* dumping stats */
PROMEX_ST_DONE, /* finished */
BUG/MINOR: contrib/prometheus-exporter: Fix applet accordingly to recent changes Since the flag EOI was added on channels, some hidden bugs in the prometheus exporter now leads to error. the visible effect is that responses are truncated. So first of all, channel_add_input() must be called when the response headers and the EOM block are added. To be sure to correctly update the response channel (especially to_forward value). Then the request must really be fully consumed. And finally, the return clause in the switch has been replaced by a break. It was totally wrong to skip the end of the function in the states PROMEX_DONE and PROMEX_ERROR. (Note that PROMEX_ERROR was never used, so it was replaced by PROMEX_END just to ease reading the code). No need to backport this patch, the Prometheus exporter does not exist in early versions.
2019-03-27 14:48:53 +00:00
PROMEX_ST_END, /* treatment terminated */
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
};
/* Prometheus exporter dumper states (appctx->st1) */
enum {
MEDIUM: contrib/prometheus-exporter: export base stick table stats I saw some people falling back to unix socket to collect some data they could not find in prometheus exporter. One of them is base info from stick tables (used/size). I do not plan to extend it more for now; keys are quite a mess to handle. This should resolve github issue #1008. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-07 19:42:38 +00:00
PROMEX_DUMPER_INIT = 0, /* initialized */
PROMEX_DUMPER_GLOBAL, /* dump metrics of globals */
PROMEX_DUMPER_FRONT, /* dump metrics of frontend proxies */
PROMEX_DUMPER_BACK, /* dump metrics of backend proxies */
PROMEX_DUMPER_LI, /* dump metrics of listeners */
PROMEX_DUMPER_SRV, /* dump metrics of servers */
PROMEX_DUMPER_STICKTABLE, /* dump metrics of stick tables */
PROMEX_DUMPER_DONE, /* finished */
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
};
/* Prometheus exporter flags (appctx->ctx.stats.flags) */
MEDIUM: contrib/prometheus-exporter: export base stick table stats I saw some people falling back to unix socket to collect some data they could not find in prometheus exporter. One of them is base info from stick tables (used/size). I do not plan to extend it more for now; keys are quite a mess to handle. This should resolve github issue #1008. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-07 19:42:38 +00:00
#define PROMEX_FL_METRIC_HDR 0x00000001
#define PROMEX_FL_INFO_METRIC 0x00000002
#define PROMEX_FL_FRONT_METRIC 0x00000004
#define PROMEX_FL_BACK_METRIC 0x00000008
#define PROMEX_FL_SRV_METRIC 0x00000010
MEDIUM: contrib/prometheus-exporter: add listen stats this was a missing piece for a while now even though it was planned. This patch adds listen stats. Nothing in particular but we make use of the status helper previously added. `promex_st_metrics` diff also looks scary, but I had to realign all lines. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-14 22:22:56 +00:00
#define PROMEX_FL_LI_METRIC 0x00000020
#define PROMEX_FL_STICKTABLE_METRIC 0x00000040
#define PROMEX_FL_SCOPE_GLOBAL 0x00000080
#define PROMEX_FL_SCOPE_FRONT 0x00000100
#define PROMEX_FL_SCOPE_BACK 0x00000200
#define PROMEX_FL_SCOPE_SERVER 0x00000400
#define PROMEX_FL_SCOPE_LI 0x00000800
#define PROMEX_FL_SCOPE_STICKTABLE 0x00001000
#define PROMEX_FL_NO_MAINT_SRV 0x00002000
MEDIUM: contrib/prometheus-exporter: export base stick table stats I saw some people falling back to unix socket to collect some data they could not find in prometheus exporter. One of them is base info from stick tables (used/size). I do not plan to extend it more for now; keys are quite a mess to handle. This should resolve github issue #1008. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-07 19:42:38 +00:00
#define PROMEX_FL_SCOPE_ALL (PROMEX_FL_SCOPE_GLOBAL | PROMEX_FL_SCOPE_FRONT | \
MEDIUM: contrib/prometheus-exporter: add listen stats this was a missing piece for a while now even though it was planned. This patch adds listen stats. Nothing in particular but we make use of the status helper previously added. `promex_st_metrics` diff also looks scary, but I had to realign all lines. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-14 22:22:56 +00:00
PROMEX_FL_SCOPE_LI | PROMEX_FL_SCOPE_BACK | \
PROMEX_FL_SCOPE_SERVER | PROMEX_FL_SCOPE_STICKTABLE)
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
MINOR: contrib/prometheus-exporter: Add promex_metric struct defining a metric This structure will be used to define a Prometheus metric, i.e its name, its type (gauge or counter) and its flags. The flags will be used to know for which entities the metric is defined (global, frontend, backend and/or server).
2021-01-20 14:19:12 +00:00
/* Promtheus metric type (gauge or counter) */
enum promex_mt_type {
PROMEX_MT_GAUGE = 1,
PROMEX_MT_COUNTER = 2,
};
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
/* The max length for metrics name. It is a hard limit but it should be
CLEANUP: assorted typo fixes in the code and comments This is fifth iteration of typo fixes
2020-03-23 17:28:40 +00:00
* enough.
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
*/
#define PROMEX_MAX_NAME_LEN 128
/* The expected max length for a metric dump, including its header lines. It is
* just a soft limit to avoid extra work. We don't try to dump a metric if less
* than this size is available in the HTX.
*/
#define PROMEX_MAX_METRIC_LENGTH 512
MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones Instead of using static labels for metrics, we now use an array of labels, filled for each metrics if necessary and passed to the dump function. This way, it is easier to extend the promex service. For now, there are at most 8 labels per metrics. This limit may be raised by changing PROMEX_MAX_LABELS value. And to ease labels addition, a label is defined as a key/value pair. The formatting is handled by the dump function. For the proxies and servers, the first entry of the array is always the proxy name. In addition, for the servers, the second entry is always the server name.
2021-01-28 10:24:17 +00:00
/* The max number of labels per metric */
#define PROMEX_MAX_LABELS 8
MINOR: contrib/prometheus-exporter: export build_info commit c55a626217a7e676e1cc ("MINOR: contrib/prometheus-exporter: Add missing global and per-server metrics") is renaming two metrics between v2.2 and v2.3: server_idle_connections_current server_idle_connections_limit It is breaking some tools which are making use of those metrics while supporting several haproxy versions. This build_info will permit tools which make use of metrics to be able to match the haproxy version and change the list of expected metrics. This was possible using the haproxy stats socket but not with prometheus export. This patch follows prometheus best pratices to export specific software informations. It is adding a new field `build_info` so we can extend it to other parameters if needed in the future. example output: # HELP haproxy_process_build_info HAProxy build info. # TYPE haproxy_process_build_info gauge haproxy_process_build_info{version="2.4-dev5-2e1a3f-5"} 1 Even though it is not a bugfix, this patch will make more sense when backported up to >= 2.0 Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-08 12:18:06 +00:00
MINOR: contrib/prometheus-exporter: Add promex_metric struct defining a metric This structure will be used to define a Prometheus metric, i.e its name, its type (gauge or counter) and its flags. The flags will be used to know for which entities the metric is defined (global, frontend, backend and/or server).
2021-01-20 14:19:12 +00:00
/* Describe a prometheus metric */
struct promex_metric {
const struct ist n; /* The metric name */
enum promex_mt_type type; /* The metric type (gauge or counter) */
unsigned int flags; /* PROMEX_FL_* flags */
};
MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones Instead of using static labels for metrics, we now use an array of labels, filled for each metrics if necessary and passed to the dump function. This way, it is easier to extend the promex service. For now, there are at most 8 labels per metrics. This limit may be raised by changing PROMEX_MAX_LABELS value. And to ease labels addition, a label is defined as a key/value pair. The formatting is handled by the dump function. For the proxies and servers, the first entry of the array is always the proxy name. In addition, for the servers, the second entry is always the server name.
2021-01-28 10:24:17 +00:00
/* Describe a prometheus metric label. It is just a key/value pair */
struct promex_label {
struct ist name;
struct ist value;
};
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
/* Global metrics */
const struct promex_metric promex_global_metrics[INF_TOTAL_FIELDS] = {
//[INF_NAME] ignored
//[INF_VERSION], ignored
//[INF_RELEASE_DATE] ignored
[INF_NBTHREAD] = { .n = IST("nbthread"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
[INF_NBPROC] = { .n = IST("nbproc"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
[INF_PROCESS_NUM] = { .n = IST("relative_process_id"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
//[INF_PID] ignored
//[INF_UPTIME] ignored
[INF_UPTIME_SEC] = { .n = IST("uptime_seconds"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
[INF_START_TIME_SEC] = { .n = IST("start_time_seconds"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
CLEANUP: contrib/prometheus-exporter: align and reorder fields - align safe_idle_connections_current field fix minor typo added in commit 37286a5ac595069a491c3e8a7a7e4faf3d9ea8ad ("MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics") - reorder info fields to be able to compare them easily - add missing ignored info fields as comment Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-01 12:11:59 +00:00
//[INF_MEMMAX_MB] ignored
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
[INF_MEMMAX_BYTES] = { .n = IST("max_memory_bytes"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
CLEANUP: contrib/prometheus-exporter: align and reorder fields - align safe_idle_connections_current field fix minor typo added in commit 37286a5ac595069a491c3e8a7a7e4faf3d9ea8ad ("MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics") - reorder info fields to be able to compare them easily - add missing ignored info fields as comment Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-01 12:11:59 +00:00
//[INF_POOL_ALLOC_MB] ignored
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
[INF_POOL_ALLOC_BYTES] = { .n = IST("pool_allocated_bytes"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
CLEANUP: contrib/prometheus-exporter: align and reorder fields - align safe_idle_connections_current field fix minor typo added in commit 37286a5ac595069a491c3e8a7a7e4faf3d9ea8ad ("MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics") - reorder info fields to be able to compare them easily - add missing ignored info fields as comment Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-01 12:11:59 +00:00
//[INF_POOL_USED_MB] ignored
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
[INF_POOL_USED_BYTES] = { .n = IST("pool_used_bytes"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
[INF_POOL_FAILED] = { .n = IST("pool_failures_total"), .type = PROMEX_MT_COUNTER, .flags = PROMEX_FL_INFO_METRIC },
[INF_ULIMIT_N] = { .n = IST("max_fds"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
[INF_MAXSOCK] = { .n = IST("max_sockets"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
[INF_MAXCONN] = { .n = IST("max_connections"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
[INF_HARD_MAXCONN] = { .n = IST("hard_max_connections"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
[INF_CURR_CONN] = { .n = IST("current_connections"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
[INF_CUM_CONN] = { .n = IST("connections_total"), .type = PROMEX_MT_COUNTER, .flags = PROMEX_FL_INFO_METRIC },
[INF_CUM_REQ] = { .n = IST("requests_total"), .type = PROMEX_MT_COUNTER, .flags = PROMEX_FL_INFO_METRIC },
[INF_MAX_SSL_CONNS] = { .n = IST("max_ssl_connections"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
[INF_CURR_SSL_CONNS] = { .n = IST("current_ssl_connections"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
[INF_CUM_SSL_CONNS] = { .n = IST("ssl_connections_total"), .type = PROMEX_MT_COUNTER, .flags = PROMEX_FL_INFO_METRIC },
[INF_MAXPIPES] = { .n = IST("max_pipes"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
[INF_PIPES_USED] = { .n = IST("pipes_used_total"), .type = PROMEX_MT_COUNTER, .flags = PROMEX_FL_INFO_METRIC },
[INF_PIPES_FREE] = { .n = IST("pipes_free_total"), .type = PROMEX_MT_COUNTER, .flags = PROMEX_FL_INFO_METRIC },
[INF_CONN_RATE] = { .n = IST("current_connection_rate"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
[INF_CONN_RATE_LIMIT] = { .n = IST("limit_connection_rate"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
[INF_MAX_CONN_RATE] = { .n = IST("max_connection_rate"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
[INF_SESS_RATE] = { .n = IST("current_session_rate"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
[INF_SESS_RATE_LIMIT] = { .n = IST("limit_session_rate"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
[INF_MAX_SESS_RATE] = { .n = IST("max_session_rate"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
[INF_SSL_RATE] = { .n = IST("current_ssl_rate"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
[INF_SSL_RATE_LIMIT] = { .n = IST("limit_ssl_rate"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
[INF_MAX_SSL_RATE] = { .n = IST("max_ssl_rate"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
[INF_SSL_FRONTEND_KEY_RATE] = { .n = IST("current_frontend_ssl_key_rate"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
[INF_SSL_FRONTEND_MAX_KEY_RATE] = { .n = IST("max_frontend_ssl_key_rate"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
[INF_SSL_FRONTEND_SESSION_REUSE_PCT] = { .n = IST("frontend_ssl_reuse"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
[INF_SSL_BACKEND_KEY_RATE] = { .n = IST("current_backend_ssl_key_rate"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
[INF_SSL_BACKEND_MAX_KEY_RATE] = { .n = IST("max_backend_ssl_key_rate"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
[INF_SSL_CACHE_LOOKUPS] = { .n = IST("ssl_cache_lookups_total"), .type = PROMEX_MT_COUNTER, .flags = PROMEX_FL_INFO_METRIC },
[INF_SSL_CACHE_MISSES] = { .n = IST("ssl_cache_misses_total"), .type = PROMEX_MT_COUNTER, .flags = PROMEX_FL_INFO_METRIC },
[INF_COMPRESS_BPS_IN] = { .n = IST("http_comp_bytes_in_total"), .type = PROMEX_MT_COUNTER, .flags = PROMEX_FL_INFO_METRIC },
[INF_COMPRESS_BPS_OUT] = { .n = IST("http_comp_bytes_out_total"), .type = PROMEX_MT_COUNTER, .flags = PROMEX_FL_INFO_METRIC },
[INF_COMPRESS_BPS_RATE_LIM] = { .n = IST("limit_http_comp"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
[INF_ZLIB_MEM_USAGE] = { .n = IST("current_zlib_memory"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
[INF_MAX_ZLIB_MEM_USAGE] = { .n = IST("max_zlib_memory"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
[INF_TASKS] = { .n = IST("current_tasks"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
[INF_RUN_QUEUE] = { .n = IST("current_run_queue"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
[INF_IDLE_PCT] = { .n = IST("idle_time_percent"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
//[INF_NODE] ignored
//[INF_DESCRIPTION] ignored
[INF_STOPPING] = { .n = IST("stopping"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
[INF_JOBS] = { .n = IST("jobs"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
[INF_UNSTOPPABLE_JOBS] = { .n = IST("unstoppable_jobs"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
[INF_LISTENERS] = { .n = IST("listeners"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
[INF_ACTIVE_PEERS] = { .n = IST("active_peers"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
[INF_CONNECTED_PEERS] = { .n = IST("connected_peers"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
[INF_DROPPED_LOGS] = { .n = IST("dropped_logs_total"), .type = PROMEX_MT_COUNTER, .flags = PROMEX_FL_INFO_METRIC },
[INF_BUSY_POLLING] = { .n = IST("busy_polling_enabled"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
[INF_FAILED_RESOLUTIONS] = { .n = IST("failed_resolutions"), .type = PROMEX_MT_COUNTER, .flags = PROMEX_FL_INFO_METRIC },
[INF_TOTAL_BYTES_OUT] = { .n = IST("bytes_out_total"), .type = PROMEX_MT_COUNTER, .flags = PROMEX_FL_INFO_METRIC },
[INF_TOTAL_SPLICED_BYTES_OUT] = { .n = IST("spliced_bytes_out_total"), .type = PROMEX_MT_COUNTER, .flags = PROMEX_FL_INFO_METRIC },
[INF_BYTES_OUT_RATE] = { .n = IST("bytes_out_rate"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
//[INF_DEBUG_COMMANDS_ISSUED] ignored
MINOR: contrib/prometheus-exporter: add recv logs_logs_total field this field was added by commit 45c457a62941a7c4a86ce4327d7755edcd4b230e ("MINOR: log: adds counters on received syslog messages.") Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-01 12:11:57 +00:00
[INF_CUM_LOG_MSGS] = { .n = IST("recv_logs_total"), .type = PROMEX_MT_COUNTER, .flags = PROMEX_FL_INFO_METRIC },
CLEANUP: contrib/prometheus-exporter: align and reorder fields - align safe_idle_connections_current field fix minor typo added in commit 37286a5ac595069a491c3e8a7a7e4faf3d9ea8ad ("MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics") - reorder info fields to be able to compare them easily - add missing ignored info fields as comment Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-01 12:11:59 +00:00
[INF_BUILD_INFO] = { .n = IST("build_info"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_INFO_METRIC },
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
};
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
/* frontend/backend/server fields */
const struct promex_metric promex_st_metrics[ST_F_TOTAL_FIELDS] = {
MINOR: promex: backend aggregated server check status - add new metric: `haproxy_backend_agg_server_check_status` it counts the number of servers matching a specific check status this permits to exclude per server check status as the usage is often to rely on the total. Indeed in large setup having thousands of servers per backend the memory impact is not neglible to store the per server metric. - realign promex_str_metrics array quite simple implementation - we could improve it later by adding an internal state to the prometheus exporter, thus to avoid counting at every dump. this patch is an attempt to close github issue #1312. It may bebackported to 2.4 if requested. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-11-07 09:18:47 +00:00
//[ST_F_PXNAME] ignored
//[ST_F_SVNAME] ignored
[ST_F_QCUR] = { .n = IST("current_queue"), .type = PROMEX_MT_GAUGE, .flags = ( PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
[ST_F_QMAX] = { .n = IST("max_queue"), .type = PROMEX_MT_GAUGE, .flags = ( PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
[ST_F_SCUR] = { .n = IST("current_sessions"), .type = PROMEX_MT_GAUGE, .flags = (PROMEX_FL_FRONT_METRIC | PROMEX_FL_LI_METRIC | PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
[ST_F_SMAX] = { .n = IST("max_sessions"), .type = PROMEX_MT_GAUGE, .flags = (PROMEX_FL_FRONT_METRIC | PROMEX_FL_LI_METRIC | PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
[ST_F_SLIM] = { .n = IST("limit_sessions"), .type = PROMEX_MT_GAUGE, .flags = (PROMEX_FL_FRONT_METRIC | PROMEX_FL_LI_METRIC | PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
[ST_F_STOT] = { .n = IST("sessions_total"), .type = PROMEX_MT_COUNTER, .flags = (PROMEX_FL_FRONT_METRIC | PROMEX_FL_LI_METRIC | PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
[ST_F_BIN] = { .n = IST("bytes_in_total"), .type = PROMEX_MT_COUNTER, .flags = (PROMEX_FL_FRONT_METRIC | PROMEX_FL_LI_METRIC | PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
[ST_F_BOUT] = { .n = IST("bytes_out_total"), .type = PROMEX_MT_COUNTER, .flags = (PROMEX_FL_FRONT_METRIC | PROMEX_FL_LI_METRIC | PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
[ST_F_DREQ] = { .n = IST("requests_denied_total"), .type = PROMEX_MT_COUNTER, .flags = (PROMEX_FL_FRONT_METRIC | PROMEX_FL_LI_METRIC | PROMEX_FL_BACK_METRIC ) },
[ST_F_DRESP] = { .n = IST("responses_denied_total"), .type = PROMEX_MT_COUNTER, .flags = (PROMEX_FL_FRONT_METRIC | PROMEX_FL_LI_METRIC | PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
[ST_F_EREQ] = { .n = IST("request_errors_total"), .type = PROMEX_MT_COUNTER, .flags = (PROMEX_FL_FRONT_METRIC | PROMEX_FL_LI_METRIC ) },
[ST_F_ECON] = { .n = IST("connection_errors_total"), .type = PROMEX_MT_COUNTER, .flags = ( PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
[ST_F_ERESP] = { .n = IST("response_errors_total"), .type = PROMEX_MT_COUNTER, .flags = ( PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
[ST_F_WRETR] = { .n = IST("retry_warnings_total"), .type = PROMEX_MT_COUNTER, .flags = ( PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
[ST_F_WREDIS] = { .n = IST("redispatch_warnings_total"), .type = PROMEX_MT_COUNTER, .flags = ( PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
[ST_F_STATUS] = { .n = IST("status"), .type = PROMEX_MT_GAUGE, .flags = (PROMEX_FL_FRONT_METRIC | PROMEX_FL_LI_METRIC | PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
[ST_F_WEIGHT] = { .n = IST("weight"), .type = PROMEX_MT_GAUGE, .flags = ( PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
[ST_F_ACT] = { .n = IST("active_servers"), .type = PROMEX_MT_GAUGE, .flags = ( PROMEX_FL_BACK_METRIC ) },
[ST_F_BCK] = { .n = IST("backup_servers"), .type = PROMEX_MT_GAUGE, .flags = ( PROMEX_FL_BACK_METRIC ) },
[ST_F_CHKFAIL] = { .n = IST("check_failures_total"), .type = PROMEX_MT_COUNTER, .flags = ( PROMEX_FL_SRV_METRIC) },
[ST_F_CHKDOWN] = { .n = IST("check_up_down_total"), .type = PROMEX_MT_COUNTER, .flags = ( PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
[ST_F_LASTCHG] = { .n = IST("check_last_change_seconds"), .type = PROMEX_MT_GAUGE, .flags = ( PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
[ST_F_DOWNTIME] = { .n = IST("downtime_seconds_total"), .type = PROMEX_MT_COUNTER, .flags = ( PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
[ST_F_QLIMIT] = { .n = IST("queue_limit"), .type = PROMEX_MT_GAUGE, .flags = ( PROMEX_FL_SRV_METRIC) },
//[ST_F_PID] ignored
//[ST_F_IID] ignored
//[ST_F_SID] ignored
[ST_F_THROTTLE] = { .n = IST("current_throttle"), .type = PROMEX_MT_GAUGE, .flags = ( PROMEX_FL_SRV_METRIC) },
[ST_F_LBTOT] = { .n = IST("loadbalanced_total"), .type = PROMEX_MT_COUNTER, .flags = ( PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
//[ST_F_TRACKED] ignored
//[ST_F_TYPE] ignored
//[ST_F_RATE] ignored
[ST_F_RATE_LIM] = { .n = IST("limit_session_rate"), .type = PROMEX_MT_GAUGE, .flags = (PROMEX_FL_FRONT_METRIC ) },
[ST_F_RATE_MAX] = { .n = IST("max_session_rate"), .type = PROMEX_MT_GAUGE, .flags = (PROMEX_FL_FRONT_METRIC | PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
[ST_F_CHECK_STATUS] = { .n = IST("check_status"), .type = PROMEX_MT_GAUGE, .flags = ( PROMEX_FL_SRV_METRIC) },
[ST_F_CHECK_CODE] = { .n = IST("check_code"), .type = PROMEX_MT_GAUGE, .flags = ( PROMEX_FL_SRV_METRIC) },
[ST_F_CHECK_DURATION] = { .n = IST("check_duration_seconds"), .type = PROMEX_MT_GAUGE, .flags = ( PROMEX_FL_SRV_METRIC) },
[ST_F_HRSP_1XX] = { .n = IST("http_responses_total"), .type = PROMEX_MT_COUNTER, .flags = (PROMEX_FL_FRONT_METRIC | PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
[ST_F_HRSP_2XX] = { .n = IST("http_responses_total"), .type = PROMEX_MT_COUNTER, .flags = (PROMEX_FL_FRONT_METRIC | PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
[ST_F_HRSP_3XX] = { .n = IST("http_responses_total"), .type = PROMEX_MT_COUNTER, .flags = (PROMEX_FL_FRONT_METRIC | PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
[ST_F_HRSP_4XX] = { .n = IST("http_responses_total"), .type = PROMEX_MT_COUNTER, .flags = (PROMEX_FL_FRONT_METRIC | PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
[ST_F_HRSP_5XX] = { .n = IST("http_responses_total"), .type = PROMEX_MT_COUNTER, .flags = (PROMEX_FL_FRONT_METRIC | PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
[ST_F_HRSP_OTHER] = { .n = IST("http_responses_total"), .type = PROMEX_MT_COUNTER, .flags = (PROMEX_FL_FRONT_METRIC | PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
//[ST_F_HANAFAIL] ignored
//[ST_F_REQ_RATE] ignored
[ST_F_REQ_RATE_MAX] = { .n = IST("http_requests_rate_max"), .type = PROMEX_MT_GAUGE, .flags = (PROMEX_FL_FRONT_METRIC ) },
[ST_F_REQ_TOT] = { .n = IST("http_requests_total"), .type = PROMEX_MT_COUNTER, .flags = (PROMEX_FL_FRONT_METRIC | PROMEX_FL_BACK_METRIC ) },
[ST_F_CLI_ABRT] = { .n = IST("client_aborts_total"), .type = PROMEX_MT_COUNTER, .flags = ( PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
[ST_F_SRV_ABRT] = { .n = IST("server_aborts_total"), .type = PROMEX_MT_COUNTER, .flags = ( PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
[ST_F_COMP_IN] = { .n = IST("http_comp_bytes_in_total"), .type = PROMEX_MT_COUNTER, .flags = (PROMEX_FL_FRONT_METRIC | PROMEX_FL_BACK_METRIC ) },
[ST_F_COMP_OUT] = { .n = IST("http_comp_bytes_out_total"), .type = PROMEX_MT_COUNTER, .flags = (PROMEX_FL_FRONT_METRIC | PROMEX_FL_BACK_METRIC ) },
[ST_F_COMP_BYP] = { .n = IST("http_comp_bytes_bypassed_total"), .type = PROMEX_MT_COUNTER, .flags = (PROMEX_FL_FRONT_METRIC | PROMEX_FL_BACK_METRIC ) },
[ST_F_COMP_RSP] = { .n = IST("http_comp_responses_total"), .type = PROMEX_MT_COUNTER, .flags = (PROMEX_FL_FRONT_METRIC | PROMEX_FL_BACK_METRIC ) },
[ST_F_LASTSESS] = { .n = IST("last_session_seconds"), .type = PROMEX_MT_GAUGE, .flags = ( PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
//[ST_F_LAST_CHK] ignored
//[ST_F_LAST_AGT] ignored
[ST_F_QTIME] = { .n = IST("queue_time_average_seconds"), .type = PROMEX_MT_GAUGE, .flags = ( PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
[ST_F_CTIME] = { .n = IST("connect_time_average_seconds"), .type = PROMEX_MT_GAUGE, .flags = ( PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
[ST_F_RTIME] = { .n = IST("response_time_average_seconds"), .type = PROMEX_MT_GAUGE, .flags = ( PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
[ST_F_TTIME] = { .n = IST("total_time_average_seconds"), .type = PROMEX_MT_GAUGE, .flags = ( PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
//[ST_F_AGENT_STATUS] ignored
//[ST_F_AGENT_CODE] ignored
//[ST_F_AGENT_DURATION] ignored
//[ST_F_CHECK_DESC] ignored
//[ST_F_AGENT_DESC] ignored
//[ST_F_CHECK_RISE] ignored
//[ST_F_CHECK_FALL] ignored
//[ST_F_CHECK_HEALTH] ignored
//[ST_F_AGENT_RISE] ignored
//[ST_F_AGENT_FALL] ignored
//[ST_F_AGENT_HEALTH] ignored
//[ST_F_ADDR] ignored
//[ST_F_COOKIE] ignored
//[ST_F_MODE] ignored
//[ST_F_ALGO] ignored
//[ST_F_CONN_RATE] ignored
[ST_F_CONN_RATE_MAX] = { .n = IST("connections_rate_max"), .type = PROMEX_MT_GAUGE, .flags = (PROMEX_FL_FRONT_METRIC ) },
[ST_F_CONN_TOT] = { .n = IST("connections_total"), .type = PROMEX_MT_COUNTER, .flags = (PROMEX_FL_FRONT_METRIC ) },
[ST_F_INTERCEPTED] = { .n = IST("intercepted_requests_total"), .type = PROMEX_MT_COUNTER, .flags = (PROMEX_FL_FRONT_METRIC ) },
[ST_F_DCON] = { .n = IST("denied_connections_total"), .type = PROMEX_MT_COUNTER, .flags = (PROMEX_FL_FRONT_METRIC | PROMEX_FL_LI_METRIC ) },
[ST_F_DSES] = { .n = IST("denied_sessions_total"), .type = PROMEX_MT_COUNTER, .flags = (PROMEX_FL_FRONT_METRIC | PROMEX_FL_LI_METRIC ) },
[ST_F_WREW] = { .n = IST("failed_header_rewriting_total"), .type = PROMEX_MT_COUNTER, .flags = (PROMEX_FL_FRONT_METRIC | PROMEX_FL_LI_METRIC | PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
[ST_F_CONNECT] = { .n = IST("connection_attempts_total"), .type = PROMEX_MT_COUNTER, .flags = ( PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
[ST_F_REUSE] = { .n = IST("connection_reuses_total"), .type = PROMEX_MT_COUNTER, .flags = ( PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
[ST_F_CACHE_LOOKUPS] = { .n = IST("http_cache_lookups_total"), .type = PROMEX_MT_COUNTER, .flags = (PROMEX_FL_FRONT_METRIC | PROMEX_FL_BACK_METRIC ) },
[ST_F_CACHE_HITS] = { .n = IST("http_cache_hits_total"), .type = PROMEX_MT_COUNTER, .flags = (PROMEX_FL_FRONT_METRIC | PROMEX_FL_BACK_METRIC ) },
[ST_F_SRV_ICUR] = { .n = IST("idle_connections_current"), .type = PROMEX_MT_GAUGE, .flags = ( PROMEX_FL_SRV_METRIC) },
[ST_F_SRV_ILIM] = { .n = IST("idle_connections_limit"), .type = PROMEX_MT_GAUGE, .flags = ( PROMEX_FL_SRV_METRIC) },
[ST_F_QT_MAX] = { .n = IST("max_queue_time_seconds"), .type = PROMEX_MT_GAUGE, .flags = ( PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
[ST_F_CT_MAX] = { .n = IST("max_connect_time_seconds"), .type = PROMEX_MT_GAUGE, .flags = ( PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
[ST_F_RT_MAX] = { .n = IST("max_response_time_seconds"), .type = PROMEX_MT_GAUGE, .flags = ( PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
[ST_F_TT_MAX] = { .n = IST("max_total_time_seconds"), .type = PROMEX_MT_GAUGE, .flags = ( PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
[ST_F_EINT] = { .n = IST("internal_errors_total"), .type = PROMEX_MT_COUNTER, .flags = (PROMEX_FL_FRONT_METRIC | PROMEX_FL_LI_METRIC | PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
[ST_F_IDLE_CONN_CUR] = { .n = IST("unsafe_idle_connections_current"), .type = PROMEX_MT_GAUGE, .flags = ( PROMEX_FL_SRV_METRIC) },
[ST_F_SAFE_CONN_CUR] = { .n = IST("safe_idle_connections_current"), .type = PROMEX_MT_GAUGE, .flags = ( PROMEX_FL_SRV_METRIC) },
[ST_F_USED_CONN_CUR] = { .n = IST("used_connections_current"), .type = PROMEX_MT_GAUGE, .flags = ( PROMEX_FL_SRV_METRIC) },
[ST_F_NEED_CONN_EST] = { .n = IST("need_connections_current"), .type = PROMEX_MT_GAUGE, .flags = ( PROMEX_FL_SRV_METRIC) },
[ST_F_UWEIGHT] = { .n = IST("uweight"), .type = PROMEX_MT_GAUGE, .flags = ( PROMEX_FL_BACK_METRIC | PROMEX_FL_SRV_METRIC) },
[ST_F_AGG_SRV_CHECK_STATUS] = { .n = IST("agg_server_check_status"), .type = PROMEX_MT_GAUGE, .flags = ( PROMEX_FL_BACK_METRIC ) },
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
};
CLEANUP: assorted typo fixes in the code and comments This is 17th iteration of typo fixes
2021-02-06 17:29:08 +00:00
/* Description of overridden stats fields */
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
const struct ist promex_st_metric_desc[ST_F_TOTAL_FIELDS] = {
MINOR: contrib/prometheus-exporter: improve service status description field Since we changed the behaviour of this metric, improve the description to better explain what is the meaning of the new gauge value; it also reflects the description we did for health check status. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-01 12:11:52 +00:00
[ST_F_STATUS] = IST("Current status of the service, per state label value."),
MAJOR: contrib/prometheus-exporter: move health check status to labels this patch is a breaking change between v2.3 and v2.4: we move from using gauge value for health check states to labels values. The diff is quite small thanks to the preparation work from Christopher to allow more flexibility in labels, see commit 5a2f938732126f43bbf4cea5482c01552b0d0314 ("MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones") this is a follow up of commit c6464591a365bfcf509b322bdaa4d608c9395d75 ("MAJOR: contrib/prometheus-exporter: move ftd/bkd/srv states to labels"). The main goal being to be better aligned with prometheus use cases in terms of queries. More specifically to health checks, Pierre C. mentioned the possible quirks he had to put in place in order to make use of those metrics through prometheus: <aggregator_function> by(proxy, check_status) (count_values by(proxy, instance) ("check_status", haproxy_server_check_status)) I am perfectly aware this introduces a lot more metrics but I don't see how we can improve the usability without it. The main issue remains in the cardinality of the states which are > 20. Prometheus recommends to stay below a cardinality of 10 for a given metric but I consider our case very specific, because highly linked to the level of precision haproxy exposes. Even before this patch I saw several large production setup (a few hundreds of MB in output) which are making use of the scope parameter to simply ignore the server metrics, so that the scrapping can be faster, and memory consumed on client side not too high. So I believe we should eventually continue in that direction and offer more granularity of filtering of the output. That being said it is already possible to filter out the data on prometheus client side. this is related to github issue #1029 Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-01 12:11:51 +00:00
[ST_F_CHECK_STATUS] = IST("Status of last health check, per state label value."),
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
[ST_F_CHECK_CODE] = IST("layer5-7 code, if available of the last health check."),
MINOR: contrib/prometheus-exporter: Add the last heathcheck duration metric ST_F_CHECK_DURATION is now part of exported server metrics, named haproxy_server_check_duration_seconds and expressed in seconds. For a given server, this value is exported only if the healthcheck is finished (the status is greater or equal to HCHK_STATUS_CHECKED). This patch fixes the issue #519. It may be backported as fat as 2.0.
2020-02-27 15:12:07 +00:00
[ST_F_CHECK_DURATION] = IST("Total duration of the latest server health check, in seconds."),
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
[ST_F_QTIME] = IST("Avg. queue time for last 1024 successful connections."),
[ST_F_CTIME] = IST("Avg. connect time for last 1024 successful connections."),
[ST_F_RTIME] = IST("Avg. response time for last 1024 successful connections."),
[ST_F_TTIME] = IST("Avg. total time for last 1024 successful connections."),
MINOR: contrib/prometheus-exporter: Report metrics about max times for sessions Now, for the sessions, the maximum times (queue, connect, response, total) are reported in addition of the averages over the last 1024 connections. These metrics are reported per backend and per server. Here are the metrics name : * haproxy_backend_max_queue_time_seconds * haproxy_backend_max_connect_time_seconds * haproxy_backend_max_response_time_seconds * haproxy_backend_max_total_time_seconds and * haproxy_server_max_queue_time_seconds * haproxy_server_max_connect_time_seconds * haproxy_server_max_response_time_seconds * haproxy_server_max_total_time_seconds This patch is related to #272.
2019-11-08 14:05:31 +00:00
[ST_F_QT_MAX] = IST("Maximum observed time spent in the queue"),
[ST_F_CT_MAX] = IST("Maximum observed time spent waiting for a connection to complete"),
[ST_F_RT_MAX] = IST("Maximum observed time spent waiting for a server response"),
[ST_F_TT_MAX] = IST("Maximum observed total request+response time (request+queue+connect+response+processing)"),
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
};
MEDIUM: contrib/prometheus-exporter: export base stick table stats I saw some people falling back to unix socket to collect some data they could not find in prometheus exporter. One of them is base info from stick tables (used/size). I do not plan to extend it more for now; keys are quite a mess to handle. This should resolve github issue #1008. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-07 19:42:38 +00:00
/* stick table base fields */
enum sticktable_field {
STICKTABLE_SIZE = 0,
STICKTABLE_USED,
/* must always be the last one */
STICKTABLE_TOTAL_FIELDS
};
const struct promex_metric promex_sticktable_metrics[STICKTABLE_TOTAL_FIELDS] = {
[STICKTABLE_SIZE] = { .n = IST("size"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_STICKTABLE_METRIC },
[STICKTABLE_USED] = { .n = IST("used"), .type = PROMEX_MT_GAUGE, .flags = PROMEX_FL_STICKTABLE_METRIC },
};
/* stick table base description */
const struct ist promex_sticktable_metric_desc[STICKTABLE_TOTAL_FIELDS] = {
[STICKTABLE_SIZE] = IST("Stick table size."),
[STICKTABLE_USED] = IST("Number of entries used in this stick table."),
};
MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones Instead of using static labels for metrics, we now use an array of labels, filled for each metrics if necessary and passed to the dump function. This way, it is easier to extend the promex service. For now, there are at most 8 labels per metrics. This limit may be raised by changing PROMEX_MAX_LABELS value. And to ease labels addition, a label is defined as a key/value pair. The formatting is handled by the dump function. For the proxies and servers, the first entry of the array is always the proxy name. In addition, for the servers, the second entry is always the server name.
2021-01-28 10:24:17 +00:00
/* Specific labels for all ST_F_HRSP_* fields */
const struct ist promex_hrsp_code[1 + ST_F_HRSP_OTHER - ST_F_HRSP_1XX] = {
[ST_F_HRSP_1XX - ST_F_HRSP_1XX] = IST("1xx"),
[ST_F_HRSP_2XX - ST_F_HRSP_1XX] = IST("2xx"),
[ST_F_HRSP_3XX - ST_F_HRSP_1XX] = IST("3xx"),
[ST_F_HRSP_4XX - ST_F_HRSP_1XX] = IST("4xx"),
[ST_F_HRSP_5XX - ST_F_HRSP_1XX] = IST("5xx"),
[ST_F_HRSP_OTHER - ST_F_HRSP_1XX] = IST("other"),
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
};
MINOR: contrib/prometheus-exporter: declare states for objects in preparation to change state gauge values as labels, declare them as enum associated with the string definition Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-27 21:40:16 +00:00
enum promex_front_state {
PROMEX_FRONT_STATE_DOWN = 0,
PROMEX_FRONT_STATE_UP,
PROMEX_FRONT_STATE_COUNT /* must be last */
};
const struct ist promex_front_st[PROMEX_FRONT_STATE_COUNT] = {
[PROMEX_FRONT_STATE_DOWN] = IST("DOWN"),
[PROMEX_FRONT_STATE_UP] = IST("UP"),
};
enum promex_back_state {
PROMEX_BACK_STATE_DOWN = 0,
PROMEX_BACK_STATE_UP,
PROMEX_BACK_STATE_COUNT /* must be last */
};
const struct ist promex_back_st[PROMEX_BACK_STATE_COUNT] = {
[PROMEX_BACK_STATE_DOWN] = IST("DOWN"),
[PROMEX_BACK_STATE_UP] = IST("UP"),
};
enum promex_srv_state {
PROMEX_SRV_STATE_DOWN = 0,
PROMEX_SRV_STATE_UP,
PROMEX_SRV_STATE_MAINT,
PROMEX_SRV_STATE_DRAIN,
PROMEX_SRV_STATE_NOLB,
PROMEX_SRV_STATE_COUNT /* must be last */
};
const struct ist promex_srv_st[PROMEX_SRV_STATE_COUNT] = {
[PROMEX_SRV_STATE_DOWN] = IST("DOWN"),
[PROMEX_SRV_STATE_UP] = IST("UP"),
[PROMEX_SRV_STATE_MAINT] = IST("MAINT"),
[PROMEX_SRV_STATE_DRAIN] = IST("DRAIN"),
[PROMEX_SRV_STATE_NOLB] = IST("NOLB"),
};
/* Return the server status. */
enum promex_srv_state promex_srv_status(struct server *sv)
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
{
MINOR: contrib/prometheus-exporter: declare states for objects in preparation to change state gauge values as labels, declare them as enum associated with the string definition Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-27 21:40:16 +00:00
int state = PROMEX_SRV_STATE_DOWN;
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
if (sv->cur_state == SRV_ST_RUNNING || sv->cur_state == SRV_ST_STARTING) {
MINOR: contrib/prometheus-exporter: declare states for objects in preparation to change state gauge values as labels, declare them as enum associated with the string definition Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-27 21:40:16 +00:00
state = PROMEX_SRV_STATE_UP;
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
if (sv->cur_admin & SRV_ADMF_DRAIN)
MINOR: contrib/prometheus-exporter: declare states for objects in preparation to change state gauge values as labels, declare them as enum associated with the string definition Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-27 21:40:16 +00:00
state = PROMEX_SRV_STATE_DRAIN;
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
}
MINOR: contrib/prometheus-exporter: Report DRAIN/MAINT/NOLB status for servers Now, following status are reported for servers:0=DOWN, 1=UP, 2=MAINT, 3=DRAIN, 4=NOLB. It is linked to the github issue #255. Thanks to Mickaël Martin. If needed, this patch may be backported to 2.0.
2019-09-06 14:10:19 +00:00
else if (sv->cur_state == SRV_ST_STOPPING)
MINOR: contrib/prometheus-exporter: declare states for objects in preparation to change state gauge values as labels, declare them as enum associated with the string definition Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-27 21:40:16 +00:00
state = PROMEX_SRV_STATE_NOLB;
MINOR: contrib/prometheus-exporter: Report DRAIN/MAINT/NOLB status for servers Now, following status are reported for servers:0=DOWN, 1=UP, 2=MAINT, 3=DRAIN, 4=NOLB. It is linked to the github issue #255. Thanks to Mickaël Martin. If needed, this patch may be backported to 2.0.
2019-09-06 14:10:19 +00:00
if (sv->cur_admin & SRV_ADMF_MAINT)
MINOR: contrib/prometheus-exporter: declare states for objects in preparation to change state gauge values as labels, declare them as enum associated with the string definition Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-27 21:40:16 +00:00
state = PROMEX_SRV_STATE_MAINT;
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
return state;
}
/* Convert a field to its string representation and write it in <out>, followed
* by a newline, if there is enough space. non-numeric value are converted in
MINOR: contrib/prometheus-exporter: better output of Not-a-Number Not necessarily mandatory but I saw a few prometheus client parsing only `NaN`. Also most librarries do output `NaN` Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-22 20:09:47 +00:00
* "NaN" because Prometheus only support numerical values (but it is unexepceted
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
* to process this kind of value). It returns 1 on success. Otherwise, it
* returns 0. The buffer's length must not exceed <max> value.
*/
static int promex_metric_to_str(struct buffer *out, struct field *f, size_t max)
{
int ret = 0;
switch (field_format(f, 0)) {
MINOR: contrib/prometheus-exporter: better output of Not-a-Number Not necessarily mandatory but I saw a few prometheus client parsing only `NaN`. Also most librarries do output `NaN` Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-22 20:09:47 +00:00
case FF_EMPTY: ret = chunk_strcat(out, "NaN\n"); break;
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
case FF_S32: ret = chunk_appendf(out, "%d\n", f->u.s32); break;
case FF_U32: ret = chunk_appendf(out, "%u\n", f->u.u32); break;
case FF_S64: ret = chunk_appendf(out, "%lld\n", (long long)f->u.s64); break;
case FF_U64: ret = chunk_appendf(out, "%llu\n", (unsigned long long)f->u.u64); break;
BUG/MINOR: contrib/prometheus-exporter: Return the time averages in seconds The metrics QTIME, CTIME, RTIME and TTIME are now returned in seconds using a float representation instead of in milliseconds. So these metrics are now consistent with their announced type and respect Prometheus naming conventions. This patch fixes the issue #288. It may be backported to 2.0. If so, the previous patch, introducing the support for float fields in stats is mantatory and should be backported first.
2019-09-24 14:35:19 +00:00
case FF_FLT: ret = chunk_appendf(out, "%f\n", f->u.flt); break;
MINOR: contrib/prometheus-exporter: better output of Not-a-Number Not necessarily mandatory but I saw a few prometheus client parsing only `NaN`. Also most librarries do output `NaN` Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-22 20:09:47 +00:00
case FF_STR: ret = chunk_strcat(out, "NaN\n"); break;
default: ret = chunk_strcat(out, "NaN\n"); break;
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
}
if (!ret || out->data > max)
return 0;
return 1;
}
/* Dump the header lines for <metric>. It is its #HELP and #TYPE strings. It
* returns 1 on success. Otherwise, if <out> length exceeds <max>, it returns 0.
*/
static int promex_dump_metric_header(struct appctx *appctx, struct htx *htx,
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
const struct promex_metric *metric, const struct ist name,
struct ist *out, size_t max)
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
{
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
struct ist type;
MINOR: contrib/prometheus-exporter: use stats desc when possible It is a followup work of commit a191b77e54c26a97064cb42ab4927d4f5c95b896 ("MINOR: contrib/prometheus-exporter: merge info description from stats") but for all other stats fields; we however keep a way to override them when needed (e.g. units, specific cases) this is another step which will avoid duplicating work between stats.c and prometheus. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-01 12:11:55 +00:00
struct ist desc;
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
switch (metric->type) {
case PROMEX_MT_COUNTER:
type = ist("counter");
break;
default:
type = ist("gauge");
}
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
MINOR: contrib/prometheus-exporter: merge info description from stats Now that units are coherent we can merge the info description from haproxy stats. Description were not always the same, but I guess we may eventually improve them in the future. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-15 21:41:39 +00:00
if (istcat(out, ist("# HELP "), max) == -1 ||
istcat(out, name, max) == -1 ||
istcat(out, ist(" "), max) == -1)
goto full;
MINOR: contrib/prometheus-exporter: use stats desc when possible It is a followup work of commit a191b77e54c26a97064cb42ab4927d4f5c95b896 ("MINOR: contrib/prometheus-exporter: merge info description from stats") but for all other stats fields; we however keep a way to override them when needed (e.g. units, specific cases) this is another step which will avoid duplicating work between stats.c and prometheus. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-01 12:11:55 +00:00
if (metric->flags & PROMEX_FL_INFO_METRIC)
desc = ist(info_fields[appctx->st2].desc);
MEDIUM: contrib/prometheus-exporter: export base stick table stats I saw some people falling back to unix socket to collect some data they could not find in prometheus exporter. One of them is base info from stick tables (used/size). I do not plan to extend it more for now; keys are quite a mess to handle. This should resolve github issue #1008. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-07 19:42:38 +00:00
else if (metric->flags & PROMEX_FL_STICKTABLE_METRIC)
desc = promex_sticktable_metric_desc[appctx->st2];
MINOR: contrib/prometheus-exporter: use stats desc when possible It is a followup work of commit a191b77e54c26a97064cb42ab4927d4f5c95b896 ("MINOR: contrib/prometheus-exporter: merge info description from stats") but for all other stats fields; we however keep a way to override them when needed (e.g. units, specific cases) this is another step which will avoid duplicating work between stats.c and prometheus. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-01 12:11:55 +00:00
else if (!isttest(promex_st_metric_desc[appctx->st2]))
desc = ist(stat_fields[appctx->st2].desc);
else
desc = promex_st_metric_desc[appctx->st2];
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
MINOR: contrib/prometheus-exporter: use stats desc when possible It is a followup work of commit a191b77e54c26a97064cb42ab4927d4f5c95b896 ("MINOR: contrib/prometheus-exporter: merge info description from stats") but for all other stats fields; we however keep a way to override them when needed (e.g. units, specific cases) this is another step which will avoid duplicating work between stats.c and prometheus. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-01 12:11:55 +00:00
if (istcat(out, desc, max) == -1 ||
istcat(out, ist("\n# TYPE "), max) == -1 ||
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
istcat(out, name, max) == -1 ||
istcat(out, ist(" "), max) == -1 ||
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
istcat(out, type, max) == -1 ||
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
istcat(out, ist("\n"), max) == -1)
goto full;
return 1;
full:
return 0;
}
/* Dump the line for <metric>. It starts by the metric name followed by its
* labels (proxy name, server name...) between braces and finally its value. If
* not already done, the header lines are dumped first. It returns 1 on
* success. Otherwise if <out> length exceeds <max>, it returns 0.
*/
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
static int promex_dump_metric(struct appctx *appctx, struct htx *htx, struct ist prefix,
MAJOR: contrib/prometheus-exporter: move ftd/bkd/srv states to labels this patch is a breaking change between v2.3 and v2.4: we move from using gauge value for frontend/backend/servers states to labels values. the main motivation being I realised it is very difficult to make use of it without hard coded quirks on prometheus client side; especially because the main use is often to group by state, which is harder when the state is the value of the metric. in order to achieve that we iterate on the status metric to generate labels, and so as many metrics. this is the first step to resolve github issue #1029 A second step should address health check states. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-27 21:40:17 +00:00
const struct promex_metric *metric, struct field *val,
MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones Instead of using static labels for metrics, we now use an array of labels, filled for each metrics if necessary and passed to the dump function. This way, it is easier to extend the promex service. For now, there are at most 8 labels per metrics. This limit may be raised by changing PROMEX_MAX_LABELS value. And to ease labels addition, a label is defined as a key/value pair. The formatting is handled by the dump function. For the proxies and servers, the first entry of the array is always the proxy name. In addition, for the servers, the second entry is always the server name.
2021-01-28 10:24:17 +00:00
struct promex_label *labels, struct ist *out, size_t max)
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
{
struct ist name = { .ptr = (char[PROMEX_MAX_NAME_LEN]){ 0 }, .len = 0 };
size_t len = out->len;
if (out->len + PROMEX_MAX_METRIC_LENGTH > max)
return 0;
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
/* Fill the metric name */
istcat(&name, prefix, PROMEX_MAX_NAME_LEN);
istcat(&name, metric->n, PROMEX_MAX_NAME_LEN);
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
if ((appctx->ctx.stats.flags & PROMEX_FL_METRIC_HDR) &&
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
!promex_dump_metric_header(appctx, htx, metric, name, out, max))
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
goto full;
MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones Instead of using static labels for metrics, we now use an array of labels, filled for each metrics if necessary and passed to the dump function. This way, it is easier to extend the promex service. For now, there are at most 8 labels per metrics. This limit may be raised by changing PROMEX_MAX_LABELS value. And to ease labels addition, a label is defined as a key/value pair. The formatting is handled by the dump function. For the proxies and servers, the first entry of the array is always the proxy name. In addition, for the servers, the second entry is always the server name.
2021-01-28 10:24:17 +00:00
if (istcat(out, name, max) == -1)
goto full;
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones Instead of using static labels for metrics, we now use an array of labels, filled for each metrics if necessary and passed to the dump function. This way, it is easier to extend the promex service. For now, there are at most 8 labels per metrics. This limit may be raised by changing PROMEX_MAX_LABELS value. And to ease labels addition, a label is defined as a key/value pair. The formatting is handled by the dump function. For the proxies and servers, the first entry of the array is always the proxy name. In addition, for the servers, the second entry is always the server name.
2021-01-28 10:24:17 +00:00
if (isttest(labels[0].name)) {
int i;
if (istcat(out, ist("{"), max) == -1)
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
goto full;
MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones Instead of using static labels for metrics, we now use an array of labels, filled for each metrics if necessary and passed to the dump function. This way, it is easier to extend the promex service. For now, there are at most 8 labels per metrics. This limit may be raised by changing PROMEX_MAX_LABELS value. And to ease labels addition, a label is defined as a key/value pair. The formatting is handled by the dump function. For the proxies and servers, the first entry of the array is always the proxy name. In addition, for the servers, the second entry is always the server name.
2021-01-28 10:24:17 +00:00
for (i = 0; isttest(labels[i].name); i++) {
if (!isttest(labels[i].value))
continue;
if ((i && istcat(out, ist(","), max) == -1) ||
istcat(out, labels[i].name, max) == -1 ||
istcat(out, ist("=\""), max) == -1 ||
istcat(out, labels[i].value, max) == -1 ||
istcat(out, ist("\""), max) == -1)
goto full;
}
if (istcat(out, ist("}"), max) == -1)
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
goto full;
MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones Instead of using static labels for metrics, we now use an array of labels, filled for each metrics if necessary and passed to the dump function. This way, it is easier to extend the promex service. For now, there are at most 8 labels per metrics. This limit may be raised by changing PROMEX_MAX_LABELS value. And to ease labels addition, a label is defined as a key/value pair. The formatting is handled by the dump function. For the proxies and servers, the first entry of the array is always the proxy name. In addition, for the servers, the second entry is always the server name.
2021-01-28 10:24:17 +00:00
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
}
MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones Instead of using static labels for metrics, we now use an array of labels, filled for each metrics if necessary and passed to the dump function. This way, it is easier to extend the promex service. For now, there are at most 8 labels per metrics. This limit may be raised by changing PROMEX_MAX_LABELS value. And to ease labels addition, a label is defined as a key/value pair. The formatting is handled by the dump function. For the proxies and servers, the first entry of the array is always the proxy name. In addition, for the servers, the second entry is always the server name.
2021-01-28 10:24:17 +00:00
if (istcat(out, ist(" "), max) == -1)
goto full;
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
trash.data = out->len;
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
if (!promex_metric_to_str(&trash, val, max))
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
goto full;
out->len = trash.data;
appctx->ctx.stats.flags &= ~PROMEX_FL_METRIC_HDR;
return 1;
full:
// Restore previous length
out->len = len;
return 0;
}
CLEANUP: assorted typo fixes in the code and comments This is fifth iteration of typo fixes
2020-03-23 17:28:40 +00:00
/* Dump global metrics (prefixed by "haproxy_process_"). It returns 1 on success,
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
* 0 if <htx> is full and -1 in case of any error. */
static int promex_dump_global_metrics(struct appctx *appctx, struct htx *htx)
{
static struct ist prefix = IST("haproxy_process_");
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
struct field val;
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
struct channel *chn = si_ic(appctx->owner);
struct ist out = ist2(trash.area, 0);
BUG/MINOR: contrib/prometheus-exporter: Respect the reserve when data are sent The previous commit e6cdfe574 ("BUG/MINOR: contrib/prometheus-exporter: Don't use channel_htx_recv_max()") is buggy. The buffer's reserve must be respected. This patch must be backported to 2.0 and 1.9.
2019-07-03 09:43:17 +00:00
size_t max = htx_get_max_blksz(htx, channel_htx_recv_max(chn, htx));
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
int ret = 1;
MINOR: stats: pass the appctx flags to stats_fill_info() Currently the stats filling function knows nothing about the caller's needs, so let's pass the STAT_* flags so that it can adapt to the requester's constraints.
2021-05-08 05:43:53 +00:00
if (!stats_fill_info(info, INF_TOTAL_FIELDS, 0))
MINOR: contrib/prometheus-exporter: use fill_info for process dump use `stats_fill_info` when possible to avoid duplicating code. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-11 19:07:49 +00:00
return -1;
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
for (; appctx->st2 < INF_TOTAL_FIELDS; appctx->st2++) {
MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones Instead of using static labels for metrics, we now use an array of labels, filled for each metrics if necessary and passed to the dump function. This way, it is easier to extend the promex service. For now, there are at most 8 labels per metrics. This limit may be raised by changing PROMEX_MAX_LABELS value. And to ease labels addition, a label is defined as a key/value pair. The formatting is handled by the dump function. For the proxies and servers, the first entry of the array is always the proxy name. In addition, for the servers, the second entry is always the server name.
2021-01-28 10:24:17 +00:00
struct promex_label labels[PROMEX_MAX_LABELS-1] = {};
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
if (!(promex_global_metrics[appctx->st2].flags & appctx->ctx.stats.flags))
continue;
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
switch (appctx->st2) {
MINOR: contrib/prometheus-exporter: export build_info commit c55a626217a7e676e1cc ("MINOR: contrib/prometheus-exporter: Add missing global and per-server metrics") is renaming two metrics between v2.2 and v2.3: server_idle_connections_current server_idle_connections_limit It is breaking some tools which are making use of those metrics while supporting several haproxy versions. This build_info will permit tools which make use of metrics to be able to match the haproxy version and change the list of expected metrics. This was possible using the haproxy stats socket but not with prometheus export. This patch follows prometheus best pratices to export specific software informations. It is adding a new field `build_info` so we can extend it to other parameters if needed in the future. example output: # HELP haproxy_process_build_info HAProxy build info. # TYPE haproxy_process_build_info gauge haproxy_process_build_info{version="2.4-dev5-2e1a3f-5"} 1 Even though it is not a bugfix, this patch will make more sense when backported up to >= 2.0 Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-08 12:18:06 +00:00
case INF_BUILD_INFO:
MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones Instead of using static labels for metrics, we now use an array of labels, filled for each metrics if necessary and passed to the dump function. This way, it is easier to extend the promex service. For now, there are at most 8 labels per metrics. This limit may be raised by changing PROMEX_MAX_LABELS value. And to ease labels addition, a label is defined as a key/value pair. The formatting is handled by the dump function. For the proxies and servers, the first entry of the array is always the proxy name. In addition, for the servers, the second entry is always the server name.
2021-01-28 10:24:17 +00:00
labels[0].name = ist("version");
labels[0].value = ist(HAPROXY_VERSION);
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
val = mkf_u32(FN_GAUGE, 1);
MINOR: contrib/prometheus-exporter: export build_info commit c55a626217a7e676e1cc ("MINOR: contrib/prometheus-exporter: Add missing global and per-server metrics") is renaming two metrics between v2.2 and v2.3: server_idle_connections_current server_idle_connections_limit It is breaking some tools which are making use of those metrics while supporting several haproxy versions. This build_info will permit tools which make use of metrics to be able to match the haproxy version and change the list of expected metrics. This was possible using the haproxy stats socket but not with prometheus export. This patch follows prometheus best pratices to export specific software informations. It is adding a new field `build_info` so we can extend it to other parameters if needed in the future. example output: # HELP haproxy_process_build_info HAProxy build info. # TYPE haproxy_process_build_info gauge haproxy_process_build_info{version="2.4-dev5-2e1a3f-5"} 1 Even though it is not a bugfix, this patch will make more sense when backported up to >= 2.0 Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-08 12:18:06 +00:00
break;
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
default:
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
val = info[appctx->st2];
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
}
MAJOR: contrib/prometheus-exporter: move ftd/bkd/srv states to labels this patch is a breaking change between v2.3 and v2.4: we move from using gauge value for frontend/backend/servers states to labels values. the main motivation being I realised it is very difficult to make use of it without hard coded quirks on prometheus client side; especially because the main use is often to group by state, which is harder when the state is the value of the metric. in order to achieve that we iterate on the status metric to generate labels, and so as many metrics. this is the first step to resolve github issue #1029 A second step should address health check states. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-27 21:40:17 +00:00
if (!promex_dump_metric(appctx, htx, prefix, &promex_global_metrics[appctx->st2],
MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones Instead of using static labels for metrics, we now use an array of labels, filled for each metrics if necessary and passed to the dump function. This way, it is easier to extend the promex service. For now, there are at most 8 labels per metrics. This limit may be raised by changing PROMEX_MAX_LABELS value. And to ease labels addition, a label is defined as a key/value pair. The formatting is handled by the dump function. For the proxies and servers, the first entry of the array is always the proxy name. In addition, for the servers, the second entry is always the server name.
2021-01-28 10:24:17 +00:00
&val, labels, &out, max))
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
goto full;
appctx->ctx.stats.flags |= PROMEX_FL_METRIC_HDR;
}
end:
BUG/MINOR: contrib/prometheus-exporter: Don't try to add empty data blocks When the response buffer is full and nothing more can be inserted, it is important to not try to insert an empty data block. Otherwise, when the function channel_add_input() is called, the flag CF_READ_PARTIAL is set on the response channel while nothing was read and the stream is uselessly woken up. Finally, we have loop while the response buffer is full. This patch must be backported to 2.0.
2019-07-04 08:03:28 +00:00
if (out.len) {
if (!htx_add_data_atonce(htx, out))
return -1; /* Unexpected and unrecoverable error */
channel_add_input(chn, out.len);
}
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
return ret;
full:
ret = 0;
goto end;
}
CLEANUP: assorted typo fixes in the code and comments This is fifth iteration of typo fixes
2020-03-23 17:28:40 +00:00
/* Dump frontends metrics (prefixed by "haproxy_frontend_"). It returns 1 on success,
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
* 0 if <htx> is full and -1 in case of any error. */
static int promex_dump_front_metrics(struct appctx *appctx, struct htx *htx)
{
static struct ist prefix = IST("haproxy_frontend_");
struct proxy *px;
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
struct field val;
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
struct channel *chn = si_ic(appctx->owner);
struct ist out = ist2(trash.area, 0);
BUG/MINOR: contrib/prometheus-exporter: Respect the reserve when data are sent The previous commit e6cdfe574 ("BUG/MINOR: contrib/prometheus-exporter: Don't use channel_htx_recv_max()") is buggy. The buffer's reserve must be respected. This patch must be backported to 2.0 and 1.9.
2019-07-03 09:43:17 +00:00
size_t max = htx_get_max_blksz(htx, channel_htx_recv_max(chn, htx));
MINOR: contrib/prometheus-exporter: use fill_fe_stats for frontend dump use `stats_fill_fe_stats` when possible to avoid duplicating code; make use of field selector to get the needed field only. this should not introduce any difference of output. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-17 17:27:46 +00:00
struct field *stats = stat_l[STATS_DOMAIN_PROXY];
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
int ret = 1;
MAJOR: contrib/prometheus-exporter: move ftd/bkd/srv states to labels this patch is a breaking change between v2.3 and v2.4: we move from using gauge value for frontend/backend/servers states to labels values. the main motivation being I realised it is very difficult to make use of it without hard coded quirks on prometheus client side; especially because the main use is often to group by state, which is harder when the state is the value of the metric. in order to achieve that we iterate on the status metric to generate labels, and so as many metrics. this is the first step to resolve github issue #1029 A second step should address health check states. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-27 21:40:17 +00:00
enum promex_front_state state;
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
for (;appctx->st2 < ST_F_TOTAL_FIELDS; appctx->st2++) {
if (!(promex_st_metrics[appctx->st2].flags & appctx->ctx.stats.flags))
continue;
MINOR: stats: hide px/sv/li fields in applet struct Use an opaque pointer to store proxy instance. Regroup server/listener as a single opaque pointer. This has the benefit to render the structure more evolutive to support statistics on other types of objects in the future. This patch is needed to extend stat support for components other than proxies objects. The prometheus module has been adapted for these changes.
2020-10-02 16:32:02 +00:00
while (appctx->ctx.stats.obj1) {
MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones Instead of using static labels for metrics, we now use an array of labels, filled for each metrics if necessary and passed to the dump function. This way, it is easier to extend the promex service. For now, there are at most 8 labels per metrics. This limit may be raised by changing PROMEX_MAX_LABELS value. And to ease labels addition, a label is defined as a key/value pair. The formatting is handled by the dump function. For the proxies and servers, the first entry of the array is always the proxy name. In addition, for the servers, the second entry is always the server name.
2021-01-28 10:24:17 +00:00
struct promex_label labels[PROMEX_MAX_LABELS-1] = {};
MINOR: stats: hide px/sv/li fields in applet struct Use an opaque pointer to store proxy instance. Regroup server/listener as a single opaque pointer. This has the benefit to render the structure more evolutive to support statistics on other types of objects in the future. This patch is needed to extend stat support for components other than proxies objects. The prometheus module has been adapted for these changes.
2020-10-02 16:32:02 +00:00
px = appctx->ctx.stats.obj1;
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones Instead of using static labels for metrics, we now use an array of labels, filled for each metrics if necessary and passed to the dump function. This way, it is easier to extend the promex service. For now, there are at most 8 labels per metrics. This limit may be raised by changing PROMEX_MAX_LABELS value. And to ease labels addition, a label is defined as a key/value pair. The formatting is handled by the dump function. For the proxies and servers, the first entry of the array is always the proxy name. In addition, for the servers, the second entry is always the server name.
2021-01-28 10:24:17 +00:00
labels[0].name = ist("proxy");
labels[0].value = ist2(px->id, strlen(px->id));
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
/* skip the disabled proxies, global frontend and non-networked ones */
MINOR: proxy: Introduce proxy flags to replace disabled bitfield This change is required to support TCP/HTTP rules in defaults sections. The 'disabled' bitfield in the proxy structure, used to know if a proxy is disabled or stopped, is replaced a generic bitfield named 'flags'. PR_DISABLED and PR_STOPPED flags are renamed to PR_FL_DISABLED and PR_FL_STOPPED respectively. In addition, everywhere there is a test to know if a proxy is disabled or stopped, there is now a bitwise AND operation on PR_FL_DISABLED and/or PR_FL_STOPPED flags.
2021-10-06 12:24:19 +00:00
if ((px->flags & PR_FL_DISABLED) || px->uuid <= 0 || !(px->cap & PR_CAP_FE))
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
goto next_px;
MINOR: contrib/prometheus-exporter: use fill_fe_stats for frontend dump use `stats_fill_fe_stats` when possible to avoid duplicating code; make use of field selector to get the needed field only. this should not introduce any difference of output. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-17 17:27:46 +00:00
if (!stats_fill_fe_stats(px, stats, ST_F_TOTAL_FIELDS, &(appctx->st2)))
return -1;
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
switch (appctx->st2) {
case ST_F_STATUS:
MINOR: proxy: Introduce proxy flags to replace disabled bitfield This change is required to support TCP/HTTP rules in defaults sections. The 'disabled' bitfield in the proxy structure, used to know if a proxy is disabled or stopped, is replaced a generic bitfield named 'flags'. PR_DISABLED and PR_STOPPED flags are renamed to PR_FL_DISABLED and PR_FL_STOPPED respectively. In addition, everywhere there is a test to know if a proxy is disabled or stopped, there is now a bitwise AND operation on PR_FL_DISABLED and/or PR_FL_STOPPED flags.
2021-10-06 12:24:19 +00:00
state = !(px->flags & PR_FL_STOPPED);
BUG/MINOR: contrib/prometheus-exporter: Restart labels dump at the right pos For some metrics, several lines are produced per entity, one per label value. For instance, the health-check status (ST_F_CHECK_STATUS) or the entity status (ST_F_STATUS). The dump may be stopped in the middle of the labels processing if the output buffer is full. This means the next time, we must take care to restart on the right label value. For now, this part is buggy and we always restart to dump all the label values again from the beginning. To be sure to restart at the right position, the field <ctx.stats.st_code> in the applet context is used to save the last position. Of course, we take care to reset this value when necessary. This fix is specific for 2.4. No backport needed.
2021-02-01 14:05:21 +00:00
for (; appctx->ctx.stats.st_code < PROMEX_FRONT_STATE_COUNT; appctx->ctx.stats.st_code++) {
MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones Instead of using static labels for metrics, we now use an array of labels, filled for each metrics if necessary and passed to the dump function. This way, it is easier to extend the promex service. For now, there are at most 8 labels per metrics. This limit may be raised by changing PROMEX_MAX_LABELS value. And to ease labels addition, a label is defined as a key/value pair. The formatting is handled by the dump function. For the proxies and servers, the first entry of the array is always the proxy name. In addition, for the servers, the second entry is always the server name.
2021-01-28 10:24:17 +00:00
labels[1].name = ist("state");
BUG/MINOR: contrib/prometheus-exporter: Restart labels dump at the right pos For some metrics, several lines are produced per entity, one per label value. For instance, the health-check status (ST_F_CHECK_STATUS) or the entity status (ST_F_STATUS). The dump may be stopped in the middle of the labels processing if the output buffer is full. This means the next time, we must take care to restart on the right label value. For now, this part is buggy and we always restart to dump all the label values again from the beginning. To be sure to restart at the right position, the field <ctx.stats.st_code> in the applet context is used to save the last position. Of course, we take care to reset this value when necessary. This fix is specific for 2.4. No backport needed.
2021-02-01 14:05:21 +00:00
labels[1].value = promex_front_st[appctx->ctx.stats.st_code];
val = mkf_u32(FO_STATUS, state == appctx->ctx.stats.st_code);
MAJOR: contrib/prometheus-exporter: move ftd/bkd/srv states to labels this patch is a breaking change between v2.3 and v2.4: we move from using gauge value for frontend/backend/servers states to labels values. the main motivation being I realised it is very difficult to make use of it without hard coded quirks on prometheus client side; especially because the main use is often to group by state, which is harder when the state is the value of the metric. in order to achieve that we iterate on the status metric to generate labels, and so as many metrics. this is the first step to resolve github issue #1029 A second step should address health check states. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-27 21:40:17 +00:00
if (!promex_dump_metric(appctx, htx, prefix, &promex_st_metrics[appctx->st2],
MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones Instead of using static labels for metrics, we now use an array of labels, filled for each metrics if necessary and passed to the dump function. This way, it is easier to extend the promex service. For now, there are at most 8 labels per metrics. This limit may be raised by changing PROMEX_MAX_LABELS value. And to ease labels addition, a label is defined as a key/value pair. The formatting is handled by the dump function. For the proxies and servers, the first entry of the array is always the proxy name. In addition, for the servers, the second entry is always the server name.
2021-01-28 10:24:17 +00:00
&val, labels, &out, max))
MAJOR: contrib/prometheus-exporter: move ftd/bkd/srv states to labels this patch is a breaking change between v2.3 and v2.4: we move from using gauge value for frontend/backend/servers states to labels values. the main motivation being I realised it is very difficult to make use of it without hard coded quirks on prometheus client side; especially because the main use is often to group by state, which is harder when the state is the value of the metric. in order to achieve that we iterate on the status metric to generate labels, and so as many metrics. this is the first step to resolve github issue #1029 A second step should address health check states. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-27 21:40:17 +00:00
goto full;
}
BUG/MINOR: contrib/prometheus-exporter: Restart labels dump at the right pos For some metrics, several lines are produced per entity, one per label value. For instance, the health-check status (ST_F_CHECK_STATUS) or the entity status (ST_F_STATUS). The dump may be stopped in the middle of the labels processing if the output buffer is full. This means the next time, we must take care to restart on the right label value. For now, this part is buggy and we always restart to dump all the label values again from the beginning. To be sure to restart at the right position, the field <ctx.stats.st_code> in the applet context is used to save the last position. Of course, we take care to reset this value when necessary. This fix is specific for 2.4. No backport needed.
2021-02-01 14:05:21 +00:00
appctx->ctx.stats.st_code = 0;
MAJOR: contrib/prometheus-exporter: move ftd/bkd/srv states to labels this patch is a breaking change between v2.3 and v2.4: we move from using gauge value for frontend/backend/servers states to labels values. the main motivation being I realised it is very difficult to make use of it without hard coded quirks on prometheus client side; especially because the main use is often to group by state, which is harder when the state is the value of the metric. in order to achieve that we iterate on the status metric to generate labels, and so as many metrics. this is the first step to resolve github issue #1029 A second step should address health check states. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-27 21:40:17 +00:00
goto next_px;
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
case ST_F_REQ_RATE_MAX:
case ST_F_REQ_TOT:
case ST_F_INTERCEPTED:
case ST_F_CACHE_LOOKUPS:
case ST_F_CACHE_HITS:
case ST_F_COMP_IN:
case ST_F_COMP_OUT:
case ST_F_COMP_BYP:
MINOR: contrib/prometheus-exporter: use fill_fe_stats for frontend dump use `stats_fill_fe_stats` when possible to avoid duplicating code; make use of field selector to get the needed field only. this should not introduce any difference of output. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-17 17:27:46 +00:00
case ST_F_COMP_RSP:
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
if (px->mode != PR_MODE_HTTP)
goto next_px;
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
val = stats[appctx->st2];
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
break;
BUG/MINOR: contrib/prometheus-exporter: Add missing label for ST_F_HRSP_1XX Since the labels are dynamically created for each metric, the "code" label of the ST_F_HRSP_1XX field is missing. To fix the bug, this metric is handled in the same way the other ST_F_HRSP_* field are. We only take care to dump the metric header only once. This bug was introduced by the commit 5a2f93873 ("MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones"). No backport needed.
2021-02-01 13:55:37 +00:00
case ST_F_HRSP_1XX:
MINOR: contrib/prometheus-exporter: use fill_fe_stats for frontend dump use `stats_fill_fe_stats` when possible to avoid duplicating code; make use of field selector to get the needed field only. this should not introduce any difference of output. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-17 17:27:46 +00:00
case ST_F_HRSP_2XX:
case ST_F_HRSP_3XX:
case ST_F_HRSP_4XX:
case ST_F_HRSP_5XX:
case ST_F_HRSP_OTHER:
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
if (px->mode != PR_MODE_HTTP)
goto next_px;
BUG/MINOR: contrib/prometheus-exporter: Add missing label for ST_F_HRSP_1XX Since the labels are dynamically created for each metric, the "code" label of the ST_F_HRSP_1XX field is missing. To fix the bug, this metric is handled in the same way the other ST_F_HRSP_* field are. We only take care to dump the metric header only once. This bug was introduced by the commit 5a2f93873 ("MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones"). No backport needed.
2021-02-01 13:55:37 +00:00
if (appctx->st2 != ST_F_HRSP_1XX)
appctx->ctx.stats.flags &= ~PROMEX_FL_METRIC_HDR;
MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones Instead of using static labels for metrics, we now use an array of labels, filled for each metrics if necessary and passed to the dump function. This way, it is easier to extend the promex service. For now, there are at most 8 labels per metrics. This limit may be raised by changing PROMEX_MAX_LABELS value. And to ease labels addition, a label is defined as a key/value pair. The formatting is handled by the dump function. For the proxies and servers, the first entry of the array is always the proxy name. In addition, for the servers, the second entry is always the server name.
2021-01-28 10:24:17 +00:00
labels[1].name = ist("code");
labels[1].value = promex_hrsp_code[appctx->st2 - ST_F_HRSP_1XX];
val = stats[appctx->st2];
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
break;
default:
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
val = stats[appctx->st2];
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
}
MAJOR: contrib/prometheus-exporter: move ftd/bkd/srv states to labels this patch is a breaking change between v2.3 and v2.4: we move from using gauge value for frontend/backend/servers states to labels values. the main motivation being I realised it is very difficult to make use of it without hard coded quirks on prometheus client side; especially because the main use is often to group by state, which is harder when the state is the value of the metric. in order to achieve that we iterate on the status metric to generate labels, and so as many metrics. this is the first step to resolve github issue #1029 A second step should address health check states. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-27 21:40:17 +00:00
if (!promex_dump_metric(appctx, htx, prefix, &promex_st_metrics[appctx->st2],
MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones Instead of using static labels for metrics, we now use an array of labels, filled for each metrics if necessary and passed to the dump function. This way, it is easier to extend the promex service. For now, there are at most 8 labels per metrics. This limit may be raised by changing PROMEX_MAX_LABELS value. And to ease labels addition, a label is defined as a key/value pair. The formatting is handled by the dump function. For the proxies and servers, the first entry of the array is always the proxy name. In addition, for the servers, the second entry is always the server name.
2021-01-28 10:24:17 +00:00
&val, labels, &out, max))
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
goto full;
next_px:
MINOR: stats: hide px/sv/li fields in applet struct Use an opaque pointer to store proxy instance. Regroup server/listener as a single opaque pointer. This has the benefit to render the structure more evolutive to support statistics on other types of objects in the future. This patch is needed to extend stat support for components other than proxies objects. The prometheus module has been adapted for these changes.
2020-10-02 16:32:02 +00:00
appctx->ctx.stats.obj1 = px->next;
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
}
appctx->ctx.stats.flags |= PROMEX_FL_METRIC_HDR;
MINOR: stats: hide px/sv/li fields in applet struct Use an opaque pointer to store proxy instance. Regroup server/listener as a single opaque pointer. This has the benefit to render the structure more evolutive to support statistics on other types of objects in the future. This patch is needed to extend stat support for components other than proxies objects. The prometheus module has been adapted for these changes.
2020-10-02 16:32:02 +00:00
appctx->ctx.stats.obj1 = proxies_list;
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
}
end:
BUG/MINOR: contrib/prometheus-exporter: Don't try to add empty data blocks When the response buffer is full and nothing more can be inserted, it is important to not try to insert an empty data block. Otherwise, when the function channel_add_input() is called, the flag CF_READ_PARTIAL is set on the response channel while nothing was read and the stream is uselessly woken up. Finally, we have loop while the response buffer is full. This patch must be backported to 2.0.
2019-07-04 08:03:28 +00:00
if (out.len) {
if (!htx_add_data_atonce(htx, out))
return -1; /* Unexpected and unrecoverable error */
channel_add_input(chn, out.len);
}
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
return ret;
full:
ret = 0;
goto end;
}
MEDIUM: contrib/prometheus-exporter: add listen stats this was a missing piece for a while now even though it was planned. This patch adds listen stats. Nothing in particular but we make use of the status helper previously added. `promex_st_metrics` diff also looks scary, but I had to realign all lines. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-14 22:22:56 +00:00
/* Dump listener metrics (prefixed by "haproxy_listen_"). It returns 1 on
* success, 0 if <htx> is full and -1 in case of any error. */
static int promex_dump_listener_metrics(struct appctx *appctx, struct htx *htx)
{
static struct ist prefix = IST("haproxy_listener_");
struct proxy *px;
struct field val;
struct channel *chn = si_ic(appctx->owner);
struct ist out = ist2(trash.area, 0);
size_t max = htx_get_max_blksz(htx, channel_htx_recv_max(chn, htx));
struct field *stats = stat_l[STATS_DOMAIN_PROXY];
struct listener *li;
int ret = 1;
enum li_status status;
for (;appctx->st2 < ST_F_TOTAL_FIELDS; appctx->st2++) {
if (!(promex_st_metrics[appctx->st2].flags & appctx->ctx.stats.flags))
continue;
while (appctx->ctx.stats.obj1) {
struct promex_label labels[PROMEX_MAX_LABELS-1] = {};
px = appctx->ctx.stats.obj1;
labels[0].name = ist("proxy");
labels[0].value = ist2(px->id, strlen(px->id));
/* skip the disabled proxies, global frontend and non-networked ones */
MINOR: proxy: Introduce proxy flags to replace disabled bitfield This change is required to support TCP/HTTP rules in defaults sections. The 'disabled' bitfield in the proxy structure, used to know if a proxy is disabled or stopped, is replaced a generic bitfield named 'flags'. PR_DISABLED and PR_STOPPED flags are renamed to PR_FL_DISABLED and PR_FL_STOPPED respectively. In addition, everywhere there is a test to know if a proxy is disabled or stopped, there is now a bitwise AND operation on PR_FL_DISABLED and/or PR_FL_STOPPED flags.
2021-10-06 12:24:19 +00:00
if ((px->flags & PR_FL_DISABLED) || px->uuid <= 0 || !(px->cap & PR_CAP_FE))
MEDIUM: contrib/prometheus-exporter: add listen stats this was a missing piece for a while now even though it was planned. This patch adds listen stats. Nothing in particular but we make use of the status helper previously added. `promex_st_metrics` diff also looks scary, but I had to realign all lines. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-14 22:22:56 +00:00
goto next_px;
li = appctx->ctx.stats.obj2;
list_for_each_entry_from(li, &px->conf.listeners, by_fe) {
if (!li->counters)
continue;
BUG/MEDIUM: contrib/prometheus-exporter: fix segfault in listener name dump We need to check whether listener is empty before doing anything; in that case, we were trying to dump listerner name while name is null. So simply move the counters check above, which validate all possible cases when the listener is empty. This is very similar to what is done in stats.c see also the trace: Thread 1 "haproxy" received signal SIGSEGV, Segmentation fault. __strlen_sse2 () at ../sysdeps/x86_64/multiarch/../strlen.S:120 120     ../sysdeps/x86_64/multiarch/../strlen.S: No such file or directory. (gdb) bt #0  __strlen_sse2 () at ../sysdeps/x86_64/multiarch/../strlen.S:120 #1  0x00005555555b716b in promex_dump_listener_metrics (htx=0x5555558fadf0, appctx=0x555555926070) at contrib/prometheus-exporter/service-prometheus.c:722 #2  promex_dump_metrics (htx=0x5555558fadf0, si=0x555555925920, appctx=0x555555926070) at contrib/prometheus-exporter/service-prometheus.c:1200 #3  promex_appctx_handle_io (appctx=0x555555926070) at contrib/prometheus-exporter/service-prometheus.c:1477 #4  0x00005555556f0c94 in task_run_applet (t=0x555555926180, context=0x555555926070, state=<optimized out>) at src/applet.c:88 #5  0x00005555556bc6d8 in run_tasks_from_lists (budgets=budgets@entry=0x7fffffffe374) at src/task.c:548 #6  0x00005555556bd1a0 in process_runnable_tasks () at src/task.c:750 #7  0x0000555555696cdd in run_poll_loop () at src/haproxy.c:2870 #8  0x0000555555697025 in run_thread_poll_loop (data=data@entry=0x0) at src/haproxy.c:3035 #9  0x0000555555596c90 in main (argc=<optimized out>, argv=0x7fffffffe818) at src/haproxy.c:3723 quit) this bug was introduced by commit e3f7bd5ae9e969cbfe87e4130d06bff7a3e814c6 ("MEDIUM: contrib/prometheus-exporter: add listen stats"), which is present for 2.4 only, so no backport needed. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-24 23:53:13 +00:00
labels[1].name = ist("listener");
labels[1].value = ist2(li->name, strlen(li->name));
MEDIUM: contrib/prometheus-exporter: add listen stats this was a missing piece for a while now even though it was planned. This patch adds listen stats. Nothing in particular but we make use of the status helper previously added. `promex_st_metrics` diff also looks scary, but I had to realign all lines. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-14 22:22:56 +00:00
if (!stats_fill_li_stats(px, li, 0, stats,
ST_F_TOTAL_FIELDS, &(appctx->st2)))
return -1;
switch (appctx->st2) {
case ST_F_STATUS:
status = get_li_status(li);
for (; appctx->ctx.stats.st_code < LI_STATE_COUNT; appctx->ctx.stats.st_code++) {
val = mkf_u32(FO_STATUS, status == appctx->ctx.stats.st_code);
labels[2].name = ist("state");
labels[2].value = ist(li_status_st[appctx->ctx.stats.st_code]);
if (!promex_dump_metric(appctx, htx, prefix, &promex_st_metrics[appctx->st2],
&val, labels, &out, max))
goto full;
}
appctx->ctx.stats.st_code = 0;
continue;
default:
val = stats[appctx->st2];
}
if (!promex_dump_metric(appctx, htx, prefix,
&promex_st_metrics[appctx->st2],
&val, labels, &out, max))
goto full;
}
next_px:
px = px->next;
appctx->ctx.stats.obj1 = px;
appctx->ctx.stats.obj2 = (px ? LIST_NEXT(&px->conf.listeners, struct listener *, by_fe) : NULL);
}
appctx->ctx.stats.flags |= PROMEX_FL_METRIC_HDR;
appctx->ctx.stats.obj1 = proxies_list;
appctx->ctx.stats.obj2 = LIST_NEXT(&proxies_list->conf.listeners, struct listener *, by_fe);
}
end:
if (out.len) {
if (!htx_add_data_atonce(htx, out))
return -1; /* Unexpected and unrecoverable error */
channel_add_input(chn, out.len);
}
return ret;
full:
appctx->ctx.stats.obj2 = li;
ret = 0;
goto end;
}
CLEANUP: assorted typo fixes in the code and comments This is fifth iteration of typo fixes
2020-03-23 17:28:40 +00:00
/* Dump backends metrics (prefixed by "haproxy_backend_"). It returns 1 on success,
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
* 0 if <htx> is full and -1 in case of any error. */
static int promex_dump_back_metrics(struct appctx *appctx, struct htx *htx)
{
static struct ist prefix = IST("haproxy_backend_");
struct proxy *px;
MINOR: promex: backend aggregated server check status - add new metric: `haproxy_backend_agg_server_check_status` it counts the number of servers matching a specific check status this permits to exclude per server check status as the usage is often to rely on the total. Indeed in large setup having thousands of servers per backend the memory impact is not neglible to store the per server metric. - realign promex_str_metrics array quite simple implementation - we could improve it later by adding an internal state to the prometheus exporter, thus to avoid counting at every dump. this patch is an attempt to close github issue #1312. It may bebackported to 2.4 if requested. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-11-07 09:18:47 +00:00
struct server *sv;
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
struct field val;
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
struct channel *chn = si_ic(appctx->owner);
struct ist out = ist2(trash.area, 0);
BUG/MINOR: contrib/prometheus-exporter: Respect the reserve when data are sent The previous commit e6cdfe574 ("BUG/MINOR: contrib/prometheus-exporter: Don't use channel_htx_recv_max()") is buggy. The buffer's reserve must be respected. This patch must be backported to 2.0 and 1.9.
2019-07-03 09:43:17 +00:00
size_t max = htx_get_max_blksz(htx, channel_htx_recv_max(chn, htx));
MINOR: contrib/prometheus-exporter: use fill_be_stats for backend dump use `stats_fill_be_stats` when possible to avoid duplicating code; make use of field selector to get the needed field only. the only difference is on `haproxy_backend_downtime_seconds_total` as stats.c is testing `px->srv`. This behaviour is present since commit 7344f4789321ef8ce2ce17cf73dabd672f7c8c6 ("MINOR: stats: only report backend's down time if it has servers"). The end result is a NaN instead of a zero when no server are present. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-25 16:29:02 +00:00
struct field *stats = stat_l[STATS_DOMAIN_PROXY];
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
int ret = 1;
BUG/MINOR: contrib/prometheus-exporter: Return the time averages in seconds The metrics QTIME, CTIME, RTIME and TTIME are now returned in seconds using a float representation instead of in milliseconds. So these metrics are now consistent with their announced type and respect Prometheus naming conventions. This patch fixes the issue #288. It may be backported to 2.0. If so, the previous patch, introducing the support for float fields in stats is mantatory and should be backported first.
2019-09-24 14:35:19 +00:00
double secs;
MINOR: promex: backend aggregated server check status - add new metric: `haproxy_backend_agg_server_check_status` it counts the number of servers matching a specific check status this permits to exclude per server check status as the usage is often to rely on the total. Indeed in large setup having thousands of servers per backend the memory impact is not neglible to store the per server metric. - realign promex_str_metrics array quite simple implementation - we could improve it later by adding an internal state to the prometheus exporter, thus to avoid counting at every dump. this patch is an attempt to close github issue #1312. It may bebackported to 2.4 if requested. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-11-07 09:18:47 +00:00
enum promex_back_state bkd_state;
enum promex_srv_state srv_state;
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
for (;appctx->st2 < ST_F_TOTAL_FIELDS; appctx->st2++) {
if (!(promex_st_metrics[appctx->st2].flags & appctx->ctx.stats.flags))
continue;
MINOR: stats: hide px/sv/li fields in applet struct Use an opaque pointer to store proxy instance. Regroup server/listener as a single opaque pointer. This has the benefit to render the structure more evolutive to support statistics on other types of objects in the future. This patch is needed to extend stat support for components other than proxies objects. The prometheus module has been adapted for these changes.
2020-10-02 16:32:02 +00:00
while (appctx->ctx.stats.obj1) {
MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones Instead of using static labels for metrics, we now use an array of labels, filled for each metrics if necessary and passed to the dump function. This way, it is easier to extend the promex service. For now, there are at most 8 labels per metrics. This limit may be raised by changing PROMEX_MAX_LABELS value. And to ease labels addition, a label is defined as a key/value pair. The formatting is handled by the dump function. For the proxies and servers, the first entry of the array is always the proxy name. In addition, for the servers, the second entry is always the server name.
2021-01-28 10:24:17 +00:00
struct promex_label labels[PROMEX_MAX_LABELS-1] = {};
MINOR: promex: backend aggregated server check status - add new metric: `haproxy_backend_agg_server_check_status` it counts the number of servers matching a specific check status this permits to exclude per server check status as the usage is often to rely on the total. Indeed in large setup having thousands of servers per backend the memory impact is not neglible to store the per server metric. - realign promex_str_metrics array quite simple implementation - we could improve it later by adding an internal state to the prometheus exporter, thus to avoid counting at every dump. this patch is an attempt to close github issue #1312. It may bebackported to 2.4 if requested. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-11-07 09:18:47 +00:00
unsigned int srv_state_count[PROMEX_SRV_STATE_COUNT] = { 0 };
MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones Instead of using static labels for metrics, we now use an array of labels, filled for each metrics if necessary and passed to the dump function. This way, it is easier to extend the promex service. For now, there are at most 8 labels per metrics. This limit may be raised by changing PROMEX_MAX_LABELS value. And to ease labels addition, a label is defined as a key/value pair. The formatting is handled by the dump function. For the proxies and servers, the first entry of the array is always the proxy name. In addition, for the servers, the second entry is always the server name.
2021-01-28 10:24:17 +00:00
MINOR: stats: hide px/sv/li fields in applet struct Use an opaque pointer to store proxy instance. Regroup server/listener as a single opaque pointer. This has the benefit to render the structure more evolutive to support statistics on other types of objects in the future. This patch is needed to extend stat support for components other than proxies objects. The prometheus module has been adapted for these changes.
2020-10-02 16:32:02 +00:00
px = appctx->ctx.stats.obj1;
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones Instead of using static labels for metrics, we now use an array of labels, filled for each metrics if necessary and passed to the dump function. This way, it is easier to extend the promex service. For now, there are at most 8 labels per metrics. This limit may be raised by changing PROMEX_MAX_LABELS value. And to ease labels addition, a label is defined as a key/value pair. The formatting is handled by the dump function. For the proxies and servers, the first entry of the array is always the proxy name. In addition, for the servers, the second entry is always the server name.
2021-01-28 10:24:17 +00:00
labels[0].name = ist("proxy");
labels[0].value = ist2(px->id, strlen(px->id));
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
/* skip the disabled proxies, global frontend and non-networked ones */
MINOR: proxy: Introduce proxy flags to replace disabled bitfield This change is required to support TCP/HTTP rules in defaults sections. The 'disabled' bitfield in the proxy structure, used to know if a proxy is disabled or stopped, is replaced a generic bitfield named 'flags'. PR_DISABLED and PR_STOPPED flags are renamed to PR_FL_DISABLED and PR_FL_STOPPED respectively. In addition, everywhere there is a test to know if a proxy is disabled or stopped, there is now a bitwise AND operation on PR_FL_DISABLED and/or PR_FL_STOPPED flags.
2021-10-06 12:24:19 +00:00
if ((px->flags & PR_FL_DISABLED) || px->uuid <= 0 || !(px->cap & PR_CAP_BE))
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
goto next_px;
MINOR: contrib/prometheus-exporter: use fill_be_stats for backend dump use `stats_fill_be_stats` when possible to avoid duplicating code; make use of field selector to get the needed field only. the only difference is on `haproxy_backend_downtime_seconds_total` as stats.c is testing `px->srv`. This behaviour is present since commit 7344f4789321ef8ce2ce17cf73dabd672f7c8c6 ("MINOR: stats: only report backend's down time if it has servers"). The end result is a NaN instead of a zero when no server are present. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-25 16:29:02 +00:00
if (!stats_fill_be_stats(px, 0, stats, ST_F_TOTAL_FIELDS, &(appctx->st2)))
return -1;
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
switch (appctx->st2) {
MINOR: promex: backend aggregated server check status - add new metric: `haproxy_backend_agg_server_check_status` it counts the number of servers matching a specific check status this permits to exclude per server check status as the usage is often to rely on the total. Indeed in large setup having thousands of servers per backend the memory impact is not neglible to store the per server metric. - realign promex_str_metrics array quite simple implementation - we could improve it later by adding an internal state to the prometheus exporter, thus to avoid counting at every dump. this patch is an attempt to close github issue #1312. It may bebackported to 2.4 if requested. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-11-07 09:18:47 +00:00
case ST_F_AGG_SRV_CHECK_STATUS:
if (!px->srv)
goto next_px;
sv = px->srv;
while (sv) {
srv_state = promex_srv_status(sv);
srv_state_count[srv_state] += 1;
sv = sv->next;
}
for (; appctx->ctx.stats.st_code < PROMEX_SRV_STATE_COUNT; appctx->ctx.stats.st_code++) {
val = mkf_u32(FN_GAUGE, srv_state_count[appctx->ctx.stats.st_code]);
labels[1].name = ist("state");
labels[1].value = promex_srv_st[appctx->ctx.stats.st_code];
if (!promex_dump_metric(appctx, htx, prefix, &promex_st_metrics[appctx->st2],
&val, labels, &out, max))
goto full;
}
appctx->ctx.stats.st_code = 0;
goto next_px;
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
case ST_F_STATUS:
MINOR: promex: backend aggregated server check status - add new metric: `haproxy_backend_agg_server_check_status` it counts the number of servers matching a specific check status this permits to exclude per server check status as the usage is often to rely on the total. Indeed in large setup having thousands of servers per backend the memory impact is not neglible to store the per server metric. - realign promex_str_metrics array quite simple implementation - we could improve it later by adding an internal state to the prometheus exporter, thus to avoid counting at every dump. this patch is an attempt to close github issue #1312. It may bebackported to 2.4 if requested. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-11-07 09:18:47 +00:00
bkd_state = ((px->lbprm.tot_weight > 0 || !px->srv) ? 1 : 0);
BUG/MINOR: contrib/prometheus-exporter: Restart labels dump at the right pos For some metrics, several lines are produced per entity, one per label value. For instance, the health-check status (ST_F_CHECK_STATUS) or the entity status (ST_F_STATUS). The dump may be stopped in the middle of the labels processing if the output buffer is full. This means the next time, we must take care to restart on the right label value. For now, this part is buggy and we always restart to dump all the label values again from the beginning. To be sure to restart at the right position, the field <ctx.stats.st_code> in the applet context is used to save the last position. Of course, we take care to reset this value when necessary. This fix is specific for 2.4. No backport needed.
2021-02-01 14:05:21 +00:00
for (; appctx->ctx.stats.st_code < PROMEX_BACK_STATE_COUNT; appctx->ctx.stats.st_code++) {
MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones Instead of using static labels for metrics, we now use an array of labels, filled for each metrics if necessary and passed to the dump function. This way, it is easier to extend the promex service. For now, there are at most 8 labels per metrics. This limit may be raised by changing PROMEX_MAX_LABELS value. And to ease labels addition, a label is defined as a key/value pair. The formatting is handled by the dump function. For the proxies and servers, the first entry of the array is always the proxy name. In addition, for the servers, the second entry is always the server name.
2021-01-28 10:24:17 +00:00
labels[1].name = ist("state");
BUG/MINOR: contrib/prometheus-exporter: Restart labels dump at the right pos For some metrics, several lines are produced per entity, one per label value. For instance, the health-check status (ST_F_CHECK_STATUS) or the entity status (ST_F_STATUS). The dump may be stopped in the middle of the labels processing if the output buffer is full. This means the next time, we must take care to restart on the right label value. For now, this part is buggy and we always restart to dump all the label values again from the beginning. To be sure to restart at the right position, the field <ctx.stats.st_code> in the applet context is used to save the last position. Of course, we take care to reset this value when necessary. This fix is specific for 2.4. No backport needed.
2021-02-01 14:05:21 +00:00
labels[1].value = promex_back_st[appctx->ctx.stats.st_code];
MINOR: promex: backend aggregated server check status - add new metric: `haproxy_backend_agg_server_check_status` it counts the number of servers matching a specific check status this permits to exclude per server check status as the usage is often to rely on the total. Indeed in large setup having thousands of servers per backend the memory impact is not neglible to store the per server metric. - realign promex_str_metrics array quite simple implementation - we could improve it later by adding an internal state to the prometheus exporter, thus to avoid counting at every dump. this patch is an attempt to close github issue #1312. It may bebackported to 2.4 if requested. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-11-07 09:18:47 +00:00
val = mkf_u32(FO_STATUS, bkd_state == appctx->ctx.stats.st_code);
MAJOR: contrib/prometheus-exporter: move ftd/bkd/srv states to labels this patch is a breaking change between v2.3 and v2.4: we move from using gauge value for frontend/backend/servers states to labels values. the main motivation being I realised it is very difficult to make use of it without hard coded quirks on prometheus client side; especially because the main use is often to group by state, which is harder when the state is the value of the metric. in order to achieve that we iterate on the status metric to generate labels, and so as many metrics. this is the first step to resolve github issue #1029 A second step should address health check states. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-27 21:40:17 +00:00
if (!promex_dump_metric(appctx, htx, prefix, &promex_st_metrics[appctx->st2],
MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones Instead of using static labels for metrics, we now use an array of labels, filled for each metrics if necessary and passed to the dump function. This way, it is easier to extend the promex service. For now, there are at most 8 labels per metrics. This limit may be raised by changing PROMEX_MAX_LABELS value. And to ease labels addition, a label is defined as a key/value pair. The formatting is handled by the dump function. For the proxies and servers, the first entry of the array is always the proxy name. In addition, for the servers, the second entry is always the server name.
2021-01-28 10:24:17 +00:00
&val, labels, &out, max))
MAJOR: contrib/prometheus-exporter: move ftd/bkd/srv states to labels this patch is a breaking change between v2.3 and v2.4: we move from using gauge value for frontend/backend/servers states to labels values. the main motivation being I realised it is very difficult to make use of it without hard coded quirks on prometheus client side; especially because the main use is often to group by state, which is harder when the state is the value of the metric. in order to achieve that we iterate on the status metric to generate labels, and so as many metrics. this is the first step to resolve github issue #1029 A second step should address health check states. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-27 21:40:17 +00:00
goto full;
}
BUG/MINOR: contrib/prometheus-exporter: Restart labels dump at the right pos For some metrics, several lines are produced per entity, one per label value. For instance, the health-check status (ST_F_CHECK_STATUS) or the entity status (ST_F_STATUS). The dump may be stopped in the middle of the labels processing if the output buffer is full. This means the next time, we must take care to restart on the right label value. For now, this part is buggy and we always restart to dump all the label values again from the beginning. To be sure to restart at the right position, the field <ctx.stats.st_code> in the applet context is used to save the last position. Of course, we take care to reset this value when necessary. This fix is specific for 2.4. No backport needed.
2021-02-01 14:05:21 +00:00
appctx->ctx.stats.st_code = 0;
MAJOR: contrib/prometheus-exporter: move ftd/bkd/srv states to labels this patch is a breaking change between v2.3 and v2.4: we move from using gauge value for frontend/backend/servers states to labels values. the main motivation being I realised it is very difficult to make use of it without hard coded quirks on prometheus client side; especially because the main use is often to group by state, which is harder when the state is the value of the metric. in order to achieve that we iterate on the status metric to generate labels, and so as many metrics. this is the first step to resolve github issue #1029 A second step should address health check states. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-27 21:40:17 +00:00
goto next_px;
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
case ST_F_QTIME:
BUG/MINOR: contrib/prometheus-exporter: Return the time averages in seconds The metrics QTIME, CTIME, RTIME and TTIME are now returned in seconds using a float representation instead of in milliseconds. So these metrics are now consistent with their announced type and respect Prometheus naming conventions. This patch fixes the issue #288. It may be backported to 2.0. If so, the previous patch, introducing the support for float fields in stats is mantatory and should be backported first.
2019-09-24 14:35:19 +00:00
secs = (double)swrate_avg(px->be_counters.q_time, TIME_STATS_SAMPLES) / 1000.0;
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
val = mkf_flt(FN_AVG, secs);
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
break;
case ST_F_CTIME:
BUG/MINOR: contrib/prometheus-exporter: Return the time averages in seconds The metrics QTIME, CTIME, RTIME and TTIME are now returned in seconds using a float representation instead of in milliseconds. So these metrics are now consistent with their announced type and respect Prometheus naming conventions. This patch fixes the issue #288. It may be backported to 2.0. If so, the previous patch, introducing the support for float fields in stats is mantatory and should be backported first.
2019-09-24 14:35:19 +00:00
secs = (double)swrate_avg(px->be_counters.c_time, TIME_STATS_SAMPLES) / 1000.0;
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
val = mkf_flt(FN_AVG, secs);
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
break;
case ST_F_RTIME:
BUG/MINOR: contrib/prometheus-exporter: Return the time averages in seconds The metrics QTIME, CTIME, RTIME and TTIME are now returned in seconds using a float representation instead of in milliseconds. So these metrics are now consistent with their announced type and respect Prometheus naming conventions. This patch fixes the issue #288. It may be backported to 2.0. If so, the previous patch, introducing the support for float fields in stats is mantatory and should be backported first.
2019-09-24 14:35:19 +00:00
secs = (double)swrate_avg(px->be_counters.d_time, TIME_STATS_SAMPLES) / 1000.0;
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
val = mkf_flt(FN_AVG, secs);
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
break;
case ST_F_TTIME:
BUG/MINOR: contrib/prometheus-exporter: Return the time averages in seconds The metrics QTIME, CTIME, RTIME and TTIME are now returned in seconds using a float representation instead of in milliseconds. So these metrics are now consistent with their announced type and respect Prometheus naming conventions. This patch fixes the issue #288. It may be backported to 2.0. If so, the previous patch, introducing the support for float fields in stats is mantatory and should be backported first.
2019-09-24 14:35:19 +00:00
secs = (double)swrate_avg(px->be_counters.t_time, TIME_STATS_SAMPLES) / 1000.0;
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
val = mkf_flt(FN_AVG, secs);
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
break;
MINOR: contrib/prometheus-exporter: Report metrics about max times for sessions Now, for the sessions, the maximum times (queue, connect, response, total) are reported in addition of the averages over the last 1024 connections. These metrics are reported per backend and per server. Here are the metrics name : * haproxy_backend_max_queue_time_seconds * haproxy_backend_max_connect_time_seconds * haproxy_backend_max_response_time_seconds * haproxy_backend_max_total_time_seconds and * haproxy_server_max_queue_time_seconds * haproxy_server_max_connect_time_seconds * haproxy_server_max_response_time_seconds * haproxy_server_max_total_time_seconds This patch is related to #272.
2019-11-08 14:05:31 +00:00
case ST_F_QT_MAX:
secs = (double)px->be_counters.qtime_max / 1000.0;
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
val = mkf_flt(FN_MAX, secs);
MINOR: contrib/prometheus-exporter: Report metrics about max times for sessions Now, for the sessions, the maximum times (queue, connect, response, total) are reported in addition of the averages over the last 1024 connections. These metrics are reported per backend and per server. Here are the metrics name : * haproxy_backend_max_queue_time_seconds * haproxy_backend_max_connect_time_seconds * haproxy_backend_max_response_time_seconds * haproxy_backend_max_total_time_seconds and * haproxy_server_max_queue_time_seconds * haproxy_server_max_connect_time_seconds * haproxy_server_max_response_time_seconds * haproxy_server_max_total_time_seconds This patch is related to #272.
2019-11-08 14:05:31 +00:00
break;
case ST_F_CT_MAX:
secs = (double)px->be_counters.ctime_max / 1000.0;
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
val = mkf_flt(FN_MAX, secs);
MINOR: contrib/prometheus-exporter: Report metrics about max times for sessions Now, for the sessions, the maximum times (queue, connect, response, total) are reported in addition of the averages over the last 1024 connections. These metrics are reported per backend and per server. Here are the metrics name : * haproxy_backend_max_queue_time_seconds * haproxy_backend_max_connect_time_seconds * haproxy_backend_max_response_time_seconds * haproxy_backend_max_total_time_seconds and * haproxy_server_max_queue_time_seconds * haproxy_server_max_connect_time_seconds * haproxy_server_max_response_time_seconds * haproxy_server_max_total_time_seconds This patch is related to #272.
2019-11-08 14:05:31 +00:00
break;
case ST_F_RT_MAX:
secs = (double)px->be_counters.dtime_max / 1000.0;
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
val = mkf_flt(FN_MAX, secs);
MINOR: contrib/prometheus-exporter: Report metrics about max times for sessions Now, for the sessions, the maximum times (queue, connect, response, total) are reported in addition of the averages over the last 1024 connections. These metrics are reported per backend and per server. Here are the metrics name : * haproxy_backend_max_queue_time_seconds * haproxy_backend_max_connect_time_seconds * haproxy_backend_max_response_time_seconds * haproxy_backend_max_total_time_seconds and * haproxy_server_max_queue_time_seconds * haproxy_server_max_connect_time_seconds * haproxy_server_max_response_time_seconds * haproxy_server_max_total_time_seconds This patch is related to #272.
2019-11-08 14:05:31 +00:00
break;
case ST_F_TT_MAX:
secs = (double)px->be_counters.ttime_max / 1000.0;
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
val = mkf_flt(FN_MAX, secs);
MINOR: contrib/prometheus-exporter: Report metrics about max times for sessions Now, for the sessions, the maximum times (queue, connect, response, total) are reported in addition of the averages over the last 1024 connections. These metrics are reported per backend and per server. Here are the metrics name : * haproxy_backend_max_queue_time_seconds * haproxy_backend_max_connect_time_seconds * haproxy_backend_max_response_time_seconds * haproxy_backend_max_total_time_seconds and * haproxy_server_max_queue_time_seconds * haproxy_server_max_connect_time_seconds * haproxy_server_max_response_time_seconds * haproxy_server_max_total_time_seconds This patch is related to #272.
2019-11-08 14:05:31 +00:00
break;
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
case ST_F_REQ_TOT:
MINOR: contrib/prometheus-exporter: use fill_be_stats for backend dump use `stats_fill_be_stats` when possible to avoid duplicating code; make use of field selector to get the needed field only. the only difference is on `haproxy_backend_downtime_seconds_total` as stats.c is testing `px->srv`. This behaviour is present since commit 7344f4789321ef8ce2ce17cf73dabd672f7c8c6 ("MINOR: stats: only report backend's down time if it has servers"). The end result is a NaN instead of a zero when no server are present. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-25 16:29:02 +00:00
case ST_F_CACHE_LOOKUPS:
case ST_F_CACHE_HITS:
case ST_F_COMP_IN:
case ST_F_COMP_OUT:
case ST_F_COMP_BYP:
case ST_F_COMP_RSP:
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
if (px->mode != PR_MODE_HTTP)
goto next_px;
MINOR: contrib/prometheus-exporter: use fill_be_stats for backend dump use `stats_fill_be_stats` when possible to avoid duplicating code; make use of field selector to get the needed field only. the only difference is on `haproxy_backend_downtime_seconds_total` as stats.c is testing `px->srv`. This behaviour is present since commit 7344f4789321ef8ce2ce17cf73dabd672f7c8c6 ("MINOR: stats: only report backend's down time if it has servers"). The end result is a NaN instead of a zero when no server are present. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-25 16:29:02 +00:00
val = stats[appctx->st2];
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
break;
BUG/MINOR: contrib/prometheus-exporter: Add missing label for ST_F_HRSP_1XX Since the labels are dynamically created for each metric, the "code" label of the ST_F_HRSP_1XX field is missing. To fix the bug, this metric is handled in the same way the other ST_F_HRSP_* field are. We only take care to dump the metric header only once. This bug was introduced by the commit 5a2f93873 ("MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones"). No backport needed.
2021-02-01 13:55:37 +00:00
case ST_F_HRSP_1XX:
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
case ST_F_HRSP_2XX:
case ST_F_HRSP_3XX:
case ST_F_HRSP_4XX:
case ST_F_HRSP_5XX:
case ST_F_HRSP_OTHER:
if (px->mode != PR_MODE_HTTP)
goto next_px;
BUG/MINOR: contrib/prometheus-exporter: Add missing label for ST_F_HRSP_1XX Since the labels are dynamically created for each metric, the "code" label of the ST_F_HRSP_1XX field is missing. To fix the bug, this metric is handled in the same way the other ST_F_HRSP_* field are. We only take care to dump the metric header only once. This bug was introduced by the commit 5a2f93873 ("MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones"). No backport needed.
2021-02-01 13:55:37 +00:00
if (appctx->st2 != ST_F_HRSP_1XX)
appctx->ctx.stats.flags &= ~PROMEX_FL_METRIC_HDR;
MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones Instead of using static labels for metrics, we now use an array of labels, filled for each metrics if necessary and passed to the dump function. This way, it is easier to extend the promex service. For now, there are at most 8 labels per metrics. This limit may be raised by changing PROMEX_MAX_LABELS value. And to ease labels addition, a label is defined as a key/value pair. The formatting is handled by the dump function. For the proxies and servers, the first entry of the array is always the proxy name. In addition, for the servers, the second entry is always the server name.
2021-01-28 10:24:17 +00:00
labels[1].name = ist("code");
labels[1].value = promex_hrsp_code[appctx->st2 - ST_F_HRSP_1XX];
MINOR: contrib/prometheus-exporter: use fill_be_stats for backend dump use `stats_fill_be_stats` when possible to avoid duplicating code; make use of field selector to get the needed field only. the only difference is on `haproxy_backend_downtime_seconds_total` as stats.c is testing `px->srv`. This behaviour is present since commit 7344f4789321ef8ce2ce17cf73dabd672f7c8c6 ("MINOR: stats: only report backend's down time if it has servers"). The end result is a NaN instead of a zero when no server are present. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-25 16:29:02 +00:00
val = stats[appctx->st2];
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
break;
default:
MINOR: contrib/prometheus-exporter: use fill_be_stats for backend dump use `stats_fill_be_stats` when possible to avoid duplicating code; make use of field selector to get the needed field only. the only difference is on `haproxy_backend_downtime_seconds_total` as stats.c is testing `px->srv`. This behaviour is present since commit 7344f4789321ef8ce2ce17cf73dabd672f7c8c6 ("MINOR: stats: only report backend's down time if it has servers"). The end result is a NaN instead of a zero when no server are present. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-25 16:29:02 +00:00
val = stats[appctx->st2];
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
}
MINOR: contrib/prometheus-exporter: use fill_be_stats for backend dump use `stats_fill_be_stats` when possible to avoid duplicating code; make use of field selector to get the needed field only. the only difference is on `haproxy_backend_downtime_seconds_total` as stats.c is testing `px->srv`. This behaviour is present since commit 7344f4789321ef8ce2ce17cf73dabd672f7c8c6 ("MINOR: stats: only report backend's down time if it has servers"). The end result is a NaN instead of a zero when no server are present. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-25 16:29:02 +00:00
if (!promex_dump_metric(appctx, htx, prefix, &promex_st_metrics[appctx->st2],
MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones Instead of using static labels for metrics, we now use an array of labels, filled for each metrics if necessary and passed to the dump function. This way, it is easier to extend the promex service. For now, there are at most 8 labels per metrics. This limit may be raised by changing PROMEX_MAX_LABELS value. And to ease labels addition, a label is defined as a key/value pair. The formatting is handled by the dump function. For the proxies and servers, the first entry of the array is always the proxy name. In addition, for the servers, the second entry is always the server name.
2021-01-28 10:24:17 +00:00
&val, labels, &out, max))
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
goto full;
next_px:
MINOR: stats: hide px/sv/li fields in applet struct Use an opaque pointer to store proxy instance. Regroup server/listener as a single opaque pointer. This has the benefit to render the structure more evolutive to support statistics on other types of objects in the future. This patch is needed to extend stat support for components other than proxies objects. The prometheus module has been adapted for these changes.
2020-10-02 16:32:02 +00:00
appctx->ctx.stats.obj1 = px->next;
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
}
appctx->ctx.stats.flags |= PROMEX_FL_METRIC_HDR;
MINOR: stats: hide px/sv/li fields in applet struct Use an opaque pointer to store proxy instance. Regroup server/listener as a single opaque pointer. This has the benefit to render the structure more evolutive to support statistics on other types of objects in the future. This patch is needed to extend stat support for components other than proxies objects. The prometheus module has been adapted for these changes.
2020-10-02 16:32:02 +00:00
appctx->ctx.stats.obj1 = proxies_list;
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
}
end:
BUG/MINOR: contrib/prometheus-exporter: Don't try to add empty data blocks When the response buffer is full and nothing more can be inserted, it is important to not try to insert an empty data block. Otherwise, when the function channel_add_input() is called, the flag CF_READ_PARTIAL is set on the response channel while nothing was read and the stream is uselessly woken up. Finally, we have loop while the response buffer is full. This patch must be backported to 2.0.
2019-07-04 08:03:28 +00:00
if (out.len) {
if (!htx_add_data_atonce(htx, out))
return -1; /* Unexpected and unrecoverable error */
channel_add_input(chn, out.len);
}
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
return ret;
full:
ret = 0;
goto end;
}
CLEANUP: assorted typo fixes in the code and comments This is fifth iteration of typo fixes
2020-03-23 17:28:40 +00:00
/* Dump servers metrics (prefixed by "haproxy_server_"). It returns 1 on success,
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
* 0 if <htx> is full and -1 in case of any error. */
static int promex_dump_srv_metrics(struct appctx *appctx, struct htx *htx)
{
static struct ist prefix = IST("haproxy_server_");
struct proxy *px;
struct server *sv;
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
struct field val;
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
struct channel *chn = si_ic(appctx->owner);
struct ist out = ist2(trash.area, 0);
BUG/MINOR: contrib/prometheus-exporter: Respect the reserve when data are sent The previous commit e6cdfe574 ("BUG/MINOR: contrib/prometheus-exporter: Don't use channel_htx_recv_max()") is buggy. The buffer's reserve must be respected. This patch must be backported to 2.0 and 1.9.
2019-07-03 09:43:17 +00:00
size_t max = htx_get_max_blksz(htx, channel_htx_recv_max(chn, htx));
MINOR: contrib/prometheus-exporter: use fill_sv_stats for server dump use `stats_fill_sv_stats` when possible to avoid duplicating code. the following metrics have a change of behaviour: haproxy_server_limit_sessions haproxy_server_queue_limit haproxy_server_check_failures_total haproxy_server_check_up_down_total haproxy_server_downtime_seconds_total haproxy_server_current_throttle haproxy_server_idle_connections_limit depending on cases, if the limit was not configured or enabled, NaN is returned instead. It should not be an issue for users, even better than before as it provides more precise info. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-25 16:29:04 +00:00
struct field *stats = stat_l[STATS_DOMAIN_PROXY];
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
int ret = 1;
BUG/MINOR: contrib/prometheus-exporter: Return the time averages in seconds The metrics QTIME, CTIME, RTIME and TTIME are now returned in seconds using a float representation instead of in milliseconds. So these metrics are now consistent with their announced type and respect Prometheus naming conventions. This patch fixes the issue #288. It may be backported to 2.0. If so, the previous patch, introducing the support for float fields in stats is mantatory and should be backported first.
2019-09-24 14:35:19 +00:00
double secs;
MAJOR: contrib/prometheus-exporter: move ftd/bkd/srv states to labels this patch is a breaking change between v2.3 and v2.4: we move from using gauge value for frontend/backend/servers states to labels values. the main motivation being I realised it is very difficult to make use of it without hard coded quirks on prometheus client side; especially because the main use is often to group by state, which is harder when the state is the value of the metric. in order to achieve that we iterate on the status metric to generate labels, and so as many metrics. this is the first step to resolve github issue #1029 A second step should address health check states. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-27 21:40:17 +00:00
enum promex_srv_state state;
MAJOR: contrib/prometheus-exporter: move health check status to labels this patch is a breaking change between v2.3 and v2.4: we move from using gauge value for health check states to labels values. The diff is quite small thanks to the preparation work from Christopher to allow more flexibility in labels, see commit 5a2f938732126f43bbf4cea5482c01552b0d0314 ("MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones") this is a follow up of commit c6464591a365bfcf509b322bdaa4d608c9395d75 ("MAJOR: contrib/prometheus-exporter: move ftd/bkd/srv states to labels"). The main goal being to be better aligned with prometheus use cases in terms of queries. More specifically to health checks, Pierre C. mentioned the possible quirks he had to put in place in order to make use of those metrics through prometheus: <aggregator_function> by(proxy, check_status) (count_values by(proxy, instance) ("check_status", haproxy_server_check_status)) I am perfectly aware this introduces a lot more metrics but I don't see how we can improve the usability without it. The main issue remains in the cardinality of the states which are > 20. Prometheus recommends to stay below a cardinality of 10 for a given metric but I consider our case very specific, because highly linked to the level of precision haproxy exposes. Even before this patch I saw several large production setup (a few hundreds of MB in output) which are making use of the scope parameter to simply ignore the server metrics, so that the scrapping can be faster, and memory consumed on client side not too high. So I believe we should eventually continue in that direction and offer more granularity of filtering of the output. That being said it is already possible to filter out the data on prometheus client side. this is related to github issue #1029 Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-01 12:11:51 +00:00
const char *check_state;
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
for (;appctx->st2 < ST_F_TOTAL_FIELDS; appctx->st2++) {
if (!(promex_st_metrics[appctx->st2].flags & appctx->ctx.stats.flags))
continue;
MINOR: stats: hide px/sv/li fields in applet struct Use an opaque pointer to store proxy instance. Regroup server/listener as a single opaque pointer. This has the benefit to render the structure more evolutive to support statistics on other types of objects in the future. This patch is needed to extend stat support for components other than proxies objects. The prometheus module has been adapted for these changes.
2020-10-02 16:32:02 +00:00
while (appctx->ctx.stats.obj1) {
MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones Instead of using static labels for metrics, we now use an array of labels, filled for each metrics if necessary and passed to the dump function. This way, it is easier to extend the promex service. For now, there are at most 8 labels per metrics. This limit may be raised by changing PROMEX_MAX_LABELS value. And to ease labels addition, a label is defined as a key/value pair. The formatting is handled by the dump function. For the proxies and servers, the first entry of the array is always the proxy name. In addition, for the servers, the second entry is always the server name.
2021-01-28 10:24:17 +00:00
struct promex_label labels[PROMEX_MAX_LABELS-1] = {};
MINOR: stats: hide px/sv/li fields in applet struct Use an opaque pointer to store proxy instance. Regroup server/listener as a single opaque pointer. This has the benefit to render the structure more evolutive to support statistics on other types of objects in the future. This patch is needed to extend stat support for components other than proxies objects. The prometheus module has been adapted for these changes.
2020-10-02 16:32:02 +00:00
px = appctx->ctx.stats.obj1;
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones Instead of using static labels for metrics, we now use an array of labels, filled for each metrics if necessary and passed to the dump function. This way, it is easier to extend the promex service. For now, there are at most 8 labels per metrics. This limit may be raised by changing PROMEX_MAX_LABELS value. And to ease labels addition, a label is defined as a key/value pair. The formatting is handled by the dump function. For the proxies and servers, the first entry of the array is always the proxy name. In addition, for the servers, the second entry is always the server name.
2021-01-28 10:24:17 +00:00
labels[0].name = ist("proxy");
labels[0].value = ist2(px->id, strlen(px->id));
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
/* skip the disabled proxies, global frontend and non-networked ones */
MINOR: proxy: Introduce proxy flags to replace disabled bitfield This change is required to support TCP/HTTP rules in defaults sections. The 'disabled' bitfield in the proxy structure, used to know if a proxy is disabled or stopped, is replaced a generic bitfield named 'flags'. PR_DISABLED and PR_STOPPED flags are renamed to PR_FL_DISABLED and PR_FL_STOPPED respectively. In addition, everywhere there is a test to know if a proxy is disabled or stopped, there is now a bitwise AND operation on PR_FL_DISABLED and/or PR_FL_STOPPED flags.
2021-10-06 12:24:19 +00:00
if ((px->flags & PR_FL_DISABLED) || px->uuid <= 0 || !(px->cap & PR_CAP_BE))
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
goto next_px;
MINOR: stats: hide px/sv/li fields in applet struct Use an opaque pointer to store proxy instance. Regroup server/listener as a single opaque pointer. This has the benefit to render the structure more evolutive to support statistics on other types of objects in the future. This patch is needed to extend stat support for components other than proxies objects. The prometheus module has been adapted for these changes.
2020-10-02 16:32:02 +00:00
while (appctx->ctx.stats.obj2) {
sv = appctx->ctx.stats.obj2;
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones Instead of using static labels for metrics, we now use an array of labels, filled for each metrics if necessary and passed to the dump function. This way, it is easier to extend the promex service. For now, there are at most 8 labels per metrics. This limit may be raised by changing PROMEX_MAX_LABELS value. And to ease labels addition, a label is defined as a key/value pair. The formatting is handled by the dump function. For the proxies and servers, the first entry of the array is always the proxy name. In addition, for the servers, the second entry is always the server name.
2021-01-28 10:24:17 +00:00
labels[1].name = ist("server");
labels[1].value = ist2(sv->id, strlen(sv->id));
MINOR: contrib/prometheus-exporter: use fill_sv_stats for server dump use `stats_fill_sv_stats` when possible to avoid duplicating code. the following metrics have a change of behaviour: haproxy_server_limit_sessions haproxy_server_queue_limit haproxy_server_check_failures_total haproxy_server_check_up_down_total haproxy_server_downtime_seconds_total haproxy_server_current_throttle haproxy_server_idle_connections_limit depending on cases, if the limit was not configured or enabled, NaN is returned instead. It should not be an issue for users, even better than before as it provides more precise info. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-25 16:29:04 +00:00
if (!stats_fill_sv_stats(px, sv, 0, stats, ST_F_TOTAL_FIELDS, &(appctx->st2)))
return -1;
MINOR: contrib/prometheus-exporter: Add a param to ignore servers in maintenance By passing the parameter "no-maint" in the query-string, it is now possible to ignore servers in maintenance. It means that the metrics for servers in this state will not be exported.
2019-11-19 13:18:24 +00:00
if ((appctx->ctx.stats.flags & PROMEX_FL_NO_MAINT_SRV) && (sv->cur_admin & SRV_ADMF_MAINT))
goto next_sv;
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
switch (appctx->st2) {
case ST_F_STATUS:
MAJOR: contrib/prometheus-exporter: move ftd/bkd/srv states to labels this patch is a breaking change between v2.3 and v2.4: we move from using gauge value for frontend/backend/servers states to labels values. the main motivation being I realised it is very difficult to make use of it without hard coded quirks on prometheus client side; especially because the main use is often to group by state, which is harder when the state is the value of the metric. in order to achieve that we iterate on the status metric to generate labels, and so as many metrics. this is the first step to resolve github issue #1029 A second step should address health check states. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-27 21:40:17 +00:00
state = promex_srv_status(sv);
CLEANUP: contrib/prometheus-exporter: align for with srv status case the for loop was wrongly indented following our recent rework Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-14 21:26:24 +00:00
for (; appctx->ctx.stats.st_code < PROMEX_SRV_STATE_COUNT; appctx->ctx.stats.st_code++) {
BUG/MINOR: contrib/prometheus-exporter: Restart labels dump at the right pos For some metrics, several lines are produced per entity, one per label value. For instance, the health-check status (ST_F_CHECK_STATUS) or the entity status (ST_F_STATUS). The dump may be stopped in the middle of the labels processing if the output buffer is full. This means the next time, we must take care to restart on the right label value. For now, this part is buggy and we always restart to dump all the label values again from the beginning. To be sure to restart at the right position, the field <ctx.stats.st_code> in the applet context is used to save the last position. Of course, we take care to reset this value when necessary. This fix is specific for 2.4. No backport needed.
2021-02-01 14:05:21 +00:00
val = mkf_u32(FO_STATUS, state == appctx->ctx.stats.st_code);
MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones Instead of using static labels for metrics, we now use an array of labels, filled for each metrics if necessary and passed to the dump function. This way, it is easier to extend the promex service. For now, there are at most 8 labels per metrics. This limit may be raised by changing PROMEX_MAX_LABELS value. And to ease labels addition, a label is defined as a key/value pair. The formatting is handled by the dump function. For the proxies and servers, the first entry of the array is always the proxy name. In addition, for the servers, the second entry is always the server name.
2021-01-28 10:24:17 +00:00
labels[2].name = ist("state");
BUG/MINOR: contrib/prometheus-exporter: Restart labels dump at the right pos For some metrics, several lines are produced per entity, one per label value. For instance, the health-check status (ST_F_CHECK_STATUS) or the entity status (ST_F_STATUS). The dump may be stopped in the middle of the labels processing if the output buffer is full. This means the next time, we must take care to restart on the right label value. For now, this part is buggy and we always restart to dump all the label values again from the beginning. To be sure to restart at the right position, the field <ctx.stats.st_code> in the applet context is used to save the last position. Of course, we take care to reset this value when necessary. This fix is specific for 2.4. No backport needed.
2021-02-01 14:05:21 +00:00
labels[2].value = promex_srv_st[appctx->ctx.stats.st_code];
MAJOR: contrib/prometheus-exporter: move ftd/bkd/srv states to labels this patch is a breaking change between v2.3 and v2.4: we move from using gauge value for frontend/backend/servers states to labels values. the main motivation being I realised it is very difficult to make use of it without hard coded quirks on prometheus client side; especially because the main use is often to group by state, which is harder when the state is the value of the metric. in order to achieve that we iterate on the status metric to generate labels, and so as many metrics. this is the first step to resolve github issue #1029 A second step should address health check states. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-27 21:40:17 +00:00
if (!promex_dump_metric(appctx, htx, prefix, &promex_st_metrics[appctx->st2],
MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones Instead of using static labels for metrics, we now use an array of labels, filled for each metrics if necessary and passed to the dump function. This way, it is easier to extend the promex service. For now, there are at most 8 labels per metrics. This limit may be raised by changing PROMEX_MAX_LABELS value. And to ease labels addition, a label is defined as a key/value pair. The formatting is handled by the dump function. For the proxies and servers, the first entry of the array is always the proxy name. In addition, for the servers, the second entry is always the server name.
2021-01-28 10:24:17 +00:00
&val, labels, &out, max))
MAJOR: contrib/prometheus-exporter: move ftd/bkd/srv states to labels this patch is a breaking change between v2.3 and v2.4: we move from using gauge value for frontend/backend/servers states to labels values. the main motivation being I realised it is very difficult to make use of it without hard coded quirks on prometheus client side; especially because the main use is often to group by state, which is harder when the state is the value of the metric. in order to achieve that we iterate on the status metric to generate labels, and so as many metrics. this is the first step to resolve github issue #1029 A second step should address health check states. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-27 21:40:17 +00:00
goto full;
}
BUG/MINOR: contrib/prometheus-exporter: Restart labels dump at the right pos For some metrics, several lines are produced per entity, one per label value. For instance, the health-check status (ST_F_CHECK_STATUS) or the entity status (ST_F_STATUS). The dump may be stopped in the middle of the labels processing if the output buffer is full. This means the next time, we must take care to restart on the right label value. For now, this part is buggy and we always restart to dump all the label values again from the beginning. To be sure to restart at the right position, the field <ctx.stats.st_code> in the applet context is used to save the last position. Of course, we take care to reset this value when necessary. This fix is specific for 2.4. No backport needed.
2021-02-01 14:05:21 +00:00
appctx->ctx.stats.st_code = 0;
MAJOR: contrib/prometheus-exporter: move ftd/bkd/srv states to labels this patch is a breaking change between v2.3 and v2.4: we move from using gauge value for frontend/backend/servers states to labels values. the main motivation being I realised it is very difficult to make use of it without hard coded quirks on prometheus client side; especially because the main use is often to group by state, which is harder when the state is the value of the metric. in order to achieve that we iterate on the status metric to generate labels, and so as many metrics. this is the first step to resolve github issue #1029 A second step should address health check states. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-27 21:40:17 +00:00
goto next_sv;
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
case ST_F_QTIME:
BUG/MINOR: contrib/prometheus-exporter: Return the time averages in seconds The metrics QTIME, CTIME, RTIME and TTIME are now returned in seconds using a float representation instead of in milliseconds. So these metrics are now consistent with their announced type and respect Prometheus naming conventions. This patch fixes the issue #288. It may be backported to 2.0. If so, the previous patch, introducing the support for float fields in stats is mantatory and should be backported first.
2019-09-24 14:35:19 +00:00
secs = (double)swrate_avg(sv->counters.q_time, TIME_STATS_SAMPLES) / 1000.0;
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
val = mkf_flt(FN_AVG, secs);
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
break;
case ST_F_CTIME:
BUG/MINOR: contrib/prometheus-exporter: Return the time averages in seconds The metrics QTIME, CTIME, RTIME and TTIME are now returned in seconds using a float representation instead of in milliseconds. So these metrics are now consistent with their announced type and respect Prometheus naming conventions. This patch fixes the issue #288. It may be backported to 2.0. If so, the previous patch, introducing the support for float fields in stats is mantatory and should be backported first.
2019-09-24 14:35:19 +00:00
secs = (double)swrate_avg(sv->counters.c_time, TIME_STATS_SAMPLES) / 1000.0;
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
val = mkf_flt(FN_AVG, secs);
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
break;
case ST_F_RTIME:
BUG/MINOR: contrib/prometheus-exporter: Return the time averages in seconds The metrics QTIME, CTIME, RTIME and TTIME are now returned in seconds using a float representation instead of in milliseconds. So these metrics are now consistent with their announced type and respect Prometheus naming conventions. This patch fixes the issue #288. It may be backported to 2.0. If so, the previous patch, introducing the support for float fields in stats is mantatory and should be backported first.
2019-09-24 14:35:19 +00:00
secs = (double)swrate_avg(sv->counters.d_time, TIME_STATS_SAMPLES) / 1000.0;
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
val = mkf_flt(FN_AVG, secs);
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
break;
case ST_F_TTIME:
BUG/MINOR: contrib/prometheus-exporter: Return the time averages in seconds The metrics QTIME, CTIME, RTIME and TTIME are now returned in seconds using a float representation instead of in milliseconds. So these metrics are now consistent with their announced type and respect Prometheus naming conventions. This patch fixes the issue #288. It may be backported to 2.0. If so, the previous patch, introducing the support for float fields in stats is mantatory and should be backported first.
2019-09-24 14:35:19 +00:00
secs = (double)swrate_avg(sv->counters.t_time, TIME_STATS_SAMPLES) / 1000.0;
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
val = mkf_flt(FN_AVG, secs);
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
break;
MINOR: contrib/prometheus-exporter: Report metrics about max times for sessions Now, for the sessions, the maximum times (queue, connect, response, total) are reported in addition of the averages over the last 1024 connections. These metrics are reported per backend and per server. Here are the metrics name : * haproxy_backend_max_queue_time_seconds * haproxy_backend_max_connect_time_seconds * haproxy_backend_max_response_time_seconds * haproxy_backend_max_total_time_seconds and * haproxy_server_max_queue_time_seconds * haproxy_server_max_connect_time_seconds * haproxy_server_max_response_time_seconds * haproxy_server_max_total_time_seconds This patch is related to #272.
2019-11-08 14:05:31 +00:00
case ST_F_QT_MAX:
secs = (double)sv->counters.qtime_max / 1000.0;
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
val = mkf_flt(FN_MAX, secs);
MINOR: contrib/prometheus-exporter: Report metrics about max times for sessions Now, for the sessions, the maximum times (queue, connect, response, total) are reported in addition of the averages over the last 1024 connections. These metrics are reported per backend and per server. Here are the metrics name : * haproxy_backend_max_queue_time_seconds * haproxy_backend_max_connect_time_seconds * haproxy_backend_max_response_time_seconds * haproxy_backend_max_total_time_seconds and * haproxy_server_max_queue_time_seconds * haproxy_server_max_connect_time_seconds * haproxy_server_max_response_time_seconds * haproxy_server_max_total_time_seconds This patch is related to #272.
2019-11-08 14:05:31 +00:00
break;
case ST_F_CT_MAX:
secs = (double)sv->counters.ctime_max / 1000.0;
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
val = mkf_flt(FN_MAX, secs);
MINOR: contrib/prometheus-exporter: Report metrics about max times for sessions Now, for the sessions, the maximum times (queue, connect, response, total) are reported in addition of the averages over the last 1024 connections. These metrics are reported per backend and per server. Here are the metrics name : * haproxy_backend_max_queue_time_seconds * haproxy_backend_max_connect_time_seconds * haproxy_backend_max_response_time_seconds * haproxy_backend_max_total_time_seconds and * haproxy_server_max_queue_time_seconds * haproxy_server_max_connect_time_seconds * haproxy_server_max_response_time_seconds * haproxy_server_max_total_time_seconds This patch is related to #272.
2019-11-08 14:05:31 +00:00
break;
case ST_F_RT_MAX:
secs = (double)sv->counters.dtime_max / 1000.0;
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
val = mkf_flt(FN_MAX, secs);
MINOR: contrib/prometheus-exporter: Report metrics about max times for sessions Now, for the sessions, the maximum times (queue, connect, response, total) are reported in addition of the averages over the last 1024 connections. These metrics are reported per backend and per server. Here are the metrics name : * haproxy_backend_max_queue_time_seconds * haproxy_backend_max_connect_time_seconds * haproxy_backend_max_response_time_seconds * haproxy_backend_max_total_time_seconds and * haproxy_server_max_queue_time_seconds * haproxy_server_max_connect_time_seconds * haproxy_server_max_response_time_seconds * haproxy_server_max_total_time_seconds This patch is related to #272.
2019-11-08 14:05:31 +00:00
break;
case ST_F_TT_MAX:
secs = (double)sv->counters.ttime_max / 1000.0;
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
val = mkf_flt(FN_MAX, secs);
MINOR: contrib/prometheus-exporter: Report metrics about max times for sessions Now, for the sessions, the maximum times (queue, connect, response, total) are reported in addition of the averages over the last 1024 connections. These metrics are reported per backend and per server. Here are the metrics name : * haproxy_backend_max_queue_time_seconds * haproxy_backend_max_connect_time_seconds * haproxy_backend_max_response_time_seconds * haproxy_backend_max_total_time_seconds and * haproxy_server_max_queue_time_seconds * haproxy_server_max_connect_time_seconds * haproxy_server_max_response_time_seconds * haproxy_server_max_total_time_seconds This patch is related to #272.
2019-11-08 14:05:31 +00:00
break;
MINOR: contrib/prometheus-exporter: Add heathcheck status/code in server metrics ST_F_CHECK_STATUS and ST_F_CHECK_CODE are now part of exported server metrics: * haproxy_server_check_status * haproxy_server_check_code The heathcheck status is an integer corresponding to HCHK_STATUS value.
2019-11-21 13:35:46 +00:00
case ST_F_CHECK_STATUS:
if ((sv->check.state & (CHK_ST_ENABLED|CHK_ST_PAUSED)) != CHK_ST_ENABLED)
goto next_sv;
MAJOR: contrib/prometheus-exporter: move health check status to labels this patch is a breaking change between v2.3 and v2.4: we move from using gauge value for health check states to labels values. The diff is quite small thanks to the preparation work from Christopher to allow more flexibility in labels, see commit 5a2f938732126f43bbf4cea5482c01552b0d0314 ("MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones") this is a follow up of commit c6464591a365bfcf509b322bdaa4d608c9395d75 ("MAJOR: contrib/prometheus-exporter: move ftd/bkd/srv states to labels"). The main goal being to be better aligned with prometheus use cases in terms of queries. More specifically to health checks, Pierre C. mentioned the possible quirks he had to put in place in order to make use of those metrics through prometheus: <aggregator_function> by(proxy, check_status) (count_values by(proxy, instance) ("check_status", haproxy_server_check_status)) I am perfectly aware this introduces a lot more metrics but I don't see how we can improve the usability without it. The main issue remains in the cardinality of the states which are > 20. Prometheus recommends to stay below a cardinality of 10 for a given metric but I consider our case very specific, because highly linked to the level of precision haproxy exposes. Even before this patch I saw several large production setup (a few hundreds of MB in output) which are making use of the scope parameter to simply ignore the server metrics, so that the scrapping can be faster, and memory consumed on client side not too high. So I believe we should eventually continue in that direction and offer more granularity of filtering of the output. That being said it is already possible to filter out the data on prometheus client side. this is related to github issue #1029 Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-01 12:11:51 +00:00
BUG/MINOR: contrib/prometheus-exporter: Restart labels dump at the right pos For some metrics, several lines are produced per entity, one per label value. For instance, the health-check status (ST_F_CHECK_STATUS) or the entity status (ST_F_STATUS). The dump may be stopped in the middle of the labels processing if the output buffer is full. This means the next time, we must take care to restart on the right label value. For now, this part is buggy and we always restart to dump all the label values again from the beginning. To be sure to restart at the right position, the field <ctx.stats.st_code> in the applet context is used to save the last position. Of course, we take care to reset this value when necessary. This fix is specific for 2.4. No backport needed.
2021-02-01 14:05:21 +00:00
for (; appctx->ctx.stats.st_code < HCHK_STATUS_SIZE; appctx->ctx.stats.st_code++) {
if (get_check_status_result(appctx->ctx.stats.st_code) < CHK_RES_FAILED)
MAJOR: contrib/prometheus-exporter: move health check status to labels this patch is a breaking change between v2.3 and v2.4: we move from using gauge value for health check states to labels values. The diff is quite small thanks to the preparation work from Christopher to allow more flexibility in labels, see commit 5a2f938732126f43bbf4cea5482c01552b0d0314 ("MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones") this is a follow up of commit c6464591a365bfcf509b322bdaa4d608c9395d75 ("MAJOR: contrib/prometheus-exporter: move ftd/bkd/srv states to labels"). The main goal being to be better aligned with prometheus use cases in terms of queries. More specifically to health checks, Pierre C. mentioned the possible quirks he had to put in place in order to make use of those metrics through prometheus: <aggregator_function> by(proxy, check_status) (count_values by(proxy, instance) ("check_status", haproxy_server_check_status)) I am perfectly aware this introduces a lot more metrics but I don't see how we can improve the usability without it. The main issue remains in the cardinality of the states which are > 20. Prometheus recommends to stay below a cardinality of 10 for a given metric but I consider our case very specific, because highly linked to the level of precision haproxy exposes. Even before this patch I saw several large production setup (a few hundreds of MB in output) which are making use of the scope parameter to simply ignore the server metrics, so that the scrapping can be faster, and memory consumed on client side not too high. So I believe we should eventually continue in that direction and offer more granularity of filtering of the output. That being said it is already possible to filter out the data on prometheus client side. this is related to github issue #1029 Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-01 12:11:51 +00:00
continue;
BUG/MINOR: contrib/prometheus-exporter: Restart labels dump at the right pos For some metrics, several lines are produced per entity, one per label value. For instance, the health-check status (ST_F_CHECK_STATUS) or the entity status (ST_F_STATUS). The dump may be stopped in the middle of the labels processing if the output buffer is full. This means the next time, we must take care to restart on the right label value. For now, this part is buggy and we always restart to dump all the label values again from the beginning. To be sure to restart at the right position, the field <ctx.stats.st_code> in the applet context is used to save the last position. Of course, we take care to reset this value when necessary. This fix is specific for 2.4. No backport needed.
2021-02-01 14:05:21 +00:00
val = mkf_u32(FO_STATUS, sv->check.status == appctx->ctx.stats.st_code);
check_state = get_check_status_info(appctx->ctx.stats.st_code);
MAJOR: contrib/prometheus-exporter: move health check status to labels this patch is a breaking change between v2.3 and v2.4: we move from using gauge value for health check states to labels values. The diff is quite small thanks to the preparation work from Christopher to allow more flexibility in labels, see commit 5a2f938732126f43bbf4cea5482c01552b0d0314 ("MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones") this is a follow up of commit c6464591a365bfcf509b322bdaa4d608c9395d75 ("MAJOR: contrib/prometheus-exporter: move ftd/bkd/srv states to labels"). The main goal being to be better aligned with prometheus use cases in terms of queries. More specifically to health checks, Pierre C. mentioned the possible quirks he had to put in place in order to make use of those metrics through prometheus: <aggregator_function> by(proxy, check_status) (count_values by(proxy, instance) ("check_status", haproxy_server_check_status)) I am perfectly aware this introduces a lot more metrics but I don't see how we can improve the usability without it. The main issue remains in the cardinality of the states which are > 20. Prometheus recommends to stay below a cardinality of 10 for a given metric but I consider our case very specific, because highly linked to the level of precision haproxy exposes. Even before this patch I saw several large production setup (a few hundreds of MB in output) which are making use of the scope parameter to simply ignore the server metrics, so that the scrapping can be faster, and memory consumed on client side not too high. So I believe we should eventually continue in that direction and offer more granularity of filtering of the output. That being said it is already possible to filter out the data on prometheus client side. this is related to github issue #1029 Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-01 12:11:51 +00:00
labels[2].name = ist("state");
CLEANUP: Apply ist.cocci This cleans up ist handling.
2021-09-15 11:58:44 +00:00
labels[2].value = ist(check_state);
MAJOR: contrib/prometheus-exporter: move health check status to labels this patch is a breaking change between v2.3 and v2.4: we move from using gauge value for health check states to labels values. The diff is quite small thanks to the preparation work from Christopher to allow more flexibility in labels, see commit 5a2f938732126f43bbf4cea5482c01552b0d0314 ("MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones") this is a follow up of commit c6464591a365bfcf509b322bdaa4d608c9395d75 ("MAJOR: contrib/prometheus-exporter: move ftd/bkd/srv states to labels"). The main goal being to be better aligned with prometheus use cases in terms of queries. More specifically to health checks, Pierre C. mentioned the possible quirks he had to put in place in order to make use of those metrics through prometheus: <aggregator_function> by(proxy, check_status) (count_values by(proxy, instance) ("check_status", haproxy_server_check_status)) I am perfectly aware this introduces a lot more metrics but I don't see how we can improve the usability without it. The main issue remains in the cardinality of the states which are > 20. Prometheus recommends to stay below a cardinality of 10 for a given metric but I consider our case very specific, because highly linked to the level of precision haproxy exposes. Even before this patch I saw several large production setup (a few hundreds of MB in output) which are making use of the scope parameter to simply ignore the server metrics, so that the scrapping can be faster, and memory consumed on client side not too high. So I believe we should eventually continue in that direction and offer more granularity of filtering of the output. That being said it is already possible to filter out the data on prometheus client side. this is related to github issue #1029 Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-01 12:11:51 +00:00
if (!promex_dump_metric(appctx, htx, prefix, &promex_st_metrics[appctx->st2],
&val, labels, &out, max))
goto full;
}
BUG/MINOR: contrib/prometheus-exporter: Restart labels dump at the right pos For some metrics, several lines are produced per entity, one per label value. For instance, the health-check status (ST_F_CHECK_STATUS) or the entity status (ST_F_STATUS). The dump may be stopped in the middle of the labels processing if the output buffer is full. This means the next time, we must take care to restart on the right label value. For now, this part is buggy and we always restart to dump all the label values again from the beginning. To be sure to restart at the right position, the field <ctx.stats.st_code> in the applet context is used to save the last position. Of course, we take care to reset this value when necessary. This fix is specific for 2.4. No backport needed.
2021-02-01 14:05:21 +00:00
appctx->ctx.stats.st_code = 0;
MAJOR: contrib/prometheus-exporter: move health check status to labels this patch is a breaking change between v2.3 and v2.4: we move from using gauge value for health check states to labels values. The diff is quite small thanks to the preparation work from Christopher to allow more flexibility in labels, see commit 5a2f938732126f43bbf4cea5482c01552b0d0314 ("MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones") this is a follow up of commit c6464591a365bfcf509b322bdaa4d608c9395d75 ("MAJOR: contrib/prometheus-exporter: move ftd/bkd/srv states to labels"). The main goal being to be better aligned with prometheus use cases in terms of queries. More specifically to health checks, Pierre C. mentioned the possible quirks he had to put in place in order to make use of those metrics through prometheus: <aggregator_function> by(proxy, check_status) (count_values by(proxy, instance) ("check_status", haproxy_server_check_status)) I am perfectly aware this introduces a lot more metrics but I don't see how we can improve the usability without it. The main issue remains in the cardinality of the states which are > 20. Prometheus recommends to stay below a cardinality of 10 for a given metric but I consider our case very specific, because highly linked to the level of precision haproxy exposes. Even before this patch I saw several large production setup (a few hundreds of MB in output) which are making use of the scope parameter to simply ignore the server metrics, so that the scrapping can be faster, and memory consumed on client side not too high. So I believe we should eventually continue in that direction and offer more granularity of filtering of the output. That being said it is already possible to filter out the data on prometheus client side. this is related to github issue #1029 Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-01 12:11:51 +00:00
goto next_sv;
MINOR: contrib/prometheus-exporter: Add heathcheck status/code in server metrics ST_F_CHECK_STATUS and ST_F_CHECK_CODE are now part of exported server metrics: * haproxy_server_check_status * haproxy_server_check_code The heathcheck status is an integer corresponding to HCHK_STATUS value.
2019-11-21 13:35:46 +00:00
case ST_F_CHECK_CODE:
if ((sv->check.state & (CHK_ST_ENABLED|CHK_ST_PAUSED)) != CHK_ST_ENABLED)
goto next_sv;
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
val = mkf_u32(FN_OUTPUT, (sv->check.status < HCHK_STATUS_L57DATA) ? 0 : sv->check.code);
MINOR: contrib/prometheus-exporter: Add heathcheck status/code in server metrics ST_F_CHECK_STATUS and ST_F_CHECK_CODE are now part of exported server metrics: * haproxy_server_check_status * haproxy_server_check_code The heathcheck status is an integer corresponding to HCHK_STATUS value.
2019-11-21 13:35:46 +00:00
break;
MINOR: contrib/prometheus-exporter: Add the last heathcheck duration metric ST_F_CHECK_DURATION is now part of exported server metrics, named haproxy_server_check_duration_seconds and expressed in seconds. For a given server, this value is exported only if the healthcheck is finished (the status is greater or equal to HCHK_STATUS_CHECKED). This patch fixes the issue #519. It may be backported as fat as 2.0.
2020-02-27 15:12:07 +00:00
case ST_F_CHECK_DURATION:
if (sv->check.status < HCHK_STATUS_CHECKED)
goto next_sv;
secs = (double)sv->check.duration / 1000.0;
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
val = mkf_flt(FN_DURATION, secs);
MINOR: contrib/prometheus-exporter: Add the last heathcheck duration metric ST_F_CHECK_DURATION is now part of exported server metrics, named haproxy_server_check_duration_seconds and expressed in seconds. For a given server, this value is exported only if the healthcheck is finished (the status is greater or equal to HCHK_STATUS_CHECKED). This patch fixes the issue #519. It may be backported as fat as 2.0.
2020-02-27 15:12:07 +00:00
break;
MINOR: stats: Expose native cum_req metric for a server Expose native cum_req metric for a server: so far it was calculated as a sum or all responses. Rename it from Cum. HTTP Responses to Cum. HTTP Requests to be consistent with Frontend and Backend.
2020-05-15 16:32:51 +00:00
case ST_F_REQ_TOT:
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
if (px->mode != PR_MODE_HTTP)
goto next_px;
MINOR: contrib/prometheus-exporter: use fill_sv_stats for server dump use `stats_fill_sv_stats` when possible to avoid duplicating code. the following metrics have a change of behaviour: haproxy_server_limit_sessions haproxy_server_queue_limit haproxy_server_check_failures_total haproxy_server_check_up_down_total haproxy_server_downtime_seconds_total haproxy_server_current_throttle haproxy_server_idle_connections_limit depending on cases, if the limit was not configured or enabled, NaN is returned instead. It should not be an issue for users, even better than before as it provides more precise info. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-25 16:29:04 +00:00
val = stats[appctx->st2];
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
break;
BUG/MINOR: contrib/prometheus-exporter: Add missing label for ST_F_HRSP_1XX Since the labels are dynamically created for each metric, the "code" label of the ST_F_HRSP_1XX field is missing. To fix the bug, this metric is handled in the same way the other ST_F_HRSP_* field are. We only take care to dump the metric header only once. This bug was introduced by the commit 5a2f93873 ("MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones"). No backport needed.
2021-02-01 13:55:37 +00:00
case ST_F_HRSP_1XX:
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
case ST_F_HRSP_2XX:
case ST_F_HRSP_3XX:
case ST_F_HRSP_4XX:
case ST_F_HRSP_5XX:
case ST_F_HRSP_OTHER:
if (px->mode != PR_MODE_HTTP)
goto next_px;
BUG/MINOR: contrib/prometheus-exporter: Add missing label for ST_F_HRSP_1XX Since the labels are dynamically created for each metric, the "code" label of the ST_F_HRSP_1XX field is missing. To fix the bug, this metric is handled in the same way the other ST_F_HRSP_* field are. We only take care to dump the metric header only once. This bug was introduced by the commit 5a2f93873 ("MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones"). No backport needed.
2021-02-01 13:55:37 +00:00
if (appctx->st2 != ST_F_HRSP_1XX)
appctx->ctx.stats.flags &= ~PROMEX_FL_METRIC_HDR;
MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones Instead of using static labels for metrics, we now use an array of labels, filled for each metrics if necessary and passed to the dump function. This way, it is easier to extend the promex service. For now, there are at most 8 labels per metrics. This limit may be raised by changing PROMEX_MAX_LABELS value. And to ease labels addition, a label is defined as a key/value pair. The formatting is handled by the dump function. For the proxies and servers, the first entry of the array is always the proxy name. In addition, for the servers, the second entry is always the server name.
2021-01-28 10:24:17 +00:00
labels[2].name = ist("code");
labels[2].value = promex_hrsp_code[appctx->st2 - ST_F_HRSP_1XX];
MINOR: contrib/prometheus-exporter: use fill_sv_stats for server dump use `stats_fill_sv_stats` when possible to avoid duplicating code. the following metrics have a change of behaviour: haproxy_server_limit_sessions haproxy_server_queue_limit haproxy_server_check_failures_total haproxy_server_check_up_down_total haproxy_server_downtime_seconds_total haproxy_server_current_throttle haproxy_server_idle_connections_limit depending on cases, if the limit was not configured or enabled, NaN is returned instead. It should not be an issue for users, even better than before as it provides more precise info. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-25 16:29:04 +00:00
val = stats[appctx->st2];
MINOR: contrib/prometheus-exporter: Add missing global and per-server metrics Following metrics are now exported by the prometheus exporter to reflect recent changes on HAProxy : * haproxy_process_failed_resolutions * haproxy_process_bytes_out_total * haproxy_process_spliced_bytes_out_total * haproxy_process_bytes_out_rate and * haproxy_server_unsafe_idle_connections_current * haproxy_server_safe_idle_connections_current * haproxy_server_used_connections_current * haproxy_server_need_connections_current
2020-07-10 13:39:39 +00:00
break;
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
default:
MINOR: contrib/prometheus-exporter: use fill_sv_stats for server dump use `stats_fill_sv_stats` when possible to avoid duplicating code. the following metrics have a change of behaviour: haproxy_server_limit_sessions haproxy_server_queue_limit haproxy_server_check_failures_total haproxy_server_check_up_down_total haproxy_server_downtime_seconds_total haproxy_server_current_throttle haproxy_server_idle_connections_limit depending on cases, if the limit was not configured or enabled, NaN is returned instead. It should not be an issue for users, even better than before as it provides more precise info. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-25 16:29:04 +00:00
val = stats[appctx->st2];
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
}
MAJOR: contrib/prometheus-exporter: move ftd/bkd/srv states to labels this patch is a breaking change between v2.3 and v2.4: we move from using gauge value for frontend/backend/servers states to labels values. the main motivation being I realised it is very difficult to make use of it without hard coded quirks on prometheus client side; especially because the main use is often to group by state, which is harder when the state is the value of the metric. in order to achieve that we iterate on the status metric to generate labels, and so as many metrics. this is the first step to resolve github issue #1029 A second step should address health check states. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-27 21:40:17 +00:00
if (!promex_dump_metric(appctx, htx, prefix, &promex_st_metrics[appctx->st2],
MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static ones Instead of using static labels for metrics, we now use an array of labels, filled for each metrics if necessary and passed to the dump function. This way, it is easier to extend the promex service. For now, there are at most 8 labels per metrics. This limit may be raised by changing PROMEX_MAX_LABELS value. And to ease labels addition, a label is defined as a key/value pair. The formatting is handled by the dump function. For the proxies and servers, the first entry of the array is always the proxy name. In addition, for the servers, the second entry is always the server name.
2021-01-28 10:24:17 +00:00
&val, labels, &out, max))
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
goto full;
MINOR: contrib/prometheus-exporter: Add a param to ignore servers in maintenance By passing the parameter "no-maint" in the query-string, it is now possible to ignore servers in maintenance. It means that the metrics for servers in this state will not be exported.
2019-11-19 13:18:24 +00:00
next_sv:
MINOR: stats: hide px/sv/li fields in applet struct Use an opaque pointer to store proxy instance. Regroup server/listener as a single opaque pointer. This has the benefit to render the structure more evolutive to support statistics on other types of objects in the future. This patch is needed to extend stat support for components other than proxies objects. The prometheus module has been adapted for these changes.
2020-10-02 16:32:02 +00:00
appctx->ctx.stats.obj2 = sv->next;
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
}
next_px:
MINOR: stats: hide px/sv/li fields in applet struct Use an opaque pointer to store proxy instance. Regroup server/listener as a single opaque pointer. This has the benefit to render the structure more evolutive to support statistics on other types of objects in the future. This patch is needed to extend stat support for components other than proxies objects. The prometheus module has been adapted for these changes.
2020-10-02 16:32:02 +00:00
appctx->ctx.stats.obj1 = px->next;
appctx->ctx.stats.obj2 = (appctx->ctx.stats.obj1 ? ((struct proxy *)appctx->ctx.stats.obj1)->srv : NULL);
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
}
appctx->ctx.stats.flags |= PROMEX_FL_METRIC_HDR;
MINOR: stats: hide px/sv/li fields in applet struct Use an opaque pointer to store proxy instance. Regroup server/listener as a single opaque pointer. This has the benefit to render the structure more evolutive to support statistics on other types of objects in the future. This patch is needed to extend stat support for components other than proxies objects. The prometheus module has been adapted for these changes.
2020-10-02 16:32:02 +00:00
appctx->ctx.stats.obj1 = proxies_list;
appctx->ctx.stats.obj2 = (appctx->ctx.stats.obj1 ? ((struct proxy *)appctx->ctx.stats.obj1)->srv : NULL);
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
}
end:
BUG/MINOR: contrib/prometheus-exporter: Don't try to add empty data blocks When the response buffer is full and nothing more can be inserted, it is important to not try to insert an empty data block. Otherwise, when the function channel_add_input() is called, the flag CF_READ_PARTIAL is set on the response channel while nothing was read and the stream is uselessly woken up. Finally, we have loop while the response buffer is full. This patch must be backported to 2.0.
2019-07-04 08:03:28 +00:00
if (out.len) {
if (!htx_add_data_atonce(htx, out))
return -1; /* Unexpected and unrecoverable error */
channel_add_input(chn, out.len);
}
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
return ret;
full:
ret = 0;
goto end;
}
MEDIUM: contrib/prometheus-exporter: export base stick table stats I saw some people falling back to unix socket to collect some data they could not find in prometheus exporter. One of them is base info from stick tables (used/size). I do not plan to extend it more for now; keys are quite a mess to handle. This should resolve github issue #1008. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-07 19:42:38 +00:00
/* Dump stick table metrics (prefixed by "haproxy_sticktable_"). It returns 1 on success,
* 0 if <htx> is full and -1 in case of any error. */
static int promex_dump_sticktable_metrics(struct appctx *appctx, struct htx *htx)
{
static struct ist prefix = IST("haproxy_sticktable_");
struct field val;
struct channel *chn = si_ic(appctx->owner);
struct ist out = ist2(trash.area, 0);
size_t max = htx_get_max_blksz(htx, channel_htx_recv_max(chn, htx));
int ret = 1;
struct stktable *t;
for (; appctx->st2 < STICKTABLE_TOTAL_FIELDS; appctx->st2++) {
if (!(promex_sticktable_metrics[appctx->st2].flags & appctx->ctx.stats.flags))
continue;
while (appctx->ctx.stats.obj1) {
struct promex_label labels[PROMEX_MAX_LABELS - 1] = {};
t = appctx->ctx.stats.obj1;
if (!t->size)
goto next_px;
labels[0].name = ist("name");
labels[0].value = ist2(t->id, strlen(t->id));
labels[1].name = ist("type");
labels[1].value = ist2(stktable_types[t->type].kw, strlen(stktable_types[t->type].kw));
switch (appctx->st2) {
case STICKTABLE_SIZE:
val = mkf_u32(FN_GAUGE, t->size);
break;
case STICKTABLE_USED:
val = mkf_u32(FN_GAUGE, t->current);
break;
default:
goto next_px;
}
if (!promex_dump_metric(appctx, htx, prefix,
&promex_sticktable_metrics[appctx->st2],
&val, labels, &out, max))
goto full;
next_px:
appctx->ctx.stats.obj1 = t->next;
}
appctx->ctx.stats.flags |= PROMEX_FL_METRIC_HDR;
appctx->ctx.stats.obj1 = stktables_list;
}
end:
if (out.len) {
if (!htx_add_data_atonce(htx, out))
return -1; /* Unexpected and unrecoverable error */
channel_add_input(chn, out.len);
}
return ret;
full:
ret = 0;
goto end;
}
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
/* Dump all metrics (global, frontends, backends and servers) depending on the
* dumper state (appctx->st1). It returns 1 on success, 0 if <htx> is full and
MINOR: stats: hide px/sv/li fields in applet struct Use an opaque pointer to store proxy instance. Regroup server/listener as a single opaque pointer. This has the benefit to render the structure more evolutive to support statistics on other types of objects in the future. This patch is needed to extend stat support for components other than proxies objects. The prometheus module has been adapted for these changes.
2020-10-02 16:32:02 +00:00
* -1 in case of any error.
* Uses <appctx.ctx.stats.obj1> as a pointer to the current proxy and <obj2> as
* a pointer to the current server/listener. */
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
static int promex_dump_metrics(struct appctx *appctx, struct stream_interface *si, struct htx *htx)
{
int ret;
switch (appctx->st1) {
case PROMEX_DUMPER_INIT:
MINOR: stats: hide px/sv/li fields in applet struct Use an opaque pointer to store proxy instance. Regroup server/listener as a single opaque pointer. This has the benefit to render the structure more evolutive to support statistics on other types of objects in the future. This patch is needed to extend stat support for components other than proxies objects. The prometheus module has been adapted for these changes.
2020-10-02 16:32:02 +00:00
appctx->ctx.stats.obj1 = NULL;
appctx->ctx.stats.obj2 = NULL;
BUG/MEDIUM: contrib/prometheus-exporter: Properly set flags to dump metrics When the metrics are dumped, in the main function promex_dump_metrics(), the appctx flags are set before entering in a new scope, among others things to know which metrics names and descriptions to use. But, those flags are not restored when the dump is interrupted because of a full output buffer. If this happens after the dump of global metrics, it may only lead to extra #TYPE and #HELP lines. But if this happens during the dump of global metrics, the following dumps of frontends, backends and servers metrics use names and descriptions of global ones with the unmatching indexes. This first leads to unexisting metrics names. For instance, "haproxy_frontend_nbproc". But also to out-of-bound accesses to name and description arrays because there are more stats fields than info fields. It is easy to reproduce the bug using small buffers, setting tune.bufsize to 8192 for instance. This patch should fix the issue #666. It must be backported as far as 2.0.
2020-06-05 06:18:56 +00:00
appctx->ctx.stats.flags |= (PROMEX_FL_METRIC_HDR|PROMEX_FL_INFO_METRIC);
BUG/MINOR: contrib/prometheus-exporter: Restart labels dump at the right pos For some metrics, several lines are produced per entity, one per label value. For instance, the health-check status (ST_F_CHECK_STATUS) or the entity status (ST_F_STATUS). The dump may be stopped in the middle of the labels processing if the output buffer is full. This means the next time, we must take care to restart on the right label value. For now, this part is buggy and we always restart to dump all the label values again from the beginning. To be sure to restart at the right position, the field <ctx.stats.st_code> in the applet context is used to save the last position. Of course, we take care to reset this value when necessary. This fix is specific for 2.4. No backport needed.
2021-02-01 14:05:21 +00:00
appctx->ctx.stats.st_code = 0;
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
appctx->st2 = INF_NAME;
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
appctx->st1 = PROMEX_DUMPER_GLOBAL;
/* fall through */
case PROMEX_DUMPER_GLOBAL:
MINOR: contrib/prometheus-exporter: filter exported metrics by scope Now, the prometheus exporter parses the HTTP query-string to filter or to adapt the exported metrics. In this first version, it is only possible select the scopes of metrics to export. To do so, one or more parameters with "scope" as name must be passed in the query-string, with one of those values: global, frontend, backend, server or '*' (means all). A scope parameter with no value means to filter out all scopes (nothing is returned). The scope parameters are parsed in their appearance order in the query-string. So an empty scope will reset all scopes already parsed. But it can be overridden by following scope parameters in the query-string. By default everything is exported. The filtering can also be done on prometheus scraping configuration, but general aim is to optimise the source of data to improve load and scraping time. This is particularly true for huge configuration with thousands of backends and servers. Also note that this configuration was possible on the previous official haproxy exporter but with even more parameters to select the needed metrics. Here we thought it was sufficient to simply avoid a given type of metric. However, more filters are still possible. Thanks to William Dauchy. This patch is based on his work.
2019-11-18 13:47:08 +00:00
if (appctx->ctx.stats.flags & PROMEX_FL_SCOPE_GLOBAL) {
ret = promex_dump_global_metrics(appctx, htx);
if (ret <= 0) {
if (ret == -1)
goto error;
goto full;
}
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
}
MINOR: stats: hide px/sv/li fields in applet struct Use an opaque pointer to store proxy instance. Regroup server/listener as a single opaque pointer. This has the benefit to render the structure more evolutive to support statistics on other types of objects in the future. This patch is needed to extend stat support for components other than proxies objects. The prometheus module has been adapted for these changes.
2020-10-02 16:32:02 +00:00
appctx->ctx.stats.obj1 = proxies_list;
appctx->ctx.stats.obj2 = NULL;
BUG/MEDIUM: contrib/prometheus-exporter: Properly set flags to dump metrics When the metrics are dumped, in the main function promex_dump_metrics(), the appctx flags are set before entering in a new scope, among others things to know which metrics names and descriptions to use. But, those flags are not restored when the dump is interrupted because of a full output buffer. If this happens after the dump of global metrics, it may only lead to extra #TYPE and #HELP lines. But if this happens during the dump of global metrics, the following dumps of frontends, backends and servers metrics use names and descriptions of global ones with the unmatching indexes. This first leads to unexisting metrics names. For instance, "haproxy_frontend_nbproc". But also to out-of-bound accesses to name and description arrays because there are more stats fields than info fields. It is easy to reproduce the bug using small buffers, setting tune.bufsize to 8192 for instance. This patch should fix the issue #666. It must be backported as far as 2.0.
2020-06-05 06:18:56 +00:00
appctx->ctx.stats.flags &= ~PROMEX_FL_INFO_METRIC;
MINOR: contrib/prometheus-exporter: Split the PROMEX_FL_STATS_METRIC flag PROMEX_FL_STATS_METRIC flag is splitted in 3 flags to easily identify the processed entity type (frontend, backend or server). Thus, now we are using PROMEX_FL_FRONT_METRIC, PROMEX_FL_BACK_METRIC or PROMEX_FL_SRV_METRIC. These flags will be used to know if a metric is defined and must be exported for a given entity type.
2021-01-20 14:02:50 +00:00
appctx->ctx.stats.flags |= (PROMEX_FL_METRIC_HDR|PROMEX_FL_FRONT_METRIC);
BUG/MINOR: contrib/prometheus-exporter: Restart labels dump at the right pos For some metrics, several lines are produced per entity, one per label value. For instance, the health-check status (ST_F_CHECK_STATUS) or the entity status (ST_F_STATUS). The dump may be stopped in the middle of the labels processing if the output buffer is full. This means the next time, we must take care to restart on the right label value. For now, this part is buggy and we always restart to dump all the label values again from the beginning. To be sure to restart at the right position, the field <ctx.stats.st_code> in the applet context is used to save the last position. Of course, we take care to reset this value when necessary. This fix is specific for 2.4. No backport needed.
2021-02-01 14:05:21 +00:00
appctx->ctx.stats.st_code = 0;
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
appctx->st2 = ST_F_PXNAME;
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
appctx->st1 = PROMEX_DUMPER_FRONT;
/* fall through */
case PROMEX_DUMPER_FRONT:
MINOR: contrib/prometheus-exporter: filter exported metrics by scope Now, the prometheus exporter parses the HTTP query-string to filter or to adapt the exported metrics. In this first version, it is only possible select the scopes of metrics to export. To do so, one or more parameters with "scope" as name must be passed in the query-string, with one of those values: global, frontend, backend, server or '*' (means all). A scope parameter with no value means to filter out all scopes (nothing is returned). The scope parameters are parsed in their appearance order in the query-string. So an empty scope will reset all scopes already parsed. But it can be overridden by following scope parameters in the query-string. By default everything is exported. The filtering can also be done on prometheus scraping configuration, but general aim is to optimise the source of data to improve load and scraping time. This is particularly true for huge configuration with thousands of backends and servers. Also note that this configuration was possible on the previous official haproxy exporter but with even more parameters to select the needed metrics. Here we thought it was sufficient to simply avoid a given type of metric. However, more filters are still possible. Thanks to William Dauchy. This patch is based on his work.
2019-11-18 13:47:08 +00:00
if (appctx->ctx.stats.flags & PROMEX_FL_SCOPE_FRONT) {
ret = promex_dump_front_metrics(appctx, htx);
if (ret <= 0) {
if (ret == -1)
goto error;
goto full;
}
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
}
MINOR: stats: hide px/sv/li fields in applet struct Use an opaque pointer to store proxy instance. Regroup server/listener as a single opaque pointer. This has the benefit to render the structure more evolutive to support statistics on other types of objects in the future. This patch is needed to extend stat support for components other than proxies objects. The prometheus module has been adapted for these changes.
2020-10-02 16:32:02 +00:00
appctx->ctx.stats.obj1 = proxies_list;
MEDIUM: contrib/prometheus-exporter: add listen stats this was a missing piece for a while now even though it was planned. This patch adds listen stats. Nothing in particular but we make use of the status helper previously added. `promex_st_metrics` diff also looks scary, but I had to realign all lines. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-14 22:22:56 +00:00
appctx->ctx.stats.obj2 = LIST_NEXT(&proxies_list->conf.listeners, struct listener *, by_fe);
MINOR: contrib/prometheus-exporter: Split the PROMEX_FL_STATS_METRIC flag PROMEX_FL_STATS_METRIC flag is splitted in 3 flags to easily identify the processed entity type (frontend, backend or server). Thus, now we are using PROMEX_FL_FRONT_METRIC, PROMEX_FL_BACK_METRIC or PROMEX_FL_SRV_METRIC. These flags will be used to know if a metric is defined and must be exported for a given entity type.
2021-01-20 14:02:50 +00:00
appctx->ctx.stats.flags &= ~PROMEX_FL_FRONT_METRIC;
MEDIUM: contrib/prometheus-exporter: add listen stats this was a missing piece for a while now even though it was planned. This patch adds listen stats. Nothing in particular but we make use of the status helper previously added. `promex_st_metrics` diff also looks scary, but I had to realign all lines. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-14 22:22:56 +00:00
appctx->ctx.stats.flags |= (PROMEX_FL_METRIC_HDR|PROMEX_FL_LI_METRIC);
appctx->ctx.stats.st_code = 0;
appctx->st2 = ST_F_PXNAME;
appctx->st1 = PROMEX_DUMPER_LI;
/* fall through */
case PROMEX_DUMPER_LI:
if (appctx->ctx.stats.flags & PROMEX_FL_SCOPE_LI) {
ret = promex_dump_listener_metrics(appctx, htx);
if (ret <= 0) {
if (ret == -1)
goto error;
goto full;
}
}
appctx->ctx.stats.obj1 = proxies_list;
appctx->ctx.stats.obj2 = NULL;
appctx->ctx.stats.flags &= ~PROMEX_FL_LI_METRIC;
MINOR: contrib/prometheus-exporter: Split the PROMEX_FL_STATS_METRIC flag PROMEX_FL_STATS_METRIC flag is splitted in 3 flags to easily identify the processed entity type (frontend, backend or server). Thus, now we are using PROMEX_FL_FRONT_METRIC, PROMEX_FL_BACK_METRIC or PROMEX_FL_SRV_METRIC. These flags will be used to know if a metric is defined and must be exported for a given entity type.
2021-01-20 14:02:50 +00:00
appctx->ctx.stats.flags |= (PROMEX_FL_METRIC_HDR|PROMEX_FL_BACK_METRIC);
BUG/MINOR: contrib/prometheus-exporter: Restart labels dump at the right pos For some metrics, several lines are produced per entity, one per label value. For instance, the health-check status (ST_F_CHECK_STATUS) or the entity status (ST_F_STATUS). The dump may be stopped in the middle of the labels processing if the output buffer is full. This means the next time, we must take care to restart on the right label value. For now, this part is buggy and we always restart to dump all the label values again from the beginning. To be sure to restart at the right position, the field <ctx.stats.st_code> in the applet context is used to save the last position. Of course, we take care to reset this value when necessary. This fix is specific for 2.4. No backport needed.
2021-02-01 14:05:21 +00:00
appctx->ctx.stats.st_code = 0;
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
appctx->st2 = ST_F_PXNAME;
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
appctx->st1 = PROMEX_DUMPER_BACK;
/* fall through */
case PROMEX_DUMPER_BACK:
MINOR: contrib/prometheus-exporter: filter exported metrics by scope Now, the prometheus exporter parses the HTTP query-string to filter or to adapt the exported metrics. In this first version, it is only possible select the scopes of metrics to export. To do so, one or more parameters with "scope" as name must be passed in the query-string, with one of those values: global, frontend, backend, server or '*' (means all). A scope parameter with no value means to filter out all scopes (nothing is returned). The scope parameters are parsed in their appearance order in the query-string. So an empty scope will reset all scopes already parsed. But it can be overridden by following scope parameters in the query-string. By default everything is exported. The filtering can also be done on prometheus scraping configuration, but general aim is to optimise the source of data to improve load and scraping time. This is particularly true for huge configuration with thousands of backends and servers. Also note that this configuration was possible on the previous official haproxy exporter but with even more parameters to select the needed metrics. Here we thought it was sufficient to simply avoid a given type of metric. However, more filters are still possible. Thanks to William Dauchy. This patch is based on his work.
2019-11-18 13:47:08 +00:00
if (appctx->ctx.stats.flags & PROMEX_FL_SCOPE_BACK) {
ret = promex_dump_back_metrics(appctx, htx);
if (ret <= 0) {
if (ret == -1)
goto error;
goto full;
}
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
}
MINOR: stats: hide px/sv/li fields in applet struct Use an opaque pointer to store proxy instance. Regroup server/listener as a single opaque pointer. This has the benefit to render the structure more evolutive to support statistics on other types of objects in the future. This patch is needed to extend stat support for components other than proxies objects. The prometheus module has been adapted for these changes.
2020-10-02 16:32:02 +00:00
appctx->ctx.stats.obj1 = proxies_list;
appctx->ctx.stats.obj2 = (appctx->ctx.stats.obj1 ? ((struct proxy *)appctx->ctx.stats.obj1)->srv : NULL);
MINOR: contrib/prometheus-exporter: Split the PROMEX_FL_STATS_METRIC flag PROMEX_FL_STATS_METRIC flag is splitted in 3 flags to easily identify the processed entity type (frontend, backend or server). Thus, now we are using PROMEX_FL_FRONT_METRIC, PROMEX_FL_BACK_METRIC or PROMEX_FL_SRV_METRIC. These flags will be used to know if a metric is defined and must be exported for a given entity type.
2021-01-20 14:02:50 +00:00
appctx->ctx.stats.flags &= ~PROMEX_FL_BACK_METRIC;
appctx->ctx.stats.flags |= (PROMEX_FL_METRIC_HDR|PROMEX_FL_SRV_METRIC);
BUG/MINOR: contrib/prometheus-exporter: Restart labels dump at the right pos For some metrics, several lines are produced per entity, one per label value. For instance, the health-check status (ST_F_CHECK_STATUS) or the entity status (ST_F_STATUS). The dump may be stopped in the middle of the labels processing if the output buffer is full. This means the next time, we must take care to restart on the right label value. For now, this part is buggy and we always restart to dump all the label values again from the beginning. To be sure to restart at the right position, the field <ctx.stats.st_code> in the applet context is used to save the last position. Of course, we take care to reset this value when necessary. This fix is specific for 2.4. No backport needed.
2021-02-01 14:05:21 +00:00
appctx->ctx.stats.st_code = 0;
MEDIUM: contrib/prometheus-exporter: Rework matrices defining Promex metrics The global and stats matrices are replaced by a simpler ones. Now we have only 2 arrays of prometheus metrics. Their flags are used to match on the entity type. This simplify a bit the metrics definition. For now, labels and descriptions are still outside of these arrays, because the labels must be reworked to be more dynamic and the descrptions must be replaced by stats ones as far as possible.
2021-01-20 14:20:53 +00:00
appctx->st2 = ST_F_PXNAME;
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
appctx->st1 = PROMEX_DUMPER_SRV;
/* fall through */
case PROMEX_DUMPER_SRV:
MINOR: contrib/prometheus-exporter: filter exported metrics by scope Now, the prometheus exporter parses the HTTP query-string to filter or to adapt the exported metrics. In this first version, it is only possible select the scopes of metrics to export. To do so, one or more parameters with "scope" as name must be passed in the query-string, with one of those values: global, frontend, backend, server or '*' (means all). A scope parameter with no value means to filter out all scopes (nothing is returned). The scope parameters are parsed in their appearance order in the query-string. So an empty scope will reset all scopes already parsed. But it can be overridden by following scope parameters in the query-string. By default everything is exported. The filtering can also be done on prometheus scraping configuration, but general aim is to optimise the source of data to improve load and scraping time. This is particularly true for huge configuration with thousands of backends and servers. Also note that this configuration was possible on the previous official haproxy exporter but with even more parameters to select the needed metrics. Here we thought it was sufficient to simply avoid a given type of metric. However, more filters are still possible. Thanks to William Dauchy. This patch is based on his work.
2019-11-18 13:47:08 +00:00
if (appctx->ctx.stats.flags & PROMEX_FL_SCOPE_SERVER) {
ret = promex_dump_srv_metrics(appctx, htx);
if (ret <= 0) {
if (ret == -1)
goto error;
goto full;
}
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
}
MEDIUM: contrib/prometheus-exporter: export base stick table stats I saw some people falling back to unix socket to collect some data they could not find in prometheus exporter. One of them is base info from stick tables (used/size). I do not plan to extend it more for now; keys are quite a mess to handle. This should resolve github issue #1008. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-07 19:42:38 +00:00
appctx->ctx.stats.obj1 = stktables_list;
MINOR: stats: hide px/sv/li fields in applet struct Use an opaque pointer to store proxy instance. Regroup server/listener as a single opaque pointer. This has the benefit to render the structure more evolutive to support statistics on other types of objects in the future. This patch is needed to extend stat support for components other than proxies objects. The prometheus module has been adapted for these changes.
2020-10-02 16:32:02 +00:00
appctx->ctx.stats.obj2 = NULL;
MINOR: contrib/prometheus-exporter: Split the PROMEX_FL_STATS_METRIC flag PROMEX_FL_STATS_METRIC flag is splitted in 3 flags to easily identify the processed entity type (frontend, backend or server). Thus, now we are using PROMEX_FL_FRONT_METRIC, PROMEX_FL_BACK_METRIC or PROMEX_FL_SRV_METRIC. These flags will be used to know if a metric is defined and must be exported for a given entity type.
2021-01-20 14:02:50 +00:00
appctx->ctx.stats.flags &= ~(PROMEX_FL_METRIC_HDR|PROMEX_FL_SRV_METRIC);
MEDIUM: contrib/prometheus-exporter: export base stick table stats I saw some people falling back to unix socket to collect some data they could not find in prometheus exporter. One of them is base info from stick tables (used/size). I do not plan to extend it more for now; keys are quite a mess to handle. This should resolve github issue #1008. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-07 19:42:38 +00:00
appctx->ctx.stats.flags |= (PROMEX_FL_METRIC_HDR|PROMEX_FL_STICKTABLE_METRIC);
appctx->st2 = STICKTABLE_SIZE;
appctx->st1 = PROMEX_DUMPER_STICKTABLE;
/* fall through */
case PROMEX_DUMPER_STICKTABLE:
if (appctx->ctx.stats.flags & PROMEX_FL_SCOPE_STICKTABLE) {
ret = promex_dump_sticktable_metrics(appctx, htx);
if (ret <= 0) {
if (ret == -1)
goto error;
goto full;
}
}
appctx->ctx.stats.obj1 = NULL;
appctx->ctx.stats.obj2 = NULL;
appctx->ctx.stats.flags &= ~(PROMEX_FL_METRIC_HDR|PROMEX_FL_STICKTABLE_METRIC);
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
appctx->st2 = 0;
appctx->st1 = PROMEX_DUMPER_DONE;
/* fall through */
case PROMEX_DUMPER_DONE:
default:
break;
}
return 1;
full:
si_rx_room_blk(si);
return 0;
error:
/* unrecoverable error */
MINOR: stats: hide px/sv/li fields in applet struct Use an opaque pointer to store proxy instance. Regroup server/listener as a single opaque pointer. This has the benefit to render the structure more evolutive to support statistics on other types of objects in the future. This patch is needed to extend stat support for components other than proxies objects. The prometheus module has been adapted for these changes.
2020-10-02 16:32:02 +00:00
appctx->ctx.stats.obj1 = NULL;
appctx->ctx.stats.obj2 = NULL;
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
appctx->ctx.stats.flags = 0;
appctx->st2 = 0;
appctx->st1 = PROMEX_DUMPER_DONE;
return -1;
}
CLEANUP: assorted typo fixes in the code and comments This is fifth iteration of typo fixes
2020-03-23 17:28:40 +00:00
/* Parse the query string of request URI to filter the metrics. It returns 1 on
MINOR: contrib/prometheus-exporter: filter exported metrics by scope Now, the prometheus exporter parses the HTTP query-string to filter or to adapt the exported metrics. In this first version, it is only possible select the scopes of metrics to export. To do so, one or more parameters with "scope" as name must be passed in the query-string, with one of those values: global, frontend, backend, server or '*' (means all). A scope parameter with no value means to filter out all scopes (nothing is returned). The scope parameters are parsed in their appearance order in the query-string. So an empty scope will reset all scopes already parsed. But it can be overridden by following scope parameters in the query-string. By default everything is exported. The filtering can also be done on prometheus scraping configuration, but general aim is to optimise the source of data to improve load and scraping time. This is particularly true for huge configuration with thousands of backends and servers. Also note that this configuration was possible on the previous official haproxy exporter but with even more parameters to select the needed metrics. Here we thought it was sufficient to simply avoid a given type of metric. However, more filters are still possible. Thanks to William Dauchy. This patch is based on his work.
2019-11-18 13:47:08 +00:00
* success and -1 on error. */
static int promex_parse_uri(struct appctx *appctx, struct stream_interface *si)
{
struct channel *req = si_oc(si);
struct channel *res = si_ic(si);
struct htx *req_htx, *res_htx;
struct htx_sl *sl;
BUG/MINOR: contrib/prometheus-exporter: decode parameter and value only we were decoding all substring and then parsing; this could lead to consider & and = in decoding result as delimiters where it should not. this patch reverses the order by first parsing and then decoding each key and value separately. we also stop parsing after number sign (#). This patch should be backported to 2.1 and 2.0 Signed-off-by: William Dauchy <w.dauchy@criteo.com>
2019-11-26 11:56:26 +00:00
char *p, *key, *value;
const char *end;
MINOR: contrib/prometheus-exporter: filter exported metrics by scope Now, the prometheus exporter parses the HTTP query-string to filter or to adapt the exported metrics. In this first version, it is only possible select the scopes of metrics to export. To do so, one or more parameters with "scope" as name must be passed in the query-string, with one of those values: global, frontend, backend, server or '*' (means all). A scope parameter with no value means to filter out all scopes (nothing is returned). The scope parameters are parsed in their appearance order in the query-string. So an empty scope will reset all scopes already parsed. But it can be overridden by following scope parameters in the query-string. By default everything is exported. The filtering can also be done on prometheus scraping configuration, but general aim is to optimise the source of data to improve load and scraping time. This is particularly true for huge configuration with thousands of backends and servers. Also note that this configuration was possible on the previous official haproxy exporter but with even more parameters to select the needed metrics. Here we thought it was sufficient to simply avoid a given type of metric. However, more filters are still possible. Thanks to William Dauchy. This patch is based on his work.
2019-11-18 13:47:08 +00:00
struct buffer *err;
int default_scopes = PROMEX_FL_SCOPE_ALL;
int len;
/* Get the query-string */
req_htx = htxbuf(&req->buf);
sl = http_get_stline(req_htx);
if (!sl)
goto error;
p = http_find_param_list(HTX_SL_REQ_UPTR(sl), HTX_SL_REQ_ULEN(sl), '?');
if (!p)
goto end;
end = HTX_SL_REQ_UPTR(sl) + HTX_SL_REQ_ULEN(sl);
BUG/MINOR: contrib/prometheus-exporter: decode parameter and value only we were decoding all substring and then parsing; this could lead to consider & and = in decoding result as delimiters where it should not. this patch reverses the order by first parsing and then decoding each key and value separately. we also stop parsing after number sign (#). This patch should be backported to 2.1 and 2.0 Signed-off-by: William Dauchy <w.dauchy@criteo.com>
2019-11-26 11:56:26 +00:00
/* copy the query-string */
len = end - p;
MINOR: contrib/prometheus-exporter: filter exported metrics by scope Now, the prometheus exporter parses the HTTP query-string to filter or to adapt the exported metrics. In this first version, it is only possible select the scopes of metrics to export. To do so, one or more parameters with "scope" as name must be passed in the query-string, with one of those values: global, frontend, backend, server or '*' (means all). A scope parameter with no value means to filter out all scopes (nothing is returned). The scope parameters are parsed in their appearance order in the query-string. So an empty scope will reset all scopes already parsed. But it can be overridden by following scope parameters in the query-string. By default everything is exported. The filtering can also be done on prometheus scraping configuration, but general aim is to optimise the source of data to improve load and scraping time. This is particularly true for huge configuration with thousands of backends and servers. Also note that this configuration was possible on the previous official haproxy exporter but with even more parameters to select the needed metrics. Here we thought it was sufficient to simply avoid a given type of metric. However, more filters are still possible. Thanks to William Dauchy. This patch is based on his work.
2019-11-18 13:47:08 +00:00
chunk_reset(&trash);
memcpy(trash.area, p, len);
trash.area[len] = 0;
p = trash.area;
BUG/MINOR: contrib/prometheus-exporter: decode parameter and value only we were decoding all substring and then parsing; this could lead to consider & and = in decoding result as delimiters where it should not. this patch reverses the order by first parsing and then decoding each key and value separately. we also stop parsing after number sign (#). This patch should be backported to 2.1 and 2.0 Signed-off-by: William Dauchy <w.dauchy@criteo.com>
2019-11-26 11:56:26 +00:00
end = trash.area + len;
MINOR: contrib/prometheus-exporter: filter exported metrics by scope Now, the prometheus exporter parses the HTTP query-string to filter or to adapt the exported metrics. In this first version, it is only possible select the scopes of metrics to export. To do so, one or more parameters with "scope" as name must be passed in the query-string, with one of those values: global, frontend, backend, server or '*' (means all). A scope parameter with no value means to filter out all scopes (nothing is returned). The scope parameters are parsed in their appearance order in the query-string. So an empty scope will reset all scopes already parsed. But it can be overridden by following scope parameters in the query-string. By default everything is exported. The filtering can also be done on prometheus scraping configuration, but general aim is to optimise the source of data to improve load and scraping time. This is particularly true for huge configuration with thousands of backends and servers. Also note that this configuration was possible on the previous official haproxy exporter but with even more parameters to select the needed metrics. Here we thought it was sufficient to simply avoid a given type of metric. However, more filters are still possible. Thanks to William Dauchy. This patch is based on his work.
2019-11-18 13:47:08 +00:00
/* Parse the query-string */
BUG/MINOR: contrib/prometheus-exporter: decode parameter and value only we were decoding all substring and then parsing; this could lead to consider & and = in decoding result as delimiters where it should not. this patch reverses the order by first parsing and then decoding each key and value separately. we also stop parsing after number sign (#). This patch should be backported to 2.1 and 2.0 Signed-off-by: William Dauchy <w.dauchy@criteo.com>
2019-11-26 11:56:26 +00:00
while (p < end && *p && *p != '#') {
value = NULL;
/* decode parameter name */
key = p;
while (p < end && *p != '=' && *p != '&' && *p != '#')
MINOR: contrib/prometheus-exporter: filter exported metrics by scope Now, the prometheus exporter parses the HTTP query-string to filter or to adapt the exported metrics. In this first version, it is only possible select the scopes of metrics to export. To do so, one or more parameters with "scope" as name must be passed in the query-string, with one of those values: global, frontend, backend, server or '*' (means all). A scope parameter with no value means to filter out all scopes (nothing is returned). The scope parameters are parsed in their appearance order in the query-string. So an empty scope will reset all scopes already parsed. But it can be overridden by following scope parameters in the query-string. By default everything is exported. The filtering can also be done on prometheus scraping configuration, but general aim is to optimise the source of data to improve load and scraping time. This is particularly true for huge configuration with thousands of backends and servers. Also note that this configuration was possible on the previous official haproxy exporter but with even more parameters to select the needed metrics. Here we thought it was sufficient to simply avoid a given type of metric. However, more filters are still possible. Thanks to William Dauchy. This patch is based on his work.
2019-11-18 13:47:08 +00:00
++p;
BUG/MINOR: contrib/prometheus-exporter: decode parameter and value only we were decoding all substring and then parsing; this could lead to consider & and = in decoding result as delimiters where it should not. this patch reverses the order by first parsing and then decoding each key and value separately. we also stop parsing after number sign (#). This patch should be backported to 2.1 and 2.0 Signed-off-by: William Dauchy <w.dauchy@criteo.com>
2019-11-26 11:56:26 +00:00
/* found a value */
if (*p == '=') {
*(p++) = 0;
value = p;
}
else if (*p == '&')
*(p++) = 0;
else if (*p == '#')
*p = 0;
BUG/MINOR: http: make url_decode() optionally convert '+' to SP The url_decode() function used by the url_dec converter and a few other call points is ambiguous on its processing of the '+' character which itself isn't stable in the spec. This one belongs to the reserved characters for the query string but not for the path nor the scheme, in which it must be left as-is. It's only in argument strings that follow the application/x-www-form-urlencoded encoding that it must be turned into a space, that is, in query strings and POST arguments. The problem is that the function is used to process full URLs and paths in various configs, and to process query strings from the stats page for example. This patch updates the function to differentiate the situation where it's parsing a path and a query string. A new argument indicates if a query string should be assumed, otherwise it's only assumed after seeing a question mark. The various locations in the code making use of this function were updated to take care of this (most call places were using it to decode POST arguments). The url_dec converter is usually called on path or url samples, so it needs to remain compatible with this and will default to parsing a path and turning the '+' to a space only after a question mark. However in situations where it would explicitly be extracted from a POST or a query string, it now becomes possible to enforce the decoding by passing a non-null value in argument. It seems to be what was reported in issue #585. This fix may be backported to older stable releases.
2020-04-23 15:54:47 +00:00
len = url_decode(key, 1);
BUG/MINOR: contrib/prometheus-exporter: decode parameter and value only we were decoding all substring and then parsing; this could lead to consider & and = in decoding result as delimiters where it should not. this patch reverses the order by first parsing and then decoding each key and value separately. we also stop parsing after number sign (#). This patch should be backported to 2.1 and 2.0 Signed-off-by: William Dauchy <w.dauchy@criteo.com>
2019-11-26 11:56:26 +00:00
if (len == -1)
goto error;
/* decode value */
if (value) {
while (p < end && *p != '=' && *p != '&' && *p != '#')
++p;
if (*p == '=')
goto error;
if (*p == '&')
*(p++) = 0;
else if (*p == '#')
*p = 0;
BUG/MINOR: http: make url_decode() optionally convert '+' to SP The url_decode() function used by the url_dec converter and a few other call points is ambiguous on its processing of the '+' character which itself isn't stable in the spec. This one belongs to the reserved characters for the query string but not for the path nor the scheme, in which it must be left as-is. It's only in argument strings that follow the application/x-www-form-urlencoded encoding that it must be turned into a space, that is, in query strings and POST arguments. The problem is that the function is used to process full URLs and paths in various configs, and to process query strings from the stats page for example. This patch updates the function to differentiate the situation where it's parsing a path and a query string. A new argument indicates if a query string should be assumed, otherwise it's only assumed after seeing a question mark. The various locations in the code making use of this function were updated to take care of this (most call places were using it to decode POST arguments). The url_dec converter is usually called on path or url samples, so it needs to remain compatible with this and will default to parsing a path and turning the '+' to a space only after a question mark. However in situations where it would explicitly be extracted from a POST or a query string, it now becomes possible to enforce the decoding by passing a non-null value in argument. It seems to be what was reported in issue #585. This fix may be backported to older stable releases.
2020-04-23 15:54:47 +00:00
len = url_decode(value, 1);
BUG/MINOR: contrib/prometheus-exporter: decode parameter and value only we were decoding all substring and then parsing; this could lead to consider & and = in decoding result as delimiters where it should not. this patch reverses the order by first parsing and then decoding each key and value separately. we also stop parsing after number sign (#). This patch should be backported to 2.1 and 2.0 Signed-off-by: William Dauchy <w.dauchy@criteo.com>
2019-11-26 11:56:26 +00:00
if (len == -1)
goto error;
}
MINOR: contrib/prometheus-exporter: filter exported metrics by scope Now, the prometheus exporter parses the HTTP query-string to filter or to adapt the exported metrics. In this first version, it is only possible select the scopes of metrics to export. To do so, one or more parameters with "scope" as name must be passed in the query-string, with one of those values: global, frontend, backend, server or '*' (means all). A scope parameter with no value means to filter out all scopes (nothing is returned). The scope parameters are parsed in their appearance order in the query-string. So an empty scope will reset all scopes already parsed. But it can be overridden by following scope parameters in the query-string. By default everything is exported. The filtering can also be done on prometheus scraping configuration, but general aim is to optimise the source of data to improve load and scraping time. This is particularly true for huge configuration with thousands of backends and servers. Also note that this configuration was possible on the previous official haproxy exporter but with even more parameters to select the needed metrics. Here we thought it was sufficient to simply avoid a given type of metric. However, more filters are still possible. Thanks to William Dauchy. This patch is based on his work.
2019-11-18 13:47:08 +00:00
CLEANUP: Apply the coccinelle patch for `XXXcmp()` on contrib/ Compare the various `cmp()` functions against zero.
2021-01-02 21:31:55 +00:00
if (strcmp(key, "scope") == 0) {
BUG/MINOR: contrib/prometheus-exporter: decode parameter and value only we were decoding all substring and then parsing; this could lead to consider & and = in decoding result as delimiters where it should not. this patch reverses the order by first parsing and then decoding each key and value separately. we also stop parsing after number sign (#). This patch should be backported to 2.1 and 2.0 Signed-off-by: William Dauchy <w.dauchy@criteo.com>
2019-11-26 11:56:26 +00:00
default_scopes = 0; /* at least a scope defined, unset default scopes */
if (!value)
goto error;
else if (*value == 0)
MINOR: contrib/prometheus-exporter: filter exported metrics by scope Now, the prometheus exporter parses the HTTP query-string to filter or to adapt the exported metrics. In this first version, it is only possible select the scopes of metrics to export. To do so, one or more parameters with "scope" as name must be passed in the query-string, with one of those values: global, frontend, backend, server or '*' (means all). A scope parameter with no value means to filter out all scopes (nothing is returned). The scope parameters are parsed in their appearance order in the query-string. So an empty scope will reset all scopes already parsed. But it can be overridden by following scope parameters in the query-string. By default everything is exported. The filtering can also be done on prometheus scraping configuration, but general aim is to optimise the source of data to improve load and scraping time. This is particularly true for huge configuration with thousands of backends and servers. Also note that this configuration was possible on the previous official haproxy exporter but with even more parameters to select the needed metrics. Here we thought it was sufficient to simply avoid a given type of metric. However, more filters are still possible. Thanks to William Dauchy. This patch is based on his work.
2019-11-18 13:47:08 +00:00
appctx->ctx.stats.flags &= ~PROMEX_FL_SCOPE_ALL;
BUG/MINOR: contrib/prometheus-exporter: decode parameter and value only we were decoding all substring and then parsing; this could lead to consider & and = in decoding result as delimiters where it should not. this patch reverses the order by first parsing and then decoding each key and value separately. we also stop parsing after number sign (#). This patch should be backported to 2.1 and 2.0 Signed-off-by: William Dauchy <w.dauchy@criteo.com>
2019-11-26 11:56:26 +00:00
else if (*value == '*')
MINOR: contrib/prometheus-exporter: filter exported metrics by scope Now, the prometheus exporter parses the HTTP query-string to filter or to adapt the exported metrics. In this first version, it is only possible select the scopes of metrics to export. To do so, one or more parameters with "scope" as name must be passed in the query-string, with one of those values: global, frontend, backend, server or '*' (means all). A scope parameter with no value means to filter out all scopes (nothing is returned). The scope parameters are parsed in their appearance order in the query-string. So an empty scope will reset all scopes already parsed. But it can be overridden by following scope parameters in the query-string. By default everything is exported. The filtering can also be done on prometheus scraping configuration, but general aim is to optimise the source of data to improve load and scraping time. This is particularly true for huge configuration with thousands of backends and servers. Also note that this configuration was possible on the previous official haproxy exporter but with even more parameters to select the needed metrics. Here we thought it was sufficient to simply avoid a given type of metric. However, more filters are still possible. Thanks to William Dauchy. This patch is based on his work.
2019-11-18 13:47:08 +00:00
appctx->ctx.stats.flags |= PROMEX_FL_SCOPE_ALL;
CLEANUP: Apply the coccinelle patch for `XXXcmp()` on contrib/ Compare the various `cmp()` functions against zero.
2021-01-02 21:31:55 +00:00
else if (strcmp(value, "global") == 0)
BUG/MINOR: contrib/prometheus-exporter: decode parameter and value only we were decoding all substring and then parsing; this could lead to consider & and = in decoding result as delimiters where it should not. this patch reverses the order by first parsing and then decoding each key and value separately. we also stop parsing after number sign (#). This patch should be backported to 2.1 and 2.0 Signed-off-by: William Dauchy <w.dauchy@criteo.com>
2019-11-26 11:56:26 +00:00
appctx->ctx.stats.flags |= PROMEX_FL_SCOPE_GLOBAL;
CLEANUP: Apply the coccinelle patch for `XXXcmp()` on contrib/ Compare the various `cmp()` functions against zero.
2021-01-02 21:31:55 +00:00
else if (strcmp(value, "server") == 0)
BUG/MINOR: contrib/prometheus-exporter: decode parameter and value only we were decoding all substring and then parsing; this could lead to consider & and = in decoding result as delimiters where it should not. this patch reverses the order by first parsing and then decoding each key and value separately. we also stop parsing after number sign (#). This patch should be backported to 2.1 and 2.0 Signed-off-by: William Dauchy <w.dauchy@criteo.com>
2019-11-26 11:56:26 +00:00
appctx->ctx.stats.flags |= PROMEX_FL_SCOPE_SERVER;
CLEANUP: Apply the coccinelle patch for `XXXcmp()` on contrib/ Compare the various `cmp()` functions against zero.
2021-01-02 21:31:55 +00:00
else if (strcmp(value, "backend") == 0)
MINOR: contrib/prometheus-exporter: filter exported metrics by scope Now, the prometheus exporter parses the HTTP query-string to filter or to adapt the exported metrics. In this first version, it is only possible select the scopes of metrics to export. To do so, one or more parameters with "scope" as name must be passed in the query-string, with one of those values: global, frontend, backend, server or '*' (means all). A scope parameter with no value means to filter out all scopes (nothing is returned). The scope parameters are parsed in their appearance order in the query-string. So an empty scope will reset all scopes already parsed. But it can be overridden by following scope parameters in the query-string. By default everything is exported. The filtering can also be done on prometheus scraping configuration, but general aim is to optimise the source of data to improve load and scraping time. This is particularly true for huge configuration with thousands of backends and servers. Also note that this configuration was possible on the previous official haproxy exporter but with even more parameters to select the needed metrics. Here we thought it was sufficient to simply avoid a given type of metric. However, more filters are still possible. Thanks to William Dauchy. This patch is based on his work.
2019-11-18 13:47:08 +00:00
appctx->ctx.stats.flags |= PROMEX_FL_SCOPE_BACK;
CLEANUP: Apply the coccinelle patch for `XXXcmp()` on contrib/ Compare the various `cmp()` functions against zero.
2021-01-02 21:31:55 +00:00
else if (strcmp(value, "frontend") == 0)
MINOR: contrib/prometheus-exporter: filter exported metrics by scope Now, the prometheus exporter parses the HTTP query-string to filter or to adapt the exported metrics. In this first version, it is only possible select the scopes of metrics to export. To do so, one or more parameters with "scope" as name must be passed in the query-string, with one of those values: global, frontend, backend, server or '*' (means all). A scope parameter with no value means to filter out all scopes (nothing is returned). The scope parameters are parsed in their appearance order in the query-string. So an empty scope will reset all scopes already parsed. But it can be overridden by following scope parameters in the query-string. By default everything is exported. The filtering can also be done on prometheus scraping configuration, but general aim is to optimise the source of data to improve load and scraping time. This is particularly true for huge configuration with thousands of backends and servers. Also note that this configuration was possible on the previous official haproxy exporter but with even more parameters to select the needed metrics. Here we thought it was sufficient to simply avoid a given type of metric. However, more filters are still possible. Thanks to William Dauchy. This patch is based on his work.
2019-11-18 13:47:08 +00:00
appctx->ctx.stats.flags |= PROMEX_FL_SCOPE_FRONT;
MEDIUM: contrib/prometheus-exporter: add listen stats this was a missing piece for a while now even though it was planned. This patch adds listen stats. Nothing in particular but we make use of the status helper previously added. `promex_st_metrics` diff also looks scary, but I had to realign all lines. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-14 22:22:56 +00:00
else if (strcmp(value, "listener") == 0)
appctx->ctx.stats.flags |= PROMEX_FL_SCOPE_LI;
MEDIUM: contrib/prometheus-exporter: export base stick table stats I saw some people falling back to unix socket to collect some data they could not find in prometheus exporter. One of them is base info from stick tables (used/size). I do not plan to extend it more for now; keys are quite a mess to handle. This should resolve github issue #1008. Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-07 19:42:38 +00:00
else if (strcmp(value, "sticktable") == 0)
appctx->ctx.stats.flags |= PROMEX_FL_SCOPE_STICKTABLE;
MINOR: contrib/prometheus-exporter: filter exported metrics by scope Now, the prometheus exporter parses the HTTP query-string to filter or to adapt the exported metrics. In this first version, it is only possible select the scopes of metrics to export. To do so, one or more parameters with "scope" as name must be passed in the query-string, with one of those values: global, frontend, backend, server or '*' (means all). A scope parameter with no value means to filter out all scopes (nothing is returned). The scope parameters are parsed in their appearance order in the query-string. So an empty scope will reset all scopes already parsed. But it can be overridden by following scope parameters in the query-string. By default everything is exported. The filtering can also be done on prometheus scraping configuration, but general aim is to optimise the source of data to improve load and scraping time. This is particularly true for huge configuration with thousands of backends and servers. Also note that this configuration was possible on the previous official haproxy exporter but with even more parameters to select the needed metrics. Here we thought it was sufficient to simply avoid a given type of metric. However, more filters are still possible. Thanks to William Dauchy. This patch is based on his work.
2019-11-18 13:47:08 +00:00
else
goto error;
}
CLEANUP: Apply the coccinelle patch for `XXXcmp()` on contrib/ Compare the various `cmp()` functions against zero.
2021-01-02 21:31:55 +00:00
else if (strcmp(key, "no-maint") == 0)
MINOR: contrib/prometheus-exporter: Add a param to ignore servers in maintenance By passing the parameter "no-maint" in the query-string, it is now possible to ignore servers in maintenance. It means that the metrics for servers in this state will not be exported.
2019-11-19 13:18:24 +00:00
appctx->ctx.stats.flags |= PROMEX_FL_NO_MAINT_SRV;
MINOR: contrib/prometheus-exporter: filter exported metrics by scope Now, the prometheus exporter parses the HTTP query-string to filter or to adapt the exported metrics. In this first version, it is only possible select the scopes of metrics to export. To do so, one or more parameters with "scope" as name must be passed in the query-string, with one of those values: global, frontend, backend, server or '*' (means all). A scope parameter with no value means to filter out all scopes (nothing is returned). The scope parameters are parsed in their appearance order in the query-string. So an empty scope will reset all scopes already parsed. But it can be overridden by following scope parameters in the query-string. By default everything is exported. The filtering can also be done on prometheus scraping configuration, but general aim is to optimise the source of data to improve load and scraping time. This is particularly true for huge configuration with thousands of backends and servers. Also note that this configuration was possible on the previous official haproxy exporter but with even more parameters to select the needed metrics. Here we thought it was sufficient to simply avoid a given type of metric. However, more filters are still possible. Thanks to William Dauchy. This patch is based on his work.
2019-11-18 13:47:08 +00:00
}
end:
appctx->ctx.stats.flags |= default_scopes;
return 1;
error:
err = &http_err_chunks[HTTP_ERR_400];
channel_erase(res);
res->buf.data = b_data(err);
memcpy(res->buf.area, b_head(err), b_data(err));
res_htx = htx_from_buf(&res->buf);
channel_add_input(res, res_htx->data);
appctx->st0 = PROMEX_ST_END;
return -1;
}
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
/* Send HTTP headers of the response. It returns 1 on success and 0 if <htx> is
* full. */
static int promex_send_headers(struct appctx *appctx, struct stream_interface *si, struct htx *htx)
{
BUG/MINOR: contrib/prometheus-exporter: Fix applet accordingly to recent changes Since the flag EOI was added on channels, some hidden bugs in the prometheus exporter now leads to error. the visible effect is that responses are truncated. So first of all, channel_add_input() must be called when the response headers and the EOM block are added. To be sure to correctly update the response channel (especially to_forward value). Then the request must really be fully consumed. And finally, the return clause in the switch has been replaced by a break. It was totally wrong to skip the end of the function in the states PROMEX_DONE and PROMEX_ERROR. (Note that PROMEX_ERROR was never used, so it was replaced by PROMEX_END just to ease reading the code). No need to backport this patch, the Prometheus exporter does not exist in early versions.
2019-03-27 14:48:53 +00:00
struct channel *chn = si_ic(appctx->owner);
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
struct htx_sl *sl;
unsigned int flags;
flags = (HTX_SL_F_IS_RESP|HTX_SL_F_VER_11|HTX_SL_F_XFER_ENC|HTX_SL_F_XFER_LEN|HTX_SL_F_CHNK);
sl = htx_add_stline(htx, HTX_BLK_RES_SL, flags, ist("HTTP/1.1"), ist("200"), ist("OK"));
if (!sl)
goto full;
sl->info.res.status = 200;
if (!htx_add_header(htx, ist("Cache-Control"), ist("no-cache")) ||
!htx_add_header(htx, ist("Content-Type"), ist("text/plain; version=0.0.4")) ||
!htx_add_header(htx, ist("Transfer-Encoding"), ist("chunked")) ||
!htx_add_endof(htx, HTX_BLK_EOH))
goto full;
BUG/MINOR: contrib/prometheus-exporter: Fix applet accordingly to recent changes Since the flag EOI was added on channels, some hidden bugs in the prometheus exporter now leads to error. the visible effect is that responses are truncated. So first of all, channel_add_input() must be called when the response headers and the EOM block are added. To be sure to correctly update the response channel (especially to_forward value). Then the request must really be fully consumed. And finally, the return clause in the switch has been replaced by a break. It was totally wrong to skip the end of the function in the states PROMEX_DONE and PROMEX_ERROR. (Note that PROMEX_ERROR was never used, so it was replaced by PROMEX_END just to ease reading the code). No need to backport this patch, the Prometheus exporter does not exist in early versions.
2019-03-27 14:48:53 +00:00
channel_add_input(chn, htx->data);
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
return 1;
full:
htx_reset(htx);
si_rx_room_blk(si);
return 0;
}
/* The function returns 1 if the initialisation is complete, 0 if
* an errors occurs and -1 if more data are required for initializing
* the applet.
*/
static int promex_appctx_init(struct appctx *appctx, struct proxy *px, struct stream *strm)
{
appctx->st0 = PROMEX_ST_INIT;
return 1;
}
/* The main I/O handler for the promex applet. */
static void promex_appctx_handle_io(struct appctx *appctx)
{
struct stream_interface *si = appctx->owner;
struct stream *s = si_strm(si);
struct channel *req = si_oc(si);
struct channel *res = si_ic(si);
struct htx *req_htx, *res_htx;
int ret;
res_htx = htx_from_buf(&res->buf);
if (unlikely(si->state == SI_ST_DIS || si->state == SI_ST_CLO))
goto out;
CLEANUP: assorted typo fixes in the code and comments This is fifth iteration of typo fixes
2020-03-23 17:28:40 +00:00
/* Check if the input buffer is available. */
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
if (!b_size(&res->buf)) {
si_rx_room_blk(si);
goto out;
}
switch (appctx->st0) {
case PROMEX_ST_INIT:
MINOR: contrib/prometheus-exporter: filter exported metrics by scope Now, the prometheus exporter parses the HTTP query-string to filter or to adapt the exported metrics. In this first version, it is only possible select the scopes of metrics to export. To do so, one or more parameters with "scope" as name must be passed in the query-string, with one of those values: global, frontend, backend, server or '*' (means all). A scope parameter with no value means to filter out all scopes (nothing is returned). The scope parameters are parsed in their appearance order in the query-string. So an empty scope will reset all scopes already parsed. But it can be overridden by following scope parameters in the query-string. By default everything is exported. The filtering can also be done on prometheus scraping configuration, but general aim is to optimise the source of data to improve load and scraping time. This is particularly true for huge configuration with thousands of backends and servers. Also note that this configuration was possible on the previous official haproxy exporter but with even more parameters to select the needed metrics. Here we thought it was sufficient to simply avoid a given type of metric. However, more filters are still possible. Thanks to William Dauchy. This patch is based on his work.
2019-11-18 13:47:08 +00:00
ret = promex_parse_uri(appctx, si);
if (ret <= 0) {
if (ret == -1)
goto error;
goto out;
}
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
appctx->st0 = PROMEX_ST_HEAD;
appctx->st1 = PROMEX_DUMPER_INIT;
/* fall through */
case PROMEX_ST_HEAD:
if (!promex_send_headers(appctx, si, res_htx))
goto out;
appctx->st0 = ((s->txn->meth == HTTP_METH_HEAD) ? PROMEX_ST_DONE : PROMEX_ST_DUMP);
/* fall through */
case PROMEX_ST_DUMP:
ret = promex_dump_metrics(appctx, si, res_htx);
if (ret <= 0) {
if (ret == -1)
goto error;
goto out;
}
appctx->st0 = PROMEX_ST_DONE;
/* fall through */
case PROMEX_ST_DONE:
MAJOR: htx: Remove the EOM block type and use HTX_FL_EOM instead The EOM block may be removed. The HTX_FL_EOM flags is enough. Most of time, to know if the end of the message is reached, we just need to have an empty HTX message with HTX_FL_EOM flag set. It may also be detected when the last block of a message with HTX_FL_EOM flag is manipulated. Removing EOM blocks simplifies the HTX message filling. Indeed, there is no more edge problems when the message ends but there is no more space to write the EOM block. However, some part are more tricky. Especially the compression filter or the FCGI mux. The compression filter must finish the compression on the last DATA block. Before it was performed on the EOM block, an extra DATA block with the checksum was added. Now, we must detect the last DATA block to be sure to finish the compression. The FCGI mux on its part must be sure to reserve the space for the empty STDIN record on the last DATA block while this record was inserted on the EOM block. The H2 multiplexer is probably the part that benefits the most from this change. Indeed, it is now fairly easier to known when to set the ES flag. The HTX documentaion has been updated accordingly.
2020-12-02 18:12:22 +00:00
/* no more data are expected. Don't add TLR because mux-h1 will take care of it */
res_htx->flags |= HTX_FL_EOM;
BUG/MINOR: contrib/prometheus-exporter: Fix applet accordingly to recent changes Since the flag EOI was added on channels, some hidden bugs in the prometheus exporter now leads to error. the visible effect is that responses are truncated. So first of all, channel_add_input() must be called when the response headers and the EOM block are added. To be sure to correctly update the response channel (especially to_forward value). Then the request must really be fully consumed. And finally, the return clause in the switch has been replaced by a break. It was totally wrong to skip the end of the function in the states PROMEX_DONE and PROMEX_ERROR. (Note that PROMEX_ERROR was never used, so it was replaced by PROMEX_END just to ease reading the code). No need to backport this patch, the Prometheus exporter does not exist in early versions.
2019-03-27 14:48:53 +00:00
appctx->st0 = PROMEX_ST_END;
/* fall through */
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
BUG/MINOR: contrib/prometheus-exporter: Fix applet accordingly to recent changes Since the flag EOI was added on channels, some hidden bugs in the prometheus exporter now leads to error. the visible effect is that responses are truncated. So first of all, channel_add_input() must be called when the response headers and the EOM block are added. To be sure to correctly update the response channel (especially to_forward value). Then the request must really be fully consumed. And finally, the return clause in the switch has been replaced by a break. It was totally wrong to skip the end of the function in the states PROMEX_DONE and PROMEX_ERROR. (Note that PROMEX_ERROR was never used, so it was replaced by PROMEX_END just to ease reading the code). No need to backport this patch, the Prometheus exporter does not exist in early versions.
2019-03-27 14:48:53 +00:00
case PROMEX_ST_END:
if (!(res->flags & CF_SHUTR)) {
res->flags |= CF_READ_NULL;
si_shutr(si);
}
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
}
out:
htx_to_buf(res_htx, &res->buf);
BUG/MINOR: contrib/prometheus-exporter: Fix applet accordingly to recent changes Since the flag EOI was added on channels, some hidden bugs in the prometheus exporter now leads to error. the visible effect is that responses are truncated. So first of all, channel_add_input() must be called when the response headers and the EOM block are added. To be sure to correctly update the response channel (especially to_forward value). Then the request must really be fully consumed. And finally, the return clause in the switch has been replaced by a break. It was totally wrong to skip the end of the function in the states PROMEX_DONE and PROMEX_ERROR. (Note that PROMEX_ERROR was never used, so it was replaced by PROMEX_END just to ease reading the code). No need to backport this patch, the Prometheus exporter does not exist in early versions.
2019-03-27 14:48:53 +00:00
/* eat the whole request */
if (co_data(req)) {
req_htx = htx_from_buf(&req->buf);
co_htx_skip(req, req_htx, co_data(req));
}
CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy It has been developped as a service applet. Internally, it is called "promex". To build HAProxy with the promex service, you should use the Makefile variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with an "http-request" rule and the corresponding HTTP proxy must enable the HTX support. For instance: frontend test mode http ... option http-use-htx http-request use-service prometheus-exporter if { path /metrics } ... See contrib/prometheus-exporter/README for details.
2019-02-07 14:38:42 +00:00
return;
error:
res->flags |= CF_READ_NULL;
si_shutr(si);
si_shutw(si);
}
struct applet promex_applet = {
.obj_type = OBJ_TYPE_APPLET,
.name = "<PROMEX>", /* used for logging */
.init = promex_appctx_init,
.fct = promex_appctx_handle_io,
};
static enum act_parse_ret service_parse_prometheus_exporter(const char **args, int *cur_arg, struct proxy *px,
struct act_rule *rule, char **err)
{
/* Prometheus exporter service is only available on "http-request" rulesets */
if (rule->from != ACT_F_HTTP_REQ) {
memprintf(err, "Prometheus exporter service only available on 'http-request' rulesets");
return ACT_RET_PRS_ERR;
}
/* Add applet pointer in the rule. */
rule->applet = promex_applet;
return ACT_RET_PRS_OK;
}
static void promex_register_build_options(void)
{
char *ptr = NULL;
memprintf(&ptr, "Built with the Prometheus exporter as a service");
hap_register_build_opts(ptr, 1);
}
static struct action_kw_list service_actions = { ILH, {
{ "prometheus-exporter", service_parse_prometheus_exporter },
{ /* END */ }
}};
INITCALL1(STG_REGISTER, service_keywords_register, &service_actions);
INITCALL0(STG_REGISTER, promex_register_build_options);
Reference in New Issue Copy Permalink