1
0
mirror of http://git.haproxy.org/git/haproxy.git/ synced 2024-12-19 18:28:33 +00:00
Commit Graph

340 Commits

Author SHA1 Message Date
Frédéric Lécaille
aab6f7c3e6 MINOR: contrib: Make the wireshark peers dissector compile for more distribs.
With a 2.6.8 wireshark, this module could not compile because of ws_version.h
missing header. This patch offers the possibility to compile this plugin without
having to include this header. Furthermore with my wireshark version a
"plugin_release" object is required to make it be loaded by wireshark. This is
a string which seems to have to match a dotted string made of you wireshark
major and minor versions.
2021-01-19 15:22:35 +01:00
William Dauchy
5d9b8f3c93 MINOR: contrib/prometheus-exporter: use fill_info for process dump
use `stats_fill_info` when possible to avoid duplicating code.

Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-13 15:19:00 +01:00
William Dauchy
1704efee89 MINOR: contrib/prometheus-exporter: avoid connection close header
it does not seem to have a reason to close connections after each
request; reflect that in tests by doing all requests within the same
client.

Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-13 15:11:38 +01:00
William Dauchy
5a982a7165 MINOR: contrib/prometheus-exporter: export build_info
commit c55a626217 ("MINOR: contrib/prometheus-exporter: Add
missing global and per-server metrics") is renaming two metrics between
v2.2 and v2.3:
  server_idle_connections_current
  server_idle_connections_limit

It is breaking some tools which are making use of those metrics while
supporting several haproxy versions. This build_info will permit tools
which make use of metrics to be able to match the haproxy version and
change the list of expected metrics. This was possible using the haproxy
stats socket but not with prometheus export.

This patch follows prometheus best pratices to export specific software
informations. It is adding a new field `build_info` so we can extend it
to other parameters if needed in the future.

example output:
  # HELP haproxy_process_build_info HAProxy build info.
  # TYPE haproxy_process_build_info gauge
  haproxy_process_build_info{version="2.4-dev5-2e1a3f-5"} 1

Even though it is not a bugfix, this patch will make more sense when
backported up to >= 2.0

Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-08 14:48:13 +01:00
Tim Duesterhus
8cb12a8292 CLEANUP: Apply the coccinelle patch for XXXcmp() on contrib/
Compare the various `cmp()` functions against zero.
2021-01-04 10:09:02 +01:00
Ilya Shipitsin
f38a01884a CLEANUP: assorted typo fixes in the code and comments
This is 13n iteration of typo fixes
2020-12-21 11:24:48 +01:00
Willy Tarreau
79d2e7bbbd CONTRIB: debug: address "poll" utility build on non-linux platforms
MSG_NOSIGNAL and MSG_MORE are not defined everywhere, let's make them
zero when not defined. It will roughly result in the same behavior,
albeit a bit less optimal, which is no big deal when debugging. This
should fix issue .
2020-12-21 08:45:26 +01:00
Willy Tarreau
03ca6054d0 CONTRIB: halog: fix signed/unsigned build warnings on counts and timestamps
Some variables were signed while they were compared to unsigned ones,
causing warnings to be issued when -Wextra is enabled.
2020-12-21 08:43:09 +01:00
Willy Tarreau
f531dfff18 CONTRIB: halog: mark the has_zero* functions unused
These ones will depend on the use of memchr() or not, let's mark them unused
to avoid the warning reported in issue .
2020-12-21 08:43:09 +01:00
Willy Tarreau
2df860cb13 CONTRIB: halog: fix build issue caused by %L printf format
%Ld isn't standard, %lld is more portable. In addition, the format
should be %llu since the printed values are unsigned. This should
address issue .
2020-12-21 08:43:09 +01:00
Miroslav Zagorac
70230c6484 CONTRIB: opentracing: add the OpenTracing filter
This commit adds the OpenTracing filter (hereinafter we will use the
abbreviated name 'the OT filter') to the contrib tree.

The OT filter adds native support for using distributed tracing in HAProxy.
This is enabled by sending an OpenTracing compliant request to one of the
supported tracers; such as Datadog, Jaeger, Lightstep and Zipkin tracers.
Please note: tracers are not listed by any preference, but alphabetically.

The OT filter is a standard HAProxy filter, so what applies to others also
applies to this one (of course, by that I mean what is described in the
documentation, more precisely in the doc/internals/filters.txt file).

The OT filter activation is done explicitly by specifying it in the HAProxy
configuration.  If this is not done, the OT filter in no way participates
in the work of HAProxy.

As for the impact on HAProxy speed, this is documented with several tests
located in the test directory, and the result is found in the README-speed-*
files.  In short, the speed of operation depends on the way it is used and
the complexity of the configuration, from an almost immeasurable impact to
a significant deceleration (5x and more).  I think that in some normal use
the speed of HAProxy with the filter on will be quite satisfactory with a
slowdown of less than 4%.

The OT filter allows intensive use of ACLs, which can be defined anywhere in
the configuration.  Thus, it is possible to use the filter only for those
connections that are of interest to us.

More detailed documentation related to the operation, configuration and use
of the filter can be found in the contrib/opentracing directory.

To make the OpenTracing filter easier to configure and compile, several
entries have been added to the Makefile.  When running the make utility,
it is possible to use several new arguments:

  USE_OT=1     : enable the OpenTracing filter
  OT_DEBUG=1   : compile the OpenTracing filter in debug mode
  OT_INC=path  : force the include path to libopentracing-c-wrapper
  OT_LIB=path  : force the lib path to libopentracing-c-wrapper
  OT_RUNPATH=1 : add libopentracing-c-wrapper RUNPATH to haproxy executable

If USE_OT is set, then an additional Makefile from the contrib/opentracing
directory is included in the compilation process.
2020-12-16 15:49:53 +01:00
Gilchrist Dadaglo
bdd034947c BUG/MEDIUM: spoa/python: Fixing references to None
As per https://docs.python.org/3/c-api/none.html, None has to be treated
exactly like other objects for reference counting.
So, when we use it, we need to INCREF and when we are done, DECREF

This patch must be backported as far as 2.0.
2020-12-09 08:39:31 +01:00
Gilchrist Dadaglo
042f697a0f BUG/MEDIUM: spoa/python: Fixing PyObject_Call positional arguments
As per https://docs.python.org/3/c-api/object.html#c.PyObject_Call,
positional arguments should be an empty tuple when not used.
Previously the code had a dictionary instead of tuple. This commit is to
fix it and use tuple to avoid unexpected consequences

This patch must be backported as far as 2.0.
2020-12-09 08:39:31 +01:00
Gilchrist Dadaglo
d1c0cf80bc BUG/MINOR: spoa/python: Cleanup ipaddress objects if initialization fails
This change is to ensure objects from the ipaddress module are cleaned
up when spoa module initialization fails.
In general the interpreter would just crash, but in a code where import
is conditional (try/except), then we would keep those objects around

This patch must be backported as far as 2.0.
2020-12-09 08:39:31 +01:00
Gilchrist Dadaglo
132d8f61e9 BUG/MINOR: spoa/python: Cleanup references for failed Module Addobject operations
As per https://docs.python.org/3/c-api/module.html#c.PyModule_AddObject,
references are stolen by the function only for success. We must do
cleanup manually if there is a failure

This patch must be backported as far as 2.0.
2020-12-09 08:39:31 +01:00
Gilchrist Dadaglo
c7485ac101 DOC: spoa/python: Fixing typos in comments
Fixing a missing letter in a comment

This patch must be backported as far as 2.0.
2020-12-09 08:39:31 +01:00
Gilchrist Dadaglo
85b25829bc DOC: spoa/python: Rephrasing memory related error messages
The old message "No more space left available" was redundant with "left
available". This commit is to rephrase that sentence and make it more
explicit we are talking about memory

This patch must be backported as far as 2.0.
2020-12-09 08:39:31 +01:00
Gilchrist Dadaglo
6884742c65 DOC: spoa/python: Fixing typo in IP related error messages
This commit fixes typos in the ps_python_set_var_ip* byte manipulation error
messages

This patch must be backported as far as 2.0.
2020-12-09 08:39:31 +01:00
Gilchrist Dadaglo
d5c428e761 BUG/MAJOR: spoa/python: Fixing return None
As per https://docs.python.org/3/c-api/none.html, None requires to be
incremented before being returned to prevent deallocating none

This patch must be backported as far as 2.0.
2020-12-09 08:39:31 +01:00
Christopher Faulet
6ad06066cd CLEANUP: connection: Remove CS_FL_READ_PARTIAL flag
Since the recent refactoring of the H1 multiplexer, this flag is no more
used. Thus it is removed.
2020-12-04 14:41:49 +01:00
Christopher Faulet
da831fa068 CLEANUP: http-ana: Remove TX_WAIT_NEXT_RQ unsued flag
This flags is now unused. It was used in REQ_WAIT_HTTP analyser, when a
stream was waiting for a request, to set the keep-alive timeout or to avoid
to send HTTP errors to client.
2020-12-04 14:41:49 +01:00
Daniel Corbett
e2370307be CONTRIB: release-estimator: Add release estimating tool
This tool monitors the HAProxy stable branches and calculates a proposed
release date for the next minor release based on the bug fixes that are in
the queue.

Print only:
    ./release-estimator.py --print

Send email:
    ./release-estimator.py --send-mail --from-email from@domain.local --to-email to@domain.local

See contrib/release-estimator/README.md for details.
2020-10-24 12:27:17 +02:00
Willy Tarreau
9018ca9655 CONTRIB: tcploop: remove unused local variables in tcp_pause()
Building with -Wall shows that "pollfd" and "ret" are not used. Silly
copy-paste...
2020-10-22 05:17:08 +02:00
Willy Tarreau
2e065cbbf6 CONTRIB: tcploop: implement a disconnect operation 'D'
This performs a connect(AF_UNSPEC) over an existing connection. This is
mainly for compatibility testing. At this step it only seems to work on
linux for TCP sockets (both listening and established), while SO_LINGER
successfully resets established connections on freebsd and aix.
2020-10-14 08:46:23 +02:00
Willy Tarreau
c3914d4fff MEDIUM: proxy: replace proxy->state with proxy->disabled
The remaining proxy states were only used to distinguish an enabled
proxy from a disabled one. Due to the initialization order, both
PR_STNEW and PR_STREADY were equivalent after startup, and they
would only differ from PR_STSTOPPED when the proxy is disabled or
shutdown (which is effectively another way to disable it).

Now we just have a "disabled" field which allows to distinguish them.
It's becoming obvious that start_proxies() is only used to print a
greeting message now, that we'd rather get rid of. Probably that
zombify_proxy() and stop_proxy() should be merged once their
differences move to the right place.
2020-10-09 11:27:30 +02:00
Willy Tarreau
6b3bf733dd MEDIUM: proxy: remove the unused PR_STFULL state
Since v1.4 or so, it's almost not possible anymore to set this state. The
only exception is by using the CLI to change a frontend's maxconn setting
below its current usage. This case makes no sense, and for other cases it
doesn't make sense either because "full" is a vague concept when only
certain listeners are full and not all. Let's just remove this unused
state and make it clear that it's not reported. The "ready" or "open"
states will continue to be reported without being misleading as they
will be opposed to "stop".
2020-10-09 11:27:30 +02:00
Amaury Denoyelle
da5b6d1cd9 MINOR: stats: hide px/sv/li fields in applet struct
Use an opaque pointer to store proxy instance. Regroup server/listener
as a single opaque pointer. This has the benefit to render the structure
more evolutive to support statistics on other types of objects in the
future.

This patch is needed to extend stat support for components other than
proxies objects.

The prometheus module has been adapted for these changes.
2020-10-05 10:48:58 +02:00
Gilchrist Dadaglo
d00ce06d9d BUG/MEDIUM: contrib/spoa-server: Fix ipv4_address used instead of ipv6_address
Typo leads to checking the wrong object for memory issues

This patch must be backported as far as 2.0.
2020-09-01 18:28:40 +02:00
Gilchrist Dadaglo
2417ebcc01 BUG/MINOR: contrib/spoa-server: Updating references to free in case of failure
When we encounter a failure, all previously borrowed references should
be freed. Especially if the program is not failing immediately

This patch must be backported as far as 2.0.
2020-09-01 18:28:40 +02:00
Gilchrist Dadaglo
c7d303a0a8 BUG/MINOR: contrib/spoa-server: Do not free reference to NULL
As per https://docs.python.org/3/c-api/refcounting.html, Py_DECREF
should not be called on NULL objects.

This patch must be backported as far as 2.0.
2020-09-01 18:28:40 +02:00
Gilchrist Dadaglo
9f0c984cf5 BUG/MINOR: contrib/spoa-server: Ensure ip address references are freed
IP addresses references passed in argument for ps_python are not freed after
they have been used. Leading to a small chance of mem leak if a lot of ip
addresses are passed around

This patch must be backported as far as 2.0.
2020-09-01 18:28:21 +02:00
Gilchrist Dadaglo
222f060be3 BUG/MAJOR: contrib/spoa-server: Fix unhandled python call leading to memory leak
The result from spoa evaluation of the user provided python code is
never passed back to the main spoa process nor freed.
Same for the keyword list passed.
This results into the elements never freed by Python as reference count
never goes down.
https://docs.python.org/3/extending/extending.html#reference-counting-in-python

This patch must be backported as far as 2.0.
2020-09-01 18:28:16 +02:00
Bertrand Jacquin
ae2132cffb MINOR: contrib/spoa-server: allow MAX_FRAME_SIZE override
MAX_FRAME_SIZE is forced to the default value of tune.bufsize, however
they don't necessarily have to be tight together.
2020-09-01 18:25:04 +02:00
William Dauchy
4896d279aa DOC: spoa-server: fix false friends actually
it seems like `actually` was wrongly used instead of `currently`

Signed-off-by: William Dauchy <w.dauchy@criteo.com>
2020-08-05 22:12:54 +02:00
William Dauchy
bbe18ac5df BUG/MINOR: spoa-server: fix size_t format printing
From https://www.python.org/dev/peps/pep-0353/
"A new type Py_ssize_t is introduced, which has the same size as the
compiler's size_t type, but is signed. It will be a typedef for ssize_t
where available."

For integer types, causes printf to expect a size_t-sized integer
argument.

This should fix github issue 

This should be backported to >= v2.2

Signed-off-by: William Dauchy <w.dauchy@criteo.com>
2020-08-05 22:12:47 +02:00
Willy Tarreau
f1ea47d896 BUG/MINOR: htx: add two missing HTX_FL_EOI and remove an unexpected one
A workaround for some difficulties encountered to anticipate end of
messages was addressed by commit 810df0614 ("MEDIUM: htx: Add a flag on
a HTX message when no more data are expected"), but there were 3 issues
in it (with minor impact):
  - the flag was mistakenly set before an EOH in Lua, which would only
    cause incomplete packets to be emitted for now but could cause
    truncated responses in the future. It's not needed to add it on
    the next EOM block as http_forward_proxy_resp() already does it.

  - one was still missing in hlua_applet_http_fct(), possibly causing
    delays on Lua services

  - one was missing in the Prometheus exporter.

All this simply shows that this mechanism is still quite fragile and
not trivial to use, especially in order to deal with the impossibility
to append the EOM, so we'll need to improve the solution in the future
and future backports should not be completely ruled out.

This fix must be backported where the patch above is backported,
typically 2.1 and later as it was required for a set of fixes.
2020-07-23 06:53:27 +02:00
Christopher Faulet
c55a626217 MINOR: contrib/prometheus-exporter: Add missing global and per-server metrics
Following metrics are now exported by the prometheus exporter to reflect recent
changes on HAProxy :

  * haproxy_process_failed_resolutions
  * haproxy_process_bytes_out_total
  * haproxy_process_spliced_bytes_out_total
  * haproxy_process_bytes_out_rate

and

 * haproxy_server_unsafe_idle_connections_current
 * haproxy_server_safe_idle_connections_current
 * haproxy_server_used_connections_current
 * haproxy_server_need_connections_current
2020-07-15 14:08:14 +02:00
Willy Tarreau
62fd12149f CONTRIB: da: fix memory leak in dummy function da_atlas_open()
The dummy function takes care of doing a bit of work using a malloc()
to avoid returning a constant but it doesn't free the tested pointer,
which coverity noticed in issue . Let's free it before testing it
for the return value.

This may be backported but is not important since this code is only
present to allow to build the device detection code and not to actually
run it.
2020-07-12 09:12:07 +02:00
Pierre Cheynier
1e36976352 CLEANUP: contrib/prometheus-exporter: typo fixes for ssl reuse metric
A typo I identified while having a look to our metric inventory.
(s/frontent/frontend)
2020-07-07 23:15:03 +02:00
Willy Tarreau
fc402cf3bc CONTRIB: debug: add missing flags SI_FL_L7_RETRY & SI_FL_D_L7_RETRY
These ones were missing from debug/flags.
2020-06-28 16:05:39 +02:00
Ryan O'Hara
f49a6049b8 BUG/MINOR: systemd: Wait for network to be online
Change systemd service file to wait for network to be completely
online. This solves two problems:

If haproxy is configured to bind to IP address(es) that are not yet
assigned, haproxy would previously fail. The workaround is to use
"option transparent".

If haproxy us configured to use a resolver to resolve servers via DNS,
haproxy would previously fail due to the fact that the network is not
fully online yet. This is the most compelling reason for this patch.

Signed-off-by: Ryan O'Hara <rohara@redhat.com>
Acked-by: Lukas Tribus <lukas@ltri.eu>
2020-06-19 21:31:10 +02:00
Willy Tarreau
54067e9d38 CONTRIB: debug: add the missing flags CO_FL_SAFE_LIST and CO_FL_IDLE_LIST
As often when flags are added they're not updated here. These ones were
missing. They're 2.2 only so no backport is needed.
2020-06-19 14:21:46 +02:00
Willy Tarreau
0a4b0ab177 BUILD: include: add sys/types before netinet/tcp.h
Apparently Cygwin requires sys/types.h before netinet/tcp.h but doesn't
include it by itself, as shown here:
  https://github.com/haproxy/haproxy/actions/runs/131943890

This patch makes sure it's always present, which is in server.c and
the SPOA example.
2020-06-11 11:22:44 +02:00
Willy Tarreau
b2551057af CLEANUP: include: tree-wide alphabetical sort of include files
This patch fixes all the leftovers from the include cleanup campaign. There
were not that many (~400 entries in ~150 files) but it was definitely worth
doing it as it revealed a few duplicates.
2020-06-11 10:18:59 +02:00
Willy Tarreau
36979d9ad5 REORG: include: move the error reporting functions to from log.h to errors.h
Most of the files dealing with error reports have to include log.h in order
to access ha_alert(), ha_warning() etc. But while these functions don't
depend on anything, log.h depends on a lot of stuff because it deals with
log-formats and samples. As a result it's impossible not to embark long
dependencies when using ha_warning() or qfprintf().

This patch moves these low-level functions to errors.h, which already
defines the error codes used at the same places. About half of the users
of log.h could be adjusted, sometimes revealing other issues such as
missing tools.h. Interestingly the total preprocessed size shrunk by
4%.
2020-06-11 10:18:59 +02:00
Willy Tarreau
6be7849f39 REORG: include: move cfgparse.h to haproxy/cfgparse.h
There's no point splitting the file in two since only cfgparse uses the
types defined there. A few call places were updated and cleaned up. All
of them were in C files which register keywords.

There is nothing left in common/ now so this directory must not be used
anymore.
2020-06-11 10:18:58 +02:00
Willy Tarreau
dfd3de8826 REORG: include: move stream.h to haproxy/stream{,-t}.h
This one was not easy because it was embarking many includes with it,
which other files would automatically find. At least global.h, arg.h
and tools.h were identified. 93 total locations were identified, 8
additional includes had to be added.

In the rare files where it was possible to finalize the sorting of
includes by adjusting only one or two extra lines, it was done. But
all files would need to be rechecked and cleaned up now.

It was the last set of files in types/ and proto/ and these directories
must not be reused anymore.
2020-06-11 10:18:58 +02:00
Willy Tarreau
1e56f92693 REORG: include: move server.h to haproxy/server{,-t}.h
extern struct dict server_name_dict was moved from the type file to the
main file. A handful of inlined functions were moved at the bottom of
the file. Call places were updated to use server-t.h when relevant, or
to simply drop the entry when not needed.
2020-06-11 10:18:58 +02:00
Willy Tarreau
4980160ecc REORG: include: move backend.h to haproxy/backend{,-t}.h
The files remained mostly unchanged since they were OK. However, half of
the users didn't need to include them, and about as many actually needed
to have it and used to find functions like srv_currently_usable() through
a long chain that broke when moving the file.
2020-06-11 10:18:58 +02:00
Willy Tarreau
6c58ab0304 REORG: include: move spoe.h to haproxy/spoe{,-t}.h
Only minor change was to make sure all defines were before the structs
in spoe-t.h, everything else went smoothly.
2020-06-11 10:18:58 +02:00