1
0
mirror of http://git.haproxy.org/git/haproxy.git/ synced 2024-12-28 15:42:30 +00:00
Commit Graph

100 Commits

Author SHA1 Message Date
Willy Tarreau
fe1b3b8777 Revert "BUG/MINOR: clock: fix a few occurrences of 'now' being used in place of 'date'"
This reverts commit aadcfc9ea6.

The parts affecting the DeviceAtlas addon were wrong actually, the
"now" variable was a local time_t in a file that's not compiled with
the haproxy binary (dadwsch). Only the fix to the calltrace is correct,
so better revert and fix the only one in a separate commit. No backport
is needed.
2023-04-27 18:14:57 +02:00
Willy Tarreau
64651d1769 BUG/MINOR: opentracing: use 'date' instead of 'now' in debug output
The filter was using "now" in visible output in debug mode, that's
not correct, we should rather use "date" since it's visible. No
backport is needed as it was mostly emphasized by commit 28360dc
("MEDIUM: clock: force internal time to wrap early after boot")
in 2.8..
2023-04-27 18:05:38 +02:00
Willy Tarreau
aadcfc9ea6 BUG/MINOR: clock: fix a few occurrences of 'now' being used in place of 'date'
Since commit 28360dc ("MEDIUM: clock: force internal time to wrap early
after boot") we have a much clearer distinction between 'now' (the internal,
drifting clock) and 'date' (the wall clock time). There were still a few
places where 'now' was being used for human consumption.

No backport is needed.
2023-04-26 19:21:25 +02:00
Willy Tarreau
ad44939e40 CLEANUP: opentracing: remove the last two occurrences of strncat()
In flt_ot_sample_to_str() there were two occurrences of strncat() which
are used to copy N chars from the source and append a zero. For the sake
of definitely getting rid of this nasty function let's replace them by
memcpy() instead. It's worth noting that the length test there appeared
to be incorrect as it didn't make provisions for the trailing zero,
unless the size argument doesn't take it into account (seems unlikely).
Nothing was changed regarding this. If the code was good, it still is,
otherwise if it was bad it still is. At least this is more obvious now
than when using a function that needs n+1 chars to work.
2023-04-07 18:14:28 +02:00
Christopher Faulet
d4eacb3683 MEDIUM: promex: Use the sedesc to report and detect end of processing
Just like for other applets, we now use the SE descriptor instead of the
channel to report error and end-of-stream.
2023-04-05 08:57:06 +02:00
Christopher Faulet
92297749e1 MINOR: applet: No longer set EOI on the SC
Thanks to the previous patch, it is now possible for applets to not set the
CF_EOI flag on the channels. On this point, the applets get closer to the
muxes.
2023-04-05 08:57:05 +02:00
Christopher Faulet
b08c5259eb MINOR: stconn: Always report READ/WRITE event on shutr/shutw
It was done by hand by callers when a shutdown for read or write was
performed. It is now always handled by the functions performing the
shutdown. This way the callers don't take care of it. This will avoid some
bugs.
2023-02-22 15:59:16 +01:00
Christopher Faulet
c1b013bc61 BUG/MINOR: promex: Don't forget to consume the request on error
When the promex applet triggers an error, for instance because the URI is
invalid, we must still take care to consume the request. Otherwise, the
error will be handled by HTTP analyzers as a server abort.

This patch must be backported as far as 2.0.
2023-01-13 09:45:23 +01:00
Christopher Faulet
6e1bbc446b REORG: channel: Rename CF_READ_NULL to CF_READ_EVENT
CF_READ_NULL flag is not really useful and used. It is a transient event
used to wakeup the stream. As we will see, all read events on a channel may
be resumed to only one and are all used to wake up the stream.

In this patch, we introduce CF_READ_EVENT flag as a replacement to
CF_READ_NULL. There is no breaking change for now, it is just a
rename. Gradually, other read events will be merged with this one.
2023-01-09 18:41:08 +01:00
Willy Tarreau
bc2f3934f9 BUILD: makefile: ensure that all USE_* handlers appear before CFLAGS are used
It happens that a few "if USE_foo" were placed too low in the makefile,
and would mostly work by luck thanks to not using variables that were
already referenced before. The opentracing include is even trickier
because it extends OPTIONS_CFLAGS that was last read a few lines before
being included, but it only works because COPTS is defined as a macro and
not a variable, so it will be evaluated later. At least now it doesn't
touch OPTIONS_* anymore and since it's cleanly arranged, it will work by
default via the flags collector.

Let's just move these late USE_* handlers upper and place a visible
delimiter after them reminding not to add any after.
2022-12-23 16:53:35 +01:00
Willy Tarreau
24f5a24c18 BUILD: 51d: fix build issue with recent compilers
With gcc-11.2 and binutils-2.37 I'm getting link errors due to multiply
defined symbols when enabling USE_51DEGREES_V4. This is caused by two
variables being present in hash.h instead of hash.c, hence they're
defined twice.

This patch just moves them to hash.c and turns their declaration to
extern.

No backport is needed since this was introduced in 2.8-dev.
2022-12-15 19:36:13 +01:00
Christopher Faulet
7edec90c00 DOC: promex: Add missing backend metrics
"haproxy_backend_agg_server_status" and "haproxy_backend_agg_check_status"
were not referenced in promex README.

"haproxy_backend_agg_server_check_status" is also missing but it is a
deprecated metric. Thus, it is better to not reference it.
2022-12-09 11:04:12 +01:00
Cedric Paillet
e06e31ea3b MINOR: promex: introduce haproxy_backend_agg_check_status
This patch introduces haproxy_backend_agg_check_status metric
as we wanted in 42d7c402d but with the right data source.

This patch could be backported as far as 2.4.
2022-12-09 10:54:48 +01:00
Cedric Paillet
7d6644e689 BUG/MINOR: promex: create haproxy_backend_agg_server_status
haproxy_backend_agg_server_check_status currently aggregates
haproxy_server_status instead of haproxy_server_check_status.
We deprecate this and create a new one,
haproxy_backend_agg_server_status to clarify what it really
does.

This patch could be backported as far as 2.4.
2022-12-09 10:54:27 +01:00
scientiamobile
6d6787ba7c LICENSE: wurfl: clarify the dummy library license.
This clarifies that LGPL is also permitted for the wurfl.h dummy file.
Should be backported where relevant.

Signed-off-by: Luca Passani <luca.passani@scientiamobile.com>
2022-12-02 05:32:46 +01:00
Dragan Dosen
a9800a0f58 MEDIUM: 51d: add support for 51Degrees V4 with Hash algorithm
This patch also adds a set of new global options:

- 51degrees-use-performance-graph { on | off }
- 51degrees-use-predictive-graph { on | off }
- 51degrees-drift <number>
- 51degrees-difference <number>
- 51degrees-allow-unmatched { on | off }

To build using the latest 51Degrees V4 engine with Hash algorithm, set
USE_51DEGREES_V4=1.

Other supported build options are 51DEGREES_INC, 51DEGREES_LIB and
51DEGREES_SRC which needs to be set to the directory that contains
headers and C files. For example:

make TARGET=<target> USE_51DEGREES_V4=1 51DEGREES_SRC='51D_REPO_PATH'/src
2022-12-01 16:12:21 +01:00
Willy Tarreau
913bea55fc BUILD: prometheus: use __fallthrough in promex_dump_metrics() and IO handler()
This avoids 11 build warnings when preprocessing happens before compiling
with gcc >= 7.
2022-11-14 11:14:02 +01:00
Amaury Denoyelle
56f50a03b7 CLEANUP: cli: rename dynamic error printing state
Rename CLI_ST_PRINT_FREE to CLI_ST_PRINT_DYNERR.

Most notably, this highlights that this is reserved to error printing.

This is done to ensure consistency between CLI_ST_PRINT/CLI_ST_PRINT_DYN
and CLI_ST_PRINT_ERR/CLI_ST_PRINT_DYNERR. The name is also consistent
with the function cli_dynerr() which activates it.
2022-11-10 16:42:47 +01:00
Ilya Shipitsin
3b64a28e15 CLEANUP: assorted typo fixes in the code and comments
This is 31st iteration of typo fixes
2022-08-06 17:12:51 +02:00
Willy Tarreau
c12b321661 CLEANUP: applet: rename appctx_cs() to appctx_sc()
It returns a stream connector, not a conn_stream anymore, so let's
fix its name.
2022-05-27 19:33:35 +02:00
Willy Tarreau
4c218fb25a CLEANUP: promex: rename all occurrences of stconn "cs" to "sc"
Function arguments and local variables called "cs" were renamed to "sc"
to avoid future confusion.
2022-05-27 19:33:35 +02:00
Willy Tarreau
cb086c6de1 REORG: stconn: rename conn_stream.{c,h} to stconn.{c,h}
There's no more reason for keepin the code and definitions in conn_stream,
let's move all that to stconn. The alphabetical ordering of include files
was adjusted.
2022-05-27 19:33:35 +02:00
Willy Tarreau
5edca2f0e1 REORG: rename cs_utils.h to sc_strm.h
This file contains all the stream-connector functions that are specific
to application layers of type stream. So let's name it accordingly so
that it's easier to figure what's located there.

The alphabetical ordering of include files was preserved.
2022-05-27 19:33:35 +02:00
Willy Tarreau
f61dd19284 CLEANUP: stconn: rename cs_{shut,chk}* to sc_*
This applies the following renaming:

cs_shutr() -> sc_shutr()
cs_shutw() -> sc_shutw()
cs_chk_rcv() -> sc_chk_rcv()
cs_chk_snd() -> sc_chk_snd()
cs_must_kill_conn() -> sc_must_kill_conn()
2022-05-27 19:33:35 +02:00
Willy Tarreau
99615ed85d CLEANUP: stconn: rename cs_rx_room_{blk,rdy} to sc_{need,have}_room()
The new name mor eclearly indicates that a stream connector cannot make
any more progress because it needs room in the channel buffer, or that
it may be unblocked because the buffer now has more room available. The
testing function is sc_waiting_room(). This is mostly used by applets.
Note that the flags will change soon.
2022-05-27 19:33:35 +02:00
Willy Tarreau
ea27f48c5a CLEANUP: stconn: rename cs_{check,strm,strm_task} to sc_strm_*
These functions return the app-layer associated with an stconn, which
is a check, a stream or a stream's task. They're used a lot to access
channels, flags and for waking up tasks. Let's just name them
appropriately for the stream connector.
2022-05-27 19:33:34 +02:00
Willy Tarreau
40a9c32e3a CLEANUP: stconn: rename cs_{i,o}{b,c} to sc_{i,o}{b,c}
We're starting to propagate the stream connector's new name through the
API. Most call places of these functions that retrieve the channel or its
buffer are in applets. The local variable names are not changed in order
to keep the changes small and reviewable. There were ~92 uses of cs_ic(),
~96 of cs_oc() (due to co_get*() being less factorizable than ci_put*),
and ~5 accesses to the buffer itself.
2022-05-27 19:33:34 +02:00
Willy Tarreau
026e8fb290 CLEANUP: stconn: tree-wide rename stconn states CS_ST/SB_* to SC_ST/SB_*
This also follows the natural naming. There are roughly 238 changes, all
totally trivial. conn_stream-t.h has become completely void of any
"conn_stream" related stuff now (except its name).
2022-05-27 19:33:34 +02:00
Willy Tarreau
4596fe20d9 CLEANUP: conn_stream: tree-wide rename to stconn (stream connector)
This renames the "struct conn_stream" to "struct stconn" and updates
the descriptions in all comments (and the rare help descriptions) to
"stream connector" or "connector". This touches a lot of files but
the change is minimal. The local variables were not even renamed, so
there's still a lot of "cs" everywhere.
2022-05-27 19:33:34 +02:00
Willy Tarreau
d869e13ed8 CLEANUP: applet: rename the sedesc pointer from "endp" to "sedesc"
Now at least it makes it obvious that it's the stream endpoint descriptor
and not an endpoint. There were few changes thanks to the previous refactor
of the flags.
2022-05-27 19:33:34 +02:00
Willy Tarreau
b605c4213f CLEANUP: conn_stream: rename the stream endpoint flags CS_EP_* to SE_FL_*
Let's now use the new flag names for the stream endpoint.
2022-05-27 19:33:34 +02:00
Willy Tarreau
d56377c5eb CLEANUP: conn_stream: apply endp_flags.cocci tree-wide
This changes all main uses of endp->flags to the se_fl_*() equivalent
by applying coccinelle script endp_flags.cocci. The se_fl_*() functions
themselves were manually excluded from the change, of course.

Note: 144 locations were touched, manually reviewed and found to be OK.

The script was applied with all includes:

  spatch --in-place --recursive-includes -I include --sp-file $script $files
2022-05-27 19:33:34 +02:00
Christopher Faulet
c9929380a4 MINOR: applet: Change return value for .init callback function
0 is now returned on success and -1 on error.
2022-05-17 16:13:21 +02:00
Willy Tarreau
0698c80a58 CLEANUP: applet: remove the unneeded appctx->owner
This one is the pointer to the conn_stream which is always in the
endpoint that is always present in the appctx, thus it's not needed.
This patch removes it and replaces it with appctx_cs() instead. A
few occurences that were using __cs_strm(appctx->owner) were moved
directly to appctx_strm() which does the equivalent.
2022-05-13 14:28:48 +02:00
Willy Tarreau
66435e5f63 CLEANUP: applet: use the appctx's endp instead of cs->endp
The few applets that set CS_EP_EOI or CS_EP_ERROR used to set it on the
endpoint retrieved from the conn_stream while it's already available on
the appctx itself. Better use the appctx one to limit the unneeded
interactions between the two sides.
2022-05-13 14:28:46 +02:00
Willy Tarreau
1c0715b12a CLEANUP: cli: move the status print context into its own context
Now that the CLI's print context is alone in the appctx, it's possible
to refine the appctx's ctx layout so that the cli part matches exactly
a regular svcctx, and as such move the CLI context into an svcctx like
other applets. External code will still build and work because the
struct cli perfectly maps onto the struct cli_print_ctx that's located
into svc.storage. This is of course only to make a smooth transition
during 2.6 and will disappear immediately after.

A tiny change had to be applied to the opentracing addon which performs
direct accesses to the CLI's err pointer in its own print function. The
rest uses the standard cli_print_* which were the only ones that needed
a small change.

The whole "ctx.cli" struct could be tagged as deprecated so that any
possibly existing external code that relies on it will get build
warnings, and the comments in the struct are pretty clear about the
way to fix it, and the lack of future of this old API.
2022-05-06 18:33:22 +02:00
Willy Tarreau
de58d245d6 CLEANUP: promex: stop using appctx->st2
Now that we have out own context, there's no need to use the cryptic st2
struct member. It's solely used to carry a field number here, so let's
add this into the context.
2022-05-06 18:13:35 +02:00
Willy Tarreau
ae1747db85 CLEANUP: promex: make the applet use its own context
The prometheus applet used to map to the stats context since it was not
convenient to have one's own context, and to reuse the fields with its
own values and enums. The obj1 pointer was used both for proxies and
stick-tables; obj2 was used both for servers and listeners.

This change makes use of the generic command context allocation so that
the there's now a properly typed context for prometheus, defined in the
code itself and independent on the stats or appctx ones. For clarity,
the types are correctly set and there's one proxy, one table, one server
and one listener. Some could be compacted using unions but that's not
necessary since the context is reasonably compact. The stats' st_code
field was used as the object state so the new field name is obj_state.

An attempt was made to change the types to const for what us only visited
but some calls pass through the stats code to retrieve info and that code
uses non-const variables due to internal API limitations (read_freq_ctr()
being used and requiring variable). That could change in the future,
though.
2022-05-06 18:13:35 +02:00
Christopher Faulet
6b0a0fb2f9 CLEANUP: tree-wide: Remove any ref to stream-interfaces
Stream-interfaces are gone. Corresponding files can be safely be removed. In
addition, comments are updated accordingly.
2022-04-13 15:10:16 +02:00
Christopher Faulet
a0bdec350f MEDIUM: stream-int/conn-stream: Move blocking flags from SI to CS
Remaining flags and associated functions are move in the conn-stream
scope. These flags are added on the endpoint and not the conn-stream
itself. This way it will be possible to get them from the mux or the
applet. The functions to get or set these flags are renamed accordingly with
the "cs_" prefix and updated to manipualte a conn-stream instead of a
stream-interface.
2022-04-13 15:10:15 +02:00
Christopher Faulet
da098e6c17 MINOR: stream-int/conn-stream: Move si_shut* and si_chk* in conn-stream scope
si_shutr(), si_shutw(), si_chk_rcv() and si_chk_snd() are moved in the
conn-stream scope and renamed, respectively, cs_shutr(), cs_shutw(),
cs_chk_rcv(), cs_chk_snd() and manipulate a conn-stream instead of a
stream-interface.
2022-04-13 15:10:15 +02:00
Christopher Faulet
62e757470a MEDIUM: stream-int/conn-stream: Move stream-interface state in the conn-stream
The stream-interface state (SI_ST_*) is now in the conn-stream. It is a
mechanical replacement for now. Nothing special. SI_ST_* and SI_SB_* were
renamed accordingly. Utils functions to manipulate these infos were moved
under the conn-stream scope.

But it could be good to keep in mind that this part should be
reworked. Indeed, at the CS level, we only need to know if it is ready to
receive or to send. The state of conn-stream from INI to EST is only used on
the server side. The client CS is immediately set to EST. Thus current
SI_ST_* states should probably be moved to the stream to reflect the server
connection state during the establishment stage.
2022-04-13 15:10:15 +02:00
Christopher Faulet
908628c4c0 MEDIUM: tree-wide: Use CS util functions instead of SI ones
At many places, we now use the new CS functions to get a stream or a channel
from a conn-stream instead of using the stream-interface API. It is the
first step to reduce the scope of the stream-interfaces. The main change
here is about the applet I/O callback functions. Before the refactoring, the
stream-interface was the appctx owner. Thus, it was heavily used. Now, as
far as possible,the conn-stream is used. Of course, it remains many calls to
the stream-interface API.
2022-04-13 15:10:14 +02:00
Christopher Faulet
b041b23ae4 MEDIUM: conn-stream: Move remaning flags from CS to endpoint
All old flags CS_FL_* are now moved in the endpoint scope and renamed
CS_EP_* accordingly. It is a systematic replacement. There is no true change
except for the health-check and the endpoint reset. Here it is a bit special
because the same conn-stream is reused. Thus, we must handle endpoint
allocation errors. To do so, cs_reset_endp() has been adapted.

Thanks to this last change, it will now be possible to simplify the
multiplexer and probably the applets too. A review must also be performed to
remove some flags in the channel or the stream-interface. The HTX will
probably be simplified too. Finally, there is now some place in the
conn-stream to move info from the stream-interface.
2022-04-13 15:10:14 +02:00
Christopher Faulet
4aa1d2838c MINOR: applet: Make .init callback more generic
For now there is no much change. Only the appctx is passed as argument when
the .init callback function is called. And it is not possible to yield at
this stage. It is not a problem because the feature is not used. Only the
lua defines this callback function for the lua TCP/HTTP services. The idea
is to be able to use it for all applets to initialize the appctx context.
2022-04-13 15:10:13 +02:00
Miroslav Zagorac
4e4b813bde Revert "BUILD: opentracing: display warning in case of using OT_USE_VARS at compile time"
This reverts commit 6c9f7faa62.

This warning is no longer needed because source code can be compiled with
enabled support for OpenTracing context transfer via HAProxy variables.

This patch must be backported in 2.5.
2022-04-08 16:31:33 +02:00
Miroslav Zagorac
52c2ec3202 MAJOR: opentracing: reenable usage of vars to transmit opentracing context
Since commit 3a4bedccc ("MEDIUM: vars: replace the global name index with
a hash") the names of HAProxy variables are no longer saved, ie their
64-bit hashes are saved instead.

This is very convenient for the HAProxy itself, but for the OpenTracing
module it is a problem because the names of the variables are important
when transferring the OpenTracing context.  Namely, this context consists
of an unknown amount of data stored in a key-value format.  The number
of these data (and the name of the variable used for this purpose) is
determined with the configuration of the OpenTracing filter, as well as
with the tracer used.  The two previous sentences seem to be in conflict,
but that is only so at first glance.  The function in the OpenTracing
filter used to read the context does not really know this, neither their
number nor its name.  The only thing that function actually knows is the
prefix of the variable names used for context transfer, and by that it
could find all the necessary data.  Of course, until the application of
the above-mentioned commit.

The problem is solved in a very simple way: in a common variable that
the filter always knows its name, the names of all variables that are the
product of the OpenTracing context are saved.  The names of these context
variables can only be added to that common variable.  When that variable
is no longer needed (when we no longer need context), it is deleted.

The format for saving data to this common variable is as follows:
  +-----+---------------+-- .. --+-----+---------------+
  | len | variable name |        | len | variable name |
  +-----+---------------+-- .. --+-----+---------------+

The amount of memory space used to store the length of the name is 1 byte,
with a sign (the minus sign is provided for inactive records, but this is
not currently used).  This means that the maximum length of the variable
name to be saved is 127 characters, which is quite enough for use in the
filter.  The buffer size for such data storage is global.tune.bufsize.

This patch must be backported in 2.5.
2022-04-08 16:31:33 +02:00
Miroslav Zagorac
6a1dfdde67 DEBUG: opentracing: display the contents of the err variable after setting
A display of the contents of the err variable has been added to the
FLT_OT_ERR() macro, once it has been set.

This patch must be backported as far as 2.4.
2022-04-08 16:31:33 +02:00
Miroslav Zagorac
854384f400 CLEANUP: opentracing: added FLT_OT_PARSE_INVALID_enum enum
It's always good to replace "hard-coded" values with something that looks
less "hard-coded" (even though that doesn't change anything in the code).
This is done here using FLT_OT_PARSE_INVALID_enum enum.

This patch must be backported as far as 2.4.
2022-04-08 16:31:33 +02:00
Miroslav Zagorac
ca09e01a13 DEBUG: opentracing: show return values of all functions in the debug output
If the OpenTracing filter is compiled using the 'OT_DEBUG=1' option, then
log messages are printed to stderr when the filter is running.  In the log
one can then find (among other things) the order in which the function is
called and the value that the function returns (if it is not a void type).

Prior to applying this patch, no value returned by a function was logged.

Log output example:
  [ 1]    0.038807 [OT]: flt_ot_init_per_thread(0x56365bd45ec0, 0x56365bd48210) {
  [ 1]    0.038807 [OT]:    ot_start(0x56365bd58920, 0x56365bd4e3a0, 0x7f561acba168:(nil)) {
  [ 1]    0.038807 [OT]:    } = 0
  [ 1]    0.038807 [OT]: } = 0

This patch must be backported as far as 2.4.
2022-04-08 16:31:33 +02:00