Commit Graph

220 Commits

Author SHA1 Message Date
Anthonin Bonnefoy
51c3aa4628 BUG/MINOR: Fix prometheus '# TYPE' and '# HELP' headers
Prometheus protocol defines HELP and TYPE as a token after the '#' and
the space after the '#' is necessary.
This is expected in the prometheus python client for example
(a8f5c80f65/prometheus_client/parser.py (L194))
and the missing space is breaking the parsing of metrics' type.

This patch must be backported to 2.0.
2019-08-12 08:51:28 +02:00
Christopher Faulet
0fe5c5e1d2 BUG/MINOR: debug: Remove flags CO_FL_SOCK_WR_ENA/CO_FL_SOCK_RD_ENA
These flags were removed by the commit 03abf2d31 ("MEDIUM: connections: Remove
CONN_FL_SOCK*").

This patch may be backported to 2.0.
2019-07-19 09:24:12 +02:00
Christopher Faulet
fc9cfe4006 REORG: proto_htx: Move HTX analyzers & co to http_ana.{c,h} files
The old module proto_http does not exist anymore. All code dedicated to the HTTP
analysis is now grouped in the file proto_htx.c. So, to finish the polishing
after removing the legacy HTTP code, proto_htx.{c,h} files have been moved in
http_ana.{c,h} files.

In addition, all HTX analyzers and related functions prefixed with "htx_" have
been renamed to start with "http_" instead.
2019-07-19 09:24:12 +02:00
Christopher Faulet
22dc248c2a CLEANUP: channel: Remove the unused flag CF_WAKE_CONNECT
This flag is tested or cleared but never set anymore.
2019-07-19 09:24:12 +02:00
Christopher Faulet
c41547b66e MINOR: proto_http: Remove unused http txn flags
Many flags of the HTTP transction (TX_*) are now unused and useless. So the
flags TX_WAIT_CLEANUP, TX_HDR_CONN_*, TX_CON_CLO_SET and TX_CON_KAL_SET were
removed. Most of TX_CON_WANT_* were also removed. Only TX_CON_WANT_TUN has been
kept.
2019-07-19 09:24:12 +02:00
Christopher Faulet
711ed6ae4a MAJOR: http: Remove the HTTP legacy code
First of all, all legacy HTTP analyzers and all functions exclusively used by
them were removed. So the most of the functions in proto_http.{c,h} were
removed. Only functions to deal with the HTTP transaction have been kept. Then,
http_msg and hdr_idx modules were entirely removed. And finally the structure
http_msg was lightened of all its useless information about the legacy HTTP. The
structure hdr_ctx was also removed because unused now, just like unused states
in the enum h1_state. Note that the memory pool "hdr_idx" was removed and
"http_txn" is now smaller.
2019-07-19 09:24:12 +02:00
Christopher Faulet
25994dacb3 MINOR: contrib/prometheus-exporter: Remove tests on the option 'http-use-htx'
Configuring the Prometheus service is now always valid for HTTP proxies. So we
don't rely anymore on the flag PR_O2_USE_HTX.
2019-07-19 09:18:27 +02:00
Aleksandar Lazic
a71447539d DOC: contrib: spoa_server Add some hints for building spoa_server 2019-07-05 16:31:50 +02:00
Christopher Faulet
0c55a15ce1 BUG/MINOR: contrib/prometheus-exporter: Don't try to add empty data blocks
When the response buffer is full and nothing more can be inserted, it is
important to not try to insert an empty data block. Otherwise, when the function
channel_add_input() is called, the flag CF_READ_PARTIAL is set on the response
channel while nothing was read and the stream is uselessly woken up. Finally, we
have loop while the response buffer is full.

This patch must be backported to 2.0.
2019-07-05 14:26:14 +02:00
Christopher Faulet
11921e6819 BUG/MINOR: contrib/prometheus-exporter: Respect the reserve when data are sent
The previous commit e6cdfe574 ("BUG/MINOR: contrib/prometheus-exporter: Don't
use channel_htx_recv_max()") is buggy. The buffer's reserve must be respected.

This patch must be backported to 2.0 and 1.9.
2019-07-03 11:47:20 +02:00
Christopher Faulet
e6cdfe574e BUG/MINOR: contrib/prometheus-exporter: Don't use channel_htx_recv_max()
The function htx_free_data_space() must be used intead. Otherwise, if there are
some output data not already forwarded, the maximum amount of data that may be
inserted into the buffer may be greater than what we can really insert.

This patch must be backported to 2.0.
2019-07-02 21:08:26 +02:00
Tim Duesterhus
86e6b6ebf8 MEDIUM: Make '(cli|con|srv)timeout' directive fatal
They were deprecated with HAProxy 1.5. Time to remove them.
2019-06-17 13:35:54 +02:00
Willy Tarreau
a8ee4b199f CLEANUP: removed obsolete examples an move a few to better places
The following example files awere removed as irrelevant by this
time :
  auth.cfg check.conf ssl.cfg haproxy.spec

The following scripts were removed as having been unused for more
than a decade :
  debug2ansi debug2html debugfind check init.haproxy stats_haproxy.sh

seemless_reload.txt was moved to doc/ where it's more suitable.

haproxy.vim was moved to contrib/syntax-highlight/

scripts/create-release was updated not to try to update haproxy.spec
anymore.
2019-06-15 21:25:06 +02:00
Willy Tarreau
d254aa8139 DOC: update few references to the linux* targets and change them to linux-glibc
The INSTALL guide, the Lua doc and the Prometheus exporter's README all
used to reference "linux2628", "linux26" or even "linux". These were all
updated to consistently reflect "linux-glibc" instead. The default options
were updated there as well so that it should build cleanly on most distros.
2019-06-15 18:03:48 +02:00
Ben51Degrees
f4a82fb26b BUILD/MINOR: 51d: Updated build registration output to indicate thatif the library is a dummy one or not.
When built with the dummy 51Degrees library for testing, the output will
include "(dummy library)" to ensure it is clear that this is this is not
the API.
2019-06-13 18:00:54 +02:00
Willy Tarreau
5e4c5003c5 CLEANUP: 51d: move the 51d dummy lib to contrib/51d/src to match the real lib
This way the directory structure remains the same as with the real lib and
one can apply the same build options regardless of where the lib is stored,
removing any possible confusion.
2019-06-13 15:56:10 +02:00
Ben51Degrees
31c3d51a18 MINOR: 51d: Added dummy libraries for the 51Degrees module for testing.
These are intended for use by HAProxy developers to ensure any changes
did not affect the 51Degrees implementation. The 51Degrees module can be
enabled and used by using the source in contrib/51d. This will run
without breaking, but will not return any meaningful information.

This is ideal for testing HAProxy core code, and other modules alongside
51Degrees, but should never be used as an actual module as it does
nothing.
2019-06-12 18:06:59 +02:00
Daniel Corbett
c802921721 DOC/MINOR: contrib/spoa_server: Fix typo in README
Fix typo in README ps_pyhton.py -> ps_python.py
2019-06-11 19:27:42 +02:00
Daniel Corbett
061766859c MINOR: contrib/spoa_server: Add random IP score
The example configuration uses sess.ip_score however this variable
is not referenced within the example scripts.  This patch adds support
for sess.ip_score to the python + lua scripts and generates a
random number between 1 and 100.
2019-06-11 19:27:42 +02:00
Daniel Corbett
4e0fa55dcd BUG/MEDIUM: contrib/spoa_server: Set FIN flag on agent frames
When communicating over SPOP the AGENT-HELLO, AGENT-DISCONNECT,
and ACK frames must have the FIN flag set.
2019-06-11 19:27:41 +02:00
Daniel Corbett
5897867ac5 MINOR: contrib/spoa_server: Upgrade SPOP to 2.0
Upgrade SPOP version to 2.0
2019-06-11 19:27:41 +02:00
Christopher Faulet
54b5e214b0 MINOR: htx: Don't use end-of-data blocks anymore
This type of blocks is useless because transition between data and trailers is
obvious. And when there is no trailers, the end-of-message is still there to
know when data end for chunked messages.
2019-06-05 10:12:11 +02:00
Christopher Faulet
142b5df778 BUG/MINOR: contrib/prometheus-exporter: Add HTX data block in one time
Since recent changes on the way HTX data blocks are added in an HTX message, we
must now be sure the prometheus service add its own blocks in one time. Indeed,
the function htx_add_data() may now decide to only copy a part of data. So
instead, we must call htx_add_data_atonce() instead.
2019-06-05 10:12:11 +02:00
Willy Tarreau
db20750fe3 CONTRIB: debug: add 4 missing connection/conn_stream flags
The following flags were missing thus added :
- CO_FL_SOCKS4_RECV
- CO_FL_SOCKS4_SEND
- CS_FL_READ_PARTIAL
- CS_FL_KILL_CONN
2019-06-03 14:26:54 +02:00
Willy Tarreau
7bb39d7cd6 CLEANUP: connection: remove the now unused CS_FL_REOS flag
Let's remove it before it gets uesd again. It was mostly replaced with
CS_FL_EOI and by mux-specific states or flags.
2019-06-03 14:23:33 +02:00
Willy Tarreau
76824a8c99 BUILD: contrib/prometheus: fix build breakage caused by move of idle_pct
The idle_pct thread-local variable was moved to struct thread_info by
commit 81036f2 ("MINOR: time: move the cpu, mono, and idle time to
thread_info") but not updated in service-prometheus.c, thus breaking
it.

No backport is needed. This fixes GH issue #110.
2019-06-02 10:38:48 +02:00
Olivier Houchard
250031e444 MEDIUM: sessions: Introduce session flags.
Add session flags, and add a new flag, SESS_FL_PREFER_LAST, to be set when
we use NTLM authentication, and we should reuse the last connection. This
should fix using NTLM with HTX. This totally replaces TX_PREFER_LAST.

This should be backported to 1.9.
2019-05-29 15:41:47 +02:00
Willy Tarreau
ce1f7a16ab CONTRIB: wurfl: address 3 build issues in the wurfl dummy library
Two of them were functions returning an string missing the return
statement and one is just the usual "set but not used".
2019-05-22 14:59:08 +02:00
mbellomi
e30e3001d9 MINOR: WURFL: call header_retireve_callback() in dummy library
The current coverage of the dummy library was limited because the callbacks
passed to wurfl_lookup() were not called. Now we do call them with one existing
and one non-existing headers to make sure that ha_wurfl_retrieve_header() is
covered by the tests as well.
2019-05-22 12:06:05 +02:00
Thierry FOURNIER
4d7bfa1cd1 MINOR/DOC: spoe-server: Add documentation
This is the documentation and examples.
2019-05-13 17:43:47 +02:00
Thierry FOURNIER
00a0225636 MINOR: spoa-server: Add python
This commit adds the Python support for the server.
2019-05-13 17:43:47 +02:00
Thierry FOURNIER
6908c95804 MINOR: spoa-server: Add Lua processing
Use the defined binding for registering Lua engine.
2019-05-13 17:43:47 +02:00
Thierry FOURNIER
a09df3f5d0 MINOR: spoa-server: Execute registered callbacks
Call the right function with the right engine for each received message.
2019-05-13 17:43:47 +02:00
Thierry FOURNIER
fbd3824868 MINOR: spoa-server: Prepare responses
This patch adds SPOP responses managament. It provides SPOP
encoding primitives. It also move the example function
ip_reputation to this new behavior.
2019-05-13 17:43:47 +02:00
Thierry FOURNIER
8b9a73bac0 MINOR: spoa-server: Load files
Declare files to be executed at the begining and execute it. The binding
between the engine and the file is done throught the extension.
2019-05-13 17:43:47 +02:00
Thierry FOURNIER
892f6647c1 MINOR: spoa-server: Allow registering message processors
This function register processor executed by any language for processing
an SPOP message.
2019-05-13 17:43:47 +02:00
Thierry FOURNIER
64eaa33214 MINOR: spoa-server: Allow registering external processes
Add struct for declaring an reistrering external processing resource.
2019-05-13 17:43:47 +02:00
Thierry FOURNIER
786e9e684b MINOR: spoa-server: With debug mode, start only one process
Because debug with processes is simpler if only one process is started.
2019-05-13 17:43:47 +02:00
Thierry FOURNIER
7de6fc6ae1 MINOR: spoa-server: Replace the thread init system by processes
I will replace thread by processes. Note that, I keep the pthread_key
system for identifiying process in the same way that threads. Note
also that I keep commented out the original thread code because I hope
to reactivate it.
2019-05-13 17:43:47 +02:00
Thierry FOURNIER
5301ed1589 MINOR: spoe-server: rename "worker" functions
"worker" name is a little bit generic and it is used in many
places, so it is hard to find the expected symbol.
2019-05-13 17:43:47 +02:00
Thierry FOURNIER
880d7e15ba MINOR: spoa-server: Externalise debug functions
Make external LOG and DEBUG function. Other process can use this ones
and later these functions will be replaced by another log system
2019-05-13 17:43:47 +02:00
Thierry FOURNIER
4aec0a4a9a MINOR: spoa-server: move some definition from spoa_server.c to spoa_server.h
This will allow to add some other files to the project
2019-05-13 17:43:47 +02:00
Thierry FOURNIER
d8b5c77b55 MINOR: spoa-server: Clone the v1.7 spoa-example project
This is a working base.
2019-05-13 17:43:47 +02:00
William Lallemand
eaa4555003 MINOR: systemd: support /etc/sysconfig/ for redhat based distrib
The patch "MINOR: systemd: Make use of master socket in systemd unit"
introduces an environment file in /etc/default.

Unfortunatly this is not supported on redhat-based system, so we add
/etc/sysconfig/haproxy for that.
2019-05-07 14:11:55 +02:00
Tim Duesterhus
eaa4b8a968 MINOR: systemd: Make use of master socket in systemd unit
Unless the EXTRAOPTS variable is overriden in /etc/default/haproxy
the unit file will use the master socket by default.

This patch may be backported to 1.9 and depends on
MINOR: systemd: Use the variables from /etc/default/haproxy.
2019-05-07 14:11:55 +02:00
Apollon Oikonomopoulos
b06014869e MINOR: systemd: Use the variables from /etc/default/haproxy
This will allow seamless upgrades from the sysvinit system while respecting
any changes the users may have made. It will also make local configuration
easier than overriding the systemd unit file.

Note by Tim:

This GPL-2 licensed patch was taken from the Debian project at [1].

It was slightly modified to cleanly apply, because HAProxy's default unit
file does not include rsyslog.service as an 'After' dependency. Also the
subject line was modified to include the proper subsystem and severity.

This patch may be backported to 1.9.

[1] https://salsa.debian.org/haproxy-team/haproxy/blob/master/debian/patches/haproxy.service-use-environment-variables.patch

Co-authored-by: Tim Duesterhus <tim@bastelstu.be>
2019-05-07 14:11:55 +02:00
Yann Cézard
bf60f6b803 BUG/MEDIUM: contrib/modsecurity: If host header is NULL, don't try to strdup it
I discovered this bug when running OWASP regression tests against HAProxy +
modsecurity-spoa (it's a POC to evaluate how it is working).  I found out that
modsecurity spoa will crash when the request doesn't have any Host header.

See the pull request #86 on github for details.

This patch must be backported to 1.9 and 1.8.
2019-04-29 16:26:05 +02:00
Yann Cézard
494ddbff47 DOC: contrib/modsecurity: Typos and fix the reject example
Thanks to https://www.mail-archive.com/haproxy@formilux.org/msg30056.html

This patch may be backported to 1.9 and 1.8.
2019-04-29 16:25:49 +02:00
David Carlier
0470d704a7 BUILD/MEDIUM: contrib: Dummy DeviceAtlas API.
Creating a "mocked" version mainly for testing purposes.
2019-04-26 17:06:32 +02:00
paulborile
cd9b9bd3e4 MINOR: contrib: dummy wurfl library
This is dummy version of the Scientiamobile WURFL C API that can be used
to successfully build/run haproxy compiled with USE_WURFL=1.
It is marked as version 1.11.2.100 to distinguish it from any real version
of the lib. It has no external dependencies so it should work out of the
box by building it like this :

   $ make -C contrib/wurfl

In order to use it, simply reference this directory as the WURFL include
and library paths :

   $ make TARGET=<target> USE_WURFL=1 WURFL_INC=$PWD/contrib/wurfl WURFL_LIB=$PWD/contrib/wurfl
2019-04-23 11:00:23 +02:00