Commit Graph

1053 Commits

Author SHA1 Message Date
Thierry FOURNIER / OZON.IO
4cac359a39 MEDIUM: log: Decompose %Tq in %Th %Ti %TR
Tq is the time between the instant the connection is accepted and a
complete valid request is received. This time includes the handshake
(SSL / Proxy-Protocol), the idle when the browser does preconnect and
the request reception.

This patch decomposes %Tq in 3 measurements names %Th, %Ti, and %TR
which returns respectively the handshake time, the idle time and the
duration of valid request reception. It also adds %Ta which reports
the request's active time, which is the total time without %Th nor %Ti.
It replaces %Tt as the total time, reporting accurate measurements for
HTTP persistent connections.

%Th is avalaible for TCP and HTTP sessions, %Ti, %TR and %Ta are only
avalaible for HTTP connections.

In addition to this, we have new timestamps %tr, %trg and %trl, which
log the date of start of receipt of the request, respectively in the
default format, in GMT time and in local time (by analogy with %t, %T
and %Tl). All of them are obviously only available for HTTP. These values
are more relevant as they more accurately represent the request date
without being skewed by a browser's preconnect nor a keep-alive idle
time.

The HTTP log format and the CLF log format have been modified to
use %tr, %TR, and %Ta respectively instead of %t, %Tq and %Tt. This
way the default log formats now produce the expected output for users
who don't want to manually fiddle with the log-format directive.

Example with the following log-format :

   log-format "%ci:%cp [%tr] %ft %b/%s h=%Th/i=%Ti/R=%TR/w=%Tw/c=%Tc/r=%Tr/a=%Ta/t=%Tt %ST %B %CC %CS %tsc %ac/%fc/%bc/%sc/%rc %sq/%bq %hr %hs %{+Q}r"

The request was sent by hand using "openssl s_client -connect" :

   Aug 23 14:43:20 haproxy[25446]: 127.0.0.1:45636 [23/Aug/2016:14:43:20.221] test~ test/test h=6/i=2375/R=261/w=0/c=1/r=0/a=262/t=2643 200 145 - - ---- 1/1/0/0/0 0/0 "GET / HTTP/1.1"

=> 6 ms of SSL handshake, 2375 waiting before sending the first char (in
fact the time to type the first line), 261 ms before the end of the request,
no time spent in queue, 1 ms spend connecting to the server, immediate
response, total active time for this request = 262ms. Total time from accept
to close : 2643 ms.

The timing now decomposes like this :

                 first request               2nd request
      |<-------------------------------->|<-------------- ...
      t         tr                       t    tr ...
   ---|----|----|----|----|----|----|----|----|--
      : Th   Ti   TR   Tw   Tc   Tr   Td : Ti   ...
      :<---- Tq ---->:                   :
      :<-------------- Tt -------------->:
                :<--------- Ta --------->:
2016-08-23 15:18:08 +02:00
Willy Tarreau
41d5e3a610 [RELEASE] Released version 1.7-dev4
Released version 1.7-dev4 with the following main changes :
    - MINOR: add list_append_word function
    - MEDIUM: init: use list_append_word in haproxy.c
    - MEDIUM: init: allow directory as argument of -f
    - CLEANUP: config: detect double registration of a config section
    - MINOR: log: add the %Td log-format specifier
    - MEDIUM: filters: Move HTTP headers filtering in its own callback
    - MINOR: filters: Simplify calls to analyzers using 2 new macros
    - MEDIUM: filters: Add pre and post analyzer callbacks
    - DOC: filters: Update the filters documentation accordingly to recent changes
    - BUG/MEDIUM: init: don't use environment locale
    - SCRIPTS: teach git-show-backports how to report upstream commits
    - SCRIPTS: make git-show-backports capable of limiting its history
    - BUG/MAJOR: fix listening IP address storage for frontends
    - BUG/MINOR: fix listening IP address storage for frontends (cont)
    - DOC: Fix typo so fetch is properly parsed by Cyril's converter
    - BUG/MAJOR: http: fix breakage of "reqdeny" causing random crashes
    - BUG/MEDIUM: stick-tables: fix breakage in table converters
    - MINOR: stick-table: change all stick-table converters' inputs to SMP_T_ANY
    - BUG/MEDIUM: dns: unbreak DNS resolver after header fix
    - BUILD: fix build on Solaris 11
    - BUG/MEDIUM: config: fix multiple declaration of section parsers
    - BUG/MEDIUM: stats: show servers state may show an servers from another backend
    - BUG/MEDIUM: fix risk of segfault with "show tls-keys"
    - MEDIUM: dumpstats: 'show tls-keys' is now able to show secrets
    - DOC: update doc about tls-tickets-keys dump
    - MEDIUM: tcp: add 'set-src' to 'tcp-request connection'
    - MINOR: set the CO_FL_ADDR_FROM_SET flags with 'set-src'
    - MEDIUM: tcp/http: add 'set-src-port' action
    - MEDIUM: tcp/http: new set-dst/set-dst-port actions
    - BUG/MEDIUM: sticktables: segfault in some configuration error cases
    - BUILD/MEDIUM: rebuild everything when an include file is changed
    - BUILD/MEDIUM: force a full rebuild if some build options change
    - BUG/MEDIUM: lua: converters doesn't work
    - BUG/MINOR: http: add-header: header name copied twice
    - BUG/MEDIUM: http: add-header: buffer overwritten
    - BUG/MINOR: ssl: fix potential memory leak in ssl_sock_load_dh_params()
    - MINOR: stream: export the function 'smp_create_src_stkctr'
    - BUG/MEDIUM: dumpstats: undefined behavior in stats_tlskeys_list()
    - MEDIUM: dumpstats: make stats_tlskeys_list() yield-aware during tls-keys dump
    - BUG/MINOR: http: url32+src should use the big endian version of url32
    - BUG/MINOR: http: url32+src should check cli_conn before using it
    - DOC: http: add documentation for url32 and url32+src
    - BUG/MINOR: fix http-response set-log-level parsing error
    - MINOR: systemd: Use variable for config and pidfile paths
    - MINOR: systemd: Perform sanity check on config before reload
    - MEDIUM: ssl: support SNI filters with multicerts
    - MINOR: ssl: crt-list parsing factor
    - BUILD: ssl: fix typo causing a build failure in the multicert patch
    - MINOR: listener: add the "accept-netscaler-cip" option to the "bind" keyword
    - MINOR: tcp: add "tcp-request connection expect-netscaler-cip layer4"
    - BUG/MINOR: init: always ensure that global.rlimit_nofile matches actual limits
    - BUG/MINOR: init: ensure that FD limit is raised to the max allowed
    - BUG/MEDIUM: external-checks: close all FDs right after the fork()
    - BUG/MAJOR: external-checks: use asynchronous signal delivery
    - BUG/MINOR: external-checks: do not unblock undesired signals
    - CLEANUP: external-check: don't block/unblock SIGCHLD when manipulating the list
    - BUG/MEDIUM: filters: Fix data filtering when data are modified
    - BUG/MINOR: filters: Fix HTTP parsing when a filter loops on data forwarding
    - BUG/MINOR: srv-state: fix incorrect output of state file
    - BUG/MINOR: ssl: close ssl key file on error
    - BUG/MINOR: http: fix misleading error message for response captures
    - BUG/BUILD: don't automatically run "make" on "make install"
    - DOC: add missing doc for http-request deny [deny_status <status>]
    - CLEANUP: dumpstats: u64 field is an unsigned type.
    - BUG/MEDIUM: http: unbreak uri/header/url_param hashing
    - BUG/MINOR: Rework slightly commit 9962f8fc to clean code and avoid mistakes
    - MINOR: new function my_realloc2 = realloc + free upon failure
    - CLEANUP: fixed some usages of realloc leading to memory leak
    - Revert "BUG/MINOR: ssl: fix potential memory leak in ssl_sock_load_dh_params()"
    - CLEANUP: connection: using internal struct to hold source and dest port.
    - DOC: spelling fixes
    - BUG/MINOR: ssl: fix potential memory leak in ssl_sock_load_dh_params()
    - BUG/MEDIUM: dns: fix alignment issues in the DNS response parser
    - BUG/MINOR: Fix endiness issue in DNS header creation code
    - BUG/MEDIUM: lua: the function txn_done() from sample fetches can crash
    - BUG/MEDIUM: lua: the function txn_done() from action wrapper can crash
    - MEDIUM: http: implement http-response track-sc* directive
    - BUG/MINOR: peers: Fix peers data decoding issue
    - BUG/MINOR: peers: don't count track-sc multiple times on errors
    - MINOR: standard: add function "escape_string"
    - BUG/MEDIUM: log: use function "escape_string" instead of "escape_chunk"
    - MINOR: tcp: Return TCP statistics like RTT and RTT variance
    - DOC: lua: remove old functions
    - BUG/MEDIUM: lua: somme HTTP manipulation functions are called without valid requests
    - DOC: fix json converter example and error message
    - BUG/MEDIUM: stream-int: completely detach connection on connect error
    - DOC: minor typo fixes to improve HTML parsing by haproxy-dconv
    - BUILD: make proto_tcp.c compatible with musl library
    - BUG/MAJOR: compression: initialize avail_in/next_in even during flush
    - BUG/MEDIUM: samples: make smp_dup() always duplicate the sample
    - MINOR: sample: implement smp_is_safe() and smp_make_safe()
    - MINOR: sample: provide smp_is_rw() and smp_make_rw()
    - BUG/MAJOR: server: the "sni" directive could randomly cause trouble
    - BUG/MEDIUM: stick-tables: do not fail on string keys with no allocated size
    - BUG/MEDIUM: stick-table: properly convert binary samples to keys
    - MINOR: sample: use smp_make_rw() in upper/lower converters
    - MINOR: tcp: add dst_is_local and src_is_local
    - BUG/MINOR: peers: some updates are pushed twice after a resync.
    - BUILD: protocol: fix some build errors on OpenBSD
    - BUILD: log: iovec requires to include sys/uio.h on OpenBSD
    - BUILD: tcp: do not include netinet/ip.h for IP_TTL
    - BUILD: connection: fix build breakage on openbsd due to missing in_systm.h
    - BUILD: checks: remove the last strcat and eliminate a warning on OpenBSD
    - BUILD: tcp: define SOL_TCP when only IPPROTO_TCP exists
    - BUILD: compression: remove a warning when no compression lib is used
    - BUILD: poll: remove unused hap_fd_isset() which causes a warning with clang
    - MINOR: tcp: add further tcp info fetchers
    - BUG/MINOR: peers: empty chunks after a resync.
    - BUG/MAJOR: stick-counters: possible crash when using sc_trackers with wrong table
    - MINOR: standard.c: ipcmp() function to compare 2 IP addresses stored in 2 struct sockaddr_storage
    - MINOR: standard.c: ipcpy() function to copy an IP address from a struct sockaddr_storage into an other one
    - MAJOR: listen section: don't use first bind port anymore when no server ports are provided
2016-08-14 12:25:21 +02:00
Joe Williams
30fcd39f35 MINOR: tcp: add further tcp info fetchers
Adding on to Thierry's work (http://git.haproxy.org/?p=haproxy.git;h=6310bef5)
I have added a few more fetchers for counters based on the tcp_info struct
maintained by the kernel :

  fc_unacked, fc_sacked, fc_retrans, fc_fackets, fc_lost,
  fc_reordering

Two fields were not added because they're version-dependant :
  fc_rcv_rtt, fc_total_retrans

The fields name depend on the operating system. FreeBSD and NetBSD prefix
all the field names with "__" so we have to rely on a few #ifdef for
portability.
2016-08-10 23:02:46 +02:00
Willy Tarreau
16e015635c MINOR: tcp: add dst_is_local and src_is_local
It is sometimes needed in application server environments to easily tell
if a source is local to the machine or a remote one, without necessarily
knowing all the local addresses (dhcp, vrrp, etc). Similarly in transparent
proxy configurations it is sometimes desired to tell the difference between
local and remote destination addresses.

This patch adds two new sample fetch functions for this :

dst_is_local : boolean
  Returns true if the destination address of the incoming connection is local
  to the system, or false if the address doesn't exist on the system, meaning
  that it was intercepted in transparent mode. It can be useful to apply
  certain rules by default to forwarded traffic and other rules to the traffic
  targetting the real address of the machine. For example the stats page could
  be delivered only on this address, or SSH access could be locally redirected.
  Please note that the check involves a few system calls, so it's better to do
  it only once per connection.

src_is_local : boolean
  Returns true if the source address of the incoming connection is local to the
  system, or false if the address doesn't exist on the system, meaning that it
  comes from a remote machine. Note that UNIX addresses are considered local.
  It can be useful to apply certain access restrictions based on where the
  client comes from (eg: require auth or https for remote machines). Please
  note that the check involves a few system calls, so it's better to do it only
  once per connection.
2016-08-09 16:50:08 +02:00
Olivier Doucet
aa1ea8a9ee DOC: minor typo fixes to improve HTML parsing by haproxy-dconv
This must be backported to 1.6 and 1.5
2016-08-07 09:29:20 +02:00
Herve COMMOWICK
8dfe863fbf DOC: fix json converter example and error message 2016-08-07 08:08:18 +02:00
Thierry FOURNIER / OZON.IO
53e381c3a0 DOC: lua: remove old functions
The functions "req_replace_value()" and "res_replace_value()"
doesn't exists in the 1.6 version. There inherited from the 1.6dev.

This patch must be backported in 1.6 version
2016-08-03 00:05:59 +02:00
Thierry Fournier / OZON.IO
6310bef511 MINOR: tcp: Return TCP statistics like RTT and RTT variance
This patch adds 4 new sample fetches which returns the RTT of the
established connexion and the RTT variance. The established connection
can be between the client and HAProxy, and between HAProxy and the
server. This is very useful for statistics. A great use case is the
estimation of the TCP connection time of the client. Note that the
RTT of the server side is not so interesting because we already have
the connect() time.
2016-07-27 13:47:09 +02:00
Ruoshan Huang
e4edc6b628 MEDIUM: http: implement http-response track-sc* directive
This enables tracking of sticky counters from current response. The only
difference from "http-request track-sc" is the <key> sample expression
can only make use of samples in response (eg. res.*, status etc.) and
samples below Layer 6.
2016-07-26 14:31:14 +02:00
Thierry FOURNIER
ab00df6cf6 BUG/MEDIUM: lua: the function txn_done() from sample fetches can crash
The function txn_done() ends a transaction. It does not make
sense to call this function from a lua sample-fetch wrapper,
because the role of a sample-fetch is not to terminate a
transaction.

This patch modify the role of the fucntion txn_done() if it
is called from a sample-fetch wrapper, now it just ends the
execution of the Lua code like the done() function.

Must be backported in 1.6
2016-07-14 16:14:24 +02:00
Dan Lloyd
8e48b8745e DOC: spelling fixes
[wt: most of them are probably valid for 1.6 and 1.5 as well]
2016-07-05 19:11:02 +02:00
Willy Tarreau
be1d34ded2 DOC: add missing doc for http-request deny [deny_status <status>]
The feature was introduced in 1.6-dev2 by commit 108b1dd ("MEDIUM:
http: configurable http result codes for http-request deny") but the
doc was missing. Thanks to Cyril for noticing.

This must be backported into 1.6.
2016-06-26 19:37:59 +02:00
Bertrand Jacquin
9075968356 MINOR: tcp: add "tcp-request connection expect-netscaler-cip layer4"
This configures the client-facing connection to receive a NetScaler
Client IP insertion protocol header before any byte is read from the
socket. This is equivalent to having the "accept-netscaler-cip" keyword
on the "bind" line, except that using the TCP rule allows the PROXY
protocol to be accepted only for certain IP address ranges using an ACL.
This is convenient when multiple layers of load balancers are passed
through by traffic coming from public hosts.
2016-06-20 23:02:47 +02:00
Bertrand Jacquin
93b227db95 MINOR: listener: add the "accept-netscaler-cip" option to the "bind" keyword
When NetScaler application switch is used as L3+ switch, informations
regarding the original IP and TCP headers are lost as a new TCP
connection is created between the NetScaler and the backend server.

NetScaler provides a feature to insert in the TCP data the original data
that can then be consumed by the backend server.

Specifications and documentations from NetScaler:
  https://support.citrix.com/article/CTX205670
  https://www.citrix.com/blogs/2016/04/25/how-to-enable-client-ip-in-tcpip-option-of-netscaler/

When CIP is enabled on the NetScaler, then a TCP packet is inserted just after
the TCP handshake. This is composed as:

  - CIP magic number : 4 bytes
    Both sender and receiver have to agree on a magic number so that
    they both handle the incoming data as a NetScaler Client IP insertion
    packet.

  - Header length : 4 bytes
    Defines the length on the remaining data.

  - IP header : >= 20 bytes if IPv4, 40 bytes if IPv6
    Contains the header of the last IP packet sent by the client during TCP
    handshake.

  - TCP header : >= 20 bytes
    Contains the header of the last TCP packet sent by the client during TCP
    handshake.
2016-06-20 23:02:47 +02:00
Emmanuel Hocdet
d294aea605 MEDIUM: ssl: support SNI filters with multicerts
SNI filters used to be ignored with multicerts (eg: those providing
ECDSA and RSA at the same time). This patch makes them work like
other certs.

Note: most of the changes in this patch are due to an extra level of
      indent, read it with "git show -b".
2016-06-20 17:15:17 +02:00
Dragan Dosen
0070cd52c8 DOC: http: add documentation for url32 and url32+src
This patch needs to be backported to 1.6 and 1.5.
2016-06-16 12:53:25 +02:00
William Lallemand
13e9b0c9ed MEDIUM: tcp/http: new set-dst/set-dst-port actions
Like 'set-src' and 'set-src-port' but for destination address and port.
It's available in 'tcp-request connection' and 'http-request' actions.
2016-06-01 11:44:11 +02:00
William Lallemand
44be6405a1 MEDIUM: tcp/http: add 'set-src-port' action
set-src-port works the same way as 'set-src' but for the source port.
It's available in 'tcp-request connection' and 'http-request' actions.
2016-06-01 11:44:11 +02:00
William Lallemand
2e785f23cb MEDIUM: tcp: add 'set-src' to 'tcp-request connection'
The 'set-src' action was not available for tcp actions The action code
has been converted into a function in proto_tcp.c to be used for both
'http-request' and 'tcp-request connection' actions.

Both http and tcp keywords are registered in proto_tcp.c
2016-06-01 11:44:11 +02:00
William Lallemand
bb933468b4 DOC: update doc about tls-tickets-keys dump
The unix socket can now dump the tls-tickets-keys.
2016-06-01 10:50:50 +02:00
Nenad Merdanovic
1516fe31dd DOC: Fix typo so fetch is properly parsed by Cyril's converter
Signed-off-by: Nenad Merdanovic <nmerdan@anine.io>
2016-05-19 22:10:33 +02:00
Maxime de Roucy
e3841395ad BUG/MEDIUM: init: don't use environment locale
This patch removes setlocale from the main function. It was introduced
by commit 379d9c7 ("MEDIUM: init: allow directory as argument of -f")
in 1.7-dev a few commits ago after a discussion on the mailing list.

Some regex may have different behaviours depending on the
locale. Some LUA scripts may change their behaviour too
(http://lua-users.org/wiki/LuaLocales).

Without this patch (haproxy is using setlocale) :

	$ cat locale.cfg
	defaults
	  mode http

	frontend test
	  bind :9000
	  mode http
	  use_backend testbk if { hdr_reg(X-Test) ^\w+$ }

	backend testbk
	  mode http
	  server s 127.0.0.1:80

	$ LANG=fr_FR.UTF-8 ./haproxy -f locale.cfg
	$ curl -i -H "X-Test: chec" localhost:9000
	HTTP/1.1 200 OK
	...

	$ LANG=C ./haproxy -f locale.cfg
	$ curl -i -H "X-Test: chec" localhost:9000
	HTTP/1.0 503 Service Unavailable
	...
2016-05-19 07:19:19 +02:00
Christopher Faulet
f34b28aa6e DOC: filters: Update the filters documentation accordingly to recent changes 2016-05-18 15:11:54 +02:00
Willy Tarreau
27b639d37f MINOR: log: add the %Td log-format specifier
As suggested by Pavlos, it's too bad that we didn't have a %Td log
format tag given that there are a few mentions of Td corresponding
to the data transmission time already in the doc, so this is now done.
Just like the other specifiers, we report -1 if the connection failed
before reaching the data transmission state.
2016-05-17 18:04:30 +02:00
Maxime de Roucy
379d9c7c14 MEDIUM: init: allow directory as argument of -f
If -f argument is a directory add all the files (and only files) it
containes to the config files list.
These files are added in lexical order (respecting LC_COLLATE).
Only files with ".cfg" extension are added.
Only non hidden files (not prefixed with ".") are added.
Symlink are followed.
The -f order is still respected:

        $ tree -a rootdir
        rootdir
        |-- dir1
        ||-- .6.cfg
        ||-- 1.cfg
        ||-- 2
        ||-- 3.cfg
        ||-- 4.cfg -> 1.cfg
        ||-- 5 -> 1.cfg
        ||-- 7.cfg -> .
        |`-- dir4
        |`-- 8.cfg
        |-- dir2
        ||-- 10.cfg
        |`-- 9.cfg
        |-- dir3
        |`-- 11.cfg
        |-- link -> dir3/
        |-- root1
        |-- root2
        `-- root3

        $ ./haproxy -C rootdir -f root2 -f dir2 -f root3 -f dir1 \
                               -f link -f root1
        root2
        dir2/10.cfg
        dir2/9.cfg
        root3
        dir1/1.cfg
        dir1/3.cfg
        dir1/4.cfg
        link/11.cfg
        root1

This can be useful on systemd where you can't change the haproxy
commande line options on service reload.
2016-05-14 07:09:33 +02:00
Willy Tarreau
7d1b48fae0 [RELEASE] Released version 1.7-dev3
Released version 1.7-dev3 with the following main changes :
    - MINOR: sample: Moves ARGS underlying type from 32 to 64 bits.
    - BUG/MINOR: log: Don't use strftime() which can clobber timezone if chrooted
    - BUILD: namespaces: fix a potential build warning in namespaces.c
    - MINOR: da: Using ARG12 macro for the sample fetch and the convertor.
    - DOC: add encoding to json converter example
    - BUG/MINOR: conf: "listener id" expects integer, but its not checked
    - DOC: Clarify tunes.vars.xxx-max-size settings
    - CLEANUP: chunk: adding NULL check to chunk_dup allocation.
    - CLEANUP: connection: fix double negation on memcmp()
    - BUG/MEDIUM: peers: fix incorrect age in frequency counters
    - BUG/MEDIUM: Fix RFC5077 resumption when more than TLS_TICKETS_NO are present
    - BUG/MAJOR: Fix crash in http_get_fhdr with exactly MAX_HDR_HISTORY headers
    - BUG/MINOR: lua: can't load external libraries
    - BUG/MINOR: prevent the dump of uninitialized vars
    - CLEANUP: map: it seems that the map were planed to be chained
    - MINOR: lua: move class registration facilities
    - MINOR: lua: remove some useless checks
    - CLEANUP: lua: Remove two same functions
    - MINOR: lua: refactor the Lua object registration
    - MINOR: lua: precise message when a critical error is catched
    - MINOR: lua: post initialization
    - MINOR: lua: Add internal function which strip spaces
    - MINOR: lua: convert field to lua type
    - DOC: "addr" parameter applies to both health and agent checks
    - DOC: timeout client: pointers to timeout http-request
    - DOC: typo on stick-store response
    - DOC: stick-table: amend paragraph blaming the loss of table upon reload
    - DOC: typo: ACL subdir match
    - DOC: typo: maxconn paragraph is wrong due to a wrong buffer size
    - DOC: regsub: parser limitation about the inability to use closing square brackets
    - DOC: typo: req.uri is now replaced by capture.req.uri
    - DOC: name set-gpt0 mismatch with the expected keyword
    - MINOR: http: sample fetch which returns unique-id
    - MINOR: dumpstats: extract stats fields enum and names
    - MINOR: dumpstats: split stats_dump_info_to_buffer() in two parts
    - MINOR: dumpstats: split stats_dump_fe_stats() in two parts
    - MINOR: dumpstats: split stats_dump_li_stats() in two parts
    - MINOR: dumpstats: split stats_dump_sv_stats() in two parts
    - MINOR: dumpstats: split stats_dump_be_stats() in two parts
    - MINOR: lua: dump general info
    - MINOR: lua: add class proxy
    - MINOR: lua: add class server
    - MINOR: lua: add class listener
    - BUG/MEDIUM: stick-tables: some sample-fetch doesn't work in the connection state.
    - MEDIUM: proxy: use dynamic allocation for error dumps
    - CLEANUP: remove unneeded casts
    - CLEANUP: uniformize last argument of malloc/calloc
    - DOC: fix "needed" typo
    - BUG/MINOR: dumpstats: fix write to global chunk
    - BUG/MINOR: dns: inapropriate way out after a resolution timeout
    - BUG/MINOR: dns: trigger a DNS query type change on resolution timeout
    - CLEANUP: proto_http: few corrections for gcc warnings.
    - BUG/MINOR: DNS: resolution structure change
    - BUG/MINOR : allow to log cookie for tarpit and denied request
    - BUG/MEDIUM: ssl: rewind the BIO when reading certificates
    - OPTIM/MINOR: session: abort if possible before connecting to the backend
    - DOC: http: rename the unique-id sample and add the documentation
    - BUG/MEDIUM: trace.c: rdtsc() is defined in two files
    - BUG/MEDIUM: channel: fix miscalculation of available buffer space (2nd try)
    - BUG/MINOR: server: risk of over reading the pref_net array.
    - BUG/MINOR: cfgparse: couple of small memory leaks.
    - BUG/MEDIUM: sample: initialize the pointer before parse_binary call.
    - DOC: fix discrepancy in the example for http-request redirect
    - MINOR: acl: Add predefined METH_DELETE, METH_PUT
    - CLEANUP: .gitignore cleanup
    - DOC: Clarify IPv4 address / mask notation rules
    - CLEANUP: fix inconsistency between fd->iocb, proto->accept and accept()
    - BUG/MEDIUM: fix maxaccept computation on per-process listeners
    - BUG/MINOR: listener: stop unbound listeners on startup
    - BUG/MINOR: fix maxaccept computation according to the frontend process range
    - TESTS: add blocksig.c to run tests with all signals blocked
    - MEDIUM: unblock signals on startup.
    - MINOR: filters: Print the list of existing filters during HA startup
    - MINOR: filters: Typo in an error message
    - MINOR: filters: Filters must define the callbacks struct during config parsing
    - DOC: filters: Add filters documentation
    - BUG/MEDIUM: channel: don't allow to overwrite the reserve until connected
    - BUG/MEDIUM: channel: incorrect polling condition may delay event delivery
    - BUG/MEDIUM: channel: fix miscalculation of available buffer space (3rd try)
    - BUG/MEDIUM: log: fix risk of segfault when logging HTTP fields in TCP mode
    - MINOR: Add ability for agent-check to set server maxconn
    - CLEANUP: Use server_parse_maxconn_change_request for maxconn CLI updates
    - MINOR: filters: add opaque data
    - BUG/MEDIUM: lua: protects the upper boundary of the argument list for converters/fetches.
    - MINOR: lua: migrate the argument mask to 64 bits type.
    - BUG/MINOR: dumpstats: Fix the "Total bytes saved" counter in backends stats
    - BUG/MINOR: log: fix a typo that would cause %HP to log <BADREQ>
    - BUG/MEDIUM: http: fix incorrect reporting of server errors
    - MINOR: channel: add new function channel_congested()
    - BUG/MEDIUM: http: fix risk of CPU spikes with pipelined requests from dead client
    - BUG/MAJOR: channel: fix miscalculation of available buffer space (4th try)
    - BUG/MEDIUM: stream: ensure the SI_FL_DONT_WAKE flag is properly cleared
    - BUG/MEDIUM: channel: fix inconsistent handling of 4GB-1 transfers
    - BUG/MEDIUM: stats: show servers state may show an empty or incomplete result
    - BUG/MEDIUM: stats: show backend may show an empty or incomplete result
    - MINOR: stats: fix typo in help messages
    - MINOR: stats: show stat resolvers missing in the help message
    - BUG/MINOR: dns: fix DNS header definition
    - BUG/MEDIUM: dns: fix alignment issue when building DNS queries
    - CLEANUP: don't ignore scripts in .gitignore
    - BUILD: add a few release and backport scripts in scripts/
2016-05-10 15:36:58 +02:00
Nenad Merdanovic
174dd37d88 MINOR: Add ability for agent-check to set server maxconn
This is very useful in complex architecture systems where HAproxy
is balancing DB connections for example. We want to keep the maxconn
high in order to avoid issues with queueing on the LB level when
there is slowness on another part of the system. Example is a case of
an architecture where each thread opens multiple DB connections, which
if get stuck in queue cause a snowball effect (old connections aren't
closed, new ones cannot be established). These connections are mostly
idle and the DB server has no problem handling thousands of them.

Allowing us to dynamically set maxconn depending on the backend usage
(LA, CPU, memory, etc.) enables us to have high maxconn for situations
like above, but lowering it in case there are real issues where the
backend servers become overloaded (cache issues, DB gets hit hard).
2016-04-25 17:23:50 +02:00
Christopher Faulet
c3fe5330be DOC: filters: Add filters documentation
The configuration documention has been updated. Doc about the filter line has
been added and a new chapter (. 9) has been created to list and document
supported filters (for now, flt_trace and flt_http_comp).

The developer documentation about filters has also been added. The is a "pre"
version. Incoming changes in the filter API will require an update.
This documentation requires a deeper review and some TODO need to be complete.
2016-04-21 07:01:41 +02:00
Daniel Schneller
eba5634f2c DOC: Clarify IPv4 address / mask notation rules
Adds some examples regarding shorthand IPv4 address notation which might
be confused with RFC 4632 CIDR notation, leading to different than
expected results.
2016-04-13 07:20:46 +02:00
Daniel Schneller
9ff96c7a62 MINOR: acl: Add predefined METH_DELETE, METH_PUT
Adds the missing HTTP verbs DELETE and PUT as predefined ACLs, similar
to GET, POST etc.
2016-04-12 11:44:09 +02:00
Coen Rosdorff
596659b023 DOC: fix discrepancy in the example for http-request redirect
Commit c8f0e78 ("DOC: typo: req.uri is now replaced by capture.req.uri")
fixed a discrepancy in the doc but the scheme is still missing, resulting
in a redirect loop. Let's fix this as well. This should be backported to
1.5.
2016-04-12 11:26:22 +02:00
Thierry Fournier
0e00dca58b DOC: http: rename the unique-id sample and add the documentation
This patch renames the ssample fetch from "uniqueid" to "unique-id".
It also adds the documentation associated with this sample fetch.
2016-04-07 19:14:58 +02:00
Thierry Fournier
ff480424ab MINOR: lua: add class listener
This class provides the access to the listener struct, it allows
some manipulations and retrieve informations.
2016-03-30 18:43:47 +02:00
Thierry Fournier
f2fdc9dc39 MINOR: lua: add class server
This class provides the access to the server struct, it allows
some manipulations and retrieve informations.
2016-03-30 18:43:47 +02:00
Thierry Fournier
f61aa6356e MINOR: lua: add class proxy
This class provides the access to the proxy struct, it allows
some manipulations and retrieve informations.
2016-03-30 18:43:42 +02:00
Thierry Fournier
eea77c0e17 MINOR: lua: dump general info
This patch adds function able to dump general haproxy information.
2016-03-30 17:27:40 +02:00
Thierry Fournier
b912567020 DOC: name set-gpt0 mismatch with the expected keyword
replace set-gpt0 by sc-set-gpt0

must be backported in 1.6
2016-03-30 17:16:46 +02:00
Baptiste Assmann
c8f0e78b25 DOC: typo: req.uri is now replaced by capture.req.uri
A configuration example was not updated after the switch from req.uri to
capture.req.uri.

backport: 1.5 and above
2016-03-30 17:13:44 +02:00
Baptiste Assmann
66025d856c DOC: regsub: parser limitation about the inability to use closing square brackets
We can't match range in regsub, since the closing bracket is evaluated
by the configuration parser.

backport: 1.6 and above
2016-03-30 17:13:24 +02:00
Baptiste Assmann
79fb45d10b DOC: typo: maxconn paragraph is wrong due to a wrong buffer size
HAProxy allocates 2 tune.bufsize, which is by default 16kB.

backport: 1.4 and above
2016-03-30 17:12:40 +02:00
Baptiste Assmann
33db600442 DOC: typo: ACL subdir match
ACL subdir match is "-m dir"

backport: 1.5 and above
2016-03-30 17:12:04 +02:00
Baptiste Assmann
123ff0463e DOC: stick-table: amend paragraph blaming the loss of table upon reload
This statement is not true anymore since we have the peers in HAProxy.

backport: 1.6 and above
2016-03-30 17:11:45 +02:00
Baptiste Assmann
2f2d2ec164 DOC: typo on stick-store response
It is used to store responses, and not requests.

backport: 1.5 and above
2016-03-30 16:53:39 +02:00
Baptiste Assmann
2e1941ec6e DOC: timeout client: pointers to timeout http-request
It worth mentionning "timeout http-request" in the "timeout client"
documentation paragraph, to ensure nobody misses this important setting.

backport: 1.5 and above
2016-03-30 16:53:34 +02:00
Baptiste Assmann
13f835380f DOC: "addr" parameter applies to both health and agent checks
It was not obvious in the documentation that the server's "addr"
parameter applies to both the agent and the health check.

backport: 1.5 and above
2016-03-30 15:49:52 +02:00
Daniel Schneller
0b547059ec DOC: Clarify tunes.vars.xxx-max-size settings
Adds a little more clarity to the description of the maximum sizes of
the different variable scopes and adds a note about what happens when
the space allocated for variables is too small.

Also fixes some typos and grammar/spelling issues re/ variables and
their naming conventions, copied throughout the document.
2016-03-24 07:50:47 +01:00
Mac Browning
e83345df1b DOC: add encoding to json converter example
Without the encoding `log-format` will issue a warning like:

$ haproxy -c -f /etc/haproxy/haproxy.cfg
[WARNING] 073/180933 (179) : parsing [/etc/haproxy/haproxy.cfg:46] :
'log-format' : sample fetch <capture.req.hdr(1),json> failed with : missing
args for conv method 'json'
2016-03-17 06:14:41 +01:00
David Carlier
840b0240bc MINOR: da: Using ARG12 macro for the sample fetch and the convertor.
Regarding the minor update introduced in the
cd6c3c7cb4 commit, the DeviceAtlas
module is now able to use up to 12 device properties via the
new ARG12 macro.
2016-03-17 05:44:33 +01:00
Willy Tarreau
8234f6dae8 [RELEASE] Released version 1.7-dev2
Released version 1.7-dev2 with the following main changes :
    - DOC: lua: fix lua API
    - DOC: mailers: typo in 'hostname' description
    - DOC: compression: missing mention of libslz for compression algorithm
    - BUILD/MINOR: regex: missing header
    - BUG/MINOR: stream: bad return code
    - DOC: lua: fix somme errors and add implicit types
    - MINOR: lua: add set/get priv for applets
    - BUG/MINOR: http: fix several off-by-one errors in the url_param parser
    - BUG/MINOR: http: Be sure to process all the data received from a server
    - MINOR: filters/http: Use a wrapper function instead of stream_int_retnclose
    - BUG/MINOR: chunk: make chunk_dup() always check and set dst->size
    - DOC: ssl: fixed some formatting errors in crt tag
    - MINOR: chunks: ensure that chunk_strcpy() adds a trailing zero
    - MINOR: chunks: add chunk_strcat() and chunk_newstr()
    - MINOR: chunk: make chunk_initstr() take a const string
    - MEDIUM: tools: add csv_enc_append() to preserve the original chunk
    - MINOR: tools: make csv_enc_append() always start at the first byte of the chunk
    - MINOR: lru: new function to delete <nb> least recently used keys
    - DOC: add Ben Shillito as the maintainer of 51d
    - BUG/MINOR: 51d: Ensures a unique domain for each configuration
    - BUG/MINOR: 51d: Aligns Pattern cache implementation with HAProxy best practices.
    - BUG/MINOR: 51d: Releases workset back to pool.
    - BUG/MINOR: 51d: Aligned const pointers to changes in 51Degrees.
    - CLEANUP: 51d: Aligned if statements with HAProxy best practices and removed casts from malloc.
    - MINOR: rename master process name in -Ds (systemd mode)
    - DOC: fix a few spelling mistakes
    - DOC: fix "workaround" spelling
    - BUG/MINOR: examples: Fixing haproxy.spec to remove references to .cfg files
    - MINOR: fix the return type for dns_response_get_query_id() function
    - MINOR: server state: missing LF (\n) on error message printed when parsing server state file
    - BUG/MEDIUM: dns: no DNS resolution happens if no ports provided to the nameserver
    - BUG/MAJOR: servers state: server port is erased when dns resolution is enabled on a server
    - BUG/MEDIUM: servers state: server port is used uninitialized
    - BUG/MEDIUM: config: Adding validation to stick-table expire value.
    - BUG/MEDIUM: sample: http_date() doesn't provide the right day of the week
    - BUG/MEDIUM: channel: fix miscalculation of available buffer space.
    - MEDIUM: pools: add a new flag to avoid rounding pool size up
    - BUG/MEDIUM: buffers: do not round up buffer size during allocation
    - BUG/MINOR: stream: don't force retries if the server is DOWN
    - BUG/MINOR: counters: make the sc-inc-gpc0 and sc-set-gpt0 touch the table
    - MINOR: unix: don't mention free ports on EAGAIN
    - BUG/CLEANUP: CLI: report the proper field states in "show sess"
    - MINOR: stats: send content-length with the redirect to allow keep-alive
    - BUG: stream_interface: Reuse connection even if the output channel is empty
    - DOC: remove old tunnel mode assumptions
    - BUG/MAJOR: http-reuse: fix risk of orphaned connections
    - BUG/MEDIUM: http-reuse: do not share private connections across backends
    - BUG/MINOR: ssl: Be sure to use unique serial for regenerated certificates
    - BUG/MINOR: stats: fix missing comma in stats on agent drain
    - MAJOR: filters: Add filters support
    - MINOR: filters: Do not reset stream analyzers if the client is gone
    - REORG: filters: Prepare creation of the HTTP compression filter
    - MAJOR: filters/http: Rewrite the HTTP compression as a filter
    - MEDIUM: filters: Use macros to call filters callbacks to speed-up processing
    - MEDIUM: filters: remove http_start_chunk, http_last_chunk and http_chunk_end
    - MEDIUM: filters: Replace filter_http_headers callback by an analyzer
    - MEDIUM: filters/http: Move body parsing of HTTP messages in dedicated functions
    - MINOR: filters: Add stream_filters structure to hide filters info
    - MAJOR: filters: Require explicit registration to filter HTTP body and TCP data
    - MINOR: filters: Remove unused or useless stuff and do small optimizations
    - MEDIUM: filters: Optimize the HTTP compression for chunk encoded response
    - MINOR: filters/http: Slightly update the parsing of chunks
    - MINOR: filters/http: Forward remaining data when a channel has no "data" filters
    - MINOR: filters: Add an filter example
    - MINOR: filters: Extract proxy stuff from the struct filter
    - MINOR: map: Add regex matching replacement
    - BUG/MINOR: lua: unsafe initialization
    - DOC: lua: fix somme errors
    - MINOR: lua: file dedicated to unsafe functions
    - MINOR: lua: add "now" time function
    - MINOR: standard: add RFC HTTP date parser
    - MINOR: lua: Add date functions
    - MINOR: lua: move common function
    - MINOR: lua: merge function
    - MINOR: lua: Add concat class
    - MINOR: standard: add function "escape_chunk"
    - MEDIUM: log: add a new log format flag "E"
    - DOC: add server name at rate-limit sessions example
    - BUG/MEDIUM: ssl: fix off-by-one in ALPN list allocation
    - BUG/MEDIUM: ssl: fix off-by-one in NPN list allocation
    - DOC: LUA: fix some typos and syntax errors
    - MINOR: cli: add a new "show env" command
    - MEDIUM: config: allow to manipulate environment variables in the global section
    - MEDIUM: cfgparse: reject incorrect 'timeout retry' keyword spelling in resolvers
    - MINOR: mailers: increase default timeout to 10 seconds
    - MINOR: mailers: use <CRLF> for all line endings
    - BUG/MAJOR: lua: segfault using Concat object
    - DOC: lua: copyrights
    - MINOR: common: mask conversion
    - MEDIUM: dns: extract options
    - MEDIUM: dns: add a "resolve-net" option which allow to prefer an ip in a network
    - MINOR: mailers: make it possible to configure the connection timeout
    - BUG/MAJOR: lua: applets can't sleep.
    - BUG/MINOR: server: some prototypes are renamed
    - BUG/MINOR: lua: Useless copy
    - BUG/MEDIUM: stats: stats bind-process doesn't propagate the process mask correctly
    - BUG/MINOR: server: fix the format of the warning on address change
    - CLEANUP: server: add "const" to some message strings
    - MINOR: server: generalize the "updater" source
    - BUG/MEDIUM: chunks: always reject negative-length chunks
    - BUG/MINOR: systemd: ensure we don't miss signals
    - BUG/MINOR: systemd: report the correct signal in debug message output
    - BUG/MINOR: systemd: propagate the correct signal to haproxy
    - MINOR: systemd: ensure a reload doesn't mask a stop
    - BUG/MEDIUM: cfgparse: wrong argument offset after parsing server "sni" keyword
    - CLEANUP: stats: Avoid computation with uninitialized bits.
    - CLEANUP: pattern: Ignore unknown samples in pat_match_ip().
    - CLEANUP: map: Avoid memory leak in out-of-memory condition.
    - BUG/MINOR: tcpcheck: fix incorrect list usage resulting in failure to load certain configs
    - BUG/MAJOR: samples: check smp->strm before using it
    - MINOR: sample: add a new helper to initialize the owner of a sample
    - MINOR: sample: always set a new sample's owner before evaluating it
    - BUG/MAJOR: vars: always retrieve the stream and session from the sample
    - CLEANUP: payload: remove useless and confusing nullity checks for channel buffer
    - BUG/MINOR: ssl: fix usage of the various sample fetch functions
    - MINOR: stats: create fields types suitable for all CSV output data
    - MINOR: stats: add all the "show info" fields in a table
    - MEDIUM: stats: fill all the show info elements prior to displaying them
    - MINOR: stats: add a function to emit fields into a chunk
    - MINOR: stats: add stats_dump_info_fields() to dump one field per line
    - MEDIUM: stats: make use of stats_dump_info_fields() for "show info"
    - MINOR: stats: add a declaration of all stats fields
    - MINOR: stats: don't hard-code the CSV fields list anymore
    - MINOR: stats: create stats fields storage and CSV dump function
    - MEDIUM: stats: convert stats_dump_fe_stats() to use stats_dump_fields_csv()
    - MEDIUM: stats: make stats_dump_fe_stats() use stats fields for HTML dump
    - MEDIUM: stats: convert stats_dump_li_stats() to use stats_dump_fields_csv()
    - MEDIUM: stats: make stats_dump_li_stats() use stats fields for HTML dump
    - MEDIUM: stats: convert stats_dump_be_stats() to use stats_dump_fields_csv()
    - MEDIUM: stats: make stats_dump_be_stats() use stats fields for HTML dump
    - MEDIUM: stats: convert stats_dump_sv_stats() to use stats_dump_fields_csv()
    - MEDIUM: stats: make stats_dump_sv_stats() use the stats field for HTML
    - MEDIUM: stats: move the server state coloring logic to the server dump function
    - MINOR: stats: do not use srv->admin & STATS_ADMF_MAINT in HTML dumps
    - MINOR: stats: do not check srv->state for SRV_ST_STOPPED in HTML dumps
    - MINOR: stats: make CSV report server check status only when enabled
    - MINOR: stats: only report backend's down time if it has servers
    - MINOR: stats: prepend '*' in front of the check status when in progress
    - MINOR: stats: make HTML stats dump rely on the table for the check status
    - MINOR: stats: add agent_status, agent_code, agent_duration to output
    - MINOR: stats: add check_desc and agent_desc to the output fields
    - MINOR: stats: add check and agent's health values in the output
    - MEDIUM: stats: make the HTML server state dump use the CSV states
    - MEDIUM: stats: only report observe errors when observe is set
    - MEDIUM: stats: expose the same flags for CLI and HTTP accesses
    - MEDIUM: stats: report server's address in the CSV output
    - MEDIUM: stats: report the cookie value in the server & backend CSV dumps
    - MEDIUM: stats: compute the color code only in the HTML form
    - MEDIUM: stats: report the listeners' address in the CSV output
    - MEDIUM: stats: make it possible to report the WAITING state for listeners
    - REORG: stats: dump the frontend's HTML stats via a generic function
    - REORG: stats: dump the socket stats via the generic function
    - REORG: stats: dump the server stats via the generic function
    - REORG: stats: dump the backend stats via the generic function
    - MEDIUM: stats: add a new "mode" column to report the proxy mode
    - MINOR: stats: report the load balancing algorithm in CSV output
    - MINOR: stats: add 3 fields to report the frontend-specific connection stats
    - MINOR: stats: report number of intercepted requests for frontend and backends
    - MINOR: stats: introduce stats_dump_one_line() to dump one stats line
    - CLEANUP: stats: make stats_dump_fields_html() not rely on proxy anymore
    - MINOR: stats: add ST_SHOWADMIN to pass the admin info in the regular flags
    - MINOR: stats: make stats_dump_fields_html() not use &trash by default
    - MINOR: stats: add functions to emit typed fields into a chunk
    - MEDIUM: stats: support "show info typed" on the CLI
    - MEDIUM: stats: implement a typed output format for stats
    - DOC: document the "show info typed" and "show stat typed" output formats
    - MINOR: cfgparse: warn when uid parameter is not a number
    - MINOR: cfgparse: warn when gid parameter is not a number
    - BUG/MINOR: standard: Avoid free of non-allocated pointer
    - BUG/MINOR: pattern: Avoid memory leak on out-of-memory condition
    - CLEANUP: http: fix a build warning introduced by a recent fix
    - BUG/MINOR: log: GMT offset not updated when entering/leaving DST
2016-03-14 00:10:05 +01:00
Willy Tarreau
5d8b979e68 DOC: document the "show info typed" and "show stat typed" output formats
These formats are more complex and are only usable if properly
documented. Let's hope it will be enough.
2016-03-11 18:05:41 +01:00