Commit Graph

462 Commits

Author SHA1 Message Date
Jamie Gloudon
801a0a353a DOC: fix name for "option independant-streams"
The correct spelling is "independent", not "independant". This patch
fixes the doc and the configuration parser to accept the correct form.
The config parser still allows the old naming for backwards compatibility.
2012-09-02 21:51:07 +02:00
Jamie Gloudon
aaa21008a7 DOC: Typos cleanup
I came across a couple of typos in configuration.txt and made this patch.
Also, there is an inconsistency between using the word analys/ze in
configuration.txt as well. However, I did not provide a patch for that.

-- Jamie Gloudon

[wt: won't fix the us/uk language mistakes, they'll always exist anyway]
2012-09-02 21:51:07 +02:00
Willy Tarreau
654694e189 MEDIUM: stats/cli: add support for "set table key" to enter values
This is used to enter values for stick tables. The most likely usage
is to set gpc0 for a specific IP address in order to block traffic
for abusers without having to reload. Since all data types are
supported, other usages are possible (eg: replace a users's assigned
server).
2012-09-02 21:51:07 +02:00
Willy Tarreau
a9fddca778 MINOR: http: add the urlp_val ACL match
It's derived from other urlp_* matches, but there was no way to check for
an integer value and it seems like it's significantly used.
2012-07-31 07:55:32 +02:00
Willy Tarreau
a7ad50cdb1 MEDIUM: pattern: add the "base" sample fetch method
This one returns the concatenation of the first Host header entry with
the path. It can make content-switching rules easier, help with fighting
DDoS on certain URLs and improve shared caches efficiency.
2012-07-26 19:08:38 +02:00
Willy Tarreau
6812bcfc94 MINOR: replace acl_fetch_{path,url}* with smp_fetch_*
Doing so allows us to support sticking on URL, URL's IP, URL's port and
path.

Both fetch functions should be improved to support an optional depth
allowing to stick to a server depending on just a few directory
components. This would help with portals, some prefetch-capable
caches and with outgoing connections using multiple internet links.
2012-07-26 19:06:40 +02:00
Vincent Bernat
6341be5b38 DOC: specify the default value for maxconn in the context of a proxy
Default value for maxconn in the context of a proxy is 2000 and is
unrelated to any other value (like global ulimit-n or global
maxconn). Without an explicit a user may think that the default value
is either no limit or equal to the global maxconn value.
2012-06-27 20:10:29 +02:00
Willy Tarreau
02c7c14ae7 [RELEASE] Released version 1.5-dev11
Released version 1.5-dev11 with the following main changes :
    - BUG/MEDIUM: option forwardfor if-none doesn't work with some configurations
    - BUG/MAJOR: trash must always be the size of a buffer
    - DOC: fix minor regex example issue and improve doc on stats
    - MINOR: stream_interface: add a pointer to the listener for TARG_TYPE_CLIENT
    - MEDIUM: protocol: add a pointer to struct sock_ops to the listener struct
    - MINOR: checks: add on-marked-up option
    - MINOR: balance uri: added 'whole' parameter to include query string in hash calculation
    - MEDIUM: stream_interface: remove the si->init
    - MINOR: buffers: add a rewind function
    - BUG/MAJOR: fix regression on content-based hashing and http-send-name-header
    - MAJOR: http: stop using msg->sol outside the parsers
    - CLEANUP: http: make it more obvious that msg->som is always null outside of chunks
    - MEDIUM: http: get rid of msg->som which is not used anymore
    - MEDIUM: http: msg->sov and msg->sol will never wrap
    - BUG/MAJOR: checks: don't call set_server_status_* when no LB algo is set
    - BUG/MINOR: stop connect timeout when connect succeeds
    - REORG: move the send-proxy code to tcp_connect_write()
    - REORG/MINOR: session: detect the TCP monitor checks at the protocol accept
    - MINOR: stream_interface: introduce a new "struct connection" type
    - REORG/MINOR: stream_interface: move si->fd to struct connection
    - REORG/MEDIUM: stream_interface: move applet->state and private to connection
    - MINOR: stream_interface: add a data channel close function
    - MEDIUM: stream_interface: call si_data_close() before releasing the si
    - MINOR: peers: use the socket layer operations from the peer instead of sock_raw
    - BUG/MINOR: checks: expire on timeout.check if smaller than timeout.connect
    - MINOR: add a new function call tracer for debugging purposes
    - BUG/MINOR: perform_http_redirect also needs to rewind the buffer
    - BUG/MAJOR: b_rew() must pass a signed offset to b_ptr()
    - BUG/MEDIUM: register peer sync handler in the proper order
    - BUG/MEDIUM: buffers: fix bi_putchr() to correctly advance the pointer
    - BUG/MINOR: fix option httplog validation with TCP frontends
    - BUG/MINOR: log: don't report logformat errors in backends
    - REORG/MINOR: use dedicated proxy flags for the cookie handling
    - BUG/MINOR: config: do not report twice the incompatibility between cookie and non-http
    - MINOR: http: add support for "httponly" and "secure" cookie attributes
    - BUG/MEDIUM: ensure that unresolved arguments are freed exactly once
    - BUG/MINOR: commit 196729ef used wrong condition resulting in freeing constants
    - MEDIUM: stats: add support for soft stop/soft start in the admin interface
    - MEDIUM: stats: add the ability to kill sessions from the admin interface
    - BUILD: add support for linux kernels >= 2.6.28
2012-06-04 00:43:45 +02:00
Justin Karneges
eb2c24ae2a MINOR: checks: add on-marked-up option
This implements the feature discussed in the earlier thread of killing
connections on backup servers when a non-backup server comes back up. For
example, you can use this to route to a mysql master & slave and ensure
clients don't stay on the slave after the master goes from down->up. I've done
some minimal testing and it seems to work.

[WT: added session flag & doc, moved the killing after logging the server UP,
 and ensured that the new server is really usable]
2012-06-03 23:48:42 +02:00
Willy Tarreau
4992dd2d30 MINOR: http: add support for "httponly" and "secure" cookie attributes
httponly  This option tells haproxy to add an "HttpOnly" cookie attribute
             when a cookie is inserted. This attribute is used so that a
             user agent doesn't share the cookie with non-HTTP components.
             Please check RFC6265 for more information on this attribute.

   secure    This option tells haproxy to add a "Secure" cookie attribute when
             a cookie is inserted. This attribute is used so that a user agent
             never emits this cookie over non-secure channels, which means
             that a cookie learned with this flag will be presented only over
             SSL/TLS connections. Please check RFC6265 for more information on
             this attribute.
2012-05-31 21:02:17 +02:00
Oskar Stolc
8dc4184c57 MINOR: balance uri: added 'whole' parameter to include query string in hash calculation
This patch brings a new "whole" parameter to "balance uri" which makes
the hash work over the whole uri, not just the part before the query
string. Len and depth parameter are still honnored.

The reason for this new feature is explained below.

I have 3 backend servers, each accepting different form of HTTP queries:

http://backend1.server.tld/service1.php?q=...
http://backend1.server.tld/service2.php?q=...

http://backend2.server.tld/index.php?query=...&subquery=...

http://backend3.server.tld/image/49b8c0d9ff

Each backend server returns a different response based on either:
- the URI path (the left part of the URI before the question mark)
- the query string (the right part of the URI after the question mark)
- or the combination of both

I wanted to set up a common caching cluster (using 6 Squid servers, each
configured as reverse proxy for those 3 backends) and have HAProxy balance
the queries among the Squid servers based on URL. I also wanted to achieve
hight cache hit ration on each Squid server and send the same queries to
the same Squid servers. Initially I was considering using the 'balance uri'
algorithm, but that would not work as in case of backend2 all queries would
go to only one Squid server. The 'balance url_param' would not work either
as it would send the backend3 queries to only one Squid server.

So I thought the simplest solution would be to use 'balance uri', but to
calculate the hash based on the whole URI (URI path + query string),
instead of just the URI path.
2012-05-22 07:56:54 +02:00
Dmitry Sivachenko
7823de3d90 DOC: fix minor regex example issue and improve doc on stats
URL rewriting should use [^\ :]* to avoid matching headers.
2012-05-16 13:49:49 +02:00
Willy Tarreau
ffb8947bb8 [RELEASE] Released version 1.5-dev10
Released version 1.5-dev10 with the following main changes :
    - BUG/MINOR: stats admin: "Unexpected result" was displayed unconditionally
    - BUG/MAJOR: acl: http_auth_group() must not accept any user from the userlist
    - CLEANUP: auth: make the code build again with DEBUG_AUTH
    - BUG/MEDIUM: config: don't crash at config load time on invalid userlist names
    - REORG: use the name sock_raw instead of stream_sock
    - MINOR: stream_interface: add a client target : TARG_TYPE_CLIENT
    - BUG/MEDIUM: stream_interface: restore get_src/get_dst
    - CLEANUP: sock_raw: remove last references to stream_sock
    - CLEANUP: stream_interface: stop exporting socket layer functions
    - MINOR: stream_interface: add an init callback to sock_ops
    - MEDIUM: stream_interface: derive the socket operations from the target
    - MAJOR: fd: remove the need for the socket layer to recheck the connection
    - MINOR: session: call the socket layer init function when a session establishes
    - MEDIUM: session: add support for tunnel timeouts
    - MINOR: standard: add a new debug macro : fddebug()
    - CLEANUP: fd: remove unused cb->b pointers in the struct fdtab
    - OPTIM: proto_http: don't enable quick-ack on empty buffers
    - OPTIM/MAJOR: ev_sepoll: process spec events after polled events
    - OPTIM/MEDIUM: stream_interface: add a new SI_FL_NOHALF flag
2012-05-14 07:26:56 +02:00
Willy Tarreau
ce887fd3b2 MEDIUM: session: add support for tunnel timeouts
Tunnel timeouts are used when TCP connections are forwarded, or
when forwarding upgraded HTTP connections (WebSocket) as well as
CONNECT requests to proxies.

This timeout allows long-lived sessions to be supported without
having to set large timeouts to normal requests.
2012-05-12 12:50:00 +02:00
Willy Tarreau
a0564f3541 [RELEASE] Released version 1.5-dev9
Released version 1.5-dev9 with the following main changes :
    - MINOR: Add release callback to si_applet
    - CLEANUP: Fix some minor typos
    - MINOR: Add TO/FROM_SET flags to struct stream_interface
    - CLEANUP: Fix some minor whitespace issues
    - MINOR: stats admin: allow unordered parameters in POST requests
    - CLEANUP: fix typo in findserver() log message
    - MINOR: stats admin: use the backend id instead of its name in the form
    - MINOR: stats admin: reduce memcmp()/strcmp() calls on status codes
    - DOC: cleanup indentation, alignment, columns and chapters
    - DOC: fix some keywords arguments documentation
    - MINOR: cli: display the 4 IP addresses and ports on "show sess XXX"
    - BUG/MAJOR: log: possible segfault with logformat
    - MEDIUM: log: split of log_format generation
    - MEDIUM: log: New format-log flags: %Fi %Fp %Si %Sp %Ts %rt %H %pid
    - MEDIUM: log: Unique ID
    - MINOR: log: log-format: usable without httplog and tcplog
    - BUG/MEDIUM: balance source did not properly hash IPv6 addresses
    - MINOR: contrib/iprange: add a network IP range to mask converter
    - MEDIUM: session: implement the "use-server" directive
    - MEDIUM: log: add a new cookie flag 'U' to report situations where cookie is not used
    - MEDIUM: http: make extract_cookie_value() iterate over cookie values
    - MEDIUM: http: add cookie and scookie ACLs
    - CLEANUP: lb_first: add reference to a paper describing the original idea
    - MEDIUM: stream_sock: add a get_src and get_dst callback and remove SN_FRT_ADDR_SET
    - BUG/MINOR: acl: req_ssl_sni would randomly fail if a session ID is present
    - BUILD: http: make extract_cookie_value() return an int not size_t
    - BUILD: http: stop gcc-4.1.2 from complaining about possibly uninitialized values
    - CLEANUP: http: message parser must ignore HTTP_MSG_ERROR
    - MINOR: standard: add a memprintf() function to build formatted error messages
    - CLEANUP: remove a few warning about unchecked return values in debug code
    - MEDIUM: move message-related flags from transaction to message
    - DOC: add a diagram to explain how circular buffers work
    - MAJOR: buffer rework: replace ->send_max with ->o
    - MAJOR: buffer: replace buf->l with buf->{o+i}
    - MINOR: buffers: provide simple pointer normalization functions
    - MINOR: buffers: remove unused function buffer_contig_data()
    - MAJOR: buffers: replace buf->w with buf->p - buf->o
    - MAJOR: buffers: replace buf->r with buf->p + buf->i
    - MAJOR: http: move buffer->lr to http_msg->next
    - MAJOR: http: change msg->{som,col,sov,eoh} to be relative to buffer origin
    - CLEANUP: http: remove unused http_msg->col
    - MAJOR: http: turn http_msg->eol to a buffer-relative offset
    - MEDIUM: http: add a pointer to the buffer in http_msg
    - MAJOR: http: make http_msg->sol relative to buffer's origin
    - MEDIUM: http: http_send_name_header: remove references to msg and buffer
    - MEDIUM: http: remove buffer arg in a few header manipulation functions
    - MEDIUM: http: remove buffer arg in http_capture_bad_message
    - MEDIUM: http: remove buffer arg in http_msg_analyzer
    - MEDIUM: http: remove buffer arg in http_upgrade_v09_to_v10
    - MEDIUM: http: remove buffer arg in http_buffer_heavy_realign
    - MEDIUM: http: remove buffer arg in chunk parsing functions
    - MINOR: http: remove useless wrapping checks in http_msg_analyzer
    - MEDIUM: buffers: fix unsafe use of buffer_ignore at some places
    - MEDIUM: buffers: add new pointer wrappers and get rid of almost all buffer_wrap_add calls
    - MEDIUM: buffers: implement b_adv() to advance a buffer's pointer
    - MEDIUM: buffers: rename a number of buffer management functions
    - MEDIUM: http: add a prefetch function for ACL pattern fetch
    - MEDIUM: http: make all ACL fetch function use acl_prefetch_http()
    - BUG/MINOR: http_auth: ACLs are volatile, not permanent
    - MEDIUM: http/acl: merge all request and response ACL fetches of headers and cookies
    - MEDIUM: http/acl: make acl_fetch_hdr_{ip,val} rely on acl_fetch_hdr()
    - MEDIUM: add a new typed argument list parsing framework
    - MAJOR: acl: make use of the new argument parsing framework
    - MAJOR: acl: store the ACL argument types in the ACL keyword declaration
    - MEDIUM: acl: acl_find_target() now resolves arguments based on their types
    - MAJOR: acl: make acl_find_targets also resolve proxy names at config time
    - MAJOR: acl: ensure that implicit table and proxies are valid
    - MEDIUM: acl: remove unused tests for missing args when args are mandatory
    - MEDIUM: pattern: replace type pattern_arg with type arg
    - MEDIUM: pattern: get rid of arg_i in all functions making use of arguments
    - MEDIUM: pattern: use the standard arg parser
    - MEDIUM: pattern: add an argument validation callback to pattern descriptors
    - MEDIUM: pattern: report the precise argument parsing error when known.
    - MEDIUM: acl: remove the ACL_TEST_F_NULL_MATCH flag
    - MINOR: pattern: add a new 'sample' type to store fetched data
    - MEDIUM: pattern: add new sample types to replace pattern types
    - MAJOR: acl: make use of the new sample struct and get rid of acl_test
    - MEDIUM: pattern/acl: get rid of temp_pattern in ACLs
    - MEDIUM: acl: get rid of the SET_RES flags
    - MEDIUM: get rid of SMP_F_READ_ONLY and SMP_F_MUST_FREE
    - MINOR: pattern: replace struct pattern with struct sample
    - MEDIUM: pattern: integrate pattern_data into sample and use sample everywhere
    - MEDIUM: pattern: retrieve the sample type in the sample, not in the keyword description
    - MEDIUM: acl/pattern: switch rdp_cookie functions stack up-down
    - MEDIUM: acl: replace acl_expr with args in acl fetch_* functions
    - MINOR: tcp: replace acl_fetch_rdp_cookie with smp_fetch_rdp_cookie
    - MEDIUM: acl/pattern: use the same direction scheme
    - MEDIUM: acl/pattern: start merging common sample fetch functions
    - MEDIUM: pattern: ensure that sample types always cast into other types.
    - MEDIUM: acl/pattern: factor out the src/dst address fetches
    - MEDIUM: acl: implement payload and payload_lv
    - CLEANUP: pattern: ensure that payload and payload_lv always stay in the buffer
    - MINOR: stick_table: centralize the handling of empty keys
    - MINOR: pattern: centralize handling of unstable data in pattern_process()
    - MEDIUM: pattern: use smp_fetch_rdp_cookie instead of the pattern specific version
    - MINOR: acl: set SMP_OPT_ITERATE on fetch functions
    - MINOR: acl: add a val_args field to keywords
    - MINOR: proto_tcp: validate arguments of payload and payload_lv ACLs
    - MEDIUM: http: merge acl and pattern header fetch functions
    - MEDIUM: http: merge ACL and pattern cookie fetches into a single one
    - MEDIUM: acl: report parsing errors to the caller
    - MINOR: arg: improve error reporting on invalid arguments
    - MINOR: acl: report errors encountered when loading patterns from files
    - MEDIUM: acl: extend the pattern parsers to report meaningful errors
    - REORG: use the name "sample" instead of "pattern" to designate extracted data
    - REORG: rename "pattern" files
    - MINOR: acl: add types to ACL patterns
    - MINOR: standard: add an IPv6 parsing function (str62net)
    - MEDIUM: acl: support IPv6 address matching
    - REORG: stream_interface: create a struct sock_ops to hold socket operations
    - REORG/MEDIUM: move protocol->{read,write} to sock_ops
    - REORG/MEDIUM: stream_interface: initialize socket ops from descriptors
    - REORG/MEDIUM: replace stream interface protocol functions by a proto pointer
    - REORG/MEDIUM: move the default accept function from sockstream to protocols.c
    - MEDIUM: proto_tcp: remove src6 and dst6 pattern fetch methods
    - BUG/MINOR: http: error snapshots are wrong if buffer wraps
    - BUG/MINOR: http: ensure that msg->err_pos is always relative to buf->p
    - MEDIUM: http: improve error capture reports
    - MINOR: acl: add the cook_val() match to match a cookie against an integer
    - BUG/MEDIUM: send_proxy: fix initialisation of send_proxy_ofs
    - MEDIUM: memory: add the ability to poison memory at run time
    - BUG/MEDIUM: log: ensure that unique_id is properly initialized
    - MINOR: cfgparse: use a common errmsg pointer for all parsers
    - MEDIUM: cfgparse: make backend_parse_balance() use memprintf to report errors
    - MEDIUM: cfgparse: use the new error reporting framework for remaining cfg_keywords
    - MINOR: http: replace http_message_realign() with  buffer_slow_realign()
2012-05-08 21:56:27 +02:00
Willy Tarreau
515393649c MINOR: acl: add the cook_val() match to match a cookie against an integer 2012-05-08 21:28:16 +02:00
Willy Tarreau
22bca61404 MEDIUM: proto_tcp: remove src6 and dst6 pattern fetch methods
These methods have been superseded by src and dst which support
multiple families. There is no point keeping them since they appeared
in a development version anyway.

For configurations using "src6", please use "src" instead. For "dst6",
use "dst" instead.
2012-05-08 21:28:15 +02:00
Willy Tarreau
ceb4ac9c34 MEDIUM: acl: support IPv6 address matching
Make use of the new IPv6 pattern type so that acl_match_ip() knows how to
compare pattern and sample.

IPv6 may be entered in their usual form, with or without a netmask appended.
Only bit counts are accepted for IPv6 netmasks. In order to avoid any risk of
trouble with randomly resolved IP addresses, host names are never allowed in
IPv6 patterns.

HAProxy is also able to match IPv4 addresses with IPv6 addresses in the
following situations :
  - tested address is IPv4, pattern address is IPv4, the match applies
    in IPv4 using the supplied mask if any.
  - tested address is IPv6, pattern address is IPv6, the match applies
    in IPv6 using the supplied mask if any.
  - tested address is IPv6, pattern address is IPv4, the match applies in IPv4
    using the pattern's mask if the IPv6 address matches with 2002:IPV4::,
    ::IPV4 or ::ffff:IPV4, otherwise it fails.
  - tested address is IPv4, pattern address is IPv6, the IPv4 address is first
    converted to IPv6 by prefixing ::ffff: in front of it, then the match is
    applied in IPv6 using the supplied IPv6 mask.
2012-05-08 21:28:14 +02:00
Willy Tarreau
28376d62cb MEDIUM: http: merge ACL and pattern cookie fetches into a single one
It's easy to merge pattern and ACL fetches of cookies. It allows us
to remove two distinct fetch functions. The new function internally
uses an occurrence number to serve both purposes, but it didn't appear
worth exposing it outside so there is no keyword argument to set it.
However one of the benefits is that the "cookie" fetch for stick tables
now automatically adapts to requests and responses, so there is no more
need for set-cookie().
2012-05-08 20:57:19 +02:00
Willy Tarreau
185b5c4a7b MEDIUM: http: merge acl and pattern header fetch functions
HTTP header fetch is now done using smp_fetch_hdr() for both ACLs and
patterns. This one also supports an occurrence number, making it possible
to specify explicit occurrences for ACLs and patterns.
2012-05-08 20:57:19 +02:00
Willy Tarreau
0ce3aa0c66 MEDIUM: acl: implement payload and payload_lv
These ones were easy to adapt to ACL usage and may really be useful,
so let's make them available right now. It's likely that some extension
such as regex, string-to-IP and raw IP matching will be implemented in
the near future.
2012-05-08 20:57:17 +02:00
Willy Tarreau
25c1ebc0c9 MEDIUM: acl/pattern: start merging common sample fetch functions
src_port, dst_port and url_param have converged between ACLs and patterns.
This means that src_port is now available in patterns and that urlp_* has
been added to ACLs. Some code has moved to accommodate for static function
definitions, but there were little changes.
2012-05-08 20:57:17 +02:00
Willy Tarreau
c0239e0425 MEDIUM: http: make all ACL fetch function use acl_prefetch_http()
All ACLs which need to process HTTP contents first call this function which
performs all the preliminary tests and also triggers the request parsing if
needed. A macro was written to simplify the code.

As a side effect, it's not required anymore to check for the HTTP ACL before
checking for HTTP contents.
2012-05-08 20:57:10 +02:00
Willy Tarreau
1122d9c03c DOC: add a diagram to explain how circular buffers work
Also add some thoughts about the existing and new design.

Note: an earlier design used the names "head" and "tail" for both sides
of the buffer, but it appears awkward as these words may be understood
in two forms (feed by head, output by tail, or make the newcomers wait
at the tail of the queue). Also there were already a few functions in the
code making use of either terminology. So better avoid this terminology
and use "input" and "output" instead.
2012-04-30 11:57:00 +02:00
Cyril Bonté
108cf6ea99 DOC: fix some keywords arguments documentation
- Typo on "dispatch" keyword arguments.
- Reindent some blocks for better parsing by automated tools.
- "option mysql-check" and "option pgsql-check" arguments were not documented
  as the others.
2012-04-24 00:26:06 +02:00
Cyril Bonté
dc4d903640 DOC: cleanup indentation, alignment, columns and chapters
This patch is a group commit simplify the parsing of the documenation :
- remove remaining tabulations
- realign some lines
- break lines at 80 columns
- add missing chapters in the summary
- fix chapter numbering format
2012-04-08 23:18:54 +02:00
William Lallemand
a73203e3dc MEDIUM: log: Unique ID
The Unique ID, is an ID generated with several informations. You can use
a log-format string to customize it, with the "unique-id-format" keyword,
and insert it in the request header, with the "unique-id-header" keyword.
2012-04-07 16:25:26 +02:00
William Lallemand
5f2324019d MEDIUM: log: New format-log flags: %Fi %Fp %Si %Sp %Ts %rt %H %pid
%Fi: Frontend IP
%Fp: Frontend Port
%Si: Server IP
%Sp: Server Port
%Ts: Timestamp
%rt: HTTP request counter
%H: hostname
%pid: PID

+X: Hexadecimal represenation

The +X mode in logformat displays hexadecimal for the following flags
%Ci %Cp %Fi %Fp %Bi %Bp %Si %Sp %Ts %ct %pid

rename logformat_write_string() to lf_text()

Optimize size computation
2012-04-07 16:05:39 +02:00
Willy Tarreau
64559c565f CLEANUP: lb_first: add reference to a paper describing the original idea
The original idea behind this implementation has been published in the
paper below :

   http://reports-archive.adm.cs.cmu.edu/anon/2012/CMU-CS-12-109.pdf
2012-04-07 09:08:45 +02:00
Willy Tarreau
04aa6a9ce8 MEDIUM: http: add cookie and scookie ACLs
The ACL matches rely on the extract_cookie_value() function as used for
for patterns. This permits ACLs to match cookie values based on the cookie
name instead of having to perform substring matching on the cookie header.
2012-04-07 08:47:26 +02:00
Willy Tarreau
c89ccb6221 MEDIUM: log: add a new cookie flag 'U' to report situations where cookie is not used
This happens when a "use-server" rule sets the server instead.
2012-04-05 21:18:22 +02:00
Willy Tarreau
4a5cadea40 MEDIUM: session: implement the "use-server" directive
Sometimes it is desirable to forward a particular request to a specific
server without having to declare a dedicated backend for this server. This
can be achieved using the "use-server" rules. These rules are evaluated after
the "redirect" rules and before evaluating cookies, and they have precedence
on them. There may be as many "use-server" rules as desired. All of these
rules are evaluated in their declaration order, and the first one which
matches will assign the server.
2012-04-05 21:14:10 +02:00
Willy Tarreau
9eeb57bd7f [RELEASE] Released version 1.5-dev8
Released version 1.5-dev8 with the following main changes :
    - MINOR: patch for minor typo (ressources/resources)
    - MEDIUM: http: add support for sending the server's name in the outgoing request
    - DOC: mention that default checks are TCP connections
    - BUG/MINOR: fix options forwardfor if-none when an alternative header name is specified
    - CLEANUP: Make check_statuses, analyze_statuses and process_chk static
    - CLEANUP: Fix HCHK spelling errors
    - BUG/MINOR: fix typo in processing of http-send-name-header
    - MEDIUM: log: Use linked lists for loggers
    - BUILD: fix declaration inside a scope block
    - REORG: log: split send_log function
    - MINOR: config: Parse the string of the log-format config keyword
    - MINOR: add ultoa, ulltoa, ltoa, lltoa implementations
    - MINOR: Date and time fonctions that don't use snprintf
    - MEDIUM: log: make http_sess_log use log_format
    - DOC: log-format documentation
    - MEDIUM: log: use log_format for mode tcplog
    - MEDIUM: log-format: backend source address %Bi %Bp
    - BUG/MINOR: log-format: fix %o flag
    - BUG/MEDIUM: bad length in log_format and __send_log
    - MINOR: logformat %st is signed
    - BUILD/MINOR: fix the source URL in the spec file
    - DOC: acl is http_first_req, not http_req_first
    - BUG/MEDIUM: don't trim last spaces from headers consisting only of spaces
    - MINOR: acl: add new matches for header/path/url length
    - BUILD: halog: make halog build on solaris
    - BUG/MINOR: don't use a wrong port when connecting to a server with mapped ports
    - MINOR: remove the client/server side distinction in SI addresses
    - MINOR: halog: add support for matching queued requests
    - DOC: indicate that cookie "prefix" and "indirect" should not be mixed
    - OPTIM/MINOR: move struct sockaddr_storage to the tail of structs
    - OPTIM/MINOR: make it possible to change pipe size (tune.pipesize)
    - BUILD/MINOR: silent a build warning in src/pipe.c (fcntl)
    - OPTIM/MINOR: move the hdr_idx pools out of the proxy struct
    - MEDIUM: tune.http.maxhdr makes it possible to configure the maximum number of HTTP headers
    - BUG/MINOR: fix a segfault when parsing a config with undeclared peers
    - CLEANUP: rename possibly confusing struct field "tracked"
    - BUG/MEDIUM: checks: fix slowstart behaviour when server tracking is in use
    - MINOR: config: tolerate server "cookie" setting in non-HTTP mode
    - MEDIUM: buffers: add some new primitives and rework existing ones
    - BUG: buffers: don't return a negative value on buffer_total_space_res()
    - MINOR: buffers: make buffer_pointer() support negative pointers too
    - CLEANUP: kill buffer_replace() and use an inline instead
    - BUG: tcp: option nolinger does not work on backends
    - CLEANUP: ebtree: remove a few annoying signedness warnings
    - CLEANUP: ebtree: clarify licence and update to 6.0.6
    - CLEANUP: ebtree: remove 4-year old harmless typo in duplicates insertion code
    - CLEANUP: ebtree: remove another typo, a wrong initialization in insertion code
    - BUG: ebtree: ebst_lookup() could return the wrong entry
    - OPTIM: stream_sock: reduce the amount of in-flight spliced data
    - OPTIM: stream_sock: save a failed recv syscall when splice returns EAGAIN
    - MINOR: acl: add support for TLS server name matching using SNI
    - BUG: http: re-enable TCP quick-ack upon incomplete HTTP requests
    - BUG: proto_tcp: don't try to bind to a foreign address if sin_family is unknown
    - MINOR: pattern: export the global temporary pattern
    - CLEANUP: patterns: get rid of pattern_data_setstring()
    - MEDIUM: acl: use temp_pattern to store fetched information in the "method" match
    - MINOR: acl: include pattern.h to make pattern migration more transparent
    - MEDIUM: pattern: change the pattern data integer from unsigned to signed
    - MEDIUM: acl: use temp_pattern to store any integer-type information
    - MEDIUM: acl: use temp_pattern to store any address-type information
    - CLEANUP: acl: integer part of acl_test is not used anymore
    - MEDIUM: acl: use temp_pattern to store any string-type information
    - CLEANUP: acl: remove last data fields from the acl_test struct
    - MEDIUM: http: replace get_ip_from_hdr2() with http_get_hdr()
    - MEDIUM: patterns: the hdr() pattern is now of type string
    - DOC: add minimal documentation on how ACLs work internally
    - DOC: add a coding-style file
    - OPTIM: halog: keep a fast path for the lines-count only
    - CLEANUP: silence a warning when building on sparc
    - BUG: http: tighten the list of allowed characters in a URI
    - MEDIUM: http: block non-ASCII characters in URIs by default
    - DOC: add some documentation from RFC3986 about URI format
    - BUG/MINOR: cli: correctly remove the whole table on "clear table"
    - BUG/MEDIUM: correctly disable servers tracking another disabled servers.
    - BUG/MEDIUM: zero-weight servers must not dequeue requests from the backend
    - MINOR: halog: add some help on the command line
    - BUILD: fix build error on FreeBSD
    - BUG: fix double free in peers config error path
    - MEDIUM: improve config check return codes
    - BUILD: make it possible to look for pcre in the default system paths
    - MINOR: config: emit a warning when 'default_backend' masks servers
    - MINOR: backend: rework the LC definition to support other connection-based algos
    - MEDIUM: backend: add the 'first' balancing algorithm
    - BUG: fix httplog trailing LF
    - MEDIUM: increase chunk-size limit to 2GB-1
    - BUG: queue: fix dequeueing sequence on HTTP keep-alive sessions
    - BUG: http: disable TCP delayed ACKs when forwarding content-length data
    - BUG: checks: fix server maintenance exit sequence
    - BUG/MINOR: stream_sock: don't remove BF_EXPECT_MORE and BF_SEND_DONTWAIT on partial writes
    - DOC: enumerate valid status codes for "observe layer7"
    - MINOR: buffer: switch a number of buffer args to const
    - CLEANUP: silence signedness warning in acl.c
    - BUG: stream_sock: si->release was not called upon shutw()
    - MINOR: log: use "%ts" to log term status only and "%tsc" to log with cookie
    - BUG/CRITICAL: log: fix risk of crash in development snapshot
    - BUG/MAJOR: possible crash when using capture headers on TCP frontends
    - MINOR: config: disable header captures in TCP mode and complain
2012-03-26 06:16:43 +02:00
Adrian Bridgett
afdb6e57f7 MINOR: patch for minor typo (ressources/resources)
The main stats page says "ressources" (French spelling) rather than
"resources" (English spelling).

One little patch attached (against v1.4.20).

Many thanks,

Adrian
2012-03-21 07:54:41 +01:00
Willy Tarreau
6580c06ba3 MINOR: log: use "%ts" to log term status only and "%tsc" to log with cookie
The difference could be seen when logging a request in HTTP mode with option
tcplog, as it would keep emitting 4 chars. Better use two distinct flags to
clear the confusion.
2012-03-12 15:50:53 +01:00
William Lallemand
b7ff6a3a36 MEDIUM: log-format: backend source address %Bi %Bp
%Bi return the backend source IP
%Bp return the backend source port

Add a function pointer in logformat_type to do additional configuration
during the log-format variable parsing.
2012-03-12 15:50:52 +01:00
William Lallemand
bddd4fd93b MEDIUM: log: use log_format for mode tcplog
Merge http_sess_log() and tcp_sess_log() to sess_log() and move it to
log.c

A new field in logformat_type define if you can use a logformat
variable in TCP or HTTP mode.

doc: log-format in tcp mode

Note that due to the way log buffer allocation currently works, trying to
log an HTTP request without "option httplog" is still not possible. This
will change in the near future.
2012-03-12 15:47:13 +01:00
Willy Tarreau
150d146d4d DOC: enumerate valid status codes for "observe layer7" 2012-03-10 08:19:02 +01:00
Willy Tarreau
f09c6603d3 MEDIUM: backend: add the 'first' balancing algorithm
The principle behind this load balancing algorithm was first imagined
and modeled by Steen Larsen then iteratively refined through several
work sessions until it would totally address its original goal.

The purpose of this algorithm is to always use the smallest number of
servers so that extra servers can be powered off during non-intensive
hours. Additional tools may be used to do that work, possibly by
locally monitoring the servers' activity.

The first server with available connection slots receives the connection.
The servers are choosen from the lowest numeric identifier to the highest
(see server parameter "id"), which defaults to the server's position in
the farm. Once a server reaches its maxconn value, the next server is used.
It does not make sense to use this algorithm without setting maxconn. Note
that it can however make sense to use minconn so that servers are not used
at full load before starting new servers, and so that introduction of new
servers requires a progressively increasing load (the number of servers
would more or less follow the square root of the load until maxconn is
reached). This algorithm ignores the server weight, and is more beneficial
to long sessions such as RDP or IMAP than HTTP, though it can be useful
there too.
2012-02-21 22:27:27 +01:00
William Lallemand
4894040fa0 DOC: log-format documentation 2012-02-09 17:03:28 +01:00
Patrick Mézard
b7aeec6333 DOC: mention that default checks are TCP connections 2012-01-29 14:16:25 +01:00
Willy Tarreau
2f1feb99a5 DOC: add some documentation from RFC3986 about URI format 2012-01-07 23:58:54 +01:00
Willy Tarreau
422246eb26 MEDIUM: http: block non-ASCII characters in URIs by default
These ones are invalid and blocked unless "option accept-invalid-http-request"
is specified in the frontend. In any case, the faulty request is logged.

Note that some of the remaining invalid chars are still not checked against,
those are the invalid ones between 32 and 127 :

    34 ('"'), 60 ('<'), 62 ('>'), 92 ('\'), 94 ('^'),
    96 ('`'), 123 ('{'), 124 ('|'), 125 ('}')

Using a lookup table might be better at some point.
2012-01-07 23:55:20 +01:00
Mark Lamourine
c2247f0b8d MEDIUM: http: add support for sending the server's name in the outgoing request
New option "http-send-name-header" specifies the name of a header which
will hold the server name in outgoing requests. This is the name of the
server the connection is really sent to, which means that upon redispatches,
the header's value is updated so that it always matches the server's name.
2012-01-05 15:17:31 +01:00
Willy Tarreau
7f051b39d2 DOC: add a coding-style file
This will help newcomers adapt to existing (or desired) style and will
save them some time.
2011-12-30 17:33:27 +01:00
Willy Tarreau
6cee7ddd5e DOC: add minimal documentation on how ACLs work internally
This will help complete the ACL to pattern migration.
2011-12-30 17:33:27 +01:00
Willy Tarreau
e428fb7b4e MEDIUM: patterns: the hdr() pattern is now of type string
This pattern previously was limited to type IP. With the new header
extraction function, it becomes possible to extract strings, so that
the header can be returned as a string. This will not change anything
to existing configs, as string will automatically be converted to IP
when needed. However, new configs will be able to use IPv6 addresses
from headers in stick-tables, as well as stick on any non-IP header
(eg: host, user-agent, ...).
2011-12-30 17:33:27 +01:00
Willy Tarreau
b6672b547a MINOR: acl: add support for TLS server name matching using SNI
Server Name Indication (SNI) is a TLS extension which makes a client
present the name of the server it is connecting to in the client hello.
It allows a transparent proxy to take a decision based on the beginning
of an SSL/TLS stream without deciphering it.

The new ACL "req_ssl_sni" matches the name extracted from the TLS
handshake against a list of names which may be loaded from a file if
needed.
2011-12-12 17:26:23 +01:00
William Lallemand
0f99e34978 MEDIUM: log: Use linked lists for loggers
This patch settles the 2 loggers limitation.
Loggers are now stored in linked lists.

Using "global log", the global loggers list content is added at the end
of the current proxy list. Each "log" entries are added at the end of
the proxy list.

"no log" flush a logger list.
2011-10-31 14:09:19 +01:00
Willy Tarreau
ac1932da3e MEDIUM: tune.http.maxhdr makes it possible to configure the maximum number of HTTP headers
For a long time, the max number of headers was taken as a part of the buffer
size. Since the header size can be configured at runtime, it does not make
much sense anymore.

Nothing was making it necessary to have a static value, so let's turn this into
a tunable with a default value of 101 which equals what was previously used.
2011-10-24 19:14:41 +02:00