Commit Graph

4982 Commits

Author SHA1 Message Date
Thierry FOURNIER
a6b6343cff CLEANUP: http/tcp actions: remove the scope member
The scope member is not used. This patch removes this entry.
2015-08-11 13:44:53 +02:00
Thierry FOURNIER
9b49f589ed CLEANUP: proto_http: remove useless initialisation
This initialisation of the opaque array is useless.
2015-08-11 13:44:51 +02:00
Willy Tarreau
a482746a29 DOC: add design thoughts on dynamic buffer allocation
This is doc/design-thoughts/dynamic-buffers.txt.
2015-08-11 12:17:42 +02:00
Willy Tarreau
5c01782340 DOC: add some thoughts on connection sharing for HTTP/2
This is doc/design-thoughts/connection-sharing.txt.
2015-08-11 12:17:41 +02:00
Willy Tarreau
e607df3f48 DOC: add design thoughts on HTTP/2
This is doc/design-thoughts/http2.txt.
2015-08-11 12:17:41 +02:00
Willy Tarreau
11dc26f4df DOC: add doc/linux-syn-cookies.txt
This is an analysis of how kernel 3.10 decides to trigger SYN cookies.
2015-08-11 12:17:41 +02:00
Willy Tarreau
4658c4dfe5 DOC: add doc/internals/entities-v2.txt
This contains some thoughts about how to improve connection management
for 1.5 in order to support SSL.
2015-08-11 12:17:41 +02:00
Willy Tarreau
f3934b8c6d MEDIUM: config: emit a warning on a frontend without listener
Commit c6678e2 ("MEDIUM: config: authorize frontend and listen without bind")
completely removed the test for bind lines in frontends in order to make it
easier for automated tools to generate configs (eg: replacing a bind with
another one passing via a temporary config without any bind line). The
problem is that some common mistakes are totally hidden now. For example,
this apparently valid entry is silently ignored :

       listen 1.2.3.4:8000
           server s1 127.0.0.1:8000

Hint: 1.2.3.4:8000 is mistakenly the proxy name here.

Thus instead we now emit a warning to indicate that a frontend was found
with no listener. This should be backported to 1.5 to help spot abnormal
configurations.
2015-08-11 12:17:41 +02:00
Willy Tarreau
80c2f2f024 CLEANUP: .gitignore: finally ignore everything but what is known.
Still too many files remain, it's easier to block everything but
what we know.
2015-08-11 12:17:41 +02:00
Willy Tarreau
921405e4a8 CLEANUP: .gitignore: ignore more test files
Exclude from "git status" many of the files that often result from
development tests and bug reports reproducers.
2015-08-11 12:17:41 +02:00
Willy Tarreau
294d0f08b3 DOC: remove documentation about appsession
Some example of typical conversions would be helpful. Probably we need
to start from Aleks' proposals.
2015-08-10 19:45:10 +02:00
Willy Tarreau
de39c9b10f CLEANUP: appsession: remove the last include files
These ones were include/common/appsession.h and include/common/sessionhash.h.
2015-08-10 19:42:30 +02:00
Willy Tarreau
3580b03fc0 CLEANUP: proxy: remove last references to appsession
The code entirely builds without using these elements, let's get rid
of them now.
2015-08-10 19:42:30 +02:00
Willy Tarreau
d38220c3ca CLEANUP: tests: remove sessionhash_test.c and test-cookie-appsess.cfg
These won't be usable anymore.
2015-08-10 19:42:11 +02:00
Willy Tarreau
9496552e7d CLEANUP: appsession: remove appsession.c and sessionhash.c
Now there's no more code using appsessions we can remove them.
2015-08-10 19:17:47 +02:00
Willy Tarreau
6db62c58d6 CLEANUP: config: remove appsession initialization
Now it asks to check the documentation.
2015-08-10 19:16:18 +02:00
Willy Tarreau
53a09d520e MAJOR: http: remove references to appsession
appsessions started to be deprecated with the introduction of stick
tables, and the latter are much more powerful and flexible, and in
addition they are replicated between nodes and maintained across
reloads. Let's now remove appsession completely.
2015-08-10 19:16:18 +02:00
Thierry FOURNIER
61e96c68b9 BUG/MEDIUM: lua: timeout error with converters, wrapper and actions.
test conf:

   global
      tune.lua.session-timeout 0
      lua-load lol.lua
      debug
      maxconn 4096

   listen test
	   bind 0.0.0.0:10010
		mode tcp
		tcp-request content lua act_test
		balance roundrobin
		server test 127.0.0.1:3304

lua test:

   function act_test(txn)
      while true do
         core.Alert("TEST")
      end
   end

The function "act_test()" is not executed because a zero timeout is not
considered as TICK_ETERNITY, but is considered as 0.

This path fix this behavior. This is the same problem than the bugfix
685c014e99.
2015-08-10 06:18:30 +02:00
Willy Tarreau
f2ee0162c3 MINOR: init: indicate to check 'bind' lines when no listeners were found.
It used to still mention '<listen>'.
2015-08-09 11:01:51 +02:00
Willy Tarreau
812c88ec12 DOC: mention that %ms is left-padded with zeroes.
That's important to emit logs.
2015-08-09 10:56:35 +02:00
Camilo Lopez
685c014e99 BUG/MEDIUM: lua: Lua tasks fail to start.
I've been trying out 1.6 dev3 with lua support, and trying to start
lua tasks seems to not be working.

Using this configuration

global
  lua-load /lua/lol.lua
  debug
  maxconn 4096

backend shard_b
  server db01 mysql_shard_b:3306

backend shard_a
  server db01 mysql_shard_a:3306

listen mysql-cluster
  bind 0.0.0.0:8001
  mode tcp
  balance roundrobin
  use_backend shard_b

And this lua function

  core.register_task(function()
    while true do
      core.Alert("LOLOLOLOLOL")
    end
  end)

I'd always get a timeout error starting the registered function.

The problem lies as far as I can tell in the fact that is possible for
now_ms to not change (is this maybe a problem on my config/system?)
until the expiration check happens, in the resume function that
actually kickstarts the lua task, making HAProxy think that expiration
time for the task is up, if I understand correctly tasks are meant to
never really timeout.
2015-08-09 10:22:34 +02:00
Andrew Hayworth
e63ac871f8 MINOR: log: Add log-format variable %HQ, to log HTTP query strings
Since sample fetches are not always available in the response phase,
this patch implements %HQ such that:

  GET /foo?bar=baz HTTP/1.0

...would be logged as:

  ?bar=baz
2015-08-09 10:16:49 +02:00
Baptiste Assmann
9f5ada32e4 MINOR: server: add new SRV_ADMF_CMAINT flag
The purpose of SRV_ADMF_CMAINT flag is to keep in mind the server was
forced to maintenance status because of the configuration file.
2015-08-08 18:18:17 +02:00
Baptiste Assmann
2359ff1de2 BUG/MEDIUM: DNS resolution response parsing broken
In some cases, parsing of the DNS response is broken and the response is
considered as invalid, despite being valid.

The current patch fixes this issue. It's a temporary solution until I
rework the response parsing to store the response buffer into a real DNS
packet structure.
2015-08-08 18:14:20 +02:00
Baptiste Assmann
37bb372ea2 MINOR: DNS counters: increment valid counter
Valid counter was never incremented.
Now it is.
2015-08-08 18:13:59 +02:00
Dragan Dosen
9373fc5811 MINOR: 51d: unable to start haproxy without "51degrees-data-file"
This patch adds a few checks on "global._51degrees.data_file_path" and allows
haproxy to start even when the pattern or trie data file is not specified.

If the "51d" converter is used, a new function "_51d_conv_check" will check
"global._51degrees.data_file_path" and displays a warning if necessary.

In src/haproxy.c, the global 51Degrees "cache_size" has moved outside of the
FIFTYONEDEGREES_H_PATTERN_INCLUDED ifdef block.
2015-08-08 12:15:55 +02:00
Willy Tarreau
067fceffb3 DOC: internals: document next steps for HTTP connection reuse
This is mostly based on the design notes and experiments that were
not turned into final code yet.
2015-08-06 16:40:59 +02:00
Willy Tarreau
30631956d6 DOC: document the new http-reuse directive
This documents the 4 strategies : never, safe, aggressive, always.
2015-08-06 16:35:03 +02:00
Willy Tarreau
449d74a906 MEDIUM: backend: add the "http-reuse aggressive" strategy
This strategy is less extreme than "always", it only dispatches first
requests to validated reused connections, and moves a connection from
the idle list to the safe list once it has seen a second request, thus
proving that it could be reused.
2015-08-06 16:29:01 +02:00
Willy Tarreau
7017cb040c MINOR: server: add a list of safe, already reused idle connections
These ones are considered safe as they have already been reused.
They will be useful in "aggressive" and "always" http-reuse modes
in order to place the first request of a connection with the least
risk.
2015-08-06 16:29:01 +02:00
Willy Tarreau
161d45ffc7 MEDIUM: backend: implement "http-reuse safe"
The "safe" mode consists in picking existing connections only when
processing a request that's not the first one from a connection. This
ensures that in case where the server finally times out and closes, the
client can decide to replay idempotent requests.
2015-08-06 11:50:53 +02:00
Willy Tarreau
efb90f9dd3 MAJOR: backend: improve the connection reuse mechanism
Now instead of closing the existing connection attached to the
stream interface, we first check if the one we pick was attached to
another stream interface, in which case the connections are swapped
if possible (eg: if the current connection is not private). That way
the previous connection remains attached to an existing session and
significantly increases the chances of being reused.
2015-08-06 11:41:06 +02:00
Willy Tarreau
8dff998b91 MAJOR: backend: initial work towards connection reuse
In connect_server(), if we don't have a connection attached to the
stream-int, we first look into the server's idle_conns list and we
pick the first one there, we detach it from its owner if it had one.
If we used to have a connection, we close it.

This mechanism works well but doesn't scale : as servers increase,
the likeliness that the connection attached to the stream interface
doesn't match the server and gets closed increases.
2015-08-06 11:34:21 +02:00
Willy Tarreau
d8fecee291 MINOR: config: add new setting "http-reuse"
For now it only supports "never", meaning that we never want to reuse a
shared connection, and "always", meaning that we can use any connection
that was not marked private. When "never" is set, this also implies that
no idle connection may become a shared one.
2015-08-06 11:14:20 +02:00
Willy Tarreau
387ebf84dd MINOR: connection: add a new flag CO_FL_PRIVATE
This flag is set on an outgoing connection when this connection gets
some properties that must not be shared with other connections, such
as dynamic transparent source binding, SNI or a proxy protocol header,
or an authentication challenge from the server. This will be needed
later to implement connection reuse.
2015-08-06 11:14:17 +02:00
Willy Tarreau
173a1c6b43 MINOR: server: add a list of already used idle connections
There's a difference with the other idle conns in that these new
ones have already been used and may be reused by other streams.
2015-08-06 11:13:47 +02:00
Willy Tarreau
4320eaac62 MINOR: stream-int: make si_idle_conn() only accept valid connections
This function is now dedicated to idle connections only, which means
that it must not be used without any endpoint nor anything not a
connection. The connection remains attached to the stream interface.
2015-08-06 11:11:10 +02:00
Willy Tarreau
323a2d925c MEDIUM: stream-int: queue idle connections at the server
Now we get a per-server list of all idle connections. That way we'll
be able to reclaim them upon shortage later.
2015-08-06 11:06:25 +02:00
Willy Tarreau
d75d40e9a8 MINOR: connection: add a new list member in the connection struct
This list member will be used to attach a connection to a list of
idle, reusable or queued connections. It's unused for now. Given
that it's not expected to be used more than a few times per session,
the member was put after the target, in the area starting at the
second cache line of the structure.
2015-08-06 11:00:34 +02:00
Willy Tarreau
600802aef0 MINOR: server: add a list of private idle connections
For now it's not populated but we have the list entry. It will carry
all idle connections that sessions don't want to share. They may be
used later to reclaim connections upon socket shortage for example.
2015-08-06 10:59:08 +02:00
Willy Tarreau
41b09f4bbc MINOR: stream-int: add new function si_detach_endpoint()
This function only detaches the endpoint from the stream-int and
optionally returns the original pointer. This will be needed to
steal idle connections from other connections.
2015-08-05 21:54:25 +02:00
Willy Tarreau
973a54235f MEDIUM: stream-int: simplify si_alloc_conn()
Since we now always call this function with the reuse parameter cleared,
let's simplify the function's logic as it cannot return the existing
connection anymore. The savings on this inline function are appreciable
(240 bytes) :

$ size haproxy.old haproxy.new
   text    data     bss     dec     hex filename
1020383   40816   36928 1098127  10c18f haproxy.old
1020143   40816   36928 1097887  10c09f haproxy.new
2015-08-05 21:51:09 +02:00
Willy Tarreau
c12b5e663d MEDIUM: backend: don't call si_alloc_conn() when we reuse a valid connection
connect_server() already does most of the check that is done again in
si_alloc_conn(), so let's simply reuse the existing connection instead
of calling the function again. It will also simplify the connection
reuse.

Indeed, for reuse to be set, it also requires srv_conn to be valid. In the
end, the only situation where we have to release the existing connection
and allocate a new one is when reuse == 0.
2015-08-05 21:42:12 +02:00
Willy Tarreau
7b00492ce3 CLEANUP: backend: factor out objt_server() in connect_server()
objt_server() is called multiple times at various places while some
places already make use of srv for this. Let's move the call at the
top of the function and use it all over the place.
2015-08-05 10:12:47 +02:00
Baptiste Assmann
c4aabae0e9 DOC: resolve-prefer default value and default-server update
By the code, resolve-prefer defaults to IPv6 and is also available in
the default-server directive.
These information were missing or wrong in the documentation.
2015-08-05 00:39:32 +02:00
Baptiste Assmann
a332299ad3 DOC: typos in tcp-check expect examples
The match keyword 'string' was missing in a few tcp-check expect example
statements.
2015-08-05 00:39:27 +02:00
Baptiste Assmann
ea849c0cca DOC: typo in 'redirect', 302 code meaning
302 means a temprary move, not a permanent one
2015-08-05 00:39:23 +02:00
Daniel Jakots
17d228be14 DOC/MINOR: fix OpenBSD versions where haproxy works 2015-07-29 17:39:35 +02:00
Remi Gacogne
7fb9de25e2 MINOR: stream: initialize the current_rule field to NULL on stream init
Currently it is possible for the current_rule field to be evaluated before
being set, leading to valgrind complaining:

==16783== Conditional jump or move depends on uninitialised value(s)
==16783==    at 0x44E662: http_res_get_intercept_rule (proto_http.c:3730)
==16783==    by 0x44E662: http_process_res_common (proto_http.c:6528)
==16783==    by 0x4797B7: process_stream (stream.c:1851)
==16783==    by 0x414634: process_runnable_tasks (task.c:238)
==16783==    by 0x40B02F: run_poll_loop (haproxy.c:1528)
==16783==    by 0x407F25: main (haproxy.c:1887)

This was introduced by commit 152b81e7b2.
2015-07-26 21:50:50 +02:00
Jérémie Courrèges-Anglas
17c3f6284c BUILD: ssl: Allow building against libssl without SSLv3.
If SSLv3 is explicitely requested but not available, warn the user and
bail out.
2015-07-26 08:11:57 +02:00