Commit Graph

37 Commits

Author SHA1 Message Date
Willy Tarreau
45cb4fb640 [MEDIUM] build: switch ebtree users to use new ebtree version
All files referencing the previous ebtree code were changed to point
to the new one in the ebtree directory. A makefile variable (EBTREE_DIR)
is also available to use files from another directory.

The ability to build the libebtree library temporarily remains disabled
because it can have an impact on some existing toolchains and does not
appear worth it in the medium term if we add support for multi-criteria
stickiness for instance.
2009-10-26 21:10:04 +01:00
Willy Tarreau
6b2e11be1e [MEDIUM] backend: implement consistent hashing variation
Consistent hashing provides some interesting advantages over common
hashing. It avoids full redistribution in case of a server failure,
or when expanding the farm. This has a cost however, the hashing is
far from being perfect, as we associate a server to a request by
searching the server with the closest key in a tree. Since servers
appear multiple times based on their weights, it is recommended to
use weights larger than approximately 10-20 in order to smoothen
the distribution a bit.

In some cases, playing with weights will be the only solution to
make a server appear more often and increase chances of being picked,
so stats are very important with consistent hashing.

In order to indicate the type of hashing, use :

   hash-type map-based      (default, old one)
   hash-type consistent     (new one)

Consistent hashing can make sense in a cache farm, in order not
to redistribute everyone when a cache changes state. It could also
probably be used for long sessions such as terminal sessions, though
that has not be attempted yet.

More details on this method of hashing here :
  http://www.spiteful.com/2008/03/17/programmers-toolbox-part-3-consistent-hashing/
2009-10-09 07:17:58 +02:00
Willy Tarreau
f89c1873f8 [CLEANUP] backend: move LB algos to individual files
It was becoming painful to have all the LB algos in backend.c.
Let's move them to their own files. A few hashing functions still
need be broken in two parts, one for the contents and one for the
map position.
2009-10-01 11:19:37 +02:00
Willy Tarreau
bc5258d514 [BUILD] add signal.o to all makefiles 2009-05-10 09:24:53 +02:00
Willy Tarreau
7f062c4193 [MEDIUM] measure and report session rate on frontend, backends and servers
With this change, all frontends, backends, and servers maintain a session
counter and a timer to compute a session rate over the last second. This
value will be very useful because it varies instantly and can be used to
check thresholds. This value is also reported in the stats in a new "rate"
column.
2009-03-05 18:43:00 +01:00
Willy Tarreau
982b6e37e4 [MEDIUM] introduce pipe pools
A new data type has been added : pipes. Some pre-allocated empty pipes
are maintained in a pool for users such as splice which use them a lot
for very short times.

Pipes are allocated using get_pipe() and released using put_pipe().
Pipes which are released with pending data are immediately killed.
The struct pipe is small (16 to 20 bytes) and may even be further
reduced by unifying ->data and ->next.

It would be nice to have a dedicated cleanup task which would watch
for the pipes usage and destroy a few of them from time to time.
2009-01-25 13:49:53 +01:00
Ross West
1df2470850 [BUILD] fix Makefile.bsd and Makefile.osx for stream_interface
Did a full compile of the 1.3.15.7 - 20081208 snapshot on Freebsd-7.x
recently, and noted that there needs to be a quick patch done on the
Makefile for bsd machines.

This was due to the stream_interface replacing the send data commands
in the rewrite Willy did a while ago.

Simple fix, and it compiled cleanly otherwise.  Thanks for the work
Willy!

Cheers,
  Ross.

-=
2009-01-25 12:21:25 +01:00
Willy Tarreau
7ad15da522 [BUILD] update MacOS Makefile to build on newer versions
This update from Dan Zinngrabe enables building on MacOS 10.4 and 10.5.
2008-02-10 17:00:13 +01:00
Willy Tarreau
7b066db3bf [MINOR] store the build options to report with -vv
Sometimes it is useful to find out how a given binary version was
built. The build compiler and options are now provided for this,
and it's possible to get them with the -vv option.
2007-12-02 11:28:59 +01:00
Willy Tarreau
e6d2e4dbdf [MINOR] merge ebtree version 3.0
Version 3.0 of ebtree has been merged in but is not used yet.
2007-11-28 14:20:44 +01:00
Willy Tarreau
e6b989479c [MAJOR] create proto_tcp and move initialization of proxy listeners
Proxy listeners were very special and not very easy to manipulate.
A proto_tcp file has been created with all that is required to
manage TCPv4/TCPv6 as raw protocols, and provide generic listeners.

The code of start_proxies() and maintain_proxies() now looks less
like spaghetti. Also, event_accept will need a serious lifting in
order to use more of the information provided by the listener.
2007-11-04 22:42:49 +01:00
Willy Tarreau
9186126e1c [MEDIUM] moved stats and buffer generic functions to new files
Neither the primitives used to write data to a buffer, nor the stats
dump functions are HTTP-specific anymore. Move them to dedicated
files
2007-10-18 14:12:21 +02:00
Willy Tarreau
92fb9836ee [MAJOR] implemented client-side support for PF_UNIX sockets
A new file, proto_uxst.c, implements support of PF_UNIX sockets
of type SOCK_STREAM. It relies on generic stream_sock_read/write
and uses its own accept primitive which also tries to be generic.

Right now it only implements an echo service in sight of a general
support for start dumping via unix socket. The echo code is more
of a proof of concept than useful code.
2007-10-18 14:11:15 +02:00
Willy Tarreau
dd81598553 [MAJOR] added generic protocol support
A new generic protocol mechanism has been added. It provides
an easy method to implement new protocols with different
listeners (eg: unix sockets).

The listeners are automatically started at the right moment
and enabled after the possible fork().
2007-10-18 14:11:12 +02:00
Willy Tarreau
ec69256382 [BUILD] centralize version and date into one file for each
The version does not appear anymore in the Makefiles nor in
the include files. It was a nightmare to maintain. Now there
is a VERSION file which contains the major version, a VERDATE
file which contains the date for this version and a SUBVERS
file which may contain a sub-version.

A "make version" target has been added to all makefiles to
check the version. The GNU Makefile also has an update-version
target to update those files. This should never be used.

It is still possible to override those values by specifying
them in the equivalent make variables. By default, the GNU
makefile tries to detect a GIT repository and always uses the
version and date from the current repository. This can be
disabled by setting IGNOREGIT to a non-void value.
2007-09-09 23:31:11 +02:00
Willy Tarreau
51041c737c [MAJOR] remove files distributed under an obscure license
src/chtbl.c, src/hashpjw.c and src/list.c are distributed under
an obscure license. While Aleks and I believe that this license
is OK for haproxy, other people think it is not compatible with
the GPL.

Whether it is or not is not the problem. The fact that it rises
a doubt is sufficient for this problem to be addressed. Arnaud
Cornet rewrote the unclear parts with clean GPLv2 and LGPL code.
The hash algorithm has changed too and the code has been slightly
simplified in the process. A lot of care has been taken in order
to respect the original API as much as possible, including the
LGPL for the exportable parts.

The new code has not been thoroughly tested but it looks OK now.
2007-09-09 21:56:53 +02:00
Willy Tarreau
f2ef8c50b9 [MEDIUM] Added easier support for Doug Lea's malloc (dlmalloc)
It's now as easy as passing "DLMALLOC_SRC=<path_to_dlmalloc.c>" to
build with support for dlmalloc. The dlmalloc source is not provided
with haproxy in order to ensure that people will use either the most
recent, or the most suited version for their platform. The minimal
mmap size is specified in DLMALLOC_THRES, which defaults to 4096. It
should be increased on platforms with larger pages (eg: 8 kB on some
64 bit systems).
2007-07-11 09:19:31 +02:00
Willy Tarreau
b21152be7a [RELEASE] Released version 1.3.12 with the following main changes :
- acl: smarter integer comparison support in ACLs
    - acl: specify the direction during fetches
    - acl: provide the argument length for fetch functions
    - acl: provide a reference to the expr to fetch()
    - acl: implement matching on header values
    - acl: support maching on 'path' component
    - acl: permit to return any header when no name specified
    - errorfile: use a local file to feed error messages
    - negation in ACL conds was not cleared between terms
    - fix segfault at exit when using captures
    - improve memory freeing upon exit
    - acl: support '-i' to ignore case when matching
    - str2net() must not change the const char *
    - provide default ACLs
    - acl: distinguish between request and response headers
    - added the 'use_backend' keyword for full content-switching
    - acl: added the TRUE and FALSE ACLs.
    - shut warnings 'is*' macros from ctype.h on solaris
2007-06-17 23:41:40 +02:00
Willy Tarreau
a3503e0b5a [RELEASE] Released version 1.3.11.4 with the following main changes :
- do not re-arm read timeout in SHUTR state
    - optimize I/O by detecting system starvation
    - the epoll FD must not be shared between processes
    - limit the number of events returned by *poll*
2007-06-03 17:27:07 +02:00
Willy Tarreau
3c6fc07e18 [RELEASE] Released version 1.3.11.3 with the following main changes :
- pre-initialize timeouts with tv_eternity during parsing
2007-05-14 14:40:25 +02:00
Willy Tarreau
fc273c230d [RELEASE] Released version 1.3.11.2 with the following main changes :
- fixed broken health-checks since switch to timeval
2007-05-14 03:42:47 +02:00
Willy Tarreau
3c5340c35b [RELEASE] Released version 1.3.11.1 with the following main changes :
- fixed ev_kqueue which was forgotten during the switch to timeval
    - allowed null timeouts for past events in select
2007-05-14 03:18:43 +02:00
Willy Tarreau
544eb40f29 [RELEASE] Released version 1.3.11 with the following main changes :
- fixed ev_sepoll again by rewriting the state machine
    - switched all timeouts to timevals instead of milliseconds
    - improved memory management using mempools v2.
    - several minor optimizations
2007-05-14 02:42:33 +02:00
Willy Tarreau
086b3b4c9f [MAJOR] ported the captures to use the new mempool v2
The "capture.c" file has also been removed since it was empty.
2007-05-13 21:45:51 +02:00
Willy Tarreau
50e608d721 [MEDIUM] implement memory pools version 2
The new pools know about their size and usage. Malloc is not used
anymore, instead a dedicated function to refill the entries is used.
2007-05-13 18:26:08 +02:00
Willy Tarreau
9ca931f976 [RELEASE] Released version 1.3.10.2 with the following main changes :
- fixed build of files including standard.h on OpenBSD
2007-05-10 07:51:17 +02:00
Willy Tarreau
13398d3896 [RELEASE] Released version 1.3.10.1 with the following main changes :
- fixed sepoll transition matrix (two states were missing)
2007-05-09 22:58:28 +02:00
Willy Tarreau
61beedfb8c [RELEASE] Released version 1.3.10 with the following main changes :
- several fixes in ev_sepoll
    - fixed some expiration dates on some tasks
    - fixed a bug in connection establishment detection due to speculative I/O
    - fixed rare bug occuring on TCP with early close (reported by Andy Smith)
    - implemented URI hashing algorithm (Guillaume Dallaire)
    - implemented SMTP health checks (Peter van Dijk)
    - replaced the rbtree with ul2tree from old scheduler project
    - new framework for generic ACL support
    - added the 'acl' and 'block' keywords to the config language
    - added several ACL criteria and matches (IP, port, URI, ...)
    - cleaned up and better modularization for some time functions
    - fixed list macros
    - fixed useless memory allocation in str2net()
    - store the original destination address in the session
2007-05-09 01:44:58 +02:00
Willy Tarreau
a84d374367 [MAJOR] new framework for generic ACL support
This framework offers all other subsystems the ability to register
ACL matching criteria. Some generic matching functions are already
provided. Others will come soon and the framework shall evolve.
2007-05-08 23:24:50 +02:00
Willy Tarreau
2e8eb67762 [BUILD] do not build rbtree anymore
Rbtree is not used anymore. Remove it from the makefiles.
2007-04-30 17:30:59 +02:00
Willy Tarreau
6e0433f0e9 [RELEASE] Released version 1.3.9 with the following changes :
- modularized the polling mechanisms and use function pointers instead
      of macros at many places
    - implemented support for FreeBSD's kqueue() polling mechanism
    - fixed a warning on OpenBSD : MIN/MAX redefined
    - change socket registration order at startup to accomodate kqueue.
    - several makefile cleanups to support old shells
    - fix build with limits.h once for all
    - ev_epoll: do not rely on fd_sets anymore, use changes stacks instead.
    - fdtab now holds the results of polling
    - implemented support for speculative I/O processing with epoll()
    - remove useless calls to shutdown(SHUT_RD), resulting in small speed boost
    - auto-registering of pollers at load time
2007-04-16 01:33:37 +02:00
Willy Tarreau
8ae42f42f8 [BUILD] enable kqueue on MacOS X too
MacOS X is said to have kqueue. Let's enable it too.
2007-04-09 16:30:28 +02:00
Willy Tarreau
4f60f16dd3 [MAJOR] modularize the polling mechanisms
select, poll and epoll now have their dedicated functions and have
been split into distinct files. Several FD manipulation primitives
have been provided with each poller.

The rest of the code needs to be cleaned to remove traces of
StaticReadEvent/StaticWriteEvent. A trick involving a macro has
temporarily been used right now. Some work needs to be done to
factorize tests and sets everywhere.
2007-04-08 16:39:58 +02:00
Willy Tarreau
42c7659669 [RELEASE] Released version 1.3.8.2 with the following changes :
- rewriting either the status line or request line could crash the
      process due to a pointer which ought to be reset before parsing.
    - rewriting the status line in the response did not work, it caused
      a 502 Bad Gateway due to an erroneous state during parsing
2007-04-03 20:30:13 +02:00
Willy Tarreau
ef6d7612d6 [RELEASE] Released version 1.3.8.1 with the following changes :
- fix reqadd when no option httpclose is used.
    - removed now unused fiprm and beprm from proxies
    - split logs into two versions : TCP and HTTP
    - added some docs about http headers storage and acls
    - added a VIM script for syntax color highlighting (Bruno Michel)
2007-04-01 11:06:22 +02:00
Willy Tarreau
d661cc06af [RELEASE] Released version 1.3.8 with the following changes :
- fixed several bugs which might have caused a crash with bad configs
    - several optimizations in header processing
    - many progresses towards transaction-based processing
    - option forwardfor may be used in frontends
    - completed HTTP response processing
    - some code refactoring between request and response processing
    - new HTTP header manipulation functions
    - optimizations on the recv() patch to reduce CPU usage under very
      high data rates.
    - more user-friendly help about the 'usesrc' keyword (CTTPROXY)
    - username/groupname support from Marcus Rueckert
    - added the "except" keyword to the "forwardfor" option (Bryan German)
    - support for health-checks on other addresses (Fabrice Dulaunoy)
    - makefile for MacOS 10.4 / Darwin (Dan Zinngrabe)
    - do not insert "Connection: close" in HTTP/1.0 messages
2007-03-26 00:24:56 +02:00
Willy Tarreau
cec2e7ad1c [BUILD] makefile for MacOS 10.4 / Darwin
Contribution from Dan Zinngrabe :

Here is a Makefile based on that for BSD that builds HAProxy 1.3.7 on
MacOS 10.4 and Darwin. I haven't tested it extensively yet, but it
does seem to work so far.
2007-03-25 16:55:56 +02:00