Commit Graph

74 Commits

Author SHA1 Message Date
Willy Tarreau
3f0c976135 [BUG] fix error checking in strl2ic/strl2uic()
The strl2ic() and strl2uic() primitives used to convert string to
integers could return 10 times the value read if they stopped on
non-digit because of a mis-placed loop exit.
2007-10-25 09:42:24 +02:00
Krzysztof Oledzki
85130941e7 [MEDIUM] stats: report server and backend cumulated downtime
Hello,

This patch implements new statistics for SLA calculation by adding new
field 'Dwntime' with total down time since restart (both HTTP/CSV) and
extending status field (HTTP) or inserting a new one (CSV) with time
showing how long each server/backend is in a current state. Additionaly,
down transations are also calculated and displayed for backends, so it is
possible to know how many times selected backend was down, generating "No
server is available to handle this request." error.

New information are presentetd in two different ways:
   - for HTTP: a "human redable form", one of "100000d 23h", "23h 59m" or
      "59m 59s"
   - for CSV: seconds

I believe that seconds resolution is enough.

As there are more columns in the status page I decided to shrink some
names to make more space:
   - Weight -> Wght
   - Check -> Chk
   - Down -> Dwn

Making described changes I also made some improvements and fixed some
small bugs:
   - don't increment s->health above 's->rise + s->fall - 1'. Previously it
     was incremented an then (re)set to 's->rise + s->fall - 1'.
   - do not set server down if it is down already
   - do not set server up if it is up already
   - fix colspan in multiple places (mostly introduced by my previous patch)
   - add missing "status" header to CSV
   - fix order of retries/redispatches in server (CSV)
   - s/Tthen/Then/
   - s/server/backend/ in DATA_ST_PX_BE (dumpstats.c)

Changes from previous version:
  - deal with negative time intervales
  - don't relay on s->state (SRV_RUNNING)
  - little reworked human_time + compacted format (no spaces). If needed it
    can be used in the future for other purposes by optionally making "cnt"
    as an argument
  - leave set_server_down mostly unchanged
  - only little reworked "process_chk: 9"
  - additional fields in CSV are appended to the rigth
  - fix "SEC" macro
  - named arguments (human_time, be_downtime, srv_downtime)

Hope it is OK. If there are only cosmetic changes needed please fill free
to correct it, however if there are some bigger changes required I would
like to discuss it first or at last to know what exactly was changed
especially since I already put this patch into my production server. :)

Thank you,

Best regards,

 				Krzysztof Oledzki
2007-10-22 21:36:23 +02:00
Krzysztof Oledzki
d9db9274fe [MINOR] report haproxy's version by default on the stats page
For people who manage many haproxies, it is sometimes convenient
to be informed of their version. This patch adds this, with the
option to disable this report by specifying "stats hide-version".

Also, the feature may be permanently disabled by setting the
STATS_VERSION_STRING to "" (empty string), or the format can
simply be adjusted.
2007-10-15 10:05:11 +02:00
Willy Tarreau
2c43a1e2f0 [MEDIUM] only consider slow checks when looking for the common interval
When one server in one backend has a very low check interval, it imposes
its value as the minimal interval, causing all other servers to start
their checks close to each other, thus partially voiding the benefits of
the spread checks.

The solution consists in ignoring intervals lower than a given value
(SRV_CHK_INTER_THRES = 1000 ms) when computing the minimal interval,
and then assigning them a start date relative to their own interval
and not the global one.

With this change, the checks distribution clearly looks better.
2007-10-15 09:33:14 +02:00
Willy Tarreau
ec69256382 [BUILD] centralize version and date into one file for each
The version does not appear anymore in the Makefiles nor in
the include files. It was a nightmare to maintain. Now there
is a VERSION file which contains the major version, a VERDATE
file which contains the date for this version and a SUBVERS
file which may contain a sub-version.

A "make version" target has been added to all makefiles to
check the version. The GNU Makefile also has an update-version
target to update those files. This should never be used.

It is still possible to override those values by specifying
them in the equivalent make variables. By default, the GNU
makefile tries to detect a GIT repository and always uses the
version and date from the current repository. This can be
disabled by setting IGNOREGIT to a non-void value.
2007-09-09 23:31:11 +02:00
Willy Tarreau
51041c737c [MAJOR] remove files distributed under an obscure license
src/chtbl.c, src/hashpjw.c and src/list.c are distributed under
an obscure license. While Aleks and I believe that this license
is OK for haproxy, other people think it is not compatible with
the GPL.

Whether it is or not is not the problem. The fact that it rises
a doubt is sufficient for this problem to be addressed. Arnaud
Cornet rewrote the unclear parts with clean GPLv2 and LGPL code.
The hash algorithm has changed too and the code has been slightly
simplified in the process. A lot of care has been taken in order
to respect the original API as much as possible, including the
LGPL for the exportable parts.

The new code has not been thoroughly tested but it looks OK now.
2007-09-09 21:56:53 +02:00
Willy Tarreau
bbd42123e1 [MINOR] add support for "stats refresh <interval>"
Sometimes it may be desirable to automatically refresh the
stats page. Most browsers support the "Refresh:" header with
an interval in seconds. Specifying "stats refresh xxx" will
automatically add this header.
2007-09-09 21:09:28 +02:00
Willy Tarreau
b21152be7a [RELEASE] Released version 1.3.12 with the following main changes :
- acl: smarter integer comparison support in ACLs
    - acl: specify the direction during fetches
    - acl: provide the argument length for fetch functions
    - acl: provide a reference to the expr to fetch()
    - acl: implement matching on header values
    - acl: support maching on 'path' component
    - acl: permit to return any header when no name specified
    - errorfile: use a local file to feed error messages
    - negation in ACL conds was not cleared between terms
    - fix segfault at exit when using captures
    - improve memory freeing upon exit
    - acl: support '-i' to ignore case when matching
    - str2net() must not change the const char *
    - provide default ACLs
    - acl: distinguish between request and response headers
    - added the 'use_backend' keyword for full content-switching
    - acl: added the TRUE and FALSE ACLs.
    - shut warnings 'is*' macros from ctype.h on solaris
2007-06-17 23:41:40 +02:00
Willy Tarreau
a3503e0b5a [RELEASE] Released version 1.3.11.4 with the following main changes :
- do not re-arm read timeout in SHUTR state
    - optimize I/O by detecting system starvation
    - the epoll FD must not be shared between processes
    - limit the number of events returned by *poll*
2007-06-03 17:27:07 +02:00
Willy Tarreau
1db37710dc [MEDIUM] limit the number of events returned by *poll*
By default, epoll/kqueue used to return as many events as possible.
This could sometimes cause huge latencies (latencies of up to 400 ms
have been observed with many thousands of fds at once). Limiting the
number of events returned also reduces the latency by avoiding too
many blind processing. The value is set to 200 by default and can be
changed in the global section using the tune.maxpollevents parameter.
2007-06-03 17:16:49 +02:00
Willy Tarreau
3c6fc07e18 [RELEASE] Released version 1.3.11.3 with the following main changes :
- pre-initialize timeouts with tv_eternity during parsing
2007-05-14 14:40:25 +02:00
Willy Tarreau
fc273c230d [RELEASE] Released version 1.3.11.2 with the following main changes :
- fixed broken health-checks since switch to timeval
2007-05-14 03:42:47 +02:00
Willy Tarreau
3c5340c35b [RELEASE] Released version 1.3.11.1 with the following main changes :
- fixed ev_kqueue which was forgotten during the switch to timeval
    - allowed null timeouts for past events in select
2007-05-14 03:18:43 +02:00
Willy Tarreau
544eb40f29 [RELEASE] Released version 1.3.11 with the following main changes :
- fixed ev_sepoll again by rewriting the state machine
    - switched all timeouts to timevals instead of milliseconds
    - improved memory management using mempools v2.
    - several minor optimizations
2007-05-14 02:42:33 +02:00
Willy Tarreau
4d2d098ea3 [MAJOR] call garbage collector when doing soft stop
When we're interrupted by another instance, it is very likely
that the other one will need some memory. Now we know how to
free what is not used, so let's do it.

Also only free non-null pointers. Previously, pool_destroy()
did implicitly check for this case which was incidentely
needed.
2007-05-14 00:39:29 +02:00
Willy Tarreau
7dcd46d471 [MEDIUM] enhance behaviour of mempools v2
- keep the number of users of each pool
- call the garbage collector on out of memory conditions
- sort the pools by size for faster creation
- force the alignment size to 16 bytes instead of 4*sizeof(void *)
2007-05-14 00:16:13 +02:00
Willy Tarreau
cf7f320f9d [MAJOR] last bunch of capture changes for mempool v2
The header captures had lots of pools. They have all been transformed.
2007-05-13 22:46:04 +02:00
Willy Tarreau
332f8bfc5b [MAJOR] ported requri to use mempools v2 2007-05-13 21:36:56 +02:00
Willy Tarreau
63963c62e7 [MAJOR] ported appsession to use mempools v2
Also during this process, a bug was found in appsession_refresh().
It would not automatically requeue the task in the queue, so the
old sessions would not vanish.
2007-05-13 21:29:55 +02:00
Willy Tarreau
e6ce59deb7 [MEDIUM] add new memory management functions
Implement pool_destroy2, pool_flush2, pool_gc2. It is safe to call
pool_gc2 to free whatever memory possible.
2007-05-13 19:38:49 +02:00
Willy Tarreau
50e608d721 [MEDIUM] implement memory pools version 2
The new pools know about their size and usage. Malloc is not used
anymore, instead a dedicated function to refill the entries is used.
2007-05-13 18:26:08 +02:00
Willy Tarreau
0481c20e66 [MINOR] add new tv_* functions
The most useful, tv_add_ifset only adds the increment if it is set. It
is designed for use in expiration computation.
2007-05-13 16:03:27 +02:00
Willy Tarreau
d825eef9c5 [MAJOR] replaced all timeouts with struct timeval
The timeout functions were difficult to manipulate because they were
rounding results to the millisecond. Thus, it was difficult to compare
and to check what expired and what did not. Also, the comparison
functions were heavy with multiplies and divides by 1000. Now, all
timeouts are stored in timevals, reducing the number of operations
for updates and leading to cleaner and more efficient code.
2007-05-12 22:35:00 +02:00
Willy Tarreau
49fa3a1453 [MINOR] time.h: added a few tv_* functions to manipulate timevals
A few more tv_* functions will be needed to switch from a millisecond-based
scheduler to a struct timeval-based one.
2007-05-12 22:29:44 +02:00
Willy Tarreau
9ca931f976 [RELEASE] Released version 1.3.10.2 with the following main changes :
- fixed build of files including standard.h on OpenBSD
2007-05-10 07:51:17 +02:00
Willy Tarreau
938b303a04 [BUILD] fixed build of files including standard.h on OpenBSD
The file standard.h included netinet/in.h without including
sys/types.h. This broke build on OpenBSD.
2007-05-10 06:39:03 +02:00
Willy Tarreau
13398d3896 [RELEASE] Released version 1.3.10.1 with the following main changes :
- fixed sepoll transition matrix (two states were missing)
2007-05-09 22:58:28 +02:00
Willy Tarreau
61beedfb8c [RELEASE] Released version 1.3.10 with the following main changes :
- several fixes in ev_sepoll
    - fixed some expiration dates on some tasks
    - fixed a bug in connection establishment detection due to speculative I/O
    - fixed rare bug occuring on TCP with early close (reported by Andy Smith)
    - implemented URI hashing algorithm (Guillaume Dallaire)
    - implemented SMTP health checks (Peter van Dijk)
    - replaced the rbtree with ul2tree from old scheduler project
    - new framework for generic ACL support
    - added the 'acl' and 'block' keywords to the config language
    - added several ACL criteria and matches (IP, port, URI, ...)
    - cleaned up and better modularization for some time functions
    - fixed list macros
    - fixed useless memory allocation in str2net()
    - store the original destination address in the session
2007-05-09 01:44:58 +02:00
Willy Tarreau
23677908dd [MEDIUM] implement SMTP health checks
Peter van Dijk contributed this patch which implements the "smtpchk"
option, which is to SMTP what "httpchk" is to HTTP. By default, it sends
"HELO localhost" to the servers, and waits for the 250 message, but it
can also send a specific request.
2007-05-08 23:50:35 +02:00
Willy Tarreau
d077a8e67c [MINOR] fixed useless memory allocation in str2net()
It was not necessary anymore to allocate memory in str2net().
Moreover, some calls to free() were missing in case of errors.
2007-05-08 23:23:38 +02:00
Willy Tarreau
3ff3dc1fb1 [MINOR] add support for variable size arrays in structures
Depending on the version of GCC, variable size arrays are not
declared in the same manner. Let's add a check for the version
and provide a macro.
2007-05-08 14:08:05 +02:00
Willy Tarreau
2b1dccd501 [MINOR] add LIST_HEAD_INIT to the list management macros
It is useful to have the ability to initialize a list while declaring it.
2007-05-08 14:07:44 +02:00
Willy Tarreau
b9c62b90da [MEDIUM] add better list management macros
The FOREACH_ITEM and FOREACH_ITEM_SAFE macros are nasty, they
require too many arguments. Let's port those from linux instead.
2007-05-08 14:07:39 +02:00
Willy Tarreau
42aae5c7cf [MEDIUM] many cleanups in the time functions
Now, functions whose name begins with '__tv_' are inlined. Also,
'tv_ms' is used as a prefix for functions using milliseconds.
2007-04-29 17:43:56 +02:00
Willy Tarreau
8d7d1497e0 [MEDIUM] implement and use tv_cmp2_le instead of tv_cmp2_ms
tv_cmp2_ms handles multiple combinations of tv1 and tv2, but only
one form is used: (tv1 <= tv2). So it is overkill to use it everywhere.
A new function designed to do exactly this has been written for that
purpose: tv_cmp2_le. Also, removed old unused tv_* functions.
2007-04-29 13:44:43 +02:00
Willy Tarreau
a6a6a93e56 [MAJOR] changed TV_ETERNITY to ~0 instead of 0
The fact that TV_ETERNITY was 0 was very awkward because it
required that comparison functions handled the special case.
Now it is ~0 and all comparisons are performed on unsigned
values, so that it is naturally greater than any other value.

A performance gain of about 2-5% has been noticed.
2007-04-29 13:44:24 +02:00
Willy Tarreau
40cf67da89 [MINOR] added support for dual-linked lists
Imported dual-linked lists management macros from the old scheduler project,
and fixed a long-standing nasty bug in the DLIST_ADD() macro.
2007-04-29 13:40:50 +02:00
Willy Tarreau
6e0433f0e9 [RELEASE] Released version 1.3.9 with the following changes :
- modularized the polling mechanisms and use function pointers instead
      of macros at many places
    - implemented support for FreeBSD's kqueue() polling mechanism
    - fixed a warning on OpenBSD : MIN/MAX redefined
    - change socket registration order at startup to accomodate kqueue.
    - several makefile cleanups to support old shells
    - fix build with limits.h once for all
    - ev_epoll: do not rely on fd_sets anymore, use changes stacks instead.
    - fdtab now holds the results of polling
    - implemented support for speculative I/O processing with epoll()
    - remove useless calls to shutdown(SHUT_RD), resulting in small speed boost
    - auto-registering of pollers at load time
2007-04-16 01:33:37 +02:00
Willy Tarreau
b40d42006c [BUILD] declare epoll_* as static when using our own functions
We will have to share this code among several implementations.
2007-04-15 23:57:41 +02:00
Willy Tarreau
8374918cce [MAJOR] implemented support for speculative I/O processing
The pollers will now be able to speculatively call the I/O
processing functions and decide whether or not they want to
poll on those FDs. The changes primarily consist in teaching
those functions how to pass the info they got an EAGAIN.
2007-04-15 20:56:27 +02:00
Willy Tarreau
167d8b5953 [BUILD] Fix limits.h once for all
Patch #cf83df3d162687d9c74783357421bd89f596eaac was stupid. Including
limits.h is portable and easier. At least it now builds on Solaris,
FreeBSD, Linux and OpenBSD.
2007-04-09 22:16:12 +02:00
Willy Tarreau
cf83df3d16 [BUILD] fix build on FreeBSD for INT_MIN/INT_MAX
FreeBSD stores INT_MIN and INT_MAX in sys/limits.h only. Other systems
(Solaris) have it in sys/types.h and do not have sys/limits.h. Let's
include sys/limits.h only if INT_MAX is not defined.
2007-04-09 15:57:51 +02:00
Willy Tarreau
42c7659669 [RELEASE] Released version 1.3.8.2 with the following changes :
- rewriting either the status line or request line could crash the
      process due to a pointer which ought to be reset before parsing.
    - rewriting the status line in the response did not work, it caused
      a 502 Bad Gateway due to an erroneous state during parsing
2007-04-03 20:30:13 +02:00
Willy Tarreau
ef6d7612d6 [RELEASE] Released version 1.3.8.1 with the following changes :
- fix reqadd when no option httpclose is used.
    - removed now unused fiprm and beprm from proxies
    - split logs into two versions : TCP and HTTP
    - added some docs about http headers storage and acls
    - added a VIM script for syntax color highlighting (Bruno Michel)
2007-04-01 11:06:22 +02:00
Willy Tarreau
d661cc06af [RELEASE] Released version 1.3.8 with the following changes :
- fixed several bugs which might have caused a crash with bad configs
    - several optimizations in header processing
    - many progresses towards transaction-based processing
    - option forwardfor may be used in frontends
    - completed HTTP response processing
    - some code refactoring between request and response processing
    - new HTTP header manipulation functions
    - optimizations on the recv() patch to reduce CPU usage under very
      high data rates.
    - more user-friendly help about the 'usesrc' keyword (CTTPROXY)
    - username/groupname support from Marcus Rueckert
    - added the "except" keyword to the "forwardfor" option (Bryan German)
    - support for health-checks on other addresses (Fabrice Dulaunoy)
    - makefile for MacOS 10.4 / Darwin (Dan Zinngrabe)
    - do not insert "Connection: close" in HTTP/1.0 messages
2007-03-26 00:24:56 +02:00
Willy Tarreau
9641e8f6ee [MINOR] read optimizations based on the MSS
Generally, if a recv() returns less bytes than the MSS, it means that
there is nothing left in the system's buffers, and that it's not worth
trying to read again because we are very likely to get nothing. A
default read low limit has been set to 1460 bytes below which we stop
reading.

This has brought a little speed boost on small objects while maintaining
the same speed on large objects.
2007-03-23 23:02:09 +01:00
Willy Tarreau
b8949f1ed0 [MEDIUM] re-implemented the multiple read polling
Multiple read polling was temporarily disabled, which had the side
effect of burning huge amounts of CPU on large objects. It has now
been re-implemented with a limit of 8 calls per wake-up, which seems
to provide best results at least on Linux.
2007-03-23 22:39:59 +01:00
Willy Tarreau
6911fa484c [MINOR] added new str2i* functions
Those functions provide faster and more flexible alternatives to atoi(),
some of which are able to work on sub-strings.
2007-03-04 18:06:08 +01:00
Willy Tarreau
5e8f066961 [MINOR] slightly optimize time calculation for rbtree
The new rbtree-based scheduler makes heavy use of tv_cmp2(), and
this function becomes a huge CPU eater. Refine it a little bit in
order to slightly reduce CPU usage.
2007-02-12 00:59:08 +01:00
Willy Tarreau
9cabf70db7 [RELEASE] Released version 1.3.7 with the following changes :
- fix critical bug introduced with 1.3.6 : an empty request header
      may lead to a crash due to missing pointer assignment
    - hdr_idx might be left uninitialized in debug mode
    - fixed build on FreeBSD due to missing fd_set declaration
2007-01-26 23:49:01 +01:00