[MEDIUM] add support for health-checks on other addresses

Patch from Fabrice Dulaunoy. Explanation below, and script
merged in examples/.

This patch allow to put a different address in the check part for each
server (and not only a specific port)

I need this feature because I've a complex settings where, when a specific
farm goes down, I need to switch a set of other farm either if these other
farm behave perfectly well.

For that purpose, I've made a small PERL daemon with some REGEX or PORT
test which allow me to test a bunch of thing.
This commit is contained in:
Willy Tarreau 2007-03-25 16:45:16 +02:00
parent 4fb20ff1cc
commit 2ea3abb7bf
7 changed files with 670 additions and 4 deletions

View File

@ -970,7 +970,8 @@ values are the following ones :
- rise : 2
- fall : 3
- port : default server port
- addr : specific address for the test (default = address server)
The default mode consists in establishing TCP connections only. But in certain
types of application failures, it is often that the server continues to accept
connections because the system does it itself while the application is running
@ -1022,6 +1023,15 @@ backup servers, the second one will only be used when the first one dies, and
so on. To force load-balancing between backup servers, specify the 'allbackups'
option.
Since version 1.1.22, it is possible to send health checks to a different port
than the service. It is mainly needed in setups where the server does not have
any predefined port, for instance when the port is deduced from the listening
port. For this, use the 'port' parameter followed by the port number which must
respond to health checks. It is also possible to send health checks to a
different address than the service. It makes it easier to use a dedicated check
daemon on the servers, for instance, check return contents and stop several
farms at once in the event of an error anywhere.
Since version 1.1.17, it is also possible to visually check the status of all
servers at once. For this, you just have to send a SIGHUP signal to the proxy.
The servers status will be dumped into the logs at the 'notice' level, as well

View File

@ -971,7 +971,8 @@ Les param
- rise : 2
- fall : 3
- port : port de connexion du serveur
- addr : adresse de connexion du serveur (par defaut: adresse du serveur)
Le mode par défaut consiste à établir des connexions TCP uniquement. Dans
certains cas de pannes, des serveurs peuvent continuer à accepter les
connexions sans les traiter. Depuis la version 1.1.16, haproxy est en mesure
@ -1032,7 +1033,11 @@ Depuis la version 1.1.22, il est possible d'envoyer les tests de fonctionnement
vers un port différent de celui de service. C'est nécessaire principalement
pour les configurations où le serveur n'a pas de port prédéfini, par exemple
lorsqu'il est déduit du port d'acceptation de la connexion. Pour cela, utiliser
le paramètre 'port' suivi du numéro de port devant répondre aux requêtes.
le paramètre 'port' suivi du numéro de port devant répondre aux requêtes. Il
est possible d'envoyer les tests de fonctionnement vers une adresse différente
de celle de service. Cela permet d'utiliser, sur la machine faisant fonctionner
HAproxy, un démon permettant des tests specifiques ( REGEX sur un résultat et
basculement de plusieurs fermes en cas d'erreur sur l'une d'elles).
Enfin, depuis la version 1.1.17, il est possible de visualiser rapidement
l'état courant de tous les serveurs. Pour cela, il suffit d'envoyer un signal

540
examples/check Executable file
View File

@ -0,0 +1,540 @@
#!/usr/bin/perl
###################################################################################################################
# $Id:: check 20 2007-02-23 14:26:44Z fabrice $
# $Revision:: 20 $
###################################################################################################################
# Authors : Fabrice Dulaunoy <fabrice@dulaunoy.com>
#
# Copyright (C) 2006-2007 Fabrice Dulaunoy <fabrice@dulaunoy.com>
#
# This program is free software; you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation; either version 2 of the License, or (at your
# option) any later version. See <http://www.fsf.org/copyleft/gpl.txt>.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
###################################################################################################################
#
###################################################################################################################
use strict;
package MyPackage;
use Config::General;
use Getopt::Std;
use LWP::UserAgent;
use URI;
use File::Basename;
# CVS VSERSION
#my $VERSION = do { my @rev = ( q$Revision: 20 $ =~ /\d+/g ); sprintf "%d." . "%d" x $#rev, @rev };
# SVN VERSION
my $VERSION = sprintf "1.%02d", '$Revision: 20 $ ' =~ /(\d+)/;
my %option;
getopts( "vHhc:", \%option );
if ( $option{ h } )
{
print "Usage: $0 [options ...]\n\n";
print "Where options include:\n";
print "\t -h \t\t\tthis help (what else ?)\n";
print "\t -H \t\t\tshow a sample config file\n";
print "\t -v \t\t\tprint version and exit\n";
print "\t -c file \t\tuse config file (default /etc/check.conf)\n";
print "\n\t This is a small program parsing the config file \n";
print "\t and checking one or more condition on one or more servers\n";
print "\t these condition could be \n";
print "\t\t HTTP return code list (with optinal Host Header and optional Basic Authentication) \n";
print "\t\t a regex over a HTTP GET (with optinal Host Header and optional Basic Authentication)\n";
print "\t\t a regex over a FTP GET ( with optional Basic Authentication)\n";
print "\t\t a TCP open port\n";
print "\t the result state is an AND over all tests \n";
print "\t this result could be \n";
print "\t\t a simple HTTP return state (\"200 OK\" or \"503 Service Unavailable\" \n";
print "\t\t a HTML page with a status OK or NOK for each test\n";
print "\t\t a HTML page with a staus OK or NOK for each test in a row of a TABLE\n";
print "\n\t The natural complement of this tools is the poll_check tool\n";
print "\t The result code of this tools is designed to fit the HAPROXY requirement (test over a port not related to the WEB server)\n";
}
if ( $option{ H } )
{
print "\t A sample config file could be:\n";
print <<'EOF';
###########################################################
# listening port ( default 9898 )
port 9899
# on which IP to bind (default 127.0.0.1 ) * = all IP
host 10.2.1.1
# which client addr is allow ( default 127.0.0.0/8 )
#cidr_allow = 0.0.0.0/0
# verbosity from 0 to 4 (default 0 = no log )
log_level = 1
# daemonize (default 0 = no )
daemon = 1
# content put a HTML content after header
# (default 0 = no content 1 = html 2 = table )
content = 2
# reparse the config file at each request ( default 0 = no )
# only SIGHUP reread the config file)
reparse = 1
# pid_file (default /var/run/check.pid )
# $$$ = basename of config file
# $$ = PID
pid_file=/var/run/CHECK_$$$.pid
# log_file (default /var/log/check.log )
# $$$ = basename of config file
# $$ = PID
log_file=/var/log/CHECK_$$$.log
# number of servers to keep running (default = 5)
min_servers = 2
# number of servers to have waiting for requests (default = 2)
min_spare_servers = 1
# maximum number of servers to have waiting for requests (default = 10)
max_spare_servers =1
# number of servers (default = 50)
max_servers = 2
###########################################################
# a server to check
# type could be get , regex or tcp
#
# get = do a http or ftp get and check the result code with
# the list, coma separated, provided ( default = 200,201 )
# hostheader is optional and send to the server if provided
#
# regex = do a http or ftp get and check the content result
# with regex provided
# hostheader is optional and send to the server if provided
#
# tcp = test if the tcp port provided is open
#
###########################################################
<realserver>
url=http://127.0.0.1:80/apache2-default/index.html
type = get
code=200,201
hostheader = www.test.com
</realserver>
<realserver>
url=http://127.0.0.1:82/apache2-default/index.html
type = get
code=200,201
hostheader = www.myhost.com
</realserver>
<realserver>
url= http://10.2.2.1
type = regex
regex= /qdAbm/
</realserver>
<realserver>
type = tcp
url = 10.2.2.1
port =80
</realserver>
<realserver>
type = get
url = ftp://USER:PASSWORD@10.2.3.1
code=200,201
</realserver>
###########################################################
EOF
}
if ( $option{ h } || $option{ H } )
{
exit;
}
if ( $option{ v } ) { print "$VERSION\n"; exit; }
use vars qw(@ISA);
use Net::Server::PreFork;
@ISA = qw(Net::Server::PreFork);
my $port;
my $host;
my $reparse;
my $cidr_allow;
my $log_level;
my $log_file;
my $pid_file;
my $daemon;
my $min_servers;
my $min_spare_servers;
my $max_spare_servers;
my $max_servers;
my $html_content;
my $conf_file = $option{ c } || "/etc/check.conf";
my $pwd = $ENV{ PWD };
$conf_file =~ s/^\./$pwd/;
$conf_file =~ s/^([^\/])/$pwd\/$1/;
my $basename = basename( $conf_file, ( '.conf' ) );
my $CONF = parse_conf( $conf_file );
my $reparse_one = 0;
$SIG{ HUP } = sub { $reparse_one = 1; };
my @TEST;
my $test_list = $CONF->{ realserver };
if ( ref( $test_list ) eq "ARRAY" )
{
@TEST = @{ $test_list };
}
else
{
@TEST = ( $test_list );
}
my $server = MyPackage->new(
{
port => $port,
host => $host,
cidr_allow => $cidr_allow,
log_level => $log_level,
child_communication => 1,
setsid => $daemon,
log_file => $log_file,
pid_file => $pid_file,
min_servers => $min_servers,
min_spare_servers => $min_spare_servers,
max_spare_servers => $max_spare_servers,
max_servers => $max_servers,
}
);
$server->run();
exit;
sub process_request
{
my $self = shift;
if ( $reparse || $reparse_one )
{
$CONF = parse_conf( $conf_file );
}
my $result;
my @TEST;
my $test_list = $CONF->{ realserver };
if ( ref( $test_list ) eq "ARRAY" )
{
@TEST = @{ $test_list };
}
else
{
@TEST = ( $test_list );
}
my $allow_code;
my $test_item;
my $html_data;
foreach my $test ( @TEST )
{
my $uri;
my $authority;
my $URL = $test->{ url };
$uri = URI->new( $URL );
$authority = $uri->authority;
if ( exists $test->{ type } )
{
if ( $test->{ type } =~ /get/i )
{
my $allow_code = $test->{ code } || '200,201';
$test_item++;
my $host = $test->{ hostheader } || $authority;
my $res = get( $URL, $allow_code, $host );
if ( $html_content == 1 )
{
if ( $res )
{
$html_data .= "GET OK $URL<br>\r\n";
}
else
{
$html_data .= "GET NOK $URL<br>\r\n";
}
}
if ( $html_content == 2 )
{
if ( $res )
{
$html_data .= "<tr><td>GET</td><td>OK</td><td>$URL</td></tr>\r\n";
}
else
{
$html_data .= "<tr><td>GET</td><td>NOK</td><td>$URL</td></tr>\r\n";
}
}
$result += $res;
}
if ( $test->{ type } =~ /regex/i )
{
my $regex = $test->{ regex };
$test_item++;
my $host = $test->{ hostheader } || $authority;
my $res = regex( $URL, $regex, $host );
if ( $html_content == 1 )
{
if ( $res )
{
$html_data .= "REGEX OK $URL<br>\r\n";
}
else
{
$html_data .= "REGEX NOK $URL<br>\r\n";
}
}
if ( $html_content == 2 )
{
if ( $res )
{
$html_data .= "<tr><td>REGEX</td><td>OK</td><td>$URL</td></tr>\r\n";
}
else
{
$html_data .= "<tr><td>REGEX</td><td>NOK</td><td>$URL</td></tr>\r\n";
}
}
$result += $res;
}
if ( $test->{ type } =~ /tcp/i )
{
$test_item++;
my $PORT = $test->{ port } || 80;
my $res = TCP( $URL, $PORT );
if ( $html_content == 1 )
{
if ( $res )
{
$html_data .= "TCP OK $URL<br>\r\n";
}
else
{
$html_data .= "TCP NOK $URL<br>\r\n";
}
}
if ( $html_content == 2 )
{
if ( $res )
{
$html_data .= "<tr><td>TCP</td><td>OK</td><td>$URL</td></tr>\r\n";
}
else
{
$html_data .= "<tr><td>TCP</td><td>NOK</td><td>$URL</td></tr>\r\n";
}
}
$result += $res;
}
}
}
my $len;
if ( $html_content == 1 )
{
$html_data = "\r\n<html><body>\r\n$html_data</body></html>\r\n";
$len = ( length( $html_data ) ) - 2;
}
if ( $html_content == 2 )
{
$html_data = "\r\n<table align='center' border='1' >\r\n$html_data</table>\r\n";
$len = ( length( $html_data ) ) - 2;
}
if ( $result != $test_item )
{
my $header = "HTTP/1.0 503 Service Unavailable\r\n";
if ( $html_content )
{
$header .= "Content-Length: $len\r\nContent-Type: text/html; charset=iso-8859-1\r\n";
}
print $header . $html_data;
return;
}
my $header = "HTTP/1.0 200 OK\r\n";
if ( $html_content )
{
$header .= "Content-Length: $len\r\nContent-Type: text/html; charset=iso-8859-1\r\n";
}
print $header. $html_data;
}
1;
##########################################################
##########################################################
# function to REGEX on a GET from URL
# arg: uri
# regex to test (with extra parameter like perl e.g. /\bweb\d{2,3}/i )
# IP
# port (optionnal: default=80)
# ret: 0 if no reply
# 1 if reply
##########################################################
##########################################################
sub regex
{
my $url = shift;
my $regex = shift;
my $host = shift;
$regex =~ /\/(.*)\/(.*)/;
my $reg = $1;
my $ext = $2;
my %options;
$options{ 'agent' } = "LB_REGEX_PROBE/$VERSION";
$options{ 'timeout' } = 10;
my $ua = LWP::UserAgent->new( %options );
my $response = $ua->get( $url, "Host" => $host );
if ( $response->is_success )
{
my $html = $response->content;
if ( $ext =~ /i/ )
{
if ( $html =~ /$reg/si )
{
return 1;
}
}
else
{
if ( $html =~ /$reg/s )
{
return 1;
}
}
}
return 0;
}
##########################################################
##########################################################
# function to GET an URL (HTTP or FTP) ftp://FTPTest:6ccount4F@brice!@172.29.0.146
# arg: uri
# allowed code (comma seaparated)
# IP
# port (optionnal: default=80)
# ret: 0 if not the expected vcode
# 1 if the expected code is returned
##########################################################
##########################################################
sub get
{
my $url = shift;
my $code = shift;
my $host = shift;
$code =~ s/\s*//g;
my %codes = map { $_ => $_ } split /,/, $code;
my %options;
$options{ 'agent' } = "LB_HTTP_PROBE/$VERSION";
$options{ 'timeout' } = 10;
my $ua = LWP::UserAgent->new( %options );
my $response = $ua->get( $url, "Host" => $host );
if ( $response->is_success )
{
my $rc = $response->{ _rc };
if ( defined $codes{ $rc } )
{
return 1;
}
}
return 0;
}
##########################################################
##########################################################
# function to test a port on a host
# arg: hostip
# port
# timeout
# ret: 0 if not open
# 1 if open
##########################################################
##########################################################
sub TCP
{
use IO::Socket::PortState qw(check_ports);
my $remote_host = shift;
my $remote_port = shift;
my $timeout = shift;
my %porthash = ( tcp => { $remote_port => { name => 'to_test', } } );
check_ports( $remote_host, $timeout, \%porthash );
return $porthash{ tcp }{ $remote_port }{ open };
}
##############################################
# parse config file
# IN: File PATH
# Out: Ref to a hash with config data
##############################################
sub parse_conf
{
my $file = shift;
my $conf = new Config::General(
-ConfigFile => $file,
-ExtendedAccess => 1,
-AllowMultiOptions => "yes"
);
my %config = $conf->getall;
$port = $config{ port } || 9898;
$host = $config{ host } || '127.0.0.1';
$reparse = $config{ reparse } || 0;
$cidr_allow = $config{ cidr_allow } || '127.0.0.0/8';
$log_level = $config{ log_level } || 0;
$log_file = $config{ log_file } || "/var/log/check.log";
$pid_file = $config{ pid_file } || "/var/run/check.pid";
$daemon = $config{ daemon } || 0;
$min_servers = $config{ min_servers } || 5;
$min_spare_servers = $config{ min_spare_servers } || 2;
$max_spare_servers = $config{ max_spare_servers } || 10;
$max_servers = $config{ max_servers } || 50;
$html_content = $config{ content } || 0;
$pid_file =~ s/\$\$\$/$basename/g;
$pid_file =~ s/\$\$/$$/g;
$log_file =~ s/\$\$\$/$basename/g;
$log_file =~ s/\$\$/$$/g;
if ( !( keys %{ $config{ realserver } } ) )
{
die "No farm to test\n";
}
return ( \%config );
}

93
examples/check.conf Normal file
View File

@ -0,0 +1,93 @@
# listening port ( default 9898 )
port 9899
# on which IP to bind (default 127.0.0.1 ) * = all IP
#host 10.2.1.1
# which client addr is allow ( default 127.0.0.0/8 )
#cidr_allow = 0.0.0.0/0
# verbosity from 0 to 4 (default 0 = no log )
log_level = 1
# daemonize (default 0 = no )
daemon = 1
# content put a HTML content after header
# (default 0 = no content 1 = html 2 = table )
content = 2
# reparse the config file at each request ( default 0 = no )
# only SIGHUP reread the config file)
reparse = 1
# pid_file (default /var/run/check.pid )
# $$$ = basename of config file
# $$ = PID
pid_file=/var/run/CHECK_$$$.pid
# log_file (default /var/log/check.log )
# $$$ = basename of config file
# $$ = PID
log_file=/var/log/CHECK_$$$.log
# number of servers to keep running (default = 5)
min_servers = 2
# number of servers to have waiting for requests (default = 2)
min_spare_servers = 1
# maximum number of servers to have waiting for requests (default = 10)
max_spare_servers =1
# number of servers (default = 50)
max_servers = 2
###########################################################
# a server to check
# type could be get , regex or tcp
# get = do a http or ftp get and check the result code with
# the list, coma separated, provided ( default = 200,201 )
# hostheader is optional and send to the server if provided
# regex = do a http or ftp get and check the content result
# with regex provided
# hostheader is optional and send to the server if provided
# tcp = test if the tcp port provided is open
#<realserver>
# url=http://127.0.0.1:80/apache2-default/index.html
# type = get
# code=200,201
# hostheader = www.test.com
#</realserver>
#<realserver>
# url=http://127.0.0.1:82/apache2-default/index.html
# type = get
# code=200,201
# hostheader = www.myhost.com
#</realserver>
<realserver>
url= http://10.2.2.1
type = regex
regex= /qdAbm/
</realserver>
<realserver>
type = tcp
url = 10.2.2.1
port =80
</realserver>
#<realserver>
# type = get
# url = ftp://FTPuser:FTPpassword@10.2.3.1
# code=200,201
#</realserver>

View File

@ -68,6 +68,8 @@ struct server {
#ifdef CONFIG_HAP_CTTPROXY
struct sockaddr_in tproxy_addr; /* non-local address we want to bind to for connect() */
#endif
struct sockaddr_in check_addr;
int set_check_addr ;
short check_port; /* the port to use for the health checks */
int health; /* 0->rise-1 = bad; rise->rise+fall-1 = good */
int rise, fall; /* time in iterations */

View File

@ -1197,6 +1197,7 @@ int cfg_parse_listen(const char *file, int linenum, char **args)
newsrv->rise = DEF_RISETIME;
newsrv->fall = DEF_FALLTIME;
newsrv->health = newsrv->rise; /* up, but will fall down at first failure */
newsrv->set_check_addr = 0;
cur_arg = 3;
while (*args[cur_arg]) {
if (!strcmp(args[cur_arg], "cookie")) {
@ -1217,6 +1218,11 @@ int cfg_parse_listen(const char *file, int linenum, char **args)
newsrv->inter = atol(args[cur_arg + 1]);
cur_arg += 2;
}
else if (!strcmp(args[cur_arg], "addr")) {
newsrv->check_addr = *str2sa(args[cur_arg + 1]);
newsrv->set_check_addr = 1;
cur_arg += 2;
}
else if (!strcmp(args[cur_arg], "port")) {
newsrv->check_port = atol(args[cur_arg + 1]);
cur_arg += 2;

View File

@ -248,8 +248,18 @@ int process_chk(struct task *t)
(setsockopt(fd, IPPROTO_TCP, TCP_NODELAY, (char *) &one, sizeof(one)) != -1)) {
//fprintf(stderr, "process_chk: 3\n");
if ( s->set_check_addr == 1 )
{
/* we'll connect to the check addr specified on the server */
sa = s->check_addr;
}
else
{
/* we'll connect to the addr on the server */
sa = s->addr;
}
/* we'll connect to the check port on the server */
sa = s->addr;
sa.sin_port = htons(s->check_port);
/* allow specific binding :