Peter van Dijk contributed this patch which implements the "smtpchk"
option, which is to SMTP what "httpchk" is to HTTP. By default, it sends
"HELO localhost" to the servers, and waits for the 250 message, but it
can also send a specific request.
The pollers will now be able to speculatively call the I/O
processing functions and decide whether or not they want to
poll on those FDs. The changes primarily consist in teaching
those functions how to pass the info they got an EAGAIN.
Generally, if a recv() returns less bytes than the MSS, it means that
there is nothing left in the system's buffers, and that it's not worth
trying to read again because we are very likely to get nothing. A
default read low limit has been set to 1460 bytes below which we stop
reading.
This has brought a little speed boost on small objects while maintaining
the same speed on large objects.
Multiple read polling was temporarily disabled, which had the side
effect of burning huge amounts of CPU on large objects. It has now
been re-implemented with a limit of 8 calls per wake-up, which seems
to provide best results at least on Linux.
This structure will consume 4 bytes per header to keep track of
headers within a request or a response without having to parse
the whole request for each regex. As it's not possible to allocate
only 4 bytes, we define a max number of HTTP headers. We set it
to (BUFSIZE+79)/80 so that 8kB buffers can contain 100 headers
(like Apache), resulting in 400 bytes dedicated to indexation,
or about 400/(2*8kB) ~= 2.4% of the memory usage.
