haproxy/src/sink.c

284 lines
7.8 KiB
C
Raw Normal View History

MINOR: sink: create definitions a minimal code for event sinks The principle will be to be able to dispatch events to various destinations called "sinks". This is already done in part in logs where log servers can be either a UDP socket or a file descriptor. This will be needed with the new trace subsystem where we may also want to add ring buffers. And it turns out that all such destinations make sense at all places. Logs may need to be sent to a TCP server via a ring buffer, or consulted from the CLI. Trace events may need to be sent to stdout/stderr as well as to remote log servers. This patch creates a new structure "sink" aiming at addressing these similar needs. The goal is to merge together what is common to all of them, such as the output format, the dropped events count, etc, and also keep separately the target identification (network address, file descriptor). Provisions were made to have a "waiter" on the sink. For a TCP log server it will be the task to wake up after writing to the log buffer. For a ring buffer, it could be the list of watchers on the CLI running a "tail" operation and waiting for new events. A lock was also placed in the struct since many operations will require some locking, including the FD ones. The output formats covers those in use by logs and two extra ones prepending the ISO time in front of the message (convenient for stdio/buffer). For now only the generic infrastructure is present, no type-specific output is implemented. There's the sink_write() function which prepares and formats a message to be sent, trying hard to avoid copies and only using pointer manipulation, where the type-specific code just has to be added. Dropped messages are already counted (for now 100% drop). The message is put into an iovec array as it will be trivial to use with file descriptors and sockets.
2019-08-11 14:38:56 +00:00
/*
* Event sink management
*
* Copyright (C) 2000-2019 Willy Tarreau - w@1wt.eu
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation, version 2.1
* exclusively.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with this library; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
#include <common/compat.h>
#include <common/config.h>
#include <common/ist.h>
#include <common/mini-clist.h>
#include <common/time.h>
#include <proto/cli.h>
MINOR: sink: create definitions a minimal code for event sinks The principle will be to be able to dispatch events to various destinations called "sinks". This is already done in part in logs where log servers can be either a UDP socket or a file descriptor. This will be needed with the new trace subsystem where we may also want to add ring buffers. And it turns out that all such destinations make sense at all places. Logs may need to be sent to a TCP server via a ring buffer, or consulted from the CLI. Trace events may need to be sent to stdout/stderr as well as to remote log servers. This patch creates a new structure "sink" aiming at addressing these similar needs. The goal is to merge together what is common to all of them, such as the output format, the dropped events count, etc, and also keep separately the target identification (network address, file descriptor). Provisions were made to have a "waiter" on the sink. For a TCP log server it will be the task to wake up after writing to the log buffer. For a ring buffer, it could be the list of watchers on the CLI running a "tail" operation and waiting for new events. A lock was also placed in the struct since many operations will require some locking, including the FD ones. The output formats covers those in use by logs and two extra ones prepending the ISO time in front of the message (convenient for stdio/buffer). For now only the generic infrastructure is present, no type-specific output is implemented. There's the sink_write() function which prepares and formats a message to be sent, trying hard to avoid copies and only using pointer manipulation, where the type-specific code just has to be added. Dropped messages are already counted (for now 100% drop). The message is put into an iovec array as it will be trivial to use with file descriptors and sockets.
2019-08-11 14:38:56 +00:00
#include <proto/log.h>
#include <proto/ring.h>
MINOR: sink: create definitions a minimal code for event sinks The principle will be to be able to dispatch events to various destinations called "sinks". This is already done in part in logs where log servers can be either a UDP socket or a file descriptor. This will be needed with the new trace subsystem where we may also want to add ring buffers. And it turns out that all such destinations make sense at all places. Logs may need to be sent to a TCP server via a ring buffer, or consulted from the CLI. Trace events may need to be sent to stdout/stderr as well as to remote log servers. This patch creates a new structure "sink" aiming at addressing these similar needs. The goal is to merge together what is common to all of them, such as the output format, the dropped events count, etc, and also keep separately the target identification (network address, file descriptor). Provisions were made to have a "waiter" on the sink. For a TCP log server it will be the task to wake up after writing to the log buffer. For a ring buffer, it could be the list of watchers on the CLI running a "tail" operation and waiting for new events. A lock was also placed in the struct since many operations will require some locking, including the FD ones. The output formats covers those in use by logs and two extra ones prepending the ISO time in front of the message (convenient for stdio/buffer). For now only the generic infrastructure is present, no type-specific output is implemented. There's the sink_write() function which prepares and formats a message to be sent, trying hard to avoid copies and only using pointer manipulation, where the type-specific code just has to be added. Dropped messages are already counted (for now 100% drop). The message is put into an iovec array as it will be trivial to use with file descriptors and sockets.
2019-08-11 14:38:56 +00:00
#include <proto/sink.h>
#include <proto/stream_interface.h>
MINOR: sink: create definitions a minimal code for event sinks The principle will be to be able to dispatch events to various destinations called "sinks". This is already done in part in logs where log servers can be either a UDP socket or a file descriptor. This will be needed with the new trace subsystem where we may also want to add ring buffers. And it turns out that all such destinations make sense at all places. Logs may need to be sent to a TCP server via a ring buffer, or consulted from the CLI. Trace events may need to be sent to stdout/stderr as well as to remote log servers. This patch creates a new structure "sink" aiming at addressing these similar needs. The goal is to merge together what is common to all of them, such as the output format, the dropped events count, etc, and also keep separately the target identification (network address, file descriptor). Provisions were made to have a "waiter" on the sink. For a TCP log server it will be the task to wake up after writing to the log buffer. For a ring buffer, it could be the list of watchers on the CLI running a "tail" operation and waiting for new events. A lock was also placed in the struct since many operations will require some locking, including the FD ones. The output formats covers those in use by logs and two extra ones prepending the ISO time in front of the message (convenient for stdio/buffer). For now only the generic infrastructure is present, no type-specific output is implemented. There's the sink_write() function which prepares and formats a message to be sent, trying hard to avoid copies and only using pointer manipulation, where the type-specific code just has to be added. Dropped messages are already counted (for now 100% drop). The message is put into an iovec array as it will be trivial to use with file descriptors and sockets.
2019-08-11 14:38:56 +00:00
struct list sink_list = LIST_HEAD_INIT(sink_list);
struct sink *sink_find(const char *name)
{
struct sink *sink;
list_for_each_entry(sink, &sink_list, sink_list)
if (strcmp(sink->name, name) == 0)
return sink;
return NULL;
}
/* creates a new sink and adds it to the list, it's still generic and not fully
* initialized. Returns NULL on allocation failure. If another one already
* exists with the same name, it will be returned. The caller can detect it as
* a newly created one has type SINK_TYPE_NEW.
*/
static struct sink *__sink_new(const char *name, const char *desc, enum sink_fmt fmt)
MINOR: sink: create definitions a minimal code for event sinks The principle will be to be able to dispatch events to various destinations called "sinks". This is already done in part in logs where log servers can be either a UDP socket or a file descriptor. This will be needed with the new trace subsystem where we may also want to add ring buffers. And it turns out that all such destinations make sense at all places. Logs may need to be sent to a TCP server via a ring buffer, or consulted from the CLI. Trace events may need to be sent to stdout/stderr as well as to remote log servers. This patch creates a new structure "sink" aiming at addressing these similar needs. The goal is to merge together what is common to all of them, such as the output format, the dropped events count, etc, and also keep separately the target identification (network address, file descriptor). Provisions were made to have a "waiter" on the sink. For a TCP log server it will be the task to wake up after writing to the log buffer. For a ring buffer, it could be the list of watchers on the CLI running a "tail" operation and waiting for new events. A lock was also placed in the struct since many operations will require some locking, including the FD ones. The output formats covers those in use by logs and two extra ones prepending the ISO time in front of the message (convenient for stdio/buffer). For now only the generic infrastructure is present, no type-specific output is implemented. There's the sink_write() function which prepares and formats a message to be sent, trying hard to avoid copies and only using pointer manipulation, where the type-specific code just has to be added. Dropped messages are already counted (for now 100% drop). The message is put into an iovec array as it will be trivial to use with file descriptors and sockets.
2019-08-11 14:38:56 +00:00
{
struct sink *sink;
sink = sink_find(name);
if (sink)
goto end;
sink = malloc(sizeof(*sink));
if (!sink)
goto end;
sink->name = name;
sink->desc = desc;
sink->fmt = fmt;
sink->type = SINK_TYPE_NEW;
/* set defaults for syslog ones */
sink->syslog_facility = 0;
sink->syslog_minlvl = 0;
sink->maxlen = BUFSIZE;
MINOR: sink: create definitions a minimal code for event sinks The principle will be to be able to dispatch events to various destinations called "sinks". This is already done in part in logs where log servers can be either a UDP socket or a file descriptor. This will be needed with the new trace subsystem where we may also want to add ring buffers. And it turns out that all such destinations make sense at all places. Logs may need to be sent to a TCP server via a ring buffer, or consulted from the CLI. Trace events may need to be sent to stdout/stderr as well as to remote log servers. This patch creates a new structure "sink" aiming at addressing these similar needs. The goal is to merge together what is common to all of them, such as the output format, the dropped events count, etc, and also keep separately the target identification (network address, file descriptor). Provisions were made to have a "waiter" on the sink. For a TCP log server it will be the task to wake up after writing to the log buffer. For a ring buffer, it could be the list of watchers on the CLI running a "tail" operation and waiting for new events. A lock was also placed in the struct since many operations will require some locking, including the FD ones. The output formats covers those in use by logs and two extra ones prepending the ISO time in front of the message (convenient for stdio/buffer). For now only the generic infrastructure is present, no type-specific output is implemented. There's the sink_write() function which prepares and formats a message to be sent, trying hard to avoid copies and only using pointer manipulation, where the type-specific code just has to be added. Dropped messages are already counted (for now 100% drop). The message is put into an iovec array as it will be trivial to use with file descriptors and sockets.
2019-08-11 14:38:56 +00:00
/* address will be filled by the caller if needed */
sink->ctx.fd = -1;
MINOR: sink: create definitions a minimal code for event sinks The principle will be to be able to dispatch events to various destinations called "sinks". This is already done in part in logs where log servers can be either a UDP socket or a file descriptor. This will be needed with the new trace subsystem where we may also want to add ring buffers. And it turns out that all such destinations make sense at all places. Logs may need to be sent to a TCP server via a ring buffer, or consulted from the CLI. Trace events may need to be sent to stdout/stderr as well as to remote log servers. This patch creates a new structure "sink" aiming at addressing these similar needs. The goal is to merge together what is common to all of them, such as the output format, the dropped events count, etc, and also keep separately the target identification (network address, file descriptor). Provisions were made to have a "waiter" on the sink. For a TCP log server it will be the task to wake up after writing to the log buffer. For a ring buffer, it could be the list of watchers on the CLI running a "tail" operation and waiting for new events. A lock was also placed in the struct since many operations will require some locking, including the FD ones. The output formats covers those in use by logs and two extra ones prepending the ISO time in front of the message (convenient for stdio/buffer). For now only the generic infrastructure is present, no type-specific output is implemented. There's the sink_write() function which prepares and formats a message to be sent, trying hard to avoid copies and only using pointer manipulation, where the type-specific code just has to be added. Dropped messages are already counted (for now 100% drop). The message is put into an iovec array as it will be trivial to use with file descriptors and sockets.
2019-08-11 14:38:56 +00:00
sink->ctx.dropped = 0;
HA_RWLOCK_INIT(&sink->ctx.lock);
LIST_ADDQ(&sink_list, &sink->sink_list);
end:
return sink;
}
/* creates a sink called <name> of type FD associated to fd <fd>, format <fmt>,
* and description <desc>. Returns NULL on allocation failure or conflict.
* Perfect duplicates are merged (same type, fd, and name).
*/
struct sink *sink_new_fd(const char *name, const char *desc, enum sink_fmt fmt, int fd)
{
struct sink *sink;
sink = __sink_new(name, desc, fmt);
if (!sink || (sink->type == SINK_TYPE_FD && sink->ctx.fd == fd))
goto end;
if (sink->type != SINK_TYPE_NEW) {
sink = NULL;
goto end;
}
sink->type = SINK_TYPE_FD;
sink->ctx.fd = fd;
end:
return sink;
}
/* creates a sink called <name> of type BUF of size <size>, format <fmt>,
* and description <desc>. Returns NULL on allocation failure or conflict.
* Perfect duplicates are merged (same type and name). If sizes differ, the
* largest one is kept.
*/
struct sink *sink_new_buf(const char *name, const char *desc, enum sink_fmt fmt, size_t size)
{
struct sink *sink;
sink = __sink_new(name, desc, fmt);
if (!sink)
goto fail;
if (sink->type == SINK_TYPE_BUFFER) {
/* such a buffer already exists, we may have to resize it */
if (!ring_resize(sink->ctx.ring, size))
goto fail;
goto end;
}
if (sink->type != SINK_TYPE_NEW) {
/* already exists of another type */
goto fail;
}
sink->ctx.ring = ring_new(size);
if (!sink->ctx.ring) {
LIST_DEL(&sink->sink_list);
free(sink);
goto fail;
}
sink->type = SINK_TYPE_BUFFER;
end:
return sink;
fail:
return NULL;
}
MINOR: sink: create definitions a minimal code for event sinks The principle will be to be able to dispatch events to various destinations called "sinks". This is already done in part in logs where log servers can be either a UDP socket or a file descriptor. This will be needed with the new trace subsystem where we may also want to add ring buffers. And it turns out that all such destinations make sense at all places. Logs may need to be sent to a TCP server via a ring buffer, or consulted from the CLI. Trace events may need to be sent to stdout/stderr as well as to remote log servers. This patch creates a new structure "sink" aiming at addressing these similar needs. The goal is to merge together what is common to all of them, such as the output format, the dropped events count, etc, and also keep separately the target identification (network address, file descriptor). Provisions were made to have a "waiter" on the sink. For a TCP log server it will be the task to wake up after writing to the log buffer. For a ring buffer, it could be the list of watchers on the CLI running a "tail" operation and waiting for new events. A lock was also placed in the struct since many operations will require some locking, including the FD ones. The output formats covers those in use by logs and two extra ones prepending the ISO time in front of the message (convenient for stdio/buffer). For now only the generic infrastructure is present, no type-specific output is implemented. There's the sink_write() function which prepares and formats a message to be sent, trying hard to avoid copies and only using pointer manipulation, where the type-specific code just has to be added. Dropped messages are already counted (for now 100% drop). The message is put into an iovec array as it will be trivial to use with file descriptors and sockets.
2019-08-11 14:38:56 +00:00
/* tries to send <nmsg> message parts (up to 8, ignored above) from message
* array <msg> to sink <sink>. Formating according to the sink's preference is
* done here. Lost messages are NOT accounted for. It is preferable to call
* sink_write() instead which will also try to emit the number of dropped
* messages when there are any. It returns >0 if it could write anything,
* <=0 otherwise.
MINOR: sink: create definitions a minimal code for event sinks The principle will be to be able to dispatch events to various destinations called "sinks". This is already done in part in logs where log servers can be either a UDP socket or a file descriptor. This will be needed with the new trace subsystem where we may also want to add ring buffers. And it turns out that all such destinations make sense at all places. Logs may need to be sent to a TCP server via a ring buffer, or consulted from the CLI. Trace events may need to be sent to stdout/stderr as well as to remote log servers. This patch creates a new structure "sink" aiming at addressing these similar needs. The goal is to merge together what is common to all of them, such as the output format, the dropped events count, etc, and also keep separately the target identification (network address, file descriptor). Provisions were made to have a "waiter" on the sink. For a TCP log server it will be the task to wake up after writing to the log buffer. For a ring buffer, it could be the list of watchers on the CLI running a "tail" operation and waiting for new events. A lock was also placed in the struct since many operations will require some locking, including the FD ones. The output formats covers those in use by logs and two extra ones prepending the ISO time in front of the message (convenient for stdio/buffer). For now only the generic infrastructure is present, no type-specific output is implemented. There's the sink_write() function which prepares and formats a message to be sent, trying hard to avoid copies and only using pointer manipulation, where the type-specific code just has to be added. Dropped messages are already counted (for now 100% drop). The message is put into an iovec array as it will be trivial to use with file descriptors and sockets.
2019-08-11 14:38:56 +00:00
*/
ssize_t __sink_write(struct sink *sink, const struct ist msg[], size_t nmsg)
MINOR: sink: create definitions a minimal code for event sinks The principle will be to be able to dispatch events to various destinations called "sinks". This is already done in part in logs where log servers can be either a UDP socket or a file descriptor. This will be needed with the new trace subsystem where we may also want to add ring buffers. And it turns out that all such destinations make sense at all places. Logs may need to be sent to a TCP server via a ring buffer, or consulted from the CLI. Trace events may need to be sent to stdout/stderr as well as to remote log servers. This patch creates a new structure "sink" aiming at addressing these similar needs. The goal is to merge together what is common to all of them, such as the output format, the dropped events count, etc, and also keep separately the target identification (network address, file descriptor). Provisions were made to have a "waiter" on the sink. For a TCP log server it will be the task to wake up after writing to the log buffer. For a ring buffer, it could be the list of watchers on the CLI running a "tail" operation and waiting for new events. A lock was also placed in the struct since many operations will require some locking, including the FD ones. The output formats covers those in use by logs and two extra ones prepending the ISO time in front of the message (convenient for stdio/buffer). For now only the generic infrastructure is present, no type-specific output is implemented. There's the sink_write() function which prepares and formats a message to be sent, trying hard to avoid copies and only using pointer manipulation, where the type-specific code just has to be added. Dropped messages are already counted (for now 100% drop). The message is put into an iovec array as it will be trivial to use with file descriptors and sockets.
2019-08-11 14:38:56 +00:00
{
char short_hdr[4];
struct ist pfx[4];
size_t npfx = 0;
MINOR: sink: create definitions a minimal code for event sinks The principle will be to be able to dispatch events to various destinations called "sinks". This is already done in part in logs where log servers can be either a UDP socket or a file descriptor. This will be needed with the new trace subsystem where we may also want to add ring buffers. And it turns out that all such destinations make sense at all places. Logs may need to be sent to a TCP server via a ring buffer, or consulted from the CLI. Trace events may need to be sent to stdout/stderr as well as to remote log servers. This patch creates a new structure "sink" aiming at addressing these similar needs. The goal is to merge together what is common to all of them, such as the output format, the dropped events count, etc, and also keep separately the target identification (network address, file descriptor). Provisions were made to have a "waiter" on the sink. For a TCP log server it will be the task to wake up after writing to the log buffer. For a ring buffer, it could be the list of watchers on the CLI running a "tail" operation and waiting for new events. A lock was also placed in the struct since many operations will require some locking, including the FD ones. The output formats covers those in use by logs and two extra ones prepending the ISO time in front of the message (convenient for stdio/buffer). For now only the generic infrastructure is present, no type-specific output is implemented. There's the sink_write() function which prepares and formats a message to be sent, trying hard to avoid copies and only using pointer manipulation, where the type-specific code just has to be added. Dropped messages are already counted (for now 100% drop). The message is put into an iovec array as it will be trivial to use with file descriptors and sockets.
2019-08-11 14:38:56 +00:00
if (sink->fmt == SINK_FMT_SHORT || sink->fmt == SINK_FMT_TIMED) {
MINOR: sink: create definitions a minimal code for event sinks The principle will be to be able to dispatch events to various destinations called "sinks". This is already done in part in logs where log servers can be either a UDP socket or a file descriptor. This will be needed with the new trace subsystem where we may also want to add ring buffers. And it turns out that all such destinations make sense at all places. Logs may need to be sent to a TCP server via a ring buffer, or consulted from the CLI. Trace events may need to be sent to stdout/stderr as well as to remote log servers. This patch creates a new structure "sink" aiming at addressing these similar needs. The goal is to merge together what is common to all of them, such as the output format, the dropped events count, etc, and also keep separately the target identification (network address, file descriptor). Provisions were made to have a "waiter" on the sink. For a TCP log server it will be the task to wake up after writing to the log buffer. For a ring buffer, it could be the list of watchers on the CLI running a "tail" operation and waiting for new events. A lock was also placed in the struct since many operations will require some locking, including the FD ones. The output formats covers those in use by logs and two extra ones prepending the ISO time in front of the message (convenient for stdio/buffer). For now only the generic infrastructure is present, no type-specific output is implemented. There's the sink_write() function which prepares and formats a message to be sent, trying hard to avoid copies and only using pointer manipulation, where the type-specific code just has to be added. Dropped messages are already counted (for now 100% drop). The message is put into an iovec array as it will be trivial to use with file descriptors and sockets.
2019-08-11 14:38:56 +00:00
short_hdr[0] = '<';
short_hdr[1] = '0' + sink->syslog_minlvl;
short_hdr[2] = '>';
pfx[npfx].ptr = short_hdr;
pfx[npfx].len = 3;
npfx++;
}
MINOR: sink: create definitions a minimal code for event sinks The principle will be to be able to dispatch events to various destinations called "sinks". This is already done in part in logs where log servers can be either a UDP socket or a file descriptor. This will be needed with the new trace subsystem where we may also want to add ring buffers. And it turns out that all such destinations make sense at all places. Logs may need to be sent to a TCP server via a ring buffer, or consulted from the CLI. Trace events may need to be sent to stdout/stderr as well as to remote log servers. This patch creates a new structure "sink" aiming at addressing these similar needs. The goal is to merge together what is common to all of them, such as the output format, the dropped events count, etc, and also keep separately the target identification (network address, file descriptor). Provisions were made to have a "waiter" on the sink. For a TCP log server it will be the task to wake up after writing to the log buffer. For a ring buffer, it could be the list of watchers on the CLI running a "tail" operation and waiting for new events. A lock was also placed in the struct since many operations will require some locking, including the FD ones. The output formats covers those in use by logs and two extra ones prepending the ISO time in front of the message (convenient for stdio/buffer). For now only the generic infrastructure is present, no type-specific output is implemented. There's the sink_write() function which prepares and formats a message to be sent, trying hard to avoid copies and only using pointer manipulation, where the type-specific code just has to be added. Dropped messages are already counted (for now 100% drop). The message is put into an iovec array as it will be trivial to use with file descriptors and sockets.
2019-08-11 14:38:56 +00:00
if (sink->fmt == SINK_FMT_ISO || sink->fmt == SINK_FMT_TIMED) {
pfx[npfx].ptr = timeofday_as_iso_us(1);
pfx[npfx].len = 27;
npfx++;
}
if (sink->type == SINK_TYPE_FD) {
return fd_write_frag_line(sink->ctx.fd, sink->maxlen, pfx, npfx, msg, nmsg, 1);
}
else if (sink->type == SINK_TYPE_BUFFER) {
return ring_write(sink->ctx.ring, sink->maxlen, pfx, npfx, msg, nmsg);
}
return 0;
}
MINOR: sink: create definitions a minimal code for event sinks The principle will be to be able to dispatch events to various destinations called "sinks". This is already done in part in logs where log servers can be either a UDP socket or a file descriptor. This will be needed with the new trace subsystem where we may also want to add ring buffers. And it turns out that all such destinations make sense at all places. Logs may need to be sent to a TCP server via a ring buffer, or consulted from the CLI. Trace events may need to be sent to stdout/stderr as well as to remote log servers. This patch creates a new structure "sink" aiming at addressing these similar needs. The goal is to merge together what is common to all of them, such as the output format, the dropped events count, etc, and also keep separately the target identification (network address, file descriptor). Provisions were made to have a "waiter" on the sink. For a TCP log server it will be the task to wake up after writing to the log buffer. For a ring buffer, it could be the list of watchers on the CLI running a "tail" operation and waiting for new events. A lock was also placed in the struct since many operations will require some locking, including the FD ones. The output formats covers those in use by logs and two extra ones prepending the ISO time in front of the message (convenient for stdio/buffer). For now only the generic infrastructure is present, no type-specific output is implemented. There's the sink_write() function which prepares and formats a message to be sent, trying hard to avoid copies and only using pointer manipulation, where the type-specific code just has to be added. Dropped messages are already counted (for now 100% drop). The message is put into an iovec array as it will be trivial to use with file descriptors and sockets.
2019-08-11 14:38:56 +00:00
/* Tries to emit a message indicating the number of dropped events. In case of
* success, the amount of drops is reduced by as much. It's supposed to be
* called under an exclusive lock on the sink to avoid multiple produces doing
* the same. On success, >0 is returned, otherwise <=0 on failure.
*/
int sink_announce_dropped(struct sink *sink)
{
unsigned int dropped;
struct buffer msg;
struct ist msgvec[1];
char logbuf[64];
while (unlikely((dropped = sink->ctx.dropped) > 0)) {
chunk_init(&msg, logbuf, sizeof(logbuf));
chunk_printf(&msg, "%u event%s dropped", dropped, dropped > 1 ? "s" : "");
msgvec[0] = ist2(msg.area, msg.data);
if (__sink_write(sink, msgvec, 1) <= 0)
return 0;
/* success! */
HA_ATOMIC_SUB(&sink->ctx.dropped, dropped);
}
return 1;
MINOR: sink: create definitions a minimal code for event sinks The principle will be to be able to dispatch events to various destinations called "sinks". This is already done in part in logs where log servers can be either a UDP socket or a file descriptor. This will be needed with the new trace subsystem where we may also want to add ring buffers. And it turns out that all such destinations make sense at all places. Logs may need to be sent to a TCP server via a ring buffer, or consulted from the CLI. Trace events may need to be sent to stdout/stderr as well as to remote log servers. This patch creates a new structure "sink" aiming at addressing these similar needs. The goal is to merge together what is common to all of them, such as the output format, the dropped events count, etc, and also keep separately the target identification (network address, file descriptor). Provisions were made to have a "waiter" on the sink. For a TCP log server it will be the task to wake up after writing to the log buffer. For a ring buffer, it could be the list of watchers on the CLI running a "tail" operation and waiting for new events. A lock was also placed in the struct since many operations will require some locking, including the FD ones. The output formats covers those in use by logs and two extra ones prepending the ISO time in front of the message (convenient for stdio/buffer). For now only the generic infrastructure is present, no type-specific output is implemented. There's the sink_write() function which prepares and formats a message to be sent, trying hard to avoid copies and only using pointer manipulation, where the type-specific code just has to be added. Dropped messages are already counted (for now 100% drop). The message is put into an iovec array as it will be trivial to use with file descriptors and sockets.
2019-08-11 14:38:56 +00:00
}
/* parse the "show events" command, returns 1 if a message is returned, otherwise zero */
static int cli_parse_show_events(char **args, char *payload, struct appctx *appctx, void *private)
{
struct sink *sink;
int arg;
args++; // make args[1] the 1st arg
if (!*args[1]) {
/* no arg => report the list of supported sink */
chunk_printf(&trash, "Supported events sinks are listed below. Add -w(wait), -n(new). Any key to stop\n");
list_for_each_entry(sink, &sink_list, sink_list) {
chunk_appendf(&trash, " %-10s : type=%s, %u dropped, %s\n",
sink->name,
sink->type == SINK_TYPE_NEW ? "init" :
sink->type == SINK_TYPE_FD ? "fd" :
sink->type == SINK_TYPE_BUFFER ? "buffer" : "?",
sink->ctx.dropped, sink->desc);
}
trash.area[trash.data] = 0;
return cli_msg(appctx, LOG_WARNING, trash.area);
}
if (!cli_has_level(appctx, ACCESS_LVL_OPER))
return 1;
sink = sink_find(args[1]);
if (!sink)
return cli_err(appctx, "No such event sink");
if (sink->type != SINK_TYPE_BUFFER)
return cli_msg(appctx, LOG_NOTICE, "Nothing to report for this sink");
for (arg = 2; *args[arg]; arg++) {
if (strcmp(args[arg], "-w") == 0)
appctx->ctx.cli.i0 |= 1; // wait mode
else if (strcmp(args[arg], "-n") == 0)
appctx->ctx.cli.i0 |= 2; // seek to new
else if (strcmp(args[arg], "-nw") == 0 || strcmp(args[arg], "-wn") == 0)
appctx->ctx.cli.i0 |= 3; // seek to new + wait
else
return cli_err(appctx, "unknown option");
}
return ring_attach_cli(sink->ctx.ring, appctx);
}
static void sink_init()
{
sink_new_fd("stdout", "standard output (fd#1)", SINK_FMT_RAW, 1);
sink_new_fd("stderr", "standard output (fd#2)", SINK_FMT_RAW, 2);
MINOR: sink: change ring buffer "buf0"'s format to "timed" This way we now always have the events date which were really missing, especially when used with traces : <0>2019-09-26T07:57:25.183845 [00|h2|1|mux_h2.c:3024] receiving H2 HEADERS frame : h2c=0x1ddcad0(B,FRP) h2s=0x1dde9e0(3,HCL) <0>2019-09-26T07:57:25.183845 [00|h2|4|mux_h2.c:2505] h2c_bck_handle_headers(): entering : h2c=0x1ddcad0(B,FRP) h2s=0x1dde9> <0>2019-09-26T07:57:25.183846 [00|h2|4|mux_h2.c:4096] h2c_decode_headers(): entering : h2c=0x1ddcad0(B,FRP) <0>2019-09-26T07:57:25.183847 [00|h2|4|mux_h2.c:4298] h2c_decode_headers(): leaving : h2c=0x1ddcad0(B,FRH) <0>2019-09-26T07:57:25.183848 [00|h2|0|mux_h2.c:2559] rcvd H2 response : h2c=0x1ddcad0(B,FRH) : [3] H2 RES: HTTP/2.0 200 <0>2019-09-26T07:57:25.183849 [00|h2|4|mux_h2.c:2560] h2c_bck_handle_headers(): leaving : h2c=0x1ddcad0(B,FRH) h2s=0x1dde9e> <0>2019-09-26T07:57:25.183849 [00|h2|4|mux_h2.c:2866] h2_process_demux(): no more Rx data : h2c=0x1ddcad0(B,FRH) <0>2019-09-26T07:57:25.183849 [00|h2|4|mux_h2.c:3123] h2_process_demux(): notifying stream before switching SID : h2c=0x1dd> <0>2019-09-26T07:57:25.183850 [00|h2|4|mux_h2.c:1014] h2s_notify_recv(): in : h2c=0x1ddcad0(B,FRH) h2s=0x1dde9e0(3,HCL) <0>2019-09-26T07:57:25.183850 [00|h2|4|mux_h2.c:3135] h2_process_demux(): leaving : h2c=0x1ddcad0(B,FRH) <0>2019-09-26T07:57:25.183851 [00|h2|4|mux_h2.c:3319] h2_send(): entering : h2c=0x1ddcad0(B,FRH) <0>2019-09-26T07:57:25.183851 [00|h2|4|mux_h2.c:3145] h2_process_mux(): entering : h2c=0x1ddcad0(B,FRH) <0>2019-09-26T07:57:25.183851 [00|h2|4|mux_h2.c:3234] h2_process_mux(): leaving : h2c=0x1ddcad0(B,FRH) <0>2019-09-26T07:57:25.183852 [00|h2|4|mux_h2.c:3428] h2_send(): leaving with everything sent : h2c=0x1ddcad0(B,FRH) <0>2019-09-26T07:57:25.183852 [00|h2|4|mux_h2.c:3319] h2_send(): entering : h2c=0x1ddcad0(B,FRH) It looks like some format options could finally be separate from the sink, or maybe enforced. For example we could imagine making the date optional or its resolution configurable within a same buffer. Similarly, maybe trace events would like to always emit the date even on stdout, while traffic logs would prefer not to emit the date in the ring buffer given that there's already one in the message.
2019-09-26 06:05:15 +00:00
sink_new_buf("buf0", "in-memory ring buffer", SINK_FMT_TIMED, 1048576);
}
static void sink_deinit()
{
struct sink *sink, *sb;
list_for_each_entry_safe(sink, sb, &sink_list, sink_list) {
if (sink->type == SINK_TYPE_BUFFER)
ring_free(sink->ctx.ring);
LIST_DEL(&sink->sink_list);
free(sink);
}
}
INITCALL0(STG_REGISTER, sink_init);
REGISTER_POST_DEINIT(sink_deinit);
static struct cli_kw_list cli_kws = {{ },{
{ { "show", "events", NULL }, "show events [<sink>] : show event sink state", cli_parse_show_events, NULL, NULL },
{{},}
}};
INITCALL1(STG_REGISTER, cli_register_kw, &cli_kws);
MINOR: sink: create definitions a minimal code for event sinks The principle will be to be able to dispatch events to various destinations called "sinks". This is already done in part in logs where log servers can be either a UDP socket or a file descriptor. This will be needed with the new trace subsystem where we may also want to add ring buffers. And it turns out that all such destinations make sense at all places. Logs may need to be sent to a TCP server via a ring buffer, or consulted from the CLI. Trace events may need to be sent to stdout/stderr as well as to remote log servers. This patch creates a new structure "sink" aiming at addressing these similar needs. The goal is to merge together what is common to all of them, such as the output format, the dropped events count, etc, and also keep separately the target identification (network address, file descriptor). Provisions were made to have a "waiter" on the sink. For a TCP log server it will be the task to wake up after writing to the log buffer. For a ring buffer, it could be the list of watchers on the CLI running a "tail" operation and waiting for new events. A lock was also placed in the struct since many operations will require some locking, including the FD ones. The output formats covers those in use by logs and two extra ones prepending the ISO time in front of the message (convenient for stdio/buffer). For now only the generic infrastructure is present, no type-specific output is implemented. There's the sink_write() function which prepares and formats a message to be sent, trying hard to avoid copies and only using pointer manipulation, where the type-specific code just has to be added. Dropped messages are already counted (for now 100% drop). The message is put into an iovec array as it will be trivial to use with file descriptors and sockets.
2019-08-11 14:38:56 +00:00
/*
* Local variables:
* c-indent-level: 8
* c-basic-offset: 8
* End:
*/