VMOD-BINLOG
NAMESYNOPSIS
DESCRIPTION
FUNCTIONS
FORMAT
DOWNLOADS
SEE ALSO
AUTHORS
BUG REPORTS
COPYRIGHT
NAME
vmod-binlog - binary log file support for Varnish Cache.
SYNOPSIS
import binlog;
VOID binlog.init(STRING dir, STRING format, STRING param);
VOID binlog.start();
VOID binlog.pack(STRING arg);
VOID binlog.commit();
DESCRIPTION
This module adds binary log files support to Varnish Cache.
Binary log file consists of an arbitrary number of equally sized records. Each record contains a UNIX timestamp in binary form and user-defined data. The file begins with a header containing auxiliary information.
Binary log files are rotated at preconfigured time intervals. In order to speed up searches, their names follow the strftime(3) pattern:
%Y%m%dT%H%M%S.log
where conversion specifiers refer to the time when the log was created. The logs are stored in an indexed database structure. Three levels of indexing are supported:
0 |
Directories indexed by the year of creation. E.g. the log for September 22, 2013 is stored in |
$dir/2013/20130922T000000.log
(where $dir refers to the top-level storage directory). This is the default indexing scheme.
1 |
Directories indexed by the year and month of creation. E.g.: |
$dir/2013/09/20130922T000000.log
2 |
Directories indexed by the year, month and day. E.g.: |
$dir/2013/09/22/20130922T000000.log
Binary logs can be viewed using the binlogcat(1) and binlogsel(1) utilities.
FUNCTIONS
Binary log support is initialized by a call to binlog.init function (normally it is done in vcl_init).
The dir
argument specifies the top-level storage directory. The
format argument defines the format of logfile
entries. It is a simplified form of Perl pack()
template. See the FORMAT section below for a detailed
description. Finally, param is a semicolon-separated
list of parameters. Each parameter is defined as a key/value
pair, with a single equals sign in between. No whitespace is
allowed on either side of the equals sign. The following
parameters are defined:
debug=NUMBER
Set debugging level.
pattern=STRING
Sets pattern to use for log file names, instead of the default. The argument can contain strftime(3) conversion specifiers. The default is
%Y%m%dT%H%M%S.log
index=NUMBER
Sets directory indexing scheme to use. Allowed values are 0, 1, and 2. See the description of indexing schemes above.
size=SIZE
Sets maximum size of a single binary log. Any records not fitting into SIZE will be discarded. SIZE can be followed by one of the usual size suffixes: K, M, G (or their lower-case equivalents), standing for kilobytes, megabytes and gigabytes, correspondingly.
Logs are mapped into memory, so the SIZE is limited. See mmap(2), for the details.
The default value is 1G.
interval=NUMBER
Sets log rotation interval, in seconds. The default is 86400.
umask=ONUM
Sets umask for creating new log files (a three-digit octal value). The default is 077.
roundts=BOOL
If set to 1, timestamps used for generating log file names will be rounded to the nearest rotation interval. E.g., if the log file was created at 01:05:18 on September 22, 2013 and roundts=1 is in effect, the log will be named 20130922T000000.log, instead of 20130922T010518.log.
reuselog=BOOL
If the file to be created already exists upon varnishd startup, the module will try to append new data to it, provided that its format matches the one actually configured. This avoids loss of data upon restarting varnishd. Setting reuselog=0 disables this behavior.
A new log record is initialized by binlog.start(). This call should be followed by as many calls to binlog.pack() as there are conversions in the format string. Each call to binlog.pack() converts its argument according to the next unused format specifier and stores it into the record being constructed. Finally, calling binlog.commit() writes the record to the log file.
For example, consider the format specification
L Z256
(i.e. a 32-bit unsigned long value, followed by a string of up to 255 characters). Given this, the following sequence of VCL commands can be used to construct the log entry from the HTTP headers:
binlog.start();
binlog.pack(http.X-Id);
binlog.pack(http.req.url);
binlog.commit();
FORMAT
Data format specification consists of a series of conversion specifiers, optionally separated by any amount of whitespace. A conversion specifier consists of a template letter optionally followed by numeric repeat count (which may be enclosed in square brackets).
The valid template letters are:
Z[N] |
A null-terminated (ASCIZ) string of at most N-1 characters, will be null padded. The repeat count is mandatory. | ||
c |
A signed char (8-bit) value. | ||
s |
A signed short (16-bit) value. | ||
S |
An unsigned short value. | ||
l |
A signed long (32-bit) value. | ||
L |
An unsigned long value. | ||
q |
A signed quad (64-bit) value. | ||
Q |
An unsigned quad value. | ||
i |
A signed integer value. | ||
I |
A unsigned integer value. | ||
n |
An unsigned short (16-bit) in "network" (big-endian) order. | ||
N |
An unsigned long (32-bit) in "network" (big-endian) order. | ||
v |
An unsigned short (16-bit) in "VAX" (little-endian) order. | ||
V |
An unsigned long (32-bit) in "VAX" (little-endian) order. | ||
f |
A single-precision float in native format. | ||
d |
A double-precision float in native format. | ||
x |
A null byte (a.k.a ASCII NUL, " 00", chr(0)) | ||
X |
Back up a byte. | ||
@ |
Null-fill or truncate to absolute position, counted from the current position. | ||
. |
Null-fill or truncate to absolute position specified by the repeat count. |
DOWNLOADS
Vmod-binlog is available for download from this location.
The latest version is vmod-binlog-2.7.
Recent news, changes and bugfixes can be tracked from the project's development page.
SEE ALSO
binlogcat(1), binlogsel(1), vcl(7), varnishd(1).
AUTHORS
Sergey Poznyakoff
BUG REPORTS
Report bugs to <gray@gnu.org>.
COPYRIGHT
Copyright
© 2013-2022 Sergey Poznyakoff
License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and
redistribute it. There is NO WARRANTY, to the extent
permitted by law.
Manpage server at man.gnu.org.ua.
Powered by mansrv 1.1