VMOD-BINLOG

NAME
SYNOPSIS
DESCRIPTION
FUNCTIONS
FORMAT
DOWNLOADS
SEE ALSO
AUTHORS
BUG REPORTS
COPYRIGHT

NAME

vmod-binlog - binary log file support for Varnish Cache.

SYNOPSIS

import binlog;

VOID binlog.init(STRING dir, STRING format, STRING param);

VOID binlog.start();

VOID binlog.pack(STRING arg);

VOID binlog.commit();

DESCRIPTION

This module adds binary log files support to Varnish Cache.

Binary log file consists of an arbitrary number of equally sized records. Each record contains a UNIX timestamp in binary form and user-defined data. The file begins with a header containing auxiliary information.

Binary log files are rotated at preconfigured time intervals. In order to speed up searches, their names follow the strftime(3) pattern:

%Y%m%dT%H%M%S.log

where conversion specifiers refer to the time when the log was created. The logs are stored in an indexed database structure. Three levels of indexing are supported:

0

Directories indexed by the year of creation. E.g. the log for September 22, 2013 is stored in

$dir/2013/20130922T000000.log

(where $dir refers to the top-level storage directory). This is the default indexing scheme.

1

Directories indexed by the year and month of creation. E.g.:

$dir/2013/09/20130922T000000.log

2

Directories indexed by the year, month and day. E.g.:

$dir/2013/09/22/20130922T000000.log

Binary logs can be viewed using the binlogcat(1) and binlogsel(1) utilities.

FUNCTIONS

Binary log support is initialized by a call to binlog.init function (normally it is done in vcl_init).

The dir argument specifies the top-level storage directory. The format argument defines the format of logfile entries. It is a simplified form of Perl pack() template. See the FORMAT section below for a detailed description. Finally, param is a semicolon-separated list of parameters. Each parameter is defined as a key/value pair, with a single equals sign in between. No whitespace is allowed on either side of the equals sign. The following parameters are defined:
debug
=NUMBER

Set debugging level.

pattern=STRING

Sets pattern to use for log file names, instead of the default. The argument can contain strftime(3) conversion specifiers. The default is

%Y%m%dT%H%M%S.log

index=NUMBER

Sets directory indexing scheme to use. Allowed values are 0, 1, and 2. See the description of indexing schemes above.

size=SIZE

Sets maximum size of a single binary log. Any records not fitting into SIZE will be discarded. SIZE can be followed by one of the usual size suffixes: K, M, G (or their lower-case equivalents), standing for kilobytes, megabytes and gigabytes, correspondingly.

Logs are mapped into memory, so the SIZE is limited. See mmap(2), for the details.

The default value is 1G.

interval=NUMBER

Sets log rotation interval, in seconds. The default is 86400.

umask=ONUM

Sets umask for creating new log files (a three-digit octal value). The default is 077.

roundts=BOOL

If set to 1, timestamps used for generating log file names will be rounded to the nearest rotation interval. E.g., if the log file was created at 01:05:18 on September 22, 2013 and roundts=1 is in effect, the log will be named 20130922T000000.log, instead of 20130922T010518.log.

reuselog=BOOL

If the file to be created already exists upon varnishd startup, the module will try to append new data to it, provided that its format matches the one actually configured. This avoids loss of data upon restarting varnishd. Setting reuselog=0 disables this behavior.

A new log record is initialized by binlog.start(). This call should be followed by as many calls to binlog.pack() as there are conversions in the format string. Each call to binlog.pack() converts its argument according to the next unused format specifier and stores it into the record being constructed. Finally, calling binlog.commit() writes the record to the log file.

For example, consider the format specification

L Z256

(i.e. a 32-bit unsigned long value, followed by a string of up to 255 characters). Given this, the following sequence of VCL commands can be used to construct the log entry from the HTTP headers:

binlog.start();
binlog.pack(http.X-Id);
binlog.pack(http.req.url);
binlog.commit();

FORMAT

Data format specification consists of a series of conversion specifiers, optionally separated by any amount of whitespace. A conversion specifier consists of a template letter optionally followed by numeric repeat count (which may be enclosed in square brackets).

The valid template letters are:

Z[N]

A null-terminated (ASCIZ) string of at most N-1 characters, will be null padded. The repeat count is mandatory.

c

A signed char (8-bit) value.

s

A signed short (16-bit) value.

S

An unsigned short value.

l

A signed long (32-bit) value.

L

An unsigned long value.

q

A signed quad (64-bit) value.

Q

An unsigned quad value.

i

A signed integer value.

I

A unsigned integer value.

n

An unsigned short (16-bit) in "network" (big-endian) order.

N

An unsigned long (32-bit) in "network" (big-endian) order.

v

An unsigned short (16-bit) in "VAX" (little-endian) order.

V

An unsigned long (32-bit) in "VAX" (little-endian) order.

f

A single-precision float in native format.

d

A double-precision float in native format.

x

A null byte (a.k.a ASCII NUL, " 00", chr(0))

X

Back up a byte.

@

Null-fill or truncate to absolute position, counted from the current position.

.

Null-fill or truncate to absolute position specified by the repeat count.

DOWNLOADS

Vmod-binlog is available for download from this location.

The latest version is vmod-binlog-2.7.

Recent news, changes and bugfixes can be tracked from the project's development page.

SEE ALSO

binlogcat(1), binlogsel(1), vcl(7), varnishd(1).

AUTHORS

Sergey Poznyakoff

BUG REPORTS

Report bugs to <gray@gnu.org>.

COPYRIGHT

Copyright © 2013-2022 Sergey Poznyakoff
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.


Manpage server at man.gnu.org.ua.

Powered by mansrv 1.1