Mfmod_openmetrics
1 Overview
2 Installation
2.1 Bootstrapping
2.2 Configuring the Package
2.3 Installing
3 Usage
3.1 Pairing Actions with Metrics
4 Functions
5 Database Errors
6 Configuration
7 Downloads and Other Links
8 Bug Reports
Appendix A License
Index
Mfmod_openmetrics
*****************
This edition of the 'mfmod_openmetrics Manual', last updated 20 June
2023, documents 'mfmod_openmetrics' version 1.2.
1 Overview
**********
This package provides OpenMetrics support for 'mailfromd', a powerful
mail filtering utility designed for use with mail transport agents
supporting the "milter" protocol (*note Mailfromd: (mailfromd)Top.).
OpenMetrics is a plaintext protocol for representing telemetry data,
which was originally introduced by 'prometheus' systems monitoring and
alerting toolkit. It is formally described in
.
The 'mfmod_openmetrics' module provides MFL functions for declaring
and manipulating metric sets from 'mailfromd' filter source file and
installs an HTTP server for answering the metric queries.
2 Installation
**************
To compile 'mfmod_openmetrics', you will need 'mailfromd' version 8.15
or later and the following packages:
1. GNU dbm version 1.19 or later.
.
2. GNU Libmicrohttpd.
For Debian users, that means installing the following additional
packages: 'libgdbm-dev', 'libmicrohttpd-dev'.
The installation procedure consists of the traditional three steps:
"configure", "make" and "install".
2.1 Bootstrapping
=================
If you have cloned the package from git, you will need to "bootstrap" it
first. Otherwise, if you are installing the package from the tarball,
skip this section and proceed directly to *note configure::.
In order to "bootstrap" the package, you will need the following
additional packages:
1. GNU autoconf .
2. GNU automake .
3. GNU libtool .
To bootstrap the package, run
autoreconf -f -i -s
If it cannot find the 'AC_MFMOD' macro, use the '-I' option to point
it to the directory where 'mfmod.m4' file is installed, e.g.:
autoreconf -f -i -s -I /usr/local/share/aclocal/
2.2 Configuring the Package
===========================
To configure the package, run 'configure' from its top-level source
directory. If you prefer to build the package in a separate directory
(e.g. 'build' located within the package top-level directory), change
to that directory and run 'configure' via its relative path, e.g.:
mkdir build
cd build
../configure
For a detailed discussion of generic 'configure' options, refer to
*Note (autoconf)Running configure Scripts::. The rest of this section
concerns 'configure' options specific for configuring
'mfmod_openmetrics'.
By default, 'configure' prepares the module parts to be installed in
the library and module directories used by mailfromd. To find these
directories, 'configure' will invoke 'mailfromd'. This means that it
should be able to find 'mailfromd' using the 'PATH' environment
variable. If it isn't the case, supply the modified 'PATH' as argument
to 'configure', like that:
./configure PATH=/usr/local/sbin:$PATH
If the need be, you can select the installation directories manually,
using the following two command line options:
'--with-mfmoddir=DIR'
Install loadable library ('mfmod_openmetrics.so') in DIR.
'--with-mfldir=DIR'
Install interface module ('openmetrics.mfl') in DIR.
It is advisable to configure the module so as to use the system
configuration directory used by the 'mailfromd' server. For example, if
your 'mailfromd' looks for configuration in '/etc/mailfromd', do:
./configure --sysconfdir=/etc/mailfromd
This will set the location where the module will look for its
configuration file (*note Configuration::).
If you are unsure what system configuration directory your
'mailfromd' installation uses, run
mailfromd --no-config --show-defaults
and examine the 'script file search path:' line. The domain part of its
first element is the system configuration directory.
2.3 Installing
==============
Once successfully configured, run 'make'. This should build the shared
library and prepare the package for installation.
Finally, run 'make install' as root, to install the package.
3 Usage
*******
Add the following line at the beginning of your filter script:
require 'openmetrics'
This will cause module to be loaded and initialized. During
initialization, the module will parse the configuration file, if it
exists, create the metric database or open an existing one, and set up
HTTP server thread for answering metric queries. If any of these
actions fails, the module signals a 'e_failure' exception.
Settings from the configuration file 'mfmod_openmetrics.conf', if it
exists in the system configuration directory, override default module
settings. *Note Configuration::, for a detailed discussion.
The "metric database file" is a GNU dbm file that keeps current
metric values. It can be "temporary", i.e. created each time
'mailfromd' starts up and destroyed when it terminates, or "persistent",
so that the data persist across program restarts. *Note database
configuration::, for details.
By default HTTP server thread listens on IP address 127.0.0.1
(localhost), port 8080. These settings can also be changed in the
configuration.
After loading the module, declare metric family names you are going
to use in your script. A metric family is defined using the following
function:
openmetrics_declare(string NAME, number TYPE, string HELP)
The NAME parameter gives the metric family name.
The TYPE parameter declares the type of the metric. It can be one of
the following: 'openmetrics_counter', 'openmetrics_gauge', or
'openmetrics_duration'. The first two are floating-point counters. The
main difference between them is that 'openmetrics_counter' can only
increase, while 'openmetrics_gauge' can both increase and decrease. The
'openmetrics_duration' type is a special case of 'openmetrics_gauge'
that represents the number of seconds elapsed since the counter was
created or reset.
The HELP parameter supplies the help text.
The best place for declaring your metric families is in the 'startup'
handler, e.g.:
prog startup
do
openmetrics_declare("accept", openmetrics_counter, "Mails accepted")
openmetrics_declare("reject", openmetrics_counter, "Mails rejected")
openmetrics_declare("tempfail", openmetrics_counter, "Mails tempfailed")
done
If the supplied metric family already exists, its type and help text
are checked against 'openmetrics_declare' arguments. If types does not
match, the 'e_inval' exception is signaled. Otherwise, if types match
but help texts don't, the help text is updated. Actual values of the
metrics in the family are not changed.
Once the metric family is declared, you can manipulate the metrics in
it using the following functions:
func openmetrics_add(string NAME, string LABELSET, number DELTA)
func openmetrics_incr(string NAME; string LABELSET)
func openmetrics_reset(string NAME; string LABELSET)
func openmetrics_set(string NAME, string LABELSET, number VALUE)
Each of them identifies the metric to operate upon by its first two
arguments: NAME, which gives the metric family name, and LABELSET, which
supplies the "label set" to qualify the metric name (the latter
parameter is optional if these are the only arguments the function
takes). For the purposes of 'mfmod_openmetrics', a "label set" is a
comma-delimited list of "labels", which take form of 'KEY=VALUE' pairs.
If label set is empty, metric name is same as the metric family name.
Otherwise, metric name is 'NAME{LABELSET}'. E.g. the following call
increases the value of the (counter) metric 'action_total':
openmetrics_incr("action_total")
Now consider the following call:
openmetrics_incr("action_total", 'action="reject"')
It is passed a non-empty label set as its second argument and
therefore increases the value of the metric
'action_total{action="reject"}'.
These functions are described in detail in chapter *note Functions::.
Finally, add calls to the appropriate functions in the right places
of your filter script. For example, to keep track of the number of
'accept', 'reject', and 'tempfail' actions run, use 'openmetrics_incr'
before these actions, e.g.:
openmetrics_incr("reject")
reject 550 5.1.0 "Sender validity not confirmed"
Now, restart 'mailfromd' and check if the metrics are returned by the
HTTP endpoint:
curl -v http://127.0.0.1:8080/metric
3.1 Pairing Actions with Metrics
================================
Placing a call to 'openmetrics_incr' before each action, as proposed
above, is cumbersome and error-prone. To make sure metrics are
increased upon executing each action, use the 'action' mailfromd hook
(*note Mailfromd: (mailfromd)action hook.). For example:
prog action
do
openmetrics_incr(milter_action_name($1))
done
This handler will be called before executing each action. Make sure
to declare metric families for each 'mailfromd' actions, like that:
prog startup
do
openmetrics_declare("accept", openmetrics_counter, "Mails accepted")
openmetrics_declare("continue", openmetrics_counter,
"Flow continued to other handlers")
openmetrics_declare("discard", openmetrics_counter, "Mails discarded")
openmetrics_declare("reject", openmetrics_counter, "Mails rejected")
openmetrics_declare("tempfail", openmetrics_counter, "Mails tempfailed")
set openmetrics_safe 1
done
The 'action' handler appeared in 'mailfromd' version 8.16.90. If you
run an earlier version, there is still a way to ensure that
corresponding metrics are increased upon executing each action. This
will require some preprocessor magic, though.
Define the following 'm4' macro:
m4_define(`ACTION',
`openmetrics_incr("$1")
$1`'m4_ifelse(`$2',`',`',`(m4_shift($@))')')
Place this definition near the top of your 'mailfromd.mfl' file and
replace literal calls to milter actions with calls to this macro. For
example, instead of
openmetrics_incr("accept")
accept
write just
ACTION(accept)
Similarly, instead of
openmetrics_incr("reject")
reject 550 5.1.0 "Sender validity not confirmed"
use
ACTION(reject, 550, 5.1.0, "Sender validity not confirmed")
This way calling an action will always increase the corresponding
metric.
4 Functions
***********
-- function: openmetrics_declare (string NAME, number TYPE, string
HELP, ...)
Declares the metric NAME with the given TYPE and HELP text.
Allowed values for TYPE are:
-- type: openmetrics_counter
Counter value. Metrics of this type can only increase over
time.
-- type: openmetrics_gauge
Counter value that can both increase and decrease.
-- type: openmetrics_duration
Special case of 'openmetrics_gauge' that represents the number
of seconds elapsed since the counter was created or reset.
This is useful for reporting uptime and similar values.
If the supplied metric already exists, its type and help text are
checked against 'openmetrics_declare' arguments. If types does not
match, the 'e_inval' exception is signaled (*note e_inval:
(mailfromd)Built-in Exceptions.). Otherwise, if types match but
help texts don't, the help text is updated. Actual values of the
metric are not changed.
Any number of optional arguments can be given. They specify the
LabelSets to initialize when creating the metric. To illustrate
their effect, consider the following two statements:
openmetrics_declare("age", openmetrics_duration,
"Seconds since the database was created")
openmetrics_declare("uptime", openmetrics_duration,
"Mailfromd instance uptime", "")
When metric database is created, the 'age' metric is created
without any actual instances of the metric. Until you create one
with 'openmetrics_set', 'openmetrics_reset' or similar function,
you won't see this metric in responses from the '/metric' endpoint.
In the contrast, the declaration of 'uptime' creates the formal
description of the metric, and defines a single instance with an
empty label set. You will see the 'uptime' variable in the
responses right after creating the database.
The 'openmetrics_declare' function can raise the following
exceptions:
e_inval
Bad number of arguments, or argument types, or bad metric type
given.
e_failure
Failed to create temporary database file.
e_dbfailure
GDBM error.
-- function: openmetrics_incr (string NAME; string LABELSET)
Increment the value of the metric NAME by one. Optional LABELSET
argument supplies a comma-separated list of 'NAME=VALUE' parameters
("labels") that qualify the counter name. If it is given, the
actual metric name is 'NAME{LABELSET}'.
This function can raise the following exceptions:
'e_inval'
Bad number or types of arguments.
'e_failure'
Failed to create temporary database file.
'e_dbfailure'
GDBM error. Most often this occurs because the database is
locked by another process. See 'database-retry' and
'database-wait' in *note Configuration::.
'e_not_found'
Metric not declared. This means the appropriate
'openmetrics_declare' call is missing from 'prog startup'.
-- function: openmetrics_add (string NAME, string LABELSET, number
DELTA)
Updates the value of the metric 'NAME{LABELSET}' (counter or gauge)
by adding DELTA to it.
This function can raise the same exceptions as 'openmetrics_incr'.
-- function: openmetrics_reset (string NAME; string LABELSET)
Resets the metric NAME (or 'NAME{LABELSET}', if LABELSET is given)
to its original value, i.e. 0, for 'openmetrics_counter' and
'openmetrics_gauge' types, and the current system local time
(seconds since Epoch), for 'openmetrics_duration'.
This function can raise the same exceptions as 'openmetrics_incr'.
-- function: openmetrics_set (string NAME, string LABELSET, number
VALUE; number IFNEW)
Sets the 'NAME{LABELSET}' metric to the given value.
If IFNEW is not null, the value will be set only unless it already
exits.
This function can raise the same exceptions as 'openmetrics_incr'.
-- function: string openmetrics_http_address()
Returns network "address" the module is listening on for HTTP
requests. The return value is formatted as 'IP:PORT'.
5 Database Errors
*****************
The module stores the collected metrics in a GNU DBM file. Normally
only one process can have write access to the database. While it is
updating the database, another processes wishing to obtain access
(whether read-write or read-only) will wait until it has finished. This
waiting consists in several "attempts". During each attempt the process
sleeps for certain amount of time, and then tries to lock and open the
database. Two "configuration variables" help tune this process:
'database-retry' sets the number of attempts to take, and
'database-wait' defines the time to sleep within each attempt. These
are discussed in detail in *note Configuration::.
Even in most meticulously configured systems, a possibility of a
timeout still remains. Such timeouts can happen in the following
functions: 'openmetrics_add', 'openmetrics_incr', 'openmetrics_set', and
'openmetrics_reset'. Beside this, another errors that can occur when
accessing database files affect these functions. By default, when a
database-related error occurs these functions raise the 'e_dbfailure'
exception. Unless properly caught and handled, this exception will
cause the filter to return temporary failure and terminate prematurely.
Obviously such behavior is not well-suited for production environments.
To avoid it, 'mfmod_openmetrics' provides the following variable:
-- Variable: number openmetrics_safe
Control gracious handling of database-related expressions.
If set to 1, instead of raising exceptions in case of error, the
above-mentioned functions will report the error using 'echo' and
return. The execution of the filter will continue normally.
By default this variable is initialized to 0. It is recommended to
set it to 1 in the 'startup' handler.
6 Configuration
***************
The module looks for file 'mfmod_openmetrics.conf' in the system
configuration directory (*note sysconfdir::). If the environment
variable 'MFMOD_OPENMETRICS_CONF' is defined, its value is used as the
full pathname if the configuration file, instead of the default
location.
The file format is described in *note Configuration File Syntax:
(mailutils)conf-syntax.
The following statements are defined:
General Configuration
---------------------
-- Configuration: namespace STRING
Sets "namespace prefix", a string which will be added (with an
underscore) at the start of each metric name in the output.
Database Configuration
----------------------
-- Configuration: persistent BOOL
If 'true', the metrics database will persist across restarts of
'mailfromd'. By default the database is temporary and is deleted
when 'mailfromd' terminates. If you set the persistent mode, the
default database name will be '/tmp/mfmetrics.db'. You can change
it using the 'database-name' configuration statement.
-- Configuration: database-name STRING
Name of the metric database file. Normally this is important only
when used together with 'persistent on'. Unless STRING is absolute
pathname, the file will be created in '/tmp'. When selecting
directory for the database file, be sure it is writable for the
user 'mailfromd' runs as.
-- Configuration: database-retry N
If the database file is in use by another process, retry opening it
N times. Default value is 10.
-- Configuration: database-wait S
Sleep S seconds (floating-point number) between two successive
attempts to open the database file. The default wait time is 0.1
second.
HTTP Server Configuration
-------------------------
-- Configuration: listen "IP:PORT"
Configure the IP and PORT to listen for HTTP requests. It can have
the following forms:
'listen "IP:PORT";'
Listen on the given IP and PORT.
'listen "IP";'
Listen on the given IP, port 8080.
'listen ":PORT";'
Listen on 127.0.0.1, port PORT.
'listen any;'
Listen on the first available port on 127.0.0.1. This is for
testing the module. The actual address the module is
listening on for HTTP is returned by the function
'openmetrics_http_address'.
'listen none;'
Disable HTTP server. This is reserved for the case when HTTP
requests are handled by some third party.
The default corresponds to 'listen "127.0.0.1:8080"'.
-- Configuration: access-log BOOL
Enable HTTP access logging. E.g.:
access-log yes;
The logs are produced on the 'mailfromd' error stream. Each log
line describes a single access and contains the following fields:
* IP address and port of the requesting machine.
* Client IP address. If the request contains the
'X-Forwarded-For' header, it is used to determine that value
(see the 'trusted-ip' statement, below). If not, this field
has the same value as the IP in the first field.
* Two dashes.
* Time when the request was replied to. It is formatted as
follows (in the 'strftime' notation):
[%d/%b/%Y:%H:%M:%S %z]
* Request method.
* Request URL with parameters.
* Response status code.
* Response size in bytes.
* Value of the 'Referer' header or '-' if no such header.
* Value of the 'User-Agent' header or '-' if no such header.
-- Configuration: trusted-ip CIDR-LIST
List of trusted proxy IP addresses. This is used to determine
source IP address for logging (a functionality, similar to
'mod_remoteip' module in 'Apache'). Any number of arguments are
allowed. Each argument is parsed as a CIDR. E.g.:
trusted-ip 127.0.0.1 10.0.1.0/16;
Default is '127.0.0.0/8'.
7 Downloads and Other Links
***************************
The program can be downloaded from
.
The source repository is available at
.
The package development page is at
.
Mailfromd home page is: .
8 Bug Reports
*************
If you think you found a bug in 'mfmod_openmetrics' or in its
documentation, please send a mail to (Sergey Poznyakoff)
or use the bug tracker at
(requires
authorization).
Appendix A License
******************
Copyright (C) 2022-2023 Sergey Poznyakoff
Permission is granted to anyone to make or distribute verbatim copies
of this document as received, in any medium, provided that the copyright
notice and this permission notice are preserved, thus giving the
recipient permission to redistribute in turn.
Permission is granted to distribute modified versions of this
document, or of portions of it, under the above conditions, provided
also that they carry prominent notices stating who last changed them.
Index
*****
* Menu:
* /tmp/mfmetrics.db: Configuration. (line 473)
* access-log: Configuration. (line 519)
* configuration file: Usage. (line 149)
* database-name: Configuration. (line 479)
* database-retry: Configuration. (line 486)
* database-wait: Configuration. (line 490)
* listen: Configuration. (line 498)
* mailfromd: Overview. (line 24)
* metric database file: Usage. (line 153)
* mfmod_openmetrics.conf: Usage. (line 149)
* mfmod_openmetrics.conf <1>: Configuration. (line 451)
* namespace: Configuration. (line 465)
* OpenMetrics: Overview. (line 28)
* openmetrics_add: Functions. (line 385)
* openmetrics_declare: Functions. (line 307)
* openmetrics_http_address(): Functions. (line 409)
* openmetrics_incr: Functions. (line 362)
* openmetrics_reset: Functions. (line 392)
* openmetrics_set: Functions. (line 400)
* persistent: Configuration. (line 472)
* Prometheus: Overview. (line 28)
* trusted-ip: Configuration. (line 553)