s-postgray.8

.\"@ s-postgray - postfix policy (graylisting) server.
.\"
.\" Copyright (c) 2022 - 2024 Steffen Nurpmeso <steffen@sdaoden.eu>.
.\" SPDX-License-Identifier: ISC
.\"
.\" Permission to use, copy, modify, and/or distribute this software for any
.\" purpose with or without fee is hereby granted, provided that the above
.\" copyright notice and this permission notice appear in all copies.
.\"
.\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
.\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
.\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
.\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
.\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
.\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
.\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
.
.Dd June 23, 2024
.ds VV \\%v0.8.3
.ds XX \\%S-POSTGRAY
.ds Xx \\%S-postgray
.ds xx \\%s-postgray
.
.Dt \*(XX 8
.Os
.Mx -enable
.
.
.Sh NAME
.Nm \*(Xx \%[\*(VV]
.Nd postfix RFC 6647 graylisting policy server
.
.
.Sh SYNOPSIS
.
.Nm \*(xx
.Op options
.Nm \*(xx
.Op options
.Fl Fl shutdown
.Nm \*(xx
.Op options
.Fl Fl startup
.Nm \*(xx
.Op options
.Fl Fl status
.Nm \*(xx
.Op options
.Fl Fl test-mode
.Op options
.
.
.Mx -toc -tree html pdf ps xhtml
.
.
.Sh DESCRIPTION
.
A RFC 6647 graylisting
.Xr postfix 1
policy service.
Graylisting defers message acceptance a configurable number of times
via a standardized SMTP response (see
.Xr allow 5 ,
RFC 5321), which does not prevent message delivery from SMTP
M(essage) T(ransfer) A(gent)s, but can help against simple spam
producing programs.
.
.Pp
Client allow (white) and block (black) lists of domain names (exact and
wildcard) as well as IPv4 and IPv6 addresses (exact or in CIDR notation)
are supported.
All the data is stored in main memory, synchronization with backing
store is only performed when the server process is started or stopped,
and the graylist database which needs to handle network data uses
a cryptographically secure hash function.
Operating-system dependent best-effort sandboxing is used, optionally
.Fl Fl untamed .
.
.Pp
The program is intended to be started via the
.Xr postfix 1
.Xr spawn 8
daemon, which takes care of the correct user and group credentials;
It's single-instance server is then managed automatically, with
a configurable lifetime: by (compile-time) default it times out without
active client connections.
An explicit server
.Fl Fl startup
may be enforced:
.Fl Fl server-timeout
is ignored in this mode.
The server
.Fl Fl status
can be queried, and its synchronized
.Fl Fl shutdown
can be enforced.
The server PID is managed in the synchronized file
.Ql NAME.pid
within
.Fl Fl store-path .
.
.Pp
Sending the
.Ql USR2
signal will save the graylist database,
whereas
.Ql USR1
logs some statistics.
Dependent upon the operating-system sandbox, and
.Fl Fl untamed ,
sending a
.Ql HUP
signal will re-evaluate the configuration, and update the graylist
database accordingly;
if supported, it may not be possible to change or add file paths
(without violating security constraints), and modifying
.Fl Fl limit Ns
s can never be hard-reflected by the (already entered) sandbox;
Sending
.Ql TERM
initiates a server shutdown, and causes active clients to terminate.
(The client ignores these signals:
.Xr pkill 1
may be used.)
.
.Pp
As recommendet by RFC 6647 messages are identified by their recipient /
sender / client_address value triple; in
.Fl Fl focus-sender
mode recipients are ignored (see there).
Here is an excessively lengthy but minimal
.Xr postconf 5
example.
(Especially address and DNS hostname checks and verifications are
performed before the policy server is involved in the decision process.)
Note the
.Ql DEFER_IF_PERMIT
.Fl Fl msg-defer
used to signal per-recipient graylisting is counted against
.Ql smtpd_hard_error_limit
and
.Ql smtpd_soft_error_limit
parameters
.Pf ( Fl Fl focus-sender
mode will generate only one error per message).
.
.Bd -literal -offset indent
#@ /etc/postfix/master.cf:

postgray unix - n n - - spawn
  user=ANON-USR
  argv=/usr/libexec/s-postgray -vvR /etc/postgray.rc -c 0

#@ /etc/postfix/main.cf:

default_privs = ANON-USR

# Client connection checks
smtpd_client_restrictions =
#   permit_inet_interfaces, OR
   permit_mynetworks,
#RELAY   permit_tls_clientcerts,
#[RELAY]   permit_sasl_authenticated,
   reject_unknown_client_hostname,
   reject_unauth_pipelining,
   permit

smtpd_data_restrictions =
   reject_unauth_pipelining,
   permit

smtpd_helo_restrictions =
#   permit_inet_interfaces, OR
   permit_mynetworks,
#RELAY   permit_tls_clientcerts,
#[RELAY]   permit_sasl_authenticated,
   reject_invalid_helo_hostname,
   reject_non_fqdn_helo_hostname,
   reject_unknown_helo_hostname,
   permit

# MAIL FROM Checks
smtpd_sender_restrictions =
#RELAY   reject_authenticated_sender_login_mismatch,
#   permit_inet_interfaces, OR
   permit_mynetworks,
#RELAY   permit_tls_clientcerts,
#[RELAY]   permit_sasl_authenticated,
   reject_non_fqdn_sender,
   # Total no-goes database, eg: qq.com reject
#   check_sender_access lmdb:/etc/postfix/sender_restrict,
#   check_sender_access inline:{$mydomain=reject},
   reject_unknown_sender_domain,
   reject_unknown_reverse_client_hostname,
   # With --focus-sender only!  And --msg-allow=permit
#   check_policy_service unix:private/postgray,
#VERIFY(..then)   reject_unverified_sender,
   permit

smtpd_relay_before_recipient_restrictions = yes

# RCPT TO checks, relay policy
# Local+auth clients may specify any destination domain
smtpd_relay_restrictions =
#   permit_inet_interfaces, OR
   permit_mynetworks,
#RELAY   permit_tls_clientcerts,
#[RELAY]  permit_sasl_authenticated,
   reject_non_fqdn_recipient,
   reject_unauth_destination,
   permit

# RCPT TO checks, spam blocking policy
smtpd_recipient_restrictions =
#   permit_inet_interfaces, OR
   permit_mynetworks,
#RELAY   permit_tls_clientcerts,
#[RELAY]  permit_sasl_authenticated,
   reject_unknown_recipient_domain,
   # Without --focus-sender only!
   check_policy_service unix:private/postgray,
#VERIFY(..then)   reject_unverified_sender,
#(VERIFY i would not)   reject_unverified_recipient,
   permit

smtpd_policy_service_default_action = DUNNO

#@ /etc/s-postgray.rc

store-path /var/lib/pg

delay-min 0
delay-progressive
gc-linger
server-timeout 0

# notorious
allow .gmail.com
allow .google.com
allow .outlook.com
allow .yahoo.com
allow .yandex.com
  allow .yandex.net

allow .paypal.com
  allow .paypal.de
.Ed
.
.
.Sh OPTIONS
.
Options may be given in short or long form,
.Fl Fl resource-file Ns
s only support the long form, and only a (logical) option subset.
The minute limit is 32767 (15-bit), the maximum duration is thus 22 days.
Other numbers have a limit of 31-bit (2147483647).
.Fl Fl test-mode
performs a dry-run configuration syntax test, and outputs a normalized
resource file.
In the following DB means database, and GC garbage collection.
.
.Bl -tag -width ".It Fl BaNg"
.Mx Fl 4-mask
.It Fl Fl 4-mask Ar mask , Fl 4 Ar mask
IPv4 mask to strip off addresses before match.
For example 24 masks all addresses in between 127.0.0.0 and 127.0.0.255.
This is desirable since in practice MX farms are used, and/or IP
addresses are selected from a pool.
.
.Mx Fl 6-mask
.It Fl Fl 6-mask Ar mask , Fl 6 Ar mask
IPv6 mask to strip off addresses before match.
Using a mask of 64 seems to be good practice (see
.Fl Fl 4-mask ) .
.
.Mx Fl allow-file
.It Fl Fl allow-file Ar path , Fl A Ar path
Load a file of whitelist entries in the syntax described for
.Fl Fl allow
from within the server or
.Fl Fl test-mode .
Lines are read as via
.Fl Fl resource-file .
.
.Mx Fl allow
.It Fl Fl allow Ar spec , Fl a Ar spec
Add a domain name or an IPv4 or IPv6 internet address, optionally in
RFC 1519 CIDR notation with network mask
.Pf ( Fl Fl test-mode
hints incorrect networks), to the list of allowed
clients (whitelist) that are accepted with
.Fl Fl msg-allow .
Domain names are matched exactly unless the first character is a period
.Ql \&. ,
in which case the given domain and all its subdomains will match.
For IP addresses the global masks
.Fl Fl 4-mask
and
.Fl Fl 6-mask
normalize the given address (range) if applicable.
All constructs are matched via dictionary, except for CIDR ranges with
masks smaller than the global ones, they are matched in the given order.
.Bd -literal -offset indent
exact.match
also.exact.match

# This matches d.a.s but also a.b.c.d.a.s
\&.d.a.s

# with --4-mask=24 this really is 127.0.0.0/24!
127.0.0.1

# with --6-mask=64 really 2a03:2880:20:6f06::/64
# --test-mode hints 2a03:2880:20:6f06:c000::/66
2a03:2880:20:6f06:face:b00c:0:14/66

# with --6-mask=64 nonetheless 2a03:2880:20:4f00::/56
# This will _not_ be matched by dictionary but in order
2a03:2880:20:4f06:face:b00c:0:14/56
.Ed
.Pp
If whitelisting is really performed that late in the processing chain
it should include all big players and all normally expected endpoints;
it may be useful to run for a few days with the special 0
.Fl Fl count
and inspect the log in order to create a whitelist.
Some MTAs are picky, so driving for a while with a low count and in
.Fl Fl verbose
mode to collect more data before increasing count etc. is worthwhile.
.Pp
It should be noted that only the two VERP (variable envelope return path
addresses) delimiters plus sign
.Ql +
and equal sign
.Ql =
are understood \(em mailing list software which chooses the hyphen-minus
.Ql -
as a VERP delimiter (ezmlm instances are known which do) make
a particularly bad choice because many mailing-lists have a hyphen-minus
as a regular part of their name, so no automatic differentiation in
between the customized address part and the regular address is possible:
such addresses can only be placed in the whitelist, otherwise each and
every received message will be graylisted.
.
.Mx Fl block-file
.It Fl Fl block-file Ar path , Fl B Ar path
Load a file of blacklist entries in the syntax described for
.Fl Fl allow-file
from within the server or
.Fl Fl test-mode .
.
.Mx Fl block
.It Fl Fl block Ar spec , Fl b Ar spec
Add a blacklist entry, syntax identical to
.Fl Fl allow .
Entries are rejected with
.Fl Fl msg-block .
(Blocking should possibly be done earlier in the processing chain.)
.
.Mx Fl count
.It Fl Fl count Ar no , Fl c Ar no
Number of SMTP message delivery retries before it is accepted.
The special value 0 will accept messages immediately, and change the
behaviour of some other settings, like
.Fl Fl limit-delay ;
it may be useful when setting up the configuration and the whitelist.
(Once regular usage begins that DB should possibly be removed.)
.
.Mx Fl delay-max
.It Fl Fl delay-max Ar mins , Fl D Ar mins
Duration until a message
.Dq is no longer a retry ,
but interpreted as a new one with a reset
.Fl Fl count .
.
.Mx Fl delay-min
.It Fl Fl delay-min Ar mins , Fl d Ar mins
Duration until a message
.Dq is a retry .
Those which come sooner do not increment
.Fl Fl count .
.
.Mx Fl delay-progressive
.It Fl Fl delay-progressive , p
If set
.Fl Fl delay-min
is multiplied with each counted retry until
.Fl Fl count
is reached.
This mode asserts that the maximum delay
.Ql delay-min * count
is smaller than
.Fl Fl delay-max .
.
.Mx Fl focus-sender
.It Fl Fl focus-sender , f
By default all of recipient (email address), sender (email address) and
client address (IPv4 or IPv6 internet address) are used to identify
messages for graylisting purposes.
With this focus is on the sender, and the recipient is ignored.
.Xr postconf 5
can then be changed to perform graylisting in
.Ql smtpd_sender_restrictions
instead of
.Ql smtpd_recipient_restrictions ,
for example to guard a following sender address verification;
to accomplish this for real
.Ql Fl Fl msg-allow Ns = Ns permit
and
.Ql Fl Fl msg-defer Ns = Ns DEFER 4.2.0 Service RFC 6647 greylisted you
should be set, so that the verification is only reached for graylisted
senders that passed the test, and
.Ql Fl Fl count Ns = Ns 1
might be sufficient.
This setting cannot be changed at runtime, and it should be ensured all
instances use the same one.
An existing DB can be reused: the next load removes recipients, so this
is one way (DB remains
.Dq compatible ) .
.
.Mx Fl gc-linger
.It Fl Fl gc-linger
Use a different GC behaviour: instead of removing
.Fl Fl gc-timeout
entries upon the next garbage collection, keep them until the DB excesses
.Fl Fl limit
(or memory constraints require DB shrinkage: condition logged).
.
.Mx Fl gc-rebalance
.It Fl Fl gc-rebalance Ar no , Fl G Ar no
Number of DB GC runs before rebalancing occurs.
Value 0 turns rebalancing off.
Rebalancing only affects shrinking of the dictionary table,
it is grown automatically as necessary, so a carefully chosen
.Fl Fl limit
may render rebalancing undesired.
.
.Mx Fl gc-timeout
.It Fl Fl gc-timeout Ar mins , Fl g Ar mins
Duration until a DB entry is seen as unused and maybe removed.
Each time an entry is used the timeout is reset.
This timeout is also an indication for how often a GC shall be
performed, but GC happens due to circumstances, too.
And see
.Fl Fl gc-linger .
.
.Mx Fl help
.It Fl Fl help , h
A short help listing (not helpful, instead see
.Fl H
or
.Fl Fl long-help ) .
.
.Mx Fl limit
.It Fl Fl limit Ar no , Fl L Ar no
Number of DB entries until new ones are not handled,
effectively turning them into accepted graylist members.
(DB maintenance tries to achieve a maximum of 88 percent fill-level.)
Data size depends on actual email (recipient /) sender / client_address
value data, but is stored compactly; accounting say 256 bytes per
entry seems to be (overly) plenty.
There is also a large continuous lookup table memory chunk,
accounting 1 MB per 10000000 entries may be proper.
When saving file size is soft-limited to 2 GiB (two gigabyte),
excess is discarded; if possible a hard limit up to that size via
.Xr setrlimit 2
sandbox is established at
.Fl Fl startup :
runtime hard limit adjustments are not possible.
.
.Mx Fl limit-delay
.It Fl Fl limit-delay Ar no , Fl l Ar no
Smaller than
.Fl Fl limit ,
this number describes a limit after which creation of a new (yet
unknown) entry is delayed by a one second sleep for throttling purposes.
The value 0 disables this feature.
By choosing the right settings for
.Fl Fl limit ,
.Fl Fl limit-delay
and
.Fl Fl gc-timeout
it should be impossible to reach the graylist bypass limit.
Not honoured for a 0
.Fl Fl count .
.
.Mx Fl msg-allow
.It Fl Fl msg-allow Ar msg , Fl ~ Ar msg
A message in
.Xr access 5
format that is passed to
.Xr postfix 1
for
.Fl Fl allow Ns
ed (recipient /) sender / client_address value combinations.
This setting cannot be changed at runtime; there is a length limit.
Defaults to
.Ql DUNNO ,
but
.Ql OK
or even
.Ql permit
seem reasonable.
.
.Mx Fl msg-block
.It Fl Fl msg-block Ar msg , Fl ! Ar msg
Like
.Fl Fl msg-allow ,
but for
.Fl Fl block Ns
ed value combinations.
Defaults to
.Ql REJECT ,
but
.Ql 5.7.1 Please go away
seems reasonable.
This setting cannot be changed at runtime; there is a length limit.
.
.Mx Fl msg-defer
.It Fl Fl msg-defer Ar msg , Fl m Ar msg
Like
.Fl Fl msg-allow ,
but used for graylisted value combinations
.Pf ( Ql DUNNO
is used for accepted ones).
The default is
.Ql DEFER_IF_PERMIT 4.2.0 Service temporarily faded to Gray ,
of which only
.Ql DEFER_IF_PERMIT
is not optional; it uses an RFC 3463 extended status code:
.Bd -literal -offset indent
# [4.2.0]
4.X.X Persistent Transient Failure
x.2.X Mailbox Status
X.2.0 Other or undefined mailbox status
# [4.1.7 (postfix during address verification in progress]
x.1.X Addressing Status
x.1.0 Other address status
x.1.7 Bad sender's mailbox address syntax
# [4.7.1 (seen in wild; less friendly and portable!)]
x.7.X Security or Policy Status
x.7.0 Other or undefined security status
x.7.1 Delivery not authorized, message refused
      This is useful only as a permanent error.
.Ed
.Pp
If
.Xr postfix 1
address verification is used in addition, it may be better to use
graylisting (maybe second-last and) before it, and return
.Ql DEFER 4.2.0
instead, so that the more expensive address verification is performed
only when graylisting permits continuation.
This setting cannot be changed at runtime; there is a length limit.
.
.Mx Fl long-help
.It Fl Fl long-help , H
A long help listing.
.
.Mx Fl once
.It Fl Fl once , o
If given the client part will only process one message.
The server process functions as usual.
.
.Mx Fl resource-file
.It Fl Fl resource-file Ar path , Fl R Ar path
A configuration file with long options (without double hyphen-minus
.Ql Fl Fl Ns ) .
Each line forms an entry, leading and trailing whitespace is removed.
If the first non-whitespace character is the number-sign
.Ql #
the line is a comment and discarded.
Empty lines are ignored, other lines can be folded over multiple input
lines with a reverse-solidus
.Ql \e
before the newline: all leading whitespace of the next line is ignored.
The server parses the configuration a second time, and from within
.Fl Fl store-path !
.Bd -literal -offset indent
# Comment \e
  line
server-\e
  queue
.Ed
.
.Mx Fl server-queue
.It Fl Fl server-queue Ar no , Fl q Ar no
The number of concurrent clients a server can handle before
.Xr accept 2 Ns
ing new ones is suspended.
This setting cannot be changed at runtime.
.
.Mx Fl server-timeout
.It Fl Fl server-timeout Ar mins , Fl t Ar mins
Duration until a \*(Xx server which does not serve any clients terminates.
The value 0 disables auto-termination; a
.Fl Fl startup
server only terminates upon request.
The statistics dumped on the signal
.Ql USR1
are not saved in the DB, they only reflect the current server lifetime.
.
.Mx Fl shutdown
.It Fl Fl shutdown , \&.
Force a running server process to exit.
The client synchronizes on the server exit before its terminating.
It exits EX_TEMPFAIL (75) when no server is running.
.
.Mx Fl startup
.It Fl Fl startup , @
Startup a permanent server, to be used in startup scripts for example.
Care should be taken to use the same user and group as
.Xr spawn 8
will use for the client.
It exits EX_TEMPFAIL (75) when a server is already running.
.
.Mx Fl status
.It Fl Fl status , %
Test whether server is running, exit according status.
.
.Mx Fl store-path
.It Fl Fl store-path Ar path , Fl s Ar path
An accessible
.Pa path
to which \*(Xx will change, and where the DB, server PID lock file,
and server/client communication socket will be created.
The directory should only be accessible by the user (and group) driving
\*(xx, no effort is taken to modify
.Xr umask 2
or path modes
.Pf ( Xr chmod 2 ) !
This setting cannot be changed at runtime.
.
.Mx Fl test-mode
.It Fl Fl test-mode , #
Enable test mode: all options are evaluated, including
.Fl Fl allow-file ,
.Fl Fl allow ,
.Fl Fl block-file
and
.Fl Fl block
which are normally processed by only the server.
Once the command line is worked the content of all white- and
blacklists, as well as the final settings of above variables are shown
in resource file format.
The exit status indicates error.
It is highly recommended to use this for configuration checks.
.
.Mx Fl untamed
.It Fl Fl untamed , u
The program always executes in a
.Xr setrlimit 2
sandbox; dependent upon operating-system and compile-time
.Pf ( Ql VAL_OS_SANDBOX )
an even more restricted compartment is entered.
In order to be as strict as possible the latter, however, may make false
assumptions on the internals of the used C library, causing security
violations at runtime (causing the
.Ql smtpd_policy_service_default_action
to take \*(xx's place).
This setting skips the latter, and cannot be changed at runtime.
.
.Mx Fl verbose
.It Fl Fl verbose , v
Increase log verbosity (two levels).
May be of interest to improve the configuration, for example
.Fl Fl allow
and
.Fl Fl block
data is logged, as is the time necessary to save and load the DB.
.El
.
.
.Sh "SEE ALSO"
.
.Xr postfix 1 ,
.Xr access 5 ,
.Xr spawn 8 ,
.Xr verify 8
.
.
.Sh AUTHORS
.
.An "Steffen Nurpmeso" Aq steffen@sdaoden.eu .
.
.\" s-ts-mode