pg_receivewal
pg_receivewal is a core utility for archiving WAL file changes as they are written (including .partial WAL files).
pg_receivewal was introduced (as pg_receivexlog) in PostgreSQL 9.2.
Usage
Though it can be run as a stand-alone utility, pg_receivewal is typically used as part of a more comprehensive backup and archiving solution, such as Barman, which use pg_receivewal (typically in combination with replication slots) to maximise the amount of WAL received from PostgreSQL, and hence minimize potential data loss.
Caveats
WAL files will be archived from the point-in-time when pg_receivewal was started, even if earlier WAL files exist on the source server.
If WAL files exist in the specified archive directory, pg_receivewal will assume these are from an earlier invocation and wait until new WAL becomes available before resuming archiving. However, as it is not possible to determine whether the WAL originated from the same server, this may lead to an effectively corrupt set of archived WAL files, if the directory contains WAL from another source.
Change history
- PostgreSQL 11
- PostgreSQL 10
- PostgreSQL 9.5
- PostgreSQL 9.4
- option
-S/--slotadded (commit 858ec118)
- option
- PostgreSQL 9.3
- PostgreSQL 9.2
- added (commit d9bae531)
Source code
pg_receivewal is built as part of pg_basebackup, as it is effectively a front-end interface to the functionality added to pg_basebackup to support streaming WAL files during a backup.
The main source file for the pg_receivewal binary is src/bin/pg_basebackup/pg_receivewal.c; much of the actual functionality is contained in src/bin/pg_basebackup/receivelog.c.
Examples
Very basic invocation of pg_receivewal:
pg_receivewal -d 'host=node1 dbname=postgres user=repuser' -D /data/wal
Note that the process will not background or daemonize itself.
This will result in archived WAL files being stored in the specified directory, e.g.:
$ ls -1 /data/wal 000000010000000000000003 000000010000000000000004 000000010000000000000005 000000010000000000000006 000000010000000000000007 000000010000000000000008 000000010000000000000009.partial
If the above process is terminated (e.g. with Ctrl-C) it may output a message like this:
pg_receivewal: not renaming "000000010000000000000009.partial", segment is not complete
References
- PostgreSQL documentation: pg_receivewal
Useful links
- Never lose a PostgreSQL transaction with pg_receivewal - October 2019 blog article by CyberTec