For this customer, CPU cycles are cheap and network bandwidth is expensive, so trade more of the former for less of the latter. Other objectives are to get the backups off site ASAP and to cope with network interruptions by transferring many smaller files rather than one huge file.

• splitcompress: Splits and compresses "large" (~130GB, so fairly small by SAP standards) full and incremental SAP ERP DB backup files into chunks and then asynchronously compresses the chunks using all available processors. Checksums are calculated at every stage - the original file, and both the compressed and uncompressed chunks.
• offsite_dirs: Copies the compressed files and checksums generated by splitcompress (see above) when they are complete.
• offsite_files: Copies all the other files that are required to restore a SAP ERP system (including DB transaction logs) off site shortly after they are complete.
• verifycksum: Verifies all the files have arrived at the DR site intact. All the compressed chunks are uncompressed and concatenated, and that result is checksummed and compared with the original checksum from the primary site.

Oracle's Data Guard was not used because: (1) DG was too new when we started, (2) we have control! - e.g., choice of compression utility, the priority to run it, etc., (3) DG only replicates Oracle, there are other files that must be replicated too, (4) No Oracle changes are required on either the primary or standby DB and (5) these scripts are extremely robust and reliable.

• Oracle DB log shipping: This runs on the production (primary) servers. Shortly after every offline redo log file is created, a checksum is calculated, the log file is compressed, and then both the compressed file and the checksum are sent off site (as described above). Unix nice and clzip are used to control resource consumption on the production (primary) systems. (Originally, gzip was used, but when the network link became saturated, it was trivial to replace it with clzip - which squishes much better!) This script is also used as part of the backup strategy for development systems.

  Development systems' redo logs are sent off site in exactly the same way as the production logs, so development system backups to tape need only be run weekly - with no extra risk of data loss.

• Oracle DB log recover: This runs on the DR (off-site) servers. As soon as a redo log file has reached a (business-specified) age, it is uncompressed, the checksum is compared to ensure the file has not been damaged, and then it's applied to the standby DB.
• Synchronization of other files: This is just a list of target directories to rsync from the production (primary) servers to the DR (off-site) servers. (All the real work is done by rsync.)