rcloneでpostgresのdumpをgoogle driveにアップロード

by

目的

debian linux上で稼働しているpostgresのバックアップをpg_dumpで行い、 rcloneでpostgresのdumpをgoogle driveにアップロードする

rcloneはさまざまなクラウド系のファイルストレージサービスを簡単に扱えるようにしてくれるツールっぽい

環境

  • ターゲットマシン
    • debian 12
    • postgresql 15.14
  • googleアカウント認証マシン
    • Mac Studio m1

rcloneをinstall

Bash
sudo apt install -y rclone
Bash

rclone config設定

Bash
$ rclone config
2025/10/15 20:58:18 NOTICE: Config file "/home/parallels/.config/rclone/rclone.conf" not found - using defaults
No remotes found, make a new one?
n) New remote
s) Set configuration password
q) Quit config
n/s/q> n
Bash

リモート接続の名前を入力する ここでは`gdrive`とした

Bash
Enter name for new remote.
name> gdrive
Bash

どのストレージ種類を使うか聞かれるのでgoogle drive(18)を選択

Bash
Option Storage.
Type of storage to configure.
Choose a number from below, or type in your own value.
 1 / 1Fichier
   \ (fichier)
 2 / Akamai NetStorage
   \ (netstorage)
 3 / Alias for an existing remote
   \ (alias)
 4 / Amazon Drive
   \ (amazon cloud drive)
 5 / Amazon S3 Compliant Storage Providers including AWS, Alibaba, Ceph, China Mobile, Cloudflare, ArvanCloud, Digital Ocean, Dreamhost, Huawei OBS, IBM COS, IDrive e2, IONOS Cloud, Lyve Cloud, Minio, Netease, RackCorp, Scaleway, SeaweedFS, StackPath, Storj, Tencent COS, Qiniu and Wasabi
   \ (s3)
 6 / Backblaze B2
   \ (b2)
 7 / Better checksums for other remotes
   \ (hasher)
 8 / Box
   \ (box)
 9 / Cache a remote
   \ (cache)
10 / Citrix Sharefile
   \ (sharefile)
11 / Combine several remotes into one
   \ (combine)
12 / Compress a remote
   \ (compress)
13 / Dropbox
   \ (dropbox)
14 / Encrypt/Decrypt a remote
   \ (crypt)
15 / Enterprise File Fabric
   \ (filefabric)
16 / FTP
   \ (ftp)
17 / Google Cloud Storage (this is not Google Drive)
   \ (google cloud storage)
18 / Google Drive
   \ (drive)
19 / Google Photos
   \ (google photos)
20 / HTTP
   \ (http)
21 / Hadoop distributed file system
   \ (hdfs)
22 / HiDrive
   \ (hidrive)
23 / In memory object storage system.
   \ (memory)
24 / Internet Archive
   \ (internetarchive)
25 / Jottacloud
   \ (jottacloud)
26 / Koofr, Digi Storage and other Koofr-compatible storage providers
   \ (koofr)
27 / Local Disk
   \ (local)
28 / Mail.ru Cloud
   \ (mailru)
29 / Microsoft Azure Blob Storage
   \ (azureblob)
30 / Microsoft OneDrive
   \ (onedrive)
31 / OpenDrive
   \ (opendrive)
32 / OpenStack Swift (Rackspace Cloud Files, Memset Memstore, OVH)
   \ (swift)
33 / Pcloud
   \ (pcloud)
34 / Put.io
   \ (putio)
35 / SMB / CIFS
   \ (smb)
36 / SSH/SFTP
   \ (sftp)
37 / Sia Decentralized Cloud
   \ (sia)
38 / Sugarsync
   \ (sugarsync)
39 / Transparently chunk/split large files
   \ (chunker)
40 / Union merges the contents of several upstream fs
   \ (union)
41 / Uptobox
   \ (uptobox)
42 / WebDAV
   \ (webdav)
43 / Yandex Disk
   \ (yandex)
44 / Zoho
   \ (zoho)
45 / premiumize.me
   \ (premiumizeme)
46 / seafile
   \ (seafile)
Storage> 18
Bash

[Option] client_id, client_secretは入力なし

Bash
Option client_id.
Google Application Client Id
Setting your own is recommended.
See https://rclone.org/drive/#making-your-own-client-id for how to create your own.
If you leave this blank, it will use an internal key which is low performance.
Enter a value. Press Enter to leave empty.
client_id> 

Option client_secret.
OAuth Client Secret.
Leave blank normally.
Enter a value. Press Enter to leave empty.
client_secret>
Bash

google driveのアクセスscopeを入力. 書き込みに行かないといけないので 1.のFull access

Bash
Option scope.
Scope that rclone should use when requesting access from drive.
Choose a number from below, or type in your own value.
Press Enter to leave empty.
 1 / Full access all files, excluding Application Data Folder.
   \ (drive)
 2 / Read-only access to file metadata and file contents.
   \ (drive.readonly)
   / Access to files created by rclone only.
 3 | These are visible in the drive website.
   | File authorization is revoked when the user deauthorizes the app.
   \ (drive.file)
   / Allows read and write access to the Application Data folder.
 4 | This is not visible in the drive website.
   \ (drive.appfolder)
   / Allows read-only access to file metadata but
 5 | does not allow any access to read or download file content.
   \ (drive.metadata.readonly)
scope> 1
Bash

[Option] service_account_file, advance configは入力なし

Bash
Option service_account_file.
Service Account Credentials JSON file path.
Leave blank normally.
Needed only if you want use SA instead of interactive login.
Leading `~` will be expanded in the file name as will environment variables such as `${RCLONE_CONFIG_DIR}`.
Enter a value. Press Enter to leave empty.
service_account_file> 

Edit advanced config?
y) Yes
n) No (default)
y/n> n
Bash

googleアカウント認証. 設定しているマシンはCUI環境しかないので n

Bash
Use auto config?
 * Say Y if not sure
 * Say N if you are working on a remote or headless machine

y) Yes (default)
n) No
y/n> n

Option config_token.
For this to work, you will need rclone available on a machine that has
a web browser available.
For more help and alternate methods see: https://rclone.org/remote_setup/
Execute the following on the machine with the web browser (same rclone
version recommended):
	rclone authorize "drive" "xxxxxxxxxxxxxxxxxx"
Then paste the result.
Enter a value.
config_token>
Bash

別のマシン(GUIが使える)でgoogleアカウント認証

上の15行目をguiが使えるマシンで実行する.

  1. 別マシン(mac)にrcloneが入っていないのでbrewでインストール
  2. インストールが終わって15行目を実行すると、ブラウザが立ち上がってgoogleアカウント認証を行う
  3. 認証が終わるとjwtコードが11行目に発行されるのでコピー
Bash
$ brew install rclone
$ rclone authorize "drive" "xxxxxxxxxxxxxx"

2025/10/15 21:11:56 NOTICE: Config file "/Users/george/.config/rclone/rclone.conf" not found - using defaults
2025/10/15 21:11:56 NOTICE: Make sure your Redirect URL is set to "http://127.0.0.1:53682/" in your custom config.
2025/10/15 21:11:56 NOTICE: If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth?state=xxxxxxxxxxxxxxxxx
2025/10/15 21:11:56 NOTICE: Log in and authorize rclone for access
2025/10/15 21:11:56 NOTICE: Waiting for code...
2025/10/15 21:12:07 NOTICE: Got code
Paste the following into your remote machine --->
jwt的なコードがここに表示されるのでコピーする
<---End paste
Bash

4. リモートマシンに戻ってjwtを貼り付ける

Bash
Option config_token.
For this to work, you will need rclone available on a machine that has
a web browser available.
For more help and alternate methods see: https://rclone.org/remote_setup/
Execute the following on the machine with the web browser (same rclone
version recommended):
	rclone authorize "drive" "xxxxxxxxxxxxxxxxxx"
Then paste the result.
Enter a value.
config_token>jwt的なコードを貼り付け
Bash

[Option] Team Driveはn

Bash
Configure this as a Shared Drive (Team Drive)?

y) Yes
n) No (default)
y/n> n
Bash

設定保存

Bash
Configuration complete.
Options:
- type: drive
- scope: drive
- token: xxxxxxxxxxxxxxxx
- team_drive: 
Keep this "gdrive" remote?
y) Yes this is OK (default)
e) Edit this remote
d) Delete this remote
y/e/d> y

Current remotes:

Name                 Type
====                 ====
gdrive               drive

e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> q
Bash

動作確認

以下コマンドでgoogle driveの中身を見れる

Bash
$ rclone lsd gdrive:
Bash

google driveの設定は終わり

postgresのバックアップ

いよいよ rcloneでpostgresのdumpをgoogle driveにアップロード するための最後の準備を行う

バックアップファイル一時置き場作成

バックアップ処理はpostgresユーザーが実行するため フォルダに権限付与

Bash
$ sudo mkdir -p /var/backups/postgres
$ sudo chown postgres:postgres /var/backups/postgres
Bash

ログ出力フォルダ作成

Bash
sudo mkdir -p /var/lib/postgresql/logs
sudo chown postgres:postgres /var/lib/postgresql/logs
Bash

バックアップスクリプト作成

Bash
sudo nano /usr/local/bin/pg_backup_simple.sh
Bash
pg_backup_simple.sh
#!/bin/bash
set -euo pipefail

cd /var/lib/postgresql

DATE=$(date +%Y-%m-%d)
BACKUP_FILE="/var/backups/postgres/pg_${DATE}.dump"
REMOTE="gdrive:postgres-backup"
LOG_FILE="/var/lib/postgresql/logs/pg_backup.log"

mkdir -p "$(dirname "$LOG_FILE")"

{
    echo "==== $(date '+%F %T') Backup start ===="
    echo "[INFO] Starting backup for database: capstore"
    echo "[INFO] Output file: $BACKUP_FILE"

    # pg_dump
    echo "[INFO] Dumping database..."
    # PostgreSQLのバックアップ（例: [database]）
    if pg_dump -U postgres [database] > "$BACKUP_FILE" 2>>"$LOG_FILE"; then
        echo "[INFO] pg_dump completed successfully."
    else
        echo "[ERROR] pg_dump failed!"
        exit 1
    fi

    # Upload
    # Google Driveにアップロード
    echo "[INFO] Uploading to Google Drive..."
    if rclone copy "$BACKUP_FILE" "$REMOTE" --progress; then
        echo "[INFO] Upload successful."
    else
        echo "[ERROR] Upload to Google Drive failed!"
        exit 1
    fi

    # Cleanup
    # ローカルの古いバックアップを7日より前のものは削除
    echo "[INFO] Cleaning old backups..."
    if find /var/backups/postgres -name "pg_*.dump" -mtime +7 -delete; then
        echo "[INFO] Old backup cleanup done."
    else
        echo "[WARN] Cleanup step encountered an issue."
    fi

    echo "[SUCCESS] Backup completed successfully ✅"
    echo "==== $(date '+%F %T') Backup end ===="
} 2>&1 | tee -a "$LOG_FILE"
ShellScript

バックアップスクリプトに実行権限を付与

Bash
sudo chmod +x /usr/local/bin/pg_backup_simple.sh
Bash

postgresユーザーでrcloneするので、設定ファイルをコピー

Bash
sudo mkdir -p /var/lib/postgresql/.config/rclone
sudo cp -r ~/.config/rclone/rclone.conf /var/lib/postgresql/.config/rclone/
sudo chown -R postgres:postgres /var/lib/postgresql/.config
Bash

cronで毎日AM2時に自動実行

Bash
$ sudo crontab -e
0 2 * * * /usr/local/bin/pg_backup_simple.sh
Bash

動作確認

即時実行する場合は以下でテスト

Bash
$ sudo -u postgres /usr/local/bin/pg_backup_simple.sh
[INFO] Starting backup for database: capstore
[INFO] Output file: /var/backups/postgres/pg_2025-10-15.dump
[INFO] Dumping database...
[INFO] pg_dump completed successfully.
[INFO] Uploading to Google Drive...
Transferred:   	   27.160 KiB / 27.160 KiB, 100%, 0 B/s, ETA -
...
Transferred:            1 / 1, 100%
Elapsed time:         2.7s
[INFO] Upload successful.
[INFO] Cleaning old backups...
[INFO] Old backup cleanup done.
[SUCCESS] Backup completed successfully ✅
==== 2025-10-15 22:30:40 Backup end ====
Bash

以上

他のpostgresの記事はこちら

コメントする