DevOps December 14, 2025 12 min read

Building a Production-Grade Backup System with rclone

A complete tutorial from zero to a reliable, automated, observable backup setup using rclone, JSON configuration, Bash scripting, Cron scheduling, and Telegram alerts.

Backups are one of those things everyone agrees are important, but very few people implement properly.

This tutorial walks through the full journey, from no backups at all to a safe, automated, production-ready backup system using:

rclone
JSON configuration
Bash scripting
Cron scheduling
Telegram alerts

This is written as a guided tutorial, not a shortcut guide. If you follow it end to end, you will understand why each decision was made, not just how to copy commands.

1. The Problem We Are Solving

Most people fall into one of these traps:

"I'll manually copy files sometimes"
"I use sync and hope nothing goes wrong"
"I have backups but don't know if they're running"
"My backups failed weeks ago and I never noticed"

A good backup system must answer all of these:

What data is backed up?
Where is it backed up?
How often does it run?
What happens on failure?
Can it delete data accidentally?
Can I audit what happened?

Our goal is to build a system that answers all six.

2. Choosing the Right Tool: Why rclone

rclone is often described as "rsync for the cloud", but that undersells it.

What rclone Gives Us

Works with local folders
Works with S3-compatible storage
Explicit commands (copy, sync)
Dry-run mode
Detailed logging
No daemon
Script-friendly

What We Intentionally Avoid

GUI backup tools
Always-running background services
Vendor-locked SaaS backup apps

This system should work anywhere Linux runs.

3. The Most Important Concept: COPY vs SYNC

Before writing any script, we must understand this.

rclone sync

Makes destination an exact mirror
Deletes files on destination if missing locally
Fast and powerful
Dangerous if misused

rclone copy

Copies new and changed files
Never deletes anything on destination
Safe for important data
Ideal for personal documents and archives

Decision Rule

Important data → copy
Disposable or reproducible data → sync

This tutorial uses copy, because safety comes first.

4. Why We Don't Hardcode Paths

Hardcoding paths inside scripts leads to:

Fragile scripts
Difficult edits
No structure
No future extension

Instead, we use a JSON configuration file.

5. Designing the JSON Configuration

The JSON file answers one simple question:

"What should be backed up, and how?"

Example backup.json

{
  "backup_sets": [
    {
      "local": "/path/to/FolderA",
      "remote": "remote:bucket"
    },
    {
      "local": "/path/to/FolderB",
      "remote": "remote:bucket"
    }
  ],
  "settings": {
    "transfers": 8,
    "checkers": 8,
    "retries": 3,
    "retries_sleep": "10s"
  },
  "app_settings": {
    "dry_run": false
  }
}

Why This Structure Works

Each folder is independent
Same remote can be reused
Performance settings are centralized
Behavior can be changed without touching the script

6. Installing Prerequisites

sudo apt install -y rclone jq curl

rclone → backup engine
jq → parse JSON safely
curl → send Telegram alerts

7. Creating the Backup Script (Core Logic)

This script will:

Load JSON
Lock execution (prevent overlaps)
Loop through backup sets
Derive folder names automatically
Run rclone copy
Log output
Send Telegram alerts

8. Understanding Execution Locking (Important)

Cron can trigger scripts even if a previous run is still running.

We prevent this using flock.

LOCK="/tmp/json-rclone-backup.lock"
exec 200>$LOCK
flock -n 200 || exit 0

This guarantees:

Only one instance runs
No race conditions
No corrupted backups

9. The Final Backup Script (Current State)

This is the complete, working, final version. All sensitive values are placeholders.

#!/bin/bash

CONFIG="$HOME/backup-config/backup.json"
LOG="$HOME/json-rclone-backup.log"
LOCK="/tmp/json-rclone-backup.lock"

BOT_TOKEN="BOT_TOKEN_HERE"
CHAT_ID="CHAT_ID_HERE"

START_TIME=$(date "+%Y-%m-%d %H:%M:%S")
START_TS=$(date +%s)

send_msg () {
  curl -s -X POST https://api.telegram.org/bot$BOT_TOKEN/sendMessage \
    -d chat_id="$CHAT_ID" \
    -d text="$1" \
    -d disable_notification=true > /dev/null
}

exec 200>$LOCK || exit 1
flock -n 200 || exit 0

TRANSFERS=$(jq -r '.settings.transfers' "$CONFIG")
CHECKERS=$(jq -r '.settings.checkers' "$CONFIG")
RETRIES=$(jq -r '.settings.retries' "$CONFIG")
RETRY_SLEEP=$(jq -r '.settings.retries_sleep' "$CONFIG")
DRY_RUN=$(jq -r '.app_settings.dry_run' "$CONFIG")

DRY=""
[ "$DRY_RUN" = "true" ] && DRY="--dry-run"

send_msg "Backup started
Host: $(hostname)
Time: $START_TIME"

> "$LOG"

jq -c '.backup_sets[]' "$CONFIG" | while read -r SET; do
  LOCAL=$(echo "$SET" | jq -r '.local')
  REMOTE=$(echo "$SET" | jq -r '.remote')
  DIR_NAME=$(basename "$LOCAL")

  rclone copy "$LOCAL" "$REMOTE/$DIR_NAME" \
    --transfers "$TRANSFERS" \
    --checkers "$CHECKERS" \
    --retries "$RETRIES" \
    --retries-sleep "$RETRY_SLEEP" \
    --log-file="$LOG" \
    --log-level INFO \
    $DRY

  if [ $? -ne 0 ]; then
    send_msg "Backup FAILED
Folder: $DIR_NAME
Check log: $LOG"
    exit 1
  fi
done

END_TIME=$(date "+%Y-%m-%d %H:%M:%S")
END_TS=$(date +%s)
DURATION=$((END_TS - START_TS))

SUMMARY=$(grep -E "Transferred:" "$LOG" | tail -n 3)

send_msg "Backup completed
Host: $(hostname)
Start: $START_TIME
End: $END_TIME
Duration: ${DURATION}s

Summary:
$SUMMARY"

10. What "Dry Run" Actually Means

In the JSON:

"dry_run": true

This causes the script to run:

rclone copy ... --dry-run

Which means:

No files are copied
No files are modified
Only logs what would happen

Always use dry-run:

On first setup
When changing paths
When testing new remotes

11. Scheduling with Cron

After manual testing succeeds:

crontab -e

Run every 15 minutes:

*/15 * * * * /usr/local/bin/json-rclone-backup.sh

Why 15 minutes:

Frequent enough for safety
Lightweight enough for laptops and servers
Locking prevents overlap

12. Telegram Alerts: Why They Matter

Logs are useless if nobody reads them.

Telegram alerts give:

Start notification
Failure notification
Completion notification
Execution duration
Transfer summary

This solves the "silent failure" problem completely.

13. Current Final State (What We Achieved)

At the end of this tutorial, the system has:

Declarative backup configuration
Safe, non-destructive backups
Automatic execution
Locking
Centralized logs
Real-time alerts
S3 compatibility
No vendor lock-in

This is production-grade, not a toy script.

14. Why This Design Works Long-Term

Bash is stable
rclone is actively maintained
JSON is portable
No background services
Easy to migrate
Easy to audit
Easy to restore from

This is the kind of system you set up once and trust.

15. Possible Future Extensions

Encrypted remotes (rclone crypt)
Date-based snapshot folders
Per-folder modes (copy vs sync)
systemd timers
Automated restore tests

Final Note

A good backup system is not clever. It is predictable, boring, and visible.

If you forget it exists, and it still works, you built it right.