Backing Up to Google Drive Using Rclone
After I recently setup rsnapshot as my backup solution, which can be found here https://blog.wretchednet.com/post/rsnapshot/ I started looking at options on how to remotely backup my workstation. I used Backblaze for a while but I wasn’t too happy with their interface. I looked at getting another cloud backup solutions but each one would either cost quite a bit more per month and or had similar or worse user interfaces.
I decided to go with Google Drive since I’ve had it for years, its free for up to 15 GB, and the interface is simple to index and use. Most of my initial backups would only be a few gigs, but whenever it came time to get the higher tier from Google Drive, I can get 100 GB for not too crazy of a price.
Let’s Get Started
First, depending on your distro of choice install rclone from your package manager. I use Arch (BTW) on my workstation but for my servers I almost exclusively run Debian and sometimes Ubuntu.
For Debian and its clones run:
$ sudo apt install curl rclone
or for Arch run:
$ sudo pacman -S curl rclone
Next we need to configure rclone by going through quite a few of options. I have noticed that other how-tos on rclone don’t provide the same options I was asked to fill out. And I’m not talking about a few more options. I was asked at least five times what the other how-tos were asked. I’m guessing rclone has had a few updates and changes for more fine-grained control since those other people wrote about rclone. Don’t worry most of them will be the default option anyways.
# Read this first before typing blindly. Type n, which will setup a new connection. Then, call it google-drive or whatever name you want to give it. Then, we type in the number 12 for Google Drive "drive" not be confused with 11 which is Google Cloud Storage. You will be prompted for client_id> and client_secret> but just press enter for both of them. I have added the default and the modified options in this code block below. Hit y for auto config if you are running rclone locally as this will open up a google pop-up in your browser where you can get an authkey back from google to be used for later.
$ rclone config
n
google-drive
12
client_id> [Press_Enter]
client_secret> [Press_Enter]
scope> 1
root_folder_id> [Press_Enter]
service_account_folder> [Press_Enter]
y/n> y
service_account_credentials> [Press_Enter]
team_drive> [Press_Enter]
auth_owner_only> [Press_Enter] ("false")
use_trash> [Press_Enter] ("true")
skip_gdocs> [Press_Enter] ("false")
shared_with_me> [Press_Enter] ("false")
trashed_only> [Press_Enter] ("false")
formats> [Press_Enter]
export_formats>
import_formats>
allow_import_name_change> [Press_Enter] ("false")
use_created_date> [Press_Enter] ("false")
list_chunk> [Press_Enter] ("1000")
imersonate> [Press_Enter]
alternate_export> [Press_Enter]
upload_cutoff> [Press_Enter] ("8M")
chunk_size> [Press_Enter] ("8M")
acknowledge_abuse> [Press_Enter] ("false")
keep_revision_forever> [Press_Enter] ("false")
v2_download_min_size> [Press_Enter] ("off")
Use auto config?
* Say Y if not sure
* Say N if you are working on a remote or headless machine or Y didn't work
y) Yes
n) no
y/n>y
Enter verification code>
If you are running rclone in a remote session via SSH then you will want to type in n for auto config as this will send you a URL link where-which you can copy and run this on a workstation that has a browser. This will run you through the same login procedure as if you typed in y and it will show an auth key. Copy the key and paste it into the Enter Verification code>. Then proceed to the next options where you will hit y when it asks you if everything is OK then q to quit the rclone google-drive installation.
Configure this as a team drive?
y) Yes
n) No
y/n> n
...
...
y) Yes this is OK
e) Edit this remote
d) Delete this remote
y/e/d> y
...
...
Name Type
==== ====
google-drive drive
e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set coniguration password
q) Quit config
e/n/d/r/c/s/q> q
# The config is officially setup now.
Setting up the rclone script
Next, we need to set up the script of the files we want to backup. What I have setup on my workstation is I have rsnapshot perform backups locally then I have rclone copy the latest hourly.0 to Google Drive. You can reference my config on rsnapshot here: https://blog.wretchednet.com/post/rsnapshots.
$ vim /home/wretchedghost/bin/rclone-home-gdrive.sh
#!/bin/bash
rclone copy --update --verbose --transfers 10 - -checkers 8 --contimeout 60s --timeout 300s --r etries 3 --low-level-retries 10 --stats 1s /rsnapbackup/hourly.0/sys76/home/wretchedghost google-drive:sys76/home
This script will copy the contents of /rsnapshot/hourly.0/sys76/home/wretchedghost to the google-drive:sys76/home directory. You can edit the google-drive:(whatever) to what you want the directory to be called on Google Drive. This script will also run 10 concurrent uploads and fail after 3 failed tries.
Set the script to executable by running the following command:
$ chmod +x /home/wretchedghost/bin/rclone-home-gdrive.sh
You may want to run the script manually at first to make sure everything uploads and backs up correctly.
$ cd ~/bin
$ ./rclone-home-gdrive.sh
The script will run and you will see a ton of information about files being checked, transferred, and a current status of the run will periodically be shown.
Setting rclone On A Schedule
The last thing we need to do is get the script we created to run on a schedule. The easiest option is to use crontab. Some crontabs will have lots of information that can help you setup scripts on a schedule while others will not. You can delete all the commented content that begins with # if you want to get rid of some of the clutter.
$ crontab -e
15 23 * * * bash /home/wretchedghost/bin/rclone-home-gdrive.sh
What this will do is run the rclone script we created to run every day at 11:15 pm. You can tweak it further if you want by modifying the options. The five time options work like this:
* * * * *
| | | | |____ Day of Week (0-7) (Sunday=0/7, Monday=1)
| | | |______ Month (1-12)
| | |________ Day of Month (1-28/29) or (1-30/31)
| |__________ Hour (0-24)
|____________ Minute (0-59)
Example crontab
30 16 1,15 * * <script>
This above crontab example will run the “script” at 4:30 pm twice a month for every month on the 1st and the 15th day of the month and can be run on any day of the week.