scinet_rclone_box
Table of Contents
Transferring files between USDA Box account and SciNet
The rclone home page is https://rclone.org/
rclone installation on SciNet
- Go to the web page at https://rclone.org/downloads/ and find the url for the AMD64 - 64 Bit Linux version, and download it e.g.
wget https://downloads.rclone.org/v1.48.0/rclone-v1.48.0-linux-amd64.zip
- Uncompress e.g.
unzip rclone-v1.48.0-linux-amd64.zip
- Put the rclone program in your bin directory, which will put it in your default PATH
mv rclone-v1.48.0-linux-amd64/rclone ~/bin/
- Install the rclone manual page (directory may not exist)
mkdir -p ~/man/man1 && mv rclone-v1.48.0-linux-amd64/rclone.1 ~/man/man1/
rclone installation on Windows
You will have to install rclone on Windows so that you can generate an authentication token for Box with the USDA VPN and eAuthentication active
- Go to the web page at https://rclone.org/downloads/ and find the Windows installer. Download it and install rclone
- Open a Windows command prompt (cmd)
- Type
rclone authorize "box"
- On the web page that shows up, click on Use Single Sign On (SSO)
- Enter your USDA email address
- Do the eAuthentication thing
- Click on the Grant access to Box button
- Go back to the command prompt window, an authentication token should be there. Copy the part between the braces {“access_token”:“ABCDEF…}
rclone configuration on SciNet
- Type
rclone config
- Type n for n) New remote
- For name> enter any name e.g. usdabox
- For Storage> you can find the number, but it's easier to just type box
- For both client_id> and client_secret> leave blank, just hit enter
- For Edit advanced config enter n
- For Remote config, Use auto config? enter n
- Paste the text from the last step from the section above that you got from your Windows computer
- Type y for y) Yes this is OK
- Type q to quit
Test
- Test directory listing using the name> you selected earlier:
rclone lsd usdabox:
- Test file listing
rclone lsl usdabox: | head
Commands
- For description of commands available see https://rclone.org/docs/#subcommands
- You installed the rclone manual page earlier, so you can also do
man rclone
- A common case might be to backup your SciNet project directory to box. You could do this with e.g.
rclone copy /project/bogus_genome usdabox:/scinetbackup/bogus_genome --verbose
scinet_rclone_box.txt · Last modified: 2019/06/25 13:03 by dsenalik