User Tools

Site Tools


scinet_rclone_box

Transferring files between USDA Box account and SciNet

The rclone home page is https://rclone.org/

rclone installation on SciNet

  1. Go to the web page at https://rclone.org/downloads/ and find the url for the AMD64 - 64 Bit Linux version, and download it e.g.
    wget https://downloads.rclone.org/v1.48.0/rclone-v1.48.0-linux-amd64.zip
  2. Uncompress e.g.
    unzip rclone-v1.48.0-linux-amd64.zip
  3. Put the rclone program in your bin directory, which will put it in your default PATH
    mv rclone-v1.48.0-linux-amd64/rclone ~/bin/
  4. Install the rclone manual page (directory may not exist)
    mkdir -p ~/man/man1 && mv rclone-v1.48.0-linux-amd64/rclone.1 ~/man/man1/

rclone installation on Windows

You will have to install rclone on Windows so that you can generate an authentication token for Box with the USDA VPN and eAuthentication active

  1. Go to the web page at https://rclone.org/downloads/ and find the Windows installer. Download it and install rclone
  2. Open a Windows command prompt (cmd)
  3. Type
    rclone authorize "box"
  4. On the web page that shows up, click on Use Single Sign On (SSO)
  5. Enter your USDA email address
  6. Do the eAuthentication thing
  7. Click on the Grant access to Box button
  8. Go back to the command prompt window, an authentication token should be there. Copy the part between the braces {“access_token”:“ABCDEF…}

rclone configuration on SciNet

  1. Type
    rclone config
  2. Type n for n) New remote
  3. For name> enter any name e.g. usdabox
  4. For Storage> you can find the number, but it's easier to just type box
  5. For both client_id> and client_secret> leave blank, just hit enter
  6. For Edit advanced config enter n
  7. For Remote config, Use auto config? enter n
  8. Paste the text from the last step from the section above that you got from your Windows computer
  9. Type y for y) Yes this is OK
  10. Type q to quit

Test

  1. Test directory listing using the name> you selected earlier:
    rclone lsd usdabox:
  2. Test file listing
    rclone lsl usdabox: | head

Commands

  1. For description of commands available see https://rclone.org/docs/#subcommands
  2. You installed the rclone manual page earlier, so you can also do
    man rclone
  3. A common case might be to backup your SciNet project directory to box. You could do this with e.g.
    rclone copy /project/bogus_genome usdabox:/scinetbackup/bogus_genome --verbose
scinet_rclone_box.txt · Last modified: 2019/06/25 13:03 by dsenalik

Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki