Skip to main content
OCLC Support

Automating SFTP access with a protected password file

  1. Last updated
  2. Save as PDF

Applies to

  • Hosted EZproxy

Answer

Passwordless private/public key access to SFTP is not available. To script automatic access to SFTP to upload or download files, we will use the Linux sshpass utility. Although the native SFTP utility allows for passing a password on the command line, it is unsafe and we will not use it. Sshpass allows for passing a password via a hidden password file.  

This is similar to using the id_rsa key file, but instead of SSH reading id_rsa, sshpass reads the hidden password file, which is stored in your .ssh directory along with other keys.

Install sshpass on your Linux system if it's not already installed. You can typically do this using your package manager. For example, on Ubuntu, you can run sudo apt-get install sshpass. Other Linux distros have their own package manager.  

This process does not use any environment variables or expose the password on the command line, both of which are unsafe.

Overview

This process requires three files:

  1. Hidden Password File: Stored in your .ssh directory, this file contains the SFTP password and will have very restrictive permissions.
  2. SFTP Response File: Contains the SFTP commands to execute.
  3. Shell Script: Executes the SFTP session using the password and response files. This script can be run manually or automated via a cron job.

Step-by-Step Instructions

Step 1: Create the hidden password file

  1. Create the Password File:

echo 'your-password-here' > ~/.ssh/.passwd

  1. Set file permissions:

Ensure only the user can read the password file:

chmod 0400 ~/.ssh/.passwd

Step 2: Create the SFTP response file

  1. Create the response file:

This file will contain the commands to be executed once logged into the SFTP server. Below is an example response file, with comments:

# Change to the local files directory

lcd /home/username/files

 

# Change to the pickup directory on the SFTP server

cd pickup

 

# Copy files starting with 't' to the local files directory

mget t*

 

# Exit SFTP and return to the shell

exit

Save the above content in a file called download-my-files.sftp.

Step 3: Create a shell script to run the response file

  1. Create the Shell Script:

From the command line, create a shell script file and make it executable:

touch ~/daily-file-download.sh

chmod 0700 ~/daily-file-download.sh

  1. Edit the shell script:

Add the following content to the daily-file-download.sh script:

#!/bin/bash

 

# Connect to the SFTP server and execute the response file

sshpass -f ~/.ssh/.passwd sftp -oBatchMode=no -b download-my-files.sftp user@scp.oclc.org

Running the script

  • Manual execution:

Run the script manually by executing:

./daily-file-download.sh

  • Automated execution:

To run the script on a schedule, add it to your cron jobs. For example, to run the script daily at midnight, add the following line to your crontab file (crontab -e):

0 0 * * * /home/username/daily-file-download.sh

Conclusion

By following these steps, you can automate SFTP file transfers securely without exposing your password on the command line or using unsafe environment variables. This method ensures your password is stored securely and your SFTP sessions can be automated.

 

Page ID

58512