Best (my) way to use CRC clusters

• PuTTY https://www.putty.org/
• Notepad++ https://notepad-plus-plus.org/downloads/
• FileZilla https://filezilla-project.org/download.php?type=client

If you are using a Mac, use the following similar software

• Terminal (already installed) to replace PuTTY.
• TextWrangler to replace Notepad++
• FileZilla can be equally used on a Mac.

See the screen below:

• Host: crcfe01.crc.nd.edu
• Username: your netid
• Password: your password
• Port: 22
• Click Quickconnect
• On the remote site (right panel), browse to the Private folder. Typically create folders and files under the Private folder.
• On the Local site (left panel), choose the files/folders to upload and drag them to the remote site folder.

See the screen below:

• Host name: crcfe01.crc.nd.edu
• Port: 22
• Connection type: ssh
• You might save the information by clicking “Save” button.
• Click on Open button
• In the new window
  • log in as: Your netid
  • password: your password (note that you will not say anything when typing. just finish typing your password).
  • Then you should login.

• pwd : print the current directory
• ls : list the file and folders
  • ls -alh: list all the information
• cd dir : change to a new directory (dir)
  • cd .. : go to last level of directory
  • cd : go the the initial directory you log in
• mkdir dir : create a new directory called dir
• rm file1 : delete a file
• rm -r dir : delete a directory
• nano : a text editor.
• chmod g+s dirname: file created belongs to the group.
• setfacl -d -m g::rwx mathfeud : set the permission for the group

• Use the ftp of Notepad++ to edit files directly.
• Show the ftp windows by the menu: Plugins – nppFTP – show NppFTP window (see the screen below)

• On the ftp window, click the settings button and then Profile settings (see the screen below)

• Then in the new window
  • Click “Add new” on the bottom left corner
  • Give a name, e.g., CRC
  • Then input the information required
    • Hostname: crcfe01.crc.nd.edu
    • Connection type: SFTP
    • Port: 22
    • Username: your netid
    • Password: leave empty and check “Ask for password”
    • Initial remote directory: the output of pwd commend in putty.
    • close the window

• Connect to ftp. Click the connection button and choose CRC as shown below.
  • Input your password when asked.
  • Your files will be listed.
  • To edit a file, click on it and it will open in Notepad++.
  • You can also right click to create/delete a directory, create/delete a file.
  • After editing, save the file directly using Notepad++.

For more information, go to http://wiki.crc.nd.edu/wiki/index.php/Submitting_Batch/SGE_jobs

• Log in using Putty
• Create a new folder -test- within Putty using command

  mkdir test

  or do that within Notepad++ using the FTP plugin

• Go the new folder test.
• Create a test R file with the following content. You can use Notepad++. If using Putty, you can use the

  nano test.R

  The content of the R file is

  rnorm(100)

  • When using nano, Ctrl+O saves the file and Ctrl+X quits the edits.
• Create another file (a submission file) called test.sh. In the file, include the following information (change the email address to yours)

  #!/bin/csh
  #$ -M xxxx@nd.edu
  #$ -m abe
  #$ -r y
  
  module load R
  
  R CMD BATCH test.R
  

• To submit a job, in Putty (has to use Putty or other SSH client), type the command

  qsub test.sh

  Your job should be submitted now.

• To check the status of your job, use

  qstat -u netid

  Your job should already finished by now, so you won't see anything for this simply example.

• You can list all the files using command ls. You will see a file called test.Rout. Any error can also be found in it.
• There will also be a file called test.sh.oxxxx (xxxx is a number). If no error, the size of it should be 0 typically.
• If there is error in your job, you will see a file like test.sh.exxxx.
• To use SAS, change module load R to module load sas. All available module can be found using module avail
• If the test job works fine, you can start your regular jobs!

Sometimes, it might be convenient to split a big jobs to smaller ones. Preferably, each small job runs within its own folder. For example, one might have a sas job in the following folders

• sasjob1
• sasjob2
• sasjob3
• sasjob4

Within each folder, there is a sas file called runsas.sas. To submit such jobs, use the batch file with contents below

#!/bin/csh
#$ -M zzhang4@nd.edu
#$ -m abe
#$ -r y
#$ -t 1-4

module load sas
cd sasjob$SGE_TASK_ID

sas runsas.sas

If one prefers to put all small jobs in one folder like

• sasjob1.sas
• sasjob2.sas
• sasjob3.sas
• sasjob4.sas

Then the following script file can be used

#!/bin/csh
#$ -M zzhang4@nd.edu
#$ -m abe
#$ -r y
#$ -t 1-4

module load sas

sas sasjob$SGE_TASK_ID.sas

To reset the CRC environment, use

/opt/crc/usr/local/bin/crc_setup