Topics Map > SCS Computing > HPC - Clusters
Topics Map > SCS Computing > How-To

SCS Clusters

info on scs clusters


Introduction and Access

In order to be able to reach the cluster(s) you need to either be on campus or using the VPN.

Lop: 

SCS maintains a computational cluster which is available for use by anyone in the School of Chemical Sciences called lop (DNS lop.scs.illinois.edu.) To get access, fill out the form located here: https://go.scs.illinois.edu/cluster-account . Once you have access, you will log in using your netid and password (The same password used for e-mail, vpn, etc)


Accessing software: 

Some of the software installed will just “work” if you try to run it when logging in. Most packages will require you loading a module first. To do so: 

  1. Type this command to see the available modules to load
    1. module avail
  2. To load a specific module, type the following command (gaussian is used as an example, replace with the one you want)
    1. module load gaussian/g16
    2. If you get a message such as "ERROR: Module 'X' depends on one of the modules 'Y" you need to load module(s) Y first.
  3.  To see which modules you currently have loaded
    1. module list
  4. To unload a module (to load a different version for example). 
    1. module unload gaussian/g16

If there is any software not currently on the cluster that you would like installed, e-mail scs-help@illinois.edu and inquire if that is possible. 

Queues:

Lop:
amd16smt: This queue is composed of dual socket AMD Opteron CPUs with 4 physical cores each with SMT (Simultaneous MultiThreading) enabled yielding 16 cores per node. They each have 64 GB of memory.

gpu: This queue is composed of nodes populated with Nvidia GPUs for use in cuda calculations. Compute-1-20 contains 4xGTX 980s. Compute-1-21 has 4x TITAN X. Compute-1-22 and Compute-1-23 both have 4xTesla k80s. This queue is only to be used for cuda calculations. Never submit a CPU compute job to this queue.

ib1: This queue is composed of dual socket AMD Opteron CPUs with 4 physical cores each with SMT enabled for 16 total cores. Nodes 2-0 through 2-16 have 64GB of memory while 2-17+ have 32GB. They are connected together via infiniband for multi-node calculations (eg. MPI)

Intel24: This queue is composed of dual socket Intel Xeon CPUs with 12 physical cores each with HyperThreading disabled for 24 cores total per node. They have 256GB or memory per node.

Lipid: (No longer active)
all.q: This queue is composed of dual socket AMD Opteron CPUS with 8 physical cores each with SMT enabled yielding 32 cores per node. They each have 32GB of memory.



Job Scheduler:

The clusters are installed using Rocks which uses SGE (Sun Grid Engine) as its job scheduler (SGE man pages). 

Quotas:

There is currently a 148 core limit per user enforced on Lop as well as lipid so that no one person can use all the resources on the cluster. In the future, this will be an adaptive quota so that resources are not left idle if only a few people are trying to use them, but for now it is static. If an individual has a pressing need to go above the quota, you can contact us at scs-help@illinois.edu and we will evaluate on a case by case basis.

The command qquota will show how many you are currently using. There is no output if you have no jobs running.

Storage:

Each user has a quota of 250GB on Lop. If more storage is required beyond that (large data sets being calculated for example) contact scs-help@illinois.edu to see if more can be allocated on a case by case basis. Also remember the  Storage Policy - Home Directories  from the access form
Students in courses are treated differently from SCS students, staff and faculty. Their access to Lop will be revoked after the end of finals for a semester and all data will be deleted. If a student wishes to keep anything, said student must remove it from lop before the end of the semester.

Software Tutorials: 

Below, find various links to tutorials for software on the cluster. Many of these are out of date but we are working to make them current: 

Unix/Linux Primer

Intro to Gaussian Part I

Intro to Gaussian Part II

Simulating Vibrationally-resolved Electronic Spectra Using Gaussian 

Determining the pKa of Simple Molecules using Gaussian

X-forwarding

A few software packages have a graphical interface so you will need to set up X forwarding. 

Windows Instructions:

MobaXterm seems to work best right after install with little configuration. https://mobaxterm.mobatek.net/ Just install, put lop.scs.illinois.edu into the quick connect box and you're off. See below for instructions on getting Xming to work, though it hasn't been as reliable since end of 2020.

  • Xming-mesa instructions

      1. Download and install Xming-mesa (not normal xming). It can be found here: https://sourceforge.net/projects/xming/files/Xming-mesa/
      2. Run the installer
      3. Open NotePad as administrator 
      4. Click on file –> Open in notepad. Open the X0.hosts file in the xming install directory as shown in this picture: scsclusters.png                               
      5. Under localhost on a new line put in 130.126.43.205 (this is the IP for lop). Save the file and close notepad 
      6. Download and install putty found here: https://the.earth.li/~sgtatham/putty/latest/w64/putty-64bit-0.74-installer.msi  
      7. Launch xming
      8. Launch putty
      9. Expand the SSH tree under Connection and click on x11. Click on the box for Enable x11 forwarding
      10. Click on the top option on the right "Session" 
      11. In the Host Name box put lop.scs.illinois.edu then click the Open button
      12. A box will pop up prompting you to "togin as" – put in your netID and hit Enter
      13. It will then prompt you for a password. Use your netID password. Nothing will show as you type, this is standard Unix behavior. 
      14. You will get a prompt. You are now connected to Lop.

Mac Instructions:

  1. Download and install xQuartz 2.7.7 (do not get any other version, they don't seem to work with Gaussview)
  2. Install the downloaded dmg
  3. Reboot your computer entirely
  4. Log back in after reboot and launch xQuartz
  5. If an xterm window did not open when you launched Quartz, right click/control click on it in the Dock and launch xterm
  6. In the opened xterm type: ssh -X netid@lop.scs.illinois.edu 
  7. Put in your netID password when prompted
  8. This will give errors about unable to connect. This is fine, this step seems to possibly set something in the user profile
  9. load the gaussian module (module load gaussian/g16) and then type gv. This will fail saying cannot connect to x-server
  10. type exit to disconnect
  11. type ssh -Y netid@lop.scs.illinois.edu
  12. load the gaussian modjule and launch gv. This should bring up the Gaussview window properly.
  13. If the above steps do not work, try making a brand new account on your mac and follow the above steps under it to see if it works there
  14. If it still does not work, contact scs-help@illinois.edu 








Keywords:lop lipid cluster   Doc ID:104365
Owner:Mark H.Group:University of Illinois School of Chemical Sciences
Created:2020-07-26 22:37 CDTUpdated:2021-06-29 13:44 CDT
Sites:University of Illinois School of Chemical Sciences
CleanURL:https://answers.uillinois.edu/scs/scs-clusters
Feedback:  0   0        
Documentation for SCS Computing services