Every submission form except TINKER's has a resources section where you choose where
the job runs and how big a slice of the machine it gets. The form only offers what your
role allows.
| Cluster | Run by | Cores per job | Time limits | GPUs | Good for |
|---|---|---|---|---|---|
| Jetstream2 | Indiana University | 1 | 1 or 6 h | — | Quick jobs, small molecules, class labs. Usually the shortest queue |
| Expanse | San Diego Supercomputer Center | 1, 8, 32 | 1, 6 or 48 h | yes | Long or large jobs. The only cluster where you can choose memory |
| Bridges-2 | Pittsburgh Supercomputing Center | 1, 8, 32 | 1, 6 or 48 h | yes | Long or large jobs |
| Delta | NCSA, University of Illinois | 1, 8, 32 | 1, 6 or 48 h | yes | Long or large jobs |
All four are NSF-funded national resources, used through the ACCESS program. Jobs on
Expanse, Bridges-2 and Delta may wait 5–15 minutes or more before starting.
auto lets ChemCompute choose. It picks the cluster with the shortest expected wait
that can run your job, and if that cluster has a problem it can move the job elsewhere for
you. If you chose the cluster yourself, a job that cluster can't take fails instead, with
an offer to resubmit somewhere else.
A cluster that's down for maintenance or having trouble disappears from the list until it
recovers. Upcoming maintenance is listed on your Dashboard.
The clusters you may use, plus auto. Each cores option shows the recent wait for that
cluster, e.g. (~4 min).
More cores make most quantum-chemistry jobs faster, and each core brings about 2 GB of
memory, so a job that runs out of memory needs more cores. On the other hand, small
molecules don't speed up much past a few cores, and bigger requests wait longer in the
queue. A reasonable rule of thumb:
Only accounts with GPU access see this. On ChemCompute, PySCF and NAMD jobs can
use a GPU, on Expanse, Bridges-2 and Delta. Choosing a GPU on the quantum-chemistry form
switches the package to PySCF. If your cluster has no GPUs (Jetstream2), the form moves
you to one that does. With a GPU the cores setting is ignored. GPU Type lets you
pick a specific card or Any. Running large jobs explains when a
GPU pays off and which card to choose.
Expanse only. Normally leave it alone. Asking for 64 GB is charged as 32 cores, whatever
number of cores you chose.
The job is stopped when it reaches this, and its status becomes TIMEOUT. Pick a limit
comfortably longer than you expect the job to need, but no longer: long limits can wait
longer in the queue. Logged-in users get 6 hours by default.
Under the resources section, the form estimates how long your job will take, from
earlier ChemCompute jobs like it:
Estimated time on the cluster: about 12 minutes ± 40%
Based on 23 atoms, B3LYP/6-31G*, geometry optimization, 1 core, …
This is time on the cluster only. Time spent waiting in the queue comes on top of it.
If similar jobs often run past your time limit, it says so and recommends a longer limit.
The estimate changes as you change the molecule, method or cores, so it's a quick way to
see what an extra core or a bigger basis set will cost.
On the guided quantum-chemistry and NAMD forms, a note can also appear just above the
Submit button when another cluster would probably finish your job well ahead of the one
you picked, once the wait in the queue is counted. It gives the total for both and has a
Switch to button that picks that cluster for you. It only speaks up when the other
cluster saves at least 10 minutes and a third of the time, and it stays quiet while
auto is chosen, since auto picks a cluster for itself.
Files the job in a project, so
your instructor or research group can see it. You can also do this later from the
Dashboard.
Jobs are charged in CPU-hours: cores × hours the job actually ran, not the time limit
you asked for. An 8-core job that runs for 15 minutes costs 2 CPU-hours. GPU jobs are
charged in service units (60 per GPU-hour). Memory on Expanse is charged as cores (64 GB
counts as 32 cores).
Your balance is on the Dashboard as CPU-hours: used / allocated. While a job is
queued or running, part of what it could cost (based on its cores and time limit) is set
aside and shown as Locked. It's released when the job finishes, and you're charged
only for the time it actually used. Because of this, a job with many cores and a long
time limit can be refused as Over allocation even though you have hours left. Ask for
less, or use Request More CPU-Hours / Resources on the Dashboard.
Each job's results page has a Job details panel with a Resources table: the cores
and memory it asked for and used, time in the queue, time running, CPU efficiency and
CPU-hours charged. It's worth a look after a test run, to choose cores and time limits for
the real one. A low CPU efficiency on many cores means the job would run about as
fast on fewer.