Guide to Submitting HTCondor jobs

High Throughput Computing (HTC)

First a note: HTC is meant for a large number of small to medium size jobs. HTC is different than HPC (High Performance Computing) that would run large size jobs that require a high memory (RAM) footprint, and/or extremely large data files.

Biochemistry Department personnel can obtain access to the in-door cluster  (Biochemistry Compute Cluster, see page BCC.) For larger projects the Center for High Throughput Computing (CHTC) may be helpful and offers free access to all UW–Madison researchers. Jobs that require even more computation may be helped by the Open Science Grid (OSG.)

In all cases, computation is managed by submitting “jobs” that are managed by a “scheduler” called HTCondor.

HTCondor

A tutorial specific to BCC is available on a separate page: tutorials-htcondor.

Below are videos providing a general introduction to HTcondor and offers example on how to submit jobs.

HTC Job Execution with HTCondor (48:23)

This was the second public lecture during OSG Virtual School 2021.

Materials and slides are available at  OSG Virtual School Materials

A User’s Guide to Submitting HTCondor jobs (54:54)

Similar material is also available on this alternate version of the video from 2020.

HTCondor manual

The latest HTCondor manual is online at: htcondor.readthedocs.io

Most useful: a QuickStart guide is available: quick-start-guide

---