How do I use it? A quick Condor tutorial

How do I set up a job?

Creating a submit description file

$(Process) is a variable substitution; it will be replaced by the process number of each process that's queued. Consult the condor_submit manpage ( man condor_submit) for more details.

Being "nice"

If you have a very large queue of jobs to run, but don't care if they finish quickly, you can add the following to your submit file as a courtesy to other users:

How do I use it? A quick Condor tutorial

How do I set up a job?

Creating a submit description file

• The Executable line tells Condor what program we want to run.
  • The default path is the current directory. If the executable is somewhere else, you need to supply the full path -- Condor will not search for it the way the shell does.
  • If you need to know the full path to a program that's in your default path, use the which command at a shell prompt. For example: which lexparser.csh

• getenv = true transfers all the environment variables that are set in the submitter's shell. This is what you want most of the time; much of our software depends on environment variables to locate binaries and libraries.
• Log indicates where the Condor log file for this job should go.
• transfer_executable = false tells Condor it does not need to copy the executable file to the compute node. This is usually the case, since the cluster nodes share a common filesystem.

How do I use it? A quick Condor tutorial

How do I set up a job?

Creating a submit description file

• The Executable line tells Condor what program we want to run.
  • The default path is the current directory. If the executable is somewhere else, you need to supply the full path -- Condor will not search for it the way the shell does.
  • If you need to know the full path to a program that's in your default path, use the which command at a shell prompt. For example: which lexparser.csh

• getenv = true transfers all the environment variables that are set in the submitter's shell. This is what you want most of the time; much of our software depends on environment variables to locate binaries and libraries.
• Log indicates where the Condor log file for this job should go.
• transfer_executable = false tells Condor it does not need to copy the executable file to the compute node. This is usually the case, since the cluster nodes share a common filesystem.

• TroubleshootingCondor - Troubleshooting Condor job problems.
• BigMemoryCondor - Tips for running jobs that need more than 2 GB of RAM.
• CondorClusterHomepage - General info about the Condor cluster.

How do I use it? A quick Condor tutorial

How do I set up a job?

Creating a submit description file

Submitting the job

Now that you have a description file, submitting it is as simple as: condor_submit foobar.cmd

• condor_q lists the job queue.
• condor_rm deletes a job from the queue.

• TroubleshootingCondor - Troubleshooting Condor job problems.
• BigMemoryCondor - Tips for running jobs that need more than 2 GB of RAM.
• CondorClusterHomepage - General info about the Condor cluster.

How do I use it? A quick Condor tutorial

How do I set up a job?

Creating a submit description file

• condor_q lists the job queue.
• condor_rm deletes a job from the queue.

Additionally, CondorView provides status graphs, updated every 15 minutes.

How do I use it? A quick Condor tutorial

How do I set up a job?

Creating a submit description file

$(Process) is a variable substitution; it will be replaced by the process number of each process that's queued. Consult the condor_submit manpage ( man condor_submit) for more details.

Things to keep in mind

If you want to put input, output, or error files on a non-shared filesystem such as /tmp, you can add stream_input=true, stream_output=true, and/or stream_error=true to your submit file. This tells Condor to pipe the output back to the original submitting system instead of creating the file on the node. It may add a slight performance penalty of you're doing a lot of I/O.

• BigMemoryCondor - Tips for running jobs that need more than 2 GB of RAM.
• CondorClusterHomepage - General info about the Condor cluster.

How do I use it? A quick Condor tutorial

How do I set up a job?

Creating a submit description file

• The Executable line tells Condor what program we want to run.
  • The default path is the current directory. If the executable is somewhere else, you need to supply the full path -- Condor will not search for it the way the shell does.
  • If you need to know the full path to a program that's in your default path, use the which command at a shell prompt. For example: which lexparser.csh

• getenv = true transfers all the environment variables that are set in the submitter's shell. This is what you want most of the time; much of our software depends on environment variables to locate binaries and libraries.

• transfer_executable = false tells Condor it does not need to copy the executable file to the compute node. This is usually the case, since the cluster nodes share a common filesystem.

/condor/examples contains some sample jobs. You may want to examine some of the submit description files there to get a better feel for how this works in different situations.

• condor_status lists available nodes and their status.
• condor_q lists the job queue.

Additionally, CondorView provides status graphs, updated every 15 minutes.

Executable = foobar

Things to keep in mind

Automating Job Submission from the Command Line

The CondorExec program allows you to send an arbitrary command line to Condor.

How do I use it? A quick Condor tutorial

How do I set up a job?

Creating a submit description file

• The Executable line tells Condor what program we want to run.
  • The default path is the current directory. If the executable is somewhere else, you need to supply the full path -- Condor will not search for it the way the shell does.
  • If you need to know the full path to a program that's in your default path, use the which command at a shell prompt. For example: which lexparser.csh

• getenv = true transfers all the environment variables that are set in the submitter's shell. This is what you want most of the time; much of our software depends on environment variables to locate binaries and libraries.
• Log indicates where the Condor log file for this job should go. Condor complains if this is located on an NFS filesystem, so putting it in a subdirectory of /tmp is a good idea. Input, output, and error files can go to your home directory.
• transfer_executable = false tells Condor it does not need to copy the executable file to the compute node. This is usually the case, since the cluster nodes share a common filesystem.

How do I use it? A quick Condor tutorial

How do I set up a job?

Creating a submit description file

Submitting the job

To keep the cluster responsive, long-running processes run on patas itself will automatically have their CPU priority lowered. Additionally, processes on patas itself are limited to no more than 2 GB of RAM. Processes submitted to Condor are not affected by this, so you should try to use Condor for anything CPU-intensive.

How do I use it? A quick Condor tutorial

How do I set up a job?

Creating a submit description file

Here's a sample Condor submit file (let's call it foobar.cmd) that does the same thing:

How do I use it? A quick Condor tutorial

How do I set up a job?

Creating a submit description file

How do I use it? A quick Condor tutorial

How do I set up a job?

Creating a submit description file

Advanced options

How do I use it? A quick Condor tutorial

How do I set up a job?

Creating a submit description file

• Log indicates where the Condor log file for this job should go. Condor complains if this is located on an NFS filesystem, so putting it in a subdirectory of /tmp is a good idea. Input, output, and error files can go to your home directory.
• transfer_executable = false tells Condor it does not need to copy the executable file to the compute node. This is usually the case, since the cluster nodes share a common filesystem.

How do I use it? A quick Condor tutorial

How do I set up a job?

Creating a submit description file

Things to keep in mind

How do I use it? A quick Condor tutorial

How do I set up a job?

Creating a submit description file

A few of these lines require explanation.

• Universe = vanilla indicates that this is an ordinary program that does not support checkpointing. Other possibilities include standard, for programs that are linked with the Condor libraries and support checkpointing and restarting; and java, for running Java programs directly. See the Condor manual for more information about these universes. The PVM universe is not currently supported, but see the PVMOnPatas Wiki page for information on how to run PVM directly.
• getenv = true transfers all the environment variables that are set in the submitter's shell. This is what you want most of the time; much of our software depends on environment variables to locate binaries and libraries.
• Log indicates where the Condor log file for this job should go. Condor complains if this is located on an NFS filesystem, so putting it in a subdirectory of /tmp is a good idea. Input, output, and error files can go to your home directory.

How do I use it? A quick Condor tutorial

How do I set up a job?

Creating a submit description file

• Universe = vanilla indicates that this is an ordinary program that does not support checkpointing. Other possibilities include standard, for programs that are linked with the Condor libraries and support checkpointing and restarting; and java, for running Java programs directly. See the Condor manual for more information about these universes. The PVM universe is not currently supported, but see the PVMOnPatas Wiki page for information on how to run PVM directly.
• getenv = true transfers all the environment variables that are set in the submitter's shell. This is what you want most of the time; much of our software depends on environment variables to locate binaries and libraries.
• Log indicates where the Condor log file for this job should go. Condor complains if this is located on an NFS filesystem, so putting it in a subdirectory of /tmp is a good idea. Input, output, and error files can go to your home directory.

Advanced options

How do I use it? A quick Condor tutorial

How do I set up a job?

Creating a submit description file

Things to keep in mind

How do I use it? A quick Condor tutorial

How do I set up a job?

Creating a submit description file

Reducing startup time

transfer_executable = false

How do I use it? A quick Condor tutorial

How do I set up a job?

Creating a submit description file

Advanced options

Things to keep in mind

How do I use it? A quick Condor tutorial

How do I set up a job?

Creating a submit description file

• condor_hold and condor_rm put a job on hold and delete it from the queue, respectively.

Advanced options

How do I use it? A quick Condor tutorial

How do I set up a job?

Creating a submit description file

• Universe = vanilla indicates that this is an ordinary program that does not support checkpointing. Other possibilities include standard, for programs that are linked with the Condor libraries and support checkpointing and restarting; and java, for running Java programs directly. See the Condor manual for more information about these universes. The PVM universe is not currently supported, but see the PVMOnPatas Wiki page for information on how to run PVM directly.
• getenv = true transfers all the environment variables that are set in the submitter's shell. This is what you want most of the time; much of our software depends on environment variables to locate binaries and libraries.

Things to keep in mind

How do I use it? A quick Condor tutorial

How do I set up a job?

Creating a submit description file

Managing jobs

• condor_status lists available nodes and their status.
• condor_q lists the job queue.
• condor_hold and condor_rm put a job on hold and delete it from the queue, respectively.

How do I use it? A quick Condor tutorial

How do I set up a job?

Creating a submit description file

A few of these lines require explanation.

• getenv = true transfers all the environment variables that are set in the submitter's shell. This is what you want most of the time; much of our software depends on environment variables to locate binaries and libraries.
• Log indicates where the Condor log file for this job should go. Condor complains if this is located on an NFS filesystem, so putting it in a subdirectory of /tmp is a good idea.

How do I use it? A quick Condor tutorial

How do I set up a job?

Creating a submit description file

Submitting the job

How do I use it? A quick Condor tutorial

How do I set up a job?

Creating a submit description file

A few of these lines require explanation.

• getenv = true transfers all the environment variables that are set in the submitter's shell. This is what you want most of the time; much of our software depends on environment variables to locate binaries and libraries.
• Log indicates where the Condor log file for this job should go. Condor complains if this is located on an NFS filesystem, so putting it in a subdirectory of /tmp is a good idea.

How do I use it? A quick Condor tutorial

How do I set up a job?

Creating a submit description file

Executable = foobar
Universe   = vanilla

• getenv = true transfers all the environment variables that are set in the submitter's shell. This is what you want most of the time; much of our software depends on environment variables to locate binaries and libraries.
• Log indicates where the Condor log file for this job should go. Condor complains if this is located on an NFS filesystem, so putting it in a subdirectory of /tmp is a good idea.

Submitting the job

Managing jobs

• condor_q lists the job queue.
• condor_hold and condor_rm put a job on hold and delete it from the queue, respectively.

Advanced options

Executable = foobar
Universe   = vanilla

Things to keep in mind