CondorClusterHomepage < Main

Main Web>CondorClusterHomepage (2012-04-27, brodbd) (raw view)EditAttach

---+ CLMS Parallel Computing Cluster
---++ What is it?

The Computational Linguistics program maintains a cluster of several servers (referred to as "compute nodes") that can run multiple processes in parallel, allowing large jobs -- if properly constructed -- to be run faster than they could on any single computer. We offer two high-performance computing frameworks:

   * [[HowToUseCondor][Condor]] can run any type of software, but leaves the details of handling the data to you. It's essentially a very flexible scheduler for matching batch jobs with available compute nodes.
   * [[PatasHadoop][Hadoop]] can only run programs written using the MapReduce framework; however, it handles many of the intermediate data processing details itself.

Equipment for the cluster is partially funded by the Student Technology Fee.
---++ How do I get an account?

If you do not yet have a Computational Linguistics account, fill out our online [[https://vervet.ling.washington.edu/db/accountrequest-form.php][account request form]]. You must have a valid UW !NetID.
---++ How do I access the cluster?

Open an SSH connection to patas.ling.washington.edu port 22 or dryas.ling.washington.edu port 22. You can also access files on the cluster directly via Samba filesystem shares. See HowToAccessPatas for more information.

---++ How do I use Condor to run my parallel jobs?

Visit the HowToUseCondor Wiki page for a quick introduction. You may also want to view the [[http://staff.washington.edu/brodbd/orientation.pdf][PDF presentation]] I wrote for orientation.
---++ Software notes and reference information
---+++ General Tips and How-Tos
   * PatasFileSystems - Overview of shared filesystems in the cluster.
   * PatasShebangLines - Script headers for installed interpreters.
   * DataProtection - What we do to protect data stored on our systems against loss, and what users need to do to protect their own data.
   * DiskQuotas
   * XmingGotchas - Problems with running Emacs (and some other apps) using the Windows X server [[http://sourceforge.net/projects/xming][Xming]]
   * [[http://www.squish.net/docs/delbs.html][Fixing backspace/delete problems in Emacs]] (will take you to an external site)
   * [[http://www.albany.edu/faculty/gms/homepage101/unix_permissions.html][Intro to UNIX file permissions]] (will take you to an external site)
   * [[LocaleSettings][Locale settings and character encodings]]
---+++ Parallel/High-Performance Computing

   * [[http://www.cs.wisc.edu/condor/manual/v7.6/2_Users_Manual.html][The official Condor manual]]
   * PatasHadoop - Using Hadoop on our cluster
   * PatasMahout - Using the Mahout scalable machine learning toolkit
   * PeformanceTips - Advice on writing fast parallel jobs
   * PerformanceProblems - Common job performance pitfalls.
   * TroubleshootingCondor - Troubleshooting Condor job problems.
   * BigMemoryCondor - Tips for running jobs that need more than 1 GB of RAM.
---+++ Programming Languages

   * [[UsingCSharpOnPatas][Using C# on Patas]]
---++ Linguistics software

A listing of linguistics software on Patas can be found in the [[https://vervet.ling.washington.edu/db/index.php][CompLing database]].

---++ Patas? What's that?

[[http://pin.primate.wisc.edu/factsheets/entry/patas_monkey][Patas monkeys]] are slender, ground-dwelling primates found in central Africa. They can run faster than any other primate, reaching speeds of up to 55 km/hr.

Topic revision: r36 - 2012-04-27 - 23:44:46 - brodbd

Main

Webs
Main
Sandbox
TWiki

Copyright &© by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback
Privacy Statement Terms & Conditions