Instructions for running IWFM on a NERSC machine:
=================================================

NERSC machines contains hundreds to thousands of nodes which are either configured as "head" or "compute" nodes. When logging onto a NERSC machine, you are automatically assigned a head node. Head nodes are used to launch a job to the compute nodes. Running jobs on the head node may result in account closure.

Log onto a NERSC machine with your <username>

<username>@franklin.nersc.gov or  <username>@hopper.nersc.gov

1) >cd /project/projectdirs/m997/scripts

There are two options for job submission to a compute node:

a) Job submission in interactive mode (to the debug queue only):

2) > qsub -IV -q debug
3) > aprun -a xt -n 1 -N 1 ./run.sh

run.sh calls the (i) preprocessor; (ii) IWFM and the (iii) budget calculator.

b) Job submission in batch mode:
2) > qsub IWFM_job

IWFM_job is a sample qsub script which launches the aprun command. 

It is most convenient to use the interactive mode for debug queue submissions. Job submission in batch mode returns a job ID to the console.

Use this ID to keep track of the job status by typing qstat <JOB_ID> or delete the job by typing qdel <JOB_ID>. On completion, an output and error files corresponding to the job ID are returned.

Building IWFM from the source repository
=======================================
All source, including HEC_lib are stored in a SVN repository on UC Davis CS departmental server.
To checkout the source, type:

>svn co svn +ssh://roger@roger.cs.ucdavis.edu/home/roger/work/dixon/IWFM/svn/IWFM

Note this is password protected and you will need to contact a IWFM project member to obtain this. 

The source is built from the makefile located under
>cd IWFM/IWFM/src/
>make

Note that no installation file is currently written. The preproc, IWFM and budget binaries located in the Intel64 directories must therefore be moved to the /project/projectdirs/m997/bin directory manually. 

The linux version of IWFM currently only builds under Intel 32 an 64 bit mode. NERSC does not support intel compilers. This means that the 64 bit intel compiler has to been installed in a user directory.

Example QSub Output from IWFM with the C2VSIM dataset
===================================================

**************************************************
TOTAL RUN TIME: 20 MINUTES 42.776 SECONDS
**************************************************
Application 141951 resources: utime 0, stime 0

 + ----------------------------------------------------------------
 +        Job name: IWFM_job
 +          Job Id: 5002135.hopper05
 +      Start Time: Tue Jan 19 18:57:04 PST 2010
 +        End Time: Tue Jan 19 19:17:48 PST 2010
 +        Walltime: 00:20:44 ( 1244 secs )
 +            User: mfdixon
 +            Host: nid00256
 +           Queue: reg_short
 +  Req. Resources: walltime=00:30:00
 +     Acct String:
 + ----------------------------------------------------------------