Fm: Marco de Vos (Newstar Project Team)
To: Ulrich Schwarz & Paul Stoppelenburg

Subject: 
Hints for batchprocessing with Newstar
--------------------------------------


A. Layout for a batch-script
----------------------------

Batch-scripts always have the follwing layout:

>>>>>>>>>>>>>>>>>>> Example follows below this line >>>>>>>>>>>
#!/bin/csh -f
#
# Startup Newstar
#
source ~newstar/src/sys/newstar_rug.csh

#
# Maybe some identification
#
echo "Name of the script: $0"     # Prints the name of the file
echo "Purpose: ....."             # Print the purpose of the file

#
# Clear all symbols
#
dwc '*$ch0'

#
# Define the symbols for this job
#
dws nmap\$ch0 /nomenu <<_EOD_
OPTION=MAKE;QUIT ! The QUIT is for final exit
LOOPS="";#       ! The # is in case we want to stop at the SCN_NODE prompt
SCN_NODE=obs2; ""/ask    ! Just set the two defaults (/ASK)
SCN_SETS=0.0.0.0.0
HA_RANGE= -90,90
SELECT_IFRS= -mm,-ff;""  ! The second answer should be ""
... etc ...
#
_EOD_

#
#  Start the program
#
dwe nmap\$ch0

#
#  End
#
<<<<<<<<<<<<<<<< End of example <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<


There may be many different dws and dwe commands in a single file.

In stead of typing  nmap\$0  you can also use  'nmap$0'  .




B. Starting a batch-script
--------------------------

To start such a batch script, just type it's name, eg:

  > nmap0.scr

This may give a message like:     nmap0.scr: Permission denied.

In that case you sould first type:

  > chmod a+x nmap0.scr

to change the mode ("protection") of the file such that All users 
can eXecute it. You probably created the script from scratch with 
an editor, so Unix treated it as an ordinary text file.



C. Starting batch jobs in the background
----------------------------------------

To start a batch job in the background, leaving your terminal free
for other work, just type the following:

   > nmap0.scr >& output_file &

All output that normally was shown on your screen will now end up in 
the file named output_file (you can use any name you like).

To check wether the job is finished you can try:

   > more output_file           shows all of the "screen output"
   > tail output_file           shows the last 20 lines or so
   > ps lws | grep nmap0.scr    shows the status of the process (with
                                 elapsed time etc.), no output means
                                 the batch is finished

There is a Newstar command which will execute a job in the background
and send the output to you by mail when it finishes. Just type:

   > nspawn nmap0.scr 

The "screen output" will end up in  nmap0.scr.output. 
The mail will be sent using elm with the command as subject.

NOTA BENE: If you start a job in the background, all parameters should
           be known in advance. So you should not use /ASK when specifying
           parameters.



D. How to set the keywords for a program
----------------------------------------

In a batch-script, you can set keywords in two different ways:

1e. Use the dwspecify (dws) command:

dws nmap\$ch0 /nomenu <<_EOD_
OPTION=first_answer;second_answer;third_answer
SCN_NODE=TEST;""
#
_EOD_

The sequence of the keywords is not relevant.
Each keyword should be set like: KEYWORD= value (not: nmap$ch0_keyword=value).
You can only specify the keywords for one program (and one stream).

You can enter the KEYWORD=value lines by hand, or edit the output
from dwsave (remove the PROGRAM$Stream_ prefixes!)

Alternatively you can extract keywords from a logfile:

   >  grep '^>' NMAP.LOG | sed -e 's/^>//'  > output_file

If this seems an interesting option, I will create a Newstar command
for it (to save innocent Unix users all the '^>' things).


2e. Use the dwrestore command:


Typically, you will now do the following:

 > dwe ncalib\$ch0 /norun/save
 > dwe nmap\$ch0 /norun/save
 > dwsave nmap\$ch0 /output=nmap0.keys

Now you may want to edit nmap0.keys, which contains lines like:

 NMAP$1_LOOPS=""";#"
 NMAP$1_OPTION="MAKE;#"
 NMAP$1_RUN="YES"
 NMAP$1_SCN_NODE=""""

In the batch script, you say:

dwrestore nmap0.keys


3e. Combine the use of dws and dwrestore


Suppose we want to do a job repeatedly with the parameters saved above,
but with only SCN_SETS differing from command to command. We may then
combine dws and dwrestore as follows:


#!bin/csh
#!/bin/csh -f
#
# Startup Newstar
#
source ~newstar/src/sys/newstar_rug.csh

#
# Maybe some identification
#
echo "Name of the script: $0"     # Prints the name of the file
echo "Purpose: ....."             # Print the purpose of the file

#
# Clear all symbols
#
dwc '*'

#
# Set up all "global" symbols for this job
#
dwrestore nmap0.keys

#
# Start a loop over the sets
#
foreach set ( 0 1 3 4 )

#
# Define SCN_SETS for ncalib
#
dws ncalib\$ch0 /nomenu <<_EOD_
SCN_SETS=0.0.$set
#
_EOD_

#
# Run ncalib for this set
#
dwe ncalib\$ch0

#
# Define SCN_SETS for nmap
#
dws nmap\$ch0 /nomenu <<_EOD_
SCN_SETS=0.0.$set
#
_EOD_

#
# Run ncalib for this set
#
dwe nmap\$ch0

#
# Next set
#
end