Date: 08/11/94
Revision history
This document is limited to configuration management on the Unix systems. The VAX system will be no longer supported at NFRA from 15/03/94, and can be maintained through the perl-scripts provided by WNB.
Reference to Newstar directories are made through the usual variables $n_src, $n_root etc. Refer to Appendix 2 for the details.
All commands assume one is logged in as the Newstar Master (newstar).
Below /newstar/master we also have an installation of: (1) mongo (in /newstar/master/mongo), of (2) perl (in /newstar/master/perl) and (3) a copy of the sources for the VAX-based Remote Tape Daemon (in /newstar/master/rmtd).
This is a working system, used by NFRA users and programmers.
This is a sources-only system which is never compiled. The Export Master source tree and the documentation are kept up-to-date from the NFRA Master system.
# # Initialise Newstar (either NFRA-Master or Export-Master) # if (-e /newstar/master/src/sys/newstar_nfra.csh) then source /newstar/master/src/sys/newstar_nfra.csh else if (-e /users/newstar/bin/newstar_init.csh) then source /users/newstar/bin/newstar_init.csh endif
This account is the owner of the NFRA-Master and the Export-Master, and is the only account that can modify them (apart from $n_import and the locking database).
+----------------------------------------------------------------------+ | Master: | | $n_src <--------+ | | | | nup build -U (in $n_import) | | | nup build -U | | | v | | | $n_exe, ... <--------+ | | | | | $n_import -->--------+ | | | ^ ^^ | | | | || | | | | || (NFRA only) | | | | |+--------------------+--<-- nsh in | | | | | | | +----------------------------------------------------------------------+ | Export-Master: | | | | | | | | | nup retrieve ... | (NFRA and others) | | | | | (NFRA only) | | | nup release | | | | | | | | nup retrieve | | | v | | | | | $n_src | | | | $n_import <------------------+ | | | +----------------------------------------------------------------------+Use of the "nup release" command is restricted to the NFRA-Master. Master systems outside NFRA use the "nup retrieve" command to get updated files from the Export-Master. The "nup update" command combines "nup retrieve" and "nup build -U".
A release is defined as a revision which involves one or more of the following items:
The updating of revision numbers is taken care of by the update script. Releases need to be explicitly indicated. The procedure for this is described in a later section.
The version number of the current Newstar configuration is given by the file $n_src/sys/version.idx
A full description of the current Newstar configuration is given by the file $n_src/sys/database.idx after the command "nup check d"
A user-oriented decription of the configuration is given in the file $n_src/doc/nnews.hlp which is shown by the command "nnews"
The version number of the Newstar executables is given by the
command
"what $n_exe/*.exe | grep %NST%"
When programmers check in their modified files, these files are copied into their local $n_import directory, together with a groupfile listing the files and executables. The same files are copied into the $n_import directory of the Export-Master. If the "nsh in" command is issued for (modified) files in a Master tree (presumably outside the NFRA), these files are not copied to the local $n_import. They are copied into the Export-Master. In all cases, a mail message describing the check-in is sent to $n_master (currently newstar@nfra.nl).
Testing should be completed before check-in! At the NFRA, an executable and/or ppd-file supplied by a programmer can at request be copied into $n_tst for testing by a broader user-group.
1e. If files originate from outside the NFRA, they should be received in $n_import of the NFRA-Master (more than one groupfile can be handled with a single update command):
>>> nup retrieve -import updxxx.grpAny file that has to be ftp'd with binary mode has to be retrieved separately.
2e. Perform some elementary checks:
If psc/pin/pef-files are changed, they should be checked against changes in keywords. If the meaning of existing keywords have been changed, the revision should be treated as a release (see the previous section).
If keywords have been removed from a psc file, it should be checked wether they have been removed from the programs as well. In such cases, special care should be taken that executables are being rebuilt synchroneously with the ppd-files. In general, removal of keywords is discouraged for revisions.
NOTE: The synchronisation of exe/ppd files is not optimal in the present structure for configuration management, but cannot be improved without structural changes in the coupling of exe and ppd files.
3e. Update libraries and executables for all architectures:
>>> nup build -Update updxxx.grp (on rzmws0) >>> nup build -Update updxxx.grp (on rzmws5)Any errors reported by the build command should be reported to and repaired by the programmer.
4e. At successful compilation, merge the files in the source tree:
>>> nup build -Update -T:none -Merge updxxx.grp (on rzmws0)The revision number will be automatically updated in version.idx.
The subject from the groupfile(s) is copied in the nnews.hlp file, and a message to be sent to local Friends of Newstar is composed (so enter useful information). Both files will be presented in the MicroEmacs editor (change buffers with ^X X command, exit with Esc Z). Comments concerning programmers only should be removed from nnews.hlp, or be prefixed by "System: ". This checking of nnews.hlp is very important since most users rely on this file for their information on changes. The message is kept in $n_import/message.RR.rr where RR is the new Release, rr the new revision of Newstar.
The message is not yet sent, this is done after the Export-Master has been
updated.
This is done with the "nup release" command, discussed later.
A mail is sent to $n_master to remind you of this revision after three days.
At remote sides, the groupfile will be deleted automatically after the next update.
<EM>Revision history not recorded</EM><P>
The description of previous releases should not be removed.
Add a hypertext link to this release at the beginning of the file.
The procedure to update the Export-Master is as follows:
>>> telnet wsrt00 >>> setenv n_remote rzmws0.nfra.nl newstar /newstar/master/src >>> nup updateThis will try to update the Westerbork installation from the NFRA-master. Any errors occurring here will almost certainly occur in other installtions as well, so they need to be repaired before the next step. Should you choose to make changes directly in the NFRA-Master, issue the following command at rzmws0 before trying to update wsrt00 again:
>>> nup check dThis will make a fresh version of the database. Once installation on wsrt00 is successfull, give the following command at rzmws0:
>>> nup releaseThis will rebuild the documentation, create a fresh database, pack all sources, libraries and executables and send them over to the Export-Master. It also tells the server program that files are waiting to be unpacked. Once this has been done, the message will be sent. In case more than one release has been pending, a fresh message for the most recent revision will be composed, containing all the subjects from previous revisions. The message will be sent to the Friends of Newstar. This relies on an elm alias friends_of_newstar.
>>> nup check dThis will build a fresh version of file database.idx The database will be updated for any direct changes in the Master (that is: without proper checkin through $n_import).
>>> nup saveThis will make a backup of the entire master tree (all files below $n_root). Three DAT tapes are used for the backups (cyclic use of Newstar_A, Newstar_B and Newstar_C). The save command will run in the background and notify by mail when it is ready.
The two most recent backups are stored at the Bank of Dwingeloo. Backups of the Export-Master are made as part of the Scissor backup routine.
After the backup, the following command should be entered:
>>> rm $n_exe/*.old $n_tst/*.oldThis will throw away old versions, which can be restored from the backup tape if necessary.
>>> nup clear -ConfirmThis will remove any obsolete files from the Master copy. See above for the deletion of obsolete groupfiles. Removing a file from a groupfile will cause a prompt for deletion here. In such cases, check wether the file has become really obsolete (e.g. by using a grep on the subroutine name). If the file has been accidentally removed from a groupfile, check out the groupfile, re-insert the file, check-in the groupfile and make a maintenance revision.
>>> nup check l (on rzmws0) >>> nup check l (on rzmws5)This will check the current libraries. If faults are reported, the libraries should be updated through the groupfile produced by the check command (instructions are given by the command). The name of the groupfile will be libyymmdd${n_arch}.grp.
The Master system has two binary trees:
Executable files are looked for in the current directory first, then in $n_uexe (if it exists) and finally in $n_exe.
A user may decide to do "setenv n_uexe $n_tst" to get access to test versions, and programmers will set $n_uexe to the binary tree of their user system.
Master (NFRA or elsewhere):
---------------------------
$n_root -+-- src = $n_src Source tree
|
+-- lib -+-- inc = $n_inc Precompiled files
| +-- sun = $n_lib Object libraries
| +-- hp
|
+-- exe -+-- sun = $n_exe Executables and ppd-files
| +-- hp
| |
| +-- html = $n_hlp Hypertext help files (elsewhere)
|
+-- hlp = $n_hlp Hypertext help files (at NFRA)
|
+-- tst -+-- sun = $n_tst Test versions
| +-- hp
|
+-- work -+-- sun = $n_work Intermediate files, files
| +-- hp necessary for debugging
|
+-- import = $n_import Import area for uploading of
revisions and programmers files.
Sites outside NFRA can have other architectures in $n_lib and $n_exe.
At most sites outside NFRA $n_hlp is a subdirectory of $n_root/exe.
This structure can be split over various filesystems. The tree can than
be realised through symbolic links. Since all system commands use the
environment variables this is not strictly necessary.
Additional directories at NFRA only:
|
+-- mongo Installation of mongo
+-- perl Executables for perl
+-- rmtd Sources for rmtd (VAX)
Possible files in $n_root: backups.txt Log of backups
updates.log Log of update-commands
updates.html Revision history (NFRA only)
Source tree:
$n_src -+-- sys Maintenance system
|
+-- doc -+- ... Documents
|
+-- wng Precompiler, files, I/O etc.
+-- dwarf Parameter interface
+-- n* The various programs
|
+-- data Calibrator models
|
+-- batch Standard batch procedures
Possible files in $n_src: upd*.log Compilation logs
User system:
------------
~programmer +... $n_uroot -+-- lib -+-- inc = $n_uinc
: | +-- sun = $n_ulib
: |
+-- exe -+-- sun = $n_uexe
|
+-- work = $n_work
|
[ +-- src = $n_usrc ]
:
+... Arbitrary directories with files
modified by the programmer.
Logging in as the owner of $n_root (Newstar manager) causes $n_u...
to point to $n_..., $n_work will point to $n_root/work/$n_arch.
If the Newstar manager uses the -NUpdate switch for update, $n_uexe
will be set to $n_tst, else it will be at $n_exe.
It is confusing that there is no $n_uwork, this departure from the
general practice may be removed in future.
Export Master (NFRA only, at ftp.nfra.nl):
------------------------------------------
/users/ftp/newstar = $n_root Newstar ftp area
/users/www/newstar = $n_www Newstar www area
/users/newstar Newstar server programs
$n_root -+-- src = $n_src Source tree
|
+-- import = $n_import Import area for uploading of
revisions and programmers files
(also from non-NFRA sites).
Files in $n_root:
nstar_src.tar.Z Archive with sources
nstar_src_??.tar Archive with additional sources
for various architectures
nstar_hlp.tar.Z Archive with documentation
nstar_exe_??.tar.Z Archive with executables for
various architectures
nstar_lib_??.tar.Z Archive with libraries for
various architectures
nstar_lib_inc.tar.Z Archive with include files
$n_www -+-- hlp Documentation (from nstar_hlp.tar)
|
+-- bug User Feedback System
|
+-- bin = $n_cgi httpd scripts for Newstar
| (sources in /users/newstar/src)
|
+-- mail Link to Newstar mail area for HjV
Files in $n_www:
homepage.html Homepage for the server area,
different from hlp/homepage.html
index.html Link to homepage.html, you get this
for http://www.nfra.nl/newstar/
example.* Some sample images
updates.html Revision history (from NFRA master)
/users/newstar -+-- src Server programs (sources)
+-- bin = $n_bin Server programs (executables)