CEMAG computer//Users environment
Last update 04-01-2009 06:18 pm / Jean-François RabasseBack to topic CEMAG computer
CEMAG Computer, users environment
Hardware configuration
The computer is made of five interconnected SGI Altix 450 machines. (See Architecture.)
The first machine, jxb, features 4 computing cores (2 Intel Montecito dual core processors). Only this machine is accessible to users for interactive tasks. (It owns logins, compilers licenses, etc.).
The other ones, jxb01 to jxb04, features 68 total cores (64 available to application runs) and are available only through the jobs queuing system. They don't accept logins.
A S330 disks enclosure is available to all the machines. The operating system is a SuSE Linux, similar to the one installed on our desktop PCs, with minor differences. (It is a SuSE Linux Enterprise Server, version 9, and it will be upgraded very soon to version 10.)
Computer access
The login machine, jxb, can be accessed only from the LRA local network. No route from the Internet, no non LRA access from the Physics Department network. External users will have to login first on one of our public servers, or a local machine.
NB: access to this computer relies on a time attribution scientific policy. So, don't contact me to ask for login access, get in touch with Michel and Steve.
Users accounts (login and password) for authorized users are those available on the LRA network. Only ssh, slogin tools are available, no telnet.
Users willing to start graphic tools, (e.g. the tasks manager xpbs) won't forget to activate X protocol routing when connecting :
slogin -X jxb
NB: the computing nodes, jxb01 to jxb04 are not connected to the laboratory network, don't try to login on them.
Disks resources
Users space
Each user owns a home directory on the login machine jxb. These directories are local to the host and different from our home directories as available on the local network. You will thus need to move yourself all what you need to work (use scp or rsync). For programs source files, a good idea will be to manage them via CVS; once connected on jxb, they will be loaded via cvs checkout commands.
These directories are provided to host standard Unix configuration files (.profile, .bash_login, .emac.d and others) and your parameters files, source code files, makefiles, miscellaneous scripts.
Never use them for data storage, runs outputs, scratch space. The total available space on the homes area (about 100 Gb) is fairly enough for everybody to work in comfortable conditions, provided it's not used as an archival space. Never forget that should this space be full, nobody could work any more. Tar and feathers for the guilty user !
Users quotas
Disk quotas have been setup on the /home area, the default is 2.5 Gb per user with a possible temporary overfill of 3.5 Gb for one week. You can check your disk usage with the quota command :
rabasse@jxb00:~> quota Disk quotas for user rabasse (uid 9043): Filesystem blocks quota limit grace files quota /dev/sda6 10912 2500000 3500000 412 0 rabasse@jxb00:~>
Data spaces
The disks space dedicated to data spreads on six filesystems, about 1.5 Tb each, named /jxb-data/01 to /jxb-data/06. This space will have to be mapped and managed, extension directories, projects directories with space sharing among users from a same project group, space allocation, etc.
For the starting phase, two data directories have been created for each account. E.g. user rabasse will own three directories, /home/rabasse, /jxb-data/01/rabasse and /jxb-data/04/rabasse.
Tasks submission
The PBS jobs manager is no longer used to process users requests. A specific tool, rplan, is available and provides allocations, tasks submissions and launch.
See the Tasks manager documentation.
Environment and tools
The working environment is standard Linux, each user will custom personal working configuration, aliases, etc.
Modules manager
The system comes with a tool, module, which is an environment configurator.
Say I'm looking for the Intel Fortran Compiler :
rabasse@jxb00:~> which ifort rabasse@jxb00:~>
it's not found !
I ask for the list of available modules :
rabasse@jxb00:~> module avail -------------- /usr/share/modules/modulefiles -------------- dot module-cvs mpich-ch-p4 intel-cc-9/9.0.031 module-info mpich-ch-p4mpd intel-fc-9/9.0.032 modules null rabasse@jxb00:~>
then load the required one :
rabasse@jxb00:~> module load intel-fc-9/9.0.032 rabasse@jxb00:~>
My environment is now properly configured :
rabasse@jxb00:~> which ifort /sw/sdev/intel-fc/9.0.032/bin/ifort rabasse@jxb00:~>
Always use the above procedure and resist to the temptation to search for tools by hand and hard-wire directories in your environment variables PATH, MANPATH, LD_LIBRARY_PATH and friends.
You would surely forget some ones, and should the system be upgraded and new versions be installed, it will be very difficult for you to select properly which compilers and/or libraries you want to work with.
Electronic mail
This machine is not supposed to support e-mail. Users tools, pine et al., are not installed. Applications that may happen to send you mails, e.g. the PBS tasks manager, will route it to the laboratory mail server. You'll recognize these mails on their specific header,
e.g. To: rabasse@jxb00.lra.ens.fr
Informations, documentations
Documentations
Useful documentations are to be made available online. As for now, are available :
Intel Compilers. These documentations are already installed on our intranet. The Itanium 64 version has been added.
PBSPro User's Guide. Presentation of Laurent Aumis on October 18th.
Altix development optimisation. Presentation of Gabriel Thabet on November 15th.
Bandela MPI communications analysis.
Other documentations will be on-lined after the next introduction sessions, as soon as available.
Miscellaneous documentations
MPI Subroutine Reference (CCR Jussieu)
Informations
A mailing list, cemag-comp, has been created. All Altix users are subscribed by default. It will be used to broadcast technical informations, new versions of compilers, system down for maintenance, etc.
It is not supposed to hold CEMAG scientific information. Don't use it neither to signal problems, but users tricks and all personal experimentations that can be useful to others will be welcome.
