October, 2002
NEC Corporation
Introduction
The aim of MathKeisan is to provide a highly tuned and well-tested collection of Math libraries for NEC high performance computers. This version is for the NEC and Cray SX-6 vector computer. See www.mathkeisan.com for other versions for the NEC SX-5, and the NEC Express 5800/1000 series running Linux (NEC's Itanium® Processor Family servers). Unless noted otherwise, all references to MathKeisan in these release notes are to MathKeisan for SX-6.
The libraries in MathKeisan are listed in Table 1.
Table 1: Libraries in MathKeisan
|
name |
description |
|---|---|
|
BLAS |
Basic Linear Algebra Subprograms |
|
LAPACK |
Linear algebra for high performance computers |
|
ScaLAPACK |
Scalable Linear Algebra package (contains PBLAS) |
|
BLACS |
Basic Linear Algebra Communication Subprograms |
|
PARBLAS |
Shared memory Parallel BLAS |
|
CBLAS |
C interface to BLAS |
|
SBLAS |
Sparse BLAS |
|
FFT |
FFT's with HP's VECLIB interface and CRAY LIBSCI 3.1 interface |
|
PARFFT |
Parallel FFT's with HP's VECLIB interface and CRAY LIBSCI 3.1 interface |
|
METIS |
Matrix/Graph ordering and partitioning library |
|
ParMETIS |
Parallel Matrix/Graph ordering and partition library |
|
SOLVER |
Direct solver for sparse symmetric systems |
|
ARPACK |
Solution of large scale eigenvalue problems |
Compatibility
MathKeisan 1.4.0 is compatible with SUPER-UX R12.1 and FORTRAN90/SX Rev.265 or later. The following compilers were used to generate the MathKeisan libraries
| Fortran | f90 for SX, Rev.253 |
| C | C++/SX, Rev.050 |
| MPI | r121-v640 |
New in MathKeisan 1.4.0
- size_t64 versions of all libraries
- performance and memory improvements for solver
- performance improvements for 2D FFT subroutines SCFFT2D, CSFFT2D, DZFFT2D, ZDFFT2D
- improvements for 3D FFT subroutines CSFFT3D, ZDFFT3D, and improvements in FFT man pages
- iterative refinement added to solver
- C code compiled with C++/SX compiler
Loading
If you are using the F90 flag -dw (the default), load with the libraries in table 2. Load libraries in the order given, or use the ld flag -h lib_cyclic. Table 4 has $(LIBDIR) for either size_t32 or size_t64 on self or cross compile machines. These are the default directories, on your machine the MathKeisan libraries may be installed elsewhere. Consult your system administrator if the libraries are not in the default directories.
Table 2: Loading for -dw
|
name |
load libries (see Table 4 for $(LIBDIR)) |
|---|---|
|
BLAS |
-L$(LIBDIR) -lblas |
|
LAPACK |
-L$(LIBDIR) -llapack -lblas |
|
ScaLAPACK |
-L$(LIBDIR) -lscalapack -lblacsF90init -lblacs -lblacsF90init -lblas -lmpi |
|
BLACS |
-L$(LIBDIR) -lblacsF90init -lblacs -lblacsF90init -lmpi |
|
PARBLAS |
-L$(LIBDIR) -lparblas -Popenmp |
|
CBLAS |
-L$(LIBDIR) -lcblas -lblas |
|
SBLAS |
-L$(LIBDIR) -lsblas |
|
FFT |
-L$(LIBDIR) -lfft |
|
PARFFT |
-L$(LIBDIR) -lparfft -Popenmp |
|
METIS |
-L$(LIBDIR) -lmetis_32 |
|
|
-L$(LIBDIR) -lmetis |
|
ParMETIS |
-L$(LIBDIR) -lparmetis_32 -lmpi |
|
|
-L$(LIBDIR) -lparmetis -lmpi |
|
SOLVER |
-L$(LIBDIR) -lsolver -lmetis -lblas -Popenmp |
|
ARPACK |
-L$(LIBDIR) -larpack -llapack -lblas |
If you are using the f90 flag -ew, load with the libraries in Table 3. Load libraries in the order given, else use the ld flag -h lib_cyclic. For information on $(LIBDIR) please see Table 4.
Table 3: Loading for -ew
|
name |
load libraries (see Table 4 for $(LIBDIR)) |
|---|---|
|
BLAS |
-L$(LIBDIR) -lblas_64 |
|
LAPACK |
-L$(LIBDIR) -llapack_64 -lblas_64 |
|
ScaLAPACK |
not available |
|
BLACS |
not available |
|
PARBLAS |
-L$(LIBDIR) -lparblas_64 -Popenmp |
|
CBLAS |
not available |
|
SBLAS |
-L$(LIBDIR) -lsblas_64 |
|
FFT |
-L$(LIBDIR) -lfft_64 |
|
PARFFT |
-L$(LIBDIR) -lparfft_64 -Popenmp |
|
METIS |
-L$(LIBDIR) -lmetis_64 |
|
ParMETIS |
-L$(LIBDIR) -lparmetis_64 -lmpiw |
|
SOLVER |
-L$(LIBDIR) -lsolver_64 -lmetis_64 -lblas_64 -Popenmp |
|
ARPACK |
-L$(LIBDIR) -larpack_64 -llapack_64 -lblas_64 |
Table 4: default $(LIBDIR)
| machine | F90 or C++ load flags | default $(LIBDIR) | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| F90 | C++ | self compile | -size_t32 (default) | -Nover2g (default) | $(LIBDIR) = /usr/lib | -size_t64 | -over2g | $(LIBDIR) = /usr/lib/lib64 | cross compile | -size_t32 (default) | -Nover2g (default) | $(LIBDIR) = /SX/usr/lib | -size_t64 | -over2g | $(LIBDIR) = /SX/usr/lib/lib64 |
Data types
The data types for MathKeisan library files are listed in Tables 5 and 6.
Table 5: Data types for MathKeisan library files
|
|
Integer and floating point data type |
|
|---|---|---|
|
name |
I32R32+I32R64 |
I64R64+I64R64 |
|
BLAS |
libblas.a |
libblas_64.a |
|
LAPACK |
liblapack.a |
liblapack_64.a |
|
ScaLAPACK |
libscalapack.a |
not available |
|
BLACS |
libblacs.a |
not available |
|
PARBLAS |
libparblas.a |
libparblas_64.a |
|
CBLAS |
libcblas.a |
not available |
|
SBLAS |
libsblas.a |
libsblas_64.a |
|
FFT |
libfft.a |
libfft_64.a |
|
PARFFT |
libparfft.a |
libparfft_64.a |
|
ARPACK |
libarpack.a |
libarpack_64.a |
Table 6: Data types for MathKeisan library files
|
|
Integer and floating point data type |
||
|---|---|---|---|
|
name |
I32R32 |
I32R64 |
I64R64 |
|
METIS |
libmetis_32.a |
libmetis.a |
libmetis_64.a |
|
ParMETIS |
libparmetis_32.a |
libparmetis.a |
libparmetis_64.a |
|
SOLVER |
not available |
libsolver.a |
libsolver_64.a |
Files in column I32R32 + I32R64 of Table 5 are for 32 bit integer data type (Fortran integer*4). The floating point data type is determined by the first letter of the subroutine or function name as follows
- s - 32 bit real (Fortran real*4)
- d - 64 bit real (Fortran real*8)
- c - 32 bit complex (Fortran complex*8)
- z - 64 bit complex (Fortran complex*16)
Code compiled with the f90 default -dw should be linked to these files.
Files in column I64R64+I64R64 have 64 bit integer and floating point data type. Subroutine and function names still have first letter s,d,c,or z, but all are for 64 bit integer and floating point data type. Code compiled with the f90 flag -ew should be linked to these files.
In Table 6, files have data type indicated by the column name, for example, column name I32R32 for 32 bit integer 32 bit real. If you are compiling with the f90 default flag –dw, link to the I32R32 file if your reals are 32 bit, or link to the I32R64 file if your reals are 64 bit. If you are compiling with the f90 flag –ew, link to the I64R64 libraries.
Man pages
- MathKeisan includes man pages. There is a man page for each library, and the following libraries have man pages for individual subroutines: BLAS, LAPACK, ScaLAPACK, FFT, SOLVER, SBLAS, ARPACK. As an example, to view the man page for BLAS, type "man blas", to view the man page for the BLAS subroutine dgemm, type "man dgemm".
Installation instructions
- MathKeisan is supplied as a gzip tar file. The gzip tar file contains
- Lib files
- Include files
- Man page files
- Install script
- User's Guide
- Release notes
- To install MathKeisan, gunzip, extract the tar file, and use the install script install.sh. The steps taken by install.sh are as follows
- Prompts for directories to install lib files, include files, man page files, and User's Guide files. Prompts for a directory to backup files that will be over written
- Finds space required
- Files in the lib, include, man page directories that will be over written are moved to the backup directory
- MathKeisan lib, include, man page, and User's Guide files are copied to the directories supplied in 1.
- An uninstall script and log file are created
- Running the uninstall script deletes the lib and man files installed in 4, and it restores files backed up in 3. Do not run the uninstall script unless you want to uninstall MathKeisan.
- A log file is produced by the install script. Please e-mail it to technical@atcc.necsyl.com. When the log file is received, you will be given access to the secure download site at www.mathkeisan.com. Patches are posted at the secure download site.
- Below is sample output from install.sh
MathKeisan Installation ...
MathKeisan contains 2109 man-page files
MathKeisan contains 1 include files
MathKeisan contains 15 English Users Guide files
MathKeisan contains 15 Japanese Users Guide files
MathKeisan contains the following size_t32 library files:
libarpack_64.a libarpack.a libblacs.a libblacsCinit.a
libblacsF90init.a libblas_64.a libblas.a libcblas.a
libfft_64.a libfft.a liblapack_64.a liblapack.a
libmetis_32.a libmetis_64.a libmetis.a libparblas_64.a
libparblas.a libparfft_64.a libparfft.a libparmetis_32.a
libparmetis_64.a libparmetis.a libsblas_64.a libsblas.a
libscalapack.a libsolver_64.a libsolver.a
MathKeisan contains the following size_t64 library files:
libarpack_64.a libarpack.a libblacs.a libblacsCinit.a
libblacsF90init.a libblas_64.a libblas.a libcblas.a
libfft_64.a libfft.a liblapack_64.a liblapack.a
libmetis_32.a libmetis_64.a libmetis.a libparblas_64.a
libparblas.a libparfft_64.a libparfft.a libparmetis_32.a
libparmetis_64.a libparmetis.a libsblas_64.a libsblas.a
libscalapack.a libsolver_64.a libsolver.a
Press RETURN to continue
This install script will do the following:
1. Copy MathKeisan library, man page, include, English User's Guide and
Japanese User's Guide files to the specified directories, and
give them the correct permission
2. Any file that is over written will be backed up in the backup
directory
3. An uninstall script is created and placed in the backup directory
Below are default directories for install and backup:
libraries (size_t32) /SX/usr/lib/
libraries (size_t64) /SX/usr/lib/lib64/
man-pages /SX/usr/man/manl/
include /SX/usr/include
English User's Guide /SX/usr/opt/MathKeisan/MK1_3_0/UsersGuide/E
Japanese User's Guide /SX/usr/opt/MathKeisan/MK1_3_0/UsersGuide/J
backup /tmp/mpack/MK1.4.0/bkpdir
The directories must exist, and you must have write permission.
The backup directory must be empty.
Is directory for installing size_t32 (the default) libraries correct y/n:
n
enter directory for size_t32 libraries
/tmp/mpack/lib
Is directory for installing size_t64 libraries correct y/n:
n
enter directory for size_t64 libraries
/tmp/mpack/lib/lib64
Is directory for installing man-pages correct y/n:
n
enter directory for man-pages
/tmp/mpack/man
Is directory for installing include files correct y/n:
n
enter directory for include files
/tmp/mpack/include
Is directory for installing English User's Guide correct y/n:
n
enter directory for English User's Guide
/tmp/mpack/EUG
Is directory for installing Japanese User's Guide correct y/n:
n
enter directory for Japanese User's Guide
/tmp/mpack/JUG
Is directory for backup correct y/n:
y
Below are chosen directories:
libraries (size_t32) /tmp/mpack/lib
libraries (size_t64) /tmp/mpack/lib/lib64
man-pages /tmp/mpack/man
include files /tmp/mpack/include
English User's Guide /tmp/mpack/EUG
Japanese User's Guide /tmp/mpack/JUG
backup /tmp/mpack/MK1.4.0/bkpdir
*** above directories exist and have correct attributes.
*** Do you want to use these directories y/n
y
finding space required for install
space required for new size_t32,64 library files = 171375860 byte
space required for new man-page files = 10617681 byte
space required for new include files = 32428 byte
space required for new English User's Guide files = 232900 byte
space required for new Japanese User's Guide files = 428778 byte
space required for backup of files over written = 0 byte
#-----------------------------------------------------------------------------#
| IMPORTANT!! only continue if you have enough space in chosen directories to |
| 1. install new library, man-page, include, English User's Guide and |
| Japanese User's Guide files |
| 2. make backup of any file over written |
#-----------------------------------------------------------------------------#
Do you want to continue y/n:
y
backing up and installing libraries
backing up and installing man pages
backing up and installing include files
backing up and installing English Users Guide
backing up and installing Japanese Users Guide
writing uninstall script
#--------------------------------------#
| Install is complete |
#--------------------------------------#
1. A log file for this install is in
/tmp/mpack/MK1.4.0/install.log8814
please e-mail a copy of this file to technical@atcc.necsyl.com.
It will be used to debug any future problems. When your log file
is received, a login and password will be issued so you can
access MathKeisan patches that are posted at www.mathkeisan.com.
2. To uninstall, save directory below, and its contents
/tmp/mpack/MK1.4.0/bkpdir
An uninstall script is in
/tmp/mpack/MK1.4.0/bkpdir/uninstall.sh
Running the uninstall script will delete the lib and man pages
that this script installed, and it will restore all files over-
written. Do not run the uninstall script unless you want to
uninstall MathKeisan
Product Description
Below are notes on each of the libraries in MathKeisan
BLAS
The BLAS (Basic Linear Algebra Subprograms) are high quality "building block" routines for performing basic vector and matrix operations. Level 1 BLAS are for vector-vector operations, Level 2 BLAS are for matrix-vector operations, and Level 3 BLAS are for matrix-matrix operations. Because the BLAS are efficient, portable, and widely available, they're commonly used in the development of high quality linear algebra software, LAPACK and ScaLAPACK for example.
The BLAS included in MathKeisan is based on the original version of BLAS which was developed by J.J. Dongarra (Argonne National Lab.), J. Du Croz (Numerical Algorithms Group Ltd.), I. S. Duff (AERE Harwell), S. Hammarling (Numerical Algorithms Group Ltd.), R. J. Hanson (Sandia National Lab.), D. Kincaid (University of Texas), F.T. Krogh, C.L. Lawson (Jet Propulsion Lab.).
PARBLAS
PARBLAS contains shared memory parallel versions of the BLAS level 2 and level 3 subroutines. The level 1 subroutines are serial, or single processor, the same as in BLAS. These subroutine have the same interface as BLAS. The number of parallel threads is specified by calling the OMP function OMP_SET_NUM_THREADS(n), where n is the number of parallel threads, or by setting the environment variable OMP_NUM_THREADS. For C shell use 'setenv OMP_NUM_THREADS n'. For Bourne shell, use 'export OMP_NUM_THREADS ; OMP_NUM_THREADS=n'. More information on setting the number of threads is in the FORTRAN90/SX Multitasking User's Guide.
The shared memory parallel BLAS included in MathKeisan is based on the original version of BLAS which was developed by J.J. Dongarra (Argonne National Lab.), J. Du Croz (Numerical Algorithms Group Ltd.), I. S. Duff (AERE Harwell), S. Hammarling (Numerical Algorithms Group Ltd.), R. J. Hanson (Sandia National Lab.), D. Kincaid (University of Texas), F.T. Krogh, C.L. Lawson (Jet Propulsion Lab.).
CBLAS
BLAS is a C language interface to the FORTRAN BLAS, a set of subroutines used to perform vector-vector(level1), matrix-vector(level2), and matrix-matrix(level3) operations.
The CBLAS is based on the BLAS Technical Forum reference implementation by K. Teranishi (University of Tennessee) with updates by J. Horner (University of Tennessee). The specification was authored by R. Whaley (University of Tennessee).
SBLAS
Sparse BLAS is a set of subroutines used to perform sparse BLAS operations. The Sparse BLAS are based on ACM Algorithm 692 by D.S.Dodson (Convex), R.G.Grimes and J.G.Lewis (Boeing).
LAPACK
LAPACK (Linear Algebra PACKage) provides routines for solving systems of simultaneous linear equations, least-squares solutions of linear systems of equations, eigenvalue problems, and singular value problems. The associated matrix factorizations (LU, Cholesky, QR, SVD, Schur, generalized Schur) are also provided, as are related computations such as reordering of the Schur factorizations and estimating condition numbers. Dense and banded matrices are handled, but not general sparse matrices. In all areas, similar functionality is provided for real and complex matrices, in both single and double precision.
LAPACK supersedes LINPACK and EISPACK. On shared memory vector and parallel processors LINPACK and EISPACK are inefficient because their memory access patterns disregard the multi-layered memory hierarchies of the machines, thereby spending too much time moving data instead of doing useful floating-point operations. LAPACK addresses this problem by reorganizing the algorithms to use block matrix operations, such as matrix multiplication, in the innermost loops. Whenever possible, LAPACK calls BLAS (usually level 2 & level 3). Because of the coarse granularity of the level 3 BLAS operations, their use promotes high efficiency.
The LAPACK included in MathKeisan is based on the original version of LAPACK version 3.0 which was developed by the LAPACK project team which was composed ofE. Anderson (University of Tennessee, Knoxville), Z. Bai (University of Kentucky and University of California, Davis), C. Bischof (Institute for Scientific Computing, Technical University Aachen, Germany), S. Blackford (University of Tennessee, Knoxville), J. Demmel (University of California, Berkeley), J. Dongarra (University of Tennessee, Knoxville, and Oak Ridge National Lab.), J. Du Croz (Numerical Algorithms Group Ltd.), A. Greenbaum (University of Washington), S. Hammarling (Numerical Algorithms Group Ltd.), A. McKenney, D. Sorensen (Rice University)
BLACS
The BLACS (Basic Linear Algebra Communication Subprograms) are a message-passing library designed for linear algebra. The computational model consists of a one-or two-dimensional process grid, where each process stores pieces of the matrices and vectors. The BLACS include synchronous send/receive routines to communicate a matrix or submatrix from one process to another, to broadcast submatrices to many processes, or to compute global data reductions (sums, maxima and minima). There are also routines to construct, change, or query the process grid. Since several ScaLAPACK algorithms require broadcasts or reductions among different subsets of processes, the BLACS permit a process to be a member of several overlapping or disjoint process grids, each one labeled by a context. In MPI this is called a communicator. The BLACS provide facilities for safe inter-operation of system contexts and BLACS contexts.
The BLACS included in MathKeisan is the original version 1.1 with patch03 written by J.J. Dongarra, and R.C. Whaley (University of Tennessee, Knoxville).
ScaLAPACK
ScaLAPACK is a library of high-performance linear algebra routines for distributed-memory message passing computers. ScaLAPACK can solve systems of linear equations, linear least squares problems, eigenvalue problems, and singular value problems. ScaLAPACK can also handle many associated computations such as matrix factorization or estimating condition numbers. Dense and band matrices are provided for, but not general sparse matrices. Similar functionality is provided for real and complex matrices. The name ScaLAPACK is an acronym for Scalable Linear Algebra PACKage, or Scalable LAPACK.
As in LAPACK, the ScaLAPACK routines are based on block-partitioned algorithms in order to minimize the frequency of data movement between different levels of the memory hierarchy. The fundamental building block of the ScaLAPACK library is a distributed memory version of the Level 1, 2, and 3 BLAS, called PBLAS (Parallel BLAS). The PBLAS are in turn built on the BLAS for computation on a single node and on a set of Basic Linear Algebra Communication Subprograms (BLACS). PBLAS is contained in the ScaLAPACK library as an integral part of the ScaLAPACK library.
The ScaLAPACK included in MathKeisan is the original version 1.6 written by L . S. Blackford (University of Tennessee, Knoxville), J. Choi (Soongsil University, Korea), A. Cleary (Lawrence Livermore National Lab.), E. D'Azevedo (Oak Ridge National Lab.), J. Demmel (University of California, Berkeley), I. Dhillon (University of California, Berkeley), J. Dongarra (University of Tennessee, Knoxville, and Oak Ridge National Lab.), S. Hammarling (Numerical Algorithms Group Ltd.), G. Henry (Intel Corporation), A. Petitet (University of Tennessee, Knoxville), K. Stanley (University of California, Berkeley), D. Walker (University of Wales, Cardiff), R. C. Whaley (University of Tennessee, Knoxville)
FFT
The Fast Fourier Transforms (FFTs) contained in MathKeisan have equivalent interface and functionality to HP's VECLIB Library and also CRAY's LIBSCI 3.1. There are 1D,2D,3D and simultaneous 1D Complex-Complex FFT's, Real-Complex FFT's and Complex-Real FFT's.
The FFT libraries were developed internally at NEC.
PARFFT
OpenMP parallel FFTs with the same functionality as for FFT above. The number of parallel threads is specified by calling the OMP function OMP_SET_NUM_THREADS(n), where n is the number of parallel threads, or by setting the environment variable OMP_NUM_THREADS. For C shell use 'setenv OMP_NUM_THREADS n'. For Bourne shell, use 'export OMP_NUM_THREADS ; OMP_NUM_THREADS=n'. More information on setting the number of threads is in the FORTRAN90/SX Multitasking User's Guide.
ARPACK
ARPACK is a collection of Fortran 77 subroutines designed to solve large-scale eigenvalue problems. ARPACK stands for ARnoldi PACKage. It is capable of solving large-scale symmetric(Hermitian), non-symmetric (non-Hermitian), standard, or generalized eigenvalue problems from significant application areas. The ARPACK library is designed to compute a few, say k, eigenvalues with user-specified features such as those of largest real part or largest magnitude using n*O(k) + O(k*k) storage. No auxiliary storage is required. A set of Schur basis vectors for the desired k-dimensional eigenspace is computed which is numerically orthogonal to working precision. Eigenvectors are also available upon request. ARPACK is dependent upon a number of subroutines from LAPACK and BLAS. The performance scales asymptotically to the Level 2 BLAS operation GEMV.
The ARPACK included in MathKeisan is based on the original version written by Rich Lehoucq, Kristi Maschhoff, Danny Sorensen and Chao Yang (Rice University).
METIS
METIS is a library for partitioning and ordering matrices/graphs. It is used by SOLVER to order the original matrix to reduce fill-ins in the factored matrix.
The METIS in MathKeisan is the original version 4.0 developed at University of Minnesota and Army HPC research center by George Karypis and Vipin Kumar.
----------------------------------------------------------------
"This software package includes/uses METIS, developed by George
Karypis and Vipin Kumar at the University of Minnesota.
Additional information about METIS can be found at
http://www.cs.umn.edu/~karypis/metis
METIS is Copyright 1997 Regents of the University of Minnesota.
Twin Cities. All Rights Reserved.
----------------------------------------------------------------
ParMETIS
PARMETIS is an MPI-based parallel library that implements a variety of algorithms for partitioning unstructured graphs and for computing fill-reducing orderings of sparse matrices.
The PARMETIS in MathKeisan is the original version 2.0 developed at University of Minnesota and Army HPC research center by George Karypis and Vipin Kumar.
SOLVER
SOLVER contains subroutines used to solve sparse symmetric linear systems. It uses the left-looking algorithm to factor a sparse matrix A into A = L D L T, where L is lower triangular with unit diagonal and D is diagonal. It takes advantage of the supernodal structure of the matrix. The current version uses the METIS library to order the matrix. Both serial and parallel numerical factorization are supported. The number of parallel threads is specified by calling the OMP function OMP_SET_NUM_THREADS(n), where n is the number of parallel threads, or by setting the environment variable OMP_NUM_THREADS. For C shell use 'setenv OMP_NUM_THREADS n'. For Bourne shell, use 'export OMP_NUM_THREADS ; OMP_NUM_THREADS=n'. More information on setting the number of threads is in the FORTRAN90/SX Multitasking User's Guide.
Auxiliary Subroutine
A subroutine mkversion in libblas.a and libblas_64.a outputs MathKeisan version information to standard output. In Fortran use "call mkversion()", in C use "mkversion_()". In both cases link with f90 to the library libblas.a or libblas_64. See also the mkversion man page. Output for MathKeisan 1.4.0 is below.
MathKeisan 1.4.0 BLAS - legacy blas LAPACK - version 3.0 + UPDATES ScaLAPACK - version 1.7 BLACS - version 1.1 + patch 03 METIS - version 4.0 PARMETIS - version 2.0