glite 26.07.2006d. zaborov, sa1-sa3 meeting, dubna1 dmitry zaborov (sinp-msu-moscow) dubna, jul 26,...

26.07.2006 D. Zaborov, SA1-SA3 meeting, Dubna 1

gLite

Dmitry Zaborov (SINP-MSU-Moscow)

Dubna, Jul 26, 2007

Testing 3.0

under SLC 4


gLite Overview

• Testing gLite 3.0 UI– A test suite has been developed specifically

to test gLite User Interface– Certain tests can also be used as functional

tests of local site/the whole GRID– Test gLite 3.0 under Scientific Linux CERN 4– Investigating the problems found

• Integration of SRM test suites into SAM– Two test suites previously existed for the

Storage Resource Manager protocol: LCG-DM tests and s2 srm tests

– Needed to extract the desired tests, recompile, etc.

– Wrapper scripts for SAM framework are developed


gLite Specificity of testing User Interface

• The aim of User Interface machine is to be an access point to the GRID services– No GRID services are running on a UI– Only client software– Not possible to do a complete test of a UI

without employing various GRID services

• User Interface installations may differ in operating system flavor, default user shell, installation directory, system libraries, etc.– Special effort should be made to ensure

integrity of the installation, ability of the software tools to load, correctness of shell environment, etc.

• No test suite for UI existed before


gLite The concept

• Usually GRID user works in command line All our tests only use command line. In a sense it is a CLI

(command line interface) test suite C/C++, java, python, … interfaces are to be tested with

other, dedicated, test suites Graphical user tools should be tested separately too

• Command line tools can be tested with a shell script Organize the test suite as a collection of shell scripts

(easy to write and read the code) It should be possible to run each test separately from the

others (without a framework) Test scripts should be easy to run, with minimum number

of command line options and no special parameter files

• Client tools do not change in time No need to integrate UI tests into an automated testing

framework. It is sufficient to re-run the tests after a software update.


gLite What to test?

• Installation integrity• Shell environment• System time synchronization (ntp)• Ability of executables to load

(implemented as “version tests”)• Basic test of the most “popular”

commands– proxy and myproxy commands– job management commands– Information system commands– LCG file catalog commands– LCG data management commands


gLite bash? tcsh?

• All GRID commands should be findable and functional regardless what shell is usedTest the two popular shell flavors: bash and tcsh

• How to make the same test for two shells?Write two copies of the test: one in bash and one

in tcsh – adopted for the shell environment testsWrite the script in such a way that it can be

understood by both shells (“multishell script”) - adopted for the “version” tests

Write tests in the simplest version of bash, “sh” – use the fact that sh does not read any startup files or manipulate shell environment – run the script twice: from tcsh and from bash – solution adopted for many of the tests


gLite Shell environment tests

• System environment: PATH, LD_LIBRARY_PATH, PYTHONPATH, …– Look for non-existing directories

• gLite-specific environment: GLITE_LOCATION, etc.– Look for undefined variables, non-existing

directories


gLite Installation integrity

• All commands mentioned in the gLite User Guide (about 100) should exist

• All libraries mentioned in the gLite User Guide (currently only a few) should exist

• A man page should exist for each command


gLite Ability to load

• The commands should return zero exit code, i.e. no error, when it is asked for its version (--version, -version, -V options)– These tests can reveal installation problems

such as, for example, python version conflict, absence or version mismatch of a system library, etc.


gLite NTP test

• Amazingly often problems in GRID are caused by a wrong computer clock

• It is not a waste of time to check the clock

• This is the only service running on UI machine to be tested


gLite Test security-related commands

• grid-cert-info with various options• grid-proxy-info with various options• grid-proxy-init/info/destroy chain• voms-proxy-info with various options• voms-proxy-init/info/destroy chain• ensure that voms-proxy-init really

uses the file given with the -userconf• myproxy-init/info/destroy command

chain


gLite Test Job Management commands

• edg-job-list-match• edg-submit/get status/get output • glite-job-list-match• glite-submit/get status/get output • glite-wms-job-list-match test• glite-wms-submit/get status/get output • glite-wms-wait-output.sh - delegate

proxy, submit/get status/get output test for the gLite WMS job submission system


gLite Test information system commands

• Get info about CEs using lcg-info• Get info about SEs using lcg-info• Get info about sites using lcg-

infosites• Get info from local GIIS using

ldapsearch


gLite Test LCG File Catalog commands

• Basic test of lfc-ls• Create a directory in LFC, list it and

remove• Create a directory in LFC, make a

symbolic link to it and clean up• Create a directory in LFC, set its

comment, list, delete comment, delete the directory

• Create a directory in LFC, list ACL, modify ACL, list ACL, delete directory


gLite Test LCG data management commands

• Create and register, and then remove, a GRID file.

• Create and register, copy back and remove a GRID file.

• Create and register, list replica (SURL), get GUID and TURL for the replica, and then delete a GRID file.

• Create and register, list aliases, create new alias, list again, remove alias, list once again and remove a GRID file.


gLite Source code and documentation

• UI package: org.glite.testsuites.ctb/UI– “Source” code: http://glite.cvs.cern.ch/cgi-

bin/glite.cgi/org.glite.testsuites.ctb/UI/tests/

• TWiki page – https://twiki.cern.ch/twiki/bin/view/LCG/

SAMTests#User_Interface_UI


gLite Employing the package for real work

• First the package was used to test new versions of gLite TarUI (tarball version of gLite UI usually installed on computer clusters)

• Also lxplus installation of gLite UI was tested

• A number of bugs and other issues have been found using this package

• Since then the package is used to test new distributions of gLite UI (before releasing)

• The package may also be helpful for GRID system administrators


gLite Problems with gLite 3.0 under SL(C) 4

• Python 2.2 vs. 2.3 version conflict (see next slide)– python is a scripting language– Tkinter (a graphics library for python) is also

affected

• Under SL(C)4 certain system libraries (e.g. libssl, libcurl) have different versions than certain gLite components ask for– The solution is to update the gLite components

• No major issues prevent to port gLite to SLC4– After a little modification a gLite 3.0 UI compiled

for SLC3 could correctly run under SLC4


gLite Problems with Python

• Python 2.2 installed with gLite 3.0 conflicts with python 2.3 installed with SL4– The problem is due to mixing of two python

versions (depends on how $PATH is defined!)– Most python scripts can be interpreted by any of

the two versions of python but fail with a mixture– The solution is to remove old python

• Certain python scripts and modules explicitly ask for python 2.2, mention python 2.2 directories etc.– The solution is to update the scripts

• A similar conflict was observed with Tkinter– The solution is to update or remove tkinter from

gLite distribution


gLiteIntegrating SRM 2.2 test suites into

SAM

• Two test suites for SRM 2.2 protocol have been integrated into the SAM testing framework– LCG-DM SRM 2.2 test suite– S-2 SRM 2.2 test suite


gLite What is SRM?

• The SRM protocol is developed as the main protocol of managing storage resources in GRID– SRM stands for Storage Resource Manager– Each storage resource in the GRID should

exhibit an SRM service– SRM protocol defines methods to:

• Bring a file into SRM’s cache• Reserve space before putting a file• Release space• Request a file transfer between two storage

elements• Get status of a request• etc.

– See http://sdm.lbl.gov/srm-wg/ for more information


gLite What is SAM?

• SAM is a testing and monitoring framework• SAM framework consists of two parts:

database server and client part– The server part is responsible for collecting and

keeping test results and presenting it in a human-readable form (usually on a web-page)

– The client part is responsible for collecting exit codes and text output (in html format) of test scripts and publishing the results to the server

• A test application is compatible with SAM if:– Its exit codes comply with SAM standards– Its text output is compatible with html– It is able to work from current directory (changing

directory disrupts writing log-files by SAM)– It comes with a (very short) definition file

• A SAM application can be written in any programming or scripting language


gLite LCG-DM SRM test suite

• Within gLite the SRM protocol is defined in the LCG Data Management (LCG-DM) package

• The LCG-DM package has a test suite for each of its components, including SRM

• Our task was to integrate the test suite for SRM 2.2 into SAM

• This is done by a wrapper script which – determines the SRM endpoint for an SE host

using the information system, – launches the test suite – and puts some html tags into the output

stream


gLite S-2 SRM test suite

• There is another test suite for SRM 2.2, written in s-2 programming language– The test suite has been integrated

into SAM by means of a wrapper script

– The wrapper interrogates the information system to determine the SRM endpoint giving a SE hostname, launches the s-2 tests and collects the results in a single html file


gLite Further information

• See also the work report at http://zaborov.web.cern.ch/zaborov/gLite/Report-Jan-Mar-2007.pdf


gLite Conclusion

• A test suite for gLite 3 User Interface has been developed and allows to– test shell environment– test for missing/non-functional

commands/libraries– test basic functionality of the most popular

commands

• gLite 3.0 UI has been tested under Scientific Linux CERN 4.4– The main problem was identified as conflict

between python 2.2 and 2.3– A few minor issues and a number of bugs found

• Two test suites, dedicated to SRM 2.2, have been integrated into SAM testing framework


gLite Acknowledgement

This work was supported by CERN-INTAS grant 05-103-7649

glite 26.07.2006d. zaborov, sa1-sa3 meeting, dubna1 dmitry zaborov (sinp-msu-moscow) dubna, jul 26,...

Documents