EGEEIII - NA4 SEE Regional Support HG Application Support

Παρουσίαση με θέμα: "EGEEIII - NA4 SEE Regional Support HG Application Support"— Μεταγράφημα παρουσίασης:

1 EGEEIII - NA4 SEE Regional Support HG Application Support
IASA Marios Chatziangelou NA4 - SEE RS Coordinator

2 Organization & Statistics
Προχωρημένα Θέματα σε Τεχνολογίες Πλέγματος 23 – 24 February 2009

3 NA4 Activity Goals and Organisation
User Community Support & Expansion Ensure that Grid technology and the EGEE infrastructure are used successfully by a broad spectrum of scientific collaborations. Motivation: General user and virtual organization support. Work directly with strategic disciplines. Build new and strengthen existing user communities. Strategic Discipline Clusters High Energy Physics Life Sciences Earth Science Grid Observatory Computational Chemistry Astronomy & Astrophysics Fusion Support Virtual Organization Support Application Porting Support Direct User Support NA4 Management Activity Coordination Regional Participation NA4 activity has been an integral part of EGEE since its first phase. The focus of the activity has always been the end user communities and their support in high level (application porting, resource exploitation, VO creation and resource allocation etc). In EGEE-III the focus of the activity has slightly shifted from being the activity of user communities to being the activity providing services for the user communities. This is reflected in the overall structure of the activity, the formulization of Scientific Clusters and the introduction of three specialized support teams: APS, VO and Direct User Support. Purpose of NA4: To have happy users  Προχωρημένα Θέματα σε Τεχνολογίες Πλέγματος 23 – 24 February 2009 3 3

4 Support Teams Direct User Support Support unit of GGUS system.
Focus on documentation development, use cases and documentation improvement. VO Support Provided through the VO Managers’ Group Registration, Treatment of VO issues Application Porting Support ( Provides assistance for new and experienced grid users who wish to port applications to the EGEE grid.  Members of the team work closely with application owners to identify suitable approaches and tools for the porting process. Regional NA4 Teams ( One team per EGEE region (total 10 teams) Responsible for first level support and signposting Ensure daily interactions with grid are effective. Προχωρημένα Θέματα σε Τεχνολογίες Πλέγματος 4 23 – 24 February 2009 4

5 Regional Support/Participation Team
Team Members Marios Chatziangelou, IASA/GRNET Slobodan Vrhovac, GRNET Nikolas Stylianides, UCY Harald Gjermundrod, UCY Marios Dikaiakos, UCY Countries supported by the team. Bulgaria, Cyprus, Greece, Israel, Romania, Serbia Region’s web-site: SEE official web-site: Προχωρημένα Θέματα σε Τεχνολογίες Πλέγματος 23 – 24 February 2009

6 Ported Applications Statistics - EGEE
61 registered applications, until now Προχωρημένα Θέματα σε Τεχνολογίες Πλέγματος 23 – 24 February 2009

7 Ported Applications Statistics - SEE
30 registered applications, until now Προχωρημένα Θέματα σε Τεχνολογίες Πλέγματος 23 – 24 February 2009

8 Ported Applications Statistics - HellasGrid
20 registered applications, until now Προχωρημένα Θέματα σε Τεχνολογίες Πλέγματος 23 – 24 February 2009

9 Available S/W and Technical Tips
Προχωρημένα Θέματα σε Τεχνολογίες Πλέγματος 23 – 24 February 2009

10 HG-Site-wide & VO-SEE SW
The available site-wide and VO-SEE SW is: How to find where the SW is installed: lcg-info --vo see --list-ce --query 'Tag=VO-see-octave*‘ How to submit a job to the appropriate site: requirements=Member("VO-see-octave ",other.GlueHostApplicationSoftwareRunTimeEnvironment); How to use the SW: export PATH="$VO_SEE_SW_DIR/octave /bin:$PATH" (this is going to be changed in the near future. It will be something like `source $VO_SEE_SW_DIR/octave /env.sh` but you will be informed about this change) New requests for GNU SW, at: MPICH-1.2.7p1 VO-see-octave MPICH VO-see-sunjdk1.6.0_04 MPI-START VO-see-NS2 HYDRA-CLIENT VO-see-MEEP GFORTRAN-4.1 VO-see-gsl-1.9 MATLAB VO-see-RNAHybrid-2.1 VO-see-Espresso (testing period) VO-see-FLUKA (testing period) Προχωρημένα Θέματα σε Τεχνολογίες Πλέγματος 23 – 24 February 2009

11 HellasGrid - MATLAB Status Trials Independent usage More info
We are in close collaboration with Mathworks to integrate their MATLAB Distributed Computing Toolkit with gLite and EGEE's grid infrastructure. Trials Trial Period - limited lifespan (3-6 months). The server side has been installed in one of HG clusters (HG-03-AUTH) lcg-info --vo see --list-ce --query 'Tag=*MATLAB*' CE: ce01.afroditi.hellasgrid.gr:2119/jobmanager-pbs-see Or in your jdl file include requirements = Member("MATLAB_R2008B",other.GlueHostApplicationSoftwareRunTimeEnvironment); Mathworks will propose a licensing scheme specific for the grid For the time being the trials are still on-going at least till March NOTE: the users should have free limited-time licenses in order to participate in the trials Independent usage Many users are using pre-compiled m files so they don't have to have MATLAB installed in the clusters Of course they use they own licenses for the software More info At: Contact person for MATLAB policy/licensing issues: Vangelis Floros Προχωρημένα Θέματα σε Τεχνολογίες Πλέγματος 23 – 24 February 2009

12 export LD_LIBRARY_PATH=`pwd`/libs:$LD_LIBRARY_PATH
Some Technical Tips Make a full static compilation of your SW Get your “special” libraries with you and use them at run-time For example: put the “special” lib in the `pwd`/libs directory on the WN you are running and add the `pwd`/libs to the LD_LIBRARY_PATH export LD_LIBRARY_PATH=`pwd`/libs:$LD_LIBRARY_PATH Use the Storage elements instead of InputSandbox and/or OutputSandbox especially if you want to transfer files >10MB (in theory this limit is at 100MB) A typical scenario (it is assumed that you have created a tarball of your “special” libraries named libs.tar): At the UI Copy the libs.tar to a storage element Pass the LFN to the .sh script that it will be executed on the target WN Submit the job At the WN Copy the libs.tar from the strorage element to the current directory Untar Run the exe Copy the results (results.tar) to a storage element Echo the second LFN Copy the results.tar from the storage elements to you current directory. Afterwards delete the results.tar from the storage element Contact us Προχωρημένα Θέματα σε Τεχνολογίες Πλέγματος 23 – 24 February 2009

13 Auxiliary Services Προχωρημένα Θέματα σε Τεχνολογίες Πλέγματος
23 – 24 February 2009

14 HellasGrid - GridMap Available at: http://gridmap.hellasgrid.gr/
It provides: Connected to the central SAM Status/availability of the HG sites (CE, SE) Running jobs Number of CPUs Προχωρημένα Θέματα σε Τεχνολογίες Πλέγματος 23 – 24 February 2009

15 Regional Application Portal (1)
Available at: Some useful characteristics: Connected to the central db at INFN (with a replica at IASA) Support all the applications using the EGEE infrastructure Provide links to the Regional Support representatives/teams Advanced searching mechanism Provide information such as: Acronym Application Name/Short Discipline Sub-Discipline Description/Abstract Scientific Contact List Web Site Region VO It will be announced very soon from the application-support and the user-support teams. Προχωρημένα Θέματα σε Τεχνολογίες Πλέγματος 23 – 24 February 2009

16 Regional Application Portal (2)
Available at: Προχωρημένα Θέματα σε Τεχνολογίες Πλέγματος 23 – 24 February 2009

17 Regional Application Portal (3)
Available at: Προχωρημένα Θέματα σε Τεχνολογίες Πλέγματος 23 – 24 February 2009

18 HellasGrid/SEE Hydra infra
What is HYDRA: Is an encrypted storage solution. This works by encrypting the files and storing them on normal storage elements. HellasGrid/SEE hydra infrastructure consists of 3 hydra servers: hydra01.egee-see.org (HG-03-AUTH) hydra02.egee-see.org (HG-06-EKT) hydra03.egee-see.org (HG-05-FORTH) A typical scenario: (available at Tip: An SEE user should request to be a member of the hydra Role by sending a signed at Προχωρημένα Θέματα σε Τεχνολογίες Πλέγματος 23 – 24 February 2009

19 Software Discovery Engine (1)
SDE: HellaGrid Search engine for the sw that is installed at the HG sites and UIs. Main Characteristics Web-base service (PHP, AJAX, mysql) Site-wide SW: Automated data retrieval using the Grid IS UI SW: A Client side SW is developed for publishing info. Easy of use Provides answers to questions such as: What kind of sw is installed to the HG sites? How can you use it? Is there extra info about the SW? What is the current status of the site which has the desired SW? And many other … Προχωρημένα Θέματα σε Τεχνολογίες Πλέγματος 23 – 24 February 2009

20 Software Discovery Engine (2)
HellasGrid SDE Main: Please try it, at: Προχωρημένα Θέματα σε Τεχνολογίες Πλέγματος 23 – 24 February 2009

21 Future Work Προχωρημένα Θέματα σε Τεχνολογίες Πλέγματος
23 – 24 February 2009

22 RPMdb schema for the VO-SEE SW (1)
It is based on the CMSSW installation/management procedure. A bootstrap script is used for the initial setup of the SEE RPMdb. The bootstrap script is modified based on SEE VO needs. An APT repository is available for the SEE SW rpms. A typical procedure is: The sgmsee user submits a job that will download and execute the bootstrap script on the target site. #!/bin/bash if test -d $VO_SEE_SW_DIR then wget export SCRAM_ARCH=slc4_ia32_gcc345 sh -x ./see_bootstrap-$SCRAM_ARCH.sh setup -path $VO_SEE_SW_DIR source $VO_SEE_SW_DIR/slc4_ia32_gcc345/external/apt/0.5.15lorg3.2-CMS3/etc/profile.d/init.sh apt-get update fi After the initial setup, the SEE rpmdb should look like the example below: ~]$ rpm -qa external+gcc CMS external+apt lorg3.2-CMS ………………….. Προχωρημένα Θέματα σε Τεχνολογίες Πλέγματος 23 – 24 February 2009

23 RPMdb schema for the VO-SEE SW (2)
The sgmsee user submits a new job for the installation of a specific SEE SW available at the SEE APT repository. The installation script could be as easy as the example provided below. #!/bin/bash PKGNAME=$1 export SCRAM_ARCH=slc4_ia32_gcc345 source $VO_SEE_SW_DIR/slc4_ia32_gcc345/external/apt/0.5.15lorg3.2-CMS3/etc/profile.d/init.sh apt-get clean apt-get update apt-get install $PKGNAME The new view of the SEE rpmdb should contain the newly installed rpm: ~]$ rpm –qa | grep see see+base-env Of course, after the SEE SW installation is made, the corresponding VO-see Tag is added. The SEE RPMdb prototype it is already deployed in two sites, the GR-06-IASA (32bit WNs) and the HG-05-FORTH (64bit WNs). It is tested and the results are more than encouraging. Προχωρημένα Θέματα σε Τεχνολογίες Πλέγματος 23 – 24 February 2009

24 Transition to EGI/NGI model
Προχωρημένα Θέματα σε Τεχνολογίες Πλέγματος 23 – 24 February 2009

25 Transition to EGI/NGI model
The EGI/NGI model will establish a sustainable European grid infrastructure User/scientific communities requirements: Ported applications should not be deprecated. Middleware evolution. No discontinuation of infrastructure operations No loss of centralized services: documentation, training, application porting support. EGI/NGI should also target sustainable user communities and collaborations. Προχωρημένα Θέματα σε Τεχνολογίες Πλέγματος 23 – 24 February 2009 25

26 4th EGEE User Forum The 4th EGEE User Forum will take place on 2-6 March 2009 in Catania, Sicily, ITALY. The event is co-located with OGF25 and OGF-Europe’s 2nd international event and will be hosted by INFN. Dates: From 02 March :00 to 06 March :00 Location: Le Ciminiere, Catania, Sicily, Italy Viale Africa Catania Chairs: Mr. Floros, Evangelos Additional info: 4th EGEE User Forum official web site: OGF25 official web site: Προχωρημένα Θέματα σε Τεχνολογίες Πλέγματος 23 – 24 February 2009

27 Useful links & s HG Applications Support team : HG Users Support team: HG GridMap: Application Support Wiki: Regional Support Registry Portal: Application Porting team (EGEE-wide): Προχωρημένα Θέματα σε Τεχνολογίες Πλέγματος 23 – 24 February 2009