data‐intensive hydrologic modeling: a cloud strategy for integrating pihm, gis, and web‐services
DESCRIPTION
AGU presentation 2010TRANSCRIPT
- 1. Dataintensivehydrologicmodeling: ACloudstrategyforintegratingPIHM,GIS, andWebServices LorneLeonard ChrisDuffy Gopal Bhatt Xuan Yu Civil&EnvironmentalEngineering,PSU, UniversityPark,PA, UnitedStates.
- 2. OurBigGoal Tobeabletorapidly prototypeanywatershed modelintheworldanytime Yes,anywhere Dorealtimeforecastingandanalysisto improveourmodels UsingNationalDatasetsofETVs Data Intensive
- 3. Issues DataComputationallyIntensive! 100sTerabytesofETVdatatomodelwatersheds anywhereintheUSA 1000sTerabytesofdataforaroundtheworld. Federaldataserversareslow NocentraldatastoreforourETVdataneeds ComplexWorkflowstoautomatedataandmodel developmentprocessing Computationrequirementsvaryperproject ITisexpensive!Wearefocusedonresearchonly
- 4. TheappealoftheCloud Cloudcomputingenablesusto: haveaccesstoDataIntensiveandHPC computationalneedsdynamically tobescalable dodataintensiveoperationsnearHPCforfaster access Enableotherresearchersandeducatorstouse ourscientificsoftwareviathewebwithoutthe needtoinstallandmaintainsoftwareand systems
- 5. OurdefinitionofCloud Dynamicallyscalable(virtualizedresources),from Desktop,HPCclustertoNCSABlueWaters,grid Resourcesareprovidedasawebbasedservice (data,software) DataIntensiveandparallelcomputing Privatecloudtoprivatecloudconduitbetween PSUandNCSAforhydrologicalresearch Thisisaprototype!
- 6. PIHMandtheCloud:WhatisPIHM Fullycoupledmultiprocessdistributedhydrological model Usessemidiscretefinitevolumemethod Unstructuredmesh(TIN) http://www.pihm.psu.edu
- 7. 2268HUC8 103,444HUC12
- 8. OurStrategy
- 9. OurStrategy
- 10. OurStrategy AtmosphericForcing(precipitation, snowcover,wind,relativehumidity, temperature, netradiation,albedo, photosyntheticatmosphericradiation, leafareaindex) Digitalelevationmodels River/StreamDischarge Soil(class,hydrologicproperties) Groundwater(levels,extent, hydrogeologicproperties) Lake/Reservoir/Wetlands(levels,extent) LandCover/Use(biomass, humaninfrastructure,demography, ecosystemdisturbance) WaterUse
- 11. OurStrategy AtmosphericForcing(precipitation, snowcover,wind,relativehumidity, temperature, netradiation,albedo, photosyntheticatmosphericradiation, leafareaindex) Digitalelevationmodels River/StreamDischarge Soil(class,hydrologicproperties) Groundwater(levels,extent, hydrogeologicproperties) Lake/Reservoir/Wetlands(levels,extent) LandCover/Use(biomass, humaninfrastructure,demography, ecosystemdisturbance) WaterUse
- 12. PIHMCloudReAnalysisandForecast WithNCSAwearedevelopinga PIHMcloudprototypeto distributethePIHMwebservice workflowandmodel componentsoverthecloudfor researchandeducation. Calibratemodels spawn100s ofdataflowexecution parameterstoprocess, compute,analyzeandvisualize thetransformedresults.
- 13. PIHMCloudReAnalysisandForecast WithNCSAwearedevelopinga PIHMcloudprototypeto distributethePIHMwebservice workflowandmodel componentsoverthecloudfor researchandeducation. Calibratemodels spawn100s ofdataflowexecution parameterstoprocess, compute,analyzeandvisualize thetransformedresults.
- 14. PIHMCloudReAnalysisandForecast WithNCSAwearedevelopinga PIHMcloudprototypeto distributethePIHMwebservice workflowandmodel componentsoverthecloudfor researchandeducation. Calibratemodels spawn100s ofdataflowexecution parameterstoprocess, compute,analyzeandvisualize thetransformedresults.
- 15. ExampleofPIHMWebServices
- 16. ExampleofPIHMWebServices
- 17. ExampleofPIHMWebServices
- 18. ExampleofPIHMWebServices
- 19. ExampleofPIHMWebServices
- 20. ExampleofPIHMWebServices
- 21. ArcPIHM PIHMwillsoonbeavailableasa toolboxforESRIusers Developmentplansinclude developingprotocolstoencourage furthermodularitysoother developerscanplugandplaycode intothePIHMworkflow.For example,otherPhysicengines, datasetsetc ConsumeCUAHSIHydroServer, HydroGML resources
- 22. InternationalCZOsitesat CreteandPlynlimon
- 23. PIHMCloudForecastExample Realtimeforecasting
- 24. Conclusion Data&ComputationallyIntensiveWatershed Simulations! 1000sTerabytesofdatarequiredtomodelany watershedintheUSA Workflowstoautomatedataprocessingand distributethecomputationonthecloud Whatisneededisfastaccesstodatacenters thatareclosetoHPCresources
- 25. Thankyouforlistening Visithttp://www.pihm.psu.edu Formoreinformationandupdates Kumar,M.,G.Bhatt,andC.J.Duffy,2009,Anefficientdomaindecompositionframeworkforaccurate representationofgeodataindistributedhydrologicmodels,IJGIS. Kumar,M.,G.Bhatt,andC.J.Duffy,2008,TheRoleofPhysical,NumericalandDataCouplingina MesoscaleWatershedModel,AdvancesinWaterResources. Bhatt,G.,M.Kumar,andC.J.Duffy,2008,Bridgingthegapbetweengeohydrologicdataand distributedhydrologicmodeling,InProceedingsofInternationalCongressonEnvironmentalModeling andSoftware