Copernicus Climate Change Service
Climate Data Store (CDS) & Toolbox.
Angel Lopez Alos [email protected] Raoult [email protected] Bergeron [email protected]
Copernicus Climate Change Service
2
Be an authoritative source of climate information for Europe
Build upon national investments and complement national climate service providers
Support the market for climate services in Europe
C3SOverall Vision and Value
Copernicus Climate Change Service
3
Copernicus Climate Change Service
Questions addressed in C3S:• How is climate changing?
• Earth observations• Reanalyses
• Will climate change continue/accelerate?• Predictions• Projections
• What are the societal impacts?• Climate indicators• Sectoral information
C3SOverall Vision and Value
Reliable and up-to-date Information for Adaptation and Mitigation
Copernicus Climate Change Service
4
ClimateData Store
SectoralInformation
System
Evaluation andQuality Control
Outreach and Dissemination
C3SComponents
4
• ECVs past, present and future
• Observed, reanalysedand simulated
• Derived climate indicators
• Tools to support adaptation and mitigation at global and European level
• Monitors quality of C3S products and services
• Ensures C3S delivers state‐of‐the‐art climate information to end‐users
• Identifies gaps in service provision
• Bridges Copernicus with the research agenda in Europe (e.g. H2020, national research projects)
• Web content • Public outreach• Coordination with national outreach
• Liaison with public authorities
• Conferences, seminars• Training and education
Copernicus Climate Change Service
5
ReanalysesMulti model seasonal
forecast products
Data reprocessing
Data collection and data rescue
Other ECVdatasets
Climateprojections
Products Landscape on the
Climate Data Store / ToolboxContent
Copernicus Climate Change Service
6
Scientific basis:• Essential Climate Variables as defined
by GCOS • GCOS Status Report (GCOS-195)• IPCC, CMIP
ObservationsObservationsGlobal estimates
of ECVs from satellite and in-
situ data
Reprocessed CDRs, reference
observations
Support for data rescue, climate data collections
Climate reanalysisClimate
reanalysisGlobal
atmosphere, ocean, land
Regional reanalysis
Coupled climate reanalysis for
100 years
Seasonal forecasts and
climate projections
Seasonal forecasts and
climate projections
Multi-model seasonal
forecast products
Access to CMIP data and products
European multi-model climate
projections
Clim
ateIndicators
Climate Data Store / ToolboxContent
Copernicus Climate Change Service
7
Stage 0/1 ‐ Proof of Concept/Pre‐operationalStage II ‐ Operational ~20 ECVs, ~5‐6 SectorsStage III ‐ Operational ~30 ECVs, ~8‐10 Sectors
2013 2014 2015 2016 2017 2018 2019 2020 2021
Stage 0 Stage I Stage II Stage III
PoC + Pre‐operational Phase
Operational Phase
C3S Timeline
Copernicus Climate Change Service
8
The Climate Data Store
(CDS)
Copernicus Climate Change Service
9
• Climate Data Store will be at the heart of the C3S infrastructure.
• Distributed System with a unique interface offering:• Information about past, present and future climate.• Essential Climate Variables and derived climate indicators. • Observations, global and regional climate reanalyses, global and regional
climate projections and seasonal forecasts.• Authoritative set of software (toolbox) that will allow the users to develop
applications that will make use of the content of the CDS.
• CDS will accommodate the needs of the highly diverse set of users that will include policy makers, experts as well as scientists.
CDS: Core Infrastructure supporting the
Climate Data Store / ToolboxVision
Copernicus Climate Change Service
10
MAIN CHALLENGES FOR
• Diversity of Products • Raw to elaborated
• Diversity of Users and Use Cases• Scientist to policy maker.
• Diversity of Volumes• From KB to PB
• Future Scalability• Increment in the number of Users, Providers,
Requirements,…
Public Authority; Urgent Requirements;
Elaborated data (charts, graphics, …);
Mb
CDS
Scientist; Research Proyect;
Raw Data; Pb
Climate Data Store / ToolboxMain Challenges
Copernicus Climate Change Service
11
• Integration of new requirements, data sources, functionalities, scenarios, …
• Guarantee high availability [24/7] for data access.
• Open to new platforms, software tools and methodologies.
• Provide with high level added-value on data services.
• Intuitive and friendly Front-end, robust and scalable backend.
• Data Processing closer to data sources
• Interoperability – Compliance with Standards.
Aim: Create an Open Framework further than a classical Web Portal
Climate Data Store / ToolboxMain Challenges
Copernicus Climate Change Service
12
Climate Data Store / ToolboxArchitecture
Monitoring / M
etrics
Computation Exec. Results/Cache
Data/Products CatalogueDiscover/View/
Download
Data/ProductMetadata Management*Users Management*Authentication
Tools/WorkflowManagement*
Application Management*
Applications Use
WEB PORTAL
BACK END
Data/Products Catalogue
Tools/Workflow Inventory
User Settings
User Requests
Web PagesContent Management
System
BROKER / Scheduler Compute
QueuesStatus
Applicationsinventory
Staging
Adaptor Adaptor Adaptor Adaptor
Data/Products Catalogue
Iso19115 Export
Tools/Workflows Repository
Computation Exec.
Mon
itorin
g / M
etric
s
DATASite 2
Tools Repository
Computation
Data Elaborated
DATASite 1
DATADATADATASite nDATA
Site 4
*Creation Modification Deletion
CLIM
ATEDATA
STORE
Toolbox Infrastructure
DATASite 3
Tools Repository
Computation
Data Elaborated
End User CDS Administrator CDS EQCExternal Information
SystemsData Supplier
Adaptor
Browse Browse API CatalogueExport
DATA
SUPPLIERS
FMI SmartMet
Copernicus Climate Change Service
13
• “Web Portal”
• Content Management System (articles, news, events, …)
• Catalogue of products • ISO19115, for interoperability with INSPIRE, GEOSS, WMO Information System, …
• Users’ settings• Profiles, preferences, licenses, …
• Users’ requests• Current and past users requests: data retrievals and computations• Request management: monitor, cancel, download results
Climate Data Store / ToolboxWeb Portal
Copernicus Climate Change Service
14
• Broker/Scheduler• Dispatches data retrieval and computation requests
to the relevant data repositories• Implements quality of service (QoS).
• Adaptors• Ensure interoperability between the broker and each of the data
repositories.
• Data Repositories (distributed)• Located at each data providers, hold datasets available via CDS.• May implement some tools to perform analytics on local data
Copernicus Climate Change Service
15
• Toolbox• Tools that run next to the data (e.g. at the data providers)
when possible• Workflows that orchestrates the invocation of tools in a
distributed fashion to combine data from different data repositories
• Web based applications that let users interact with workflows
• Compute layer• Will be used by the toolbox when tools cannot be run on the
data• Will make use of cloud technologies
• REST Based API• Provides batch access to products and tools.
Copernicus Climate Change Service
16
• Monitoring• Reporting
• Capacity planning• Usage statistics
• Service level agreement• On-call and support• Help desk
• High-availability• Backup
Climate Data Store / ToolboxOperations
Copernicus Climate Change Service
17
Climate Data Store / ToolboxToolbox Challenges
CDS-ToolBox particular challenges:
• Data residing at different locations
• Interoperability, efficiency
• User-defined workflows
• Variety of presentation methods
• Need for interactivity
• Access via API
maps
text
controls
graphs
Copernicus Climate Change Service
18
Example #1 (hypothetical):Monthly average of temperature at selected location
• ERA5: 40 years reanalysis, hosted at ECMWF (GRIB, in Kelvin)• CIMP6: 2000 years climate projections, hosted in an ESGF node
(NetCDF, Kelvin)• Observation: time series of temperature measured at a given station,
hosted in ClimatDBase (SQL, imaginary dataset)
Copernicus Climate Change Service
19
Stakeholders, UX Inputs, ..
System Architecture, Mockups, Flow Diagrams, …
Development and Implementation
Agile Methodology: Brings flexibility and adaptability during development process.
Climate Data Store / ToolboxAgile Development
Copernicus Climate Change Service
20
Copernicus Climate Change Service
21
KO Sprint 0 Sprints 1-6 Sprints 7-13
6Se
p 16
04 O
ct 1
6SC1 End
30 S
ep 1
7
5 Se
p 16
Initial Product Backlog
Development Environment
Roadmap Release 0.1
Roadmap Release 1.0
Q1 Review
Q1 Review
v0.1 Review
v0.1 Review
v1.0Review
v1.0Review
4 weeks 6 x 4 weeks 6 x 4 weeks
Review Meetings(at ECMWF)
Climate Data Store ToolboxDevelopment Timeline to release 1.0
04 M
ar 1
7
Copernicus Climate Change Service
22
Sectorial Information Systems (SIS)
Copernicus Climate Change Service
23
Proof Of Concept Development of
sectorial applications
Copernicus Climate Change
Service
Sectoral Information SystemsOverview
Copernicus Climate Change Service
24
Copernicus Climate Change Service
Seven proof of concept SIS contracts have been awarded:
SIS water management: SWICCA (Service for Water Indicators in Climate Change adaptation) – lead(Sweden) EDgE (End-to-End demonstrator for improved decision making in the water sector in
Europe) – Lead CEH (UK)
SIS energy: CLIM4ENERGY (Climate for Energy) – Lead CEA (France) ECEM (European Climatic Energy Mixes) – Lead UEA (UK)
SIS others: AgriCLASS (Agriculture Climate Advisory Services) – Lead Telespazio (UK) WISC (Windstorm Information Service) – Lead (UK) URBAN-SIS (touching health, infrastructure, water) – Lead SMHI (Sweden)
Sectoral Information SystemsOverview
Copernicus Climate Change Service
25
•Simple presentation•Maps and time-series plots•Narrative films
•Uncertainty and model skill•For seasonal forecasts & long-term projections•How best to visualise/ disseminate?
•Access to underlying data•Subset of point data•Gridded data to drive models•NetCDF/ csv files/ GIS format•Standard Services (WXS)
SIS Interfaces
Copernicus Climate Change Service
26
Focus Groups & Case Studies in 3 contrasting parts of Europe:
Spain
Norway
United Kingdom
SIS Use Cases
Use Case driving Approach (e.g. EDgE)
Copernicus Climate Change Service
27
Predicting changes in disruption risks
Predicting changes in renewable resource
Predicting changes in conditions for operating, maintenance, commissioning
and decommissioning
Balancing grid load in an increasingly variable production mix
Energy: CLIM4ENERGY
Copernicus Climate Change Service
28 http://swicca.eu
Water manager
Purveyor
Data Provider
(C3S)
28
C3S
Sectoral Information SystemsLife-cycle (eg. SWICCA)
Copernicus Climate Change Service
29
Req
uire
men
ts
Constrained by CDS-ToolBox and SIS Projects Roadmaps
Current Status
CDS V0.1
-
CDS V1.0
Toolbox V1
CDS V2.0
Toolbox V2
Agile Development
CDS Vx.0
Toolbox Vx
Sectoral Information SystemsIntegration with CDS - ToolBox
SIS “A”
SIS “B”
Catalogue MetadataData Services
Copernicus Climate Change Service
30
Thank You!
climate.copernicus.eu