NSO8055 Okeanograafiline prognoosJüri Elken elken@phys.sea.ee

Andmete haldamise küsimusi(väga lai temaatika, ühtne kontseptsioon puudub, siiski: http://www.iode.org)

ICES ajalooline andmehõiveperfokaardi formaat (andmed)ROSCOP vorm (meta-andmed)geograafiline kodeering (Mardseni kvadraadid)

Andmete haldamise tendentsid MyOcean, Sea-SEARCH & SeaDataNet

Meta-andmed: EDIOS

Card from a Fortran program: Z(1) = Y + W(1)

1881, Herman Hollerith

IBM 80 column punch card formatkasutusel alates 1928 kuni ca 1980andmestruktuur kanti üle magnetlintidele

numbrid 1 auk, tähed mitu aukukasutati ka “overpunch”FORTRAN reeglid põhinevad perfokaardile

Perfokaart

ICES ajalooline andmehõive

ICES Punch Card format (Data) 1968ROSCOP Cruise Report format (MetaData)

ICES perfokaardid

Hydro Master Cardiga jaama kohta 1, sisaldab meteoandmeid

Hydrochemistry Cardiga sügavuse kohta 1 kaart, sh riigi, laeva, koordinaatide, aja andmed

Hydrography Cardiga sügavuse kohta 1 kaart, kohandatud CTD-le (rohkem tüvenumbreid)

andmete lahutusasukoht minutsügavus mT, S sajandikbiogeenide suured kontsentratsioonid “overpunch” abilNB! puudub info metoodika ja kvaliteedikontrolli kohta

Submitters of data should, if at all possible, use either of the formats described here. Receivers of data from ICES will receive it in only the first described format, the ICES

Oceanographic 'punch card' format but software is normally supplied with any request to help the user read data sets prepared in this format, including an export facility to data bases and spreadsheets. The user should note that this format has been modified from that published by ICES in 1979 in several important respects. In particular provision was made to include position information to .01 of a degree, and time to the nearest minute. Other changes include a re-definition of the > (greater than) overpunch in the nutrient fields (type '56' chemistry record was replaced by type '76') and record type 'P6' was introduced to accommodate the very high nutrient levels reported from some coastal regions. In both of these record types chlorophyll 'a' is stored to only one decimal place (the '56' record type was 2 decimal places).

From early 1994, additional features were added to the format (03 record) to accommodate extra decimal places common in CTD records. This affected only columns previously used for derived quantities (sigma-t, dynamic depth).For data received after ca 1997 parameters not supported by the above record types were included by the inclusion of the '0Z' record (Additional Parameter Record). This record type allows for any number of parameters, so long as it is specifified in the

BODC/JGOFS data dictionary.

ICES andmehõive kaasaegne juhend(ka HELCOM seire kasutab seda)

http://www.ices.dk/Ocean/formats/

(Position 79-80 = "76" or *Position 79-80 = "P6" or Position 79-80 = "56")

PositionParameter Description 01-27 Parameter Copy of contents of positions 01-27 in the Hydromaster record. 28-31 Depth/Pressure Pressure (decibars) or Depth (meters) no implied decimals Assumes unit as in "03" record above. 32-35 Temperature Temperature Given in degrees Celsius (°C) 2 implied decimal places Negative temperatures are indicated by "}" (Closing Brace) in field 32 36-39 Salinity Salinity given in PSS-78 scale for post-1978 data. 2 implied decimal places 40-42 Oxygen Oxygen contents. Given in cm3 02 / dm3 water at STP. 2 implied decimals (If position 78 = "K" then /kg) 43-45 Phosphate Phosphate phosphorus. Given in µmol/l. (if position 78 = "K" then µmol/kg) 2 implied decimals (* 1 implied decimal) 46-48 Tot.Phosphorus Total Phosphorus contents. Given in µmol/l. (if position 78 = "K" then µmol/kg) 2 implied decimals (* 1 implied decimal) 49-51 Silicate Silicate contents (Silicate Silicon). Given in µmol/l. (if position 78 = "K" then µmol/kg) 1 implied decimal (* 0 implied decimal) 52-54 Nitrate Nitrate contents (Nitrate Nitrogen). Given in µmol/l. (if position 78 = "K" then µmol/kg) 1 implied decimal (* 0 implied decimal) 55-57 Nitrite Nitrite contents (Nitrite Nitrogen). Given in µmol/l. (if position 78 = "K" then µmol/kg) 2 implied decimals (* 1 implied decimal) 58-60 Ammonium Ammonium contents (Ammonium Nitrogen) Given in µmol/l. (if position 78 = "K" then µmol/kg) 1 implied decimal (* 0 implied decimal) 61-63 Tot. Nitrogen Total nitrogen contents. Given in µmol/l. (if position 78 = "K" then µmol/kg) 1 implied decimal (* 0 implied decimal) 64-66 Hydrog. Sulph. Hydrogen Sulphide contents (Sulphide Sulphur).Given in µmol/l. (if position 78 = "K" then µmol/kg) 1 implied decimal (* 0 implied decimal) 67-69 pH Hydrogen ion concentration in situ 2 implied decimals 70-73 Alkalinity Alkalinity. Given in milliequivalents(millival)/dm3 water at 20°C.( if position 78 = "K" then meq/kg) 3 implied decimal places 74-76 Chlorophyll a Chlorophyll a. Given in ug/dm3 water at 20°C.(if position 78 = "K" then ug/kg) 1 implied decimal (if 56 record, 2 implied decimals) 77 None Reserved 78 Unit Indicator Not K = All Chemistry units in /l (per volume) K = All Chemistry units in /kg (per mass) 79 Indicator Always "7" (Seven) (**or "P" in which case all nutrients and H2S X10, ie decimals-1) 80 Record type Always "6" (Six)

Position 79-80 = "0J "

PositionParameter Description 01-02 Country Coded according to IOC country codes. 03-04 Ship Coded according to IOC ship codes. 05-08 Station No. Station number (within a given year start counting one at the 1 Jan 0000 Hrs UTC is recommended). 09-12 Latitude Geographical latitude in degrees and minutes (decimals see below). 13-17 Longitude Geographical longitude in degrees and minutes (decimals see below). 18 Quadrant Indicator of quadrant on globe: 0 = Latitude North Longitude East 1 = Latitude North Longitude West 2 = Latitude South Longitude East 3 = Latitude South Longitude West 0° Latitude is defined as being North 0° Longitude is defined as being East 180° Longitude is defined as being West North, South relative to the equator East, West relative to Greenwich meridian 19-21 Year Last 3 digits of year. 22-23 Month Number of the month within a year 24-25 Day Number of the day within a month 26-27 Time Starting time of hydrographic station in UTC (minutes given later). 28-31 Depth Corrected depth to bottom in meters 32-45 None Reserved (usually specifies origin of data) 46-64 None Weather information (rarely used in recent data) 65-66 Latitude ct'd Decimals of Latitude minutes 67-68 Longitude ct'd Decimals of Longitude minutes 69-70 Time ct'd Minutes of time 71-74 None Reserved 75-77 Secchi Secchi Disk Depth (metre, 1 implied decimal) 78 None 79 Indicator Always "0" (Zero) 80 Record Type Always "J" (Juliett)

ICES perfokaardi näited:

Hydro Master jaHydrochemistry

ROSCOP (Cruise Summary Report)

ROSCOP (Report of Observations/Samples collected by Oceanographic Programmes) kinnitati IOC poolt 1960-ndate lõpus

Ekspeditsioonide lühiinfomida millega mõõdetikus (Marsden kvadraadid)kelle käest küsida mõõtmisandmeid

kaasajal analoog: meta-andmed

Kirjelduste grupid:Meteorology (6)Physical Oceanography (18)Chemical Oceanography (16)Marine Contaminants/Pollution (8)Marine Biology/Fisheries (27)Marine Geology/Geophysics (16)Other (>30)

Näide

Data. We make distinction between information and data. Information is describing a status or situation (e.g. today is cold); data quantify the status or situation (e.g. temperature is 10°C). We define data broadly to include experimental or in situ observations, model outputs and images.

Visualization. The first significant examples of Marine Information Systems were developed during the '60s. Until the '80s the data management was composed of many independent processes: data collection; pre-processing; storage in files. Graphical representation and dissemination were part of the scientific study of the ocean dynamics. Today an efficient information strategy includes visual representation of data (graphs, maps, ...), as a tool for dissemination of data among users and the public.

Quality control. There are areas were the use of term 'data' is often controversial: a) processed versus raw measurements, b) model outputs versus observations, c) images versus digital underpinnings. Scientist strive to fully characterise their data to enable a better understanding of its limitations. Use of data can be limited by the lack of certain attributes such as: procedures for collection, conditions during collection, instrumentation, temporal and spatial referencing, error or uncertainty, indications on quality assurance procedures.

Analysis. Working with data provides opportunities for quantitative analysis and reasoning, broad discussion and debate to evolve scientific understanding.

Models. We define as model an idealisation that embodies certain aspects of the 'real ocean'. Models provide an experimental apparatus for the scientific rationalisation of the ocean phenomena. In the presentation of ocean model fundamentals, it is useful to start with a discussion on fluid kinematics.

Data Tools and Models (SeaDataNet)

Kui andureid ja platvorme oli vähe, koguti andmed suurtesse keskustesse

Kaasajal hoitakse andmed enamasti mõõtja juures, formaadid jms on kirjeldatud, ligipääs üle veebi, rahvuslikud andmekeskused on siiski tugevad:

Meta-andmed kirjeldavad andmeid (parameetri definitsioon, metoodika, platvorm, kvaliteedikontroll jne)

Andmete otsimine läbi meta-andmete kataloogi

Andmeülekande protokollid (ftp, OpeNDAP jne)

Operatiivne okeanograafia: mõõdetavaid parameetreid vähe, lihtsam suuri “süsteemide süsteeme” kokku panna

Interdistsiplinaarne mereteadus: parameetreid tohutult, juba defineerimine keerukas

Andmete haldamise tendentsid

Operatiivne süsteem

Operatiivne süsteem

Andmete “otsimise” projekt (juba lõppenud)

http://www.sea-search.net/

SEADATANET has developed an efficient distributed Marine Data Management Infrastructure for the management of large and diverse sets of data deriving from in situ and remote observation of the seas and oceans.

The on-line access to in-situ and remote sensing data, meta-data and products is provided through a unique portal interconnecting the interoperable node platforms constituted by the SeaDataNet data centres. The development and adoption of common communication standards and adapted technology ensure the platforms interoperability. The quality, compatibility and coherence of the data issuing from so many sources, is assured by the adoption of standardized methodologies for data checking, by dedicating part of the activities to training and preparation of synthesised regional and global statistical products from the most comprehensive in-situ and remote sensing data sets made available by the SeaDataNet partners.

http://www.seadatanet.org

The marine observing system is highly fragmented: more than 600 scientific data collecting laboratories from governmental organizations and private industry have been identified. They collect data by using various sensors on board of research vessels, submarines, fixed and drifting platforms, airplanes and satellites, to measure physical, geophysical, geological, biological and chemical parameters, biological species etc. The collected data are neither easily accessible, nor standardized. They are not always validated and their security and availability have to be insure in the future.

SeaDataNet üldine andmete haldamise kontseptsioon

http://www.seadatanet.org

CDI

Otsing 2005-2008

Andmeid on veel vähe!

väljavõte 2010

SeaDataNet produktid

Soolsuse klimatoloogia 1975-2005

Jaanuar Aprill

Juuli Oktoober

Interpolatsioon: 4D Data-Interpolating Variational AnalysisSoft: DIVA GHER (University of Liege) http://modb.oce.ulg.ac.be/projects/1/diva

http://odv.awi.de/en/home/ • vabalt kasutatav• Windows (7, Vista, XP, 9x, Me, NT, 2000), Mac OS X, Linux, and UNIX (Solaris, Irix, AIX)• oma andmete formaat, kuid loeb ka NetCDF• rannajoon, sügavused

The BODC Parameter Dictionary

In the 1980s, when we first started managing oceanographic data our dictionary contained less than twenty parameters.

The BODC Parameter Dictionary is a collection of controlled vocabularies for parameter management. The BODC Parameter Usage Vocabulary (8 MB) contains almost 19,000 terms that are designed to label data values. These have been systematically constructed using a semantic model.

Navigation through such a large number of parameters is a daunting task. To help with this, a 3-layer hierarchy of discovery keywords is provided. The top level is the SeaDataNet Parameter Disciplines, followed by the SeaDataNet Agreed Parameter Groups and the BODC Parameter Discovery Vocabulary.

XML formaadis, kasutatav vastava tarkvaraga

http://www.bodc.ac.uk/data/codes_and_formats/parameter_codes/

BODC = British Oceanographic Data Centre

Milliseid andmeid tuleb käsitleda?

Term definition

Administration and dimensions

Parameters related to spatial and temporal co-ordinates, entity referencing (eg record numbering and keys) and access control

Atmosphere The atmospheric sciences domain

Biological oceanography The biological oceanographic science domain

Chemical oceanography The chemical oceanographic science domain

Cross-discipline No specific association with an identified domain

CryosphereThe cryosphere science domain, including ice on both land and sea

Marine geology The marine geological science domain

Physical oceanography The physical oceanographic science domain

Terrestrial The terrestrial science domain

SeaDataNet Parameter Disciplines

asendatud veebisõnastikuga http://seadatanet.maris2.nl/v_bodc_vocab/welcome.aspx/

SeaDataNet Agreed Parameter Groups term term

Acoustics Optical properties

Administration and dimensions Other biological measurements

Amino acids Other inorganic chemical measurements

Atmospheric chemistry Other organic chemical measurements

Bacteria and viruses Other physical oceanographic measurements

Biota composition PCBs and organic micropollutants

Birds, mammals and reptiles Phytoplankton

Carbon, nitrogen and phosphorus Pigments

Carbonate systemRate measurements (including production, excretion and grazing)

Cryosphere Rock and sediment age and dating

Currents Rock and sediment biota

Dissolved gases Rock and sediment chemistry

Fatty acids Rock and sediment lithology and mineralogy

Fish Rock and sediment physical properties

Fluxes Sea level

Gravity, magnetics and bathymetry Sediment pore water chemistry

Halocarbons (including freons) Sedimentation and erosion processes

Hydrocarbons Sonar and seismics

Isotopes Suspended particulate matter

Metal concentrations Terrestrial

Meteorology Water column temperature and salinity

Microzooplankton Waves

Nutrients Zooplankton

asendatud veebisõnastikuga http://seadatanet.maris2.nl/v_bodc_vocab/welcome.aspx/

EDIOS Meta-andmete formaat (MIF) (1)

asendatud veebivormiga http://seadatanet.maris2.nl/v_edios/search.asp

EDIOS Meta-andmete formaat (MIF) (2)

asendatud veebivormiga http://seadatanet.maris2.nl/v_edios/search.asp

Marine Strategy Framework Directive

foresees adoption of methodological standards for the assessment of the status of the marine environment, monitoring, environmental targets and the adoption of technical formats for the purposes of transmission and processing of data in line with INSPIRE Directive. In respect of each marine region or subregion, Member States shall make an initial assessment of their marine waters, taking account of existing data where available. Member States sharing a marine region or subregion shall draw up monitoring programmes and shall, in the interest of coherence and coordination, endeavour to ensure that:

(a) monitoring methods are consistent across the marine region or subregion so as to facilitate comparability of monitoring results;

(b) relevant transboundary impacts and transboundary features are taken into account.

Uued arengud (1)

• Lot 1 – Hydrographic data • Lot 2 – Marine geological data • Lot 3 – Chemical data • Lot 4 – Biological data

EMODNETEuropean Marine Observation and Data NETwork

EMODNET will improve availability of high quality data. EMODNET will provide data on scales defined by the regions and subregions of the Marine Strategy Framework Directive. The parameters to be collated are chosen to fit in with the requirements of the Directive.

Four service contracts were launched for creating pilot components:

Uued arengud (2)

EMODNET vision: components

Läänemere andmeid veebis

Operatiivsed andmed (ainult näha) BOOS www.boos.org

Ajaloolised andmebaasidICES vaba http://www.ices.dk/ocean/BED vaba http://nest.su.se/models/bed.htmBALTEX parooliga http://www.gkss.de/baltex/data/index.html

FMI ja SYKE (endine FIMR)SMHI (SHARK)