EU Horizon 2020 project

A large scale EU Horizon 2020 research and innovation project contributing to the Trans-Atlantic Research Alliance and GEO.

62 Partners, 18 Countries

International integration of Atlantic ocean observing activities – further supporters / members are welcome.

Integrated Data Portal

integrated-data-portal-1One long term goal of AtlantOS is to ensure that Atlantic data from different and diverse in-situ observing networks are readily accessible and useable to the wider community, international ocean science community and other stakeholders in this field.

The AtlantOS Integrated Data Portal designed under WP7 “Data flow and integration” is not a new system. It is an integrated access to existing systems, these being enhanced to ingest more in-situ observation data on Atlantic and to better serve the users, in a harmonised way across the systems.

The actors: Networks and Integrators

The Networks involved in Data integration for AtlantOS are:

  • Ship-based observation Networks: GO-SHIP (Global Ocean Ship-based Hydrographic Investigations Program), VOS (Voluntary Observing Ship)/SOOP (Ship of Opportunity Program), CPR (Continuous Plankton Recorder), fish and plankton surveys (ICES), seafloor mapping
  • Autonomous observing Networks: Argo, Gliders, Drifters, OceanSITES, EATN (European Animal Tracking Network)
  • Coastal observing systems: Ferrybox, FOS (Fishery Observing System), coastal profilers, fixed moorings

Some Networks are organized with DACs and GDACs components. A DAC is a Data Assembly Centre typically operating at either the national or regional scale. A DAC manages data and metadata for its area with a direct link to scientists and operators. The DAC pushes observations to the network GDAC. A GDAC is a Global Data Assembly Centre. It is designed for a global observation network such as Argo, OceanSITES, EGO for Gliders, etc. The GDAC aggregates data and metadata provided by Network DACs, in RT (Real Time) and DM (Delayed Mode).Therefore it is a portal to access, at any time, the “best version” of the Network data.

The European infrastructures or global assembly centres involved as Integrators in AtlantOS are:

  • For marine environmental data: SeaDataNet for validated and archived data;  and the In-Situ Thematic Assembling Centre (INS TAC) component of Copernicus Marine Environment Monitoring Service (CMEMS) for NRT data and for the past 60 years of historical data assembled for reanalysis needs
  • for marine biodiversity data: the ICES system, and EurOBIS

The Portals involved as Integrators in AtlantOS are:

  • EMODnet lots (physics, chemistry, bathymetry, biology) fed by Copernicus INS TAC, SeaDataNet and EurOBIS
  • GEOSS

The UE integrated data system within AtlantOS

> Starting point

Data management situation at the beginning of AtlantOS project

Data management situation at the beginning of AtlantOS project

To summarize the situation at the beginning of AtlantOS project, the data acquired by the different in situ observing networks contributing to the AtlantOS project were processed and distributed using different methodologies and means. Depending on the network, the data were either processed following recommendations elaborated by the network and made accessible through a unique portal (FTP or Web), or were processed by individual scientific researchers and made available through National Data Centres or at the Institution level. Some datasets were available through Integrators by ad-hoc links that were developed in past years within projects such as Copernicus, EMODNet, SeaDataNet, etc.

> Target

integrated-system-to-be-developed-within-the-atlantos-projectwhite

Integrated system to be developed within the AtlantOS project

standardiztion-between-networks-from-acquisition-to-service-to-users

To facilitate access to the broad array of Atlantic observations and avoid “mixing apples and oranges”, the actors first agreed on common standards for metadata and data description (detailed in D7.1 Data Harmonization and D7.2 QC Report),.

  • Mandatory Metadata (unique ids for platforms/stations, unique codes for the Institutions that provide data) for guaranteeing a continuum between data-platform-institution in an unambiguous way across the Networks
  • Common vocabularies for metadata and data description (AtlantOS mapping vocabulary)
  • Minimum NRT (24 hours to several days) QC procedures for a core of 7 variables (measured by more than one network) : T, S, Current, O2, Chl, Nitrate, Sea level, Carbon
to-plu-the-data-on-the-backbonewhite

to plug the data on the backbone

to facilitate integration into the Integrators

to facilitate integration into the Integrators

data-exchange-backbone

relying on existing international standards and protocols

  • Services: OGC standards (WMS, WFS, SWE)
  • Network and product catalogue: Geonetwork
  • Metadata: EDMO SeaDataNet catalogue for Institutions, C17  SeaDataNet Platform catalogue for most platforms and ICES directory for stations
  • Parameters: SeaDataNet P01 and P06, CF convention (P07) and WoRMS for Taxa
  • Data citation: DOI (Data Object Identifier) technique

Facilitate discovery, viewing and downloading by the users.

  • Platform catalogue (Index files/Copernicus technique)
  • Network and Product Catalogue (Sextant back office tool)
  • Controlled vocabularies (NERC/BODC) for parameters
  • Access by FTP to quality controlled data
  • Visualisation and advanced data access services
  • Detailed network and platform descriptions (SensorML marine profile)

Ease access to existing observationsservices-users

  • Facilitate discovery through Network and Product catalogue, AtlantOS catalogue, based on ISO standards
  • Provide OGC services ( WMS, WFS) to facilitate development of customised user interfaces either through Integrators or directly from Networks
  • Provide enhanced download facilities either through Integrators or directly from Networks
  • Facilitate visibility of existing data and gap identifications

> The roadmap

The roadmap towards that integrated EU data system within AtlantOS is (detailed in D7.4 Data Management Handbook) :

  • for the Networks to (1) implement the AtlantOS recommendations for standardization across Networks, (2) plan NRT QC procedures enhancement if needed and (3) facilitate access to Network data,  
  • for the Integrators to enhance (1) their tools for data integration (update their ingestion procedure to integrate new Network data, enhance viewing and downloading services on Network data, perform cross Network assessments and provide feedback to Networks, develop traceability and monitoring facilities for providers and users) and (2) the services to users