cbsdl 2015 koch_wg_v01

Cloud‐Based Services in Digital Libraries workshopVocabulary Web Services, Business Intelligence and Digital Curation

Walter Koch ‐ Gerda KochAIT‐Angewandte Informationstechnik Forschungs‐GmbH, Graz ‐ Austria

LoCloud is funded by the European Commission's ICT Policy Support Programme

2015‐09‐17 Gerda & Walter Koch (AIT) 2

locoud.eu

About:– LoCloud is a Best Practice Network co‐funded under the CIP ICT‐PSP programme of the European Commission.

– LoCloud is supporting small and medium‐sized institutions in making their content and metadata available to Europeana.

– A cloud‐based technology infrastructure will enable the aggregation of local content, and

– a number of micro‐services will help to reduce technical, semantic and skills barriers and to render the content more discoverable and interoperable.


Relevant Work Packages

Specify, modify, test, implement core infrastructure components: MINT, MoRe, Lightweight digital library – all build on existing work Specify, modify, test, implement core infrastructure components: MINT, MoRe, Lightweight digital library – all build on existing work

WP 2WP 2Design and implementation of aggregation Design and implementation of aggregation infrastructure

Collaborative cloud‐based testlab, involving partners and users; develop and implement a suite of SaaS: geolocation enrichment; metadata enrichment; multilingual vocabularies; historic place‐names; Wikimedia and crowdsourcing

Collaborative cloud‐based testlab, involving partners and users; develop and implement a suite of SaaS: geolocation enrichment; metadata enrichment; multilingual vocabularies; historic place‐names; Wikimedia and crowdsourcing

WP 3WP 3Micro servicesMicro services for small and medium institutions


The LoCloudAggregation Structure

EnrichmentServices

Europe

ana

Europe

ana

Clou

dTestlab


Reference Model

Micro-services(Vocabulary,...)

Operating Environment(Linux, Windows, MAC)

Virtual Engines(KVM, XEN, VMDK, ...)

Micro-services(Vocabulary,...)

Outcome / Value

Process / Procedure


Testlab Implementation(opennebula.org)

Private Cloud

Private Cloud

Public Cloud


OpenNebula ‐ Frontend

http://test186.ait.co.at

Vocabulary WebServices


Evolution of Architecturespwc: Technology Forecast: Rethinking Integration, Issue 1, 2014, page 3


Microservice

Wikipedia: https://en.wikipedia.org/wiki/Microservices (2015‐09‐15)

[...] microservices is a • software architecture style in which • complex applications are composed

of small, independent processes• communicating with each other using language‐agnostic APIs.

• These services are small, highly decoupled and focus on doing a small task,facilitating a modular approach to system‐building.


http://vocabulary.locloud.eu/

• Vocabulary Microservices • This is the entry page for the LoCloud vocabulary application.

• LoCloud uses a variety of standard vocabularies during the metadata enrichment process.These vocabularies have been established with the TemaTres tool and are available in the LoCloud cloud testlab.

• In addition, it is possible to feed the tool with your own vocabularies, establish new vocabularies or add languages to existing vocabularies.


Vocabulary WebService20+ Functions, API

Parameters

Example


Example: SearchUNESCO‐Thesaurus

http://test113.ait.co.at/tematres/unesco/services.php?task=search&arg=peace


SPARQL‐ExampleDISMARC‐Genres

• Query: "Find all Broader Term for 'Acid Jazz‘“• SPARQL End Point: http://test113.ait.co.at/tematres/vocab/sparql.php

QueryResult

ThesaurusEntry


Service Call („fetchTopTerms“)embedded in a test application

Micro-Service

Vocabulary Web Service


Test Application DevelopmentDesigner View

Development of a Drop Down List

Development of a Drop Down List


The Test Application:Part of Data Entry

Path to the Thesaurus Item

„Candidate Term“ Crowdsourcing activity

Next Slide(Vocabulary Management)


Vocabulary ManagementBPMN ‐ Diagramme

„CandidateTerm“

Crowdsrc.Activity

CandidateNew Item


Business Intelligence


Business Intelligence

Wikipedia: https://en.wikipedia.org/wiki/Business_intelligence (2015‐09‐15)• Business intelligence (BI) is the set of techniques and tools for the transformation of raw data into meaningful and useful information for business analysis purposes.

• From Business Analysis to Cultural Analyticshttps://en.wikipedia.org/wiki/Cultural_analytics (2015‐09‐15)

• One Tool: ETL – „Extract Transform – Load“https://en.wikipedia.org/wiki/Extract,_transform,_load (2015‐09‐15)

• Example: Pentaho – Data Integration („Kettle“)http://www.pentaho.com/product/data‐integration


OpenUp! ‐ Natural HistoryAggregator for Europeana

Micro Service:Common Name

Service

Micro Service:Common Name

Service


OpenUp!Overall Workflow

Transformation Process


OpenUp!Transformation Workflow

Invoke Micro Service:

Common Name Service


OpenUp!Transformation Parameters

Micro Service:Common Name Service

Service End Point (Cache)


Result: OpenUp!Enriched Meta Data Set

Latin Name

Common Names


Pentaho‐PDISelected Tools

... and many more !


Digital Curation


Digital CurationState‐of‐the‐Art Examples (2010)

• Archivematica– Linux VM based toolset allows producers and archivists to implement a small scale workflow

• UC3 / CDL Merritt– Micro‐Services specifications – implement building blocks for the Digital Curation Lifecycle


ArchivmaticaMicro Services


California Digital LibraryUC3 Merritt

• California Digital Library UC Curation Center• Independant, Interoperable Micro‐Services• Complex curation function through combination of atomistic services

Knowledge Enhancement

Annotation


Micro‐Services(Merritt vs. Archivematica)

CDL/UC3-MERRITT ARCHIVEMATICA


High‐Level Workflow

• OAIS activities providea sound basis

• Archivematica Use Cases

• Merritt specifications• Micro‐Services are thebuilding blocks


CDLCuration Micro Services

1) „An Emergent Micro‐Services Approach to Digital Curation Infrastructure“, IPRES 2009https://escholarship.org/uc/item/5313h6k9

2) UC Curation Center / California Digital Library, merritt: An Emergent Micro‐services Approach to Digital Curation Infrastructure; Rev. 0.6, 2010‐03‐25

1) Annotation service 2)

The Annotation service provides a mechanism by which the consumers of managed digital content can enrich that content through additional description, association, or related content. The service supports the following methods:

• Annotate

• Submit.


Contact

Gerda Koch, [email protected] Koch, [email protected]

http://www.ait.co.at

cbsdl 2015 koch_wg_v01

Software