Web ServicesNovember 2001

Web Services as Program Components

• A Web Service is a URL addressable resource that returns requested data, e.g. current weather or the map for a neighborhood.

• Web Services use standard web protocols: HTTP, XML, SOAP, WSDL allow computer to computer communication, regardless of their language or platform.

• Web Services are reusable components, like ‘LEGO blocks’, that allow agile development of richer applications with less effort.

• Web services can transform the web from a medium for viewing and downloading to data/knowledge-exchange and distributed computing platform. (Berners-Lee)

Web Services: Connect, Communicate, Describe, Discover

Enabling Protocols of the Web Services architecture:

• Connect: Extensible Markup Language (XML) is the universal data format that makes connection and data sharing possible.

• Communicate. Simple Object Access Protocol (SOAP) is the new W3C protocol for data communication, e.g. making requests.

• Describe. Web Service Description Language (WSDL) describes the functions, parameters and the returned results from a service

• Discover. Universal Description, Discovery and Integration (UDDI) is a broad W3C effort for locating and understanding web services.

Interoperability through a Layered Protocol Stack

• Web Services are implemented on a layered stack of technologies and standards• The lower layers enable binding and exchange of messages; higher levels enable interoperability• Applications are formed dynamically from distributed components through publish-find-bind

mechanisms

TCP/IP, HTTP, FTP

ASCII, XML, etc.

HTML, XML OGC -GML

OGC Coverage, CoordTransfom, WMS

HTTP, SOAP

WSDL

UDDI OGC Catalog

WSFL, XLANG

StandardsInteroperability

Comm. Protocols

Data Encoding

Data Schema

Data Binding

Web Service

Service Integr.

Service Discovery

Service Descript.

Connectivity

Web Services Components and Actions

• Service providers publish services to a service broker.

• Service users find the needed service and get access key from a service broker• With the access key, users bind to the service provider

• The result is a dynamic binding mechanism between the service users and providers

Service Broker

Service Provider

PublishFind

BindServiceUser

Components: Provider – User – Broker

Actions: Publish – Find - Bind

Web Service Standards

Service Broker

Service Provider

PublishUDDI, WSDL

FindUDDI, WSDL

Access

SOAP, XML

Each operation is governed by standard protocols:

• Discovery and Integration: UDDI (Universal Description, Discovery and Integration)

• Service Description: WSDL (Web Services Description Language)

• Content Envelope: SOAP (Simple Object Access Protocol)

• Data Encoding: XML (Extensible Modeling Language)

ServiceUser

Web Application: Chained Web Services

• A Web Service Provider may also be a User of other services

• Multiple web services can be chained into an interactive workflow system

• The result is an agile application that can be created ‘just in time’ by the user for a specific need

Service Broker

Service Provider/User

PublishFind

BindServiceUser

Chain

Service Provider

Bind

Chain

Web PublishHTTP, FTP

4D Data Access though a Web Service Portal

Service Broker

PublishUDDI, WSDL

Service Consumer

FindUDDI, WSDL

Access

SOAP, XML

Ordinary web content can be delivered as a Web Service through a Proxy Server.

• The Proxy Server supplies a web server- to - web service ‘wrapper’

• The Proxy Server publishes the web service to the Broker

• The User accesses the Proxy to get the distributed Web Server data

Service Proxy

WebServer

ServiceUser Chain

Distributed Data Browser Architecture

XML WebServices

Satellite

Vector

GIS Data

XDim Data

OLAPCube

SQLTable

HTTPServices

Text Data

WebPage

TextData

Time Chart

Scatter Chart

Text, Table

Data ViewsLayered Map

Cursor

Session Manager (Broker)

Data View

Manager

Connection

Manager

Data Access

Manager

Cursor-Query

Manager

OpenGISServices

Data are rendered by linked Data Views (map, time, text)

Distributed data of multiple types (spatial, temporal text)

The Broker handles the views, connections, data access, cursor

Web Services Standards

• Discovery and Integration: UDDI (Universal Description, Discovery and Integration)

• Service Description: WSDL (Web Services Description Language)

• Content Envelope: SOAP (Simple Object Access Protocol)

• Data Encoding: XML (Extensible Modeling Language)

• Service Chaining: WSFL (Web Services Flow Language, IBM) or XLANG (Message flow language, Microsoft)

ChainWSFL/XLANG

Service Broker

Service Provider/User

PublishUDDI, WSDL

FindUDDI, WSDL

BindSOAP, XML

ServiceUser

Chain

Service Provider

Bind

Web Application: Chained Web Services

• Any Web Service Provider may be a User of other services

• Multiple web services can be chained into an interactive ‘workflow’ system

• The result is an agile application that can be created ‘just in time’ by the user for a specific need

Data/ServiceCatalog

VoyagerData Service

PublishFind

BindServiceUser

Chain

Data Provider

Bind

Chain

Dvoy_Services: Generic Software components

Webservice

Param 1

Param2

Service state

Webservice Adaptor

User Interface Module

Selector

state I/o ports

state I/o ports

Web service calls

Web serviceOutput data

Web serviceInput data

User Interface Module

UIM extracts relevant UI parameters from STATE

User changes UI parameters

UIM transmits modified UI parameters to STATE

Service Chain STATE Module

Contains the state params for all services in the chain

Has ports for getting/setting state params

Service Adopter Module

Gets input data from upsteam service

Gets service params from STATE

Make service call

Service Adopter ModuleGets input data from upsteam service

Gets service params from STATE

Make service call

Web service Service Module

Gets service call from Adopter module

Executes service

Returns output data

PointAccess->Grid->GridRender Service Chain

• The service chain interpreter make ONLY 2 sequential calls, stated in the data flow program:– GetMapPointDataAdaptor

– RenderMapviewPoint Adaptor

GetMapPointDatadataset_abbr: IMPROVE

Param_abber SOILf

datatime: 2001-04-16

sql_filter:

RenderMapviewPoint

dataset_url:

output_format:

out_image_width:

Etc…..

Service state

GetMapPointData Adaptor

RenderMapviewPoint Adaptor

GetMapPointData Selector

RenderMapviewPoint Selector

state I/o ports

state I/o ports

Web service calls

Web serviceOutput data

Service state

PointAccess->Grid->GridRender Service Chain

• The service chain interpreter make ONLY 3 sequential calls, stated in the data flow program:– GetMapPointDataAdaptor

– GridMapviewPointAdaptor

– RenderMapviewGridAdaptor

GetMapPointDatadataset_abbr: IMPROVE

Param_abber SOILf

datatime: 2001-04-16

sql_filter:

RenderMapviewGriddataset_url:

output_format:

out_image_width:

Etc…..

GetMapPointData Adaptor

RenderMapviewGrid Adaptor

GetMapPointData Selector

RenderMapviewGrid Selector

GridMapviewPoint Selector

GridMapviewPointdataset_url:

output_format:

out_image_width:

Etc…..

GridMapviewPoint Adaptor

state I/o ports

state I/o ports

Web service calls

Web serviceOutput data

Service state Service state Service state

SQLDataAdapter1

CustomDataAdapter

ImageDataAdapter2

SQLServer1

ImageServer2

LegacyServer

Presentation

Data Access & Use

Provider Tier Heterogeneous Data

Proxy Tier

Data Homogenization, etc.

Member ServersProxy Server

User Tier

Data Consumption

Processing

Integration

Federated Data Warehouse

Firewall; Federation ContractWeb Service, Uniform Query & Data

Federated Data Warehouse Architecture• Three-tier architecture consisting of

– Provider Tier: Back-end servers containing heterogeneous data, maintained by the federation members – Proxy Tier: Retrieves Provider data and homogenizes it into common, uniform Datasets – User Tier: Accesses the Proxy Server and uses the uniform data for presentation, integration or further processing

• The Provider servers interact only with the Proxy Server in accordance with the Federation Contract– The contract sets the rules of interaction (accessible data subsets; types of queries submitted by the Proxy)– The Proxy layer allows strong security measures, e.g. through Secure Socket layer

• The data User interacts only with the generic Proxy Server using flexible Web Services interface– Generic data queries, applicable to all data in the Warehouse (e.g. space, time, parameter data sub-cube)– The data query is addressed to a Web Service provided by the Proxy Server of the Federation – Uniformly formatted, self-describing XML data packages are handed to the user for presentation or further machine processing

Voyager: The Program

• The Voyager program consists of a stable core and adoptive input/output section• The core executes the data selection, access portrayal tasks• The adoptive, abstract I/O layer connects the core to evolving web data, flexible

displays and to the a configurable user interface:– Wrappers encapsulate the heterogeneous external data sources and homogenize the access– Device Drivers translate generic, abstract graphic objects to specific devices and formats – Ports expose the internal parameters of Voyager to external controls

Data Sources

Controls

Displays

Voyager Core

Data Selection

Data Access

Data Portrayal

Adoptive Abstract I/O Layer

Dev

ice

Dri

vers

Ports

Wra

pp

ers

ASP.NET: Dynamic Web pages

• Easy WebApp development – drag-drop of controls on a WebForm

– Binding control properties to class members and event handlers

• Controls execute on serve but render themselves in HTML

• User input at browser is posted to the server as class data and as properties

• No need to know HTML – rendering is done by the controls

• Clear separation of UI (form layout) and ‘business logic’ behind the form

• ASP.Net is compiled for high performance

• Automatic Validation Controls on the client makes input more robust

• Automatic data biding of data source to controls – i.e Data Grid

ADO.NET: DataSet and DataAdapter, another

• The DataSet is a container object for one or more DataTables, DataRelations and Constraints. To draw a comparison to classic ASP/ADO, the DataTable is analogous to a RecordSet, and the DataSet is a container for one or more DataTables. DataSet knows nothing about particular data access interfaces like OLEDB or ODBC

• The DataAdapter is the bridge between a DataSet and the data source, such as a Microsoft SQL Server database. The DataAdapter manages creating and opening a Connection, executing a Command, returning a DataReader, populating a DataTable and closing the DataReader and Connection. This can be done multiple times to populate multiple DataTables in the same DataSet.

• Using a DataSet and DataAdapter is more memory intensive than using a DataReader, since all of the records returned are populated into a DataTable (taking up valuable system resources). The DataReader streams data, using up the memory required for only one record at a time. With that said, there are instances when you would want to use a DataSet and DataAdapter, such as when you need to manipulate the data, iterate through it, or alter it and update it.

ADO.NET: Remote Database AccessMicrosoft ADO.NET PPT

• ADO.Net is a set of classes (DataSet and DataAdapter) to access remote data sources; it decouples data source from data consumer through indirection

• DataSet is a data container object of structured information on a set of tables; it can can locally cash portions of the database and synchronize changes

• Each DataSet object is associated with a subclass of DataAdapter tailored to interact with a particular data source, e.g. SQLAdopter

• To change the data source, the consumer need only to change the DataAdapter

• User can access tables as properties of DataSet – no need to know SQL

• While in transit, XML data can pass through firewalls over regular HTTP port.

DataSet: a Small Relational Database

DataRelation objects link related DataTable objects in the DataSet, providing referential integrity features similar to a database.

The DataTable objects can be nested to whatever depth is necessary to replicate the structure of a hierarchical XML document or a relational database.

A DataTable can access its relevant linkages using its child and parent Data Relation collections.

DataRows make the navigation in a hierarchy even simpler using the GetChildRows() command with the DataRelation name as a parameter.

The DataTable fulfills the role of a database table

The DataColumn determines the data type and name of the column

The DataConstraint adds extended information such as primary key,…

A collection of DataRow objects holds the data in the DataTable

The DataRow also plays the part of an updateable cache ( a cursor???)

DataSet Common client data store

• Relational View of Data– Tables, Columns, Rows,

Constraints, Relations

• Directly create metadata and insert data

• Explicit Disconnected Model– Disconnected, remotable object

– No knowledge of data source or properties• Common Behavior• Predictable performance characteristics

– Array-like indexing

– Strong Typing

DataSetDataSetTablesTables

TableTable

ColumnsColumnsColumnColumn

ConstrainConstraintsts ConstraintConstraintRowsRows

RowRowRelationsRelations

RelationRelation

DataAdapter

• Loads a table from a data store and writes changes back.– Exposes two methods:

• Fill(DataSet,DataTable)

• Update(DataSet,DataTable)

– Provides mappings between tables & columns

– User provides insert/update/delete commands

• Allows use of Stored Procedures

• Autogen component available

– Allows single DataSet to be populated from multiple different datasources

Data store

DataAdapter

MappingsMappingsMappings

InsertCommand

UpdateCommand

DeleteCommand

SelectCommand

Fill() Update()

DataSet

DataAdapter: Accessing the Data

• Each .NET data provider has a DataAdapter object: OleDbDataAdapter, SqlDataAdapter (can we make DataAdapters to legacy data servers?)

• The SelectCommand property of the DataAdapter retrieves data from the data source.

• The InsertCommand, UpdateCommand, and DeleteCommand properties manage updates to the data in the data source.

• The Fill method populates a DataTable object in a DataSet with the results of the SelectCommand

An Introduction to GXA: Global XML Web Services Architecture

Summary: Discusses the Global XML Web Services Architecture (GXA), an open architecture for application internetworking. (2 printed pages)

Extensible Markup Language (XML) Web services are the fundamental building blocks in the move to distributed computing on the Internet. Open standards and the focus on communication and collaboration among people and applications have created an environment where XML Web services have become the platform for application integration. Applications are constructed using multiple XML Web services from various sources that work together regardless of where they reside or how they were implemented. XML Web services are successful for two reasons: They are based on open standards that making them interoperable, and the technology used to implement them is ubiquitous.

XML Web services are built on XML, the Simple Object Access Protocol (SOAP), the Web Services Description Language (WSDL), and Universal Description, Discovery, and Integration (UDDI) specifications. These constitute a set of baseline specifications that provide the foundation for application integration and aggregation. From these baseline specifications, companies are building real solutions and getting real value from them. But, as companies develop XML Web services, their solutions have become more complex and their need for standards beyond this baseline is readily apparent. The baseline specifications are a strong start for XML Web services, but today developers are compelled to implement higher-level functionality such as security, routing, reliable messaging, and transactions in proprietary and often non-interoperable ways.

Addressing this need for additional specifications for XML Web services in these areas, Microsoft® and IBM presented an architectural sketch for the evolution of XML Web services at the W3C Workshop on Web Services in April 2001. This sketch was the forerunner of the Microsoft Global XML Web Services Architecture (GXA). GXA provides principles, specifications and guidelines for advancing the protocols of today's XML Web services standards to address more complex and sophisticated tasks in standard ways, allowing XML Web services to continue to meet customer needs as the fabric of application internetworking.

GXA is based on four design tenets:

                     Modular—GXA uses the extensibility of the SOAP specification to deliver a set of composable modules that can be combined as needed to deliver end-to-end capabilities. As new capabilities are required, new modular elements can be created.

                     General Purpose—GXA is designed for a wide range of XML Web services scenarios, ranging from B2B and EAI solutions to peer-to-peer applications and B2C services.

                     Federated—GXA is fully distributed and designed to support XML Web services that cross organizational and trust boundaries and requires no centralized servers or administrative functions.

                     Standards-Based—As with previous XML Web services specifications, GXA protocols will be submitted to appropriate standards bodies and Microsoft will work with interested parties to complete their standardization.

The revolution of the Web was about browsing—making information available for people. While very successful, this left a gap. Applications and companies were still disconnected islands. Just as ubiquitous support of HTML enabled the World Wide Web, ubiquitous support of SOAP enables XML Web services. XML Web services are bringing us a new revolution that allows businesses to interconnect on a programmable Internet. The Global XML Web Services Architecture is the framework for the future of XML Web services.

Portal Server• A portal solution seemed to fit BHCS's goals well. Commercial portals like Excite or Yahoo! are well known. They provide a specially

formatted window onto a variety of data, whose source is invisible to users. Enterprise portals work the same way, letting users in an organization view a customized presentation of information from many sources. Interactive portal vendors like Sequoia—the vendor BHCS ultimately chose—also standardize access to information sources, including legacy apps and databases. This permits updates to data, merging data from different sources, and supporting search capabilities based on the context of the data

Mapping Legacy Apps to XMLHow do you handle all those legacy apps? XPS can adapt to each individual legacy app's schemas, mapping their native fields and formats to XML tags and permitting XML-based indexing in context. Once legacy data has been converted to XML, administrators can then map data elements from different information sources that actually correspond to the same logical entity. For example, "name" in one document may be the same as "insurance subscriber" in another document; in a new app, these could both map to "patient."

Distributed Data Browser Architecture

XML WebServices

Satellite

Vector

GIS Data

XDim Data

OLAPCube

SQLTable

HTTPServices

Text Data

WebPage

TextData

Time Chart

Scatter Chart

Text, Table

Data ViewsLayered Map

Cursor

Session Manager (Broker)

Data View

Manager

Connection

Manager

Data Access

Manager

Cursor-Query

Manager

OpenGISServices

Data are rendered by linked Data Views (map, time, text)

Distributed data of multiple types (spatial, temporal text)

The Broker handles the views, connections, data access, cursor

.NET Data Providers - A Basic Tutorial

• Now, the .NET Data provider can be manifested as SQL Server data provider, OLEDB data provider or the ODBC Data Provider. For the time being I'll be taking you through this article in the context of the first two Data providers.

SQL Server Data provider is the most efficient way to connect to an SQL Server Database(version 7.0 onwards). It uses a proprietary protocol for connecting to the Database which results in optimized usage of the SQL Server database along with faster data transactions.The System.Data.Sqlclient namespace contains classes for the SQL Server data provider.

• OLEDB Data Provider is used with databases that support the OLE DB interfaces. ADO.NET supports the following OLE DB Providers

• SQLOLEDB - Microsoft OLE DB Provider for SQL ServerMSDAORA - Microsoft OLE DB Provider for OracleMicrosoft.Jet.OLEDB.4.0 - OLE DB Provider for Microsoft Jet

DVoy Queries as Web Services

Purpose: Locating relevant data measures fir specific location and time

Abstract Query: Find available measures in MyDataCube

Web Service: input: MyDataCube; output: List of measures, MeasureDataCube

Design: Measure table with bounding MeasureDataCube cubes

Implementation: SQL measures table with MeasureDataCubeSELECT Measures, MeasureDataCube WHERE MeasureDataCube in MyDataCube

Distinct Locations, Times, Heights

DataAdapter: Accessing the Data

• Each .NET data provider has a DataAdapter object: OleDbDataAdapter, SqlDataAdapter (can we make DataAdapters to legacy data servers?)

• The SelectCommand property of the DataAdapter retrieves data from the data source.

• The InsertCommand, UpdateCommand, and DeleteCommand properties manage updates to the data in the data source.

• The Fill method populates a DataTable object in a DataSet with the results of the SelectCommand

Diff Calculus

• Sun's slant, Engstrom says, is to provide a complete "stack" of Web services infrastructure, which includes development tools, application server platforms, and Sun's server hardware. In his own department, Engstrom says Sun is "all about making developers more productive—by doing the little things the standard doesn't address."

• One example of this process is a tool to bring existing applications (such as SAP, PeopleSoft, and IBM CICS programs) into the Web-services arena with XML adapters. "A big issue for us is how to retrofit," Engstrom says. "Because the applications running today are not going away."

Service Registry

Service Provider

PublishUDDI, WSDL

Provider Proxy

Service User

Web Service Actions and Components

Service Broker

Service Provider

Publish

Service User

Find

Access

• Web Services architecture requires three operations: publish, find, and bind.

• Service providers publish services to a service broker.

• Service users find the services and get access key from a service broker

• With the key, users access the service.

Web Service Standards

Service Broker

Service Provider

PublishUDDI, WSDL

Service User

FindUDDI, WSDL

Access

SOAP, XML

Each operation is governed by standard protocols:

• Discovery and Integration: UDDI (Universal Description, Discovery and Integration)

• Service Description: WSDL (Web Services Description Language)

• Content Envelope: SOAP (Simple Object Access Protocol)

• Data Encoding: XML (Extensible Modeling Language)

Web Service Standards

Service Broker

Service Provider

PublishUDDI, WSDL

Service User

FindUDDI, WSDL

Access

SOAP

• Each operation is governed by a separate standard.• Service providers publish services to a service broker.• Service users find the services and get access key from a service broker• With the key, users access the service.

Web Service Actions and Components

Service Broker

Service Provider

PublishFind

Access

• Web Services architecture requires three operations: publish, find, and bind.

• Service providers publish services to a service broker.

• Service users find the services and get access key from a service broker

• With the key, users access the service.

ServiceUser

Diff Calculus

• Sun's slant, Engstrom says, is to provide a complete "stack" of Web services infrastructure, which includes development tools, application server platforms, and Sun's server hardware. In his own department, Engstrom says Sun is "all about making developers more productive—by doing the little things the standard doesn't address."

• One example of this process is a tool to bring existing applications (such as SAP, PeopleSoft, and IBM CICS programs) into the Web-services arena with XML adapters. "A big issue for us is how to retrofit," Engstrom says. "Because the applications running today are not going away."

Service Broker

Service Provider

Publish

Service User

Find

Access

Dvoy Web Services

Service Broker

Service Provider

PublishFind

BindServiceUser

getMeasureList->MeasureList

getMeasureRec(MeasureID)->MeasureRec

getDimensions(MeasureID)->DimIDList

getDimTable(MeasureID, > DimTable

getFactTable(MeasureID, FactQuery> FactTable

Web Services Enabled by Standards

Web Services operate ‘on top’ of many layers of Internet standards, TCP/IP, HTTP…

Web services also the use an array of its own standards - some still in development.

The data sharing standards for are to facilitate discovery, description and invocation

Discovery

UDDI

Disco

Description

WSDL, XSchema

XSD, XSI

Invocation

SOAP

XML

On top of these Internet and Web Service Standards, we will need to develop our own:

Naming conventions

Metadata standards

Uniform database schemata, etc

Major Service CategoriesAs envisioned by Open GiS Consortium (OGC)

Service Category Description

Human Interaction Managing user interfaces, graphics, presentation.

 

Info. Management Managing and storage of metadata, schemas, datasets.

 

Workflow Services that support specific tasks or work-related activities.

 

Processing Data processing, computations; no data storage or transfer  

Communication Services that encode and transfer data across networks.

 

Sys. Management Managing system components, applications, networks (access).

 

0000 OGC web service [ROOT]1000 Human interaction1100 Portrayal1110 Geospatial viewer1111 Animation1112 Mosaicing1113 Perspective1114 Imagery1120 Geospatial symbol editor1130 Feature generalization editor1200 Service interaction editor1300 Registry browser2000 Information Management2100 Feature access2200 Coverage access2210 Real-time sensor2300 Map access2400 Gazetteer2500 Registry2600 Sensor access2700 Order handling3000 Workflow3100 Chain definition3200 Enactment3300 Subscription4000 Processing4100 Spatial4110 Coordinate conversion4120 Coordinate transformation4130 Representation conversion4140 Orthorectification4150 Subsetting4160 Sampling4170 Feature manipulation4180 Feature generalization4190 Route determination41A0 Positioning4200 Thematic4210 Geoparameter calculation4220 Thematic classification4221 Unsupervised4222 Supervised4230 Change detection4240 Radiometric correction4250 Geospatial analysis4260 Image processing4261 Reduced resolution

generation4262 Image manipulation4263 Image synthesis4270 Geoparsing4280 Geocoding4300 Temporal4310 Reference system

transformation4320 Subsetting4330 Sampling4340 Proximity analysis4400 Metadata4410 Statistical analysis4420 Annotation5000 Communication5100 Encoding5200 Format conversion5300 Messaging6000 System Management

OGC code

Service class

SOAP and WSDL

SOAP

Envelope for message description and processing

A set of encoding rules for expressing data types

Convention for remote procedure calls and responses

A binding convention for exchanging messages

WSDL

Message format

Ports