scribe insight 04 insight 7.9.0
TRANSCRIPT
Insight 7.9.0
New Features Deep Dive
4
Session Abstract
This session will cover details around the threading capability of Insight with non-queue based integrations, and how to apply this new feature. During the discussion you will hear detailed information regarding the processor grouping feature, such as, where it is available and how consultants should approach projects where they want to utilize this feature. We will also share best practices around new global connections such as moving from test to prod. During the discussion we will preview new features coming out this fall.
© 2015 Scribe Software Corporation. All rights reserved. 5
Insight 7.9.0 New Features Agenda
• Intro and Overview of New Features
• New Feature Deep Dives
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 6
Build
• Workbench provides UI for:
◦ Building data and logical transformations
◦ Testing
• Adapters connect and provide:
◦ Access to data and metadata
◦ Application-specific processing
Manage
• Platform provides automated processing but also much more
• It is how you save, manage, and communicate the parameters that define your integration environment.
◦ Source system knowledge
◦ Server resource considerations
◦ Message queueing and delivery
◦ Process multi-threading
◦ Target system knowledge
◦ Error handling and notifications
Insight Overview
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 7
Insight 7.9.0 New Features: Overview
• Global DTS connections
• Configure services during install
Usability/Install
• New multi-processor capabilities
Processing Control
• Message Processor Groups
Processing Control: Enterprise Focus
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved.
Insight 7.9.0 New FeatureGlobal DTS Connections
What’s New?
How it Works
Transporting DTS Files and Connections
What’s Next?
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 9
Global DTS Connections: What’s New?
• Connection storage moved from DTS files to internal database
◦ Single point of modification for connection changes
◦ All DTS connections with the same name point to the same set of connection credentials and parameters
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 10
Global DTS Connections: What’s New?
• Usage field in the Connection Manager is re-purposed
◦ Differentiate between global connections and DTS connections
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 11
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 12
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 13
Global DTS Connections: What’s New?
• Connection dialogs in the Workbench are reorganized to highlight:
◦ Global connection parameters – required to establish the connection
◦ DTS Connection parameters – options and settings
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 14
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 15
Global DTS Connections: How it Works3 Questions
• How did we change the storage of connection information?
• How are we handling connection transport for…
◦ DTS upgrades?
• Connection transport from legacy systems
• Conversion from DTS storage to database storage
◦ DTS sharing?
• Connection transport from peer Insight servers
• Movement of connections between Scribe databases
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 16
Global DTS Connections: How it WorksStorage
• Connection information is stored in the internal database.
• Two tables:
◦ DATAPROVIDERS – standard Scribe connection properties (username, password, connection type, etc.)
◦ DRSPROPERTIES – connection-specific properties (Dynamics CRM org name, etc.)
• Identifiers (GUID, connection name) stored in the DTS.
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 17
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 18
Global DTS Connections: How it WorksTransport
• Transporting DTS Files and Connections
◦ From older versions of Insight
• Change in the definition of “unique connection”
• De-duping and merging of connections
◦ From one environment to another (e.g. test to production)
• DTS files no longer contain connection information
• Re-creation of connection list in a new install
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 19
System Upgrades• Design Challenge
◦ Changing from unique/DTS to unique/Insight Server
• What makes a connection unique?
◦ Connection Parameters◦ Name
• Supporting Insight UI Changes
• User Technique
◦ “Clean” legacy connections
Environment Migrations• Design Challenge
◦ DTS no longer contains the connection parameters
• What makes a connection unique?
◦ GUID◦ Name
• Supporting Insight UI Changes
• User Technique
◦ Import/Export
Transporting DTS Files and Connections
20
Duplicate Connection Parameters
21
Duplicate Connection Name
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 22
Global DTS Connections: What’s Next?
• Connection Substitutions
◦ Unlimited connection substitutions during transport
◦ Target connection substitutions
23
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved.
Insight 7.9.0 New FeatureConfigure Service Credentials During Install
What’s New
How it Works
What’s Next?
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 25
Configure Service Credentials: What’s New?
• Ability to set your Scribe service user during install
◦ Prompted for Windows user credentials
◦ One less step to perform after the install
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 26
Configuring Service Credentials: How it Works
• Accessible when the internal database is installed using Windows authentication
• Credentials are not tested
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 27
Configuring Service Credentials: What’s Next?
• Ability to configure Windows user service log on when internal database installed using SQL authentication
◦ Services still need proper network rights
◦ Best practice is not to have services log on as LocalSystem
• Service user name preserved during upgrade
◦ Dialog auto-populated
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved.
Insight 7.9.0 New FeatureMore Multi-Processor Capabilities
What’s New?
How it Works
Tips and Tricks
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 29
Multi-Processors: What’s New?
• File-, time-, and query-based Integration Processes
◦ Could previously only run sequentially
◦ Can now run either simultaneously or sequentially
• New field determines if the DTS will run…
◦ Sequentially, through a single process
OR
◦ From the ScribeIn queue: multiple processors can be pulling DTS files out of the queue at the same time
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 30
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 31
Multi-Processors: How it Works
ScribeInQueue
EventManagerService
PublishersF,T,Q Processes
Serial Processor Group
Default Processor Group
MessageServerService
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 32
Multi-Processor: Tips and Tricks
• DTS process initiation is multi-threaded
◦ Row processing is not
• Route around processes with long run-times
◦ Periodic critical data refreshes don’t need to be blockers
• Resource considerations
◦ Feature addresses queueing (not resource) bottlenecks
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved.
Message Processor Groups
What’s New?
How it Works
Tips and Tricks
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 34
Message Processor Groups: What’s New?
• Available with Enterprise licenses
• Previously
◦ A Single message processor pool pulling from a single queue
• Now
◦ Multiple processor pools pulling from multiple queues
• New UI for defining groups and associated queues
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 35
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 36
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 37
Processor Groups: How it Works
ScribeInQueue
Serial Processor Group
Publishers
Default Processor Group
User Defined Processor Groups
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 38
Message Processor Groups: Tips and Tricks
• Guarantee simultaneous processing
◦ Process routing and priority
◦ Isolate processes
• Resource considerations
◦ Similar to multi-threading feature
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 39
Q+A
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 40
THANK YOU