introduction to osgi
TRANSCRIPT
Introduction
● Elegance always matters, especially when creating software – and modularity lies at its very heart
● In this presentation, we'll briefly introduce the fundamentals of OSGi - a dynamic module system for the Java Virtual Machine
● The author would like to thank:– Professor Paolo Bellavista, for his kind and valuable
advice and suggestions
– Neil Bartlett, author of the beautiful, free, Creative Commons-licensed book OSGi in practice
Outline
● Why the traditional Java class loading mechanism is far from being perfect
● Overview of the module system provided by OSGi● Details of OSGi's class loading● Creating OSGi bundles● First steps in Apache Felix● Example multi-module application with Scala +
JavaFX + OSGi + Gradle
Section 1
Why OSGi?
Modularity
● Modularity is paramount for developing complex systems, as it fosters:
– Separation of concerns
– Parallel development
– Design simplicity
– Code reuse
– Ease of maintenance and repair
What is a module?
● The main defining attributes of a module are:– Self-contained: from the outside, a module must
appear as a logical whole
– Highly cohesive: a module should have just one, well-defined purpose, and achieve it
– Loosely coupled: a module should know nothing about the implementation of the modules it depends on – it must only rely on their public interface
● The JVM employs JAR files - archives containing bytecode and resources. Are JARs modules?
JAR hell
● JAR files are not modules! For several reasons:– JARs do not exist at runtime: when searching for a
class, the JVM performs a linear scan of the CLASSPATH, starting from its left-most item → this is a very inefficient and unreliable process – in particular, a class might load not a companion class, but a totally unrelated - yet preceding - class in the CLASSPATH
– JARs, by default, contain no explicit declaration about dependencies (except a file-system based, very specific, Class-Path declaration)
JAR hell (2)
– JARs are not versioned – it's the developer that must find a way to track versions – for example, stamping them in every JAR file name
– There is no information hiding between JARs: once a class is declared public, it can be accessed by all the other classes in its own JAR, but also by any other class in the system – this can lead to potential side-effects
● So, apart from the fact that they keep files together, for example simplifying Internet downloads, JAR files are not modules – at all
What about Gradle, Maven, Ivy...?
● Build systems are brilliant – they automate and extremely simplify the build process, hiding an immense deal of complexity – e.g., they introduce artifact versioning, track down transitive dependencies and perform file-based JAR versioning
● Alas, such versioning is static, a contract respected by the tools only: the JVM keeps working with its linear CLASSPATH, making JAR files lose their boundaries
● Anyway, build systems are precious compile-time complementary tools for OSGi
JVM – Class loading
● The JVM loads classes by means of class loaders, instances of java.lang.ClassLoader subclasses
● ClassLoader has 2 duties:
1)Find the bytes of a class – which could reside anywhere
2)Turn them into a Class
● We can only override methods to customize task (1)● Task (2) is performed by defineClass(), a native
final method – so we are unable to change it
Class loaders – The delegation tree
● Class loaders are organized in a hierarchical structure: each class loader has a parent, except the native class loader, which is the root of the tree
● A class loader first asks its parent to load a class – and tries to locate such class only if the parent fails (parent-first delegation)
● This is due to security reasons – the loading of core Java classes is enforced to be carried out by the native class loader, and not by a possibly malicious one
Java SE – Typical class loader tree
Native class loader
Extension class loader
Application class loader
Java core classes(rt.jar)
Extension JARs (in $JAVA_HOME/lib/ext)
These libraries are available to all apps
App-specific JARs
Java EE traditional class loader tree
Native class loader
Extension class loader
Application class loader
Java core classes(rt.jar)
Extension JARs (in $JAVA_HOME/lib/ext)
Server-level shared JARs
EAR 1 class loader
WAR 1 c.l. EJB 1 c.l.
EAR 2 class loader
WAR 2 c.l. EJB 2 c.l.
(...)
JAR hell: unpredictable, unreliable
● Not only might a class happen to reference an unknown class contained in another JAR file, in lieu of the expected companion...
● ...but such misbehavior might occur at any moment, perhaps just under certain conditions
● What's more, the resulting errors are usually not trivial (LinkageError, ClassNotFoundException, NoClassDefFoundError)
● Finally, different compile-time versions of a JAR just can't coexist at runtime
OSGi
● OSGi is “The dynamic module system for Java™”● “The OSGi Alliance is a worldwide consortium of
technology innovators that advances a proven and mature process to create open specifications that enable the modular assembly of software built with Java technology. Modularity reduces software complexity; OSGi is the best model to modularize Java.” - https://www.osgi.org/
OSGi bundles
● Bundle = a module in OSGi. Bundles are plain JAR files with just additional metadata, declared in the standard META-INF/MANIFEST.MF file
● Therefore, OSGi is fully backward-compatible● Bundle metadata:
– Symbolic name (required)
– Version (optional, but advised)
– Description
– List of imported and exported packages
– …
Immediate benefits of OSGi
● Each bundle has its own class loader● Dependencies are explicit, by means of
import/export declarations for packages● Bundle encapsulation: only exported packages are
visible to other bundles● Multiple versions of the same bundle – and even of
the same package – can coexist at once within an application
● OSGi is dynamic: bundles can be installed, updated and uninstalled without restarting applications
OSGi: from trees to graphs
● Traditional Java class loaders compose a tree – with Java EE apps structured like self-contained silos – involving scalability issues, in addition to all the drawbacks imposed by a CLASSPATH within the boundaries of each app
● The OSGi framework reads the declarative import/export package statements and dynamically connects (wires) bundles – bypassing the inefficient, non-scalable global sequential scan
● It's the same as trying to retrieve an element by indexing a hash map in lieu of scanning a huge list
The OSGi graph – Further advantages
● There is no more a node-subnode relationship between class loaders – only a network of peers, that can be providers of some packages and consumers of other packages
● In case of missing dependencies, an error occurs immediately, with detailed explanations
● In the graph, dependency links express the consumer/provider relationship for single packages: that is, bundle A can depend on bundle B for package P1, and depend on bundle C for package P2
Part 2
Bundles in detail
Fine-grained versioning
● Bundles are versioned, just like Gradle or Maven artifacts...
● ...but – which is most important - packages are versioned as well!
● A package version is arbitrary, provided that it's well-formed
● In OSGi, dependencies are mainly expressed in terms of packages – any bundle could provide a package required by another bundle
Version syntax
● A version string is subject to the following syntax: MAJOR[.MINOR[.MICRO[.QUALIFIER]]]
● MAJOR, MINOR and MICRO must be numeric, and they are sorted accordingly, as expected
● QUALIFIER is a string: whenever the other components are all equal, QUALIFIER is checked via string comparison. There are a few qualifier patterns:
– Development phase: alphaXX/betaXX/final
– Timestamp: YYYY-MM-DD_HHMM => scales better
Import / export – Headers & Versions
● An exported version must be exact or missing (defaulting to: 0.0.0) → it is like a point in the space
● An imported version must be a range or missing (defaulting to: any version) → it is like a segment
● This design greatly increases flexibility, as a module does not necessarily require recompilation whenever a new version of an imported package is available - provided that only the implementation has changed, not the shared contract
Version ranges
● Version ranges are expressed as mathematical ranges:
– [V1,V2] → from V1 to V2, both included
– (V1, V2) → from V1 to V2, both excluded
– [V1, V2) → from V1 to V2, only V1 included
– (V1, V2] → from V1 to V2, only V2 included
● Special cases:– V1 (where a range is expected) → V1 or any later
– [V1,V1] → V1 only
Bundle encapsulation
● All the packages in a bundle are private by default, to minimize namespace pollution and to foster encapsulation
● Bundles must keep implementation as hidden as possible, only exposing interface information
OSGi – Class loading in detail
Bundle A
Bundle class loader
Parent class loader
Bundle B
Bundle C
(1) java.* and boot delegation(2) Imported packages
(3) Required bundles
Internal types
(4) Internal classes
Fragment X
(5) Fragments
OSGi – The application class loader
Every resolved bundle has its own class loader, not relying on the application class loader - which is still employed by OSGi in 2 cases:
1)Only the native class loader – at the top of the standard class loader tree - can load java.* packages – otherwise, a SecurityException would be thrown
2)A few classes might assume that parent delegation is in use – to avoid breaking compatibility, OSGi defines a bootdelegation parameter – packages that will be loaded just by the parent class loader. It should be used for very particular requirements
The system bundle● The parent class loader transparently loads java.*
and bootdelegation classes, as usual● However, the JRE and its CLASSPATH include
several packages – for example, javax.swing.* - which are not automatically loaded: in OSGi, they must be explicitly imported – and it is the system bundle that exports them
● System bundle = meta-bundle encapsulating the OSGi framework at runtime; its id is always 0
● The packages exported by the system bundle are not standard, but they can be enforced via properties: org.osgi.framework.system.packages[.extra]
OSGi – Loading packages
● Class loading is delegated, whenever possible, to the class loaders of other bundles, transitively, following the dependency trail (called wiring)
● Imported packages prevail over local packages, for:– Optimization: a class is globally loaded just once –
and not once per bundle
– Coherence: because class identity = <class FQN> + <class loader>,having a versioned FQN loaded by just one class loader prevents cases of ClassCastException
● A bundle can even export its imported packages
Version resolution
● Different versions of the same package might be available – but a bundle can export at most one
● Whenever an import requested by a bundle – via the Import-Package header - must be resolved:
1)If one matching version - that is, one version belonging to the range of the import declaration – is available, it is automatically wired
2)If two or more matching versions are available, the highest of them is chosen
3)Otherwise, it's the bundle with the lowest id that provides the package
Required bundles
● Via the Require-Bundle header, a bundle can directly depend on another bundle, not on packages
● Depending on bundles is definitely discouraged:– The efficient granularity at the heart of OSGi is lost
– The architecture becomes fragile, as splitting a bundle could cause unresolved runtime missing dependencies, not eagerly detected by the OSGi framework
● If a package cannot be found via import/export wiring, such package is looked for in the Export-Package header of the required bundles
Internal types
● A bundle is a regular JAR file, so it includes types and resources
● Packages can be private or exported – but they can all be used by the bundle itself, of course
● Packages are private by default, unless they are listed in the Export-Package header
– NOTE: always check your tools, as they might actually export packages by default
– Furthermore, tools might create a Private-Package pseudo-header, ignored by the OSGi framework
Fragments
● Fragment = set of types and resources that cannot exist on its own – it needs to be attached to a bundle, providing additional types and resources.
● Fragments are especially valuable in a few common situations:
– Multi-platform support: usually, fragments integrate a cross-platform bundle by adding a few platform-specific implementations
– Resource bundles: to provide small customer-specific variations – for example, logos, icons, or localization
Bundle dependencies
● Dependency = Set of assumptions made by a software component about its context
● Bundles can require 3 types of dependencies:
1)Execution environment specification
2)Package imports
3)Bundle requirements
● OSGi even supports simultaneous bundle resolution, thus enabling circular dependencies
Environment specification
● Bundle-RequiredExtensionEnvironment header● Describes the expected Runtime Enviroment, for
example JavaSE-1.8.● JRE classes are extended, version after version, with
new methods and properties – which, if called in the context of an older JRE, would cause an unpredictable NoSuchMethodError
● The solution is twofold:
1)Compile your code using exactly the target JDK
2)Enforce the requested JRE with the OSGi header
Package imports
● The header for declaring required packages is:
Import-Package: <package>[;version=<range>][,<package>[;version=<range>]][,....]
● version is optional and, if omitted, means “any version”
● Employing a punctual version means “that version or later”
Bundle requirements
● Requiring a bundle is really discouraged● Anyway, the header syntax is:
Require-Bundle: <bundle symbolic name>[;bundle-version=<version>][,<bundle symbolic name>[;bundle-version=<version>]][,...]
● Versioning works like package versioning● Bundle-Version is the related header, which has an
important side-effect: it enables multiple, different versions of a bundle to be loaded at the same time
Bundle lifecycle
INSTALLED
RESOLVED
STARTING
ACTIVE
STOPPING
UNINSTALLED
Install
Resolve/Install
Start
Stop
Refresh/Update
Refresh/Update
UninstallUninstall
Bundle lifecycle - States
● INSTALLED: the bundle has been loaded, but nothing is known about its dependencies
● RESOLVED: the 3 kinds of dependencies have been resolved and the bundle is ready to run
● ACTIVE: the bundle is exporting packages, and its services are running
● UNINSTALLED: the bundle has been removed from the environment
● STARTING, STOPPING: temporary states, where the bundle activator is running
Further details on the lifecycle● A bundle can move from INSTALLED to
RESOLVED if its dependencies are RESOLVED or they can be transitively resolved at that moment
● Calling Start on an INSTALLED bundle moves it to RESOLVED, then to STARTING, and finally to STARTED - if dependencies are satisfied, of course
● Update loads a new version of a bundle● Refresh re-wires bundle dependencies after one or
more bundles have been updated● The state of a bundle can also be affected by the
state of bundles on which it depends
BundleActivator
● Every bundle can have a bundle activator – a class implementing the BundleActivator interface and registered with the following manifest header:
– Bundle-Activator: <Activator FQN>
● Its start() and stop() methods are called in the STARTING and STOPPING phases – so, they should be as fast as possible; however, for long-running tasks, an activator can spawn threads
● The bundle activator can reside in a private package● A new activator instance is created for every
STARTING / STOPPING pair
BundleContext
● BundleContext is a handle to access the framework● It includes methods for programmatically inspecting
and manipulating the current bundles:– Read metadata
– Change lifecycle state
– Install new bundles
– …
● A common way of obtaining a BundleContext is via BundleActivator's start(BundleContext context) method
BundleListener
● A bundle listener is called on lifecycle events triggered by any bundle
● It must implement the BundleListener interface● It can be registered an unregistered using
BundleContext's methods:– addBundleListener(...)
– removeBundleListener(...)
OSGi = Modularity + Services
● Up to now, OSGi might appear as a brilliant, package-level runtime extension of the versioning ideas introduced by build systems
● Actually, OSGi is much more - it provides a full-fledged lifecycle management for services and a rich API to publish, locate and consume them
● We won't deal with services in this presentation, but most books, as well as the reference documentation, describe them in great detail
Section 3
Creating OSGi bundles
Bundle = JAR + OSGi metadata
● To create a bundle, one needs to add standard key: value metadata to the manifest file, located at:
– META-INF/MANIFEST.MF within the JAR
● Bundle-SymbolicName is the only mandatory header ● Bundle-SymbolicName + Bundle-Version uniquely
identify a bundle in the OSGi runtime● The manifest file should never be manually edited,
as it must follow very specific JVM constraints● A tool is therefore needed: Bnd
Bnd
● Bnd can be:– A command-line program
– An IDE extension
– A plugin for a build system (Gradle, Maven, ...)
● No matter its form, bnd reads a list of directives (for example, in a .bnd file) and generates a full OSGi jar, with a compliant manifest file.
● You can include the usual OSGi headers, as well as syntactic shortcuts
Bnd: quick reference
● The full documentation of this tool is available on its website. We'll only deal with its basic principles
● Bnd supports all the OSGi headers, but shortcuts as well:
– * in a package name is a pattern-making character matching anything – for example, javafx.*
– A leading ! in a package name/pattern excludes it
● The Export-Package, Private-Package and Import-Package directives are paramount, and should be learnt; however, always check that your Bnd tools actually support them in a standard way
Bnd and Gradle
● Gradle supports OSGi by providing the osgi plugin, which internally employs Bnd
● To make a Gradle project OSGI-aware, use:– apply plugin: 'osgi'
● Bundle generation can be customized via Bnd:jar { manifest { instruction '<Directive>', ['<value>'[,'value2'...]] [, instruction '<Directive>', …] }}
Bnd in the JVM
● The JVM is now hosting more and more languages that, while introducing innovative syntax, still are able to compile to bytecode
● In particular, Scala (http://scala-lang.org/) is an extremely flexible, elegant, blended language, supporting both the OOP and the functional paradigm
● Bnd detects packages not in the source code, but in the bytecode – which makes OSGi compatible with the vast ecosystem of the new languages
Section 4
Introduction to Apache Felix
OSGi implementations
● Equinox: the reference implementation, a fundamental element of the Eclipse IDE
● Apache Felix: another widespread open source implementation
● … and many more! ^__^!● As long as the standard framework is employed, the
choice of the specific implementation depends on other factors (robustness, performances, simplicity, compatibility with other products, …) and does not have to be permanent
Why Apache Felix
● OSGi-compliant● Minimalist, simple, easy to learn● Employed in other enterprise-level Apache products
(Karaf, ServiceMix, ...)● Open source
Running Apache Felix
● Felix is distributed as a zip bundle, which can be downloaded from its official website
● Felix can be started:– Embedded in a Java application - it is a standard JAR
– As a standalone jar: cd to its directory, and run:
java -jar bin/felix.jar
NOTE: running directly from within bin will NOT work: Felix will search the current directory for a bundle subdir, for retrieving startup bundles
Gogo – Felix's command line
● When running Felix as an application, you will see a command line
● Type help to see the list of available commands● Type help <command> to see specific help● Every OSGi shell may have specific commands - for
example, in Equinox you might have to employ ss to show the list of bundles in the system, whereas in Felix's Gogo one employs lb
Common Gogo tasks
● List all the current bundles → lb● Install a bundle → install <URL>● Start an installed/resolved bundle → start <id>● Stop a running bundle → stop <id>● Update a bundle → update <id> [<new URL>]
Example project
● A small demonstrative project, called OSGi-Test, is available on GitHub. It's a multimodule Java SE application employing a variety of technologies:
– OSGi
– Scala
– JavaFX 8
– Gradle
● The repository is hosted on GitHub: https://github.com/giancosta86/OSGi-Test
Running the example project
● The suggested way to run the application is MoonDeploy, a minimalist, open source tool to automate browser-based software deployment (à la Java Web Start):
– https://github.com/giancosta86/moondeploy
– In this case, you just have to click and open the file App.moondeploy from the project's download area
● Otherwise, you can download the zip file, extract it and run one of the scripts from the bin sub-directory
● For further information, please refer to the example project's page on GitHub
Example project - Enhancements
● The GUI bundle directly depends on a Scala object provided by the hotswap-message bundle: this purposely demonstrates how reloading a bundle can reload its client bundles without stopping the app
● However, a drawback exists: a new main window is created! Since we want to depend on a static object, we need to reload and re-initialize the client bundle
● A more elegant solution would be to introduce an OSGi service, whose interface and implementation can be decoupled, in order to be able to change the latter without interfering with clients
OSGi in the real world
● OSGi is not an abstract concept - it is a general-purpose, widespread technology. Examples are:
– Eclipse: Equinox, OSGi's reference implementation, is developed as a core component of the IDE
– IntelliJ and NetBeans equally employ modular, elegant OSGi-based kernels
– Enterprise-level solutions, such as Apache Karaf and Apache ServiceMix, rely on OSGi
– The OSGi Alliance is a consortium of technology innovators actively employing OSGi
– Scala's core (scala-library) is an OSGi bundle
Final considerations
● This presentation briefly introduced a minimal, basic part of OSGi, without any claim of completeness but still showing how OSGi can prove an effective technology even at its fundamental core, greatly enhancing class loading – in terms of conceptual clarity, robustness and performances
● Much more could be said about:– Its dynamic aspects, revolving around the concept of
service and the classes ServiceReference, ServiceTracker and ServiceTrackerCustomizer
– The use of annotations, to reduce boilerplate code
Further references● OSGi - Official website: https://www.osgi.org/● OSGI – Wiki: http://wiki.osgi.org/● Equinox: http://www.eclipse.org/equinox/● Apache Felix: http://felix.apache.org/● Apache Karaf: http://karaf.apache.org/● Apache ServiceMix: http://servicemix.apache.org/● OSGi in practice:
http://njbartlett.name/osgibook.html● Professor Paolo Bellavista's website:
http://www.lia.deis.unibo.it/Staff/PaoloBellavista/