Home » miscellaneous » Sandbox

Composite Bundles

1. Overview

The OSGi framework supports deploying bundles into a flat and basically globally bundle space. The idea behind this approach can be summarized as, "the deployed set of bundles is your application configuration." This approach has performed well over the years; however, as OSGi technology is used in more and more complicated scenarios, this approach is not always sufficient. For example, when trying to run multiple applications in a single framework instance or when applications become so large that sets of bundles start mapping onto logical subsystems. In these types of situations, it is possible for the configurations of different applications or subsystems to interfere with each other. To address such issues, this proposal introduces a composite bundle concept built on top of virtual bundles.

The two main goals of this proposal are:

  1. To provide an isolation mechanism for groups of bundles, while still allowing collaboration among those groups.
  2. To implement this mechanism as a layer above the framework.

The remainder of this proposal describes the technical approach for achieving these two goals.

2. Use cases

Some potential use cases for composite bundles:

This list is not intended to be exhaustive.

3. Terminology

The following terms are used in this document:

4. Technical approach

The overall technical approach is to use the virtual bundle concept (proposed separately) to manage composite bundles as a layer above the framework. This proposal forgoes an API-based approach to support a simple, declarative approach. The technical approach is divided into two halves:

  1. A simple composite model that supports only package and service sharing.
  2. A transparent composite model that extends the simple composite model with bundle sharing.

NOTE 1: The purpose is not to actually treat the two models as separate entities, but to more clearly illustrate the extra cost in complexity by supporting transparency.

NOTE 2: Although this document is API-less, the goal is not to completely rule out any API, but to keep things simple until some real-world experience is gained at which point API could potentially be introduced.

4.1 Simple composite model

The simple composite model supports installing groups of bundles as a single composite bundle with the ability to import packages/services from the parent framework into the composite bundle and to export packages/services from the composite bundle to the parent framework. The lifecycle of the constituent bundles are managed by the lifecycle of the composite bundle in the parent framework.

4.1.1 Simple composite description

A composite bundle is declared using a set of manifest-like headers, which is familiar to bundle developers and fits well with the virtual bundle proposal, where virtual bundles are installed with a given set of headers. The following headers are reused for declaring a composite bundle:

From the parent framework's perspective, the composite bundle is a normal bundle importing/exporting packages and services. Internally, the imported/exported packages and services are mapped onto and made available to the composite bundle's constituent bundles by the composite layer. A new header is introduced to declare the composite bundle's constituent bundles:

The constituent bundles of a composite bundle are isolated from the other bundles in the parent framework. In other words, they can only see their sibling constituent bundles and bundles in the parent framework can only see the composite bundle, not its constituent bundles.

To simplify mapping a composite bundle's exported packages to its constituent bundles, this proposal introduces a new from directive for Export-Package, which is used to specify the symbolic name of the providing bundle. Consider the following composite declaration:

Bundle-ManifestVersion: 2
Bundle-Name: Paint Program
Bundle-SymbolicName: org.foo.paint.composite
Include-Bundle: \
 file:/Users/rickhall/Projects/book-trunk/code/chapter04/paint-example/bundles/shape-4.0.jar, \
 file:/Users/rickhall/Projects/book-trunk/code/chapter04/paint-example/bundles/paint-4.0.jar, \
 file:/Users/rickhall/Projects/book-trunk/code/chapter04/paint-example/bundles/circle-4.0.jar, \
 file:/Users/rickhall/Projects/book-trunk/code/chapter04/paint-example/bundles/square-4.0.jar, \
Export-Package: org.foo.shape; from:=org.foo.shape.bundle; version="4.0"
Import-Package: org.osgi.service.log; version=1.0.0
Import-Service: org.foo.shape.SimpleShape

This composite contains five bundles and exports org.foo.shape from the bundle with the symbolic name org.foo.shape.bundle. Further, it also imports the log service package and any services implementing the org.foo.shape.SimpleShape interface from the package it exports.

These declarative headers define the entire capabilities of a simple composite bundle. To summarize, these capabilities are: containing bundles, importing/exporting packages, and importing/exporting services.

NOTE 1: It is not clear if Import-Package should essentially support an "export as" directive where the composite creator explicitly specifies how the imported package gets converted to an export internally or if this should be somehow automatically derived from the actual injected wire. If the latter, then this relates to the rich wiring section in the open issues.

NOTE 2: Any "uses" constraints for the composite bundle's exported packages must be specified in the metadata, which will be used by parent framework to ensure consistency like normal.

4.1.2 Simple composite lifecycle management

Since composite bundles are implemented as virtual bundles, access to their content and portions of their lifecycle are controlled by an external manager. This section describes various lifecycle management issues for simple composite bundles. Composite manager

The composite manager results from the use of virtual bundles and is largely responsible for realizing the capabilities embodied in the composite description. This means it is the composite manager's responsibility to:

The precise approach the composite manager uses to accomplish these responsibilities is not specified, but one potential approach is for the composite manager to create a separate framework instance for each composite bundle. Another approach would be to create a static wiring of the constituent bundles and just mimic framework behavior for them. Installing composites

If an "install hook" is introduced in the virtual bundle proposal, then the composite manager can use it to seamlessly install composite bundles via the BundleContext.installBundle() method like a normal bundle. If install hooks are not proposed, then it could provide a simple service for installing composites. A composite is installed with a complete composite description, which forms the manifest of the installed virtual bundle. As such, composite installation is effectively atomic from the perspective of the parent framework. Resolving composites

The composite bundle's wires for its imported packages are injected into the composite's virtual module by the framework, like for all virtual bundles. The composite manager uses these wires for delegation purposes out to the parent framework for the constituent bundles. If a composite bundle is resolved, then it is possible to load classes from it.

NOTE: Resolving a composite bundle could actually be combined with some sort of verification step, where the manager verifies whether or not the composite bundle can provide what it says it can provide. It is not clear if this needs to be specified, since such verification does not happen for normal bundles. In other words, it is a reasonable approach to just trust the metadata. Starting and stopping composites

Starting a composite bundle starts all internal constituent bundles. Likewise, stopping a composite bundle stops all constituent bundles. Composite bundles do not have user-defined activators, although the composite manager may make use of an activator. For active composites, the composite manager must provide constituent bundles access to services imported from the parent framework and must make exported services available in the parent framework. Conversely, when a composite bundle is no longer active, it must stop providing access to these services.

After a composite bundle is stopped, it should remain resolved and continue to provide access to its exported packages.

NOTE: Since this proposal does not propose an API to expose the constituent bundles, individual lifecycle manipulation of the constituent bundles is not expected. To keep things simple, constituent bundles are either active or not based on the state of their composite bundle. If more fine-grained control is required it would be possible. Starting and stopping individual constituent bundles would offer no real issue, although allowing them to be refreshed or uninstalled might cause the composite manager to force the outer composite bundle to refresh if the export signature is impacted. Updating a composite

When a composite bundle is updated, the composite manager must continue to support the associated virtual module; i.e., it must still be possible to load classes from it. At the point in time when the bundle is refreshed and returned to the installed state, then the composite manager can dispose of the old virtual module. If the composite was updated to a normal bundle (or a different kind of virtual bundle), then the composite manager will no longer manage it. On the other hand, if was updated to another composite bundle, then the composite manager will reinstall a new virtual module for it. Uninstalling a composite

When a bundle is uninstalled, that does not mean that it is no longer in use by the framework, since it must still be possible to load classes from it. Unfortunately, the framework provides no additional callbacks or state changes to notify when it is really done with an uninstalled bundle. As a result, if a composite bundle is uninstalled, the composite manager must immediately refresh it to perform proper clean up, since it will not get a later lifecycle callback when the uninstalled bundle is eventually refreshed.

NOTE: This could be improved with a VirtualModule.dispose() method, which would be invoked by the framework to indicate when it was done with the virtual module. Refreshing a composite

When refreshing a composite bundle, all constituent bundles are refreshed and the composite bundle returns to the installed state. Following normal framework behavior, any bundles depending on the composite bundle will also be refreshed. Likewise, if the composite bundle depends on another bundle being refreshed in the parent framework, then it too will be refreshed. Relationship to composite manager lifecycle

Since the composite manager manages all aspects of the composite's content, its active lifetime scopes its managed composites. In other words, if the composite manager is stopped, then it explicitly causes all of its managed composites to refresh and return to the installed state.

4.2 Transparent composite model

The simple composite model fits fairly well within the constraints of the OSGi framework since it aligns well with the concepts embodied in the original OSGi specification (i.e., packages and services). However, some use cases may require support beyond these original concepts. For such cases, this proposal defines a transparent composite model that extends the simple composite model to include additional support for provided and required bundles at the expense of added complexity.

4.2.1 Transparent composite description

Transparent composite bundles can require bundles using the following header:

The composite manager makes the required bundles available to the composite bundle's constituent bundles. A new header is defined to provide access to constituent bundles in the parent framework:

The provided bundles will be manifested in the parent framework as virtual bundles themselves. This means that in addition to the composite bundle in the parent framework, there will ultimately be additional virtual bundles installed by the composite manager for each provided bundle; since this is related to the transparent composite bundle lifecycle, more details will be present in the next section.

4.2.2 Transparent composite lifecycle management

To provide access to constituent bundles into the parent framework, the composite manager must proxy provided constituent bundles as separate virtual bundles in the parent framework. This complicates lifecycle management since it creates separate points of control for the composite bundle and it also complicates maintaining class space consistency for clients of the provided bundle. This section discusses these issues in more detail.

NOTE: Depending on how composite bundles are implemented, these issues may also apply to required bundles inside the composite bundle. Two-phase resolve

When a composite providing access to a constituent bundle is first installed, the provided bundle cannot be made available immediately. Once the composite bundle is resolved (i.e., the composite's associated virtual module is injected with its wires), then the composite manager can install a virtual bundle proxying the provided bundle. At this point, the proxy provided bundle is available for use by other bundles in the parent framework, although it will still be in the installed state in the parent until someone actually causes it to resolve. However, resolving the proxy provided bundle technically has no real effect since it is actually resolved internal to the composite. Maintaining class space consistency

If the packages exported by a provided bundle do not have uses constraints or if the uses constraints are confined to other packages exported by the provided bundle itself, then the proxying described in the last section is sufficient to provide access in the parent framework. On the other hand, if the provided bundle's exported packages have uses constraints on packages imported by the provided bundle, then this poses a potential issue for clients of the proxy provided bundle in the parent framework. The provided bundle's uses constraints must be modeled in the parent framework so it can maintain class space consistency.

To achieve this, when such a situation is detected, the composite manager must install an additional virtual bundle in the parent framework that acts as a uses constraint proxy bundle by exporting any packages imported by the provided bundle that are part of a uses constraint. Additionally, the proxy provided bundle must be generated such that it explicitly imports its packages from the uses constraint proxy bundle in the parent framework. This will ensure the parent framework correct observes the uses constraints and enforces them for potential clients. To clarify via an example, consider the following hypothetical (and completely irrational) composite description:

Bundle-ManifestVersion: 2
Bundle-Name: HTTP Composite
Bundle-SymbolicName: org.foo.composite.http
Include-Bundle: \
 http://www.foo.org/org.foo.http.jar, \
Provide-Bundle: org.foo.http

This composite contains two constituent bundles and provides access to the org.foo.http constituent bundle. Assume the org.foo.http bundle has the following metadata:

Bundle-ManifestVersion: 2
Bundle-Name: HTTP Service
Bundle-SymbolicName: org.foo.http
Import-Package: javax.servlet, javax.servlet.http
Export-Package: org.foo.http; uses:="javax.servlet", org.foo.http.util

In this case the org.foo.http imports two packages (javax.servlet and javax,servlet.http), presumably both come from the other constituent bundle in the composite, and exports two packages (org.foo.http and org.foo.http.util), where org.foo.http has a uses constraint on the imported javax.servlet package. To properly proxy this provided bundle, the composite manager would install a virtual bundle in the parent framework that looked like this:

Bundle-ManifestVersion: 2
Bundle-Name: HTTP Service
Bundle-SymbolicName: org.foo.http
Import-Package: javax.servlet; bundle-symbolic-name="USES.12312"
Export-Package: org.foo.http; uses:="javax.servlet", org.foo.http.util

This proxy provided bundle enables access to the exported packages of the original provided bundle and correctly models its uses constraints on a second uses constraint virtual bundle. The composite manager generates the proxy provided bundle metadata so that it can only resolve to the generated uses constraint virtual bundle, which would look something like this:

Bundle-ManifestVersion: 2
Bundle-Name: HTTP Service Uses Constraints
Bundle-SymbolicName: USES.12312
Export-Package: javax.servlet

Since there is no uses constraint on the javax.servlet.http package, then it need not be provided by the uses constraint bundle. On the other hand, if there was then it would need to be exported as well. Further, if these exported packages had uses constraints on other imported packages, then these would need to be modeled as well. However, these could be modeled by simply having the uses constraint bundle export them in addition to the original exports (effectively reexporting the imported packages).

NOTE: The approach to proxy the provided bundle as two bundles (the proxy bundle and the uses constraint bundle) is necessary to maintain the semantics of Require-Bundle which only gives access to the target bundle's exported packages. A different, but not completely consistent approach is to just proxy the provided bundle and turn all of its imports into exports so that all uses constraints are satisfied by the proxy itself. The main downside of this approach is that client bundles in the parent framework would end up with greater visibility of packages than if they required the bundle directly.

If a composite provides multiple bundles, then shared and potentially conflicting packages among the provided bundles would need to be correctly modeled. For each provided bundle, a package space would need to be calculated for any imported package participating in a uses constraint. Any common packages with the same provider among the provided bundles would need to be modeled on a common uses constraint bundle in the parent framework, where the same package coming from different providers would need to be modeled with a separate uses constraint bundles. Non-overlapping packages could be lumped into a single uses constraint bundle. This algorithm would be non-trivial, but since it is just walking existing wires, it should not suffer from similar performance issues like the resolver algorithm.

One special case to note, if the provided bundle has a uses constraint on an imported package that was actually imported from the parent framework via the composite description, then it is not necessary to model this import in the parent framework since it already exists. For this case, the generated proxy provided bundle must simply explicitly import the package from the original bundle in the parent framework.

NOTE: This approach requires richer wiring information as discussed in the open issues. Lifecycle of proxied bundles

As discussed, the support for providing bundles results in the composite manager installing proxy bundles for the provided bundle and its uses constraints in the parent framework in addition to the original composite bundle. This raises questions about the lifecycle of proxied bundles.

The lifetime of the proxied bundles is dependent on the resolved lifetime of the associated composite bundle. If the composite is refreshed, then the provided bundles should be uninstalled and refreshed. (Technically, it would be possible to simply refresh them and leave them unresolvable.)

Performing individual lifecycle operations on the proxied bundles should function like normal in the parent framework, but should have no impact on the internal constituent bundles of the composite. For example, you can start, stop, and even uninstall provided bundles, but this just impacts the state of the bundles in the parent framework, which may render them unresolvable.

5. Open issues

5.1 Rich wiring information

Currently, the wiring information provided by the virtual bundle proposal has been kept purposely simplistic. To fully implement aspects of composites, like requiring/providing bundles, it is necessary to get richer information from the wires, such as the type of capability. Further, the wiring information needs to be at the module-level (i.e., bundle revision level), not at the bundle level. The refactoring of the Package Admin API addresses some of these issues, but not all of them.

6. Considered alternatives

Scoping approach

Another potential approach for providing similar capabilities is to try to use virtual bundles to implement a scoping approach. Scoping can be modeled reasonably well as manifest rewriting (i.e., mandatory attributes and renaming). If it were possible to install bundles and "lock" them in the INSTALLED state, then these bundles could be used like templates for creating scopes via manifest rewriting in a virtual bundle. The same template bundle could be copied into different scopes using different virtual bundles in the different scopes or could be shared among scopes by appropriately rewriting the metadata. This approach could also scope the service registry, since it would be possible to inject proxied bundle contexts into the scoped bundles (via their virtual bundle wrapper) that only show services in the appropriate scope.

The biggest issue here is achieving complete fidelity with the OSGi specification for handling of bundles. The virtual bundle mechanism would need to include support for dynamic imports, fragments, and lazy activation. All of these are potentially feasible, but would need to be fleshed out.

Rev. 1700393 by cziegeler on Tue, 1 Sep 2015 06:04:06 +0000
Apache Felix, Felix, Apache, the Apache feather logo, and the Apache Felix project logo are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.