|
Francisco J. Ballesteros Fabio Kon
Universidad Carlos III de Madrid University of Illinois at Urbana-Champaign
Sergio Arevalo Roy H. Campbell
Universidad Politecnica de Madrid University of Illinois at Urbana-Champaign
nemo@gsyc.escet.urjc.es sarevalo@fi.upm.es {f-kon,roy}@cs.uiuc.edu
We believe that, behind it all, the reason is the inadequate support that existing operating systems offer for mobility, replication, and DOS in general. The lack of proper support promotes ad-hoc solutions and discourages sharing.
We propose a different organization in which operating system and middleware cooperate to support efficient distributed object systems. A flexible microkernel is responsible for securely multiplexing distributed hardware. Adaptable middleware is built on top of this minimal network microkernel and implements a flexible object model that can be customized to meet application-specific requirements.
In distributed object systems such as Obliq, Emerald, CORBA, DCOM, and Java RMI, the middleware - and not the OS - implements the support for concurrency control, coherency protocols, data marshaling, method dispatching, object reference resolution, object state storage, etc. Different middleware implementations perform these operations in different ways with little help from the local OS. They all tend to use general purpose mechanisms which cannot be adapted to obtain a better fit for each of the different distributed applications.
Database systems tend to use ad-hoc solutions because OS and middleware are usually tailored for general-purpose applications. And databases behave very differently from ``general-purpose'' applications. Their data access patterns, for example, differ from the ones encountered in traditional file systems. Therefore, database developers are forced to implement their own mechanisms for data storage and retrieval, scheduling, concurrency, consistency, etc. In distributed environments, the discrepancies are maintained, distributed databases utilize their own algorithms for replication, communication, and mobility.
The same problem occurs in other fields like parallel numeric applications [5], distributed multimedia, and distributed real-time.
On one hand, many recent approaches consider support for general-purpose DOS applications [3,10,13]. They do not focus on providing basic mechanisms that could be shared to implement different distributed object models. For example, applications might benefit from relaxing concurrency control and enforcing state reconciliation recovery mechanisms; but they might want to do the opposite! Most existing systems (experimental or not) tend to enforce an entire consistency model which does not tolerate adaptations.
On the other hand, middleware supplies fine-grain abstractions such as object references, state storage, threads, and events which are usually built on top of heavy-weight OS abstractions like virtual memory, persistent stores, processes, files, and network connections. This leads not only to duplication of work but also to space and time overhead.
In summary, the OS supplies its own abstractions (which differ from OS to OS, i.e., from node to node) and the middleware reinvents them [9] to provide uniform and convenient distributed services. It follows that these abstractions differ from one middleware system to another.
kernel architecture
(DAMN1) in which applications
use local resources as a cache for network-wide resources
[1].
Following this philosophy, we are proposing a framework (see figure 1b) in which the work is performed either by the minimal OS or by flexible middleware [11]. Both layers cooperate to support customized distributed object systems so that specific issues can be addressed on a per application basis and different approaches can coexist. Note the difference with respect to the implementation of conventional middleware systems on conventional operating systems (figure 1a).
Applications running with the help of our middleware will not only be able to run on a highly-optimized microkernel environment but will also be able to run on and interoperate with traditional systems such as Solaris and Windows. The services provided by our minimal OS can be easily ported to commercial OSes. This can be accomplished either by modifying the commercial OS to export a subset of their available physical resources (as shown in figure 1c) or by writing libraries that would act as an adaptation layer.
In section 2, we discuss briefly the problems with current approaches for supporting mobility and replication. We present an approach that enables applications to customize system behavior optimizing its performance and meeting specific requirements. Section 3 shows how the proposed framework addresses these issues. Section 4 discusses related work. We conclude with a brief note on the current implementation status and future work (section 5).
The characterization of object mobility and replication can be drawn from aspects such as naming, protection, reference resolution, transport protocols, object state, and migration decisions. We, now, discuss how these aspects are handled by traditional DOS implementations pointing at their inefficiencies.
Object Naming is provided by the middleware, duplicating the work performed by the OS naming system. Distributed objects usually employ high-level, relocatable references which are translated to logical names known to the OS. The latter are then, once more, translated to physical names.
We propose to handle logical naming entirely inside the middleware. Should the OS support names for distributed physical resources, abstract resource (i.e., object) names can be implemented without duplication of efforts.
Protection is usually enforced by the object run-time system because mobile objects share the run-time with local native objects and are sometimes untrusted (see figure 1a). Another layer of protection is found in the underlying OS. In spite of using two different protection layers, mobile agent systems are still insecure and have raised new security issues [4]. Besides, databases might want coarse-grain ACL-based access control (e.g. before a whole query) and agent systems could prefer fine-grain capability-based access control: different security policies must coexist.
We propose that the OS cooperate with the middleware to protect resources. If the middleware can select the protection policy for each physical resource it is granted, mobile objects can be assigned their own physical resources and protect them even from the hosting environment (see figure 1b). The middleware can then implement abstract resource protection based on the physical protection provided by the OS.
Reference Resolution and method invocation in mobile and replicated objects is complicated because traditional OS facilities are not aware of migration and replication. Distributed references as provided by existing OSes are usually too heavy-weight to be used by middleware systems. Besides, there is no OS support to handle references while and after objects move. Therefore, the middleware must encapsulate its objects to turn them into mobile entities [9]. Such encapsulation requires extra work and leads to performance degradation.
If the OS supports call redirection, calls to objects in a problematic state (i.e., moving, or with inconsistent state) can be delegated to the middleware. It can then take control and keep client objects unaware of location or state changes, without reimplementation of IPC services.
Transport, location, and concurrency control mechanisms are
implemented by the OS and, again, by the middleware.
Usually, when mobile objects are implemented over distributed OSes or
kernel-based OSes, all applications share the same network protocol
stack. In many cases, location, concurrency control, and coherency
protocols are also fixed and cannot be adapted on a per-application
basis.
Such a fixed scheme is not likely to suit all kinds of applications.
Database management systems, for example, tend to maintain knowledge about
the physical location of its data. However, more typical applications
tend to rely on some underlying location mechanism.
As another example, a database using transactions could interpret a
network disconnection as a failure and abort affected transactions.
A mobile application, on the other hand, could see the disconnection
as a transitory state.
We propose that the OS (and possibly, even the middleware) delegate these protocols to protocol implementors specified by the application: each application will select its own model and share it with others. We consider [2] to be a step in the right direction
Object State. As mentioned above, middleware encapsulates objects maintaining a portable view of object state. Since some of this state is already maintained by the OS, work is duplicated once more. The OS can instead export such state to the middleware.
Migration Decisions. In traditional OSes, resource allocation status is either carefully hidden or exposed in a machine-dependent way [8]. Hence, DOS supporting explicit migration must re-export state information in a platform-independent way so that applications can decide when and where to move their objects.
In the framework proposed, both OS and middleware state is exported using a common well-defined interface. Users rely on it to gather information about system state and available resources in the network. This can be considered to be a generalization of facilities provided in MOS [12] to inspect processor availability and to allocate processor time. By using this information, applications and system services can make migration decisions, leading the way to customized load balancing, fault-tolerance, and availability.
Our research focus on the development of Off++, a flexible object-oriented microkernel implemented as a collection of cooperating frameworks. Its major goal is to distribute network hardware resources in a secure manner. Off++ abstractions [6] are simpler and lighter than the ones in traditional OSes. Therefore, the middleware has more freedom to implement its own abstractions in a efficient and highly-optimized way. In nodes in which running a new native OS will not be possible, its interface will be emulated by loadable kernel modules or by user-level libraries.
Off++ enables the use of per-application transport, location, and coherency protocols which can be selected and changed dynamically by each application through the system's reflective interface. The distributed IPC mechanism can be adjusted on demand by applications, as [2] suggests. We believe that this architecture can provide significant performance improvements for middleware systems in which objects migrate and are replicated.
System objects implement an interface for reading and writing state in a serialized form. ``Frozen'' object state is transported to different locations to implement migration and replica updates. A frozen object can be ``melted'' with the same name in different places when replication is desired. The system automatically redirects calls to objects marked as ``frozen'' by raising exceptions (handled by the middleware or by the application). Thus, the middleware can handle object references and invocations while they are moving or while they have an inconsistent state.
The exact semantics of replication - regarding coherency, method invocation, and the like - is defined by the application. By supplying adequate protocols, different coherency semantics and concurrency protocols can be employed.
For naming and protection, Off++ uses distributed physical names corresponding to the physical resources it allocates to the middleware. A flexible protection framework allows the middleware to select among different protection schemes such as capabilities, access control lists, or active security agents. The middleware assigns a concrete protection implementation for each of the allocated distributed system resources and the kernel is responsible for enforcing it. A migrating object uses a ticket [15] that includes its access rights in a format suitable to the selected protection scheme. Thus, protection can be enforced wherever the object goes.
Finally, our model will be extended to conventional OSes by making them export physical resources (which usually requires minor changes in the resource allocation subsystem) and support a flexible IPC with redirection. We believe that this goal can be achieved without excessive effort. The modifications implemented in some modern UNIX systems to support soft real-time [16] are very close to what we need. The practical consequence will be a thiner middleware cooperating with the OS rather than fighting against it.
Systems based on virtual machines and interpreters are slow and pose a number of security concerns [10,13]. Other middleware-based systems which build heavily on OS support impose their own distributed object implementation to every application [14,13,3]. We explore distribution at a lower level of abstraction.
Our physical resource protection model is comparable to TACOMA's
resource containers [7]. TACOMA containers,
handled outside the OS, are now handled by the resource multiplexor,
the Off++ kernel.
Exokernel [8] proposes a flexible architecture in which
user-level libraries implement system services in a highly customized
and efficient way. We extend their centralized model by considering
distribution issues and, thus, developing a distributed adaptable
kernel which can perform well for distributed objects.
Paramecium [14] is a novel OS supporting the notion of distributed shared objects as a means to implement system services. The system can be adapted and heavy middleware layers could possibly be avoided. The problem, though, is that it imposes a single distributed object implementation and it is not clear yet how different approaches for replication and concurrency control could coexist on it.
We are currently implementing Off++, an object-oriented redesign of the Off
kernel which extends its model. In the mean time, OS services are being
implemented as distributed object systems to be run on top of Off++ as well
as on existing systems.