Patterns and Frameworks for E&P Middleware
Introduction to Architecture of PetroCOVERY
Several international studies have shown that integration of Information technology has generated more return from the industry as a whole. Similar interoperability of information technology in petroleum industry has yielded immense results. Appreciating and depending on object-oriented (OO) techniques and continuing with similar processes in design patterns, framework and component etc techniques, designing of a framework like distributed discipline-specific application and develop pluggable services in robust, extensible, portable and efficient way. With detailed introduction of PetroCOVERY architecture author examines some key features about E&P middleware and evaluates its architecture, this paper presents a domain-specific middleware definition for E&P domain on personal view. Introduction of Object-oriented techniques provides an innovative methodology to analysis and model E&P domain knowledge, information and data (KID).
E&P middleware should tier different encapsulation layers, which in higher level supports interoperability among different discipline-specific applications via the middleware bus, and harness the development process of discipline-specific applications, which are exclusively against some object-oriented services and uniform data model schema in distributed and heterogeneous environment. As a conclusion, application interoperability, data integration, distributed object computing (DOC) and adaptive service-oriented architecture should characterize an E&P middleware.
The use of information technology and its various components like effective application systems and repositories has helped humans in dealing with different subsurface aspects like geology, geophysics, reservoir etc. till date. To continue with associated benefits and its interoperability in petroleum industry, those involved in the E&P petroleum industry like companies, vendors, certification organization etc. have contributed their ideas to develop interoperability between data and application, application and application, to make the computing ability powerful (Klie et al. 2005; Parashar et al. 2005) and to extend their functionality. Interoperability here refers to the ability of a system to work with or use the parts or equipment of another system.
Various companies are working to create interoperability between data and between applications of varied vendors and oil companies. Petroleum Open Software Consortium (POSC) which was established in 1990 have been developing certain projects that focuses on energy standards and that provides open, detailed and precise presentations for information modeling, information management, and data and application integration over the life cycle of E&P assets exampled by POSC site. But limited capability of POSC and its strength as a ‘single vendor’ solution has made it useless for fulfilling the demands of the petroleum companies and the interoperability among vendor solutions. Though in theory, the concepts of “interoperability levels” suggested by POSC are beneficial as they help us understand as to which level would benefit us more (Rainaud, 2005) with the ultimate users of the product at one of those level could run many applications against different data, which could be pointed out as soon as there is a change in the encapsulated data objects. A basic conceptual structure of the application developed by OpenSpirit takes precedent over this level. The global architecture of the product developed by OpenSpirit is shown in Figure 1. This product helps bring about a base framework which presents a set of services based on common object request broker architecture (CORBA). It pervades specifications for distributed middleware products, a set of specific objects like session, links, coordinate system, etc., and data modules for domain-specific fields like drilling, subsurface, seismic, etc. Other famous E&P software providers have also been focusing on similar work by refining its existing products or developing new software based on middleware architecture like Paradigm.
Logic based on “interoperability levels” gives a new kind of description for applications based on middleware architecture like data store, middleware and application, yet the requirement of E&P industry to meet emerging trends and other changes emphasizes on the service oriented approach. This services-oriented architecture would satisfy the requirement of interoperability of data and different applications, but also supports the powerful capability of grid-enabled computing, and the extensibility of pluggable discipline-specific applications connected to some domain-specific services. This in turn would help the application developers to focus on their knowledge of domain upon the problem at hand. It is time to share the reasonable definition for E&P domain-specific middleware, considering that middleware technology is currently leading the software development in different domains (Wolski et al., 1999; Beynon et al., 2001; Allen et al., 2001; Emmerich, 2000). This definition provides the E&P industry players with innovative domain-specific middleware architecture as shown in figure 2, for developing new discipline-specific applications and integrating legacy applications, in addition to fledging the functionalities suggested by “interoperability levels.” The four main topics that E&P middleware should mention are thin and rich applications, distributed object computing, domain intelligence platform, and data integration. These are the general ideas adopted by successful middleware product of the world like the product line of Business Object, TongTech, Adaptable Communication Environment etc.
Middleware came into existence in 1980s, and was used to describe the network connection management software. But revolutionary changes in the application of information technology tools in mid 1990s, made sufficient condition for the development of middleware into a much richer set of paradigms and services offered to help make it easier and more manageable to build distributed applications. Different kinds of middleware have been developed till now. They vary in terms of the provision for programming abstractions and the kinds of heterogeneity beyond network and hardware, they are just like there are layers of networking protocols as shown in figure 3 (Douglas, 1995, 1996, 1997, 1998, 1999).
The core of middleware is a class of software technologies which is designed to help manage the complexity and heterogeneity inherent in distributed systems (Zinky et al., 1997; Bernstein, 1996; Campbell et al. 1999) , especially combined with different design patterns. It is defined as different layers of software, beginning from hardware but, below the application program that provides a common programming abstraction across a distributed system. Middleware helps domain experts to develop their applications with only mention to domain-specific logics, but not to detailed lower level application programming interfaces (APIs) which is more prone to errors.
2.1 Operating System & Protocol
As opposed to hardware, different operating systems (OS) and network protocols like Unix/Linux, Windows, VxWorks, TCP, UDP, etc. provide mechanisms to manage end system resources. For e.g., CPU scheduling and dispatching, virtual memory management, secondary storage, persistence and file systems, local and remote inter-process communication (IPC), etc.
2.2 Host Infrastructure Middleware
In order to create network programming components that can be reused, different OS vendors are known to encapsulate or enhance its native OS mechanisms. This would help the programmer in avoiding the usage of several tiresome and error-prone low-level OS APIs. Examples include Java virtual machine (JVM), Common Language Runtime (CLR), etc.
2.3 Distribution Middleware
Distribution middleware refers to higher-level distributed programming models with reusable APIs and components which help automate and extend native OS capabilities across different types of networks like wide area net (WAN) and local area net (LAN). The benefits of distribution middleware are that it avoids hard-coding client and the dependence of server application on object location, language, OS, protocols, and hardware. At the same time, it provides the abstraction of an object, which is remote yet whose methods can be implemented just like those of an object in the same address space as the caller. Practically, accessing the services of a remote object should be as simple as calling a method on that object like obj->op(args). With the use of distributed objects, all the software engineering benefits of object-oriented techniques like encapsulation, inheritance, polymorphism etc. are available to the distributed application developer.
Examples of two predominating standards of distributed object computing (DOC) are OMG’s CORBA and Microsoft’s DCOM (Distributed Component Object Model).
2.4 Common Middleware Services
Distribution middleware can be strengthened by defining higher-level domain-independent services that focus on programming “business logic.” For E.g., Sun’s J2EE, Microsoft’s .NET etc. Common middleware is playing a very important role in the current information technology scenario since they support many recurring distributed system capabilities like transactional behaviour, authentication and authorization, database connection pooling and concurrency control, active replication, etc.
2.5 Domain-Specific Middleware
Besides the general “business logic,” there exist special needs for different domains that should be suited to above middleware layers. These include telecom, e-commerce, health care, process automation, aerospace, etc. Since E&P middleware is a new addition to the middleware family, visualization, interpretation, inversion, etc. of domain-specific logic remains to be resolved.
Based on services, object-oriented services supplied by domain-specific, common, and distributed middleware, rich and thin applications can run well in an extensible and flexible way based on interfaces on behalf of those services. Discipline-specific applications may be referred to as “rich” applications and web browser can be an example of the “thin” one.
3 E&P Domain-specific Middleware
From figure 2, the E&P middleware location is clear and is at the uppermost including the applications layer of that ladder structure. It should have a modern, sophisticated services-oriented architecture and should be flexible and extensible. The E&P middleware should be able to integrate legacy systems, manage and grow with the diverse demands of E&P domain. It should also be able to harness the development process of E&P domain–specific applications in distributed and heterogeneous environment. PetroCOVERY, a domain-specific middleware, focuses on how to set up powerful infrastructure which helps in data exchange and messaging across WAN and LAN. It contributes the standard geometry definition for visualization and provides domain-specific specialists with productive and typical discipline-specific actions like pluggable services.
3.1.1 Powerful Infrastructure
Decisions on development of domain-specific applications are challenged by heterogeneous computing environment, integrated, diverse information formats, and various programming styles. It is also very important that we integrate the existing legacy systems into the large-scale enterprise solution. PetroCOVERY, a domain-specific middleware, which complies with specifications given by Object Management Group (OMG) CORBA is designed and implemented in a powerful infrastructure for distributed and heterogeneous environment. For example, with the help of interface definition language (IDL) and its compiler supplied by a CORBA vendor, one could very well get stub classes in java language which can be developed into a java application in client side and skeleton classes in C++ language representing one domain-specific and object-oriented service. Both kinds of classes merely set up a distributed framework using different language styles. We only need to connect and narrow down to one service object with its name and to activate one function of this object upon the middleware bus, namely object request broker (ORB) and object adapter (OA). It is not be required to marsh and unmarsh parameters, package data against one kind of transport protocol, multithreading connections etc. In addition to some common services like event service, naming service etc., which assure the infrastructure’s reliability, availability, and stability, PetroCOVERY also specifies some common domain-specific service objects. For instance, data integration service as a data hub or broker can bring data from the data base and is then able to convert those data respectively into different data formats understood by different applications, e.g., XML format for one web application, SEGY format for seismic inversion application, and so on.
3.1.2 Discipline-specific Application Framework
It is a known fact that sharing of more experimental information helps us know more about the subsurface. That is, data obtained from one discipline can be adopted by other disciplines for a more correct understanding, in this case, the subsurface. Major disciplines like geophysics, stratigraphy, and tectonics, which are used for studying subsurface aspects, should be connected to one intelligence platform for an integrated work and then respond to those calling of all different discipline-specific applications. And the implementation of one application framework is implicitly trapped into a distributed environment for sharing its understanding. PetroCOVERY provides a new method for customizing a discipline-specific application framework using its developer interfaces and lets those applications work tightly as a whole. Thus, based on PetroCOVERY developer interfaces (item 3.2.2), one developer should be able to set up a discipline-specific application framework that should merge with those stub classes which are proxies of different disciplines.
3.1.3 Viewing Pipeline
Disciplines used by the E&P industry, especially up stream, need to have well developed rendering effects for discovering hidden information. It is tiring and unreasonable that different applications have specific graphic package about different kinds of primitive definition and rendering processes. Geometry primitives provided by geography information system (GIS) may be referred and adapted to by different domain-specific features, since they can not only specify reasonable geometry primitive definition but also have a powerful ability for spatial analysis. Implementation of rendering perfects and viewing pipeline for various applications will benefit with the use of a computer graphic package called Simple Features Specification for SQL Reversion 1.1 (SFS) (Open GIS Consortium, 1999), based on OpenGIS specification, and developed by petroCOVERY. It is important to note that following the same geometry schema, spatial analysis service can answer the same result for different applications under the same geometry convention.
3.1.4 Common Domain-specific Classes
It has been observed that different disciplines of E&P domain share same subjects or topics. For example, the basic input for their study include seismic and log data and they normally follow the same structure definition for time window limitation. Thus, E&P middleware should recognize and abstract geological objects in object-oriented principles like inheritance, and package them into a utility class library which is supplementary to a discipline-specific application framework except for those stub classes. It is generally seen that common domain-specific classes are a source of uniform data base schema. They are also able to give a correct understanding of data fetched by the data service to a discipline-specific application.
3.2 Inner Tiers
PetroCOVERY consists of separate, yet interconnected tiers similar to the architecture of a domain-specific middleware. The tiers include user interaction, developer interfaces, platform services, and data services, as illustrated in figure 5.
3.2.1 User Interaction
The two main types of user interactions are rich and thin applications. Rich applications refer to different discipline frameworks specified by IDL interfaces on behalf of domain services. Domain and data services running on the same middleware bus are closely integrated with the rich applications. Discipline-specific applications mainly serve specialists’ demands, while web applications, also called thin applications are for corporate information accessing and decision making to satisfy other customers’ demands like those of auditors, managers etc. It is also used to improve the ability of information exchange and interoperability. Generally, thin and rich applications running on the same middleware make up one cluster which can exchange its inner information with other clusters, even other kinds of applications, like those based on DCOM middleware using XML techniques.
3.2.2 Developer Interfaces
The E&P middleware mainly consists of a powerful set of domain and data services. Therefore, there is a need to expose them through an extensive set of software development kits (SDKs) including discipline-specific interfaces based on native language (C++ and Java) or J2EE,.NET and web service common middleware like J2EE etc. (Box et al., 2002). The functionality of the server-side components is hosted by discipline interfaces, namely discipline-specific frameworks. It also acts as the translation layer between end-user and domain or data services while enclosing activities belonging to one discipline. That is, the end-user is provided with higher level understanding of an application of a discipline. They also make reasonable abstraction and implementation using object-oriented techniques like encapsulation, inheritance, parameterized type, exception handling etc. PetroCOVERY that is tightly integrated with Java and Microsoft-based platforms via native J2EE,.NET and web service SDKs, also specifies “enterprise-based” interfaces communicating with the third party’s application server to deploy enterprise application. All end-user applications provided by E&P middleware are based on these SDKs. They follow different discipline specifications to make sure that developers are able to access all aspects of the system.
3.2.3 Platform Services
The core processing and management engine for different E&P disciplines and root purposes is the platform service, namely domain-specific services like inversion, spatial analysis etc. The domain services tier is also called the intelligence and special processing tier because this is the area of data processing, key algorithm execution and communication.
Domain services that are divided into a series of specialized services for various activities are categorized into different disciplines and are connected by a highly-optimized communication framework called the middleware bus. This middleware bus is responsible for the movement of information and cooperation among the domain-specific and data services, and provides end-user information access, delivery, and interaction. The individual service can be run on a single or multiple processors on the same or different machines, which may be organized into a virtually clustered multi-server environment making grid-enabled distributed computing. This is in such a way that while an end-user runs the inversion and spatial analysis services on the same cluster, one can run seismic data interpretation service on a separate machine of another cluster. The end-user may choose to run multiple instances of each service on a system, if the entire set of system services is running on a multi-processor computer. It is useful when the end-user wants powerful computing ability.
3.2.4 Data Services
For PetroCOVERY to support comprehensive and flexible data access and to get a correct understanding of all services plugged in the same middleware bus, it needs a central management service to define one meta-based semantic environment to manage all services. With the help of the semantic layer, the central management service can easily monitor and control everything occurring on the same bus and produce reasonable response, like correctly answer inversion application as to where, how many and which types of inversion services are running. The information service also monitor whatever is occurring and tell corresponding services or applications on the basis of event or time-schedule mechanism. The main task of data services is the ability to integrate different vendors’ data. They do this by specifying different ways to import and export data in specified format which is understood by both the applications.
There are four essential characteristics of E&P middleware, derived from the tier description of PetroCOVERY:
3.3.1 Data Integration
E&P middleware should automate and integrate disparate data so as to solve problems related to growing data volumes and the need for data sharing among different disciplines. It should also ensure researchers’ confidence in adopting, reporting, query and analysis. Mapping of object-oriented and relational theories and data warehouse techniques is essential for building long or short-term information store. Keeping in mind the persistence of those common-specific classes, a uniform data base schema for different pluggable services was put up by PetroCOVERY. This schema has two main roles to play. It supports import and export of different data formats of vendors like GeoFrame. OpenWorks etc. and interoperability among other vendors’ applications in a semi-automatic way.
3.3.2 Pluggable Services
The open nature of E&P middleware enables integration of different object-oriented services into this middleware framework with the same specification. As soon as these services are plugged into this architecture, they take advantage of the performance, failover, load balancing, and end-user interfaces. Different services are categorized on the basis of their intentions and functions. For example, services supply interface specification for different discipline framework, platform services are the core for managing and monitoring E&P middleware runtime status, and data services support a flexible set of options for accessing, organizing, and exposing data.
3.3.3 Distributed Object Computing
One of the main problems faced by the E&P industry like reservoir simulation is large-scale computation. Thus, the ability to duplicate the same service and then execute the task using the same information store at the same time is also the main characteristic of E&P middleware. The services of E&P middleware can be scaled vertically to take full advantage of the hardware they are running on and horizontally to take advantage of the multiple computers over the network environment.
3.3.4 Application Interoperability
With the use of developer interfaces which can be hooked and used with the use of special language or against special OS, different discipline applications like seismic data interpretation and seismic data inversion can run on the same software bus including message notification, data packaging and exchanging. Legacy applications or other third vendors’ applications can encapsulate new service embracing legacy application adopting its exposed interface for communication (E.g., add-ins technology) and then be plugged into this service. It can also communicate with other services with the help of an event or a message channel.
4. Patterns and Frameworks
Advances in information technology is enabling and strengthening PetroCOVERY. Information technology is getting cheaper, faster and better with time. There is a growing acceptance of a network-centric component pattern, like distributed applications with a range of quality of service (QoS) needs to be constructed by integrating components and frameworks through various communication mechanisms. For example, component middleware is developing and spreading everywhere, model driven development is integrating generative software technologies with QoS-enabled component middleware, for example, the general flow of development is to model, analyze, synthesize and optimize, and provision and deploy. PetroCOVERY is supported by the emerging OO principles, methods and tools, so as to avoid problems caused by functional decomposition techniques which result in non-extensible system architectures. Language idioms, like encapsulated classes, dynamic binding, inheritance, parameterized type etc. decrease the potential runtime error which maybe caused by type mismatches, which are common bugs in functional source code. Design patterns are used for solving common software problems arising out of a certain context, for resolving key software design forces, and for understanding structures and dynamics that occur again and again among software participants so that successful designs can be used again and again. They also codify expert knowledge of design strategies, constraints and “best practices.” For example, the use of proxy pattern makes an inversion object free from situations where we need intensive memory requirement for large data object E.g., A seismic data object. This proxy object loads trace data only when this inversion object needs them, using dereference operators (* or -> in C++), as shown in figure 5.
A discipline-specific application framework specified by PetroCOVERY is an integrated set of classes and those stub classes jointly work to produce a reusable architecture that can be applied widely. A framework is generally defined as a “semi-complete” application that exhibit “inversion control” at runtime through callbacks. It also provides integrated domain-specific structures and functionality. At the higher level, it will run well with defining abstract actions. For example, a seismic inversion framework comprises of stub classes on behalf of inversion service, data service, common utility classes for visualization, and those abstract classes which will be hooked at the runtime by concrete implementations. One possible implementation of this framework is the horizons interpreted by sequence stratigraphy which gives more reasonable time window for inversion algorithm, inversion results or inputs is in MySQL Database, which runs only on Sun Solaris UNIX Platform (Figure 6).
Let us discuss the component technique for PetroCOVERY. A component refers to an encapsulation unit with one or more interfaces that provide client, those discipline-specific applications, with access to domain-specific or domain-independent “business logic” and interact with each other through “ports.” It should also have interfaces which specify what it can do, connection points required by other service components to work together, event sinks or sources which keep receiving or sending events or messages, as shown in figure 7. While designing an IDL interface, suggestions and conventions of component technique play an important role. This would be implemented as a PetroCOVERY service plugged into middleware bus and ultimately make an intelligence platform with common services as core part of PetroCOVERY, as shown in figure 8.
5. Summary and Conclusions
The poor situation of IT interoperability in the E&P industry is changing, thanks to the development of E&P middleware, data integration, application interoperability, distributed object computing and object-oriented service. The mature study of different disciplines of petroleum and object-oriented techniques are making the development of one E&P middleware more feasible. The author intends to describe different object-oriented techniques in detail in the future issues. These object-oriented techniques are applied to resolve issues related to discipline framework, uniform data model, distributed object computing and others.