Modeling Canonical Data with ArchiMate

Almost all large organisations have many applications which were developed based on different data models and data definitions, yet they all need to share data. So the basic idea of a canonical data model is “let’s use a common language when exchanging information between systems.” So, if system A wants to send data to system B, it translates the data into an agreed-upon standard syntax (canonical format) without regard to the data structures, syntax or protocol of system B.  When B receives the data, it translates the canonical format into its internal format and everything is good.

The benefit of this approach is that system A and B are now decoupled; if A needs to be upgraded, enhanced or replaced, as long as it continues to reformat the data for external systems into canonical form, then B or any other system is not impacted. Simple and elegant!  Under this approach, organisations with hundreds of application systems can now happily upgrade and enhance individual systems without the complexity (and cost/risk) of coordinating the analysis, design and testing with all the other systems.

However, there is a catch.  While A and B are no longer coupled to each other, they are now coupled to the canonical physical format.  So now if the canonical model changes, every single system that uses it to send or receive information is potentially impacted.The results endless analysis and discussions among business analyst, architects and designers of any proposed change to the canonical model.

The complexity of data Governance is often avoided by creating  "quick and dirty" service implementations, which focus only on the requirements at hand so as to speed up implementation. Over time, these services create increasingly reductant services and restricts the flexibility and extensibility of the overall SOA.

One symptom of this approach is revealed by the inspection of the data structures being exposed through service messages. If there are many services exposing arbitrary variations of the same business information, or many instances of data mapping running in the enterprise service bus (ESB) and business processes, then it is likely that a simpler and more reusable solution could have been developed. Left unchecked, this approach can lead to a solution so complex that all the benefits of the SOA approach are lost.

A good place to look for possible candidates for the CDM is the Information Structure Viewpoint and Application co-operation viewpoint in ArchiMate model.

A structure like this would indicate possibility of a redundant information in both the customer model.

A target Information model for this could be to have a specialised representation of customer as shown. However it is not evident from this model that customer object is a part of the Canonical data model. We can use group to explicitly  state this fact as shown in the next model.

The customer object is now a part of a group is represents a set of  canonical data objects

Note that canonical data model is often achieved though Enterprise service bus patterns. We do not model the ESB patterns in the above diagram. This model merely points to the presence of a CDM  and not how its implemented.

Starting with ArchiMate -2

In Starting with ArchiMate -1 we had a overview of the the ArchiMate meta models. In this section we discuss the possible relations between the components.

Here is a list of relations in ArchiMate.





All relations have a direction, except Association which is bidirectional (Assignment was also bidirectional in ArchiMate 1.0).
ArchiMate, also offers a mechanism to create ‘shortcuts’ called ‘derived structural relations’. The derived relation between two objects is the weakest relation found on the route

ArchiMate 2.1 released

ArchiMate 2.1 has been released.

Apart from the editorial corrections the major change is change of classification of the concepts within the Business , Application and Technology layers.
In ArchiMate 2.0 the core concepts section describes three types of elements :-active structure elements, behavioral elements and passive structure.

However, within the the layers the concepts are categorized as 

• Structural Concepts

• active entities

• passive entities

• Behavioral Concepts

• Informational Concepts

This was inconsistent with the core concepts. Also the at times it is ambiguous whether an entity is a passive structure or information as both do not invoke behavior (as active entity) or are themselves a behavior.
This has been fixed in 2.1 release and the new classification within the layers stand as

• Active Structure Concepts

• Behavioral Concepts

• Passive Structure Concepts

Impact on your existing models:
All references to business objects and data objects which may be categorized as Structural objects in the metadata of the element must be changed to Passive Structure Object. The Information Object category itself should be renamed to Passive Structure objects.

Starting with ArchiMate -1

In this blog I would start sharing my experiences and leanings in Enterprise architecture. In my fist blog about EA I would discuss the basic of modeling Enterprise architecture using ArchiMate.

ArchiMate, an Open Group Standard, is an open and independent modelling language for enterprise architecture. Just as any modeling standards  purpose of a enterprise architecture modeling standards is to reduce the ambiguities that occur in natural-language descriptions and ArchiMate is a decent language for  that. ArchiMate, with its business layer, application layer and technical layer helps us to model the different aspects of IT Architecture It great way to illustrate high-level couplings of business processes, people (as roles), applications, data and infrastructure in an organization.

Features :-

• A language for describing Architecture
• Framework describes the business layer,application layer and Technology Layer
- With relations between the layers
• Graphical language with formal semantics
• Techniques for Visualization and Analysis for various stakeholders
• Open standard maintained by The Open Group

There are three core layers in ArchiMate:

1. The business layer models the products and services offered to both internal and external customers, which are realized in the organization by business functions and processes, performed by business actors. It also represents the business objects utilized by people and processes.

2. The application layer models the applications and software services, which support the business layer.

3. The technology layer models the infrastructure services, which are required to run the applications, realized by communication hardware and system software.

Also, the core layers consist of three types of elements, which are:

• objects that act (active objects)
• objects that represent the behavior of those ‘objects that act’ (behavioral objects)
• objects that cannot act and which are acted upon by that behavior (passive objects)

Here are the core ArchiMate artifacts

Active Structural Entities
*	A business actor is an organisational entity that is capable of performing behaviour
*	A business role is an entity responsible for performing specific behaviour
*	A business collaboration is an aggregate of two or more business roles, which work together to perform a collective behaviour
Behaviour Entities
*	A business process groups behaviour based on an ordering of activities, intended to produce a defined set of outcomes, i.e., products and services
*	A business event is something that happens (internally or externally) and influences a behaviour
*	A business interaction describes the behaviour of a business collaboration
Passive Entities
*	A business object is a passive element, which has relevance from a business perspective
Informational Entities
*	A representation is defined as a perceptible form of the information carried by a business object
Active Structural Entities
*	An application component is a modular, deployable, and replaceable part of a software system that encapsulates its behaviour and data, and exposes these through a set of interfaces
*	An application collaboration is an aggregate of two or more application components that work together to perform collective behaviour
Behaviour Entities
*	An application function groups automated behaviour that can be performed by an application component
*	An application service exposes an automated behaviour of an application
Passive Entities
*	A data object is suitable for automated processing
Structural Entities
*	A node is a computational resource upon which artifacts may be stored or deployed for execution
*	A device is a hardware resource upon which artifacts may be stored or deployed for execution
*	An artifact is a physical piece of data that is used or produced in a software development process, or by deployment and operation of a system

In the next section we discuss about modeling  relationship and  derived relationship.

Change general project to Java project in ecllipse

To change a general project which may have been downloaded from repository to a Java  project close the project in Eclipse. Open the .project file in a text editor. Change the <natures> </natures> to
<natures> <nature>org.eclipse.jdt.core.javanature</nature></natures>   and <buildSpec> </buildSpec> to

<buildSpec><buildCommand> <name>org.eclipse.jdt.core.javabuilder</name><arguments> </arguments></buildCommand></buildSpec>.
To change to another type like a cordys project use <natures> <nature>org.eclipse.jdt.core.javanature</nature>      <nature>com.cordys.coe.CEP.nCordysProjectNature</nature></natures>

Open the project again.

Validating XML against multiple schema's within the same namespace

I was trying to validate XML against a XSD schema which in turn contains included schemas. I was getting this strange exception "src-resolve: Cannot resolve the name someName to a(n) 'type definition' component" where someName is the component that is declared in the included schema. I tried to load the schema using newSchema(Array) constructor but that did nor help either.
Schema schema = schemaFactory.newSchema(sources);
Where sources is an array of the StreamSource pointing to the main schema as well as the included schemas.
After googling I found this is actually a known bug in Xerces.
https://issues.apache.org/jira/browse/XERCESJ-1130
One way to bypass this problem is to write a custom ResourceResolver. However after lot of trial and error I found a way to simpler way to do this. The trick is to create a URL object and set systemID of the source to this URL.
//open the main schema

File file = new File("schema1.xsd");
 FileInputStream ss = new FileInputStream(file);
//Get the URL for this file

URL url = file.toURI().toURL();
// Create the StreamSource for the schema
StreamSource source = new StreamSource(url.openStream());
// Most Important: set the systemID to this URL. The included schema will be searched from this location
source.setSystemId(url.toString());
Now create the schema and validation should work fine

 SchemaFactory schemaFactory =
SchemaFactory.newInstance(XMLConstants.W3C_XML_SCHEMA_NS_URI);
Schema schema = schemaFactory.newSchema(source);

Removing XML namespaces

I was working on a 2 versions of a BPM product the older one without support for namespaces and the newer one supporting namespaces. The messages from external systems were with namespaces. When the old system was called it was required to strip the namespaces. Initially we thought of using xslt to achieve this. But then again regular expression came to our rescue. The cute little function below can remove namespaces from xml string


public static String removeXmlStringNamespaceAndPreamble(String xmlString) {


return xmlString.replaceAll("(<\\?[^<]*\\?>)?", ""). /* remove preamble */
replaceAll("xmlns.*?(\"|\').*?(\"|\')", "") /* remove xmlns declaration */
.replaceAll("(<)(\\w+:)(.*?>)", "$1$3") /* remove opening tag prefix */
.replaceAll("(</)(\\w+:)(.*?>)", "$1$3"); /* remove closing tags prefix */


}
Love the power of regular expression :)