Cetrix Blog

Challenges When Introducing an Enterprise Integration Platform

Paul Nilsen - May 11, 2018 10:59:26 AM

What is Enterprise Integration and Where is Needed?

Today, organizations rely heavily on technologies – which are often of different nature - and applications, running in different business units and likely based on different  technologies. So, integrating these applications into a unified set of business process has emerged as a priority. Enterprise Integration provides a mechanism to build a composite application and share processes and data.

As several studies indicate (here and here), Enterprise integration started as point-to-point integration and evolved to cloud based enterprise service bus architectures. In a point-to-point integration, integrating between two applications requires code development and creating information you could exchange between them. This may prove cost-saving and effective on small networks, but it results in exponential growth of cost and required effort for enterprises. Forrester Research estimates that up to 35% of development time is spent to create application interfaces.

Fig 1. Point-to-point integration

In an Enterprise Integration platform. Enterprises integrate between applications through an Enterprise service bus (a middleware platform) which orchestrates the information exchanged between the integration endpoints, reducing the number of interconnections. Additionally, Enterprise Integration platforms usually provide features such as

- Scalability, high availability and sizing

- Use of industry standards as interface technologies (no custom development-universal connectivity)

- Transactional processing, since the information should be consistent within multiple applications

- Process driven approach embedding logic into integration flows

- Code abstraction, lowering development cost

- Monitoring and managing components, making all transaction executions accessible and clear.

- Deployment management components  

Fig 2. Enterprise service bus integration

Platform Challenges

A typical integration project has the following phases:

  1. Project planning and conceptual solution definition (business review, project scoping and planning)
  2. Detailed analysis and design of the solution (architecture and design, development environment setup)
  3. Development and testing of the solution (Development, testing, rollout planning, production environment setup)
  4. Go live to production (Rollout preparation, go live)

Fig 3. Integration project phases

Most of the challenges that may arise come from over-focusing on the development (phase c), compared to project planning analysis and design (a and b). While this is common on all IT projects, it is essential for enterprise integration.

Technical challenges can be grouped to the following categories:

Enterprise Integration Challenges

 

Incomplete or Not Well-Defined Analysis

Business people should know exactly what they are looking in an integration project, and what the deliverables will be. The delivered solution may include processes for several areas, departments, so the project team should have an overall knowledge of business processes.

This also apply for IT people, since applications and systems involved are numerous and may run in different platforms and heterogeneous environments. In terms of technical analysis and design and project management, this makes the composition of individual systems and application into a composite one, difficult to manage. Additionally high consulting rates prevents often, receiving external support.

Not open platforms

There are systems that provide limited interface or non-standard capabilities. Other systems require a license to upgrade and extra costs to customize. In such cases, you might need a custom development (from integration platform side) solution, resulting in extra effort, time and cost.

If a system is unsupported, closed and does not have an interface, you should dispose it.

Vendor management

Clear vendor management is critical for an Enterprise Service Bus Integration project when:

- Vendors cannot do modifications to application for several reasons, or require a new license

- The result of a requested modification may not fulfil time specifications

- Arguments and conflicts regarding areas of responsibility may arise

- Application vendor may not support custom modifications in future upgrades

In these cases, you must strictly define your requirements, responsibilities milestones, and deliverables within your vendor agreement

Performance, Sizing and High Availability  

You should know your performance objectives upfront. Some of things you should consider include:

- What is the most important factor, number of supported concurrent users, response time or system throughput?  

- Is a stress test required?

- How someone can measure bottlenecks or poor performance?  

As general guide to face performance issues:

- In design components that talk frequently should be placed in the same network

- If possible, workload should be distributed

- Bottlenecks should be located during tests

- Focus on processes that take the 80% processing

- Perform step by step changes. Measure impact before apply

- Keep historical logs

Scalability is the ability of a system to handle increased workload. Basic question you should ask in the first step is:  vertical or horizontal scalability?

The first is the quick, less expensive solution (e.g. increase memory in a server). The second is the long term: adding a new server in a clustered environment, implementing an external load balancer, etc. Usually, a combination of both is applicable.

Every business that runs an enterprise service bus integration platform wants their integration platform to be available as much as possible. Since this is strongly relates to cost and performance, you should figure out how much downtime your business can tolerate prior to design and implementation.

Transaction Processing

Transactional integrity: while it is very common for multiple integration services to execute atomically, we require either an “all complete” or “all no complete”. (Most RDBMs and ERPs provide such functionality)

Most integration platforms provide transactional processing capabilities with some additionally guaranteeing the processing of documents.

Setting Up Test Environments. Incomplete Tests

Setting up in integration project test environments requires duplicating (simulating) your production ones, including all systems and applications involved. Additionally, it requires that you establish all network connections between test components.

This may prove difficult in some cases:

- Integration between real time applications or applications that require hardware resources not available (e.g. control systems)

- Integration between cloud applications or integration between applications that may require extra licensing for implementing a test environment  

Usually, you perform tests in batches on specific dates -according to the project plan-, with the intermediate time dedicated to corrections and improvements. This requires management effort for not having any incomplete tests.

Security

Since data can come from different applications, you need data access control in your integration platform when that Integration platform has an end user component.

Data mapping

If some of your applications include entity information, data mapping comes 1st priority when you integrate these applications.

Usually, you resolve this by applying the “canonical” model, a defined superset of entity’s fields and attributes that the integration endpoints should follow.

Monitoring Tools are not Implemented

You often find integration management software missing from your final solution, usually for cost reasons.

Management and monitoring tools provide information to end user, regarding integration executions, execution statuses, timestamps, which are often valuable in troubleshooting  

In conclusion, in an enterprise service bus integration project, your  biggest challenge is to focus on analysis and design -both operational and technical. This is because these projects often deal with different applications, technologies and processes that make these projects complex. 

Find out how to get more from your cloud-based and on-premises systems without sacrificing security.

Download the eBook Now.

GET THE EBOOK

Composite Cloud

Composite Cloud

Topics: Application Integration

Paul Nilsen

Paul Nilsen

Paul Nilsen is the director of solution development at Cetrix Cloud Services. He also works with the Solution Development Department at Cetrix Technologies, where he headed the department for three years. Paul has over 20 years of experience in IT, working on software engineering projects, hardware-software product integration, design of special purpose systems using artificial intelligence, and implementing CRM and ERP software systems.

Previous Post

Best Microservices Framework for Java

0 Comments

CETRIX BLOG

Subscribe To Cetrix Blog Notification

Cetrix blog posts discuss latest topics on CRM, marketing and analytics technologies and use in different sectors. Get notified of new blogs.

Subscribe to Email Newsletter

100% Privacy. No Spam.

Salesforce For Small Business

Salesforce For Small Business - Cetrix Cloud Services

Higher-Education.jpg

Salesforce Platform

We adapt the World’s No. 1 CRM to your processes to help you increase sales efficiency towards your strategic goals.

 

LEARN MORE

surgery.jpg

Healthcare

Download and learn how the right CRM system can provide your business with the tools it needs to keep customers happy and grow.

Download

Related posts

Best Microservices Framework for Java
Elastic load balancer: What it is and how does it work?
Design Principles of Application Integration Architecture