Cetrix Blog

    Challenges When Introducing an Enterprise Integration Platform

    Paul Nilsen - May 11, 2018 10:59:26 AM

    What is Enterprise Integration and Where is Needed?

    Today, organizations rely heavily on technologies – which are often of different nature - and applications, running in different business units and likely based on different  technologies. So, integrating these applications into a unified set of business process has emerged as a priority. Enterprise Integration provides a mechanism to build a composite application and share processes and data.

    As several studies indicate (here and here), Enterprise integration started as point-to-point integration and evolved to cloud based enterprise service bus architectures. In a point-to-point integration, integrating between two applications requires code development and creating information you could exchange between them. This may prove cost-saving and effective on small networks, but it results in exponential growth of cost and required effort for enterprises. Forrester Research estimates that up to 35% of development time is spent to create application interfaces.

    Fig 1. Point-to-point integration

    In an Enterprise Integration platform. Enterprises integrate between applications through an Enterprise service bus (a middleware platform) which orchestrates the information exchanged between the integration endpoints, reducing the number of interconnections. Additionally, Enterprise Integration platforms usually provide features such as

    - Scalability, high availability and sizing

    - Use of industry standards as interface technologies (no custom development-universal connectivity)

    - Transactional processing, since the information should be consistent within multiple applications

    - Process driven approach embedding logic into integration flows

    - Code abstraction, lowering development cost

    - Monitoring and managing components, making all transaction executions accessible and clear.

    - Deployment management components  

    Fig 2. Enterprise service bus integration

    Platform Challenges

    A typical integration project has the following phases:

    1. Project planning and conceptual solution definition (business review, project scoping and planning)
    2. Detailed analysis and design of the solution (architecture and design, development environment setup)
    3. Development and testing of the solution (Development, testing, rollout planning, production environment setup)
    4. Go live to production (Rollout preparation, go live)

    Fig 3. Integration project phases

    Most of the challenges that may arise come from over-focusing on the development (phase c), compared to project planning analysis and design (a and b). While this is common on all IT projects, it is essential for enterprise integration.

    Technical challenges can be grouped to the following categories:

    Enterprise Integration Challenges

     

    Incomplete or Not Well-Defined Analysis

    Business people should know exactly what they are looking in an integration project, and what the deliverables will be. The delivered solution may include processes for several areas, departments, so the project team should have an overall knowledge of business processes.

    This also apply for IT people, since applications and systems involved are numerous and may run in different platforms and heterogeneous environments. In terms of technical analysis and design and project management, this makes the composition of individual systems and application into a composite one, difficult to manage. Additionally high consulting rates prevents often, receiving external support.

    Not open platforms

    There are systems that provide limited interface or non-standard capabilities. Other systems require a license to upgrade and extra costs to customize. In such cases, you might need a custom development (from integration platform side) solution, resulting in extra effort, time and cost.

    If a system is unsupported, closed and does not have an interface, you should dispose it.

    Vendor management

    Clear vendor management is critical for an Enterprise Service Bus Integration project when:

    - Vendors cannot do modifications to application for several reasons, or require a new license

    - The result of a requested modification may not fulfil time specifications

    - Arguments and conflicts regarding areas of responsibility may arise

    - Application vendor may not support custom modifications in future upgrades

    In these cases, you must strictly define your requirements, responsibilities milestones, and deliverables within your vendor agreement

    Performance, Sizing and High Availability  

    You should know your performance objectives upfront. Some of things you should consider include:

    - What is the most important factor, number of supported concurrent users, response time or system throughput?  

    - Is a stress test required?

    - How someone can measure bottlenecks or poor performance?  

    As general guide to face performance issues:

    - In design components that talk frequently should be placed in the same network

    - If possible, workload should be distributed

    - Bottlenecks should be located during tests

    - Focus on processes that take the 80% processing

    - Perform step by step changes. Measure impact before apply

    - Keep historical logs

    Scalability is the ability of a system to handle increased workload. Basic question you should ask in the first step is:  vertical or horizontal scalability?

    The first is the quick, less expensive solution (e.g. increase memory in a server). The second is the long term: adding a new server in a clustered environment, implementing an external load balancer, etc. Usually, a combination of both is applicable.

    Every business that runs an enterprise service bus integration platform wants their integration platform to be available as much as possible. Since this is strongly relates to cost and performance, you should figure out how much downtime your business can tolerate prior to design and implementation.

    Transaction Processing

    Transactional integrity: while it is very common for multiple integration services to execute atomically, we require either an “all complete” or “all no complete”. (Most RDBMs and ERPs provide such functionality)

    Most integration platforms provide transactional processing capabilities with some additionally guaranteeing the processing of documents.

    Setting Up Test Environments. Incomplete Tests

    Setting up in integration project test environments requires duplicating (simulating) your production ones, including all systems and applications involved. Additionally, it requires that you establish all network connections between test components.

    This may prove difficult in some cases:

    - Integration between real time applications or applications that require hardware resources not available (e.g. control systems)

    - Integration between cloud applications or integration between applications that may require extra licensing for implementing a test environment  

    Usually, you perform tests in batches on specific dates -according to the project plan-, with the intermediate time dedicated to corrections and improvements. This requires management effort for not having any incomplete tests.

    Security

    Since data can come from different applications, you need data access control in your integration platform when that Integration platform has an end user component.

    Data mapping

    If some of your applications include entity information, data mapping comes 1st priority when you integrate these applications.

    Usually, you resolve this by applying the “canonical” model, a defined superset of entity’s fields and attributes that the integration endpoints should follow.

    Monitoring Tools are not Implemented

    You often find integration management software missing from your final solution, usually for cost reasons.

    Management and monitoring tools provide information to end user, regarding integration executions, execution statuses, timestamps, which are often valuable in troubleshooting  

    In conclusion, in an enterprise application and process integration project based on enterprise service bus architecture, your  biggest challenge is to focus on analysis and design -both operational and technical. This is because these projects often deal with different applications, technologies and processes that make these projects complex. 

    Find out how to get more from your cloud-based and on-premises systems without sacrificing security.

    Download the eBook Now.

    GET THE EBOOK

    Composite Cloud

    Composite Cloud

    Topics: Application Integration

    Paul Nilsen

    Paul Nilsen

    Paul Nilsen is the director of solution development at Cetrix Cloud Services. He also works with the Solution Development Department at Cetrix Technologies, where he headed the department for three years. Paul has over 20 years of experience in IT, working on software engineering projects, hardware-software product integration, design of special purpose systems using artificial intelligence, and implementing CRM and ERP software systems.

    Previous Post

    Best Microservices Framework for Java

    Next Post

    Artificial Intelligence in Digital Process Automation

    0 Comments

    Integration Architecture Patterns for Digital Transformation-01

    WATCH DEMOS

    CASE STUDIES

    GET THE EBOOK

    Related Blogs