Integration middleware for enterprise systems

One hub in place of isolated integrations.

The problem

The integrations between an enterprise ERP and third-party systems were isolated, with no security standard or traceability, and had already duplicated hundreds of records in the target system.

Overview

Source
- Enterprise ERPMembers and contracts
Staging area
- Intermediate tablesValidates before sending and keeps each record’s history
Integration platform
- Periodic incremental loadOnly what changed
- Event queueProcessed every few seconds
- Contract routineTest send and dry run by default
Destination
- Third-party systemsOne send per record, no duplicates
Monitoring
- Monitoring dashboardRead-only
- Failure report by e-mailIn business language

Across the whole platformAuthentication, credential vault, auditing and automatic reprocessing: temporary failures are rescheduled with growing waits; only the administrator resends, one record at a time.

How the ERP and the third-party systems talk to each other without talking directly.

What I built

A central integration platform with authentication, a credential vault, auditing and a scheduler ready to use.
An architecture built for reuse: a new integration is a new module, with no need to rebuild the base.
Failure detection and automatic reprocessing, with a history of what was synchronized.

Under the hood

A waiting area between the two systems

Data goes through intermediate tables before moving on to the partner. This decouples one system from the other, lets us validate before sending and keeps the history of each record.

Three independent processes

A periodic incremental load, an event queue processed every few seconds, and a contract routine that validates with a test send and runs in dry-run mode by default. This ended the manual change for every new contract.

Failure handled as routine

Temporary failures are rescheduled with growing waits between attempts. Past the limit, the record is marked as an error and the business area gets an e-mail report in business language, not in error codes.

Resending without duplicating

Only the administrator resends, one record at a time, and a resend does not overwrite what already exists in the target. A read-only monitoring dashboard shows the state of each integration.

The result

More than 115,000 records scanned in a full load; about 111,000 synchronizations in 24 hours.
About 40,000 historical inconsistencies found and fixed.
Failures detected within 8h30 at most and reprocessed within 30 minutes.

Have a similar problem?

Tell me the scenario and I will say, no strings attached, where I would start.

Get in touch