How to document software integration

How to document your software and data integrations so that you save time and avoid errors?

Roughly speaking, there are 3 areas that must be documented

Functional process description.
The functional process that is automated by the integration is explained in this document.
Integration catalog.
A list of integrations, with which he functional process can be automated
Integration Design
The technical design of the integration(s) to be realized.

Functional process description.

It starts with a functional description of the process which will be automated by the integrations. This functional description has been written in plain language e.g. everyone must be able to read and understand it well.

For example:When an invoice is created in Salesforce, it is automatically posted in Exact. In Salesforce it is possible to choose different products for individual invoice lines. These products are managed in Exact. Salesforce must show whether the invoices have already been paid.

This description can be so short and to the point, but the more detailed the better.

Integration catalog.

The integration catalog is an overview of all integrations that are required. The purpose of this overview is to record which information flows are present in your project.

In the integration catalog you record the following:

What goes from A to B
Source system
Target system
Is the information Realtime or Batch?
How often should the information go back and forth (weekly, daily, every minute)
Which direction does the data go, is that from A to B or B to A.
Unique identifier for each integration (eg int-0001)
Who is the master of the data. (system of record)
Type Data returned:
- json
- xml
- csv

Download an example of an Integration Catalog.

Integration design

For each integration in the catalog the integration design describes the following: :

Mapping– This describes where data from the source fields should go. In the mappings you will also record which information is mandatory. “Source field A should be copied to destination field B”
Transformation If the weight on the source is registered in kilograms and the weight field on the destination is registered as gram, a transformation should be applied as well. “Source field A / 1000 should be copied to destination field B”
Flow:
the flow design you determine which process steps are being made to get information from A to B. It is important here that both the happy flow and the non-happy flow are designed.
Integration scenario: in the integration scenario document you record which steps must be taken and how this can be tested. Download an example of an integration scenario.

Integration flow diagram, example.

Points of interest when designing a batch job;

How often do I want to repeat a job if it is wrong?
How do I handle transient errors? Am I going to do a re-try or let the job fail right away and send an email.
How do I check whether a batch is successful or not? It is often a good idea to create success lists and email them onSuccess. A central email address that the whole team can access is a good idea.
How do I notify the end user that a batch has failed? You can do this, for example, by sending an automatic mail onFail or to immediately return an error to the source system.

Points to consider when designing and testing an API:

What is the maximum load time?
What is the responsibility of my API?
How do I arrange authentication [Credentials, Token, etc]
It is particularly important in the preliminary phase that you consider:
- What data will be included?
- What data do I return?
- Which codes do I return and how do I test this (200,201,400,401,403,500)
Make sure you always create an OpenApi / Swagger document for the consumer of your API. This makes communicating and testing your API easier.
Tip: start a project in postman. For every call that can be made to your API, make sure there is an example request + test.
Tip: if you are going to consume third party APIs, it is best to use postman. You can document all requests and responses here (and even share them with your teammates).