API Integrations
API Integrations are one of the most important parts of building an application. Integrating APIs allows applications to share data and communicate with each other freely.
According to Statista, a business is connected to an average of 110 SaaS applications, and some of the biggest companies can have thousands of API integrations.
An engineering team’s resources are finite. The more time they spend integrating APIs, the less time they have to focus on their product’s core functionality.
For every third party API integration engineers build, they will face the following challenges
1. API Authorization
Engineers need to understand the API provider’s authorization protocol before connecting to their API. For example, an API provider may require OAuth2 authorization for connection.
2. Understanding API provider’s documentation
One of the most common complaints about a third party’s API from engineers is poor documentation. Poor documentation hinders engineers’ progress on their applications. Poor documentation slows down the process of building new integrations.
They need to understand what is required to communicate with these APIs and the data being returned.
3. Maintenance
Once the integration is built, an engineer needs to handle maintenance and upgrading. Any changes made to the product’s functionality can mess up the entire process of accessing or manipulating data.
Maintaining APIs can be frustrating because APIs can change for reasons such as
- Authentication type changing
- Endpoints being added, modified, or removed
- Changing the data returned by endpoints
Building a Single integration
In the simplest scenario, building a single API integration will look something like the diagram below, and the integration must address the same challenges mentioned earlier.
Building multiple API integrations
Engineering teams spend significant effort building one integration, and their product will more than likely have multiple integrations.
For every integration built, engineers face the same challenges mentioned above.
With the pressure to deliver software as quickly as possible, small engineering teams will likely spend most of their time building these integrations instead of working on their core business functionality.
Building multiple API integrations will look something like the diagram below, and it must address the same challenges mentioned earlier for each integration. (For simplicity, only three API integrations are shown as there could be more.)
As you can see below, it is easy to see how things can get complicated very quickly.
Goal
During my time at Poindexter Inc., we wanted to launch a new product called Gridabl and move their existing customers from their legacy product to Gridabl and integrate with the following applications quickly
- Xero
- Quickbooks
- Stripe
- Google Analytics
- Salesforce
Past decisions made
In the past, our team previously built a SaaS application called Poindexter, which is integrated with Xero.
We faced the challenges mentioned above and below is our experience with each of these challenges
1. Authorization
Our team needed to understand the OAuth2 authorization workflow to integrate with Xero, and we faced some challenges.
OAuth2 authorization involved understanding which scopes we were required to access based on the data we needed.
Our team needed to build the logic to track the expiration date of the access token to request a new access token from Xero’s authorization server by providing a refresh token.
Poindexter’s Token Refresh Workflow
- If the access token is set to expire soon, Poindexter sends a requests to Xero’s authorization server with a refresh token for a new access token.
- With the new access token, Poindexter makes a request to Xero’s resource server for a protected resource
If the access token is not about to expire, there is no need to request a new access token from Xero’s Authorization server.
2. Documentation
Our team spent significant time understanding Xero’s documentation and how it fits into the application. There was also some difficulty getting the answers we needed as we are located in different countries.
3. Maintenance
Our team would maintain the integration with Xero, and if anything went wrong, we needed to fix it immediately. As mentioned above, maintenance adds more time working on the integration, and our responsibility is to make sure that the integration is working smoothly.
For example, if we wanted to access bank transactions from Xero, we would need to access the bank transactions endpoint from Xero’s accounting API. Xero could change the name of their endpoint, and we would need to track the change and implement a fix.
Lessons learned
We wanted to avoid the same mistakes with Gridabl and did not want to go through the challenges of integrating multiple APIs. We needed a way to integrate APIs quickly and abstract the challenges of building API integrations.
Solution
We decided to use an Integration as a platform tool to accomplish this objective. We evaluated different tools based on cost, functionality, security, and usability.
We chose Paragon to solve this problem.
We leveraged Paragon’s Connect API, a proxy that communicates with any API we want to integrate. The difference between building API integrations directly with Xero and using Paragon’s ConnectAPI to Xero was significant. The effort integrating with Xero took minutes compared to months integrating directly with Xero.
Integrating with APIs using Paragon Connect
The Paragon ConnectAPI handles the challenges of building integrations such as understanding documentation, maintenance, and authorization.
- The user requests a Paragon user token from the server, and the Paragon user token is needed to access Paragon’s ConnectAPI.
(API is built in Ruby on Rails)
a. When the user requests a Paragon user token, the user’s id is sent to the server as part of the request. Within the server, the Paragon controller handles the request in the API.
b. A payload with the user_id, time token was issued at, and the expiration date and time.
c. With the signing key provided by Paragon, a Private key is created.
d. With the payload created earlier and the private key, the JWT is encoded, and returned as a response to the client.
2. User authenticates with Paragon using the Paragon user token
3. Paragon ConnectAPI communicates with the APIs and receives a response
4. Paragon ConnectAPI returns the response to the client
Challenge faced with Paragon
Paragon has limited documentation, which made it a little bit difficult to translate, and if we had any questions related to the tool, there weren’t many resources online to help resolve the issue since the company is very young.
One of the issues we encountered was not accessing the reports endpoint from Xero’s API, and it was because Paragon did not have that scope that was required to access that endpoint when we were adding the integration.
To resolve the issue we were having with accessing one of Xero’s endpoints, we reached out to Paragon’s engineering team and let them know that we needed to add a specific scope to access that endpoint since we had to rely on them to make the change and their engineers were able to implement a fix quickly.
Summary
To recap about API Integrations, here are some points to remember
- Integrating APIs allows applications to share data and communicate with each other freely
- The most common challenges of integrating APIs are API authorization, understanding documentation, and maintaining API integrations
- Understanding the API provider’s documentation
- Authorization to access APIs
- Maintaining API Integrations
- Engineering teams will have to face the same challenges every time they integrate a new API
- Integration as a Platform tools such as Paragon abstract away all of the challenges of integrating APIs, so that engineers can focus on other tasks
- The advantage to integrating directly with APIs is that data trust and technical trust can be ensured
Resources
- https://www.krit.com/blog/custom-integrations-are-a-total-headache-heres-why-theyre-worth-it
- https://www.saastr.com/the-top-saas-companies-have-an-average-of-350-integrations/
- https://www.digitalocean.com/community/tutorials/an-introduction-to-oauth-2
- https://www.quora.com/What-are-the-difficulties-in-API-integration
- https://www.quora.com/How-difficult-is-it-to-integrate-a-web-API-and-whats-the-process-you-take-when-you-integrate-an-API-What-challenges-do-you-face
- https://www.useparagon.com/blog/6-things-you-should-never-build-in-your-saas#integrations
- https://api2cart.medium.com/what-does-it-take-to-build-an-api-integration-c73f5ec88d45
- https://developers.google.com/identity/protocols/oauth2/web-server#uri-validation
- https://blog.cloudrail.com/bad-api-documentation-why-and-what-you-can-do-about-it/
- https://www.pandium.com/blogs/3-best-practices-for-api-documentation
