DECISION NODE

December 15, 2023
Decision Nodes

Extensible and Flexible 3rd Party Integrations: Leveraging the Infrastructure Persistence Layer

Leeho Lim and Rohin Bhargava
,ㅤ
Founders @ Multithread

Leeho and Rohin are the founders of Multithread, an engineering consultancy. Rohin previously spent half a decade at Microsoft on the government cloud (IC3) team, and previously founded Pigeon, a YC-backed company. Leeho has a background in product and software engineering. He previously founded 2 YC-backed companies, Encord and Parlan, and before this, was an early employee at a unicorn company.

At Multithread.ai, we frequently work with many venture backed founders who integrate other 3rd party services into their application for core services including payments, billing, authentication, customer service, logging, notifications, and more.

When designing for such services, we find that teams often do not account for 3rd party vendor flexibility and heavily couple services within their codebase. Commonly, startups have to outright swap vendors for particular functionality or accommodate for multiple vendors that provide the same type of application functionality in different regions. In these situations, teams can save significant time if the code is designed to be re-usable from the beginning.

We recently worked with a fintech provider who leveraged a 3rd party service to provide its users with bank accounts, debit cards, and more. In this case, the fintech provider would eventually work with 3rd party banking providers in different nations to provide the same repeatable services in multiple regions.

A first stab: naive service implementation using direct service invocations

When challenged with this task, we could very well have leveraged direct instance calling or even a simple factory locator. We find that many startups often implement 3rd party services into their code bases using the below two implementation approaches.

/////////////////////////////
// Naive implementation 1
/////////////////////////////
class BankService {
  constructor() {
    // Direct invokation
    this.externalBankService = new AThirdPartyProviderService()
  }
  
  // further method implementations
}

/////////////////////////////
// Naive implementation 2 
/////////////////////////////
class BankService {
  constructor() {
    // Factory locator service using inversion of control (ioc) 
    // but invoked within the service
    let externalBankServiceFactory = ServiceLocator.GetExternalBankFactory()
    this.externalBankService = externalBankServiceFactory
      .CreateExternalBankService();
  }
  
  // further method implementations
}
Open in Eraser

Most of the time, such an implementation is sufficient: for certain services within your application, you might not ever intend to swap out a 3rd party vendor or add additional 3rd party vendors to cover the same services (e.g. multiple vendors for payments). For example, most startups do not anticipate ever having more than 1 authentication provider (e.g. Auth0, Firebase, Supabase, ...). In this case, the following implementations, which invoke the 3rd party service directly into the constructor, might make sense. Such an implementation still allows the dependency class to invoke the 3rd party service, and admittedly, this design is much quicker to implement, optimized for speed over reusability.

This approach would especially be nice if you are designing a lightweight service that leverages a 3rd party service that you do not anticipate needing to swap out. Furthermore, if you are not wholly certain what common methods you might need from your 3rd party services, direct instance calling might be the best time-vs-value optimized implementation.

However, in the implementations above, there are a few noticeable issues:

  1. What if a banking provider leaves us and we are forced to find a new banking provider?
  2. What if the externalBankService is used in other classes? Are we expected to change each constructor?
  3. What if we have multiple external banking services that we would like our BankService to leverage from? For example, as presented above, we might have different currencies for different banking providers.
  4. What if we need to support multiple places where BankService is instantiated? What if each of those invocations has a different lifetime? What if the externalBankService has different lifetimes?

Introducing the infrastructure persistence layer

In order to address the issues above, we introduce the notion of the infrastructure persistence layer, which leverages the core concepts of domain driven design (repository pattern) and dependency injection.

Open in Eraser
class BankService() {
  constructor(thirdPartyBankService: IThirdPartyBankService) {
    this.thirdPartyBankService = thirdPartyBankService
  }
}

With dependency injection, the responsibility to choose and pass the right implementation is now the responsibility of the calling class (IoC). Extending this logic, we can now ascend the syntax tree and register the dependency at the appropriate level of our tree. This has the following advantages:

  • Code re-use is maximized
  • There is granular control of dependency lifetimes all in one place
  • The code is more domain driven, as the constructor for classes now list the dependencies

We now need to implement our IThirdPartyBankService :

// Note: method parameters have been simplified for readability. 

interface IThirdPartyBankService {
  createBankAccount(
    firstName: string,
    lastName: string,
    currency: Currency,
    accountType: AccountType,
    ...
  ) => BankAccount
  
  retrieveAccountBalance(
    accountNumber: string
  ) => AccountBalance
  
  depositFunds(
    accountNumber: string,
    amount: float
  ) => float 
  
  withdrawFunds (
    accountNumber: string,
    amount: float
  ) => float
}

Our IThirdPartyBankService allows us to accomplish our goal of standardizing a set of method inputs and outputs that we can use to implement a set of services for multiple 3rd party vendors. Consequently, any logic that touches our BankService will never have to worry about and be safely decoupled from the implementation details of the specific 3rd party services.

Now let's implement the set of services from each specific vendor:

class FakeThirdPartyBankBService implements IThirdPartyBankService {
  constructor(key: string) {
    // initializations for FakeTPBankB service
  }
  
  createBankAccount(
    firstName: string,
    lastName: string,
    currency: Currency,
    accountType: AccountType,
    ...
  ) {
    // Execution logic...
    ...
  }
  
  // and so on and so forth for all the methods that you want to implement...
}

Once we have our implementations, note that we can swap in and out of our implementations very easily with the following:

// You can easily instantiate BankService with either 
// thirdPartyBankAService or thirdPartyBankBService
const thirdPartyBankAService = new FakeThirdPartyBankAService(key=key)
const thirdPartyBankBService = new FakeThirdPartyBankBService(key=key)

// Option A
const bankService = new BankService(thirdPartyBankAService)

// Option B
const bankService = new BankService(thirdPartyBankBService)

You can swap Option A and Option B , and any other services or further layers that depend on BankService now are sufficiently protected if we change the implementation logic in either Bank A or Bank B. Furthermore, other code that depends on BankService are also protected if we need to add an implementation for Bank C.

While you could take this abstraction further down onto a method level rather than a service level, we find that this design is the most compromising for startup founders who need to optimize on both quality and speed. Such an implementation would future proof an early stage startup's code base, especially in highly volatile stages of constant vendor and code pivots.

Pitfalls

The primary pitfall to the infrastructure persistence layer is that it requires a well-planned interface that all 3rd party repository implementations can refer to. Any drastic changes to the general interface of a service would be no better than direct instance calling a service directly into its dependencies.

For example, imagine that we had the following:

interface IThirdPartyBankService {
  someMethod(paramA, paramB, paramC) {}
  someOtherMethod(paramD, paramE, paramF) {}
}

class SomeBankService implements IThirdPartyBankService {
  someMethod(paramA, paramB, paramC) {}
  someOtherMethod(paramD, paramE, paramF) {} 
}

class SomeOtherBankService implements IThirdPartyBankService {
  someMethod(paramA, paramB, paramC) {}
  someOtherMethod(paramD, paramE, paramF) {} 
}

If we were to update the method parameters for someMethod , we would have to go ahead an update the method parameters for all implementations of someMethod in SomeBankService and SomeOtherBankService . Furthermore, we would have to find every invocation of someMethod in our code base, and update that to match the updated parameter types. In this example, such changes would be no better than the naive method that we presented above.

TL;DR

In this article, we discussed the pros and cons of different approaches to implementing 3rd party integrations into an application.

The initial approaches, which propose leveraging direct class instance invocations or naive usages of a factory pattern, do not accommodate for the evolving and uncertain needs of startups, including vendor swapping or adding additional vendors. For example, we have worked with clients who had adopted 3rd party vendor solutions, only to rip out these solutions in 3 months time and spend another month doing so.

We avoid the pitfalls of the initial approaches with a sufficiently optimal approach. This approach leverages a sufficient degree of abstraction by implementing a infrastructure persistence layer: it introduces the notion of a generalized contract that multiple 3rd party services of the same type can implement, and it leverage the idea of dependency injection when instantiating a service object that can accommodate for swappable services. Using these two concepts, we've now presented an approach that allows startups to make  pivots or additions to their code base without spending additional weeks or months of implementation time.

We present the caveats to the infrastructure persistence layer, and consequently, we strongly advise that startups that conduct planning around the common interface for their 3rd party repository implementations.

Sources

  1. www.martinfowler.com/bliki/AnemicDomainModel.html
  2. www.infoq.com/articles/ddd-in-practice/
  3. learn.microsoft.com/en-us/dotnet/architecture/microservices/microservice-ddd-cqrs-patterns/infrastructure-persistence-layer-design

Subscribe for Updates

Exploring forks in the road of software design with a weekly newsletter.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.