Deploying APIs on Netlify

Investigating a better way to serve APIs that scale

Ever get sick of dealing with servers? Complicated workflows? Deployment hassles? How about costs when you’re still bootstrapping a project, or when it’s growing? Worried about having to balance between being overly scrappy and overbuilding? There are solutions for web interfaces but how about APIs?

The building blocks for great software are certainly there, but the process of putting them together is simply too hard. A movement is well underway to ditch the infrastructure concepts of the 90’s and move as processing away from the server. Rich applications can live in the browser and have lightweight communication with a backend via REST API.

Deploying this web code is fairly easy—you’ll need to compile and bundle-up static assets like CSS, HTML and JavaScript. These need to be distributed quickly and cached by end users whose browsers do the hard work of rendering and visualizing the app. A global CDN can easily do the trick, providing it can keep up with the needs of the development flow.

The API, however, isn’t quite as simple. There is typically a large amount of software needed to implement the API routing, payload parsing, error handling, etc. This can be heavy weight, hard to configure, and expensive to deploy. Imagine you have a global CDN serving assets all around the world—to what back end will end users talk to? Do you need a global distributed set of API servers? Sounds pricey and difficult to manage.

So if one could have a global CDN, a fast, serverless model for API execution, and a simple workflow for deploying to all of these that is friendly for developers, we’d be set, right? There are a number of serverless systems in production now, but let’s be honest, they’re a bunch of work to deploy to.

A remarkably elegant solution comes from Web Tooling company Netlify, which has taken a developer-first approach to workflow, bundled with a proprietary global CDN and connection to Amazon Lambda. Developers simply push code to their SCM of choice and Netlify will build, configure, and deploy across the world—all for free. Simply push code to your master branch and within a few seconds, Netlify has built and distributed your code across the world. No webhooks, custom scripts, etc. You can literally run your entire workflow through GitHub without much fuss.

But what about the difficulties of the API? How can Netlify + AWS Lambda make our dev process great?

By a simple configuration in the Netlify configuration, JavaScript files in a specified folder will be deployed as AWS Lambda functions. They are then available through a simple URL based on the name of the implementation file (/.netlify/functions/{file}). Execution runs under Netlify’s account so you don’t need to pass any AWS keys to them. Heck you don’t even need an AWS account to use Lambda now! If you exceed the generous limits on the free plan, you simply pay for more. It’s not only reasonable, but the right way to manage and deploy your software.

That’s great, but it’s still too hard. At this point we have a great development flow, deployment process, and infrastructure to do work. But how about that API? Just calling an HTTP endpoint doesn’t really help a bunch, because we have parameters, headers, payloads, response codes, CORS, etc. There is still a lot left to do!

Luckily there’s a solution. Naturally it involves the OpenAPI Specification and some simple but critical open-source dependencies. What we really want is to create our OpenAPI definition and specify all the methods in it. The operations will be backed by JavaScript functions, which are passed the arguments as defined in the OpenAPI definition. It will return a response after performing our precious business logic, and the caller will receive exactly what they expect, exactly in the format we told them. Sound friendly?

The glue between OpenAPI definition and JavaScript function is a tiny library called swambda (get it? Swagger + Lambda = swambda). The OpenAPI definition says exactly what file to instantiate (controller), the method to invoke (method) and what arguments to pass (parameters). We then extract, validate, and pass on the arguments to the method and return a format that is appropriate to the Netlify framework.

Let’s see how this works. Given a snipped of our OpenAPI definition:

/pets/{petId}:
  get:
    summary: Info for a specific pet
    operationId: getPetById
    x-swagger-router-controller: Pets
    tags:
      - pets
    parameters:
      - name: petId
        in: path
        required: true
        type: string
    responses:
      200:
        description: The pet requested
        schema:
          $ref: "#/definitions/Pet"

We have an extension x-swagger-router-controller which specifies the controller file to use. The operationId will specify the method in that file, and parameters to define what arguments will be passed to the method.

Thus in the Pets.js file, the method getPetById will be called like such:

const getPetById = exports.getPetById = (args) => {
  const petId = args.petId;
 /* return a promise and resolve with the pet data */
}

That’s it! The hard work of routing, value coercion and validation is all done for you. The Lambda function is actually simple to create and initialize with the swambda library:

"use strict";

let swambda = require("swambda");
const handler = exports.handler = (event, context, callback) => {
///

Inside the handler, we create a cache. Why? Because there is some overhead in initializing the swambda function, such as parsing the OpenAPI definition and resolving references. Caching to global (I know, this sounds dirty) allows this setup work to be preserved, and since AWS will keep Lambda instances running for a bit when idle, the setup time could have a non-trivial performance boost. This is a common pattern when using Lambda to avoid setting up database connections, etc. It looks like this:

  swambda.cacheWith(global);

  swambda.fromCache()
    .catch((err) => {
      // not in cache, need to create

      // load via webpack yml-loader
      const swagger = require("yml-loader!./swagger.yaml");

      // set the route path
      return new Swambda("/.netlify/functions/main")
        .cors()
        .controllerDir("controllers")
        .load(swagger)
        .then(router => {
          return router;
        });
  })
  .then((router) => {
    return router.process(event);
  })
  .then((result) => {
    callback(null, result);
  })
  .catch((err) => {
    callback(null, err);
  });
};

Walking through this setup:

• We’re using global as our cache
• We try to get the swambda router from the cache. If it doesn’t exist, we’ll set it up
• We tell swambda that requests will be received on the /.netlify/functions/main basepath. Everything after this path will be used for routing to paths in the OpenAPI document
• We’re enabling CORS support
• Business logic controllers will live in a folder called controllers

Requests are then passed through the router and eventually returned to the callback function.

Here is a full example, including a webpack configuration file:

https://github.com/fehguy/swambda/tree/master/examples

So now, we can use any tool we want to create the OpenAPI definition. We can implement our business logic without knowledge of routing framework, deployment process, or other. The swambda framework will perform the function of gluing the Lambda system and your code simple, and along with the same developer-friendly workflow as the web assets.

This gives a solid base to build a product from. Traditionally APIs are expensive to deploy and hard to manage, or are low performing and in general hacky. With this new approach, there are no idle server costs, setup fees, pre-production provisioning steps. If it takes off, we have near infinite scale via AWS Lambda. We get a huge meal for free, and once we’ve exceeded that, there is a palatable fee associated with the infrastructure. We don’t need complicated integrations between 3rd party services, and the entire system—soup-to-nuts is both transparent and portable. If you don’t like Netlify, you can simply redo the plumbing that provide on your own. Sound like we have cake and get to eat it too, right?