Tips for tool-building using OpenAPI

One such task for us at Nord Security is to create client SDKs for the APIs and keep them up to date.

What's the problem with client SDKs?

Let's say we build a simple gamer API that would allow gamers to register:

1
POST /api/v1/gamers

We then create a php-gamer-sdk client so that other projects could use it. After some discussion with the other team, you find out that we need a golang SDK as well. Easy - you create a go-gamer-sdk for everyone to use.

Then you receive a message from the nodejs gang: why is there no client for the almighty typescript? They do have a point, so you add yet another client - js-gamer-sdk.

Oh, shoot! We forgot to add an endpoint for retrieving a gamer by UUID!

1
GET /api/v1/gamers/{uuid}

Now comes the tedious and lengthy task of updating all the client-SDKs. All the developers now need to know 3 languages to be able to maintain the codebase.

An alternative to manual SDK updates

Instead of creating the SDKs manually, we can use OpenAPI. OpenAPI is a standard for describing APIs using plain old YAML files. This standard (previously known as Swagger) uses a format that can be read by both humans and machines.

How to get started

Every OpenAPI file starts with the basic info:

1
openapi: 3.0.0
2
info:
3
  title: Gamer API
4
  description: An API for creating and managing gamers
5
  version: 0.1.0
6
servers:
7
  - url: https://gamer-api.pci.test/api/v1
8
    description: Local development server

The first line defines the version of the standard used. We then add additional info about the microservice, including the servers where you can access our API.

Now it's time to actually describe the endpoints.

1
paths:
2
  /gamers:
3
    post:
4
      operationId: createGamer
5
      summary: Create a gamer
6
      responses:
7
        '200':
8
          description: Gamer creation succeeded!
9
        '500':
10
          description: Unexpected issue detected, try again later.

We first describe the path of our endpoint as /gamers and the method that is used to access it as post. After that, you assign a name to this endpoint via operationId and give a brief description in the summary field. Finally, you specify what response codes could be returned by the API.

One thing is missing, though: there is no info on what kind of data the API receives to create a gamer. Let's fix that in the next section.

Accepting request parameters

Describing incoming data is pretty easy using the requestBody property.

1
paths:
2
  /gamers:
3
    post:
4
      operationId: createGamer
5
      summary: Create a gamer
6
      requestBody:
7
        required: true
8
        content:
9
          application/json:
10
            schema:
11
              type: object
12
              properties:
13
                # define all your json properties you will pass in the request

We want to show that the request parameters are non-optional, so we mark the content as required. Note that we will pass JSON to the API endpoint, which is why we tell OpenAPI that the request must be in application/json format.

Now comes the important part: we describe the type of the request data. There are many types to choose from, but for this example, we will describe the data as an object. Now all that's left is to define its properties.

1
paths:
2
  /gamers:
3
    post:
4
      operationId: createGamer
5
      summary: Create a gamer
6
      requestBody:
7
        required: true
8
        content:
9
          application/json:
10
            schema:
11
              type: object
12
              properties:
13
                nickname:
14
                  type: string
15
                  example: 'monkKey'
16
                clan:
17
                  type: string
18
                  example: 'chillpill'

As you can see, you can add examples of the fields you can pass. For the gamer to create an endpoint, the following object can be passed:

1
{
2
  "nickname": "monkKey",
3
  "clan": "chillpill"
4
}

The same approach can be used to describe responses.

1
      responses:
2
        '200':
3
          description: Gamer creation succeeded!
4
          content:
5
            application/json:
6
              schema:
7
                type: object
8
                properties:
9
                  uuid:
10
                    type: string
11
                    format: uuid
12
                    example: 'e5159ff7-9e12-11ec-ad6c-2cdb0742b873'
13
                  nickname:
14
                    type: string
15
                    example: 'monkKey'
16
                  clan:
17
                    type: string
18
                    example: 'chillpill'

As you can see, the response of our API is almost identical to the request, except that an additional uuid field is returned. Note the format property, which serves as a hint to the content of the property. Judging from the current OpenAPI file, the response is as follows:

1
{
2
  "uuid": "e5159ff7-9e12-11ec-ad6c-2cdb0742b873",
3
  "nickname": "monkKey",
4
  "clan": "chillpill"
5
}

Here's what we have so far:

1
openapi: 3.0.0
2
info:
3
  title: Gamer API
4
  description: An API for creating and managing gamers
5
  version: 0.1.0
6
servers:
7
  - url: https://gamer-api.pci.test/api/v1
8
    description: Local development server
9
paths:
10
  /gamers:
11
    post:
12
      operationId: createGamer
13
      summary: Create a gamer
14
      requestBody:
15
        required: true
16
        content:
17
          application/json:
18
            schema:
19
              type: object
20
              properties:
21
                nickname:
22
                  type: string
23
                  example: 'monkKey'
24
                clan:
25
                  type: string
26
                  example: 'chillpill'
27
      responses:
28
        '200':
29
          description: Gamer creation succeeded!
30
          content:
31
            application/json:
32
              schema:
33
                type: object
34
                properties:
35
                  uuid:
36
                    type: string
37
                    format: uuid
38
                    example: 'e5159ff7-9e12-11ec-ad6c-2cdb0742b873'
39
                  nickname:
40
                    type: string
41
                    example: 'monkKey'
42
                  clan:
43
                    type: string
44
                    example: 'chillpill'

Using references

Even though our specification file describes a simple endpoint, it already looks like a mess - it is pretty difficult to read. There is, however, an easy way around that - references.

The idea behind a reference is simple - you can define something elsewhere in the specification and point to it. It's basically like defining a variable and using it later.

So first, define your request and response as reusable components.

1
omponents:
2
  schemas:
3
    GamerCreateRequest:
4
      type: object
5
      properties:
6
        nickname:
7
          type: string
8
        clan:
9
          type: string
10
    GamerCreateResponse:
11
      type: object
12
      properties:
13
        uuid:
14
          type: string
15
          format: uuid
16
        nickname:
17
          type: string
18
        clan:
19
          type: string

Then, you can reference both request and response definitions using the $ref keyword.

1
paths:
2
  /gamers:
3
    post:
4
      operationId: createGamer
5
      summary: Create a gamer
6
      requestBody:
7
        required: true
8
        content:
9
          application/json:
10
            schema:
11
              $ref: '#/components/schemas/GamerCreateRequest'
12
      responses:
13
        '200':
14
          description: Gamer creation succeeded!
15
          content:
16
            application/json:
17
              schema:
18
                $ref: '#/components/schemas/GamerCreateResponse'

Here I am referencing the schemas I defined earlier. As you can see, the file now looks much cleaner!

And with that, we are officially done with our specifications. Now, we can use it to generate things!

Generating a client

For client generation, we use an amazing tool called OpenAPI generator. This tool allows us to generate clients in a variety of languages. Let's use it to generate a PHP client.

First, you need to save all the API definitions we have so far into a YAML file and pass the path to it.

1
$ openapi generate -i /path/to/specification.yaml

Next, you need to pass the language you want to use for generation. We pass php, but we can just as easily generate libraries for java, nodejs, or even ruby if we want to.

1
$ openapi generate -i /path/to/specification.yaml -g php

We then pass the folder where we want to put the generated code using -o.

1
$ openapi generate -i /path/to/specification.yaml -g php -o ./gen/php

It's also important to name the package properly so that it could be added to package repositories. For that, --git-user* options are a godsend.

1
$ openapi generate -i /path/to/specification.yaml -g php -o ./gen/php --git-user-id=nordsec --git-repo-id=gamer-sdk

And finally, you can pass a bunch of additional configurations that slightly tweaks the naming of the namespaces, folders, and variables.

1
$ openapi generate -i /path/to/specification.yaml -g php -o ./gen/php --git-user-id=nordsec --git-repo-id=gamer-sdk --additional-properties=variableNamingConvention=camelCase,packageName=GamerApiClient,modelPackage=Dto,invokerPackage=NordsecGamer

And you're done - the command can now be run! Check your gen/php directory for the generated source code.

The OpenAPI generator not only creates a client but data transfer objects for requests and responses as well. Here's an example of how you could use the generated code:

1
$gamerCreateRequest = new GamerCreateRequest(); // \NordsecPayments\Dto\GamerCreateRequest
2
$gamerCreateRequest->setNickname('rocket')
3
$gamerCreateRequest->setClan('rocketRoll')
4

5
$gamerCreateResponse = $apiInstance->createGamer($gamerCreateRequest);
6
echo $gamerCreateResponse->getUuid();

Protip: If you don't want to set up an additional tool just to play around with OpenAPI, you can even use PhpStorm to generate code.

Just open your OpenAPI file and use the icons at the top.

Phpstorm OpenAPI support

Generating documentation

Fortunately for us, generating SDKs for APIs is not the only benefit of OpenAPI. There are tools that use the specification file to generate API references. This is a cheap way to create simple documentation.

For that, we use widdershins. Here's how you can quickly generate a slate-compatible documentation file:

1
$ mkdir slate
2

3
$ widdershins --search false --language_tabs shell php go javascript --summary /path/to/specification.yaml -o ./slate/index.html.md

This will generate documentation with code examples that are compatible with slate in the slate directory.

Generated slate documentation

Not only does this produce beautiful documentation, but it also doesn't require any additional effort. Nice!

Using Postman

That wasn't even the last possible use-case for the OpenAPI specification! OpenAPI can help you query your endpoints using postman. Instead of defining all endpoints yourself, use the import feature.

Navigate to File -> Import:

The postman import window

Then choose what you want to use the specification file for. We are going to use it as a "Test Suite":

Use the spec file as test suite

And finally, we click import and enjoy the results.

The main postman window

Once the file has been imported, you can see the newly created API and fire requests to it. This saves a lot of configuration, especially for both new developers and QA engineers.

OpenAPI - a powerful tool

Even though we've touched on quite a few usages of OpenAPI, we've barely scratched the surface. Because the format is readable by both humans and machines, more and more tools are created that take advantage of it. Therefore, OpenAPI is definitely a powerful tool for any API developer.