In Part 1 of this series, we learned about the WebSocket protocol and how to set up our own WebSocket server in Node.js. Next, let’s explore how to use a public WebSocket API to access smart devices around a connected home.

REST and WebSockets for a connected home

When it comes to transmitting data in a connected home environment, both REST and WebSockets are commonly used protocols, but they have different characteristics and use cases.

REST follows a request-response pattern, where a client sends a request to a server, and the server responds with the requested data. This is useful for accessing and controlling smart devices and services, and works well for scenarios where data updates are not required in real-time. For example, you could use a REST API to turn on a smart light.

On the other hand, WebSockets enables bidirectional communication between a client and server, enabling real-time data transmission. This is useful for applications that require continuous data updates, such as real-time monitoring of sensor data and displaying live dashboards. For example, you could use a WebSocket API to continuously monitor the temperature in a room over a persistent connection.

In the next section, let’s take a look at a popular home automation platform that provides both REST and WebSocket APIs.

Home Assistant for home automation

Home Assistant is a popular open-source home automation platform that lets you control and monitor smart devices from different brands using a unified interface. Instead of using separate applications to control the kitchen lights, thermostat, and other connected devices all manufactured by different producers, you can manage almost everything from a single Home Assistant web dashboard running on a Raspberry Pi or other dedicated server within your local network.

Home Assistant is ideal for DIY smart-home tinkerers because it supports a wide range of integrations and protocols, allowing you to customize automation scenarios based on events, schedules, and sensor readings.

Next, let’s take a look at Home Assistant’s WebSocket API.

Home Assistant WebSocket API

In addition to a REST API, Home Assistant also contains a WebSocket API to stream information. To learn how to authenticate the WebSockets connection and send saved messages to the Home Assistant server, follow along with this step-by-step tutorial, watch the video, and reference the sample collection.

Using a long-lived token, you can use Postman to establish a connection with our Home Assistant server running locally, and then send and receive messages using the WebSocket API.

You can also configure your own Saved Messages to create your own customized themes and sequences.

Home Assistant also provides a REST API. Explore Home Assistant’s WebSocket and REST APIs side-by-side in Postman to better understand the differences between the two protocols.

Additional resources

You can work in Postman using different API patterns and protocols. Check out these Postman resources to learn more about WebSockets:

• Guide to Postman WebSockets collection
• Using WebSocket requests docs
• WebSocket requests video

Browse the Program smart lights public workspace for APIs from other providers, such as Philips Hue and Elgato, to automatically control smart lights in your home or office. And let us know in the comments below what kind of projects you want to learn about, and what you’re doing with WebSockets.

Photo by Intricate Explorer on Unsplash

I’ve previously shared how to build resilient APIs with chaos engineering and security hacking with the Big List of Naughty Strings. I’ve also broken some hearts, so I have some experience with breaking things. In this post, let’s go beyond basic test automation in Postman, and learn why intentionally trying to break things increases the resilience of your applications. We’ll also explore some common negative test scenarios.

Happy path vs. Unhappy path

During product design and development, stakeholders focus on intended user behavior. Product owners describe this happy path with business requirements. Engineers push code to enable users to do these things. A user who demonstrates this expected behavior follows the happy path.

These are examples of happy paths:

• User creates a new account
• User signs in with their new credentials
• User retrieves information about their account

But what happens when there’s unintended behavior, like when a user submits invalid inputs? These unhappy path scenarios are often neglected in design and development. But with enough traffic, unexpected inputs will inevitably come from both well-intentioned and malicious users.

These are examples of unhappy paths:

• User creates a new account without providing all required inputs
• User signs in with invalid credentials
• User pastes a SQL query that gets executed (maliciously or unintentionally)

A strong test plan covers both happy and unhappy paths. Positive test cases describe happy path scenarios, where errors result in failed tests. Negative test cases describe unhappy path scenarios, where expected errors result in passed tests.

Trying to break the Unbreakable API

In this livestream with my colleagues Trent McCann, Quality Engineering Manager, and Evan Lindsey, Lead SDET, I tried to break the Unbreakable API. This API was engineered by Evan to demonstrate the importance of validating inputs and handling errors.

“How to break an API” livestream

Typically, architects and engineers of an API are very familiar with the vulnerabilities of that API. However, they’re focused on enabling happy path user stories. They may not be thinking through all possible scenarios a user might encounter when interacting with the API. In the next section, let’s explore some conventional ways to break an API.

A Recommended Testing Flow

*Unbreakable API Lite* is a simplified version of *Unbreakable API* used in the “How to Break an API” livestream. Instead of managing authorization for multiple roles, the role of *admin* is removed and *employee* is renamed to *user*. In *Unbreakable API Lite*, once you create a user and set the returned token, you will have access to all available endpoints.

The collection [Testing Flow for Lite](https://www.postman.com/postman/workspace/unbreakable-api/documentation/1559645-c2db0576-6f29-4338-908e-25892275fe03) includes examples of positive and negative test scenarios for the API referenced in Unbreakable API Lite. For engineers, Testing Flow for Lite can be a guide for input validation and error handling. And for testers, it can be a recipe for mischief.

Try it out in Postman

1. Fork the collection: Fork the collection [Testing Flow for Lite](https://www.postman.com/postman/workspace/unbreakable-api/documentation/1559645-c2db0576-6f29-4338-908e-25892275fe03) to your workspace. You may need to enable your public profile if you haven’t already.
2. Create a new user: Find the User create request, and update the values under the Body tab. Hit Send to create the new user, and also set a collection variable called userToken that can be used in subsequent calls.
3. Step through the collection: Explore the positive and negative test scenarios outlined in the remaining folders. Under the Authorization tab, notice the authorization method used for each request. Under the Pre-request Script and Tests tabs, notice code that runs before and after you send each request.
4. Run the collection automatically: This collection can also be run to automate your testing flow. This is done using the Runner in Postman, Newman from the command line, or Monitors on Postman servers. Remember to set up a new, unique user under the User create request before running the collection in its entirety.

Positive and Negative testing

The examples in the Positive folder of Testing Flow for Lite describe happy path scenarios, such as:

• Retrieve all movies
• Create a new movie
• Retrieve the new movie by ID

For positive test cases, you write a test asserting a successful response like receiving a 200 OK, or similar, HTTP status code. If there’s errors in those assertions, the test should fail. For example, to test the endpoint of a user creating a new movie, write a Postman test and assertion like this:

pm.test("Status code is 200", function () {

pm.response.to.have.status(200);

});

The examples in the Negative folder describe unhappy path scenarios, such as:

• Retrieve all movies, using an authorization token when not needed
• Create a new movie, using an invalid authorization token
• Delete the movie, when cascade is not set on database cleanup

For negative test cases, you write tests expecting certain errors. If your application doesn’t return expected errors or swallows exceptions, your application isn’t handling unexpected input gracefully. In these cases, your tests should fail if you don’t see the expected errors.

For example, what happens when a user creates a new movie and submits an invalid authorization token? We expect the server to return a 401 Unauthorized, or similar, HTTP status code. To test that case, send a token that you know is invalid, and write a Postman test and assertion like this:

pm.test("Status code is 401", function () {

pm.response.to.have.status(401);

});

The importance of negative testing

With negative testing, you are asserting the application accommodates unexpected user behaviors. When unexpected user behavior crashes the whole system, engineering rushes to put failsafes in place, adding tests to ensure they are handling that exact scenario correctly going forward.

With negative testing, you’re proactively exploring these edge cases to increase the resilience of your APIs in production. It’s important to establish a workflow and foster a supportive organizational culture to do this, before your system fails.

Level up your testing in Postman with these additional resources:

• Test examples in Postman public workspace
• Acing your API tests — what you need to know for test automation blog and examples
• Chaos engineering to increase resilience blog and recipe
• How to break an API livestream
• Security hacking with the Big List of Naughty Strings livestream

🙏🏼 Technical review by Evan Lindsey.

In the Marvel Cinematic Universe, Bifrost is the name of the rainbow bridge that allows instantaneous travel between the realms of gods and humanity. Similarly, and equally magically, our Bifrost websocket gateway lets Postman clients instantaneously connect to Postman services.

Photo by Toni Reed on Unsplash

As I’ve previously shared in How Postman Engineering Does Microservices, all software architectures are a continuous work in process. Operating in the real world means occasionally re-evaluating old ways of thinking to adjust to new circumstances. That is the natural evolution of software design.

Here is the story of how Postman engineers developed the Bifrost websocket gateway by chipping away at a service that grew too big.

Development teams at Postman

Most development teams at Postman work in cross-functional squads focused on a single core domain, such as documentation or version control. Guided by the principles of domain-driven design, each squad develops internal microservices and Postman features for Postman users.

While most engineers work in squads, some work in functional teams that build components shared across the entire engineering organization. The Server Foundation team is an example of a functional team at Postman. These engineers create the utilities used by other squads to build, ship, and observe their own features. This team is also where the resident AWS and infrastructure experts reside.

the Server Foundation team is an example of a functional team at Postman that creates and manages stuff used across the entire engineering organization

Most microservices at Postman are loosely coupled so they can evolve independently of other teams. Unfortunately, a service can sometimes grow too big, providing a breadth of seemingly unrelated services. These services allow the team to rapidly iterate, but may start acting more like a bloated monolith, a big ball of mud, or whatever you want to call these unwieldy creatures.

When this happens at Postman, many engineers across different teams end up contributing to the code, requiring careful coordination across every team for each and every update.

The monolithic Sync service

One of the Postman services that grew too big to be managed efficiently is called Sync. It has the daunting task of synchronizing all the activity in the Postman client on your local machine with Postman servers. Every user action in Postman results in a series of API calls handled over websocket connections, following a publish-subscribe pattern, so that information flows in real time between users and across teams.

For example, this is what happens when you log into Postman and update a collection:

1. You add a parameter to the Postman collection.
2. Postman keeps a record of the update in version control stored with your profile.
3. Postman displays the latest information to viewers of the collection in real time.

Sync was originally intended to handle database transactions, like updating a collection. However, this time last year, Sync also managed additional activities, such as notifying and displaying the latest version to everyone subscribed to the collection.

Sync under pressure

When you’re building a car, the frame is the main supporting structure to which all other components are attached. A tiny crack in the frame might not seem like a big deal. It could probably go unnoticed driving around at low speeds. At higher speeds, however, there’s a ripple effect escalating misalignments. The seemingly insignificant crack allows vibrations to amplify throughout the rest of the vehicle until it escalates into a flaming wreckage.

“Stuff that goes unnoticed in smaller systems becomes inescapable in more complex systems.”
 — Kunal Nagpal, engineering manager at Postman

Sync was one of the earliest services at Postman, and its monolithic architecture allowed the team to ship Postman features quickly. Over time, it began handling more and more responsibilities. To this day, the Sync service still has widespread influence across the engineering organization, and lots of engineers feel the pain when Sync behaves unexpectedly or there’s scheduled downtime.

In 2019, Sync was handling both websocket connections and database transactions. With more and more collaboration happening among our 11 million users at that time, Postman was approaching a million concurrent connections at peak load.

As the foundation for virtually every microservice at Postman, the strain on Sync was growing.

• Cascading failure due to backpressure: Every deployment to Sync results in disconnecting Postman clients connected over websockets. When a million sockets reconnect, server resources are degraded, which can then result in more disconnections, causing a predictable but unavoidable surge that can take 6 to 8 hours to recover.
• Impacting user experience: Even though it didn’t happen often, dropped connections meant an occasional delay in seeing the latest updates and activity in a Team Workspace.
• Higher cost of maintenance: Since every squad relied on Sync, virtually every engineer at Postman had to learn how to handle dropped connections, initiate new ones, and then reconcile any conflicts in the data.

The Server Foundation team knew they wanted to increase the efficiency of websocket connections, and also handle them separately from the Sync service. The destination was clear, but the path to get there was not.

“This is the natural evolution of software design. Microservices start nimble, but they build up, and need to be broken back down. We wanted to separate socket handling from Sync because we were about to introduce a lot more functionality.” -Yashish Dua, software engineer at Postman

some internal services grow too big requiring careful coordination across teams

Here is what happened

Step 1: We got organizational buy-in

The first challenge to tackle was not a technical one. This was not Postman’s first ambitious rollout. Engineering had learned from past go-arounds to begin with the people. Starting in October of 2019, the Server Foundation engineers held a series of reviews dedicated to communicating the goal to the broader organization, and explaining the benefit for all dependent services.

If this new system succeeded, handling dropped connections and dealing with the aftermath would no longer be commonplace. This was a real incentive for the other engineering teams to support and migrate to the new system. This open communication and coordination would continue throughout the duration of this project.

Step 2: We identified the unknown unknowns

Engineering knew the direction they were heading in. Despite that, they took some time to think through all the scenarios and better understand the underlying concepts. The engineers scheduled exploratory sessions with other stakeholders to identify unknown unknowns, the unforeseeable conditions which can pose a greater risk than the known knowns.

While the Postman organization is used to researching and planning, this part of the process took a lot longer than normal due to the critical nature of this change. They researched different options, considered auxiliary requirements, and came up with a plan over the course of two months.

Step 3: We built the Bifrost websocket gateway

The Bifrost is composed of two parts:

• Public gateway: The gateway uses the Fastify web framework and Amazon AWS ElastiCache for Redis as a central message broker to manage all websocket connections.
• Private API: The API also uses Fastify as a low overhead web framework to proxy traffic to other internal Postman services.

Bifrost is composed of two parts: a public gateway and a private API

Step 4: We tested the new gateway

When Postman engineers are ready to ship a feature, they are expected to test the feature, along with any related features. Since almost every Postman feature relies on websockets, this meant every single feature had to be tested for this release. Furthermore, a framework for automated testing of websockets had not yet been set up at Postman, so all the testing was completed manually before Bifrost could be used in production.

This was an arduous journey, but by the end of January 2020, engineering had a working proof of concept.

Step 5: We migrated to the new gateway

All Postman clients, such as the Electron app or the Web, rely on an initial bootstrap call to another core service named Godserver. This server determines the clients’ access and configuration, and is how engineering controls incremental product rollouts. Because this was all predetermined and controlled by the Godserver, migrating to the Bifrost gateway would not require a single Postman client code update.

The Server Foundation team outlined the squads’ migration steps, the required code changes and configuration to apply. Over the course of a few weeks, dependent services began transitioning from relying on Sync to Bifrost-based handling of their websocket connections. The Godserver diverted more and more traffic to the new websocket gateway to see how Bifrost handled the load and responded to edge cases.

“It’s like changing an engine on an aircraft in mid-flight.” -Numaan Ashraf, director of engineering at Postman

Step 6: We scaled the service

The Bifrost gateway was working!

But Postman had acquired another million or so users while the gateway was in planning and development. And the other engineering teams had not paused their own work during this time. New collaboration features, like version control and role-based access control (RBAC), relied heavily on websockets for information to be updated in real time. There was a flood of upcoming product releases that would truly test the new gateway.

Bifrost was ready to support the increased demands and scale websocket handling.

• Horizontal scaling: Most of the time, Postman services handle increased usage by either scaling to higher capacity instances or by adding more compute instances to the fleet. So engineers at Postman usually scale up a service by increasing the size and computing power of AWS EC2 instances, for example, by using AWS Elastic Beanstalk. But for Bifrost, websocket handling scales out by using more machines. Its optimum efficiency is achieved when smaller-sized instances are used in large numbers. This type of hyper-horizontal scaling works well for Bifrost because clients don’t require high network throughput, and limiting each machine to fewer connections limits the blast radius of failures.
• New load factor of CPU and memory: Most Postman services can effectively scale with a single dimension of scaling metric, like CPU, memory, or latency. However, for Bifrost, things get a bit more nuanced because both memory and CPU usage have different impacts on operations at various levels of throughput. To account for that, Bifrost uses a custom scaling metric based on load factor. The load factor is a multidimensional calculation that imparts a custom non-linear scaling profile.

Let’s dig into the architectural and technology decisions made by Postman engineering.

The Bifrost architecture and tech stack

The Bifrost system has two major components-a Gateway and an API. This two-part architecture is the secret sauce to the stability and scalability of the system.

The Gateway acts as the termination point for all websocket connections. Even though commercial gateways are available for purchase, it was important to preserve the legacy business logic accumulated over years of optimization. Postman engineers also wanted to fully control how websockets are handled, for example, if they wanted to tap into the protocol handshake. For the Bifrost gateway, they used Amazon ElastiCache for Redis allowing them to query the Redis cache using reader and writer nodes. Splitting the traffic into two nodes for read and write operations further allows the team to optimize the performance.

“Bifrost is our gateway for all websocket connections. It’s a proxy for all Postman clients, and responsible for handling low-level socket operations for internal Postman services.” -Mudit Mehta, software engineer at Postman

Most every other service at Postman uses Sails as a real-time MVC framework for Node.js. For the Bifrost gateway, however, the engineers needed a performant backend framework capable of handling high volumes with speed and optimized memory usage. Once again, they wanted to go deeper into the socket layer, below the higher-level abstractions provided by Sails. So they turned to Fastify and forked the socketio-adapter middleware to optimize for their own use cases.

the Bifrost gateway used AWS, Redis, and Fastify to handle websockets

In addition to the gateway, the other component of Bifrost is the private API that proxies traffic to other Postman services. It is based on flexible business rules, and so constantly re-evaluated for how and where to forward inbound traffic.

“Simple components. Complex logic.” -Kunal Nagpal, engineering manager at Postman

For both components, the engineering team decided to roll their own. Although the gateway part of Bifrost isn’t updated frequently, the team has full control over what happens in the deeper layers of websocket handling. The API part of Bifrost is the brains of the operation and converts incoming real-time messages to standard HTTP calls. It can also be updated more quickly as an independent component from Sync and the Bifrost gateway.

Remember that secret sauce? Decoupling Bifrost into these two discrete systems allows both parts to optimize for their own objectives.

The journey is far from over

As with all juicy engineering stories, this isn’t the end. I’ll leave you with a few cliffhangers about what’s coming up next for Postman engineering.

• Build additional redundancy: The Redis cache is a central message broker. Websocket handling still relies on a single point of failure, so what happens if the cache ever goes down?
• Increase bandwidth and throughput: The gateway is currently capable of handling 10x concurrency, but the Postman community is growing fast and engineering is building out more collaboration features. The need to handle more websocket traffic is coming up quickly.
• Continue breaking down the monolith: The Sync service contains a jumble of other services entwined within its codebase. Decoupling socket handling from Sync loosens its grip on other services, so other services can now be more easily peeled off.

This was another behind-the-scenes peek at how Postman engineering operates. Stay tuned for more stories from the trenches.

🙏🏼 Technical review by Kunal Nagpal, Yashish Dua, Mudit Mehta, and Shamasis Bhattacharya.

Originally published at https://blog.postman.com on December 22, 2020.

I’ve previously talked about the traits of a good collection. While not all collections grow up to be documentation, collections are the foundational building blocks for all Postman documentation viewable on the web. Here, I’m going to talk about the traits of good documentation.

Postman documentation has become widely adopted across the API community because it enables better collaboration and API adoption. Let’s learn from the thousands of publishers who document their APIs in Postman — like Microsoft, Twitter, and Dropbox — and find out what makes their documentation successful.

What is good documentation?

Effective documentation teaches someone how to use your API. Since the intended audience is people, and not machines, effectiveness is a subjective measure. Still, we can find some shared similarities across good documentation.

Take a look at this breakdown of nine helpful documentation tips and see how your own process measures up.

An interactive version of this blog post in a collection called The Good Documentation Checklist

In true Postman fashion, I’ve documented these guidelines in a collection. You can experience an interactive version of this blog post in The Good Documentation Checklist template, which is comprised of examples from publishers who document their APIs in Postman.

Step 1. Create a Postman Collection

✅ Generate from an API specification

If you’re using an API specification format, such as OpenAPI, you can import a spec file into Postman to automatically generate a collection. Once you have a collection, you can automatically generate web-viewable Postman documentation.

✅ Find a collection

It’s possible someone else has already done the heavy lifting of creating a collection. Check with your teammates to see if they have a collection that can provide a starting point for the full-fledged collection. Or, you may discover that community members are sharing unofficial collections documenting your public API.

✅ Start from scratch

If you’re starting from scratch, you can import cURL as raw text for each API call. Or as a last resort, input every endpoint’s URL, method, etc.

Step 2. Organize the API metadata

✅ Verify the request metadata

If you previously used a machine-readable data format, consider renaming your requests to be more human-readable. Verify the API metadata (e.g. method, URL, headers) displays in the way you want.

✅ Organize logically in folders

If your collection contains lots of requests, stay clean and tidy by ordering and grouping requests in folders to make the documentation easier to navigate.

✅ Add and format descriptions

Provide a description for each parameter, request, folder, and collection. Style your descriptions, add tables, and embed screenshots using Markdown so readers can quickly identify the most relevant details. You will author the content inside Postman in Markdown, but consumers will view it in the web browser as HTML.

Twitter API documentation with metadata, examples, and folders

Step 3. Include a Getting Started guide

✅ Add headings in the Introduction

The collection description should include relevant information about rate limiting, content types, or settings that apply across endpoints. Any h1 heading that you add to the collection description automatically displays in the side navigation. This makes it easier for the reader to scan for important sections of the documentation.

✅ Explain authorization

The collection description is also a good place to be explicit about how to authorize requests — including instructions explaining what developers need to do to acquire and use credentials. Additionally, Postman has auth helpers under the Authorization tab to help get users interacting with your API quickly. Some publishers choose to manually add authorization headers under the Headers tab of a request, but you can also add an authorization type for an entire folder or collection.

Dropbox documentation includes headings in the Introduction and explains Authorization using screenshots for more guidance

Step 4. Keep it DRY with variables

✅ Use variables

Just like in programming, Postman variables make it easy to change a value in one spot and propagate those changes throughout a collection so you can keep your code DRY (Don’t Repeat Yourself) instead of WET (Write Everything Twice).

✅ Consider variable scopes

Postman supports several variable types and scopes to support different scenarios. For example, a collection variable is useful when the value is inextricably linked to a particular collection. Environment variables can be decoupled from the collection to be used with another collection (e.g. user authorization tokens or server configuration details).

✅ Add placeholder text

Postman relies on string substitution to render variable values in the web documentation. Use placeholder text as a variable’s value to indicate what value to use. If you leave the value blank, Postman displays no information in the documentation.

Step 5. Show use cases

✅ Provide examples

Save examples of requests and responses so users can understand the endpoint and troubleshoot issues more quickly. This is especially useful when a user is browsing the documentation and doesn’t yet have their access credentials. Ensure that these examples contain readable, prototypical usage of your API endpoints.

✅ Demonstrate scenarios

Good documentation is more than just API reference. It includes user scenarios to show what is possible with the API. Guide new users through the proper sequence of API calls to accomplish a specific workflow using variables to pass along data. These use cases can be grouped in folders or as a stand-alone collection.

Step 6. Be secure

✅ Use variables for secrets

Another great use for variables is to store your secrets instead of hard-coding your confidential data. Remember to update your secrets with placeholder text, so you don’t leak any confidential information.

✅ Set up a sandbox environment

Some publishers have an isolated testing environment called a sandbox. This allows users to play around with the API without incurring any costs or side effects in a production environment. You can show users how to interact with your sandbox by pre-populating a collection and environment with the sandbox domain and test credentials.

Step 7. Share private API documentation

✅ Collaborate with teammates

Share your API documentation with teammates by sharing the collection to a team workspace. The default role is that anyone is a “Viewer” of the collection but you can grant “Editor” capabilities to trusted team members. Fork and then merge the collection so you can choose when to work independently versus collaboratively.

Step 8. Share public API documentation

✅ Publish the documentation

In addition to sharing API documentation privately, you can also share collections and environments publicly as web-viewable documentation. Documentation is based on a Postman collection, so you can generate it from an existing collection or create it in conjunction with a new collection.

✅ Embed the Run in Postman button

The Run in Postman button is one way to share a Postman collection (and optional environment). This button is found in the Postman API Network, at the top of published documentation in the web, and also where publishers embed the stand-alone button (like in a README or developer portal).

✅ Add it to the API Network

When you share API documentation publicly, you can also list those collections in the Postman API Network for broader discovery. Make sure to add a concise description and tags so users can easily identify relevant APIs.

✅ Spruce up your profile

When you publish a collection, you can do that from either a personal profile or team profile. Taking the time to fill out your team profile gives readers a clear sense of who a collection is published by helping them to decide whether to try it out or not. You can add a name, description, photo, and custom URL.

Example of a Team profile page listing multiple collections in the Postman API Network

If you’ve ticked most of these boxes, then you can confidently say that you have good documentation. For all you overachievers, let’s forge ahead to a bonus round.

Step 9. Advanced stuff

✅ Add scripts

You can add JavaScript that executes before or after a request runs. These pre-request and test scripts are useful for setting and getting variable values. Writing scripts is also good for debugging, like logging output to the console. If you find yourself reusing a lot of scripts, you can add them to entire collections or folders to run along with every request within that collection or folder.

✅ Keep track of metrics

Postman displays some information like views and imports of your documentation collection. Some publishers will add a custom header to requests in their Postman collection for the purpose of inspecting traffic that comes from the Postman client. This gives them more traceability and insights into how users are interacting with their endpoints from Postman.

✅ Helpful error messages

Publishers have developed clever ways to provide feedback to users who are learning their APIs. Some use helpful error messages returned from the server to guide users along their journey. Others rely on Postman log statements or tests to interpret the client request or standard server response and prompt users to update their API call appropriately.

✅ Visualize the API response

Another way you can make each API request more informative is to use the Postman Visualizer to visualize the response or provide more detail about what actions can be taken next by the API consumer. Providing built-in visualizations and responses can guide consumers through the API’s highlights, making it more of a hands-on journey.

More examples of good documentation

If you’re looking for benchmarks of good documentation collections, check out these stars:

Ping Identity

Ping Identity has a number of public APIs listed on their Ping Identity team profile page. In the PingOne Platform APIs collection, endpoints are organized neatly into folders, markdown tables summarize required and optional variables, and technical diagrams provide a general overview for readers.

Dropbox

Dropbox has a single documentation collection with workflows for team admins listed on their Dropbox team profile page. This collection is unique in that it’s not intended for API reference, but rather for Dropbox team admins to execute specific workflows and tasks. The collection is kind of like an admin panel or internal tool, executing scripts in Postman.

Twitter

Twitter has two profile pages for their Developer Labs and Twitter Ads teams. For the Twitter API v2, they’ve included examples demonstrating practical scenarios like 429 Rate limit exceeded and 200 Request Tweet fields. When you import this collection, you can also see a collection-level script that runs before every request in the collection to retrieve a Bearer token from the client credentials that you provide in your environment file.

Rich Data Services

Rich Data Services provides a wealth of information with their data APIs, providing all the information you need to understand what is being offered with their COVID-19 APIs. Notice that every API request has a visualization that guides you through how to use each individual API.

Many of the better practices we’ve talked about are universal truths about docs, whether or not you’re documenting your APIs with Postman. If you’ve read this far, chances are you’re thinking about investing in proper onboarding and developer education for your APIs.

Come on over and join the Postman API Network. Or dig deeper into documentation best practices and check out this post I wrote about The Ultimate API Publisher’s Guide. If you’re doing something neat with API documentation, let us know in a comment below so that we can share it with the rest of the Postman community.

Photo by Phil Botha on Unsplash

SpaceX is launching thousands of Starlink satellites to assemble a giant interconnected constellation in space. If you look up at just the right time, you might be lucky enough to spot some. But how can you know ahead of time when a satellite is going to pass overhead?

You don’t have to count on luck to see these tiny silver ants parading across the night sky. This tutorial shows you how to set up a scheduled job to check if a satellite approaches and send an SMS alert out to notify you.

Tutorial Requirements

• Python version 3
• A personal phone number to receive SMS notifications
• A freeTwilio account and a Twilio phone number to send SMS notifications
• A free Docker Hub account, Docker Desktop (for Mac or Windows) or Docker Engine (for Linux) to build and share containerized applications
• A free KubeSail account to deploy your code to a Kubernetes cluster

Set up the Python virtual environment

Make a new project directory, and change into the directory from the command line.

$ mkdir starlink-alert
$ cd starlink-alert

Create a virtual environment called venv. Activate the virtual environment, and then install the required Python packages inside the virtual environment. If you’re on Unix or Mac operating systems, enter these commands in a terminal.

$ python3 -m venv venv
$ source venv/bin/activate
(venv) $ pip install twilio python-dotenv datetime pytz skyfield

If you’re on a Windows machine, enter these commands in a command prompt window.

$ python -m venv venv
$ venv\Scripts\activate
(venv) $ pip install twilio python-dotenv datetime pytz skyfield

The Python packages we’re using are:

• Twilio’s Python Library — a Python module for communicating with the Twilio API
• python-dotenv — to manage environment variables
• datetime and pytz — to handle dates, times, and timezones
• skyfield — to compute positions for the satellites in orbit around the Earth

Write the Python script

The purpose of the script is to check if a visible satellite is approaching a given city location. If so, your Twilio phone number sends an SMS alert to your personal phone number.

Copy the following code into a file called tracker.py:

import os
import math
from dotenv import load_dotenv
load_dotenv()
from skyfield.api import Topos, load
from datetime import timedelta
from pytz import timezone
from twilio.rest import Client

# load the satellite dataset from Celestrak
starlink_url = 'https://celestrak.com/NORAD/elements/starlink.txt'
starlinks = load.tle_file(starlink_url)
print ('Loaded', len(starlinks), 'satellites')

# update city location and timezone
location = Topos('37.7749 N', '122.4194 W')
tz = timezone('US/Pacific')

# establish time window of opportunity
ts = load.timescale()
t0 = ts.now()
t1 = ts.from_datetime(t0.utc_datetime()+ timedelta(hours=2))

# loop through satellites to find next sighting
first_sighting = {}
for satellite in starlinks:

filter out farthest satellites and NaN elevation

elevation = satellite.at(t0).subpoint().elevation.km
isNan = math.isnan(elevation)
if elevation > 400 or isNan: continue
print ('considering: {} at {}km'.format(
satellite.name,
round(elevation)
))

find and loop through rise / set events

t, events = satellite.find_events(location, t0, t1, altitude_degrees=30.0)
for ti, event in zip(t, events):

check if satellite visible to a ground observer

eph = load('de421.bsp')
sunlit = satellite.at(t1).is_sunlit(eph)
if not sunlit: continue

filter by moment of greatest altitude - culminate

name = ('rise above 30°', 'culminate', 'set below 30°')[event]
if (name != 'culminate'): continue

find earliest time for next sighting

if (not first_sighting) or (ti.utc < first_sighting['time']):
first_sighting['time_object'] = ti
first_sighting['time'] = ti.utc
first_sighting['satellite'] = satellite

if (first_sighting):

create body for SMS

next_sighting = ('next sighting: {} {}'.format(
first_sighting['satellite'].name,
first_sighting['time_object'].astimezone(tz).strftime('%Y-%m-%d %H:%M')
))

send SMS via Twilio if upcoming sighting

account_sid = os.environ.get('TWILIO_ACCOUNT_SID')
auth_token = os.environ.get('TWILIO_AUTH_TOKEN')
client = Client(account_sid, auth_token)

message = client.messages.create(
body=next_sighting,
from_=os.environ.get('TWILIO_PHONE_NUMBER'),
to=os.environ.get('MY_PHONE_NUMBER')
)

print ('Message sent:', message.sid, next_sighting)

else:

print ('No upcoming sightings')

Update variables location and tz on rows 16 to 17 with your own geographic coordinates and timezone. You can find your latitude and longitude here. See all the timezones by running the following code in a Python shell. Once you find the timezone that applies to your location, exit the Python shell.

(venv) $ python
>>> import pytz
>>> pytz.all_timezones
>>> exit()

The tracker.py script relies on a Python package called Skyfield to handle some of the complex space computations. The script loads the satellite dataset, looping through the satellites to find the next good sighting. Skyfield figures out when a satellite rises and sets near you and sees if the satellite is sunlit for maximum visibility. If there are any sightings predicted within the next two hours, Twilio sends the details with an SMS to your phone number.

The Starlink 3 train rides across the early morning sky, photo by Forest Katsch on Unsplash

To enable the Twilio SMS, store your credentials and other configuration information in a file called .env (notice the dot in front of this filename).

TWILIO_ACCOUNT_SID=
TWILIO_AUTH_TOKEN=
TWILIO_PHONE_NUMBER=
MY_PHONE_NUMBER=

To find your Twilio Account SID and Auth Token, log into the Twilio Console, then click on “Settings” in the sidebar and scroll down to “API Credentials”. You will also need your Twilio phone number to send the SMS.

Save your updates to the .env file, and then run the script from the command line. If you’re running the script during the day, you might determine a satellite is passing overhead, but not be able to see it very well in the daylight. Once you set up a CronJob, you can specify when to run the script for maximum satellite visibility, like at dusk or dawn.

(venv) $ python tracker.py

After you try the script and confirm that it is working, create a requirements file to store the dependencies, so that we can install them in the next step.

(venv) $ pip freeze > requirements.txt

Now, let’s create a Docker image to run the script.

Create a Docker image and push to Docker Hub

Make sure Docker is installed and running on your machine. Add the following code in a file called Dockerfile (notice there’s no file extension) that describes how to build our image.

# creates a layer from the python:3 Docker image
FROM python:3

# copy and install dependencies
COPY requirements.txt /
RUN pip install -r requirements.txt

# add script
COPY tracker.py /

# define the command to run the script
CMD [ "python", "./tracker.py" ]

Next, we are going to build and run a Docker image for the Python script from the terminal. In the following commands, the image we are building is called python-starlink and the container is called starlink_test. Since we don’t want to include our environment variables in the container for security reasons, we run the container using the --env-file flag in the command line. If you’re on a Linux operating system, precede these docker commands with sudo or create a docker group.

$ docker build --tag python-starlink .
$ docker run --name starlink_test --env-file=.env --rm python-starlink

Once you verify that the container runs successfully, share the Docker image on the Docker Hub registry so it can be accessed and run from the cloud. Log into your Docker Hub account from the command line with docker login. Then create a repository and push your image to Docker Hub using the following commands:

$ docker tag python-starlink /python-starlink:latest
$ docker push /python-starlink:latest

In the above commands, Your Docker ID is the username you have registered with Docker Hub.

In the next step, let’s schedule a Kubernetes CronJob to run the Python script.

Deploy a CronJob on Kubernetes

Make sure you are logged into KubeSail. KubeSail has public YAML examples. I made a CronJob template called [satellites](https://kubesail.com/template/loopDelicious/satellites) that has two kinds of Kubernetes resources:

• Secrets to securely store environment variables
• CronJob to complete a task on a schedule

Go to the [satellites](https://kubesail.com/template/loopDelicious/satellites) template. Under “Required Variables”, replace the default DOCKER-IMAGE with the base image you just pushed to Docker Hub. Then input the remaining environment variables.

If you want to inspect the YAML, click “Edit Yaml” to expand the code editor and review the underlying configuration for each resource. This YAML describes the resource CronJob.

apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: starlink
labels:
app: starlink
spec:
schedule: "0/30 19-21 * * *"
jobTemplate:
spec:
template:
spec:
restartPolicy: Never
containers:

• name: starlink-tracker
  image: "{{DOCKER-IMAGE|joycelin79/python-starlink:latest}}"
  envFrom:
  • secretRef:
    name: starlink-secret
    imagePullPolicy: Always
    successfulJobsHistoryLimit: 3
    failedJobsHistoryLimit: 1

This YAML describes the resource Secret to store our environment variables.

apiVersion: v1
kind: Secret
metadata:
name: starlink-secret
labels:
app: starlink
stringData:
TWILIO_ACCOUNT_SID: "{{TWILIO-ACCOUNT-SID}}"
TWILIO_AUTH_TOKEN: "{{TWILIO-ACCOUNT-SID}}"
TWILIO_PHONE_NUMBER: "{{TWILIO-PHONE-NUMBER}}"
MY_PHONE_NUMBER: "{{MY-PHONE-NUMBER}}"

When you’re ready, hit “Launch Template” to deploy a cluster to your Kubernetes namespace on KubeSail.

In the sidebar under “Resources”, you can see all the resources running in this Kubernetes context including the scheduled jobs.

This is also where you can update the deployment. For example, update your environment variables in the Secret resource. To trigger a job every minute while you’re testing, update the YAML configuration for the CronJob resource and set the schedule to "* * * * *"(surrounded in double-quotes) and Apply your changes.

To edit the underlying Python script, save the changes locally in tracker.py. Then build and push the updated image to Docker Hub as we did before. The Kubernetes CronJob in KubeSail automatically pulls the latest version in Docker Hub because we specified imagePullPolicy: Always in our container configuration.

And that’s it!

Conclusion

Creating an SMS notification is pretty straightforward. There’s a bunch of ways to do this — and now you know how to schedule a CronJob on Kubernetes.

For further exploration, you can:

• Adjust the criteria: Each batch of Starlink satellites spreads out over time traveling to a higher elevation. If you want to increase the sensitivity of your tracker to see more satellites, adjust the elevation in your script. Or update the Cron schedule to run when satellites are most visible from the ground — at dusk and dawn.
• Change the deployment: Set up a different Kubernetes context in KubeSail to deploy the CronJob to a cluster running locally or on another cloud provider. Or experiment with other CronJob options in Kubernetes.
• Track something else: If satellites aren’t your thing, swap it out for a new dataset of celestial bodies.

Let me know if you spot something out of this world!

Photo by Nina PhotoLab on Unsplash

Just like Minecraft is more fun when a friend shows you how to make torches and avoid creepers, Kubernetes is also less scary with someone to guide the way.

The Minecraft wiki has lots of pointers for setting up your own server. But it doesn’t say how to do it with Kubernetes. I recently learned how easy it is to deploy an app to a local Raspberry Pi with Kubernetes.

Let’s try it again — this time to run a free Minecraft server on Kubernetes.

Running a Minecraft server on Kubernetes? Piece of cake!

Why Kubernetes instead of just Docker?

At a busy airport, air traffic controllers manage the flow of aircraft into and out of the airport airspace.

• Aircraft transport passengers and cargo from one city to another. The pilots don’t know much about how the airport is managed. They don’t worry about the other aircraft because they rely on air traffic controllers to keep them safe and moving.
• Air traffic controllers guide pilots during takeoff and landing and monitor aircraft as they travel through the skies. They’re not opinionated about the aircraft and don’t mind any loose chickens running through the aisles 🐔

At a less busy airport, you might not need a controller to manage the safety and efficiency of the aircraft. But having one makes operations run smoothly.

There’s a similar relationship between Docker and Kubernetes. Docker containers package software. Kubernetes is the orchestration system coordinating the behavior of those containers.

There’s a separation of concerns — so your containers don’t care where they run, and the orchestration system doesn’t care what is running inside those containers. If you want a refresher, I’ve previously written an intro to the basics.

separation of concerns between Docker and Kubernetes

Kubernetes has a reputation of being “hard”. So why go through the trouble of using it on top of Docker for a simple game server? Even if you run a single app with a single data store, as we’ll do today, Kubernetes can simplify deployment.

The friendly, but independent, relationship between Docker and Kubernetes gives us the flexibility to run our app the way we want. For example, I plan to deploy a lightweight C++ version of Minecraft. Later on, I can update my app to the default Java Minecraft server by swapping out the image of the deployment.

Let’s run Minecraft for free by deploying with Kubernetes on a self-hosted Raspberry Pi server.

A recipe for deploying Minecraft on Kubernetes with KubeSail

The Minecraft wiki lists a bunch of tips for setting up your own server. The game developers behind Minecraft offer a free Java version of server software. The open-source community has developed a C++ version that’s more lightweight so we can run it on a Raspberry Pi.

Let’s roll our own Minecraft server using Kubernetes.

• Step 0 — Pre-requisites
• Step 1 — Add a cluster to the Raspberry Pi
• Step 2 — Fork a public template
• Step 3 — Deploy on Kubernetes
• Step 4 — Connect to the Minecraft server

Let’s set up a server today!

run a Minecraft server on a single-node cluster on Raspberry Pi

Step 0: pre-requisites

KubeSail

Sign up for a free KubeSail account — to deploy and manage the apps on our cluster.

Kubernetes

Decide where to run your Kubernetes cluster. I’m using a $35 Raspberry Pi 4. But you can also use a spare machine if it has enough input/output (I/O) performance to run Minecraft.

Step 1: add a cluster to the Raspberry Pi

I already have a Raspberry Pi 4 to run a Kubernetes cluster. This very good tutorial shows us how to set up a cluster on a new Raspberry Pi.

• Flash an operating system, like Ubuntu, onto an SD card
• Enable Secure Shell (SSH) to operate a headless Raspberry Pi without a separate keyboard and monitor
• Connect the Raspberry Pi directly to the ethernet to reduce latency for gaming — or enable Wi-Fi on the SD card
• Move the SD card over to the Raspberry Pi
• Log in to the Raspberry Pi using SSH
• Install MicroK8s — a lightweight Kubernetes recommended for IOT apps, so your terminal commands will be microk8s.kubectl instead of kubectl

To connect the Raspberry Pi to KubeSail, apply the configuration file to your personal namespace from the Raspberry Pi’s command line. This might take a couple of minutes to spin up.

$ sudo microk8s.kubectl apply -f https://byoc.kubesail.com/.yaml

1) add a cluster on the Raspberry Pi using `microk8s.kubectl` instead of `kubectl`, and 2) give it a name

The Raspberry Pi is now a single-node Kubernetes cluster!

You can manage this new cluster in the KubeSail dashboard. Find the new cluster, give it a name, and then select Namespaces from the sub-menu. This is where you can add a new Kubernetes namespace, like one called games.

Add a new namespace called `games`

Now we have a Kubernetes context consisting of a cluster, a namespace, and a user — everything that we need to set up our deployment.

Step 2: fork a public template

KubeSail has a bunch of Kubernetes templates to browse YAML examples and customize your own deployments.

Under Templates, I made this one for Minecraft on Raspberry Pi. If you’re not using a Raspberry Pi, use this template instead.

minecraft-raspberry-pi: Minecraft server with a Persistent Volume Claim to store your world data, on Raspberry Pi 🍓. This template also runs on any board with an arm64 CPU*.*

The minecraft-raspberry-pi template has two kinds of Kubernetes resources.

• Deployment — to describe how an application is deployed
• Persistent Volume Claim — a request for a particular type of storage

fork the minecraft-raspberry-pi template

Fork the template. You can rename it, and also toggle the new template’s setting to Private if you don’t want the template to be listed publicly as one of your own.

The YAML explained

This section explains some of the underlying YAML for these resources if you want to make any adjustments. Feel free to skip ahead to the next step.

In your new KubeSail template, click the Edit Yaml button to open the YAML editor on the right side of the page. Find the Deployment document tab in the YAML editor, and notice some details.

• [**replicas**](https://kubernetes.io/docs/concepts/workloads/controllers/replicaset/#replicas): to specify the number of pods to run
• [**image**](https://kubernetes.io/docs/concepts/containers/images/): the base Docker image for the container is [joycelin79/cuberite](https://hub.docker.com/r/joycelin79/cuberite), a C++ version of Minecraft (instead of the default Java) that is more lightweight for the Raspberry Pi or any machine with an arm64 CPU
• [**volumes**](https://kubernetes.io/docs/concepts/storage/volumes/): there’s a single volume, or data store, that points to our second resource which is a PersistentVolumeClaim
• [**volumeMounts**](https://kubernetes.io/docs/concepts/storage/volumes/#background): uses subpaths to specify where to mount the volumes into the container according to how the base image is structured
• [**ports**](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.10/#containerport-v1-core): exposes the port by setting the hostPort and containerPort — these two ports should be the same value, the default Minecraft port 25565
• [**resources**](https://kubernetes.io/docs/tasks/configure-pod-container/assign-memory-resource/): allocates your cluster’s memory to run your game smoothly, you may need to increase this amount, but most game servers should run below 1 CPU and 2Gi of memory

Review the deployment details

Find the PersistentVolumeClaim document tab in the YAML editor. This claim asks the cluster for a particular type of storage, to be provisioned as a PersistentVolume.

• **name**: points back to the volume specified in the Deployment document
• [**resources**](https://kubernetes.io/docs/tasks/configure-pod-container/assign-memory-resource/): this is a request for resources for the volume, or the data stored, and 2Gi should be enough for the Minecraft world.

Review PersistentVolumeClaim

The PersistentVolume in this deployment allows us to save the state of the Minecraft world so that we can quit our game and keep all of our precious minerals when the server reboots ⛏💎

Step 3: deploy on Kubernetes

We’re ready to deploy our app on Kubernetes!

In the dropdown menu next to the Launch Template button, select the Kubernetes context from Step 1.

Select a Kubernetes context to deploy the app

Hit Launch Template, and KubeSail will deploy Minecraft. In this example, I deploy to the games namespace on my Raspberry Pi cluster.

Our Minecraft server is alive!

Step 4: Connect to the Minecraft server

To connect to the Minecraft server, start Minecraft on the machine where you want to play. Select Multiplayer from the main menu. Click the Add Server button, and enter the IP or web address of the Raspberry Pi server. If you’re on the same local network, you should see the server suggested in the local list.

scanning for games on the local network

If you want to allow access to the Minecraft server via the internet, set up port forwarding to point port 25565 to the IP of your Raspberry Pi. If you need a refresher, here is a tutorial to forward ports on your router.

Provide this public IP to friends you invite to multiplayer Minecraft. Or you can set up an A record with your DNS provider that points to your home network like we did previously, so friends can input a URL instead of an IP address.

Players need to run the same version of Minecraft as the server. As of the writing of this article, the latest version of cuberite is 1.12, so players should set their Minecraft version to 1.12.

So that’s it!

We set up a Minecraft server on Raspberry Pi using Kubernetes with a few terminal commands and clicks. Hosting a game server from your living room is free and easy. With the time I saved setting up a Minecraft server, I now have more time to spend on mining precious minerals ⛏

🤓 Fun Fact: If my IP address changes while I’m away from home, I might have trouble accessing the game server or any other self-hosted services. Check out this other tutorial I wrote for setting up a dynamic DNS service, also on Raspberry Pi using Kubernetes!

Final thoughts on deploying with Kubernetes

When people talk about Kubernetes, they picture huge companies with tons of traffic hitting their services. My little game server certainly doesn’t need Kubernetes for any advanced capabilities. But Kubernetes can also make deployment easier. I’ve done the “hard part” by configuring the Minecraft server to run on Kubernetes. And now if I want to run Kubernetes on a different cluster, I switch the context in KubeSail to deploy my changes with a single click.

Most people also picture Kubernetes running on a managed cloud provider, which can get expensive. But now we know how to run a cluster for free on a local machine like a Raspberry Pi.

There’s a bunch of reasons why people use Kubernetes. For this game server, I’m using Kubernetes to simplify deployment and because it’s free to run on a self-hosted server.

Try it yourself, and let me know what you think!

A couple years ago, Postman CTO and co-founder Ankit Sobti shared Postman’s struggle to escape from a microservices dependency hell. If you want to learn more about how and why Postman Engineering ventured into microservices, check out Ankit’s story.

This is an update — here’s how Postman Engineering does microservices today.

Who is Postman Engineering?

Postman Engineering has 100+ engineers working across eight locations around the world. The company just announced a Series C round of funding, so things are bound to evolve as with any growing startup.

the sheer beauty of Postman Engineering

Let’s dive into how these teams are organized. As we’ll soon see, this really impacts the microservices implementation.

With a microservice architecture, how do you organize your teams?

According to Conway’s Law, software starts looking like the organizations that create them. How people communicate, collaborate, and even their tooling has an impact. And the way information flows between related teams influences the output.

Any organization… will produce a design whose structure is a copy of the organization’s communication structure.

— Conway’s law

This is one reason why organizations like Amazon and Netflix work in small, independent teams. It enables API-first design and development. And is represented as the well-known microservices deathstar.

microservices deathstar

You can observe Conway’s Law by comparing products from teams grouped by function, by product, or by workflow.

teams grouped by function, by product, or by workflow

How can you take advantage of this phenomenon?

The Inverse Conway Maneuver proposes that you can change your organizational structure in order to change the technical structure that you desire. By dictating how your organization communicates and collaborates, you begin converging on your ideal shape.

Keeping this in mind, development teams at Postman are called squads. Squads operate on domain-driven design (DDD) principles, which free up each squad to focus on its core domain.

For example, one squad owns the Identity domain. The Identity squad produces services like Create a User and Authenticate a User, among others.

examples of services produced by the squad owning the Identity domain

Across our company, there’s 12 squads that produce 40 services for Postman engineering. For the most part, each squad independently manages their roadmap and sprints. They have some freedom to choose their tools and configure their work processes.

Although squads are focused on their specific service, they are cross-functional, with dedicated members from design, product, engineering, security, quality, technical writing, and developer relations (that’s me!).

cross-functional squad focused on a domain

There’s many ways to implement microservices, but here’s how Postman engineering does it.

With a microservice architecture, how do you create and maintain new services?

Imagine the Identity squad wants to produce a new service — to authenticate an existing user.

Step 1 | Create a team workspace

In Postman, the Identity service owner is the producer of this new service. This producer creates a team workspace as the single source of truth for the new service.

a team workspace is the single source of truth for a service

A workspace is the foundation for collaboration in Postman. As we’ll soon see, the workspace contains the expected behavior of the service. Unit and integration tests live here. And it’s the central thread for all the communication during and after development.

Step 2 | Draft a blueprint collection to describe use cases

Within the new workspace, the Identity service owner drafts a collection to describe their service. This collection is like a blueprint with proposed examples of a request and potential server responses to describe various use cases. In fact, the name of the collection includes #blueprint as a tag indicating its purpose.

start with a proposed blueprint of the new service

Step 3 | Negotiate how the proposed service will function

The blueprint collection is a proposal for the service. Potential consumers of the service weigh in on the design, either offline or via threaded comments in the app, to shape what the service will become.

threaded comments to negotiate the proposed service

Step 4 | Parallel development with mocks

Once all stakeholders successfully negotiate the design of the service, the collection becomes an agreement, or contract, for the new service. Those examples of requests and responses inform mock servers that enable parallel development.

paired requests and responses inform mocks which enable parallel development

The mock is a contract between the producer and consumers of this service. Both parties can start developing their code at the same time. Tests are written. Documentation begins. All this activity occurs in parallel and is driven by an agreed-upon vision of the new service.

With a microservice architecture, how do we know when there’s a breaking change?

When we’re ready to deploy the service, we test to ensure that no workflow is broken. One of the most powerful tools that Postman engineering has adopted to manage microservices is consumer-driven contract (CDC) tests.

CDC testing

In the previous section, we talked about how the service owner relies on the blueprint collection to propose, negotiate, and then establish how their service will be used.

Once that contract is agreed upon, consumers write tests using the mocks as a reference point. The consumers only test the endpoints and properties that they need, saving the tests in a collection. Once again, the name of these collections include #contract as a tag indicating its purpose. These test collections are maintained within the producer’s team workspace.

consumer writes tests that are run as part of the producer’s CI pipeline

When the producer is ready to deploy their service, the consumer’s contract tests are run against the code as part of the continuous integration (CI) pipeline.

The service owner pulls all the collections within the team workspace using the Postman API, looking for any collections with the #contract tag to run. They use Docker to configure their own environments and deploy their code in the CI/CD pipeline, and rely on Newman to run the collections in the container.

The code changes deploy only if the contract tests pass.

Continuous testing

In addition to running contract tests on demand as part of the CI pipeline, Postman Engineering runs a plethora of other tests on a regular basis.

Our engineers schedule monitors that run test collections from Postman servers. These alerts and summaries are piped into Slack and email to keep tabs on the service throughout the day.

continue running health and security checks on monitors

So Postman engineers rely on tests that run on demand and continuously. Incorporating CDC testing in the CI pipeline allows service owners to deploy updates with confidence that they’re not breaking somebody else’s code. Continuously running other tests on monitors also ensures nothing is broken after deployment.

1) Newman to run tests on demand at build time, and 2) Monitors to run tests on a scheduled cadence

All of this informs an ongoing feedback loop that allows the team to continue evolving and updating their services.

Is that happily ever after?

No blog post about microservices will ever conclude with a tidy bow. All architectures are a continuously evolving work in progress.

As you can probably imagine, Postman engineering uses the Postman product quite a bit. Which means that we’re the first canaries in the coal mine. If something doesn’t work smoothly or it can be done better, we have the opportunity to feel the pain first. This allows us to iterate quickly on new features.

Conclusion

This was a peek at how Postman is working with microservices today. We’re continuing to address some challenges.

• Dependencies on other squads require additional coordination. One squad’s roadmap might be blocked by another squad’s progress, or at least be subject to their service levels.
• Squads are a relatively new concept at Postman, and each squad is independently figuring out what processes work best.
• In domain-driven design, domains require constant evaluation. For example, squads focused on large domains that require a large share of responsibilities debate the trade-offs between managing a microservice vs. a monorepo vs. a monolith.

If you want to learn more about microservices, then check out Ankit’s original blog series about Postman’s initial foray into the microservice architecture. And keep your eyes peeled. We hope to share more updates and stories from the trenches soon.

In a recent Livestream with my teammate Arlemi, we unboxed some lights and messed around with the Philips Hue Lights API. We figured out how to turn on the light and change the colors using an API.

Then Arlemi queried a weather API — he wanted to update the light based on the weather forecast.

the original events inspiring this project

But the light was with me in San Francisco. And Arlemi was in London. So I was the only one who could update the device during the Livestream.

So the challenge began: how could I let Arlemi, and the viewers, change the lights in my home from the comfort of their own pajamas?

Dear impatient readers*: If you don’t care how I did it, skip ahead to the end, and you too can turn on the lights in my kitchen **💡*

Why can’t anyone else turn on the lights?

You can log in to the Philips Hue mobile app with your Philips Hue account for remote access to your lights.

To use the Philips Hue Lights API directly, there’s a couple of options. For example, if I’m sending these API calls from Postman on my laptop there are really only two ways to do it.

Local Area Network

I can send the request from within the same local network. Since my laptop and lights are connected to the same home WiFi, I can use Postman to hit the API. But if I go across the street and use my neighbor’s WiFi, then I’m no longer on the same local network.

Local Area Network: devices connected to the same network can access the light

So unless I’m connected to my local network, I won’t be able to reach my lights without providing additional information.

Remote access

The Philips Hue Lights API allows me to give additional information to authorize remote access, so my lights can be accessed from a different network.

Remote access: additional info required to authorize

But I don’t want to share my private account information with anyone else.

Let’s walk through a few ways to enable public access to our light, without sharing personal credentials.

• Port forwarding — the most straightforward option, but don’t do it
• Proxy in the cloud — place a proxy in front of your light, in the cloud
• Proxy in the local network — place a proxy in front of your light, in the local network

Option 1 — port forwarding

The most straightforward option is to set up port forwarding for your connected light. Port forwarding enables external requests to your public IP on a specified port to be forwarded to a connected device.

Port forwarding: light is exposed directly to the internet

Don’t do this. Bots and malicious attackers scan IP addresses for exposed and vulnerable devices. And if my Philips Hue light has a vulnerability, an attacker can infiltrate my connected device and access other clients connected to my local network.

Also, you tell the users your public IP address for your home, which can reveal some personal details like your general location.

Next, let’s buffer against potential attacks and obscure our home IP.

Option 2 — proxy in the cloud

This option places a proxy in the cloud between users and the connected device. This buffers against some unwanted attention. By directing requests through an app, we can build custom checkpoints like validating user inputs or rate-limiting into our app.

validate user inputs and rate limiting

Deploy your app on a cloud hosting provider like Digital Ocean or Heroku. This means you can invite users to send requests to the app hosted on the cloud, instead of your home IP address.

Now we control traffic from our friendly users. But we still need to set up port forwarding to route traffic from our proxy, so our light is still exposed to the internet.

Proxy in the cloud: place a proxy in front of the light

Even though you’re now handling legitimate requests from your users more cautiously, the light is still exposed. There’s nothing standing between potential attackers and the light.

Next, let’s thwart attackers by moving the proxy inside our local network.

Option 3 — proxy in the local network

The only improvement with this option is the placement of the proxy in the local network. If we move the same proxy from the cloud to the local network, we handle potential attackers more effectively.

This time we set up port forwarding to route traffic to our proxy. Attackers can still hit our public IP, but now we have a barrier between the light and the internet.

Once again, we control the data passed through and returned, but this time for every user.

Proxy in the local network: move the proxy inside

Tradeoffs and additional considerations

We talked about three options to open up your connected device to the public, and a few more considerations to further fortify connected devices. As always, the option you choose will depend on your specific situation.

pros and cons for each option to enable public access to a connected device

CDN

You can use a content delivery network (CDN) in front of any of these three options to obscure your home IP address. A CDN provider like Cloudflare also offers free protection against distributed denial of service (DDOS) attacks to throttle an intentional, or unintentional, surge in traffic.

Rate limiting*: You can add rate limits at the infrastructure-level or application-level. Check out my tutorial [to build application-level rate limits with Node and Redis](https://codeburst.io/api-rate-limiting-with-node-and-redis-95354259c768).*

NAT traversal

You can use a gateway to maintain a network address translation (NAT) connection by opening a two-way tunnel between a cloud proxy and your home server. This means you don’t need to configure your router for port forwarding and is secure from attackers who scan IPs.

Dynamic DNS

You can set up a dynamic DNS (DDNS) service for any option that requires router configuration for port forwarding. Check out my tutorial for using a free DDNS client like ddclient to automatically update your DNS record when your IP address changes, so access to your home server is maintained.

Here’s what I did

I wanted my Philips Hue light to be accessible to Livestream viewers. So I ran an Express server on a Raspberry Pi in my local network (option 3). I used Cloudflare for DNS and KubeSail for NAT traversal so that I didn’t need port forwarding and could direct users to a public domain: [https://light.meowsergirl.com](https://light.meowsergirl.com).

here’s what I did

Go ahead, turn on the lights in my kitchen

Now, as promised, go ahead and turn on the lights in my kitchen.

• Example API calls in Postman — to update the lights in my kitchen
• Code sample in GitHub — to enable public access to your own connected devices

Conclusion

Turning on these lights is more de-light-ful when you can see them. So follow Postman on Twitch and tune in to a livestream to see the effects of your API calls in realtime. Thanks for reading, I hope you found this entertaining!

Photo by Ludovic Charlet on Unsplash

Rate limiting can protect and improve the availability of your API-based services. If you’re talking to an API and receive the HTTP 429 Too Many Requests response status code, you’ve been rate limited. That means you’ve exceeded the number of requests allowed within a given period of time. Slow your roll and wait a bit, before trying again.

Why rate limit?

When you’re thinking about limiting your own API-based service, you need to balance tradeoffs between user experience, security, and performance.

The most common reason to control the flow of data is to maintain availability for your API-based services. But there are security benefits too. A single unintentional, or intentional, surge in inbound traffic can tie up valuable resources and impact the availability for other users.

By controlling the rate of incoming requests, you can:

• protect services and resources from being overwhelmed
• slow down brute-force attacks
• prevent distributed denial-of-service (DDOS) attacks

How can you implement rate limiting?

Rate limiting can be implemented at the client-level, application-level, infrastructure-level, or anywhere in between. There are a few ways to control inbound traffic to your API service.

• By user — track calls made by a user with an API key, access token, or IP address
• By geography — such as decreasing rate limits at peak times of the day for each geographic region
• By server —if you have multiple servers handling different calls to your API, you might implement stricter rate limits for access to more expensive resources

You can use any one (or even a combination of) these types of rate limiting.

No matter how you choose to implement it, the goal of rate limiting is to establish a checkpoint that either rejects or passes through requests to access your resources. Many programming languages and frameworks have inbuilt capabilities or middleware to do this. There are also options for various rate limiting algorithms.

This is one way to make your own rate limiter with Node and Redis:

1. Create a Node app
2. Add a rate limiter with Redis
3. Test in Postman

💻 Check out the code sample on GitHub.

Before you begin, make sure you have Node and Redis installed on your machine.

Step 1: Create a Node app

Set up a new Node app from the command line. Walkthrough the CLI prompts, or add the --yes flag to accept the default options.

$ npm init --yes

Create a file called index.js for your entry point if you accepted the default option during the project setup.

$ touch index.js

Install the Express web framework, and initialize the server in index.js.

const express = require('express')

const app = express()

const port = process.env.PORT || 3000

app.get('/', (req, res) \=> res.send('Hello World!'))

app.listen(port, () \=> console.log(`Example app listening at http://localhost:${port}`))

Start the server from the command line.

$ node index.js

Back in index.js, create a route to first check the rate limit, and then allow access to resources if the user is not over the limit.

app.post('/', async (req, res) \=> {

async function isOverLimit(ip) {

// to define

}

// check rate limit

let overLimit = await isOverLimit(req.ip)

if (overLimit) {

res.status(429).send('Too many requests - try again later')

return

}

// allow access to resources

res.send("Accessed the precious resources!")

})

application-level rate limiting

In the next step, we’ll define the rate limiter function isOverLimit.

Step 2: Add a rate limiter with Redis

Redis is an in-memory key-value data store, so it can retrieve data very quickly. It’s also pretty straightforward to implement rate limiting with Redis.

• Store a key like a user’s IP address
• Increment the number of calls made from that IP
• Expire the record after a specified timeframe

The rate limiting algorithm shown below is an example of a sliding window counter. A user who submits a modest number of calls, or spaces them out over time, never reaches a rate limit. A user who exceeds the maximum requests within a 10-second window must wait for enough time to pass to resume their requests.

rate limiting algorithm: sliding window counter

Install a Redis client for Node called ioredis from the command line.

$ npm install ioredis

Start the Redis server locally.

$ redis-server

Then require and initialize the Redis client in index.js.

const redis = require('ioredis')

const client = redis.createClient({

port: process.env.REDIS_PORT || 6379,

host: process.env.REDIS_HOST || 'localhost',

})

client.on('connect', function () {

console.log('connected');

});

Define the isOverLimit function that we started writing in the previous step, by following this pattern from Redis to keep a counter by IP.

async function isOverLimit(ip) {

let res

try {

res = await client.incr(ip)

} catch (err) {

console.error('isOverLimit: could not increment key')

throw err

}

console.log(`${ip} has value: ${res}`)

if (res > 10) {

return true

}

client.expire(ip, 10)

}

And that’s our rate limiter.

When a user makes a call to the API, we check Redis to see if the user is over the limit. If they are, the API immediately returns the HTTP 429 status code with the message Too many requests — try again later. If the user is within the limit, we proceed to the next code block where we can allow access to protected resources like a database.

During the rate limit check, we find the user’s record in Redis and increment the count of their requests. If there’s no record of the user in Redis, then we create a new record. And lastly, each record will expire within 10 seconds of the latest activity.

In the next step, let’s make sure our rate limiter is working correctly.

Step 3: Test in Postman

Save your changes, and restart the server. We’ll use Postman to send POST requests to our API server running locally at http://localhost:3000.

within rate limit

Continue sending requests in rapid succession to hit your rate limit.

exceeds rate limit — HTTP 429 too many requests

Final thoughts about rate limiting

This was a simple example of a rate limiter with Node and Redis. This is just the start. There’s a bunch of strategies and tools available to structure and implement your rate limits. And there are other enhancements to explore with this example, like:

• Letting the user know how much time they should wait before trying again, either in the response body or as a Retry-after header
• Logging the requests reaching the rate limit, for insight into user behavior and to alert of malicious attacks
• Trying a different rate limiting algorithm or other middleware

Remember, when you’re exploring API limits you’re balancing tradeoffs between performance, security, and user experience. Your ideal solution for rate limiting will change over time along with these considerations.

Photo by NASA on Unsplash

Most internet service providers (ISPs) give residential customers a dynamic IP address. They pull from a pool of IP addresses and allocate one to a home. Since the IP address might change in a few months or even in a few hours, it can be hard to consistently access your personal computer when you’re away from home.

Setting up a dynamic DNS (DDNS) service gives you reliable access to self-hosted services, like a personal website, a Minecraft server, or just your home computer. If you point a hostname to your home network, you can have Kubernetes update the DNS record when your IP inevitably changes.

What is Dynamic DNS?

Since most residential customers have dynamic IPs, there’s a few options to manage a changing IP address.

• Static IP: if your ISP offers this option, get a dedicated IP address so it never changes
• Router: if your router has a dynamic DNS option, configure DDNS on your router to always maintain an active connection
• DDNS services: Cloudflare recommends a few options, such as updating your IP using a dynamic DNS client application

Making sure your IP is up to date, or remains fixed, guarantees reliable access to your self-hosted services.

Don’t gamble on accessing your self-hosted services when you’re away from home

For example, I have a self-hosted Minecraft server on a Raspberry Pi at home. I want to set up DDNS so that I can access this game server even if the IP changes while I’m out of town.

How can we run a dynamic DNS service?

Let’s use a dynamic DNS client application called ddclient. Services like these keep an eye on your IP address. When the IP changes, the service automatically updates your DNS record using the DNS provider’s API.

DDNS client apps like ddclient periodically check your public IP for changes

If you’d like to learn how to set up your DNS records, take a spin through this very good tutorial that walks through how to register a domain and run ddclient using Docker on its own.

In this tutorial, let’s run ddclient on Kubernetes.

Wait, wait… You’re using Kubernetes to run a self-hosted app?!

Isn’t this overkill?

— anyone reading this

Why use Kubernetes in addition to Docker?

Docker containers are useful for packaging and running software in a way that’s predictable and consistent. While I’ve previously talked about using Kubernetes to reduce the complexity of managing containers and to scale services, we can also use Kubernetes to simplify the deployment process.

Docker for packaging software, and Kubernetes for deploying it

My self-hosted apps will never have Netflix- or Slack-level traffic. But if we can use Kubernetes to ease the pain of deployment, for free, then why not?

There’s a bit of a learning curve with Kubernetes, but a bunch of new tools are making it easier than ever to get started.

Let’s forge ahead.

Where can I deploy a Kubernetes cluster?

When I think about deploying on Kubernetes, I imagine working with a managed cloud service, like when we previously deployed a cluster on Amazon EKS.

But Kubernetes can also run on a local machine, like a Raspberry Pi at home.

You can deploy a Kubernetes cluster on a local machine, cloud, on-prem datacenter; or choose a managed Kubernetes cluster. You can also create custom solutions across a wide range of cloud providers, or bare metal environments.

— Kubernetes docs

A cluster running on a self-hosted Raspberry Pi server is perfect for my own DDNS and gaming needs. Setting up Kubernetes on a local machine is also a great way to learn Kubernetes without racking up cloud costs.

Let’s deploy ddclient on Kubernetes running on a local Raspberry Pi. To do this, we’ll use KubeSail, which can deploy on any type of cluster.

A recipe for deploying ddclient on Kubernetes with KubeSail

automatically update IP using ddclient

The ddclient app keeps an eye on our IP address. When the IP changes, ddclient automatically updates our DNS record using the Cloudflare API. Ddclient works with a bunch of DNS providers to keep DNS records up to date, but we’ll use Cloudflare.

• Step 0 — Pre-requisites
• Step 1 — Add a cluster to the Raspberry Pi
• Step 2 — Fork a public template
• Step 3 — Update your DNS secrets
• Step 4 — Update the deployment details (skip)
• Step 5 — Deploy on Kubernetes

Let’s get started.

0 | Pre-requisites

Do this stuff first.

KubeSail

Sign up for a free KubeSail account — we’ll use their tools to manage the apps on our Raspberry Pi.

DNS server

You’ll need to own your own domain and be able to change your domain’s nameservers. I’m using a free Cloudflare account to route my web traffic through the Cloudflare network.

Kubernetes

Decide where to run your Kubernetes cluster. I’m using a Raspberry Pi. The DDNS application should be running on the same home network as your self-hosted services, since the app works by checking its own network IP.

1 | Add a cluster to the Raspberry Pi

I’m using a Raspberry Pi 4 to run my Kubernetes cluster. This very good tutorial shows us how to set up a cluster on a new Raspberry Pi.

• Flash an operating system onto an SD card — I used Ubuntu
• Enable Secure Shell (SSH) to operate a headless Raspberry Pi — or use a separate keyboard and monitor
• Enable Wi-Fi also on the SD card — or I plugged directly into ethernet
• Insert the SD card into the Raspberry Pi
• Log in to the Raspberry Pi using SSH
• Install MicroK8s — a lightweight Kubernetes recommended for IOT apps, so your commands will use microk8s.kubectl instead of just kubectl

To connect the Raspberry Pi to KubeSail, apply the configuration file to your namespace from the Raspberry Pi’s command line.

$ sudo microk8s.kubectl apply -f https://byoc.kubesail.com/.yaml

The Raspberry Pi is now a single-node Kubernetes cluster!

The Raspberry Pi is now a single-node Kubernetes cluster!

The Raspberry Pi will appear in the KubeSail clusters dashboard after a minute or so. In the dashboard under Clusters, select the new cluster and give it a name. Then select Namespaces from the sub-menu. This is where you can add a new Kubernetes namespace, or use a default namespace called default.

Add a new namespace, or use the default

So now we have a cluster, a namespace, and a user —all the parameters required for our Kubernetes context.

2 | Fork a public template

I previously talked about the basic concepts behind Docker and Kubernetes. YAML is used to write instructions to deploy and manage your containers.

If you’re just learning Kubernetes, or getting started with a new project, KubeSail lists community examples. You can browse these examples, and cobble together relevant YAML samples to customize your own Kubernetes apps.

Under Templates, I made this one for ddclient.

ddclient: ddclient for dynamic DNS updating for self-hosted services at a residential network 🌐

The ddclient template has two kinds of Kubernetes resources.

• Secret — to store your confidential information
• Deployment — to describe how your application is deployed

fork the ddclient template

Fork the template on this page. Then toggle the new template’s setting to Private because we’ll add our secrets in the next step.

3 | Update your DNS secrets

If you haven’t already done so, add an A record to Cloudflare that points to your home network. For now, this can point to any IP, such as 1.1.1.1 because ddclient will update the IP to the correct address when we deploy it later.

add an A record for the home network

Here’s the data you need from Cloudflare.

• zone — your domain name like example.com
• login — your Cloudflare account email address
• password — your global API key from under API Tokens on your Cloudflare profile
• the line beneath password — full domain name to keep updated, including subdomain like home.example.com

Back in your new private KubeSail template, click the Edit Yaml button to open the YAML editor on the right side of the page.

There’s a couple ways to decouple Kubernetes configuration details. Using a Kubernetes Secret will store your confidential data, and inject the data into your containers only when it’s needed.

Find the Secret document tab in the YAML editor, and update your Cloudflare credentials beneath ddclient.conf.

Update your Cloudflare credentials in the `Secret` document

Here’s some other interesting details in the ddclient.conf.

• **daemon**: ddclient will run this process every 300 seconds (five minutes)
• **web**: dclient will check if the IP address has changed, using this web service: https://domains.google.com/checkip

When you’re done, save the template.

4 | Update the deployment details (skip)

You don’t have to do anything here, but it’s helpful to understand some of the underlying YAML for this resource.

The YAML explained

Find the Deployment document tab in the YAML editor, and notice the interesting bits.

• [**volumes**](https://kubernetes.io/docs/concepts/storage/volumes/): there’s a single Kubernetes volume, or data store, that points to our secrets from the last step
• [**image**](https://kubernetes.io/docs/concepts/containers/images/): the base Docker image for our containerized application is [linuxserver/ddclient](https://hub.docker.com/r/linuxserver/ddclient) from the smart people at Linux Server
• [**resources**](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/): the request for how much CPU or memory a container needs to run ddclient as a fairly lightweight Perl program

Review the deployment details

If you make any changes, remember to save the template.

5 | Deploy on Kubernetes

We’re ready to deploy our app on Kubernetes! This part is easy peasy lemon squeezy since we already added a Kubernetes cluster and namespace in the first step.

In the dropdown menu next to the Launch Template button, select a Kubernetes context to deploy our app.

Hit Launch Template, and KubeSail will deploy ddclient.

In the example below, I deploy to the default namespace on my Raspberry Pi cluster.

deploy to the `default` namespace on the `Raspberry Pi` cluster

You can see success notifications and logs under your deployment details.

Verify ddclient is working

Once your app is finished deploying, return to Cloudflare to verify that ddclient updated 1.1.1.1 to our actual network IP. Every five minutes, ddclient will check if our network IP has changed.

Besides the initial update, there’s one more check to verify that ddclient is working correctly. Once again, change the IP of your home network’s A record in Cloudflare to 1.1.1.1. Turn off your internet router long enough to ensure your ISP allocates a new IP address. I left my router unplugged overnight. In the morning when I powered up my router, I verified that ddclient updated my IP in Cloudflare.

In summary, we deployed ddclient on Kubernetes. Every five minutes, ddclient will check the IP address of the Raspberry Pi. If there’s been a change to the IP, ddclient will update our DNS record using the Cloudflare API.

Doing this, allows me to continue playing Minecraft on my Raspberry Pi when I’m away from home, and my ISP changes my IP unexpectedly.

🤓 Fun Fact: The Raspberry Pi at home hosts both ddclient and my Minecraft server. While both servers don’t need to run on the same cluster, they should be in the same home network since ddclient works by checking its own network IP. Stay tuned to learn how to set up a Minecraft server on Raspberry Pi using Kubernetes.

Final thoughts on deploying on Kubernetes

Kubernetes for simplifying deployments

Deploying projects is something I do very rarely. So when I do, it takes a while to dust off the cobwebs, and it’s pretty slow going.

Even though Kubernetes is best known for complex container orchestration at massive scale, it can also simplify modest deployments.

Share your templates

I’m new to Kubernetes. So as a n00b does, I hack on something until it works. For this project, starting with other people’s YAML examples was super helpful. This great blog post, other public templates, and these code samples saved me a ton of time.

If you come up with something that works for you, share it with the rest of us as a public template 🙏

Photo by Serena Repice Lentini on Unsplash

For my last Mac, I had a friendly hamster face emoji in my terminal bash prompt. After I recently upgraded to a new Mac, I discovered Apple has replaced bash with zsh as the default shell, beginning with macOS Catalina.

Starting with macOS Catalina, your Mac uses zsh as the default login shell and interactive shell.

You can still switch back to bash, for now, but the trend is moving to zsh as the command-line interpreter for the login shell and interactive shell.

Here’s how you can add an Octopus emoji and other custom items to your zsh prompt to make life in your terminal more hospitable.

Change your default shell

If you aren’t already using zsh in your terminal, change the shell by updating the shell path to /bin/zsh, with the following terminal command.

$ chsh -s /bin/zsh

Switch to a zsh prompt

Zsh recognizes a different set of prompt specifiers than bash, so you may need to modify some settings previously used in your bash profile.

If you’re starting from scratch, here’s 3 steps to add a custom prompt with an emoji as well as other information about the current git repository.

Step 1: install oh-my-zsh

Install Oh My Zsh, an open source framework for managing your zsh configuration, by running this curl command in your terminal.

$ sh -c "$(curl -fsSL https://raw.github.com/ohmyzsh/ohmyzsh/master/tools/install.sh)"

Step 2: update your theme and plugins

Use your favorite text editor to open your ~/.zshrc config file located in your $HOME directory.

$ code ~/.zshrc

This config file is where you can update your theme for a different look and feel by selecting one of the many available themes, or leave it the default theme.

# in ~/.zshrc
# pick theme
ZSH_THEME="robbyrussell"

You can also add a plugin to this config file by selecting a few of the many available plugins, like [git-prompt](https://github.com/ohmyzsh/ohmyzsh/tree/master/plugins/git-prompt) to display information about the current directory.

# in ~/.zshrc
# add plugin
plugins=(
git-prompt
)

Step 3: customize the prompt

Use your favorite text editor to open your oh my zhs theme to further customize your terminal prompt.

$ code ~/.oh-my-zsh/themes/robbyrussell.zsh-theme

Tweak the theme by updating various parts of the PROMPT string. You can use a zsh prompt generator to easily configure, bold, or colorize more elements. Paste in your own emoji, or use Ctrl + Cmd + Space on Mac to open up the emoji keyboard.

customize prompt with emoji and git status

Another option is to add a conditional emoji based on the success of the previous command. In this example, an Octopus will display in the prompt if the preceding command succeeds, otherwise a Dinosaur.

customize with conditional emojis

Save your changes, and open a new terminal tab to inspect your newest Octopus friend 🐙!

When I first moved to San Francisco from Denver several years ago, I traded in most of my outdoorsy activities for techie ones. But camping in Yosemite has been on my wish list. Year after year, I notice that it’s really tough to reserve a spot. So I’ve still never been.

There’s a bit of an issue with reserving a campsite at Yosemite, but in this case, my techie self can help out my outdoorsy self.

The problem

In 2018, a bunch of state and federal park services launched an online reservation system, Recreation.gov, allowing users to book picnic spots and campsites across America.

Located in California, Yosemite National Park is one of the most popular destinations that you can book through this website. It’s so popular that Yosemite releases a new block of campsites four months in advance.

On the 15th of every month at 7:00 a.m. Pacific Time, eager campers will scramble to book a spot at one of the prime campsites in Yosemite valley closest to the most iconic landmarks in the park.

Seconds make the difference between getting your reservation or not.

— RECREATION.GOV

The competition is fierce. Thousands will try to reserve a few select spots, and the campsites will be fully booked within 20 minutes. How is this humanly possible?

There are bots among us.

Enter the bots

A quick search in GitHub reveals scripts developed to gain an advantage in booking one of these in-demand campsites.

• A Python script to scrape campground availability
• Another one using Selenium to automate the process of browsing the website

There’s also paid services (like this one and that one) that offers to monitor the reservation website and alert you of cancellations.

Another quick search in your favorite search engine reveals articles and forum chatter about the dastardly nature of these bots, with discussion about the unfairness of it all. People are frustrated with the reservations processes going back several years.

Let’s dig deeper into what happens when reserving a campsite.

Breaking down a campsite reservation

Go to Recreation.gov in your browser. Open your browser’s DevTools and head over to the Network tab to see all of the network calls sent to the servers as you browse the site.

Checking availability

On the website, select your preferred campground, and then hit View by Availability. In DevTools under Network, you’ll see a request submitted to an endpoint called /availability. Right click on this request, and select “Copy as cURL” to copy it to your clipboard.

copy as cURL

We’ve previously used Postman to reverse engineer an API. Let’s switch over to Postman now to inspect what’s happening and modify what we send to Recreation.gov’s servers.

Paste the cURL from your clipboard to import it into Postman as raw text.

import cURL as raw text

This GET request does not require authorization, and retrieves the availability of all campsites for your selected campground. We can further inspect the headers and other components of this request.

Hit Send, and inspect the response.

inspect elements like request headers and response body

This shows us what’s happening in the background as we browse the website.

Booking a reservation

Back on the website, click “Book Now” to temporarily call dibs on a campsite for a specific date. Once you do this, you’ll have 15 minutes to complete the reservation before that spot is released back into inventory.

In DevTools under Network, you’ll see a request submitted to an endpoint called /multi. Import the request into Postman like we did before.

Under Headers, we see this POST request requires a Bearer token as an authorization header. Under Body, we see a JSON object with our reservation information.

Hit Send. The Recreation.gov servers already received a request submitted from the browser, and now we’re trying to submit the exact same request from the Postman client.

Well, that’s not going to work. But we understand a little more about what the Recreation.gov servers are expecting from the client, so we can begin building our bot.

Build your own bot

If you’re technically savvy and can write a bit of code, you can build your own bot with some googling and ingenuity. Really, there’s a few ways to get this done.

As you write your own bot, or use someone else’s, here’s some things to keep in mind.

Reuse and recycle

There’s several public solutions available as starters for various parts of the booking process. For example, you may find an app to intercept web requests and allow you to modify the payload before submitting a reservation. You can also choose to script any or all aspects of the reservation process.

Customize your criteria

Your strategy is completely up to you and your preference of campsites, dates, and other criteria. For example, someone may only be available for one week in the month of May while someone else may be flexible on dates but only want to stay in the Upper Pines camping area.

Speed is of the essence

Automation does not imply speed. You can automate a task to accomplish something very slowly. Optimize your program to run in an efficient manner allowing you to compete against other bots. For example, limiting your search to a few campsites takes less time than searching all the campsites in the campground.

complete your reservation within the 15 minute window

Is this fair?

It’s not illegal. But legal standards are not the same as ethical standards, which are established by societal norms.

What is fair

Recreation.gov promotes the programmatic use of their data via a public API. While it provides a lot of information, you can’t see campsite availability or book a reservation. Their website also uses an open JSON API to display information like availability, so that your program can access the same information that a human sees while browsing the website.

What is not fair

If you’re planning on reselling your camping spot, this is prohibited by most state and national parks. However, prosecutors typically go after folks who are commercially profiting or harming the website.

For just booking your own reservations, I couldn’t find anything on the current Recreation.gov website that explicitly prohibits the use of bots. However, I came across a number of comments online expressing frustration toward these bots. People think using a bot is “wrong”, “dishonest”, and “unfair”. Some comments go further, piling on a disgust of computers, technology, and tech workers.

Bots are simply programs created by humans. They can be helpful and automate a lot of tedious tasks. They can also be a nuisance and used for nefarious purposes. What they ultimately do, is up to the human writing the program.

With great power, comes great responsibility

-Uncle Ben

Final thoughts

It’s an interesting discussion about ethics, but I just want to go camping. It would be lovely if the National Park Service came up with a fair process that was also sustainable for our public parks, and everyone could camp freely without writing a bot.

For those who want to book a campsite in the Yosemite valley, the conversation about fairness focuses on how public access to natural resources has been co-opted by the technologically literate. Indeed the digital divide still exists even though more people now have access to the internet and technical resources.

I’m privileged because I know how to use technology to my advantage. Like most people, I rely on free resources and public communities to improve my technological literacy. If you want to learn how to do something, you can do it. The circle of life involves consuming, and then sharing, technical knowledge in forums and articles like this one.

Photo by Lenny Kuhne on Unsplash

A specification is a technical document that tells you how a thing works.

If you’re building cars, a specification tells you how a car works. It includes the most important details and perhaps a list of requirements that the end product should fulfill. Once the car gets built, the owner’s manual shows a driver how to operate and maintain the car.

Similarly, if you’re building APIs, a specification tells you how an API works, along with the most important details. The documentation shows a consumer how to work with the API.

We’ve previously talked about software development in an API-first world and also how to design APIs like you design user experience.

Now let’s see how API descriptions help us build and manage APIs.

A brief history of API descriptions

Since the beginning of APIs, various specification formats have evolved in an effort to standardize the way we describe them.

• Web Services Description Language (WSDL) was created by Ariba, IBM, and Microsoft as a machine-readable way to describe services in 2001.
• Web Application Description Language (WADL) was from Sun Microsystems as a RESTful equivalent to WSDL in 2009.
• Swagger was a project from 2011, acquired by SmartBear, then donated to the Linux Foundation’s Open API Initiative, eventually to be renamed OpenAPI Specification.
• RESTful API Modeling Language (RAML) was from MuleSoft and a group of other enterprise companies in 2013 to focus on API Design.
• OpenAPI Specification (OAS) is currently the market leader in specification formats and supported by a number of large enterprises.

These are a few of the formats that gained popularity in recent years. Their goals are all to standardize the way in which you describe your resources, and they’re all useful in different ways for developing APIs.

And the evolution continues. For example, some community members want to introduce more human-readable elements, describe more use cases for the end consumer, and find other ways to govern API design and development in a scalable manner.

Some confusing terms

In addition to a continually evolving landscape, there’s a bunch of ambiguous terminology. Industry experts don’t entirely agree on the terms, and they’re often used interchangeably when they really don’t mean the same thing.

it’s time to get pedantic

Schemas — this is how data looks

Schemas include metadata referring to your data models and provide information about the resource representations that your API accepts or returns. For example, JSON Schema is a specification declaring the proper structure of JavaScript Object Notation (JSON) data used frequently with web APIs, and is used to validate the correct format for the body of a request or response.

API description formats — this is a way to describe APIs

These formats are sometimes called API specification formats, and are usually based on a specification format (such as OpenAPI Specification) that combines a schema language (such as JSON Schema) and other information to describe endpoints, headers, and all the stuff that’s not in the body (that’s covered in a schema).

API description document — this is how a particular API works

These documents are sometimes called API specifications, and is a file that contains all the stuff from your description format fleshed out with information related to your actual API. This file describes how the API works. It can be used as a contract to programmatically build workflows useful for API development, such as tests or documentation.

API documentation — this is how to use a particular API

Documentation talks about how to use an API, and will typically include a technical reference as well as functional guides for how to interact with the API across various use cases. An API consumer who reads the API documentation should understand how to use the API.

There are meaningful differences between all of these concepts. But once again, experts don’t always agree on these meanings, and they’re frequently referred to interchangeably or ambiguously. For example, when someone says “API specification”, they could be referring to either the format for describing APIs, the file intended to drive API development, or the file generated to document an API.

The important thing is to understand these basic concepts, and then determine what is helpful for your organization’s desired workflow.

If your head hurts from reading this, so does mine. Let’s forge ahead and walk through a couple examples.

Example #1: Building blocks

From the building blocks of modern software to the building blocks of life — let’s talk about APIs and DNA.

DNA is the blueprint for every living organism

In biochemistry, DNA is the blueprint for every living organism. A schema establishes the rules for how biological molecules pair together to form chemical strands of a double helix 🧬

The human genome is a description format offering a complete genetic blueprint for building human beings. This mapping provides detailed information about the structure, organization, and function of the complete set of human genes.

My personal DNA sequence can be captured in a description file. My DNA follows the general rules of DNA, particularly that of a human’s DNA, and not an alligator or mushroom 🐊🍄

My documentation helps others understand how to interact with me. I must be fed 4–5 times a day, and I’ll get grumpy if you talk to me while I’m busy working with my headphones on.

Let’s walk through one more example — this time with our friendly, neighborhood Postman.

Example #2: Postman collections to describe an API

A Postman collection can be used as an API description format, API description document, or API documentation — depending on the context

The concept of a Postman collection is an API description format (or API specification format), in the sense that it’s how you can describe an API. The underlying JSON representation of a collection must be formatted a certain way as dictated by a schema to ensure the collection is valid.

Now, imagine Twitter has an API. When Twitter uses a Postman collection to describe how their particular API works, Twitter’s collection becomes an API description document (or API specification). Maybe the Twitter team includes a few different examples to demonstrate authorization use cases, or adds some code to their collection to show off different workflows.

As an API description document (or API specification), Twitter’s Postman collection can now be used to generate code samples, virtualized services, or API documentation to help developers learn how to use their API.

Ok , enough with the examples — it’s time to get tactical!

Test APIs and lint specifications with OpenAPI Specification, Newman, and Spectral

Just like a clothing lint roller creates a more uniform and consistent appearance by removing bits of fuzz and fur from all of your surfaces, a code linter does the same.

You may have used a code linter to ensure that your code is error-free, readable, and contains impeccable syntax for whatever programming language you’re using.

We’ve already talked about writing tests in Postman and different ways to automate your testing. Now let’s use an API specifications linter and create a custom style guide to bring law and order to our untamed APIs.

We can either run these checks locally from the command line, automatically using git hooks, or as a build step during our continuous integration workflow.

Let’s run tests and lint specifications in our CI pipeline.

Let’s run tests and lint specifications in our CI workflow

Step 0: Pre-requisites

If you want to skip to the punchline, go ahead and clone this example, and follow the README to try it out locally. Otherwise, follow these pre-requisites.

1. You’ll need a Postman account to use the Postman API to programmatically access data stored in your Postman account. Skip this step if you’re working with standalone files of your Postman collection, environment, and OpenAPI specification.
2. Make sure you have Node.js and a package manager like npm installed on your machine. Then, start a new project and install your dependencies locally.

$ npm init
$ npm install newman @stoplight/spectral

Newman — an open-source library by Postman for running Postman collections and tests

Spectral — an open-source JSON / YAML linter that supports OpenAPI Specification and JSON Schema

Step 1: Lint an OpenAPI specification file locally

Download this OpenAPI specification file to your project directory. In your terminal, use Spectral to lint this specification file, and then decide which errors or warnings you want to fix.

$ ./node_modules/.bin/spectral lint cosmos.yaml

Run the lint check using Spectral from the command line

There’s a number of options for installing Spectral, and depending on how you’ve completed the installation or have Node set up, you may need to fiddle with it. If you plan to use Spectral across multiple projects, install it globally to run:

$ spectral lint cosmos.yaml

Step 2: Generate a Postman collection from the OAS file

In the Postman app, sign in to your Postman account, and then import the specification file as an API. Give this new API a name, select OpenAPI 3.0 and YAML from the options, and then hit Save.

Now you can automatically generate a Postman collection from your specification file. For now, we’ll just write API tests. But remember, you can also generate documentation, code samples, monitors, or even virtualize mock services in Postman.

Import an OAS 3.0 specification and generate a Postman collection

💡 POSTMAN TIP*: [Import your specification file](https://learning.getpostman.com/docs/postman/design-and-develop-apis/the-api-workflow/#importing-a-file) into the Postman app to [generate a collection](https://learning.getpostman.com/docs/postman/design-and-develop-apis/the-api-workflow/#generating-a-collection-from-a-schema). The collection is the foundation for creating [documentation](https://learning.getpostman.com/docs/postman/api-documentation/intro-to-api-documentation/), [code samples](https://learning.getpostman.com/docs/postman/sending-api-requests/generate-code-snippets/), [monitors](https://learning.getpostman.com/docs/postman/monitors/intro-monitors/), or [mock services](https://learning.getpostman.com/docs/postman/mock-servers/intro-to-mock-servers/) in Postman.*

You’ll notice this newly generated collection describes the API just like the OAS file does, but it’s not quite actionable yet.

Create a new Postman environment with a key baseUrl and the value [https://e8c086f7-a5c9-4752-b81f-bfac0d0dc5d2.mock.pstmn.i](https://e8c086f7-a5c9-4752-b81f-bfac0d0dc5d2.mock.pstmn.io,)o.

Now you can submit your first request — go ahead and try it out!

Step 3: Write some Postman tests

We won’t make any changes in this folder so that you can refer back to it. Instead, duplicate this folder to add some tests.

Under the Tests tab, add a few tests using the snippets on the right or writing your own from scratch. For the first request [GET List all cosmos](https://documenter.getpostman.com/view/1559645/SW7T7WmM?version=latest#3a5e52d8-969b-46e9-88a2-d1213aa96aca), let’s also randomly pick one of the items returned in the response, parse the ID, and then save that information as an environment variable so we can use it in the next request. Remember to hit Send to execute this code.

Under the Tests tab, write Postman tests and save data as an environment variable to be used later

In the next request [GET Info for a specific constellation](https://documenter.getpostman.com/view/1559645/SW7T7WmM?version=latest#96cd8068-7d50-4d94-b221-4be0ee36f4bb), look under the Params tab and change the value of the cosmoId key from <string> to {{constellationId}}. This is the name of the environment variable that you set from the previous response, allowing this data to be accessed for this request.

Step 4: Run the Postman tests locally

In the Postman app, we can run all of our Postman tests using the collection runner. Another option is to export the collection and environment as JSON files to run tests from the command line using Newman.

$ newman run Cosmos.postman_collection.json -e cosmos.postman_environment.json

Run the API tests using Newman from the command line

Step 5: Run tests and lint specifications in CI/CD

Instead of running our checks locally, let’s add it to our continuous integration pipeline. We’ll use a bash script, but you can execute this in any manner that you like. Create a new folder called bin/ to hold a new executable file called deploy.sh.

Once again, we’ll use Newman to run our collection and Spectral to lint our specification. Last time, we used static files located within our project directory, but a better practice is to pass these elements as a URL. Using the Postman API to dynamically retrieve the latest versions, means that we can make changes in Postman without needing to re-export the latest files.

Here’s an example of how you can structure your deploy script:

Run API tests and lint API specifications in a continuous integration workflow, like this example script

💡 POSTMAN TIP*: Instead of exporting and then referencing static files, use the [Postman API](https://docs.api.getpostman.com) to dynamically retrieve the latest version of your collection, environment, or specification, and then pass these elements as a URL.*

Once you’ve written the script, remember to add the path of the deploy script file to your package.json file.

...
"scripts": {

"deploy": "./bin/deploy.sh"

},

...

Then you can run your deploy script from the command line.

$ npm run deploy

If you’re using a tool like Jenkins or Travis CI to manage your Continuous Integration / Continuous Deployment (CI/CD) pipeline, simply add a build step to execute this command.

If we experience any failures or errors, our code won’t deploy.

Step 6: Create a custom style guide

We’re testing. We’re linting. These are all good things!

But hold up. Just like how tests are only as good as we make them, linting is only as good as the rules we’ve defined.

If you work on a team, there’s probably style preferences and code requirements that everyone abides by. When pull requests are submitted, deviations are hopefully caught in code review, but frequently they’re missed. And I can already see there’s default lint warnings that I have no intention of ever addressing.

Let’s make a style guide by creating a new file called .spectral.yaml where we will define our custom rules to use with Spectral. For example, I want all of our URL paths to be camelCase, and not some ridiculous kebab-case!

Create a style guide with your own custom linting rules, like this example

Style guides ensure that everyone agrees to specified conventions. Creating a style guide in this manner also provides an easy way to enforce these conventions.

A final thought about API descriptions and linting

Using an API specification can be useful to standardize how data is exchanged between services. Furthermore, it can be useful to standardize how information is relayed between collaborating teams and throughout various phases of development.

Using a specifications linter allows you to save time otherwise spent on manual reviews. Creating a custom style guide allows you to align your team’s opinions about API design and documentation. And perhaps even more importantly, it’s also an easy way to automatically enforce these rules and govern the API development process.

Don’t waste customers time forcing them to try and figure out your inconsistencies.

Don’t waste all API developers time learning to memorizing style guides.

Don’t waste the API governance teams time reviewing APIs manually.

Don’t waste everyone’s time fixing inconsistencies in production later.

- Phil Sturgeon, APIs you won’t hate

As with all of these better practices, the important thing is to understand the possible solutions, and then determine what is most helpful to achieve your organization’s desired workflow.

I don’t deal with failure well. But I’ve gotten better at it because I fail a lot. In fact, the more I fail, the better I anticipate all the ways I can possibly crash and burn and shore up those vulnerabilities.

This anticipation could keep me up at night, but I’m stronger for it instead.

On Better Practices, we’ve previously talked about how working in the cloud is growing increasingly complex as teams continue to adopt microservice and distributed architectures. Some approaches are to enable flexibility with an API-first approach or to increase velocity by using containers and an orchestration engine.

Another approach is with chaos engineering.

Distributed systems are inherently chaotic, and a number of organizations are embracing this chaos as they move to the cloud. They intentionally inject failure into their system in order to identify previously unknown vulnerabilities.

What happens when one of your dependencies fail?

What happens when one of your dependencies fail? By proactively controlling the conditions of a failure, you can learn from your system’s response on your own terms, instead of during an unanticipated outage.

What is Chaos engineering?

When Netflix famously transitioned from a monolithic architecture over to a distributed cloud architecture, they worried about how potential downtime would impact their users. The Netflix team introduced Chaos Monkey to pseudo-randomly kill instances in order to simulate random server failure.

They wanted to make sure they could survive this type of failure and better understand the impact of such a failure on their customers. The goal was to fortify their distributed systems to expect and tolerate failure from other systems on which they depend.

[Chaos engineering] incentivizes engineers to build their services to anticipate that some servers will suddenly go away… so they have to build their services to be redundant, highly available, and fault tolerant.

-Casey Rosenthal, CEO at Verica

Shortly thereafter, the Netflix team introduced a virtual Simian Army, with each new Monkey tasked with inducing another type of failure — such as degrading a service or killing an entire region. Since then, the chaos community has developed a number of tools to more precisely cause chaos in a controlled manner.

Today, there’s a number of companies who have embraced the principles of chaos engineering. Some teams might focus on failure injection testing or disaster recovery training, while adopting only the tools and practices that work for their specific goals.

Across all of these programs, a disciplined approach teaches them how to improve their systems so they can better tolerate future, unplanned failures.

Why worry about something that isn’t going to happen?

- HBO miniseries “Chernobyl”

Why look for trouble?

A chaos experiment is like running a fire drill. Imagine that you’ve already created an emergency protocol in case of a fire, sent it out to everyone, and you test the fire alarm once a month to make sure it’s functioning.

Now what happens when you run the fire drill? You might discover that there’s a few rooms where the alarm can’t be heard, most people don’t remember where they’re supposed to meet, while others ignore the drill and stay at their desks.

By proactively testing how a system responds to failure conditions, you can identify and fix failures before they become public-facing outages. Failures in production are costly, so Chaos Engineering lets you validate what you think will happen with what is actually happening in your systems.

“Breaking things on purpose” in order to build more resilient systems prepares your team in the event of an earthquake, a zombie attack, or whatever else comes your way.

Who is responsible for Chaos engineering?

In a microservice architecture, the engineers building the services frequently have responsibility for deployment and uptime. In an organization with a traditional DevOps team, those DevOps engineers may own these service levels. Some companies have specialized teams of Site Reliability Engineers (SRE) or Production Engineers (PE) tasked with continuous improvement and production support.

Initially, the people with the highest motivation to implement chaos engineering are those who feel the pain of a failure in production, like the ones on call.

I started doing [chaos engineering] so I would get woken up less in the middle of the night and better understand my software.

It boils down to who gets paged — if that’s an SRE or Ops team, they have the most incentive to start doing this work and making their lives better.

— Kolton Andrus, CEO at Gremlin

Other people with a vested interest are the ones tasked with incident management or post-mortem analysis. The difference here is that incident management is a reactive process with steps taken to prevent reoccurrence. With chaos engineering, your experiments are conducted proactively by anticipating what could go wrong.

Right now, the vast majority of people implementing chaos engineering tend to be quality-driven and production-focused operations engineers.

However, it also makes sense to introduce the responsibility of resiliency earlier in the development cycle, when the cost of bugs is lowest, to reduce financial and other consequences in production. As such, there’s an emerging trend of testers who focus on production testing in addition to traditional pre-release testing.

So who owns Chaos?

• Specialized roles — Site Reliability Engineers (SRE), Production Engineers (PE)
• Functional teams — DevOps, Test and Quality Assurance (QA), Research and Development (R&D)
• Domain knowledge experts — Traffic, database, data, storage

The initial owners of chaos engineering will be determined by your team’s current infrastructure, talent, and goals. Furthermore, this responsibility may shift as more organizations migrate to the cloud, and chaos engineering becomes more mainstream and begins to augment traditional testing.

How can you start a chaos program?

You don’t need a special job title or even Netflix-level traffic to begin dabbling in chaos. The chaos community has developed a number of shared resources to help advance this emerging discipline. Organizations like Google, Twilio, PagerDuty, and many more have adopted their own approaches to chaos engineering.

In some cases, it’s a curious tester who kicks off a single chaos experiment after notifying the rest of her team. Maybe it’s a handful of engineers huddled together to plan their failures during a gameday. Or it’s a directive from senior management to scope out a chaos program after one particularly costly outage.

Once again, the implementation you choose will be determined by your team’s current infrastructure, talent, and goals.

Perhaps aggregate bits and pieces from different [resilience engineering] frameworks that appeal to you, and then create a practice around it. You’ll likely be the first person to create a similar practice in your particular context.

I wish the best of luck to you in that undertaking, but I wouldn’t wager that you get it right on your first try. Or your second.

-Casey Rosenthal, CEO at Verica

A common pitfall is not clearly communicating the reasons why you are adopting chaos engineering practices in the first place. When the rest of the organization doesn’t yet understand or believe in the benefit, there’s going to be fear and confusion. Worse yet, there’s the perception that you’re just breaking stuff randomly and without a legitimate hypothesis.

How do you run a chaos experiment?

While there’s many ways to run your chaos experiments, most processes echo the scientific method with a continuous cycle of hypothesis and experimentation.

Make improvements, and automate the experiments to run continuously

Plan an experiment. First, create a hypothesis about the steady-state behavior of your systems. Focus on if your systems do work, not how they work. Then think about what could go wrong. Perhaps a server goes down (as they do). Maybe it’s your third-party payment system, a specific cluster, or an entire region that’s experiencing an outage.

Start small. Initiate an attack that’s small enough to give you information about how your systems react in a production environment. You can’t always predict how users will behave. Instead of testing code in an isolated environment, test a complete system in production comprised of users, code, environment, infrastructure, a singular point in time, and more.

Measure the impact. Observe the impact of your attack by comparing it to your steady-state metrics to reveal any known and unknown issues. At this point, you can either turn up the juice on your attack or roll it back if there were unintended and potentially harmful side effects.

Learn more about your systems. Validate or update your hypothesis, and shore up your vulnerabilities. Make these improvements, and then be prepared to automate your experiments to run continuously.

Just like a fire drill, your team is developing a muscle memory by practicing how to respond when the stakes are controlled, instead of practicing during an actual emergency. By methodically performing these automated tests, you can learn more about your systems’ real-world behavior.

If you’re just getting your feet wet with chaos engineering, consider starting with your APIs. We’ve previously talked about using APIs to create additional load or security injections and other user behavior. Now you can test your fallbacks by simulating the outage of external third-party APIs or killing your own internal APIs.

When a company measures their critical services, APIs are often considered second-class citizens.

But APIs are a core part of an organization’s infrastructure, and not understanding their weaknesses can lead to performance issues and downtime.

- Tammy Butow, Principal Site Reliability Engineer at Gremlin

One way to experience an API outage is by simply unplugging a server. But what if the servers aren’t yours, or they’re hard to reach? Or you realize you’ve unplugged the wrong machine? Oops. You can’t just plug it back in and expect service to resume right away. Don’t do that.

Another way to experience an API outage is by using a Postman mock to return a 500 internal server error. This works, but it’s still just a simulation. While we can simulate outages in a test environment, there’s only one way to capture the unquantifiable conditions that cannot be replicated in an isolated test environment.

Eventually we want to break stuff in production 😈

A Postman recipe for creating chaos with Amazon EKS and Gremlin

Let’s start with an example e-commerce app where users can browse items, add them to the cart, and purchase them. Then we’ll shut down a container and see what hijinks ensue.

• Trigger: Gremlin is a failure-as-a-service and offers a free version with limited attack types. We’ll be using Postman with the Gremlin API to trigger our attacks. Spoiler alert . . . this is so we can easily automate these chaos tests with our continuous integration pipeline.
• Target: We’ve previously talked about deploying scalable apps with Docker and Kubernetes. Amazon EKS is a managed Kubernetes service that runs on AWS. It ain’t cheap. If you’re already running on hosts, containers, or another cloud platform, swap out EKS with your own target.
• Observability: We’ll use Prometheus as our time-series database and Grafana to visualize the effects of our attacks. Both are open-source and have a free version. If you’re already using something else for steady-state monitoring, swap out Prometheus and Grafana with your own toolset.

a Postman recipe for creating chaos

Set up Gremlin and create a Kubernetes cluster on EKS

If you want to jump to the end, go ahead and clone this example. Otherwise, let’s begin with this helpful guide to install Gremlin to use with Amazon EKS. You’ll need an AWS account, the AWS CLI configured to use eksctl to create the EKS cluster, and a Gremlin account.

• Step 0 — Verify your account AWS CLI Installation
• Step 1 — Create an EKS cluster using eksctl
• Step 2 — Load up the kubeconfig for the cluster
• Step 3 — Deploy Kubernetes Dashboard
• Step 4 — Deploy a Microservice Demo application
• Step 5 — Run a Shutdown Container Attack using Gremlin (skip)

After that, you should have a sample app deployed on AWS EKS and Gremlin running on your Kubernetes dashboard.

1) Sample app deployed in EKS, and 2) Gremlin installed using Kubernetes Dashboard

Set up Grafana and Prometheus

If you’re still with me, let’s set up Monitoring Kubernetes Clusters with Grafana. This particular guide starts with Google Kubernetes Engine (GKE) instead of EKS, but most of the remaining steps are the same after you create your cluster.

• Step 0 — Create a GKE cluster (skip)
• Step 1 — Lots and lots and lots of yaml configuration
• Step 2 — Configure your cluster settings on Grafana (skip)

Instead of configuring your cluster settings on Grafana, you can simply import an existing dashboard if your monitoring tools are running on your cluster. Check out the README in the sample repo for more details on how to manage external access to the apps running inside the cluster.

After that, you should be able to observe the steady-state metrics of all the nodes in your cluster.

1) Add a datasource of Prometheus type, and 2) import dashboard 3131 for an overview of all nodes in your cluster

Programmatically manage your chaos experiments

A Kubernetes pod is composed of one or more containers that share a network stack. If we attack a single container within a pod, the impact should be observed for all containers collocated within that pod. Optionally, we can further specify container ports to restrict the impact of our attack.

In the last step of the Gremlin tutorial, we shut down a container using Gremlin’s UI. We could also use an API to run our attacks so that we can programmatically manage our chaos experiments.

Let’s use Postman to shut down a single container via the Gremlin API.

In the Postman app, import the template Chaos engineering which includes a corresponding environment chaosEngineering. Check out the Chaos engineering documentation for step-by-step instructions.

Click this enticing button to import the template

You’ll first need to update the Postman environment with your gremlin_api_key and your_deployed_app_url. Then look for the folder called Shut down a container.

1. Get a list of our active containers
2. Shut down a specified container
3. Verify the health of our app
4. Stop the attack (if you need to)

Our hypothesis is that if we shut down the container running cartservice, EKS will give us a new one, and we won’t suffer any downtime. However, if we experience unintended effects, let’s be prepared to stop the attack.

1) Shut down a container using the Gremlin API, and 2) see the effects in your browser

Yikes! When we shut down the cartservice container, we see a 500 error on our shopping cart page. We see this in our browser and also in Postman if we make a GET request to our deployed app’s URL.

If you completed the last step of the Gremlin tutorial, you already know this is because cartservice uses Redis instead of Redis Cluster. Now that we have revealed this vulnerability, we can have a broader discussion with other team members about your data solution.

In the meantime, let’s make a DELETE request using the Gremlin API to stop our attack, and add some code to Postman to only stop our attack if we see a 500 internal server error.

1) Stop the attack if we see a 500 internal server error in Postman, and 2) Run our chaos tests automatically in the collection runner

In this manner, we can use Postman and the Gremlin API to explore what happens when we break a specific part of our system. Beyond manual testing, however, we’ve also seen how to automatically run our integration tests using the collection runner and a little bit of code.

🎓 Advanced challenge: now that we know how to use the Postman app to manage our chaos experiments, how can we can automate chaos testing with our continuous integration (CI) pipeline or continuous testing process? Here’s a clue. And here’s a better clue.

A final thought about Chaos Engineering

So what happens when one of your dependencies fail? A valuable chaos test will not only teach you about your systems, but also your people. The human response is frequently one of the more difficult aspects to test in a proactive and systematic fashion. It’s probably easier to test the impact of a server crash.

When chaos experiments impact production traffic and users in real life, other teams will be impacted and hopefully notified before any experiments begin. Your team’s ability to communicate and collaborate will inevitably impact the resiliency of your systems.

The biggest limitation in the fear of delivering software faster is the focus on adding more pre-release testing.

Chaos engineering is all about building trust in our resiliency and mean time to recovery. In time, we have less fear that any one change will bring down our products and when issues do occur, we are practiced in triaging and deploying fixes faster, building confidence that we aren’t fragile.

- Abby Bangser, Platform Test Engineer at MOO

For many teams, chaos engineering will require a mental shift in how failures are perceived in the organization. It’s great to identify a failure and bring it to the attention of the rest of your team. However, it’s even better to create a process that anticipates potential failures, and then reveals new information to help shore up your vulnerabilities.

Chaos engineering is not just about building more resilient software, but also building a culture of resiliency within an organization. A team that truly celebrates failures, instead of hiding them, will enable the broader organization to learn and grow stronger from these experiences.

Photo by chuttersnap on Unsplash

As cloud applications grow in complexity, sometimes teams reorganize into a distributed microservice architecture and software delivery cycles get faster. Throughout all of this, DevOps engineers keep looking for ways to streamline and automate the continuous deployment of code.

Some teams use containers, like Docker, to simplify and automate the deployment process for their applications. This is an effective way of packaging software in a way that is predictable and consistent. You can expect the software to behave similarly whether you’re on a laptop or in the cloud.

Once you get to a point where you’re running multiple containers across multiple machines, spinning up more instances of your components doesn’t scale linearly and dealing with this growing complexity gets a little hairy. Many teams will address this complexity by also using an orchestration engine, like Kubernetes. Teams are using Kubernetes as a higher-level abstraction to manage Docker container technology and further simplify the pipeline to enable their teams to go faster.

We’re already seeing tremendous benefits with Kubernetes — improved engineering productivity, faster delivery of applications and a simplified infrastructure.

Teams who were previously limited to 1–2 releases per academic year can now ship code multiple times per day!

Chris Jackson, Director for Cloud Platforms & SRE at Pearson

You don’t need to have Google- or Facebook-level traffic to care about making your web applications scalable. You might have a varying number of users, need to accurately predict your infrastructure costs, or just want to manage your systems more efficiently.

To better understand software containers, let’s first talk about physical shipping containers.

Why use containers?

Before someone invented physical shipping containers, dock workers needed specialized skills to handle different types of precious cargo. Physical containers allow us to standardize how our cargo is shipped.

There’s many reasons to use a container to ship something like bundles of firewood, for example.

• Portability — your container can be loaded onto any ship, transported by any shipping provider, or even transferred to a truck to travel over the road

Photo by Erwan Hesry on Unsplash

• Modularity — containers are all the same size and dimension, so the same crane that is used at any port to handle your container of firewood can also be used to load and unload a container of loose chickens 🐔
• Security — your container is isolated from other containers, so that someone shipping fish tanks won’t slosh fish water 🐟 onto your bundle of firewood
• Scalability — your firewood will only take up the space needed, so you can either occupy a small corner of one container or ramp up to reserve a bunch more containers across a fleet of ships

Similar to physical shipping containers, software containers are useful for standardization. In this case, software containers standardize IT infrastructure. Containers are a tidy way to package code with their dependencies into building blocks that can be deployed consistently and efficiently.

• Portability — a container is decoupled from the host operating system, so that it can run on anything from a laptop to your cloud of choice

Photo by Markus Spiske on Unsplash

• Modularity — containers give you the flexibility to create easily interchangeable application stack components, so you have a separation of concerns between components like your web server and your database
• Security — containers are immutable so updates are made by replacing the container in its entirety, making it easy to issue security patches or rollback an update quickly
• Scalability — containerized applications can scale up to handle additional load or ramp down to conserve resources during a lull

How do containers work?

Let’s revisit our physical shipping containers analogy and imagine a busy seaport where ships are coming and going all day long. There’s a container manifest that lists the contents and the loading sequence of everything getting stuffed into the container. The container gets stuffed according to this manifest and then loaded onto a ship. The dock workers will orchestrate the logistics, keeping a schedule of when the containers get loaded and unloaded, managing the arrival and departure of the ships, and coordinating with the freight carriers. At a busy port like this, we’d have some pretty hefty tools to coordinate and govern all of these details.

Now, back to the world of software containers.

Docker is one of the most popular, open-source container technologies that allows you to build, run, test, and deploy distributed applications. There’s a lot of terminology, so let’s contain our excitement, and just tackle some of the basics.

Container image

This image will inform how a container is instantiated, determining which software components will run and how. You can also create an image from a container, and share these specifications with someone else, so that an app runs the same way on a developer’s laptop as it would in production.

Container

This describes a virtual environment that bundles the application code with all the binaries and libraries that are required to run an application. Since the container includes all of its dependencies, you don’t have to install anything on the host operating system, keeping it separate and pristine.

Container orchestration

This refers to coordinating behaviors for containers and between containers, such as scheduling, resource management, and load balancing. In complex or dynamic ecosystems, teams will use an orchestration engine to control, manage, and automate this activity.

contain your excitement

After the Postman engineering team reorganized into a microservice architecture, every service now uses Docker to configure their own environments. Every service owner defines their own Dockerfile from which an image is generated when new code is deployed as part of the CI/CD pipeline. The resulting image is pushed to the team’s container registry, and their Beanstalk environments are configured to pull the image from the registry to run the containers.

Every service gets the flexibility of configuring how to run their services. So services engineers can focus on building the application while platform engineers can focus on how to build and deploy automatically.

Docker takes over the responsibility of configuring the environment and standardising the deployment pipeline. This gives us faster deployment and scaling time because the build happens only once during CI.

— Saswat Das, Platform engineer at Postman

Why use Kubernetes?

With a microservice architecture, a number of services can go into making a single application, and each of these services can live in its own container. Even a simple web application might not live in a single container. You might have one container for the web frontend, one for the backend APIs, and maybe another for data storage. If you start seeing some traffic, you can spin up more instances of your app’s components.

However, simply spinning up more instances doesn’t scale linearly. Containers allow you to scale, but managing these operations at scale can be complicated. When you’re operating at scale, you will be coordinating behavior for containers and between containers.

That’s when an orchestration engine like Kubernetes comes into play. Kubernetes is an open-source, orchestration system for automating deployment, scaling, and management of containerized applications. As a higher-level abstraction to deal with container management, there’s a somewhat steep learning curve to get set up, but then it makes day-to-day life easier.

The Kubernetes master is responsible for maintaining the desired state for your cluster

Kubernetes simplifies the deployment process for your application, and provides tools to make your application super robust.

With Kubernetes, you get rolling deployments with no downtime, service discovery, and the flexibility to change cloud providers easily.

— Dan Pastusek, Founder of Kubesail

A Postman recipe for deploying a Node application with Docker and Kubernetes

Let’s try it out! We’ll start with a simple Node app that functions like a URL shortener. In our case, we’ll transform one URL into a different one using cat verbs, cat adjectives, and cat emojis 🐱 — and when you input your custom URL into a browser, you’ll be redirected back to the original website.

The beauty of using containers is that even if I’m developing this app on a machine with my Operating System and a different version of Node, you can rely on my container image to prescribe the exact specifications you’ll need to run the same application seamlessly on your machine, or in the cloud, or wherever you choose to deploy.

Use containers to test and deploy the app

If you want to follow along, go ahead and clone this example, and follow the README steps to spin up a local version of these APIs.

1. Develop
2. Test
3. Deploy

Develop the app

Let’s start with a Node app using React for the frontend and Express for the backend. To simplify this demonstration, we won’t be implementing persistent data storage.

The backend boils down to 2 APIs.

1. Transform the original URL

2. Redirect the new URL back to the original URL

Even before you begin developing the frontend, you can use Postman as a client to send requests to our local server. It’ll save us time every time we update our payload and inspect the server response.

This is the quintessential use case most people know about when they think of Postman. You probably knew this already. Don’t worry. We’ll learn a few more ways to increase our efficiency with Postman.

💡POSTMAN TIP: use a Postman environment to save configuration parameters so you can switch quickly between server setups. For example, I used an environment variable called *url* in my collection and created separate templates for local development and production. Now, selecting the desired Postman environment allows me to toggle between [*http://localhost:5500*](http://localhost:5500) to [*https://cat-kube-stateless-prod-2ea747e90b.kubesail.io*](https://cat-kube-stateless-prod-2ea747e90b.kubesail.io), and other configuration options, more quickly.

Start these endpoints on your local server, and then let’s move over to the Postman app. Find the Postman template called catURL in the Postman app, and then import the example collection and environment.

Import the catURL collection and catURL-local environment from the Postman templates

We’ll use the catURL collection along with the catURL-local environment to debug and test our 2 APIs. Read through the collection documentation, and try it out for yourself.

Test the app

In this recipe, we won’t cover the basics of how to write tests in Postman, or how to run these tests in Postman. However, you can see a few tests already written under the Tests tab of the catURL collection. Feel free to add your own tests to validate the behavior of our APIs.

Make sure to update your general Postman settings to disallow redirects so that we can inspect our server’s response headers before the redirection. More on how to do that here.

Now that we’ve manually tested our APIs in Postman, or used the collection runner as our first step towards automation, let’s use Postman’s open source library Newman to run our collection at build time. We have 3 options:

• Good practice: test against your APIs running on a local server using static Postman collection and environment JSON files located within the project directory
• Better practice: still testing against your local server, run these tests using the Postman API to dynamically pull the latest versions of your collection and environment
• Even better practice: still using the Postman API, test against your APIs running on a container hosted by your local server so that your test environment exactly replicates your production environment

Let’s try out the last scenario — the even better practice. Add a deployment script that builds and starts our APIs in a local container. If any Postman tests fail, Newman will return an error code to immediately terminate the remainder of the script before any deployment steps are executed.

If the tests don’t pass, your code doesn’t deploy.

💡POSTMAN TIP: use the Postman API to pull the latest version of your collection or environment. Using a static JSON file doesn’t ensure you’re running the latest version of your tests.

Then, run the collection and environment using Newman as part of your Continuous Deployment / Continuous Integration (CI/CD) pipeline. The process is the same if you’re using Jenkins, Travis CI, or any similar build system. You will add a build time script telling Newman to run your Postman tests. If the tests don’t pass, your code doesn’t deploy.

Deploy the app

In this example, we will be using Kubernetes to deploy our frontend and backend to separate containers. Consequently, we’ll be describing our deployment steps in separate YAML files. These files will include your container image, resource allocation, the desired number of replicas, and other important information.

In this example, I only designated a single replica of our APIs. However, if I wanted to have 3 replicas, Kubernetes would keep a steady eye on my app instances and replace them if any single one crashed for some reason.

Configuration file for backend deployment

For the deployment, I used a hosted Kubernetes provider called Kubesail that creates a free managed namespace. However, the underlying deployment utility [npx deploy-to-kube](https://github.com/kubesail/deploy-to-kube) supports any Kubernetes cluster. By running it inside your app’s directory, this utility will automatically generate a Dockerfile, build and push deployment images, generate Kubernetes configuration files, and trigger a deployment on your Kubernetes cluster.

It’s a litter bit amazing.

Once our app is in production, we can continue testing our production APIs periodically to ensure they’re still functioning properly.

• Good practice: on an ad hoc basis, use the Postman collection runner to run the same tests along with a Postman environment that is set up with your production configuration.
• Better practice: set up a Postman monitor to schedule your tests on a recurring frequency, or set up a custom cron job using Newman to do the same thing.

Continue testing your production APIs to ensure they’re functioning properly

In addition to basic monitoring, the Kubernetes community has developed a bunch of open source resources for introspection. Tools like Prometheus and Istio provide more advanced features for logging and debugging for your Kubernetes clusters.

In summary, we used a local container to test our Node app, and then Kubernetes to deploy both the frontend and backend on containers in the cloud. Along the way, we used Postman to test our APIs locally, then before releasing our code, and finally once they were in production.

A final thought about containers and orchestration

In case you missed it, I wanted to highlight the importance of using containers during testing. Container images make it easier to reproduce and run an application that more closely replicates the production environment.

You can use your container images to reliably run your tests against an application in a test environment, like the Postman engineering team does. Containers also allow you to scale up more instances for concurrent performance testing. And in the event you’ve released a bug into production, containers make it really easy to quickly roll back an update to a previous version of the application.

There’s a bunch of ways that you can use containers in your development, testing, and deployment processes. The strategy you choose will be determined by your team’s current infrastructure, talent, and goals.

Photo by Lum3n.com from Pexels

When you think about reverse engineering an API, perhaps corporate espionage or something legally or ethically questionable comes to mind. Hackers gonna hack. However, there’s also legitimate reasons for reverse engineering an API.

Why reverse engineer an API

APIs aren’t always documented. When you’re debugging an API, you can diagnose and resolve issues more quickly when you can visualize all the data that is sent and received. Doing this allows you to gain a deeper understanding of an API.

Reverse engineering, also called back engineering, is the process by which a man-made object is deconstructed to reveal its designs, architecture, or to extract knowledge from the object.

Knowing how an API actually behaves enables you to identify flaws and security vulnerabilities like accidental data leakage. This also performance tests your API so you can isolate bottlenecks that could benefit from additional caching and compression.

Of course, there’s also selfish reasons for reverse engineering an API.

Selfish reasons for reverse engineering an API

• Accessing a service programmatically that doesn’t offer a public API
• Creating an interface that suits your needs because theirs just ain’t cutting it
• Contribute to a feature that the engineering team is slow 🐌 to deliver
• Hack with the hopes that you’ll dazzle the engineering team and they’ll offer you a job
• Order extra bacon on your pizza when the available app limits you to a sub-optimal amount of bacon
• Cheat at games — there’s an entire subreddit dedicated to this among others
• Scale up your catfishing side business by unleashing chatbots on popular dating apps

When you’re reverse engineering a private API that isn’t yours, make sure to check the terms of service. Some developers wear their cease and desist orders like a badge of honor, but you might get sued or banned from their services.

What is an HTTP/S proxy?

A web proxy server is like a middleman that sits between your client application and the server. The web proxy is a substitute server that can intercept HTTP traffic between a web browser and the website’s real server.

It’s all there! Black and white, clear as crystal!

— Willy Wonka, The Chocolate Factory

Reasons to use a web proxy

• Monitor and eavesdrop on HTTP network traffic by recording and displaying all traffic between your client and server
• Hide your public IP address while browsing websites and to access blocked content
• Filter or redirect requests to limit access to content or optimize systems performance

Postman is a proxy that captures the HTTP/S request

Free web proxy tools

• Postman is a free tool with a bunch of different proxies, including a built-in proxy to capture HTTP requests in the native apps for Mac, Windows, or Linux
• Mitproxy is an open-source proxy with a command line interface, web interface, and Python API
• Fiddler is a free web debugging proxy with support for a wide variety of browsers, systems, and platforms
• Burp has a free community edition of a web proxy server that lets you view and modify requests and responses

The remainder of this article will focus on how you can use Postman to intercept the traffic between your client and server.

A Postman recipe for reverse engineering an API

It’s time to get our hands dirty, sniffing and inspecting to our heart’s content. First, we’ll import a single request into the Postman app. Then we’ll use Postman as a proxy to capture a stream of HTTP/S requests from a variety of clients like a desktop web browser, a mobile device, and an Electron app.

To follow along in Postman, click the orange +New button in the top left of the Postman app. Under the Templates tab, search for Reverse engineering an API, and import the sample collection into your instance of the Postman app.

Import this collection and follow along with these examples

Read through the descriptions in the Postman app for details, or check out the web documentation for step-by-step instructions and screenshots.

Examples of inspecting HTTP requests

Import a single request

If you want to inspect a request in Postman, here’s a super simple way to import it as cURL from your browser. In this example, we will use Chrome DevTools to inspect and select a request. You can follow similar steps when using other web browsers.

Copy the cURL request from Chrome DevTools

Paste the cURL request as raw text in Postman

Inspect a stream of requests

If you want to inspect a stream of requests from your client, you can use the Postman built-in proxy to capture these requests. Postman has a bunch of different proxies. In this scenario, we’ll rely on the Postman built-in proxy in the native apps for Mac, Windows, or Linux. You can capture requests sent from your client, like a desktop web browser, mobile device, or an Electron app.

You can pipe this stream of requests to your Postman History and save them to a Postman Collection. Then you can revisit these requests for inspection at a later date, or share them with a teammate.

Postman as a proxy to capture HTTP/S requests from web browser

Currently, the Postman built-in proxy in the native apps only captures HTTP request traffic. Fortunately, most websites have HTTP Strict Transport Security (HSTS) enabled for an additional layer of security. Unfortunately, this means the Postman built-in proxy cannot capture requests sent over HTTPS if the website has HSTS enabled.

Note: As of the publication of this article, the Postman built-in proxy captures HTTP traffic, but not traffic from HTTPS websites with HSTS enabled. Interceptor integration and HTTPS proxy is slated for development in the Postman roadmap for developers.

A final thought about reverse engineering an API

This recipe is just the tip of the iceberg of how you can sniff and inspect HTTP traffic to start understanding what’s going on under the hood. For tougher nuts to crack, you may have to dig into SSL certificate pinning, spoof particular client attributes, or learn how to sign and authenticate more complex requests.

Although these tools and methods are powerful and can be used for selfish reasons, they can and should be used for good.

Be excellent to each other.

— Bill S. Preston, Esq.

Tools like Postman can enhance the visibility of client requests, making them easier to replicate and tweak, so you can diagnose and resolve issues faster. Ultimately, this will help you gain a deeper understanding of a public or private API, especially for APIs that aren’t well-documented.

Photo by Denisse Leon on Unsplash

As Quality Engineers, we set the quality standard on our products and hold our development teams accountable.

Writing tests for our microservices keeps our products running smoothly. We would not be able to release with the high level of confidence we currently have without these tests in place to ensure that our services are running and integrated as expected.

Trent McCann, Lead Quality Engineer at Cvent

Part 1: API tests

Part 2: integration tests

Part 3: other stuff that people talk about when writing tests

Part 4: a recipe for writing tests in Postman

As our applications and services grow, it becomes more and more time consuming to execute regression testing on the existing codebase every time new functionality is introduced.

⚠️ The testing community at times conflates common testing terminology, or refers to concepts interchangeably. Regardless of how your team decides to define particular terms, first understand the purpose for why you’re running each type of test.

Part 1: API tests

API test for a single API request

When some people talk about “units tests” for an API, they are likely referring to a test of a single HTTP API request. Similar to how software unit tests are written to assess the smallest unit of functionality, you can think of an API test like a unit test.

With this type of testing, you may or may not have access to the underlying code. However, you can validate that an API behaves as expected from the user’s perspective.

Testing a publicly facing API through your UI is not enough!

Amber Race, Senior SDET at Big Fish Games

Examples of API tests

• Status: the correct response code was returned
• Performance: the response was returned within a certain time
• Syntax: the content type of the returned content was as expected
• Syntax: the server accepts correct input
• Error handling: the server rejects incorrect input
• Error handling: excluding a required parameter results in an error
• Error handling: submitting incorrect data types results in an error
• Error detection: negative testing to identify exceptions and race conditions
• Schema: the response payload conforms to an expected structure or format
• Functional: the server returns a predictable value based on the input condition
• Functional: the request predictably modifies a resource
• Security checks: SQL injections do not result in data leakage

Tests are the documentation for your code. They should document the desired functionality of the software, and also address edge cases and error scenarios.

Valentin Despa, Software Developer at AOE

Part 2: Integration tests

Integration tests are built across multiple units to verify that components are working together as expected. This can encompass two or more individual components, or endpoints. Integration tests can include internal services, third-party services, and other external dependencies.

We write tests because we want them to fail someday, to warn us that something is our application has changed or behaves differently. While this seems rather obvious, tests that never fail are quite common.

It can be that the test run report is not properly understood by the CI/CD tool and marked as passed, or that assertions themselves are not executed, are faulty, or too permissive. So when you write a test, make sure it can fail.

Valentin Despa, Software Developer at AOE

Setup and teardown

Sometimes your test cases require some initial groundwork to prepare your test environment. Perhaps you’re generating new users, creating authentication tokens, or simply initializing variables.

After your tests run, you may need to clean up the test conditions, so that you’re not littering new users, records, and other side effects throughout your test environment.

Automate the setup and teardown of your test conditions

Automating the setup and teardown steps allows you to quickly re-create your test conditions in a consistent way so that you can repeat your tests quickly and easily. Creating repeatable tests allows you to more efficiently tweak other variables, isolate the system under test, and observe the results in a scalable manner.

Scenario tests

Make sure your application is robust and stable across a spectrum of scenarios. Testing both expected and unexpected use cases is critical to a good user experience. Visualize the user’s workflow and think about how they interact with the application.

Write and run your API tests in a sequence that mirrors a typical user workflow, commonly identified in the user story business requirements. Testers should also identify test cases for atypical user behaviors.

Generally, I take a black box approach — follow the happy-path to ensure it meets the defined functional specs and start going off that path. Try negative and edge case scenarios to see how the application will respond.

A good rule of thumb to keep in mind is: if a user can do it, they will at some point actually do it, no matter how obscure it may seem.

Trent McCann, Lead Quality Engineer at Cvent

Part 3: Other stuff that people talk about when writing tests

Performance tests

You can run these test cases to performance test your APIs. Increasing concurrent load at controlled increments allows you to identify bottlenecks and validate service-level agreements (SLAs) for internal APIs or public services.

Exploratory load testing to gain a deeper understanding of your systems

Mocking dependencies

If your test case involves shared or production resources, some teams will use mock services. Mocking the actual service allows you to rely on a predictable response so you can isolate the component under test when debugging issues.

You can also use mocks to simulate error conditions or rare instances that might be difficult or problematic to set up in a production environment.

Contract testing

As organizations continue to transition to a microservices architecture, teams become increasingly dependent on internal services. An API-first approach assumes the design and development of an application programming interface (API) comes before the implementation.

Consumer-Driven Contract Testing (CDC Testing) ensures that your service doesn’t break when a service that you rely on is updated. This type of contract testing requires the consumer of the service to define the format of the expected responses, including any validation tests. The provider must pass these tests before releasing any updates to the service.

Consumer-Driven Contract Testing ensures that your service doesn’t break when a service that you rely on is updated

We take an API-First Design approach to ensure that our API specifications are defined and approved before cutting any code. This provides us with a unified design to build upon, and allows our developers and QE to simultaneously develop our application and tests, thereby avoiding the waiting period before the handoff from dev to QE.

We leverage the Postman mock service to ensure that our tests are developed and ready for use when handoff takes place. Then it is simply a matter of swapping variables and our QEs are off to the races.

Trent McCann, Lead Quality Engineer at Cvent

Regression testing

Run initial smoke tests to verify that your build is stable enough to proceed with more rigorous, regression testing.

Regression tests can be either unit tests, integration tests, or both. Run your entire test suite after a patch, upgrade, or bug fix. Even if you’re working on a part of the code that you don’t believe impacts a previous patch, you should run your regression tests to ensure that new functionality doesn’t break something that was previously working.

Tests give you confidence when adding new features and changing your code. They should not be a burden to write and maintain but a tool that can empower you as a development team to build better software.

Valentin Despa, Software Developer at AOE

Part 4: a recipe for writing Postman tests

Now it’s time to get tactical! Let’s check out some examples.

To follow along with these, and other, examples, click the orange +New button in the top left of the Postman app. Under the Templates tab, search for Intro to writing tests — with examples and import the sample collection and environment into your instance of the Postman app.

Import this collection and follow along with these examples

Read through the descriptions in the Postman app for details, or check out the web documentation for step-by-step instructions and screenshots.

• Basic test syntax
• Considerations for using variables
• Test automation in Postman
• And examples aplenty!

Examples of writing tests in Postman

A final thought about writing tests

Once you’ve written your tests, the world is your oyster. Do more with your time, and continue your journey from manual to automated testing.

Couple your tests with a CI/CD pipeline to ensure no failure conditions are released into your server environments. Continue running your tests in production at regular intervals to monitor your endpoints and make sure they’re behaving as expected. Run exploratory or regular tests to assess the performance of your APIs with real-world loads.

Automating simple API tests and more complex integration tests allows you to catch issues earlier in the development cycle when the cost of debugging is the lowest. Running these tests in an easily repeatable manner allows you scale your development with confidence as the code base grows.

If you have examples of tests that you use, you too can share them with the rest of the Postman community. Publish your examples in a template.

As more organizations move to the cloud, they implement processes to deal with new microservices architectures, containerization, and continuous delivery. Whether they’re adopting cloud services or transitioning to a cloud infrastructure, an API-first approach can help you manage the complexity of working in the cloud.

The traditional code-first approach to app development sometimes results in delays, rework, or a disjointed frankenstein-esque experience for the developer, especially in this cloud-driven landscape.

An API-first approach assumes the design and development of an application programming interface (API) comes before the implementation. Your team begins by creating an interface for their application. After the API has been developed, the team will rely on this interface to build the rest of the application.

By introducing new features as an independent service accessed by API, the rest of the app can be stitched together, along with any other future apps.

Design and build the API first, before building out the rest of the app.

In a world where speed-to-market holds a premium, why would you spend extra time to focus on the API first?

When the code comes first

With a code-first approach, you might start with your integrated development environment (IDE) to type out a few lines of code. Somewhere in the back of your mind, you know that eventually an interface will be created to deliver this service. However, your main focus is on what you call “core functionality” not the “interface”.

When it comes time to implement the interface (the API), the core service is mostly done so it drives the implementation of the API. In some cases, the API might need to be shoehorned into place to accommodate the behavior of the core service. For example, you can’t retrieve information the way you want to due to the way access is granted or the way a database is structured.

To the user who will ultimately use this API, it can feel tacked on, like an afterthought to the core service.

Besides a disjointed developer experience, this approach leaves your team vulnerable to bottlenecks.

Once a version of the API is completed, developers will commit the code to a shared repository and release it to a server environment. Only then can your dependent teams begin. At this point, there’s a host of other tools that you might use to consume the API, to develop test cases, write documentation, and provide feedback.

Testing and documentation wait for development before getting started.

This is a synchronous flow that’s frequently encountered in the traditional waterfall software development process. Delays at any point in time will hold up dependent teams and push back the overall delivery of the product.

Feedback is gathered later in the development process, so changes are more costly since valuable time and resources have already been invested into this earlier version of the product.

When the API comes first

With an API-first approach, instead of starting with code, you could start with design, planning, mocks, and tests.

As you might already suspect, this process is aligned with the popular agile software development principle of iterating rapidly. This allows the team and any other stakeholders to weigh in on the proposed direction and functionality, early and often.

Gathering feedback at this early stage allows you to make changes more easily before a lot of time and effort is sunk into the project. Using mocks or an API development environment makes the current state of the project more accessible to both technical and non-technical team members.

By also separating the design of the API from its implementation, the architect is constrained only by the data model and business logic. Your API can now evolve unfettered by any existing user interface or legacy engineering frameworks.

Once the direction has been solidified, the design then serves as the contract that all teams can begin working towards in parallel, and only then does coding officially begin.

Testing, documentation, and development can begin independently of other teams’ progress.

To drill down a little bit deeper on the API-first approach, let’s talk about API-first development and API-first design.

API-first development

This concept refers to developing the actual API first and foremost. When you’re developing new functionality, the functionality should first be exposed as an API in your organization. The developers responsible for the rest of the application will be the first consumers of this API. This ensures the quality, predictability, and stability to withstand web clients, mobile users, and other developer consumers. Other projects requiring this functionality can now independently consume the functionality via this API.

API-first design

This approach takes it a step further and requires planning the intended API’s functionality before building the API itself. What functionality will the API have? What data will it expose? What will the developer experience be like? How will it scale? How will we add new functionality in the future?

When people talk about API-first, sometimes they’re only referring to API-first development and sometimes they’re including API-first design as well. For the remainder of this article, API first will encompass both API-first development and API-first design.

Word of Caution: focusing on the design first does not mean ruminating endlessly on all the what-if scenarios. A good design should have the flexibility to adapt to changing circumstances and accommodate new insights. As some teams can attest, mandating that specifications drive development can backfire.

Most developers still prefer documenting existing code as specifications, rather than writing the specification first. An interface is intended to be an abstraction for interacting with a system, while the implementation is the way the system does its work.

Hyrum Wright observed that it’s problematic to fully separate an interface from its implementation, especially in large scale systems.

Hyrum’s Law differentiates between an explicitly documented interface that is planned for and an implicit interface that will become evident only with usage. Consumers will come to rely on behavior observed from both the explicit and implicit interface, at which point any changes to the implementation will violate their expectations.

With a sufficient number of users of an API, it does not matter what you promise in the contract: all observable behaviors of your system will be depended on by somebody.

- Hyrum’s Law

Relevant XKCD

Why API first?

Clearly, pausing to flesh out the API delays valuable building time. Is it worth it? While not all organizations have the liberty to fully plan out their work, there are several benefits for choosing this approach that potentially outweigh delaying your time-to-market.

• Earlier validation: getting early feedback on the design allows the team to pivot or adapt to any new inputs while the cost of change is still relatively low. This reduces overall cost over the lifetime of the project.
• Clear abstraction layer: showing only the necessary details to the intended user hides internal complexity. This allows others to ramp up more quickly when implementing the new service.
• Decoupling dependencies: by adhering to a contract between internal services, dependencies are decoupled so that work can progress independent of other teams’ work. Working in parallel minimizes the project’s overall development time.
• Faster growth: building out the API design at an early stage takes future functionality into account, laying the groundwork for expansion, and accommodates the ability to quickly scale to other apps, devices, and platforms.
• Freedom from constraints: focusing first on the API instead of the code and implementation frees the design from legacy constraints.

What does API first look like in modern organizations?

Some organizations launched as API-only businesses, like Twilio or Algolia, who are both known for offering their services most importantly as an API. Twilio virtualized traditional telecommunications infrastructure allowing developers to replicate any communications experience using APIs. The business recently announced Twilio Build, a partner program designed with an API-first approach.

Twilio’s unique API-first cloud platform is tailor-made to support an ecosystem of partners that are differentiated by the innovations they deliver for their customers.

- Ron Huddleston, Twilio

As some organizations adopt agile software development practices, an API-first approach allows other companies to optimize their total development time. For example, Etsy switched to an API-first approach after facing the common challenge of implementing their logic twice.

All of the code that was built for the website then had to be rebuilt in our API to be used by our iOS and Android apps.

- Stephanie Schirmer, Etsy

Besides increasing extensibility for new devices and features, Etsy’s upgrade optimized their platform’s performance by enabling concurrent API calls previously limited by their PHP code base.

Another example of an API-first company is Netflix. Accounting for a large percent of total internet traffic, Netflix also re-configured their API architecture to allow concurrency. In the process, they distributed their API development so that each client application team was capable of implementing and operating their own endpoints.

A single team should not become a bottleneck nor need to have expertise on every client application to create optimized endpoints. Rapid innovation through fast, decoupled development cycles across a wide variety of device types and distributed ownership and expertise across teams should be enabled.

- Ben Christensen, Netflix

In this context, many teams treat APIs as a separate product or platform. Today, many engineering teams have a completely separate group for building new API functionality and maintaining scalability of their APIs.

Dedicating resources for designing and implementing the APIs demonstrates the value these organizations have for an API-first approach. This is in stark contrast with the code-first mentality which might propose building the application and then retrofitting an API as an afterthought.

Ok, it’s time to try it out.

I love pushups 💪 — who doesn’t? Let’s build a web app that reminds us to get some exercise. I can picture it in my mind, kind of.

Gif by CLTerry

At this point, wecould probably just pull up our integrated development environment (IDE) and plunge right in and get started coding.

Instead, let’s walk through an API-first approach, and learn about one way to build a new product or feature in Postman.

1. Design the new feature
2. Get feedback about the new feature
3. Build the feature
4. Deploy the changes

#1 Design the API

Let’s open the Postman app, and log in.

Create a Postman collection. This will be used to group our requests, and will come in handy when we start testing our API. Let’s name this collection after our project called Pushup.

Create a Postman environment also called Pushup to store variables so that we can update a value in a single place and propagate the changes easily. Store our base url [https://api.pushup.com/v1](https://api.pushup.com/v1) under a key called base_url. When we’re ready for development, we can swap it out to [https://localhost:3000/v1](https://localhost:3000/v1), for example. In addition to storing useful values for configuration, this would be a good spot to keep any sensitive and confidential information like API keys and tokens.

Store config values or confidential information in a Postman environment.

Let’s add a Postman folder to this collection. Similar to a collection, we can further group our requests in a meaningful way. You can nest these folders, and also dedicate variables and scripts to use within the folder scope. Let’s call our first folder Pushups after an endpoint we plan to make /pushups and follow a predictable REST architectural style.

1. GET request to retrieve a list of all types of pushups (/pushups)
2. GET request to retrieve an existing pushup (/pushups/:pushupId)
3. POST request to create a new type of pushup (/pushups)
4. PUT request to update an existing pushup (/pushups/:pushupId)
5. DELETE request to delete an existing pushup (/pushups/:pushupId)

Group requests in a collection and sub-folders.

Add a second Postman folder called “Get Started”. For developers new to this API, this folder can include a few requests walking through an example workflow. This is helpful for other team members to visualize what is happening, and also give you a head start on testing this API. Let’s duplicate some of our previous requests and drag them over to our new folder.

1. POST request to create a new type of pushup (/pushups)
2. PUT request to update an existing pushup (/pushups/:pushupId)

Create a separate folder to help others visualize an example workflow.

For every request, add an example illustrating what you’d like the response to look like. For example, we can add headers, HTTP status code, and a response body. These examples will come in handy later for mocking and documentation, before we have a working endpoint.

Add examples to demonstrate a response.

#2 Get feedback on the API design

The design is just about perfect! We’re actually ready to start developing the API, but it can’t hurt to ask our teammates what they think. Changing the API now will be much less painful than waiting until later once we’ve coded the whole dang thing.

Collaborate in workspaces: Go ahead and share the Pushup collection from your personal workspace to a team workspace so they can look it over. If anybody wants to make any changes, you can give them edit permissions.

Allow others read and write permissions for collaboration.

Activity feed: an activity feed displays updates to the collection, so that you can keep track of who is making changes, and what kind of changes.

Revert changes: if anyone makes any changes that we don’t like, restore the collection and roll it back to the point right after their change was made.

Review the collection’s activities and restore the collection to a previous point in time.

Mock the API: your boss loves pushups too, but he’s not part of your Postman team. We can mock up the responses using the examples that we previously saved for each request.

Mock the API to visualize intended behavior, all before you have a working endpoint.

Now your boss can visualize the intended behavior of these endpoints by hitting the mock endpoints, before the real endpoints are up and running.

Send a request to the mock endpoint along with the unique path in your saved example.

Turns out our teammates had some good feedback. Let’s implement some of their suggestions.

Update the /pushups endpoint to be inclusive of other exercises. Pushups are great, but what if we want to expand the functionality of the app to include burpees, arm circles, and anything else? Let’s find and replace* the text by replacing “pushup” with “exercise”, and similar instances of the word. In fact, let’s rename our collection as “Workout”.

**Note:* Find and Replace is available in Postman’s Canary v6.2.2.

Find and replace text related to “pushups” to be more inclusive of other exercises.

Add a third Postman folder called “Account” for account-related requests. It would be nice to let other people use this app too, but we don’t want to mix up our workout schedules.

1. POST generate an access token (/oauth2/token).
2. GET request to retrieve the exercise history of a user (/account/:userId)

Defining what our POST request to create an access token could look like.

Now you can initialize and save the resulting access token as an environment variable so it can be used in subsequent requests.

In fact, add an Authorization header with our access token to all of our other Pushup API requests. Don’t forget to duplicate this “Create an account” request, and add it to the beginning of our “Get Started” workflow folder.

Include the access token as an Authorization header with all other API requests.

Time for another round of feedback. Our collection updates are made in real time, so our teammates can once again review the latest version of the collection in our team workspace. The examples are also updated in real time, so your boss can view the latest version of the mock responses too.

We can continue repeating this loop until everyone feels good about what we’re building. This allows us to iterate quickly. It’s better to update the design and the mock at this stage before we invest our time and resources into development.

Alright, we’re ready to start building!

#3 Build the API

I think the collection helped people visualize the app a little bit better. It seems like there’s a few more teammates who want to help out now. Fortunately, your boss has agreed to let them work on this for one week.

How can we do this in a week?!?

With parallel development.

By gaining alignment on the plan up front, we have established a public contract of sorts to avoid any major pivots and integration failures down the line. We’ve also decoupled a number of dependencies, so that no one is waiting on anyone else.

We have everything we need to get started.

• Document the API: your technical writer has everything they need to begin describing each endpoint and laying the groundwork for the API documentation. This API is a private one, and we’ll use it internally within our organization, but thorough documentation will be helpful for anyone working on the project and especially new team members. Furthermore, this API documentation will continue to socialize the purpose and behavior of the planned service.
• Write tests: Since we saved examples, your QA engineer doesn’t have to wait for development to get going. They can write tests and assertions for each endpoint, run each test against the mock responses, or create new examples to mock any external dependencies that are not yet available.
• Develop the backend: your back-end engineer will be helping out with the API and database development. Every request in Postman represents a server-side route that must be developed. The mock responses will also provide insight into how to set up the database and structure the database queries.
• Develop the user interface: you can handle the front-end development this time. Both back-end and front-end development will be referring to the mocks to develop their respective code. You know how and where to send client requests, what kind of server response to expect, and can make requests to the mock server until a time the real endpoints become available. As a shortcut, remember that you can use Postman to generate a code snippet for every request you plan to make to the server.

Generate code snippets

Once you’ve finalized and saved your requests in Postman, you can generate a code snippet in your preferred language or framework to add to your own application. Let’s walk through how to generate code snippets in Postman.

Identify the request

In your app, imagine there’s a button called “See all exercises”. If a user pushes that button, we’d like the client to make an authenticated GET request to the /exercises endpoint.

In our Workout collection (previously called Pushup), expand the Exercises folder, and click on the GET request to list all the exercises to load it up in the request builder.

Select an environment if you’re relying on any environment variables.

When you have the request working as you’d like it, click the Code link on the right side to open the GENERATE CODE SNIPPETS modal.

Use the Code link to generate a code snippet of the current request.

Select the programming language and framework

From the dropdown, select the language and framework that you’re using for your own app. Our team is using Node.js, specifically the Request framework, to develop our web app. You can update the snippet* in this editor.

*Note: the code snippet may include a Postman-Token header. It’s a holdover from the deprecated Postman Chrome app, and can be deleted in your own app.

Select your preferred programming language and framework to customize your code snippet.

When you’re ready, copy and paste this code directly into your IDE where you’re developing your own app.

If you’re using a Postman environment variable to store your secrets, Postman will use string substitution while generating the code snippet. In your own app, you will likely have a way to pull in environment variables so that you’re not hardcoding any secrets or leaking any sensitive information.

#4 Deploy the API

Finally, everyone is ready to deploy our app to a staging server where we will test the app before letting it go to production.

• Share the API documentation: your technical writer has finished documenting all the endpoints and team members can view the private documentation in a web browser as an easy reference.
• Run the tests: your QA engineer has swapped out the mock endpoints for the real ones already deployed to the staging server by the backend engineer. They added test cases in Postman, which we will run for any new build and must pass prior to any future deployments. We’re using Postman as the single source of truth for our API tests, API documentation, and how our APIs are being consumed.

For the API documentation and API tests in Postman, team members with write permissions have made their updates. Postman automatically merge d their changes into the shared collection so that everyone can work off the latest version. If we decide to undo any changes, we can restore the collection to a previous point in time.

• Deploy server-side code: your backend engineer has committed the code updates made in their IDE to Git and pushed them to our shared repository on GitHub. We’re using GitHub as the single source of truth for our code base, including the implementation for our APIs.
• Deploy client-side code: commit the code updates made in your IDE to Git, and pushed the client-side code to GitHub.

As part of our branching workflow, the server-side and client-side code must undergo a peer review before it will be merged into our main branch. Once approved, the code is integrated and released to the staging server environment for manual and automated testing.

For this particular project, we’ll use the Postman collection runner to execute the test suite written by QA. If any tests fail, we debug the issues until the code is running as intended. Only after all the tests have passed, will we release the code to production.

A closer look at API-first development where work progresses in parallel, avoiding bottlenecks.

In the future if we decide to set up a Continuous Integration and Continuous Delivery (CI/CD) process to automate our build, we’ll use Newman to handle test automation from our CI environment.

Rinse and repeat

Huzzah! It’s been 3 glorious weeks since we released the app to production, and we are pumped, figuratively and literally 🏋️

There’s been such a fitness buzz in the office, that we’re ready to expand the functionality of the app.

Since we followed an API-first approach, we’re set up favorably to handle new feature requests. The team can use the Workout API to build out additional application functionality or expand to other platforms. Alternatively, a new team can jump in to help out and get up to speed quickly by referencing our API documentation.

Let’s continue this API-first approach for our next feature. Adding another feature will follow the same process as when we first created the app.

1. Design the new feature
2. Get feedback about the new feature
3. Build the feature
4. Deploy the changes

Repeat as many times as necessary.

Want to check it out in the Postman app? Click the orange “Run in Postman” button below to download the Postman collection and environment containing these sample requests, or check out the documentation.

click this button to download the collection and environment

What is the single source of truth for your APIs?

We agreed earlier that our shared repository, in this example GitHub, is the single source of truth for our code base, including the implementation for our APIs. This code base is deployed on your own servers, a cloud provider, or on an API gateway if you’re managing an extensive system of services.

Even though the code lives in your source control system, teams will sometimes say that Postman serves as the single source of truth for their APIs.

With Postman Pro, we’ve created a single source of truth for all API endpoints that’s shared across the organization.

- David Esposito, BetterCloud

Some teams use Postman to write their API tests, write their API documentation, and then run their API tests before any build is permitted to release. Postman is the single source of truth for their APIs.

For a lot of teams, Postman is used as a reference for the latest endpoints, use cases, and examples of how their APIs should be functioning.

Although any updates in Postman still require one more step to update the code base, via code generation or other means, the data maintained in Postman serves as the single source of truth for their APIs.

This source of truth is valuable for all API stakeholders, including Product Managers, Sales, Customer Support, DevOps, or anybody who will interact with or consume the API.

Closing thoughts on API-first software development

We’ve reviewed a number of reasons why organizations choose an API-first approach for software development. Some organizations are under the gun to push something into the market, and doing it quickly outweighs any headaches they might have to deal with down the line.

For other teams, let’s make APIs a first class citizen. It’s conceivable that one day, if not today, components will be talking to each other and services will be relied on internally and externally. The world of code is only increasing in complexity.

Consider your API, one that is resilient, versatile, and future-friendly. Designing and building the API first will save time, resources, and headaches over the lifetime of your project.