Your tests aren’t the problem — it’s your untestable code
At some point, we’ve all moaned and groaned about writing tests. We already did loads of testing while we made it, so why do we have to waste our time trying to cram in automated tests? They’re a pain to write, and will break as soon as we make new changes. I shared this perspective at one point, but it changed once I realized a few things:
- Good test coverage means less manual testing
- Regression tests ensure bugs don’t come back
- Refactor as much as you want, so long as the tests still pass
- A failing test is more useful than a passing one
You had me at unlimited refactoring, but consider the time-savings you can achieve by investing in your automated tests. If they can reliably cover the most important flows, then repeated manual testing becomes a waste of time! But this doesn’t change the fact that tests can be difficult to write, and difficult to maintain; how do we make the investment worthwhile? By making your code testable.
But how do you write testable code? You treat your tests as first-class citizens; by architecting your services for testing first, and for running second. In this guide, I’ll show you the strategies I’ve learned by example: implementing part of the well-known Swagger Petstore API.
I’ve tried to keep the code as focused on the main goal of this guide as possible. I’ve omitted any code that doesn’t directly contribute to that goal, so the code may seem unfinished or non-robust as a result.
The samples provided here were extracted from a sample project. To see how it all fits together, you can find it here.
We’re going to use two major strategies to accomplish our goal today, mocks and dependency injection.
Mocks are a way for us to mimic a real object by replacing its behaviour with one we can control. When we build the tests for our application, we replace the external dependencies with mocked versions. There are libraries that can accomplish this using reflection, but I prefer to write mine myself.
Dependency injection is a simple concept: it essentially boils down to passing in all of our database and client classes as parameters to our business logic, rather then accessing them statically. It prompts us to be more thoughtful in how we architect our service, but it will also act as the delivery mechanism for our mocks.
For the sake of this guide, let’s start from the bottom-up: at our Database Access Object (DAO) — or Repository. This is our
Pet model, and we want to retrieve it from a MySQL database.
We might come up with a DAO like this… I used plain JDBC here, but most ORMs will let you inject your own
Connection anyway. In order to keep our future tests simple, we ensure there is no business logic embedded in our DAO; it should be focused on access only.
The important thing to note here is that we inject a
DataSource into the constructor for the DAO to use. This means that we’re free to create this DAO and connect it to any database we want.
Now you might think that testing database code is a huge pain, because it involves setting up a database in your CI environment, but it doesn’t have to be. If we can compromise on a few superfluous features, we can write SQL that’s compatible with both MySQL and H2: a pure java database with an in-memory mode. So let’s go add that to our project.
The first thing we need to do is create some sort of harness we can re-use in our tests. This
PetsDao.mock extension function will build a DAO that’s connected to the test database via dependency injection.
If you want, you can even dump your database seed into a SQL file and execute that.
Here’s our first test. All it needs to do is create a new DAO that’s connected to H2, and run our tests. Since a new database is created for every test, we don’t need to worry about tests affecting each other.
Our first test was really easy; it validated the code in our DAO, and gave us a stable platform to build upon.
The next thing I want to cover is API clients. These can be fully tested, with the right HTTP client. I highly recommend http4k; it’s incredibly easy to inject a fake server into a client, and can run without using the network.
For this example, we’ll integrate a third-party image hosting service so we don’t have to worry about storing Exabytes of cat pics. Let’s make a client.
The important thing once again is our constructor parameter, where we inject an
http4k , the
HttpHandler is literally just a
(Request) -> Response . This is very powerful, because while we can inject an actual http client — like okhttp or java http — we can also inject our own fake server.
You might notice how there’s no hostname or credentials, but those can just be injected into the
HttpHandler by a
Filter. Our test backend doesn’t care about hostname and credentials, so our client doesn’t necessarily need to care; feel free to disagree and disregard, but we’ll add those later when we’re ready to run for real.
An alternate approach would be to make an
ImageClient interface, make a fake version of that, and then not have to deal with a fake http server. This is ok, but it means we don’t have any test coverage in our clients, which can hide some of the worst bugs.
For now, let’s write our fake server.
See how our fake is also an
HttpHandler? We can inject this directly into our client and all requests will be routed to it. Now, at this point, we could make some tests, but I don’t usually test API clients directly; once again, feel free to disagree and disregard; but it will get tested when we test our business logic.
One thing to note is that the ids generated by this fake are sequential, which makes the side-effects and return values easy to know ahead of time. Adding randomness to tests can make them brittle, so it’s usually safer to work with something more predictable.
So I know we’ve been doing this backwards, but here is where we’re finally going to write some business logic to tie it all together.
The service code itself is pretty basic; once again you should notice that our
ThirdPartyImageClient are both injected in via the constructor. You should also notice there’s absolutely no database transaction logic polluting our business logic.
Many frameworks will train you to couple your service calls with database transactions, but we’re not going to do that here. Why? Because for all our business logic knows here, our
PetsDao could be backed by anything, like DynamoDB, memory, or an API.
PetsDaoTest is where things start to get really interesting. In order to construct our
PetService we need to build a
PetsDao with the fake database, and a
ThirdPartyImageClient with the fake server. Then we can start doing some very interesting tests.
add image test as the prime example: we start off with an existing
Pet so we seed the test by calling
petsDao.create. We do not call
PetService.create because it could potentially be hiding side-effects or other logic that we haven’t tested yet. If the test fails, we wouldn’t know which call to
PetService is at fault.
This way, if the test fails, and our
PetsDaoTest methods are passing, we are in a better position to diagnose the fault. Alternatively, you can load a global seed from a SQL file.
Once our test calls
petsDao.uploadImage we do three verifications. The first is to verify that the
Pet retuned by
uploadImage looks how we expect. But then we know there should be two side effects, so we need to validate them. In order to do that, we’ll perform assertions on our
Now that we have business logic in our service, the next step is to provide an interface for clients to use it. We’ve made our service modular enough that the interface could be REST, RPC, WebSocket, a stream listener, or even a ViewModel for a native app.
But for this example, we’ll build a REST Server. Once again, I highly recommend http4k, because it can be tested without starting a server or using the network; Ktor is capable of this too.
For this REST API, our only dependency is on our
PetService . Notice how it can return an
HttpHandler which is the same thing we inject into our
Client backends and servers are the same thing; neat!
In our test, we create a
PetService the same way we did in our
PetServiceTest , but then we inject it into our
RestApi , which provides an
HttpHandler, which we can make requests against.
I recommend you don’t go overboard with your API tests; they’re harder to maintain, and after a certain point, they don’t provide any coverage that our
PetServiceTest doesn’t already provide. If you create a new version of the API, it might end up using the exact same
PetService calls, so the extra tests for that API are extra redundant. A good rule of thumb might be to only test the different variations of status codes you expect to receive. For example, the 200, 400, and 404 cases.
You might also notice that we’re still verifying some side-effects here. We don’t need to go too overboard, but we do need to verify that the things we wanted to happen did indeed happen. It’s entirely possible the API is just returning some sample data and not actually doing anything.
So after all this time, we’ve completely tested our service, but we haven’t actually run it for real yet. No problem. We can just build a
Runner module that gathers our configs builds our
RestApi , and then hooks it up to a real server.
You can gather your config parameters however you want, but ENV gets the job done here. In essence, this runner isn’t too different from setting up a test, but there are a few important differences:
MySqlDataSource, pointing to a real MySQL server
ThirdPartyImageClienttakes a real client as the
HttpHandlerbut wraps it in a
Filterthat adds the hostname and credentials to the request
RestApiis converted into a real server and started
Now we can just run our new
main method and our app is running for real.
So testing in isolation is all well and good, but there were several assumptions we made in the
ThirdPartyImageClient and the
PetsDao . These assumptions may not be correct, so this is where pretty much the only manual testing is required.
We need to run our app for real, making calls against the real MySQL database and third-party image hosting service. If we run into any errors, it’s very important we modify our automated tests or fake backends to reproduce the error, and then fix our production code. This allows our manual tests to actually contribute regression tests to our suite, reducing the need for future manual testing.
For example, say the 3rd party image hosting site returns a 404 because we got the path wrong. Mistakes like this are inevitable when we bake assumptions into our mocks, but by incorporating the fix into both our mocks and our production code, we adjust our incorrect assumptions and reinforce our correct ones. If we make a mistake in our production code that’s caught by our mock, that’s incredibly powerful feedback that we might not have encountered at all through manual testing.
I hope the examples shown in this guide help improve the testability of your code. As your test coverage increases, you’ll start to feel several advantages:
- While automated tests can focus on the details, manual testing can be limited to simple smoke tests
- Less time wasted on testing means greater development velocity
- Tests create a safety net that makes it feasible to make large refactors
- Greater confidence in deployments and releases
I wish you luck in making your services testable; let me know how it works for you, and thank you for reading!
This guide was written around a sample repo, which is available on Github. It may provide a more complete picture to help you make your own testable code.
You can even mock the AWS SDK! There are two methods:
- Create a mock implementation of the client’s interface
- Start a fake AWS server and override the endpoint of your clients
You can always create your own mock implementations of SDK clients, but since AWS has a well-known and understood API, there are some off-the-shelf tools that can save you a lot of time.
Moto is well known and has two operating modes. It can either be integrated directly into a python application, or it can start a mock server. This is my tool of choice for python applications.
The AWS SDK is really just a bloated client for their REST API, so http4k has a side-project to offer featherweight pre-built clients for some of the most popular AWS services. This tool seems mainly intended to replace the AWS SDK, but it does offer mock backends for some of its supported services. It’s great for serverless environments since you can replace Jackson and Apache HttpClient for adapters that don’t have abysmal cold-start performance.
I wanted a tool that was pure java, could be injected easily and supported the Dynamo DB Mapper. None of the tools above could give me all of those things.
So I started this project for my co-workers and me to use in all of our services. It supports the most commonly needed AWS services and gets the job done 95% of the time. To use it, you just have to create a mock client and inject it in place of an AWS SDK interface. I hope you’ll check it out!