Open sourcing packages used for testing at kununu
In the recent years open source and it’s contributors started to gain the recognition they deserve for the outstanding work that they have been doing for the community. With the step forward taken by tech giants like Facebook or Google in open sourcing some of their proprietary innovations like React, Kubernetes or GO, the open source community is growing and through an open collaboration worldwide, innovation bursts.
At kununu, and mostly probably like at 99.99% of the companies out there, we use packages from the open source community in our applications and services, like the Symfony framework in our PHP services or React in our micro frontends. Contributing back to the community has been a topic among the engineers at kununu, and after a couple of months as a trello card gaining dust in our Backend GUILD board, together with my extremely skilled colleague Hugo Gonçalves, we decided to use one of the two hackweeks we have per year at kununu to move this trello card forward and open source two repositories that we developed at kununu.
Testing at kununu
Testing is an essential part of the development process at kununu and no pull request is merged if it does not has tests in place that ensure consistency, reliability, security and high performance which further results in time saving, cost effectiveness and customer satisfaction.
In his book Succeeding with Agile, Mike Cohn, came up with the concept of Test Pyramid which helps you to think about different layers of testing as well how much testing to do in each layer.
In the beginning of 2019 my team started to work in a project that involved the migration of one of the core parts of kununu, the profiles. Before starting this new project we did a retrospective on the previous projects and one point that we identified as cumbersome and difficult to maintain was our integration tests specially how we were loading fixtures.
Therefore, we decided that we needed to improve how we load fixtures in our services and defined the requirements:
- We don’t want to depend on Doctrine ORM — Doctrine ORM it’s great, still in order to ease the writing of the fixtures we can easily end up with Anemic Domain Models with a bunch of setters in our entities. Another point that we had into consideration is that we have services that do not rely on Doctrine ORM and use the Doctrine DBAL directly. With this two points we had no other choice than drop doctrine/data-fixtures as an option.
- Database schema is not touched when loading fixtures —When we use Doctrine ORM we map our entities and generate migrations using Doctrine Migrations out of those mappings. Still, we have cases which Doctrine ORM does not support, like Virtual Columns, and as such we cannot recreated the database easily when we load fixtures. With this point we had no other choice than drop liip/LiipTestFixturesBundle as an option.
- We really want to hit the database — The approach of the package dmaicher/doctrine-test-bundle is to begin a transaction before every test case and roll it back again after the test finishes. This is definitely a valid approach, still by actually hit the database we are sure we cover the whole integration even if it means a small performance penalty.
- We would like it to be a standard in all services — We want that all of our services follow the same approach making it easier to contribute to services owned by other teams.
- We would like to have the same approach for all storages — No matter the storage that we load fixtures into, we want to have the same behavior making it easier to reason about.
- We can use it in our development and test environments — We didn’t only want to use this solution when running tests but also open up the possibility to easily load fixtures for our services in our development and testing environments.
After looking at the requirements and analyse the options we had in the open source community we decided that we needed to implement our in-house solution.
Today we open source our packages which we have been using for more than a year in our services: kununu/data-fixtures and kununu/testing-bundle.
kununu/data-fixtures
As mentioned previously, at kununu we rely on data fixtures in our tests as well in our development and testing environments. A good definition of what fixtures are is the one from the documentation of Doctrine Fixtures Bundle in which the design and implementation of this package was heavily based on.
Fixtures are used to load a “fake” set of data into a database that can then be used for testing or to help give you some interesting data while you’re developing your application.
The kununu/data-fixtures package provides a simple way to manage and execute the loading of data fixtures in any storage. It’s design and implementation was heavily based on the doctrine/data-fixtures package.
Currently this package allows you to load fixtures with:
If you are interested in knowing more about the concepts of the package or you need to create a new fixture type you can read the chapter How to create a new Fixture Type.
In order to give you a glimpse on how you can use this package lets look at an example that loads fixtures with a Doctrine DBAL Connection.
Example — Load Fixtures with a Doctrine DBAL Connection
Create your fixture class
Load Fixtures
And that’s it. Simple.
If you are interested in exploring kununu/data-fixtures check out the documentation in GitHub.
kununu/testing-bundle
Most of the services at kununu are built on top of the Symfony framework. In order to enable the integration of the kununu/data-fixtures package in our services we created kununu/testing-bundle which is a Symfony Bundle.
A bundle is similar to a plugin in other software, but even better. … Bundles are first-class citizens in Symfony. This gives you the flexibility to use pre-built features packaged in third-party bundles or to distribute your own bundles. It makes it easy to pick and choose which features to enable in your application and to optimize them the way you want.
Apart from integrate with kununu/data-fixtures package, this bundle provides some utilities that makes testing easier, like a Request Builder that turns testing controllers more expressive.
In order to give you a glimpse on how you can use this bundle lets look at an example where we load fixtures with a Doctrine DBAL Connection and test a controller.
Example — Testing a Controller
Lets imagine that you have a route named profile_create which is protected (A valid access token needs to be provided) and expects a json to be provided in the body of the request with the data required to create a new profile.
Using concepts provided by this bundle, like Loading Fixtures, the RequestBuilder and the WebTestCase our test could like:
And that’s it. Simple.
If you are interested in exploring kununu/testing-bundle check out the documentation in GitHub.
I hope that this post open up your curiosity towards writing tests while using the packages kununu/data-fixtures and kununu/testing-bundle that we use at kununu and are now open source. If you aren’t, don’t worry, I know that the open source community is huge and there are plenty of amazing options out there that you can choose from.
Thank you!
