So far, I have published two pretty successful open source projects along with a few that are flying under the radar. In this post, I will share what I learned along the way with one of them.

How it started

When I joined trivago in 2016, a new central automation team was just formed (this did not turn out to be the best of decisions, but this is another story). This team was responsible for aligning and reorganizing all automated testing activities. A large part of that was also the creation of a new test framework as the old one proved not to be the best choice for our setup and use cases. This included adding new features, solidifying its architecture to make it resilient and maintainable and, of course, documenting it all.

Why Cucumber?

There is always some discussion about the Gherkin language in tests. Is it needed? Isn't it just overhead?

For us, it was a great idea for our end to end tests:

• It keeps the tests readable for all involved parties, even for non-technical stakeholders
• It is easy to implement
• It has lots of documentation
• Its community is very likable and helpful

Frankly, as it is often the case, we do not religiously follow the BDD (Behavior Driven Development) methodology. It is a great concept but just never fully worked for us. The ubiquitous language aspect, though, changed our tests for the better.

In case you are not familiar with Gherkin, here is a small example:

Feature: QA Meetup
    Scenario: Ticket distribution
        Given we have 49 registered people
        When I go through the registration process
        Then there should be 50 registered people

For each step, you need to implement code (in our case, Java code) to connect it to some programmatic functionality. This is an overhead compared to code-only tests but keeps the scenarios concise and understandable.

Reporting

In the beginning of our Cucumber based testing, we used pretty much the de facto standard of test reporting: cucumber-reporting. To make it clear, the following paragraph is not about bashing it or saying it is bad. It is just that it did not quite work for us since it shows a lot of information that we didn't need.

Here, you can see that a lot of the numbers in the bottom part distract from the important information of what scenarios failed and why.

The Hackathon

In 2017, I decided to try to create a custom reporting Maven plugin to solve this and have a solution that we can maintain and extend ourselves. Since I had already quite some experience with Java templating via my previous work involving JSF (Java Server Faces), plus the fact that Cucumber spits out easy to parse JSON files that include all relevant information about the tests, this did not seem like such a daunting task after all.

The tech

In Cluecumber, Google's Gson library is used to parse and pre-process the JSON files. For turning the deserialized...