Been A Long Time Since I Wrote And Rolled: Automation Assist To The Rescue!

Hello! It’s been a long time since I’ve rock-n-roll, er, I mean, written. For the past few months, I’ve been mostly heads-down working for multiple clients, helping to deliver quality software both as a strategist and a practitioner. I’ve had some interesting challenges to solve, and I’d like to share how we helped solve one of them.

For a bit of historical context, I’ve written previously about what I call “automation assist”. I’ve also written about my first “professional” software creation, which was way back in the early 90’s; I didn’t know it at the time, but it was my first introduction to automation assist.

Fast forward to 2025… I once again had the chance to make testing more effective and more efficient, which I love. I was working with a client who needed to add a Spanish version of their website to their offerings. How the heck do we automate the testing of that?

We don’t. What we do is help the testers be, as I mentioned above, more effective and/or efficient. Bluntly, I wanted automation to help them do their jobs, not replace them. In this case, them was me.

So, what did we do? Based on the inspiration of my friend and colleague Bill Roske, I created a program to filter out obvious misses so that the testers would not waste their time on stuff that we already knew was broken. What was an obvious miss? A DOM element on the English site that had the same text on the Spanish site; we classified that as untranslated and, therefore, a likely error.

The main algorithm was
• For each URL in the site map, snag all the elements
• Use the web page’s feature to switch to the Spanish site
• Snag all the elements on the now Spanish site
• Compare the number of elements on each English page to the number of elements on the corresponding Spanish page. If they are different, report a discrepancy; based on the implementation expectations, the translation should be 1-for-1, meaning, regardless of content, the DOM’s element structure must be the same (oops… keep reading to see what happened)
• If the above succeeds, we now know that we have the same number of elements on each page
• Next, we iterate through both lists of elements
• If the text in any element from the English elements matches the text from the corresponding Spanish element, the text in that element was probably not translated, so we flag that as a discrepancy.
• The end of the execution was not pass or fail; it was “did the script find anything that was not expected?”

This was awesome for the first phase of the translations.

Uh Oh! A base assumption failed… or was changed… or was misunderstood… or was never true.

When I started this implementation, I was told that the DOM for the English site, with respect to element hierarchy, would be the same for each translated page, meaning that if the number of elements didn’t match, that was an automatic issue. That assumption did not hold. This kind of thing happens more than any of us would like. Advice: always check your assumptions.

We had a choice:
• Continue without any automation. This was not a good idea because of the volume of pages on the website.
• Evolve the automation to handle the DOM variations. This was also not a good idea because of the complexity and cost of implementing this solution; this may not even be a workable solution.
• Continue with the automation basically as is. We decided this was the most appropriate solution because only minimal code changes were required, and we still obtained some value from the automation, just not as much as we’d expected.

Of course, as there always are, there were downsides to this “basically as is” approach.

We received, what I’m calling, “repeated failures”, i.e., failures that are either a re-report of previously reported failures or failures that are reported due to previously discovered failures. As an analogy, it’s similar to compiler errors that are only reported due to previously reported errors. For example, missed a semicolon? Many errors shall be reported; it’s the nature of parsing. Despite this “noise”, did it provide value? To this team, it did.

Also, using this approach, the results are inconclusive. We could only count on the automation to tell us did the checks “egregiously fail”, or did they not. In the parlance of testing and QA, we didn’t have a pass/fail report. We had a “good or not good” report. All other information had to be gleaned from the report and distilled by the tester, in this case, me.

Is that a problem? Not necessarily. As much as red/green, pass/fail results can be desirable, sometimes that’s not realistic or economical. The results of this tool’s execution gave us “you have data”. Did it save the team some effort? In our case, yes, it did. I had to wade through the results, but it took far less effort than for me to “turn the crank” to produce the data and then wade through it.

To be fair, this implementation has a very weak heuristic. The heuristic was stronger when we thought there was DOM equality, but that said, the heuristic was rather weak to begin with: ”Dom element in English matches the same element in the translated DOM, but not in English”. There were cases where that was OK, but in other cases, that was an issue. For example, street addresses were intentionally not translated. Sometimes the automation could recognize that, but not always. The human, the tester, needed to filter that out when reporting issues. Again, in this case, the tester was me.

Reenter the “automation assist” aspect of test automation. This is a great example of what I’ve been calling automation assist: non-test-case based mechanization of part of our jobs that we can trust and makes us more effective or more efficient.

Remember, automation is not an “all or nothing” endeavor. If it provides value and helps us do our job, it’s probably worth it!

Like this? Catch me at or book me for an upcoming event!