How do we fulfill the promise of automation for quality assurance professionals?

Fully automated testing is still quite a ways away. However, that doesn't mean that the AI we have now isn't incredibly powerful, and Functionize attempts to harnass the latest developments in the science to make powerful, reliable tests for QA. The goal is to make tests that you can't ever break, even if you tried.

When I arrived at Functionize, you could argue the AI was too good - we had a world class backend, with some of the most advanced XPath replacements in the entire industry. Lost in all the money and time spent on the model was a similar level of effort to make the software polished and functional. Without a modern Javascript framework or a dedicated UX designer, the website was slow, inconsistent, hard to parse, and ultimately didn't demo very well. I was brought in to turn it around.

I worked with Elise Carmichael (VP Product) and Andrew Gassanoff (Visual Designer) on this project.

This page covers my work as the Lead Product Designer. If you are looking for my work as a developer and accessibility specialist, you can go here:

Goals

Visual clarity

Previous designers had set up the product to be angled at power users. This created a steep learning curve and numerous knock on effects. We had low conversion of free trial users, lots of customer questions about basic functions of the software, and a general lack of enthusiasm for the product's UI when demoing to potential clients. The backend was the star of the show, and that meant the frontend of the site had been neglected for some time.

Less dialogs

Due to technical limitations and the experience level of the dev team when the site was built, every major function occurred in a dialog. While completely removing them on the first pass was technically infeasible, I cut them down as much as I could. Dialogs create accessibility problems, but they also limit the ability of the user to engage in a nonlinear workflow. The main component of the website - site actions - was completely reimagined to use tabs, expansion panels, and clear headings to allow multiple actions to be edited to their full extent without ever needing to go into a modal.

Testing automation for everyone

I'm passionate about making digital experiences first and foremost for those within the accessibility community. Initial Axe scans revealed a site that never had anyone audit it for a11y, with hundreds of critical and severe issues. Contrast issues were omnipresent, screen readers read text covered by other elements, keyboard navigation didn't exist, the dialogs weren't made to work in accessible context, and the list goes on. As the team hadn't worked on these kinds of bugs before, I became a strong proponent of Vuetify - an open source UI library using the Material Design specification that included many a11y best practices. Vuetify gave us a floor to prevent the most severe accessibility problems from making their way into production.

Information Architecture

Style Guide

High Fidelity Mockups

1

2

3

4

5

6

7

Flow Prototype

Nicknamed Flow, this secondary product aimed to automatically track areas of test coverage and assess coverage gaps based on risk.

UX Overhaul Video Overview

The recent 5.1 release was the first step in the overhaul of the front end, and the first time many of my updates went live. Most of the changes were aimed at the core area of the product, the test detail page. This video was made with Visual Effects by Andrew Gassanoff and voiceover by Judy Bossi.

Takeaways

Listen to your customers

Customers often have great suggestions for ways to improve the product. I enjoyed talking to testers and sdets who have used Functionize as their automation tool and what their pain points where.

Don't reinvent the wheel

Even if the UX is new, the component pieces of UI should be standardized and recognizable. There were multiple places where a custom one-off piece of CSS determined something that vaguely resemebled a table. These were often completely inaccessible, and some of them exhibited glitchy behavior. All of these areas benefitted from a standardized vuetify data table, with built-in sorting and pagination. This also created consistency in the way we displayed data.

Design systems are worth the hype

No one ever wants to let someone go for behavior problems. We want to believe that people will understand the way they affect people and work on themselves, but some never do. We track every single change to a case with a permanent, unchangeable activity log. We keep bad actors accountable, so when the time comes the paperwork trail is immaculate.

Thank you for reading!