Test Matrixes

February 25, 2022 Engineering

  • When working on complex systems, there are often more than one test parameter that changes. For example when working on a component for your design system, you need to test across all these different parameters:
    • Responsive Breakpoints
    • Dark/Light Mode
    • Accesibility options (screen reader, mouse-only, keyboard-only)
    • Different browser versions
  • A good abstraction to help cope with this increasing complexity is by thinking of combinational matrixes.
    • In code, this can be multi dimensional arrays or nested loops. Jest even has a feature for it.
    • In PR descriptions, it's helpful to draw a table to make sure you test all possible combinations throghoutly.
  • Increasing the dimensions can make a full test run very costly though, so you can’t just blindly add more.
  • A very common pattern where this is userful is for manual pull request descriptions where you want to provide proof that you have exhaustively tested a new feature. A two-dimensional table might be used to show screenshots of e.g. different browsers and different responsive breakpoints.

Other Notes

June 13, 2024
Engineering/The 50-50 Goal
May 17, 2024
May 2, 2024
April 3, 2024
Engineering/Infrastructure/Deploy Workers Programatically
April 2, 2024
March 7, 2024
Engineering/Feature Flags
February 21, 2024
Engineering/Demo Culture
February 16, 2024
February 1, 2024
Engineering/ML/Embeddings
May 5, 2023
Engineering/ML/Jaccard Similarity
May 4, 2023
May 2, 2023
Engineering/Front-End/Modern Front-End Problems
November 3, 2022
Engineering/Test Matrixes
February 25, 2022
February 25, 2022
Engineering/Front-End/React’s Escape Hatch
February 21, 2022
Other/Notes
January 1, 2022

About the author

Philipp Spiess
Philipp Spiess [ˈʃpiːs]

Engineer at Sourcegraph.
Prev: UI Engineer at Meta, curator of This Week in React, React DOM team member, and Team Lead at PSPDFKit.