On End-to-End Tests During Redesign

By Radosław Miernik · Published on 31 August 2026 · Comment on Reddit

Table of contents

• Intro
• Priorities
• Configuration
• Tests
• Closing thoughts

Intro

Almost a year ago, I wrote On How to Handle a Redesign. During that time, we did a lot of groundwork, including a proprietary design system (that’s a topic for a separate post) and applied it to a few “pilot” features – new ones that were already planned with the new styles in mind.

Today, with a few business-critical views being worked on, we started working on the automated test setup for the transition period. The goal is simple: have a test suite for both designs simultaneously. I’ll explain the secondary challenges priorities later.

Even though our end-to-end test setup is rather common – a Playwright suite orchestrated with GitHub Actions – the idea is applicable to other test runners and environments. For further details about our setup, including the complete workflow, please refer to On Playwright in GitHub Actions.

Priorities

Assuming infinite resources – either team- or time-wise – we could maintain two separate test suites (one for each version of the system). Of course, it’s far from being feasible, especially because we plan to do a gradual rollout¹.

Therefore, our first priority is to reduce the amount of maintenance needed while the redesign is happening, i.e., have the redesigned views tested as soon as possible. That, to some extent, implies a high degree of code reuse².

Another implied practice is preservation of the semantic structure wherever possible. That goes for the basics, e.g., “Where do we use the <button> tag?”, but also for test-specific things, e.g., “What data-test attribute does the Save action have?”³. Of course, it’s not always possible, but at least try.

Next in the priorities is the developer’s experience. Obvious, I know. But the more annoying it is, the less people will want to do it. And we’re talking about new React components for virtually every single thing on the screen, so things will break, at least during development.

And lastly, the easier it will be to clean it up afterward, the better. Like, if we could somehow make it in a way that git diff of removing it all will consist mostly of removals (not changes), we’re golden. Half points if it’ll be the case with whitespaces ignored⁴.

Configuration

The approach I suggest is trivial: run tests with some environmental variable that implies they should assume the redesigned components will be used. Yes, that means we’ll need to run the same tests twice, but we’ll get to that later. We also need to mark the tests somehow as if they aren’t affected, there’s no need to run them more than once. And finally, we’d like to keep them separate in our reporting tool⁵, to track failures in both versions independently.

Let’s start with the workflow file. We’ll name the flag REDESIGN, and add it to the env block. In code, it’ll look like this:

     - name: Run Playwright tests
       run: npx playwright test --project=${{ matrix.project }} --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}
+      env:
+        REDESIGN: 'true'
     - uses: actions/upload-artifact@v3
       if: always()

But wait, this will run all tests exclusively for the redesign. All projects (well, browsers: chromium, firefox, webkit) and all shards will run with the same flag. Luckily, there’s an option to change the matrix to list all interesting cases explicitly, using matrix.include. The change is rather verbose:

     strategy:
       fail-fast: false
       matrix:
-        project: [chromium, firefox, webkit]
-        shardIndex: [1, 2, 3, 4]
-        shardTotal: [4]
+        include:
+        - {shardIndex: 1, shardTotal: 4, project: 'chromium'}
+        - {shardIndex: 2, shardTotal: 4, project: 'chromium'}
+        - {shardIndex: 3, shardTotal: 4, project: 'chromium'}
+        - {shardIndex: 4, shardTotal: 4, project: 'chromium'}
+        - {shardIndex: 1, shardTotal: 4, project: 'firefox'}
+        # ...
+        - {shardIndex: 4, shardTotal: 4, project: 'webkit'}
     timeout-minutes: 60
     steps:

Why on Earth would we do that? Only to be able to add more, redesign-specific cases, like this⁶:

         - {shardIndex: 3, shardTotal: 4, project: 'webkit'}
         - {shardIndex: 4, shardTotal: 4, project: 'webkit'}
+        - {shardIndex: 1, shardTotal: 1, project: 'chromium', redesign: 'true'}
+        - {shardIndex: 1, shardTotal: 1, project: 'firefox', redesign: 'true'}
+        - {shardIndex: 1, shardTotal: 1, project: 'webkit', redesign: 'true'}
     timeout-minutes: 60
     steps:

Now we can use the new matrix.redesign variable! Let’s use it for the job name⁷, flag value, and the artifact name (to make sure it’s unique):

 jobs:
   test:
-    name: 'Test on ${{ matrix.project }} (${{ matrix.shardIndex }}/${{ matrix.shardTotal }})'
+    name: 'Test on ${{ matrix.project }} (${{ matrix.shardIndex }}/${{ matrix.shardTotal }}${{ matrix.redesign && ', redesign' || '' }})'
     runs-on: ubuntu-latest
     container:

     - name: Run Playwright tests
       run: npx playwright test --project=${{ matrix.project }} --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}
       env:
-        REDESIGN: 'true'
+        REDESIGN: ${{ matrix.redesign }}
     - uses: actions/upload-artifact@v3
       if: always()
       with:
-        name: playwright-report-${{ matrix.project }}-${{ matrix.shardIndex }}
+        name: playwright-report-${{ matrix.project }}-${{ matrix.shardIndex }}-${{ matrix.redesign && '-redesign' || '' }}
         path: playwright-report/
         retention-days: 30

The last step would be to filter only the tests that should run. This is not a problem either – we can use tags and the grep option. Configuration will look as follows:

 const config: PlaywrightTestConfig = {
   // ...
+  grep: process.env.REDESIGN === 'true' ? /@redesign/ : undefined,
   projects: [
     { name: 'chromium', use: { ...devices['Desktop Chrome'] } },

The only problem left is that in our reporting tool, some of the tests are now added twice. We considered different approaches but decided to do a quick-and-dirty approach, i.e., create dedicated projects with different names for it. This way we have a clear cut in the test run summary.

         - {shardIndex: 3, shardTotal: 4, project: 'webkit'}
         - {shardIndex: 4, shardTotal: 4, project: 'webkit'}
-        - {shardIndex: 1, shardTotal: 1, project: 'chromium', redesign: 'true'}
-        - {shardIndex: 1, shardTotal: 1, project: 'firefox', redesign: 'true'}
-        - {shardIndex: 1, shardTotal: 1, project: 'webkit', redesign: 'true'}
+        - {shardIndex: 1, shardTotal: 1, project: 'chromium-redesign', redesign: 'true'}
+        - {shardIndex: 1, shardTotal: 1, project: 'firefox-redesign', redesign: 'true'}
+        - {shardIndex: 1, shardTotal: 1, project: 'webkit-redesign', redesign: 'true'}
     timeout-minutes: 60
     steps:

+const suffix = process.env.REDESIGN === 'true' ? '-redesign' : '';
 const config: PlaywrightTestConfig = {
   // ...
   grep: process.env.REDESIGN === 'true' ? /@redesign/ : undefined,
   projects: [
-    { name: 'chromium', use: { ...devices['Desktop Chrome'] } },
-    { name: 'firefox', use: { ...devices['Desktop Firefox'] } },
-    { name: 'webkit', use: { ...devices['Desktop Safari'] } },
+    { name: `chromium${suffix}`, use: { ...devices['Desktop Chrome'] } },
+    { name: `firefox${suffix}`, use: { ...devices['Desktop Firefox'] } },
+    { name: `webkit${suffix}`, use: { ...devices['Desktop Safari'] } },
   ],
 };

Tests

Now, with all configuration done, we can finally get to the tests. The first thing we need to do is to add the @redesign tag we just filtered:

 // Based on Playwright examples.
-test('has title', async ({ page }) => {
+test('has title', { tag: '@redesign' }, async ({ page }) => {
   await page.goto('https://playwright.dev/');
   await expect(page).toHaveTitle(/Playwright/);
 });

This is enough to make the test run twice, once with and once without the flag. But how does the test “know” it’s supposed to do something else? We must once again use the REDESIGN flag⁸:

 // Based on Playwright examples.
 test('has title', { tag: '@redesign' }, async ({ page }) => {
   await page.goto('https://playwright.dev/');
-  await expect(page).toHaveTitle(/Playwright/);
+  // Easy, it's just an example.
+  if (process.env.REDESIGN === 'true') {
+    await expect(page.getByRole('link', { name: 'Get started' })).toBeVisible();
+  } else {
+    await expect(page).toHaveTitle(/Playwright/);
+  }
 });

Of course, in reality, our tests are not that simple. How can we apply it to our page object model approach? You guessed it – use the REDESIGN flag. The same goes for helpers, fixtures, and every other piece of code your tests need.

Just remember to add a setup step that opts in for the redesign, if needed. In our case, we added it in the user registration flow, so the entire suite is covered automatically. Alternatively, you can make it explicit, but then it’s on you to remember about it.

Closing thoughts

Let’s review the priorities and see if we did a good job. Can we reuse the existing tests easily? Yes, all we need to do is tag them. Can we reuse the rest of the code (e.g., page objects or helpers)? Sure, they can rely on the REDESIGN flag. Does it impact the components structure? Not at all. Can we remove it “cleanly”? Yep, all we’ll do is remove a bunch of conditions here and there.

To sum up – it works, is rather straightforward, and can be wiped out quickly. A nice thing is that we can use the exact same approach for other things, e.g., testing the mobile version of the application with different navigation patterns.

Now let me get back to the actual redesign…