Developer Experience

Guide: Integrating Argos Visual Testing in Vercel Preview

February 1, 2024

Jeremy Sfez

Argos now seamlessly integrates with Vercel Preview for visual testing, offering a streamlined approach to catch visual regressions in a production-like environment. In this step-by-step guide, I show my journey to run Argos on Vercel Preview via GitHub Actions, and share my source code to do so.

Staircase / eye in library — Photo by Petri Heiskanen

Why Visual Testing with Vercel Preview? Key Advantages

Reducing CI cost: Eliminating the need to set up a second environment to run the tests.
Increasing test reliability: Visual testing on a production-like environment.
Increasing test reliability: Visual testing on a production-like environment.

Note: This method is optimal for repeatable tests that don't significantly alter the environment, as certain tests (e.g. those involving user deletion) may not be applicable.

Getting Started with Argos: Setup Guide

Prerequisites

Before integrating Argos into Vercel Preview for visual testing, I have:

Set up a Next.js project with Playwright.
Integrated Playwright tests into CI/CD.
Deployed the project on Vercel.

1. Project Importation to Argos

Signed up into Argos and import my project. In response, Argos generated an ARGOS_TOKEN for my repository that I saved it for later usage.

Argos GitHub token

2. Package Installation

npm install --save-dev @argos-ci/playwright

3. Setup Argos in Playwright config file

Update the playwright.config.ts file:

import { defineConfig } from "@playwright/test";

export default defineConfig({
  reporter: [
    process.env.CI ? ["dot"] : ["list"],
    [
      "@argos-ci/playwright/reporter",
      {
        uploadToArgos: !!process.env.CI,
        token: process.env.ARGOS_TOKEN,
      },
    ],
  ],
  use: {
    trace: "on-first-retry",
    screenshot: "only-on-failure",
  },
});

4. Screenshot my app

I utilized the Screenshot pages script from Argos's documentation for capturing my application's homepage. This script is flexible for adding more pages in the future.

import { argosScreenshot } from "@argos-ci/playwright";
import { test } from "@playwright/test";

const baseUrl = "http://localhost:3000";
const pages = [
  { name: "homepage", path: "/" },
  // { name: "faq", path: "/faq" },
  // { name: "contact", path: "/contact-us" },
  // { name: "pricing", path: "/pricing" },
];

test("screenshot pages", async ({ page }, workerInfo) => {
  for (const { name, path } of pages) {
    const browserName = workerInfo.project.name;
    await page.goto(`${baseUrl}${path}`);
    await argosScreenshot(page, `${name}-${browserName}`);
  }
});

5. Set up ARGOS_TOKEN and test integration

First, I set up ARGOS_TOKEN as a repository secret on GitHub. Then, I'm ready to push my code to GitHub monitor the status checks to ensure everything is working as expected: Playwright, Vercel, and Argos.

Github tests with Argos

Run Argos on Vercel Preview

To run my tests on Vercel Preview, I updated my CI workflow file:

name: Playwright + Argos Tests
# 👉 Trigger on deployment event
on: deployment_status:
jobs:
  test:
    # 👉 Run tests only if the deployment is successful
    if: github.event_name == 'deployment_status' && github.event.deployment_status.state == 'success'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v3
      - name: Install dependencies
        run: npm ci && npx playwright install --with-deps
      - name: Run Playwright tests
        run: npx playwright test
        # 👉 Add this variables
        env:
          BASE_URL: ${{ github.event.deployment_status.environment_url }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          ARGOS_BRANCH: ${{ github.event.deployment_status.environment == 'Production' && 'main' || '' }}

For a comprehensive explanation of each step, read Argos documentation.

Then, I updated my Playwright's config file:

export default defineConfig({
  use: {
    // URL generated for the preview
    baseURL: process.env.BASE_URL,
  },
  extraHTTPHeaders: {
    // Hide vercel toolbar in tests
    "x-vercel-skip-toolbar": "0",
  },
});

After my changes are merged, upon a successful deploy, the E2E workflow start on the preview:

Github pull-request check statuses

Explore the Source Code

Explore my GitHub Repository to read to code source of this integration.