#Playwright in CI/CD: Performance Testing, Visual Regression, and Production Monitoring

Integrating Playwright into your CI/CD pipeline transforms it from a testing tool into a comprehensive quality assurance platform. This guide covers advanced CI/CD integration, performance testing, visual regression detection, and production monitoring strategies.

#CI/CD Pipeline Integration

#GitHub Actions Configuration

yaml

# .github/workflows/playwright.yml
name: Playwright Tests
on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]
  schedule:
    # Run tests daily at 2 AM UTC
    - cron: '0 2 * * *'

jobs:
  test:
    timeout-minutes: 60
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        shard: [1, 2, 3, 4]

    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 18
          cache: 'npm'

      - name: Install dependencies
        run: npm ci

      - name: Install Playwright Browsers
        run: npx playwright install --with-deps

      - name: Run Playwright tests
        run: npx playwright test --shard=${{ matrix.shard }}/${{ strategy.job-total }}
        env:
          PLAYWRIGHT_BASE_URL: ${{ secrets.PLAYWRIGHT_BASE_URL }}

      - uses: actions/upload-artifact@v4
        if: failure()
        with:
          name: playwright-report-${{ matrix.shard }}
          path: playwright-report/
          retention-days: 30

      - name: Upload test results to Tesults
        if: always()
        run: |
          npm install tesults
          node upload-results.js
        env:
          TESULTS_TOKEN: ${{ secrets.TESULTS_TOKEN }}

  merge-reports:
    if: always()
    needs: [test]
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 18
      - name: Install dependencies
        run: npm ci
      - name: Download all reports
        uses: actions/download-artifact@v4
        with:
          pattern: playwright-report-*
          path: all-reports/
      - name: Merge reports
        run: npx playwright merge-reports --reporter html ./all-reports
      - name: Upload merged report
        uses: actions/upload-artifact@v4
        with:
          name: final-playwright-report
          path: playwright-report/

#Docker Integration

dockerfile

# Dockerfile.playwright
FROM mcr.microsoft.com/playwright:v1.40.0-focal

WORKDIR /app

# Copy package files
COPY package*.json ./

# Install dependencies
RUN npm ci --only=production

# Copy test files
COPY tests/ tests/
COPY playwright.config.js .

# Set environment
ENV CI=true
ENV PLAYWRIGHT_BASE_URL=http://host.docker.internal:3000

# Run tests
CMD ["npx", "playwright", "test"]

#Performance Testing with Playwright

#Core Web Vitals Monitoring

javascript

// tests/performance/core-web-vitals.spec.js
import { test, expect } from '@playwright/test'

test.describe('Core Web Vitals', () => {
  test('measures LCP, FID, and CLS', async ({ page }) => {
    await page.goto('/')

    // Inject Web Vitals library
    await page.addScriptTag({
      url: 'https://unpkg.com/web-vitals@3/dist/web-vitals.iife.js',
    })

    const metrics = await page.evaluate(() => {
      return new Promise(resolve => {
        const vitals = {}

        webVitals.onLCP(metric => {
          vitals.lcp = metric.value
        })

        webVitals.onFID(metric => {
          vitals.fid = metric.value
        })

        webVitals.onCLS(metric => {
          vitals.cls = metric.value
          resolve(vitals)
        })

        // Trigger layout shift measurement
        setTimeout(() => {
          if (!vitals.cls) {
            webVitals.onCLS(
              metric => {
                vitals.cls = metric.value
                resolve(vitals)
              },
              { reportAllChanges: true }
            )
          }
        }, 3000)
      })
    })

    // Assert Core Web Vitals thresholds
    expect(metrics.lcp).toBeLessThan(2500) // Good LCP < 2.5s
    expect(metrics.fid).toBeLessThan(100) // Good FID < 100ms
    expect(metrics.cls).toBeLessThan(0.1) // Good CLS < 0.1

    console.log('Core Web Vitals:', metrics)
  })
})

#Network Performance Testing

javascript

// tests/performance/network.spec.js
import { test, expect } from '@playwright/test'

test.describe('Network Performance', () => {
  test('measures resource loading performance', async ({ page }) => {
    const performanceData = []

    page.on('response', response => {
      performanceData.push({
        url: response.url(),
        status: response.status(),
        timing: response.timing(),
        size: response.headers()['content-length'],
      })
    })

    await page.goto('/')

    // Wait for all network activity to complete
    await page.waitForLoadState('networkidle')

    // Analyze performance
    const slowRequests = performanceData.filter(
      req => req.timing && req.timing.responseEnd - req.timing.requestStart > 1000
    )

    const totalSize = performanceData
      .filter(req => req.size)
      .reduce((sum, req) => sum + parseInt(req.size), 0)

    // Performance assertions
    expect(slowRequests.length).toBeLessThan(3) // Max 2 slow requests
    expect(totalSize).toBeLessThan(2 * 1024 * 1024) // Max 2MB total

    console.log(`Total requests: ${performanceData.length}`)
    console.log(`Slow requests: ${slowRequests.length}`)
    console.log(`Total size: ${(totalSize / 1024).toFixed(2)} KB`)
  })

  test('tests performance under slow network conditions', async ({ page }) => {
    // Simulate slow 3G
    await page.context().route('**/*', (route, request) => {
      // Add artificial delay
      setTimeout(() => route.continue(), 100)
    })

    const startTime = Date.now()
    await page.goto('/')
    await page.waitForSelector('[data-testid="main-content"]')
    const loadTime = Date.now() - startTime

    // Should still load in reasonable time even on slow network
    expect(loadTime).toBeLessThan(8000) // 8 seconds max on slow 3G
  })
})

#Memory and CPU Monitoring

javascript

// tests/performance/memory.spec.js
import { test, expect } from '@playwright/test'

test.describe('Memory Performance', () => {
  test('monitors memory usage during navigation', async ({ page }) => {
    const memoryUsage = []

    // Function to collect memory metrics
    const collectMemory = async () => {
      const metrics = await page.evaluate(() => {
        const memory = performance.memory
        return {
          usedJSHeapSize: memory.usedJSHeapSize,
          totalJSHeapSize: memory.totalJSHeapSize,
          jsHeapSizeLimit: memory.jsHeapSizeLimit,
          timestamp: Date.now(),
        }
      })
      memoryUsage.push(metrics)
    }

    // Initial memory measurement
    await page.goto('/')
    await collectMemory()

    // Navigate through multiple pages
    const routes = ['/products', '/about', '/contact', '/blog']
    for (const route of routes) {
      await page.goto(route)
      await page.waitForLoadState('networkidle')
      await collectMemory()
    }

    // Check for memory leaks
    const initialMemory = memoryUsage[0].usedJSHeapSize
    const finalMemory = memoryUsage[memoryUsage.length - 1].usedJSHeapSize
    const memoryIncrease = finalMemory - initialMemory

    // Memory should not increase by more than 50MB during navigation
    expect(memoryIncrease).toBeLessThan(50 * 1024 * 1024)

    console.log('Memory Usage:', memoryUsage)
  })
})

#Visual Regression Testing

#Screenshot Comparison

javascript

// tests/visual/visual-regression.spec.js
import { test, expect } from '@playwright/test'

test.describe('Visual Regression', () => {
  test.beforeEach(async ({ page }) => {
    // Consistent font loading
    await page.addStyleTag({
      content: `
        * { 
          font-family: -apple-system, BlinkMacSystemFont, sans-serif !important; 
          -webkit-font-smoothing: antialiased !important;
        }
      `,
    })
  })

  test('homepage visual test', async ({ page }) => {
    await page.goto('/')

    // Wait for images and animations to load
    await page.waitForFunction(() => {
      const images = document.querySelectorAll('img')
      return Array.from(images).every(img => img.complete)
    })

    // Hide dynamic content
    await page.addStyleTag({
      content: `
        [data-testid="timestamp"],
        [data-testid="live-chat"],
        .animate-pulse {
          visibility: hidden !important;
        }
      `,
    })

    await expect(page).toHaveScreenshot('homepage.png', {
      fullPage: true,
      threshold: 0.3, // 30% difference threshold
    })
  })

  test('component visual tests', async ({ page }) => {
    await page.goto('/components')

    // Test individual components
    const components = ['button', 'card', 'modal', 'form', 'navigation']

    for (const component of components) {
      const element = page.locator(`[data-testid="${component}-demo"]`)
      await expect(element).toHaveScreenshot(`${component}.png`, {
        threshold: 0.2,
      })
    }
  })

  test('responsive visual tests', async ({ page }) => {
    const viewports = [
      { width: 320, height: 568, name: 'mobile' },
      { width: 768, height: 1024, name: 'tablet' },
      { width: 1440, height: 900, name: 'desktop' },
    ]

    for (const viewport of viewports) {
      await page.setViewportSize(viewport)
      await page.goto('/')

      await expect(page).toHaveScreenshot(`homepage-${viewport.name}.png`, {
        fullPage: true,
      })
    }
  })
})

#Cross-browser Visual Testing

javascript

// playwright.config.js - Visual testing configuration
export default defineConfig({
  projects: [
    {
      name: 'chromium-visual',
      use: {
        ...devices['Desktop Chrome'],
        // Consistent rendering settings
        colorScheme: 'light',
        locale: 'en-US',
        timezoneId: 'America/New_York',
      },
      testDir: './tests/visual',
    },
    {
      name: 'firefox-visual',
      use: {
        ...devices['Desktop Firefox'],
        colorScheme: 'light',
        locale: 'en-US',
        timezoneId: 'America/New_York',
      },
      testDir: './tests/visual',
    },
  ],

  expect: {
    // Global screenshot settings
    threshold: 0.2,
    mode: 'default',
  },
})

#Production Monitoring

#Health Check Automation

javascript

// tests/monitoring/health-checks.spec.js
import { test, expect } from '@playwright/test'

test.describe('Production Health Checks', () => {
  test('API endpoints are responsive', async ({ request }) => {
    const endpoints = ['/api/health', '/api/users', '/api/products', '/api/orders']

    for (const endpoint of endpoints) {
      const response = await request.get(endpoint)
      expect(response.status()).toBe(200)

      const responseTime = response.timing().responseEnd - response.timing().requestStart
      expect(responseTime).toBeLessThan(2000) // 2 second max response time
    }
  })

  test('critical user journeys work', async ({ page }) => {
    // Test critical path: Homepage -> Product -> Checkout
    await page.goto('/')
    await expect(page.locator('h1')).toBeVisible()

    // Navigate to product
    await page.click('[data-testid="featured-product"]')
    await expect(page.locator('[data-testid="product-details"]')).toBeVisible()

    // Add to cart
    await page.click('[data-testid="add-to-cart"]')
    await expect(page.locator('[data-testid="cart-count"]')).toContainText('1')

    // Start checkout
    await page.click('[data-testid="checkout"]')
    await expect(page.locator('[data-testid="checkout-form"]')).toBeVisible()
  })

  test('error pages are properly configured', async ({ page }) => {
    // Test 404 page
    const response = await page.goto('/non-existent-page')
    expect(response.status()).toBe(404)
    await expect(page.locator('h1')).toContainText('Page Not Found')

    // Test 500 error handling
    await page.route('**/api/error-test', route => {
      route.fulfill({ status: 500, body: 'Server Error' })
    })

    await page.goto('/error-test-page')
    await expect(page.locator('[data-testid="error-message"]')).toBeVisible()
  })
})

#Performance Monitoring

javascript

// tests/monitoring/performance-monitoring.spec.js
import { test, expect } from '@playwright/test'

test.describe('Performance Monitoring', () => {
  test('tracks performance metrics over time', async ({ page }) => {
    const metrics = []

    // Monitor multiple pages
    const pages = ['/', '/products', '/about']

    for (const pagePath of pages) {
      const startTime = Date.now()
      await page.goto(pagePath)
      await page.waitForLoadState('networkidle')

      const performanceMetrics = await page.evaluate(() => {
        const navigation = performance.getEntriesByType('navigation')[0]
        const paint = performance.getEntriesByType('paint')

        return {
          domContentLoaded:
            navigation.domContentLoadedEventEnd - navigation.domContentLoadedEventStart,
          loadComplete: navigation.loadEventEnd - navigation.loadEventStart,
          firstPaint: paint.find(p => p.name === 'first-paint')?.startTime || 0,
          firstContentfulPaint:
            paint.find(p => p.name === 'first-contentful-paint')?.startTime || 0,
        }
      })

      metrics.push({
        page: pagePath,
        timestamp: new Date().toISOString(),
        ...performanceMetrics,
        totalTime: Date.now() - startTime,
      })
    }

    // Log metrics for external monitoring systems
    console.log('Performance Metrics:', JSON.stringify(metrics, null, 2))

    // Assert performance thresholds
    metrics.forEach(metric => {
      expect(metric.firstContentfulPaint).toBeLessThan(1500) // FCP < 1.5s
      expect(metric.domContentLoaded).toBeLessThan(2000) // DCL < 2s
    })
  })
})

#Advanced Reporting and Analytics

#Custom Reporter

javascript

// reporters/custom-reporter.js
class CustomReporter {
  onBegin(config, suite) {
    console.log(`Starting test run with ${suite.allTests().length} tests`)
    this.startTime = Date.now()
  }

  onTestEnd(test, result) {
    const duration = result.duration
    const status = result.status

    // Send metrics to external system
    this.sendMetrics({
      testName: test.title,
      duration,
      status,
      project: test.parent.project()?.name,
      retry: result.retry,
    })
  }

  onEnd(result) {
    const duration = Date.now() - this.startTime

    console.log(`Test run completed in ${duration}ms`)
    console.log(`Passed: ${result.passed}, Failed: ${result.failed}`)

    // Send summary to monitoring system
    this.sendSummary({
      totalDuration: duration,
      passed: result.passed,
      failed: result.failed,
      timestamp: new Date().toISOString(),
    })
  }

  sendMetrics(data) {
    // Integration with monitoring systems like DataDog, New Relic, etc.
    if (process.env.DATADOG_API_KEY) {
      // Send to DataDog
    }
  }

  sendSummary(data) {
    // Send summary to Slack, Teams, etc.
    if (process.env.SLACK_WEBHOOK) {
      // Send Slack notification
    }
  }
}

module.exports = CustomReporter

#Integration with Monitoring Services

javascript

// utils/monitoring.js
export class MonitoringIntegration {
  static async sendToDataDog(metrics) {
    if (!process.env.DATADOG_API_KEY) return

    const response = await fetch('https://api.datadoghq.com/api/v1/series', {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
        'DD-API-KEY': process.env.DATADOG_API_KEY,
      },
      body: JSON.stringify({
        series: [
          {
            metric: 'playwright.test.duration',
            points: [[Math.floor(Date.now() / 1000), metrics.duration]],
            tags: [`test:${metrics.testName}`, `status:${metrics.status}`],
          },
        ],
      }),
    })

    return response.json()
  }

  static async sendSlackNotification(summary) {
    if (!process.env.SLACK_WEBHOOK) return

    const color = summary.failed > 0 ? 'danger' : 'good'
    const message = {
      attachments: [
        {
          color,
          title: 'Playwright Test Results',
          fields: [
            { title: 'Passed', value: summary.passed, short: true },
            { title: 'Failed', value: summary.failed, short: true },
            { title: 'Duration', value: `${summary.totalDuration}ms`, short: true },
          ],
        },
      ],
    }

    return fetch(process.env.SLACK_WEBHOOK, {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify(message),
    })
  }
}

#Conclusion

Integrating Playwright into CI/CD pipelines creates a comprehensive quality assurance system that goes beyond simple functional testing. Key benefits include:

#Performance Monitoring:

Core Web Vitals tracking ensures optimal user experience
Network and memory monitoring catches performance regressions
Real-time production monitoring provides early warning systems

#Visual Quality:

Visual regression testing catches UI changes automatically
Cross-browser visual consistency ensures brand integrity
Component-level visual testing enables design system validation

#Production Readiness:

Health check automation verifies system availability
Critical user journey monitoring ensures core functionality
Error handling verification provides confidence in failure scenarios

#DevOps Integration:

Automated reporting provides visibility to stakeholders
Integration with monitoring services enables data-driven decisions
Slack/Teams notifications keep teams informed of quality status

This comprehensive approach transforms Playwright from a testing tool into a quality assurance platform that continuously monitors and validates your application's performance, functionality, and visual integrity across the entire development lifecycle.