Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: add a few more k6 browser examples #1430

Merged
merged 3 commits into from
Nov 30, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,8 @@ Uses the `BrowserContext` to create a new [Page](https://grafana.com/docs/k6/<K6
import { browser } from 'k6/experimental/browser';

export default async function () {
const page = browser.newPage();
const context = browser.newContext();
const page = context.newPage();

try {
await page.goto('https://test.k6.io/browser.php');
Expand Down
72 changes: 72 additions & 0 deletions docs/sources/next/using-k6-browser/metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,3 +98,75 @@ When the test is run, you should see a similar output as the one below.
✓ browser_web_vital_lcp..........................: avg=460.1ms min=460.1ms med=460.1ms max=460.1ms p(90)=460.1ms p(95)=460.1ms
browser_web_vital_ttfb.........................: avg=339.3ms min=258.9ms med=339.3ms max=419.7ms p(90)=403.62ms p(95)=411.66ms
```

## Measure custom metrics

Through k6 browser's `page.evaluate` function, you can call the [Performance API](https://developer.mozilla.org/en-US/docs/Web/API/Performance_API) to measure the performance of web applications. As an example, if you want to measure the time it takes for your users to complete actions, such as a search feature, then you can use the [`performance.mark`](https://developer.mozilla.org/en-US/docs/Web/API/Performance/mark) method to add a timestamp in your browser's performance timeline.

To measure the time difference between two performance markers, you can use the [`performance.measure`](https://developer.mozilla.org/en-US/docs/Web/API/Performance/measure) method.

The time duration that is returned by the performance measure can be added as custom metric in k6 browser using [Trends](https://k6.io/docs/javascript-api/k6-metrics/trend/).
mdcruz marked this conversation as resolved.
Show resolved Hide resolved

{{< code >}}

```javascript
import { browser } from "k6/experimental/browser";
import { Trend } from "k6/metrics";

export const options = {
scenarios: {
ui: {
executor: "shared-iterations",
options: {
browser: {
type: "chromium",
},
},
},
},
};

const myTrend = new Trend('total_action_time');

export default async function () {
const page = browser.newPage();

try {
await page.goto('https://test.k6.io/browser.php');
page.evaluate(() => window.performance.mark('page-visit'));

page.locator('#checkbox1').check();
page.locator('#counter-button"]').click();
page.locator('#text1').fill('This is a test');

page.evaluate(() => window.performance.mark('action-completed'));

// Get time difference between visiting the page and completing the actions
page.evaluate(() =>
window.performance.measure(
'total-action-time',
'page-visit',
'action-completed',
)
);

const totalActionTime = page.evaluate(() =>
JSON.parse(JSON.stringify(window.performance.getEntriesByName('total-action-time')))[0].duration
);

myTrend.add(total_action_time);
} finally {
page.close();
}
}
```

{{< /code >}}

When the test is run, you should see a similar output as the one below.
mdcruz marked this conversation as resolved.
Show resolved Hide resolved

```bash
iteration_duration..........: avg=1.06s min=1.06s med=1.06s max=1.06s p(90)=1.06s p(95)=1.06s
iterations..................: 1 0.70866/s
total_action_time.............: avg=295.3 min=295.3 med=295.3 max=295.3 p(90)=295.3 p(95)=295.3
```
Original file line number Diff line number Diff line change
Expand Up @@ -9,5 +9,6 @@ weight: 100

This section presents some examples and recommended practices when working with the `k6 browser` module to leverage browser automation as part of your k6 tests.

- [Hybrid approach to performance](https://grafana.com/docs/k6/<K6_VERSION>/using-k6-browser/recommended-practices/hybrid-approach-to-performance)
- [Page object model pattern](https://grafana.com/docs/k6/<K6_VERSION>/using-k6-browser/recommended-practices/page-object-model-pattern)
- [Selecting elements](https://grafana.com/docs/k6/<K6_VERSION>/using-k6-browser/recommended-practices/selecting-elements)
Original file line number Diff line number Diff line change
@@ -0,0 +1,197 @@
---
title: 'Hybrid approach to performance'
heading: 'Hybrid performance with k6 browser'
head_title: 'Hybrid performance with k6 browser'
excerpt: 'An example on how to implement a hybrid approach to performance with k6 browser'
weight: 01
---

# Hybrid performance with k6 browser

An alternative approach to [browser-based load testing](https://grafana.com/docs/k6/<K6_VERSION>/testing-guides/load-testing-websites/#browser-based-load-testing) that is much less resource-intensive is combining a low amount of virtual users for a browser test with a high amount of virtual users for a protocol-level test.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is <K6 VERSION> here correct?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I've looked at the existing examples and seems to be correct :)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's correct! It's how we handle versioned links in the Grafana docs, it'll change based on which version of the docs the user is in. So if they're seeing latest, the links on that page will replace <K6_version> with latest.

mdcruz marked this conversation as resolved.
Show resolved Hide resolved

Hybrid performance can be achieved in multiple ways, often using different tools. To simplify the developer experience, k6 browser can be easily combined with core k6 features, so you can easily write hybrid tests in a single script.
mdcruz marked this conversation as resolved.
Show resolved Hide resolved

## Browser and HTTP test

The code below shows an example of combining a browser and HTTP test in a single script. While the backend is exposed to the typical load, the frontend is also checked for any unexpected issues. Thresholds are defined to check both HTTP and browser metrics against pre-defined SLOs.
mdcruz marked this conversation as resolved.
Show resolved Hide resolved

{{< code >}}

```javascript
import http from "k6/http";
import { check } from "k6";
import { browser } from "k6/experimental/browser";

const BASE_URL = __ENV.BASE_URL;

export const options = {
scenarios: {
load: {
exec: 'getPizza',
executor: 'ramping-vus',
stages: [
{ duration: '5s', target: 5 },
{ duration: '10s', target: 5 },
{ duration: '5s', target: 0 },
],
startTime: '10s',
},
browser: {
exec: 'checkFrontend',
executor: 'constant-vus',
vus: 1,
duration: '30s',
options: {
browser: {
type: 'chromium',
},
},
}
},
thresholds: {
http_req_failed: ['rate<0.01'],
http_req_duration: ['p(95)<500', 'p(99)<1000'],
browser_web_vital_fcp: ["p(95) < 1000"],
browser_web_vital_lcp: ["p(95) < 2000"],
},
};

export function getPizza() {
let restrictions = {
maxCaloriesPerSlice: 500,
mustBeVegetarian: false,
excludedIngredients: ['pepperoni'],
excludedTools: ['knife'],
maxNumberOfToppings: 6,
minNumberOfToppings: 2
}

let res = http.post(`${BASE_URL}/api/pizza`, JSON.stringify(restrictions), {
headers: {
'Content-Type': 'application/json',
'X-User-ID': customers[Math.floor(Math.random() * customers.length)],
},
});

check(res, {
'status is 200': (res) => res.status === 200
});
}

export async function checkFrontend() {
const page = browser.newPage();

try {
await page.goto(BASE_URL)
check(page, {
'header': page.locator('h1').textContent() == 'Looking to break out of your pizza routine?',
});

await page.locator('//button[. = "Pizza, Please!"]').click();
page.waitForTimeout(500);
page.screenshot({ path: `screenshots/${__ITER}.png` });

check(page, {
'recommendation': page.locator('div#recommendations').textContent() != '',
});
} finally {
page.close();
}
}

```

{{< /code >}}

## Browser and failure injection test

A browser test can also be run with a failure injection test via the [xk6-disruptor](https://github.com/grafana/xk6-disruptor) extension. This approach lets you find issues in your front end if any services it depends on are suddenly injected with failures, such as delays or server errors.
mdcruz marked this conversation as resolved.
Show resolved Hide resolved

The code below shows an example of introducing faults to a Kubernetes service. While this happens, the `browser` scenario is also executed, which checks the frontend application for unexpected errors that might not have been handled properly.
mdcruz marked this conversation as resolved.
Show resolved Hide resolved

To find out more information about injecting faults to your service, check out our [get started guide with xk6-disruptor](https://grafana.com/docs/k6/<K6_VERSION>/javascript-api/xk6-disruptor/get-started/).
mdcruz marked this conversation as resolved.
Show resolved Hide resolved

{{< code >}}

```javascript
import http from "k6/http";
import { check } from "k6";
import { browser } from "k6/experimental/browser";

const BASE_URL = __ENV.BASE_URL;

export const options = {
scenarios: {
disrupt: {
executor: "shared-iterations",
iterations: 1,
vus: 1,
exec: "disrupt",
},
browser: {
executor: "constant-vus",
vus: 1,
duration: "10s",
startTime: "10s",
exec: "browser",
options: {
browser: {
type: "chromium",
},
},
},
},
thresholds: {
browser_web_vital_fcp: ["p(95) < 1000"],
browser_web_vital_lcp: ["p(95) < 2000"],
},
};

// Add faults to the service by introducing a delay of 1s and 503 errors to 10% of the requests.
const fault = {
averageDelay: "1000ms",
errorRate: 0.1,
errorCode: 503,
}

export function disrupt() {
const disruptor = new ServiceDisruptor("pizza-info", "pizza-ns");
const targets = disruptor.targets();
if (targets.length == 0) {
throw new Error("expected list to have one target");
}

disruptor.injectHTTPFaults(fault, "20s");
}

export async function checkFrontend() {
const page = browser.newPage();

try {
await page.goto(BASE_URL)
check(page, {
'header': page.locator('h1').textContent() == 'Looking to break out of your pizza routine?',
});

await page.locator('//button[. = "Pizza, Please!"]').click();
page.waitForTimeout(500);
page.screenshot({ path: `screenshots/${__ITER}.png` });

check(page, {
'recommendation': page.locator('div#recommendations').textContent() != '',
});
} finally {
page.close();
}
}

```

{{< /code >}}

## Recommended practices

- **Do start small**. Start with a low amount of browser-based virtual users. A good starting point is to have 10% virtual users or less to monitor the user experience for your end-users, while around 90% of traffic should be emulated from the protocol level.
mdcruz marked this conversation as resolved.
Show resolved Hide resolved
- **Combine browser test with different load testing types**. To fully understand the impact of different traffic patterns on your end-user experience, experiment with running your browser test with different [load testing types](https://grafana.com/docs/k6/<K6_VERSION>/testing-guides/test-types/).
- **Cover high-risk user journeys as a start**. Consider identifying the high-risk user journeys first so you can start to monitor the web performance metrics for these user journeys while your backend applications are being exposed to high traffic or service faults.
mdcruz marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: 'Page object model'
heading: 'Page object model with k6 browser'
head_title: 'Page object model with k6 browser'
excerpt: 'An example on how to implement page object model design pattern with k6 browser'
weight: 01
weight: 02
---

# Page object model
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: 'Selecting elements'
excerpt: 'A guide on how to select elements with the browser module.'
weight: 02
weight: 03
---

# Selecting elements
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,8 @@ Uses the `BrowserContext` to create a new [Page](https://grafana.com/docs/k6/<K6
import { browser } from 'k6/experimental/browser';

export default async function () {
const page = browser.newPage();
const context = browser.newContext();
const page = context.newPage();

try {
await page.goto('https://test.k6.io/browser.php');
Expand Down
72 changes: 72 additions & 0 deletions docs/sources/v0.47.x/using-k6-browser/metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,3 +98,75 @@ When the test is run, you should see a similar output as the one below.
✓ browser_web_vital_lcp..........................: avg=460.1ms min=460.1ms med=460.1ms max=460.1ms p(90)=460.1ms p(95)=460.1ms
browser_web_vital_ttfb.........................: avg=339.3ms min=258.9ms med=339.3ms max=419.7ms p(90)=403.62ms p(95)=411.66ms
```

## Measure custom metrics

Through k6 browser's `page.evaluate` function, you can call the [Performance API](https://developer.mozilla.org/en-US/docs/Web/API/Performance_API) to measure the performance of web applications. As an example, if you want to measure the time it takes for your users to complete actions, such as a search feature, then you can use the [`performance.mark`](https://developer.mozilla.org/en-US/docs/Web/API/Performance/mark) method to add a timestamp in your browser's performance timeline.

To measure the time difference between two performance markers, you can use the [`performance.measure`](https://developer.mozilla.org/en-US/docs/Web/API/Performance/measure) method.

The time duration that is returned by the performance measure can be added as custom metric in k6 browser using [Trends](https://k6.io/docs/javascript-api/k6-metrics/trend/).

{{< code >}}

```javascript
import { browser } from "k6/experimental/browser";
import { Trend } from "k6/metrics";

export const options = {
scenarios: {
ui: {
executor: "shared-iterations",
options: {
browser: {
type: "chromium",
},
},
},
},
};

const myTrend = new Trend('total_action_time');

export default async function () {
const page = browser.newPage();

try {
await page.goto('https://test.k6.io/browser.php');
page.evaluate(() => window.performance.mark('page-visit'));

page.locator('#checkbox1').check();
page.locator('#counter-button"]').click();
page.locator('#text1').fill('This is a test');

page.evaluate(() => window.performance.mark('action-completed'));

// Get time difference between visiting the page and completing the actions
page.evaluate(() =>
window.performance.measure(
'total-action-time',
'page-visit',
'action-completed',
)
);

const totalActionTime = page.evaluate(() =>
JSON.parse(JSON.stringify(window.performance.getEntriesByName('total-action-time')))[0].duration
);

myTrend.add(total_action_time);
} finally {
page.close();
}
}
```

{{< /code >}}

When the test is run, you should see a similar output as the one below.

```bash
iteration_duration..........: avg=1.06s min=1.06s med=1.06s max=1.06s p(90)=1.06s p(95)=1.06s
iterations..................: 1 0.70866/s
total_action_time.............: avg=295.3 min=295.3 med=295.3 max=295.3 p(90)=295.3 p(95)=295.3
```
Loading
Loading