Testing a website that is a PWA hangs when using Role or reload

[eroetman]

What is your Scenario?

We want to test our product that also supports PWA (Progressive WebApp).

What is the Current behavior?

When testing with Testcafe 3.5.0 in native mode tests hang when using a Role or reloading during testing.

What is the Expected behavior?

We should be able to test a PWA just like any other website with a service worker.

What is the public URL of the test page? (attach your complete example)

https://demo.tigrenpwa.com/
This is not our site but the behavior is the same for the demonstartion of the issues.

What is your TestCafe test code?

import { Selector, Role } from "testcafe";

const url = "https://demo.tigrenpwa.com/";
const element = Selector("button").withText("EXPLORE DEMOS");

fixture`Test a PWA Demo site`.page(url).before(async(ctx)=>{
  // Allow notifications
  const CDP = require("chrome-remote-interface");
  const cdpClient = await CDP();
  await cdpClient.Browser.grantPermissions({
    permissions: ["notifications"],
    origin: url,
  });
  console.log("Notifications granted (using CDP).");
});

const role = Role(url, async (t) => {
  console.log("role initialized");
  return true;
});

test("Open PWA then refresh page", async (t) => {
  console.log("opened");
  await t.expect(element.exists).ok();
  console.log("refresh");
  await t.eval(() => location.reload());
  console.log("after refresh");
  await t.expect(element.exists).ok();
});

test("Open PWA with Role", async (t) => {
  console.log("opened");
  await t.expect(element.exists).ok();
  await t.useRole(role);
  console.log("with role");
  await t.expect(element.exists).ok();
});

Your complete configuration file

{
  "assertionTimeout": 2000,
  "debugMode": false,
  "disableNativeAutomation": false,
  "disablePageCaching": true,
  "inspectBrk": false,
  "pageLoadTimeout": 2000,
  "selectorTimeout": 2000,
  "skipJsErrors": false,
  "skipUncaughtErrors": false,
  "speed": 1.0,
  "screenshots": {
    "path": "reports/screenshots",
    "takeOnFails": true
  },
  "reporter": [
    {
      "name": "spec"
    },
    {
      "name": "html",
      "output": "reports/report.html"
    }
  ],
}

Your complete test report

 Running tests in:
 - Chrome 120.0.0.0 / Sonoma 14

 Test a PWA Demo site
Notifications granted (using CDP).
opened
refresh
after refresh
 ✖ Open PWA then refresh page (screenshots:
 /Users/workspace/demo/reports/screenshots/2024-01-18_17-55-50/test-1/Chrome_120.0.0.0_Sonoma_14/errors/1.png)

   1) AssertionError: expected false to be truthy

      Browser: Chrome 120.0.0.0 / Sonoma 14
      Screenshot:

   /Users/workspace/demo/reports/screenshots/2024-01-18_17-55-50/test-1/Chrome_120.0.0.0_Sonoma_14/errors/1.png

         23 |  console.log("opened");
         24 |  await t.expect(element.exists).ok();
         25 |  console.log("refresh");
         26 |  await t.eval(() => location.reload());
         27 |  console.log("after refresh");
       > 28 |  await t.expect(element.exists).ok();
         29 |});
         30 |
         31 |test("Open PWA with Role", async (t) => {
         32 |  console.log("opened");
         33 |  await t.expect(element.exists).ok();

         at <anonymous> (/Users/workspace/demo/testcafe-tigren.ts:28:34)
         at fulfilled (/Users/workspace/demo/testcafe-tigren.ts:5:58)


opened
role initialized
with role
 ✖ Open PWA with Role (screenshots:
 /Users//workspace/demo/reports/screenshots/2024-01-18_17-55-50/test-2/Chrome_120.0.0.0_Sonoma_14/errors/1.png)

   1) AssertionError: expected false to be truthy

      Browser: Chrome 120.0.0.0 / Sonoma 14
      Screenshot:

   /Users/workspace/demo/reports/screenshots/2024-01-18_17-55-50/test-2/Chrome_120.0.0.0_Sonoma_14/errors/1.png

         31 |test("Open PWA with Role", async (t) => {
         32 |  console.log("opened");
         33 |  await t.expect(element.exists).ok();
         34 |  await t.useRole(role);
         35 |  console.log("with role");
       > 36 |  await t.expect(element.exists).ok();
         37 |});
         38 |

         at <anonymous> (/Users/workspace/demo/testcafe-tigren.ts:36:34)
         at fulfilled (/Users/workspace/demo/testcafe-tigren.ts:5:58)



 2/2 failed (2m 29s)

Screenshots

No response

Steps to Reproduce

1.Run the test suppplied with the issue. Possibly this is flalky so if it succeeds please run again.
2.
3.

TestCafe version

3.5.0

Node.js version

v20.11.0

Command-line arguments

testcafe chrome:emulation:cdpPort=9222

Browser name(s) and version(s)

120.0.6099.234

Platform(s) and version(s)

macos 14.2.1

Other

No response

[github-actions]

We appreciate you taking the time to share information about this issue. We reproduced the bug and added this ticket to our internal task queue. We'll update this thread once we have news.

[PavelMor25]

Hello @eroetman,

We've investigated this usage scenario and identified the cause of this behavior. TestCafe executes tests on the page by intercepting responses with content-type: Document and injecting all necessary scripts into the document. TestCafe achieves this using the Chrome DevTools Protocol (CDP).

Unfortunately, CDP methods do not allow intercepting and handling requests/responses that have already been processed by a service worker. As a result, after a reload or role initialization, TestCafe cannot inject the necessary scripts because the required type of request has been processed by your service worker - sw.js.

Currently, as a workaround, we suggest disabling the interception of requests by the service worker using CDP. Add this code to the the Fixture.beforeEach Method:

.beforeEach( async t => {
  const idConnection = t.testRun.browserConnection.id;
  const cdpClient    = t.testRun.browserConnection.getNativeAutomation(idConnection)._client;
  await cdpClient.Network.setBypassServiceWorker({bypass: true}) // Toggles ignoring of service worker for each request.
})

Please let us know your results.

[eroetman]

This fixed our issue, thanks!

[PavelMor25]

Since this workaround has helped you, we will close the issue. If you need further assistance, please let us know.