Chrome browser disconnect and execution restarts

[TsvetanTsvetkovClevertap]

What is your Scenario?

There is a corner case issue happening due to the combination:

• TestCafe (tested latest two versions 3.7.0, 3.7.1)
• Chrome engine - reproducible with: Chrome (131, 132), Chromium - tested several versions including a version from July 2024
• Our application -> Clevertap Dashboard

The following is observed when the automation script reaches the publish campaign stage (the root cause is not clear, for sure it happens a few steps before Publish, but it is most obvious in the UI after Publish):

1/ Chrome browser "freezes". It looks the same as if JS is disabled from the browser dev-tool options. This is visible in the UI because the Publish confirmation popup "Campaign Saved successfully" does not disappear (it should disappear after a couple of seconds)

There are no console errors in the Chrome instance pointing to anything unusual

Network calls (ajax calls from our app) keep coming through.

2/ DEBUG=testcafe:* -> after 3 min, we see "browser disconnect +3m" in the console
There is no other error or anything suspicious in the debug logs.

3/ TestCafe restarts the execution all over from the beginning. Please note quarantine mode is not enabled

4/ steps 1 to 3 are repeated 3 times until finally the executions stops giving the very generic Browser disconnect error

NOTE 1:
Issue is not OS related. We are reproducing it on Ubuntu and MacOS.

NOTE 2:
When we change the browser to Firefox or Safari - the exact same test script passes successfully without any issues.

What is the Current behavior?

Chrome browser disconnects, execution is restarted when it should not

What is the Expected behavior?

Test passes on Chrome like it does on Firefox.

What is the public URL of the test page? (attach your complete example)

N/A cannot provide due to security.

What is your TestCafe test code?

N/A cannot provide due to security.

Your complete configuration file

No response

Your complete test report

No response

Screenshots

Sending a video of the execution

TestCafe_Chrome_Disconnect.mp4

Steps to Reproduce

As per the video

TestCafe version

3.7.0, 3.7.1

Node.js version

20.10

Command-line arguments

n/a

Browser name(s) and version(s)

Chrome 131, Chrome 132, Chromium 134.0.6970.0, Chromium 113.0.5637.0

Platform(s) and version(s)

Mac OS 15.2, Ubuntu 20.04

Other

We totally understanding that it is impossible for you to debug with the provided information.

We are hoping to organise a debug session over zoom or another tool.

We are OK also to install some "debug" testcafe version that could give more insights as to what is happening.

Although there is a workaround with Firefox, this is not a desirable long term solution for us, because there are other problems arising from using Firefox.

Thank you in advance!

[MirKml]

maybe you can test it without native automation see - https://testcafe.io/documentation/404237/guides/intermediate-guides/native-automation-mode

You mentioned that in Firefox, Safari your tests work. And for these browsers no native automation is needed. For Chrome it's used by default.
So you can try same test runner settings for Chrome, maybe it will work.

[TsvetanTsvetkovClevertap]

Thank you @MirKml
The issue is reproducible with the proxy (disabled native automation) as well.

[deepanshugoyal1989]

I am also experiencing this issue. TestCafe hangs when clicking "Confirm" on a pop-up window. After that, the same test is restarted and hangs again, even though quarantine mode is disabled. After three attempts, it says 'The Chrome 132.0.0.0 / Ubuntu 22.04 browser disconnected.'

I have tried several options (one at a time), including:

disableNativeAutomation
disableMultipleWindows
Adjusting timeouts
Setting setNativeDialogHandler
However, none of these seem to resolve the issue. It appears that TestCafe loses the browser context when the confirmation window closes and continues waiting for a response from the now-closed window.

[Suchiraz]

I am also seeing the same issue. Tests are hanging in the pipeline
Tried running tests on older version of chrome 129 but that does not seem to solve the problem
Tried the following:
Adding explicit wait
Increasing/Decreasing timeouts
disableMultipleWindows
disableNativeAutomation

[Bayheck]

Hello,

Thank you for submitting a bug report.

We are hoping to organise a debug session over zoom or another tool.

We do not offer such services. Please share a simple code example that reliably reproduces the bug. Refer to the following help topic for more information: How To Create a Minimal Working Example When You Submit an Issue.

We look forward to your response.

[TsvetanTsvetkovClevertap]

Hello @Bayheck,

here are the reproduction instructions:

ACCESS
I created a TEST project and i have to sent an invitation to grant you access to this project.
Obviously this must not be public so I sent an email to [email protected] to arrange this

After this is arranged whoever is going to debug the issue have to
Accept the email invitation
follow the registration instructions -> setup a secure password and authenticator app

SETUP
The attached zip contains 3 files:
ct.config.js - configuration file
ct.debug.test.js - the test code which reproduces the issue
ct.runner.js - runner file which triggers TestCafe

1. Unzip the archive in the root of your project where package.json is located
2. Add the following script to your package.json
   "clevertap": "NODE_OPTIONS='--max-old-space-size=4096' node ct.runner.js",
3. Add the following devDependencies to the package.json
   "testcafe-reporter-allure-plus": "1.2.1"
"luxon": "1.21.2"
4. install and use node 20.10
5. install yarn
6. run yarn install (to install the above dependencies)

REPRODUCTION
1/ run the script clevertap with yarn
2/ the execution will stop on debug on the login screen, login manually and then hit RESUME in the debugger
3/ wait until the execution freezes after publishing the campaign and then it restarts => same as on the attached video in this ticket

IMPORTANT: Please note that I have observed 1 in every few executions (lets say 5-6) is successful and does not reproduce the problem. If this happens please rerun the script.

ct.chrome.disconnect.reproduction.zip

[aleks-pro]

Hello @TsvetanTsvetkovClevertap ,

Thank you for sharing the reproduction instructions. We appreciate your detailed report.

Unfortunately, we cannot debug examples that include sensitive data. However, we will be happy to research an example that requires no authentication. If possible, please create an example that does not depend on password-protected resources. We will do our best to find a solution.

[TsvetanTsvetkovClevertap]

Hello @aleks-pro

I beg you to reconsider 🙏

It is impossible for us to provide reproduction example without login.

You just need to do the login step manually after completing a simple 2 min registration process - 1 email address is needed and we will provide access to the TEST account

The attached zip with test code, configuration and runner files does not include any sensitive information.

This is a serious issue and blocker for us!

2 other people are reporting the same in this very thread.

I really beg you for help!

Best Regards,
Tsvetan

[TsvetanTsvetkovClevertap]

Hello @aleks-pro @Bayheck,

We managed to isolate and create a very simple example that reproduces the issue
both on Firefox and Chrome.

The only code needed is

await t
.navigateTo('https://tsvetantsvetkovclevertap.github.io/TestCafeBrowserDisconnect/main-app-encrypted-5-mb.js')
.expect(Selector('body').innerText).contains('undefined')
`

see also video of the reproduction with the above code

Please note I have cut parts of the video where waiting for the timeout to be hit, to make it shorter and smaller size.

https://tsvetantsvetkovclevertap.github.io/TestCafeBrowserDisconnect/BrowserDisconnect-Firefox.mp4

Best Regards,
Tsvetan

[github-actions]

We appreciate you taking the time to share information about this issue. We reproduced the bug and added this ticket to our internal task queue. We'll update this thread once we have news.

[TsvetanTsvetkovClevertap]

Hello,

just checking, when is a fix expected to enter a release?
Thanks!

Best Regards,
Tsvetan

[Bayheck]

At present, we cannot give you an exact ETA for fixes, as predicting timelines can be challenging and may vary.

[Bayheck]

Hello, I researched your example issue and managed to determine its root cause.
You see, your page (https://tsvetantsvetkovclevertap.github.io/TestCafeBrowserDisconnect/main-app-encrypted-5-mb.js) has a MIME type of application/javascript. It is a direct link to a .js file rather than a regular html page. Testcafe can not interact with such pages.
Please clarify if the page that you are trying to access also has the MIME type of application/javascript?.

[TsvetanTsvetkovClevertap]

Hi @Bayheck

I specifically created this "page" via github pages,
https://tsvetantsvetkovclevertap.github.io/TestCafeBrowserDisconnect/main-app-encrypted-5-mb.js

it is a web server which reads a public repo
Everything is built in and I do not control the headers.

This reproduces 100%, all the time, the same behaviour of the issue we are experiencing while testing our application.

I would also include my colleague Saurabh Agarwal to try to help you in this debug / investigation 🙏
Thank you for your support!

[Suchiraz]

We are also seeing same issue when toast pop up messages appear and tests hang. These were old tests and worked perfectly in chrome but suddenly stopped working from Jan this year. Firefox it works OK

[Bayheck]

Hello, I understand that the https://tsvetantsvetkovclevertap.github.io/TestCafeBrowserDisconnect/main-app-encrypted-5-mb.js was created just as an example of the issue.

Please clarify if the page that you are trying to access also has the MIME type of application/javascript?.

What I was asking in my previous comment was to check the MIME type of pages in your actual application. Please check and let us know your results.

Other than that, there is nothing to debug for me at present. As I have mentioned earlier, your example page has a MIME type of application/javascript. TestCafe simply does not interact with such pages because the testing page could only be an HTML page. This behavior is by design.