Cookie retrieval #3068

kloknibor · 2025-01-02T09:50:30Z

Added support of retrieval of Cookies from webview components

Currently, cookies can only be retrieved with javascript. However, HTTP-only cookies can not be retrieved, and a native implementation with Python is easier.

PR Checklist:

All new features have been tested
All new features have been documented
I have read the CONTRIBUTING.md file
I will abide by the code of conduct

update ios code add the funtctionality for macos as well

kloknibor · 2025-01-02T11:41:37Z

I might need some guidance on how to implement the testbed, since cookies are dynamic. Should I :

just check if there is any result?
Find/create a page with static cookies?

freakboy3742

This is a nice first pass - the high level structure all looks pretty good; I've flagged a couple of minor API design and testing issues inline.

Regarding the testbed: If at all possible, we'd prefer to avoid a dependency on external servers. My suggestion would be to use a javascript blob to programatically set some cookie values, and then retrieve then with the new API.

If that proves prohibitively complex, an alternative approach might be to use a "known reliable" URL to check real world behavior. Historically, we've used github.com for this purpose; the testing risk is that we're not in control of Github's cookie usage, so it will be difficult to construct a test that won't break for reasons outside our control.

One other detail that will come out of adding the testbed test is that the testbed test will fail on Android and GTK - platforms that have a WebView, but won't have an implementation of this API. You'll need to provide a stub implementation on each of those platforms that uses factory.not_implemented().

cocoa/src/toga_cocoa/widgets/webview.py

dummy/src/toga_dummy/widgets/webview.py

core/tests/widgets/test_webview.py

changes/3068.feature.rst

core/src/toga/widgets/webview.py

Co-authored-by: Russell Keith-Magee <[email protected]>

making call more pythonic as suggested

use the ocrrect NSHTTPCookie parameters

freakboy3742

I've flagged a couple of things inline, but they're all either fairly minor naming convention or formatting issues, or structural issues with testing that are a bit harder to explain - so I've pushed an update that includes those fixes.

Thanks for the PR - This will make a nice addition to Toga's API.

freakboy3742 · 2025-01-08T04:02:44Z

android/src/toga_android/widgets/webview.py

+    def cookies(self, on_result=None):
+        # Create the result object
+        result = CookiesResult(on_result=on_result)
+        result.set_result(None)


Good catch on returning a "real" result here; however, to minimise the need for workaround code, it might be helpful to return an empty CookieJar, rather than None. If nothing else, it will remove the need for an | None extension to any type annotations :-)

freakboy3742 · 2025-01-08T04:04:34Z

dummy/src/toga_dummy/widgets/webview.py

+        print("kaas")
+        print(cookies)


Looks like some stray debug.

Suggested change

print("kaas")

print(cookies)

freakboy3742 · 2025-01-08T04:33:05Z

core/src/toga/widgets/webview.py

@@ -135,6 +139,23 @@ def set_content(self, root_url: str, content: str) -> None:
        """
        self._impl.set_content(root_url, content)

+    def cookies(


We're generally deprecating the use of the on_result "callback" version of asynchronous APIs; so this could be turned into an effectively async property.

freakboy3742 · 2025-01-08T04:33:20Z

core/src/toga/widgets/webview.py

+        **This is an asynchronous method**. There is no guarantee that the function
+        has finished evaluating when this method returns. The object returned by this
+        method can be awaited to obtain the value of the expression.
+        An CookieJar object will be returned


Suggested change

An CookieJar object will be returned

Return values should be documented explicitly - see below.

freakboy3742 · 2025-01-08T04:34:30Z

core/src/toga/widgets/webview.py

+
+        **Note:** This is not yet currently supported on Android or Linux.
+
+        :param on_result: A callback function to process the cookies once retrieved.


Suggested change

:param on_result: A callback function to process the cookies once retrieved.

:param on_result: A callback function to process the cookies once retrieved.

:returns: A CookieJar object with all the cookies currently defined on the page.

freakboy3742 · 2025-01-08T04:35:20Z

core/src/toga/widgets/webview.py

+
+        :param on_result: A callback function to process the cookies once retrieved.
+        """
+        return self._impl.cookies(on_result=on_result)


For consistency with other backend APIs, this should remain a get_cookies() method. It's not a public API, it's an internal one.

freakboy3742 · 2025-01-08T04:47:01Z

core/tests/widgets/test_webview.py

+    result = widget.cookies()
+
+    # Get the cookie jar from the future
+    cookie_jar = await result.future  # Await the future to get the CookieJar


You should be able to await result directly - there's no need to dig into the future.

freakboy3742 · 2025-01-08T04:47:24Z

core/tests/widgets/test_webview.py

+    # Get the cookie jar from the future
+    cookie_jar = await result.future  # Await the future to get the CookieJar
+
+    # Validate the cookies in the CookieJar


This should also validate that the result is a cookiejar, not just that it is iterable.

freakboy3742 · 2025-01-08T05:04:09Z

testbed/tests/widgets/test_webview.py

+
+    cookie_jar = await result.future  # Await the future to get the CookieJar
+
+    if toga.platform.current_platform not in {"windows", "iOS", "macOS"}:


We try not to reference specific platforms in the individual tests - instead, we put methods and properties on the test probe that raise a skip if the platform doesn't implement the feature.

freakboy3742 · 2025-01-08T05:38:45Z

testbed/tests/widgets/test_webview.py

+    """Cookies can be retrieved asynchronously."""
+    # A page must be loaded to set cookies
+    await wait_for(
+        widget.load_url("https://example.com/"),


We lean on github.com as a "known URL" because it should be a local lookup when running on GitHub Actions (also - at time of writing, example.com appears to be down for me, which isn't a good sign...)

kloknibor · 2025-01-08T07:05:23Z

Good catches! Thanks for the support both here and at discord, it's very much appreciated :-) This was a fun project

mhsmith · 2025-01-12T16:28:02Z

One of these new tests has just failed on the main branch on macOS.

freakboy3742 · 2025-01-12T23:03:49Z

Yeah - there were multiple failures of this kind in the weekly dependabot updates. I'll need to investigate options for a more robust test.

kloknibor added 6 commits January 2, 2025 10:13

Working cookie retrieval for windows

476366f

Add support for MacOS and iOS

e43f452

update ios code add the funtctionality for macos as well

making uniform with evaluate javascript

c384e2a

Creating tests for cookie retrieval

41a96af

made documentation more extensive

147bec1

added changes file

196c58f

Remove print statement on Windows

17f2841

freakboy3742 requested changes Jan 3, 2025

View reviewed changes

kloknibor and others added 2 commits January 3, 2025 13:38

Update changes/3068.feature.rst

5fe0de7

Co-authored-by: Russell Keith-Magee <[email protected]>

Using Cookiejar as return as suggested

4b67b9a

kloknibor force-pushed the cookie-retrieval branch from 102075b to 4b67b9a Compare January 4, 2025 11:57

fixing tests

9702a73

making call more pythonic as suggested

kloknibor force-pushed the cookie-retrieval branch from e624ede to 9702a73 Compare January 4, 2025 19:19

kloknibor added 4 commits January 4, 2025 20:26

fixing doc linting issue

4c53dc8

fixing testbed test

932c5e8

adding stub implementation for not yet supported platforms

c360cee

testbed fixes

b476de3

use the ocrrect NSHTTPCookie parameters

kloknibor force-pushed the cookie-retrieval branch from f524c0c to b476de3 Compare January 4, 2025 21:57

kloknibor added 2 commits January 7, 2025 08:28

Fix MacOS and iOS tests and cookie session only return

d1778c9

Adding stub implementation unsupported platforms

2b5f5c4

kloknibor requested a review from freakboy3742 January 7, 2025 09:06

freakboy3742 added 2 commits January 8, 2025 12:47

Merge branch 'main' into cookie-retrieval

4e47888

Minor cleanups and consistency fixes.

248b6e2

freakboy3742 approved these changes Jan 8, 2025

View reviewed changes

freakboy3742 merged commit 738f444 into beeware:main Jan 8, 2025
41 checks passed

freakboy3742 mentioned this pull request Jan 13, 2025

Add a retry for setting the cookie in webview tests. #3103

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cookie retrieval #3068

Cookie retrieval #3068

kloknibor commented Jan 2, 2025

kloknibor commented Jan 2, 2025

freakboy3742 left a comment

freakboy3742 left a comment

freakboy3742 Jan 8, 2025

freakboy3742 Jan 8, 2025

freakboy3742 Jan 8, 2025

freakboy3742 Jan 8, 2025

freakboy3742 Jan 8, 2025

freakboy3742 Jan 8, 2025

freakboy3742 Jan 8, 2025

freakboy3742 Jan 8, 2025

freakboy3742 Jan 8, 2025

freakboy3742 Jan 8, 2025

kloknibor commented Jan 8, 2025 •

edited

Loading

mhsmith commented Jan 12, 2025

freakboy3742 commented Jan 12, 2025


		Note: This is not yet currently supported on Android or Linux.

		:param on_result: A callback function to process the cookies once retrieved.

	:param on_result: A callback function to process the cookies once retrieved.
	:param on_result: A callback function to process the cookies once retrieved.
	:returns: A CookieJar object with all the cookies currently defined on the page.


		cookie_jar = await result.future # Await the future to get the CookieJar

		if toga.platform.current_platform not in {"windows", "iOS", "macOS"}:

Cookie retrieval #3068

Cookie retrieval #3068

Conversation

kloknibor commented Jan 2, 2025

PR Checklist:

kloknibor commented Jan 2, 2025

freakboy3742 left a comment

Choose a reason for hiding this comment

freakboy3742 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kloknibor commented Jan 8, 2025 • edited Loading

mhsmith commented Jan 12, 2025

freakboy3742 commented Jan 12, 2025

kloknibor commented Jan 8, 2025 •

edited

Loading