Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flatpak package and Flathub publication #86

Open
k8ieone opened this issue Jul 24, 2024 · 42 comments
Open

Flatpak package and Flathub publication #86

k8ieone opened this issue Jul 24, 2024 · 42 comments
Labels
help wanted Extra attention is needed tangent-electron Issues relating to the Tangent Application itself. User Request Features & Enhancements requested by users.

Comments

@k8ieone
Copy link
Contributor

k8ieone commented Jul 24, 2024

Really liking the app so far, it would be nice if the project was available as a Flatpak.

@taylorhadden taylorhadden added help wanted Extra attention is needed tangent-electron Issues relating to the Tangent Application itself. User Request Features & Enhancements requested by users. labels Jul 25, 2024
@taylorhadden
Copy link
Contributor

Sounds like a good idea to me! I'm not really a linux user myself. Is this something you're knowledgeable on / would mind helping out with?

@k8ieone
Copy link
Contributor Author

k8ieone commented Jul 26, 2024

No, infortunately I haven't done anything like this before. But I'll try looking into it, my guess is that it shouldn't be too difficult.

@k8ieone
Copy link
Contributor Author

k8ieone commented Jul 27, 2024

Alright, I did some initial research and I'm trying to throw a manifest together. I have a question tho. Would you like to publish the finished application on Flathub yourself?

Flathub's policy is that users are allowed to submit applications that they're not the developer of. https://docs.flathub.org/docs/for-app-authors/submission#theres-an-app-that-id-like-to-see-on-flathub-but-im-not-the-developer

But it seems that having the developer publish the application is preferred. https://docs.flathub.org/docs/for-app-authors/submission#someone-else-has-put-my-app-on-flathubwhat-do-i-do

This is still in the future, the manifest is nowhere near ready yet. I just wanted to shoot this question out beforehand.

I'm hoping this might not even matter because once a repository in https://github.com/flathub has been created the publisher gets write permissions and I hope more people can be invited to also have write access. This way we could both manage the application on Flathub. https://docs.flathub.org/docs/for-app-authors/submission/#after-submission

@taylorhadden
Copy link
Contributor

I have no strong preferences about being the primary publisher. If you or anybody else want to help out here, I'd be quite pleased! That's why Tangent is open source now.

@k8ieone
Copy link
Contributor Author

k8ieone commented Jul 28, 2024

I might have spoken too soon 😅

Tangent being an Electron app using npm for builds complicates things because Flathub doesn't allow network access from the sandbox where the application is built. The workaround for this is to pull the dependencies before the build process starts and then use npm in offline mode for the build. They have tooling and a guide for this, but the Typewriter fork being a local module seems to break their script for dependency fetching. I'm afraid I'm not enough of an npm guru for this 😅

In case anyone wants to pick this up, feel free to do so.

I'll post an update if I manage to get any further, but my hopes aren't so high anymore 😔

This is what I have so far:

id: io.github.suchnsuch.Tangent
runtime: org.freedesktop.Platform
runtime-version: '23.08'
sdk: org.freedesktop.Sdk
base: org.electronjs.Electron2.BaseApp
base-version: '23.08'
sdk-extensions:
  - org.freedesktop.Sdk.Extension.node18
# Electron doesn't use a traditional locale format
separate-locales: false
writable-sdk: true
command: run.sh
finish-args:
  - --device=dri
  - --share=ipc
  - --socket=x11
  - --socket=pulseaudio
  # Probably not needed
  #- --share=network
  - --filesystem=home
modules:
  - name: tangent
    buildsystem: simple
    build-options:
      append-path: /usr/lib/sdk/node18/bin
      env:
        NPM_CONFIG_LOGLEVEL: info
        XDG_CACHE_HOME: /run/build/tangent/flatpak-node/cache
        npm_config_cache: /run/build/tangent/flatpak-node/npm-cache
        npm_config_nodedir: /usr/lib/sdk/node18
        npm_config_offline: 'true'
    subdir: main
    sources:
      - type: git
        url: https://github.com/suchnsuch/Tangent
        tag: tangent-v0.7.6
      - type: script
        dest-filename: run.sh
        commands:
          - zypak-wrapper.sh /app/main/electron-sample-app "$@"
      #- type: file
      #  path: io.github.suchnsuch.Tangent.desktop
      #- type: file
      #  path: io.github.suchnsuch.Tangent.svg
      #- type: file
      #  path: io.github.suchnsuch.Tangent.metainfo.xml
    build-commands:
      # build linux
      - npm install --offline
      # install
      #- install -Dm644 -t /app/share/icons/hicolor/scalable/apps/ io.github.suchnsuch.Tangent.svg
      #- install -Dm644 -t /app/share/applications io.github.suchnsuch.Tangent.desktop
      #- install -Dm644 -t /app/share/metainfo io.github.suchnsuch.Tangent.metainfo.xml
      #- install -Dm755 -t /app/bin/ run.sh

@taylorhadden
Copy link
Contributor

Does recursive mode help at all? What kind of errors are you getting?

@k8ieone
Copy link
Contributor Author

k8ieone commented Jul 31, 2024

Initially I had an issue with the Typewriter fork, but that was my fault. I didn't realize it was a submodule and that it isn't cloned by default.

Now I cloned the Tangent repo at version 0.7.6 including the submodules (git submodule update --init --recursive) and ran flatpak-node-generator npm package-lock.json.

I'm getting

Reading packages from lockfiles...
Traceback (most recent call last):
  File "/home/k8ie/.local/bin/flatpak-node-generator", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/k8ie/.local/share/pipx/venvs/flatpak-node-generator/lib/python3.12/site-packages/flatpak_node_generator/main.py", line 277, in main
    asyncio.run(_async_main())
  File "/usr/lib/python3.12/asyncio/runners.py", line 194, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/asyncio/base_events.py", line 687, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/k8ie/.local/share/pipx/venvs/flatpak-node-generator/lib/python3.12/site-packages/flatpak_node_generator/main.py", line 198, in _async_main
    packages.update(lockfile_provider.process_lockfile(lockfile))
  File "/home/k8ie/.local/share/pipx/venvs/flatpak-node-generator/lib/python3.12/site-packages/flatpak_node_generator/providers/npm.py", line 166, in process_lockfile
    yield from self._process_packages_v2(lockfile, data)
  File "/home/k8ie/.local/share/pipx/venvs/flatpak-node-generator/lib/python3.12/site-packages/flatpak_node_generator/providers/npm.py", line 138, in _process_packages_v2
    raise NotImplementedError(
NotImplementedError: Don't know how to handle package packages/tangent-test-workspace-generator in package-lock.json

@k8ieone
Copy link
Contributor Author

k8ieone commented Jul 31, 2024

I think this could be a forgotten package in the package-lock.json file. I removed the section mentioning packages/tangent-test-workspace-generator and got a bit further now:

Reading packages from lockfiles...
1070 packages read.
Generating packages [1069/1070] app-builder-bin @ 4.0.0                                                                                                                                                                                       
Traceback (most recent call last):
  File "/home/k8ie/.local/bin/flatpak-node-generator", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/k8ie/.local/share/pipx/venvs/flatpak-node-generator/lib/python3.12/site-packages/flatpak_node_generator/main.py", line 277, in main
    asyncio.run(_async_main())
  File "/usr/lib/python3.12/asyncio/runners.py", line 194, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/asyncio/base_events.py", line 687, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/k8ie/.local/share/pipx/venvs/flatpak-node-generator/lib/python3.12/site-packages/flatpak_node_generator/main.py", line 229, in _async_main
    await progress.run()
  File "/home/k8ie/.local/share/pipx/venvs/flatpak-node-generator/lib/python3.12/site-packages/flatpak_node_generator/progress.py", line 78, in run
    await coro
  File "/usr/lib/python3.12/asyncio/tasks.py", line 631, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "/home/k8ie/.local/share/pipx/venvs/flatpak-node-generator/lib/python3.12/site-packages/flatpak_node_generator/progress.py", line 68, in _generate
    await self.module_provider.generate_package(package)
  File "/home/k8ie/.local/share/pipx/venvs/flatpak-node-generator/lib/python3.12/site-packages/flatpak_node_generator/providers/npm.py", line 365, in generate_package
    await self.special_source_provider.generate_special_sources(package)
  File "/home/k8ie/.local/share/pipx/venvs/flatpak-node-generator/lib/python3.12/site-packages/flatpak_node_generator/providers/special.py", line 482, in generate_special_sources
    await self._handle_playwright(package)
  File "/home/k8ie/.local/share/pipx/venvs/flatpak-node-generator/lib/python3.12/site-packages/flatpak_node_generator/providers/special.py", line 373, in _handle_playwright
    metadata = await RemoteUrlMetadata.get(dl_url, cachable=True)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/k8ie/.local/share/pipx/venvs/flatpak-node-generator/lib/python3.12/site-packages/flatpak_node_generator/url_metadata.py", line 41, in get
    async for part in Requests.instance.read_parts(url, cachable=False):
  File "/home/k8ie/.local/share/pipx/venvs/flatpak-node-generator/lib/python3.12/site-packages/flatpak_node_generator/requests.py", line 57, in read_parts
    async for part in self._read_parts(url, size):
  File "/home/k8ie/.local/share/pipx/venvs/flatpak-node-generator/lib/python3.12/site-packages/flatpak_node_generator/requests.py", line 30, in _read_parts
    async with self._open_stream(url) as stream:
  File "/usr/lib/python3.12/contextlib.py", line 210, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/k8ie/.local/share/pipx/venvs/flatpak-node-generator/lib/python3.12/site-packages/flatpak_node_generator/requests.py", line 24, in _open_stream
    async with session.get(url) as response:
  File "/home/k8ie/.local/share/pipx/venvs/flatpak-node-generator/lib/python3.12/site-packages/aiohttp/client.py", line 1197, in __aenter__
    self._resp = await self._coro
                 ^^^^^^^^^^^^^^^^
  File "/home/k8ie/.local/share/pipx/venvs/flatpak-node-generator/lib/python3.12/site-packages/aiohttp/client.py", line 696, in _request
    resp.raise_for_status()
  File "/home/k8ie/.local/share/pipx/venvs/flatpak-node-generator/lib/python3.12/site-packages/aiohttp/client_reqrep.py", line 1070, in raise_for_status
    raise ClientResponseError(
aiohttp.client_exceptions.ClientResponseError: 404, message='The specified blob does not exist.', url=URL('https://playwright.azureedge.net/builds/firefox/1440/firefox-ubuntu-18.04.zip')

@k8ieone
Copy link
Contributor Author

k8ieone commented Jul 31, 2024

I tried looking around, but I don't know where that URL comes from.

@taylorhadden
Copy link
Contributor

That looks like it's trying to pull binaries for the development dependencies. Playwright is used for automated testing.

tangent-test-workspace-generator is just a different tool in the mono-repo. Hopefully you should be able to ignore it?

After doing some poking around, you might be hitting this issue?

@k8ieone
Copy link
Contributor Author

k8ieone commented Aug 5, 2024

Yes, that seems to be it exactly. Nice find.

Regarding tangent-test-workspace-generator, I think it was moved at some point to a different directory but not updated in the package-lock.json file. If you take a look at that error message, it was looking for it in packages/tangent-test-workspace-generator, but in the repo it's located in the apps folder.

@taylorhadden
Copy link
Contributor

Ah! I should probably do a package update to refresh package-lock.json.

@k8ieone
Copy link
Contributor Author

k8ieone commented Aug 5, 2024

If you do so, could you please backport it to the tangent-v0.7.6 tag? Or perhaps create a new one for that like tangent-v0.7.7.

@taylorhadden
Copy link
Contributor

Hmm.. would it be kosher to move ahead with the v0.8.x version as a test case and release on Flathub when v0.8 is ready?

@k8ieone
Copy link
Contributor Author

k8ieone commented Aug 6, 2024

They only have a policy that says no unstable releases, so I'd say yes. We can work with 0.8.x and release the package along with 0.8.

@k8ieone
Copy link
Contributor Author

k8ieone commented Aug 24, 2024

I was kinda hoping they would have fixed that error in flatpak-node-generator by now, but they didn't. So I'm running a modified copy that includes the fix.

After manually removing the missing packages/tangent-test-workspace-generator package, the command passed and generated the file needed for the Flatpak builder.

@k8ieone
Copy link
Contributor Author

k8ieone commented Aug 24, 2024

Alright, I've run into another thing.

Every time I try to build the Flatpak, I get to bwrap: Can't chdir to /run/build/tangent/main: No such file or directory after checking out the submodule.

I tried building https://github.com/flathub/electron-sample-app as well, but I get the same result.

I've also tried to build the Flatpak in a privileged Docker container to make sure this isn't an issue specific to Arch or my system install.

The only related issue I was able to find is flathub/com.visualstudio.code-oss#78, but that's pretty old and was solved by using a newer flatpak-builder version. I'm already using the latest one. I've also tried using the flatpak-builder Flatpak instead of the Arch package, but the result is the same.

@k8ieone
Copy link
Contributor Author

k8ieone commented Aug 24, 2024

Nevermind, that issue was caused by the subdir: main statement in the Flatpak manifest. Removing it gets me a little further.

@k8ieone
Copy link
Contributor Author

k8ieone commented Aug 24, 2024

I have great news, I was finally able to build and run Tangent in a Flatpak!

Now it's just a matter of going through the Flathub guidelines and making sure it passes them. I'll post a link to the Flatpak packaging repo here once it's at least somewhat ready.

@k8ieone
Copy link
Contributor Author

k8ieone commented Aug 25, 2024

I was also able to test Tangent on a mobile phone. I couldn't before because it runs Alpine and Alpine doesn't exactly see eye-to-eye with AppImages. Likely because it uses musl.

Now with the Flatpak it works flawlessly. Tangent even scales down pretty nicely. The note takes up a lot of horizontal space, forcing the user to scroll. Works great in landscape mode tho. Just thought I'd share.

@taylorhadden
Copy link
Contributor

This is awesome news! Thank you for all your hard work!

Using it with a phone is pretty interesting. Try turning down the "min panel width" in the settings. That should help!

@k8ieone
Copy link
Contributor Author

k8ieone commented Aug 27, 2024

I did have to make some changes in this repo as well, I'll open a PR for those.

@k8ieone
Copy link
Contributor Author

k8ieone commented Aug 27, 2024

@taylorhadden Here are the Flatpak sources in their current state, I recommend looking at the Metainfo file, it contains most of the metadata that will be visible on Flathub and in stores.

I wasn't able to find a .desktop file in the main repo, so I made one as well.

The .desktop file and Metainfo are the two main things where I could have made a mistake, I encourage you to check both and let me know it you don't like something or if you'd like something added.

https://github.com/k8ieone/flathub/tree/new-pr

@k8ieone
Copy link
Contributor Author

k8ieone commented Aug 27, 2024

Here's a preview of how the metadata looks when rendered.

image

image

@taylorhadden
Copy link
Contributor

This is super cool! I'll take a look at the description.

@k8ieone
Copy link
Contributor Author

k8ieone commented Aug 31, 2024

@taylorhadden I have one issue to report and that's that the spell-checking doesn't seem to work.

Does Tangent use some kind of system-wide dictionary that could be missing in the Flatpak? I know some programs use the files in /usr/share/dict for example.

@taylorhadden
Copy link
Contributor

Huh, that's odd. It just uses the default Chromium spellcheck system.

@k8ieone
Copy link
Contributor Author

k8ieone commented Sep 23, 2024

Alright, it turns out the spell-checking system automatically downloads the language files needed. This means it needs internet access.

When I initially put the manifest together I disabled networking because I thought an editor probably doesn't need it. Well, I was wrong 😅

I'll update the manifest to include network access. After doing this spell-checking works again.

@taylorhadden
Copy link
Contributor

Hmm, we probably also want some flag to disable the built-in auto-update behavior, right? Are you seeing any issues with that?

@k8ieone
Copy link
Contributor Author

k8ieone commented Sep 26, 2024

I thought that this behavior was controlled by the APPIMAGE env variable.

I see this during startup:

APPIMAGE env is not defined, current application is not an AppImage

I'm not sure how the app behaves when it tries to update itself. I haven't seen that happen yet.

But yes, it would make sense to disable auto-updates when running in Flatpak.

@k8ieone
Copy link
Contributor Author

k8ieone commented Oct 22, 2024

@taylorhadden Do you have a plan for the next stable release? If you have a date in mind, keep me posted please. It's possible some new issue may pop up with the new release and I'd like to be ready for it.

@k8ieone
Copy link
Contributor Author

k8ieone commented Oct 22, 2024

Nevermind, I just checked the releases page 😄

I'll try to get to finishing up the publication as soon as I can.

@k8ieone
Copy link
Contributor Author

k8ieone commented Oct 31, 2024

Flathub PR open! ⬆️

@k8ieone
Copy link
Contributor Author

k8ieone commented Nov 1, 2024

@taylorhadden Quick update in case you're not following the PR:

They had an issue with the screenshots being in the flatpak repo and not somewhere upstream. I'll fix that by using the screenshots from the website.

Second more serious issue is that they would like the .desktop and metainfo files to also be upstream. I'll try getting a PR ready when I get to it. Is there a good place for these files in the project? I don't want to break your existing packaging by adding them. We'll also need to do another release after these are added, or we can wait until you release 0.9.0. I'll leave that up to you, although I'm a bit afraid if the changes take too long the PR will fall into a pile of TODO PRs. I was a bit shocked when I saw the number of open PRs on Flathub.

@taylorhadden
Copy link
Contributor

Thanks for pushing this through! Those files can live in the "build" folder of Tangent. Maybe in "build/flatpak"?

There are some bug fixes that want to get into a 0.8.2, so no worries there.

@k8ieone
Copy link
Contributor Author

k8ieone commented Nov 12, 2024

Thanks for the merge. Do you think you could re-tag the last commit? Either that or create another tag. I feel like we're really close here 🤞

@taylorhadden
Copy link
Contributor

Done!

@k8ieone
Copy link
Contributor Author

k8ieone commented Nov 14, 2024

Seems like we're blocked until Electron is able to fix this electron/electron#43819

Either we wait for this to get resolved or we use an older version.

@k8ieone
Copy link
Contributor Author

k8ieone commented Dec 27, 2024

@taylorhadden Seems like the fix isn't going to get backported to older versions of Electron any time soon. Is there any significant amount of work that would need to be done to use Electron 34 when it releases?

@taylorhadden
Copy link
Contributor

I wouldn't expect it to be a problem. Electron updates have been largely painless. In fact, I think they've fixed some contenteditable issues for me.

@k8ieone
Copy link
Contributor Author

k8ieone commented Dec 27, 2024

Awesome, I'd rather wait for 34 to release instead of downgrading it just to pass on Flathub. But it's your project, so this decision is up to you. We could find a working version in the issue report, a few were mentioned as working.

@taylorhadden
Copy link
Contributor

Given that my last upgrade fixed some other bugs, I don't really want to downgrade. I'm okay waiting on 34.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed tangent-electron Issues relating to the Tangent Application itself. User Request Features & Enhancements requested by users.
Projects
None yet
Development

No branches or pull requests

2 participants