updated to v3.29.1 #11
Conversation
|
Started test build 34386 |
|
Build 34386 failed |
|
@muelli the library runs an install script that downloads the binaries from the GitHub release depending on platform/arch. You could manually do that, but then you don't need the library in the first place. :) |
|
ffmpeg-ffprobe-static is forked from https://github.com/eugeneware/ffmpeg-static which also downloads ffmpeg from the internet. (I used that package up until now) I don't understand how ffmpeg-ffprobe-static is any different, other than also downloading ffprobe in addition |
|
The ffmpeg-static package supports, albeit clunky, pre-downloaded executables: eugeneware/ffmpeg-static#37 |
I'd love to. But it's unclear to me, how I would do that. |
Doesn't seem like it: https://github.com/eugeneware/ffmpeg-static/search?q=FFMPEG_BIN |
|
Is it possible to enable internet access during build? Or use the prebuilt linux binary from here: https://github.com/mifi/lossless-cut/releases/tag/v3.29.1 |
No. This is not possible. Flathub doesn't allow that. For good reasons. No serious software distribution does that, AFAIK. It should be known at build time, which bytes go into the build which produces the artefacts produced for users.
That'd be possible, but it'd be much preferred to build from source. |
That makes sense
Understandable. But using ffmpeg-static, ffmpeg is also not built from source, because that package uses prebuilt ffmpeg binaries. Or does this flathub build script also build ffmpeg from source? |
|
I do build ffmpeg/ffprobe from source for losslesscut automaticlaly using github actions, but only for MacOS as of now: https://github.com/mifi/ffmpeg-build-script |
Right. I think there are two parts to that. Firstly, it'd be nice to build everything from source. It's not a strict requirement, though. The offline build is. And building v3.29.0 requires ffmpeg-static which can only be installed when certain files have been put in place before, i.e. the ffmpeg executables. Cf. no.mifi.losslesscut/no.mifi.losslesscut.yaml Lines 131 to 134 in d9c7959 Secondly, but I think that's a minor detail, there is an ffmpeg executable in the runtime: but I don't think it is being used.So yeah, taking the whole app as a BLOB and shipping it to users is an option, but I consider it the worst. Making it build with all dependencies pre-loaded is my favourite. Bonus points if all sub-dependencies (e.g. ffmpeg) can be either built or taken from the system. |
|
I could add code in the app that uses the system installed ffmpeg and ffprobe for linux, but I would then need to know how to query this runtime, and how to get the paths to those (PATH?) |
|
Yeah, that could work. I don't understand why you would need to find the path rather than just executing the name and hoping for the best, but I consider that secondary, anyway. Right now, for the flathub package, the ffmpeg-ffprobe-static dependency is a problem and needs to be capable of being installed offline. It'd be nice if it could check whether the contents it wants to download already exists. |
|
I could execute the ffmpeg and ffprobe commands from PATH, and in such case I would use https://github.com/npm/node-which which would find this for me automatically. I'm not sure how I think the optimal would be to go back to ffmpeg-static, but then someone would need to implement the ffprobe support there |
yeah, that sounds good :)
I guess this line makes it work: https://github.com/eugeneware/ffmpeg-static/blob/55975393f91e091a2083400477c18393f197a09e/install.js#L28 The ffmpeg-ffprobe-static package has a very similar mechanism: So it could work by placing something in the right folder. But instead of me guessing which files the package expects there, I'd appreciate if @marcello3d could chime in to tell me what file (and preferably its hash) to put where.
I'm not entirely convinced. Unless @marcello3d objects, I have the feeling that enabling the package to be installed offline is the better approach. But in the end it doesn't matter much. |
|
Great find! I think it would be like this then: |
It doesn't build, though :( It seems the directory is re-created.
Here we can see that the files are indeed in that directory. When the build fails, the diretory looks different, i.e. as if it has been re-created.
|
Started test build 34608 |
|
Doesn't work, unfortunately :( It seems the directory is re-created, i.e. the files disappear. |
|
Build 34608 failed |
|
All ffmpeg-static does is make it so you don't have to manually download and copy these prebuilt static libraries and gives you the path to them in node_modules. My fork just adds ffprobe. If you're downloading them manually/"offline" then you don't need the module. |
|
The problem is that the official build process of losslesscut doesn't manually download and copy the files. If it's a lot of trouble getting it all to work, then maybe I should ditch the whole ffmpeg-ffprobe-static and instead create a pre-build step in the official build process that downloads the right ffmpeg first. |
|
It seems like what you want to do, since this is Linux, is not bundle ffmpeg/ffprobe at all and depend on a shared system version. That'll make the app a lot smaller, too. |
|
Yea if flathub can specify an ffmpeg version requirement in its "spec" file, that ensures the user has already installed a specific version of ffmpeg, then we could do that |
I don't agree with your reasoning, but I don't think it matters. I think the author of a piece of software could very well depend on the package reliably providing "ffprobe" while a distributor, at the same time, builds that software offline. Would you object if anybody provided a patch to make the module offline installable? |
AFAIU, the "Freedesktop Runtime" as of 20.08 commits to shipping ffmpeg 4.3. |
and it seems to remove the ability to install the package offline :-( |
That was a change on the original I'm sorry it doesn't work for you, feel free to make your own module! :-) As I said, they're not doing much of anything other than packaging the binaries in a manner more useful for node/electron. |
Oh and losslesscut flathub depends on freedesktop runtime? Then the only problem is that I need to know somehow inside the JS code that we are running the flathub version. Or I just default to trying the system ffmpeg/ffprobe if the included binaries fail to run on linux. Ohh and somehow we must make the ffmpeg-ffprobe-static installation not crash if offline, am i right? Or remove it entirely |
Ah, yes:
Apps run inside a flatpak container expose
yes. This is the actual blocker here. All the other things like running the correct ffmpeg executable can easily be achieved, I think, if only by symlinking executables into the right place or manipulating PATH. |
ah. weird then. Maybe @derhuerst can comment on whether or how it's possible to install the package offline or what caused this ability to be removed. |
|
Maintainer of
It does. You can set the export FFMPEG_BIN=/bin/bash
npm i ffmpeg-static
# > [email protected] install /Users/j/playground/node_modules/ffmpeg-static
# > node install.js
#
# ffmpeg is installed already.
# + [email protected]
node -p 'require("ffmpeg-static")'
# /bin/bashWhen doing this, I don't really see the point of using The
Yes, because it couldn't do the thing it's supposed to do: Provide a self-contained (as in independent of system-installed binaries) & platform-specific binary to npm-based tools.
As I said in the linked comment, I agree by now that adding |
|
Regarding the "no network access during builds" thing: You could of course run But I don't think it's worth the hassle/complexity, I'd rather argue that we should all work towards a trusted host/API from which ffmpeg binaries for all plattforms/arches can be downloaded with a simple one-liner. (That could of course be the |
|
Thanks for your input!
Right. We made it work for the ffmpeg-static package. That capability seems to be missing from the ffmpeg-ffprobe-static package, though. At least, I cannot see how to make that work.
I don't think these goals are mutually exclusive. Here, we have the case that a software package depends on (a variant of) ffmpeg-static, because it wants to ensure that an ffmpeg executable is present. Regardless of operating system or architecture. In order to package this software up for distribution to users, we, as distributors, now want to say, "please, please, don't download anything from the Internet, as we consider it a big waste of resources and makes auditing the software nearly impossible, and take this ffmpeg instead". |
If you want something auditable, then relying on ffmpeg binaries provided by random strangers on the internet™ is probably not a good idea either. 😃 Let's set up a repo with the sole purpose of hosting the ffmpeg binaries, with their hashes, links to build scripts, etc. Eventually, we could enhance it to build the binaries from source. The losslesscut build would then pull its binary from this repo, just like |
Sure. You are absolutely right. I wish the whole ecosystem worked entirely differently. But that ship has sailed. So two things there: Firstly, with ffmpeg-static, rather than ffmpeg-ffprobe-static, we can ignore the BLOB from the Internet and have our own version run. Secondly, we're not even trying to fix everything, but make things better step by step. And if we can control the build, as in, not relying on the Internet as an input to our build, we can perform audits much much easier, because the build is deterministic. Hence, less fragile, less dependent on the mercy of other parties, it has chances of being reproducible, the build infrastructure can be greatly simplified, and many other benefits. |
As |
|
In the I could assist with this or proof-read it, but I have neither the energy nor the Electron-specific knowledge to drive this by myself. Anyone willing to do this? |
except mac Remove ffmpeg-ffprobe-static, use scripts instead flathub/no.mifi.losslesscut#11
|
superseded |
This doesn't build, because the new package, ffmpeg-ffprobe-static requires Internet access during build time :(
Maybe @marcello3d can shed some light into making it build offline, e.g. by telling how to provide the required executables upfront.