bundling/minifying cronicle

[mikeTWC1984]

I've been experimenting with esbuild for a while, trying to bundle/minify cronicle into a single file. I can conclude that it is possible and viable option, however some tweaks are needed to make it work.

So the objective is to get rid of node_modules folder after bundling js fille. Here are some challenges you may face, and how to fix them:

• Storage plugins - those are imported via dynamic require. Bundler doesn't understand that, so it keep reference to engine/ folder, which won't exist in context of bundled main.js. I guess the best way to fix it is to statically require common plugins (fs/s3), baking it into bundle. However you can also make a copy of engine folder to lib. In this case you should also bundle each plugin separately in order to get rid of node_modules.
• Shared modules - some of the node modules are shared between frontend and cronicle engine. Cronicle's build tool is creating symbolic link in many case. Need to change it to "copy", so you can drop node_modules later
• Socket.io client library - in current setup socket.io is just serving client library (socket.io.js). This is causing similar problem as plugin issue, websocket engine will keep reference to node_module where this file is located. Solution is to copy socket.io.min.js to htdocs and update script reference on index.html (you can also disable serving client). This probably a good thing to do (regardless to bundling), because current setup might require some extra configuration for reverse proxies.
• Minification issue - if you try to minify cronicle bundle it will fail. Turns out problem is silly and comes from pixl-server-web. This component is missing __name property (needs to be WebServer). Cronicle gets it's name from class name, which will be changed by minification process, but other components still expects it to be "WebServer". @jhuckaby please update pixl-server-web to include __name property

Here is a minimal working demo you can test in docker

FROM alpine:3.16 

RUN apk add nodejs npm bash coreutils procps git 

WORKDIR /opt/cronicle

RUN  git clone https://github.com/jhuckaby/cronicle . && \
     npm i esbuild -g && npm i 

# patch setup.json - avoid creating symlinks from node_modules (just copy)
RUN sed -i 's/symlinkCompress/copyFiles/' sample_conf/setup.json && \
    cp -r node_modules/pixl-webapp/js htdocs/js/common  && \
     node bin/build dist 

RUN esbuild --bundle --platform=node --outfile=lib/cron.js lib/main.js && \
     ln -s $PWD/package.json lib/package.json 

# since storage plugins are loaded via dinamic require, need to build it separetly.
RUN esbuild --bundle --minify --platform=node --outfile=lib/engines/Filesystem.js \
     node_modules/pixl-server-storage/engines/Filesystem.js \
     && ln -s /opt/cronicle/lib/engines /opt/cronicle/bin/engines

# generate entrypoint script
RUN echo -e '#!/bin/bash\n\
/opt/cronicle/bin/control.sh setup || true \n\
/opt/cronicle/lib/cron.js --echo 1 --foreground 1 --master 1\n'\
>> entrypoint.sh  && chmod +x entrypoint.sh 

# patch socket.io.js 
RUN rm htdocs/index.html.gz && cp node_modules/socket.io/client-dist/socket.io.min.js htdocs/js/ \
     && sed -i 's/\/socket.io\/socket.io.js/js\/socket.io.min.js/' htdocs/index.html

# bundle all other js tools and plugins located in bin
RUN esbuild --bundle --platform=node --outdir=bin --allow-overwrite  bin/*.js && cp -r lib/engines bin/


# docker run -it --rm -p 3012:3012 cronbld ./entrypoint.sh

It will generate lib/cron.js bundle you can launch via ./entrypoint.sh script. It also bundle other js tools and FS storage plugin separately. After bundling, node_modules folder can be removed.
The size of cronicle bundle is about 3MB. Minified version would take 1.7MB. There are other js tools eating up some space, but technically you can bake them into one file. When I bundle cornicle with S3 (v3 client) and redis plugins total size is about 4MB.
Frontend would eat most of the space - fonts alone take ~ 4MB. If you replace font files with specific svg icons, I'd think entire app can be squeezed to ~10MB.

Also, not a huge deal but bundled cronicle seemed to start faster (~300ms vs ~1000ms)

[jhuckaby]

Minification issue - if you try to minify cronicle bundle it will fail. Turns out problem is silly and comes from pixl-server-web. This component is missing __name property (needs to be WebServer). Cronicle gets it's name from class name, which will be changed by minification process, but other components still expects it to be "WebServer". @jhuckaby please update pixl-server-web to include __name property

This was done deliberately. pixl-server-web is not missing anything. It was migrated from an extremely old class polyfill library (from before JavaScript had proper classes) to use real native ES5 classes with proper names. The class name IS, in fact, WebServer. This negates the need for the legacy __name property hack, so it was removed. Cronicle works fine with this. It sounds like your esbuild is doing something really funky. Maybe change it to minimize without munging class and function names?

[mikeTWC1984]

I believe renaming variables and functions (while keeping property names) is a standard transformation for any minifier. So if you tie property name to function name no surprise minification fails. In this particular case __name unintentionally plays role of "component_id", which prevent such issues.
Anyway, I realized user can just explicitly declare properties for components on main.js to avoid this issues (if bundling is needed)

Btw, since regular classes are available, is there any good reason to keep using generator functions? E.g. you define WebServer as regular class but then still wrap it with Class function before exporting (why not doing this with some static method or util?). That prevents static analysis/intellisense features you could get otherwise. Actually while experimenting with bundling I rewrote all your modules used by cronicle in regular classes (thinking dynamic classes might cause bundling issues), and cronicle ran without any problem.

[jhuckaby]

I believe renaming variables and functions (while keeping property names) is a standard transformation for any minifier.

I thought it was the job of the minifier to determine if variables and function names are used externally, and if so, do not touch them. It's only supposed to rename internal variables and functions. If esbuild isn't smart like that, you may need to supply it with a list of exceptions at build time.

From the esbuild docs:

https://esbuild.github.io/api/#minify-considerations

By default esbuild does not preserve the value of .name on function and class objects. This is because most code doesn't rely on this property and using shorter names is an important size optimization. However, some code does rely on the .name property for registration and binding purposes. If you need to rely on this you should enable the keep names option.

Looks like you need: https://esbuild.github.io/api/#keep-names

Btw, since regular classes are available, is there any good reason to keep using generator functions?

ES5 classes have some shortcomings that I find annoying, so I wrote a wrapper called class-plus to address them. I wrote about my reasons in that README. I didn't know it messed with static analysis/intellisense features. That's unfortunate.

[mikeTWC1984]

Looks like you need: https://esbuild.github.io/api/#keep-names

It does work indeed, thanks

I wrote about my reasons in that README.

I see. Looks like as of now most of your concerns are addressed, like you can add properties on the class including static/private (sounds like very recent features but I just tried on my default 14.17 node and it works). I guess the only thing really missing is the option to load mixins. Mixins are kind of "evil" from static analysis perspective, since IDE won't be aware of properties/methods you are adding, but doesn't sound like there is a good alternative. In many statically typed languages there are "extension" methods (external methods you can tie to a specific class), but such thing is not even on the radar for JS unfortunately.
Technically even plain js project can be covered with static analysis using classes, typings (TS type declaration you can put in @types folder) and JSDoc, but need to get rid of mixins, dynamic require and other magic.

I didn't know it messed with static analysis/intellisense features. That's unfortunate

That's not an error, Class function just returns plain object (so IDE is not aware of what it is). I doubt it would make much sense at this point but to fix that you can make Class function to modify class itself, not return a new class

[mikeTWC1984]

Wrapping up all recent changes and notes, below is the bundle.sh script to generate cronicle bundle. Need to be at the root of cronicle repo.
Upon bundle script completion need to run bin/control.sh setup and then bin/cronicle.js --echo --foreground --master
Also, node_modules, lib and some other folder can be safely removed. Total app size could be close to 10MB (including fonts and S3 engine).

# some prep for plain alpine installation
# apk add npm bash nodejs git procps sed && git clone https://github.com/jhuckaby/cronicle

npm i esbuild -g && npm i

mkdir -p htdocs/js/external htdocs/css htdocs/fonts

cp \
 node_modules/jquery/dist/jquery.min.js \
 node_modules/moment/min/moment.min.js \
 node_modules/moment-timezone/builds/moment-timezone-with-data.min.js \
 node_modules/chart.js/dist/Chart.min.js \
 node_modules/jstimezonedetect/dist/jstz.min.js \
 node_modules/zxcvbn/dist/zxcvbn.js \
 node_modules/socket.io/client-dist/socket.io.min.js \
 htdocs/js/external/

 cp \
  node_modules/font-awesome/css/font-awesome.min.css \
  node_modules/mdi/css/materialdesignicons.min.css \
  node_modules/pixl-webapp/css/base.css \
  htdocs/css/

  cp -r node_modules/pixl-webapp/js htdocs/js/common

  cp \
   node_modules/font-awesome/fonts/* \
   node_modules/mdi/fonts/* \
   node_modules/pixl-webapp/fonts/* \
   htdocs/fonts/

 sed -i 's/\/socket.io\/socket.io.js/js\/external\/socket.io.min.js/' htdocs/index-dev.html 
 mv htdocs/index-dev.html htdocs/index.html
 ln -s sample_conf conf
 ln -s ../package.json bin/package.json

esbuild --bundle --minify --platform=node --outdir=bin --allow-overwrite bin/*.js 
esbuild --bundle --minify --platform=node --outdir=bin/engines node_modules/pixl-server-storage/engines/Filesystem.js
esbuild --bundle --minify --platform=node --outdir=bin/engines node_modules/pixl-server-storage/engines/S3.js
esbuild --bundle --minify --keep-names --platform=node --outfile=bin/cronicle.js lib/main.js

# should be safe to remove anything besides bin, htdocs, conf/sample_conf and package.json
# rm -rf node_modules lib docs .git README.md

# try it out
# bin/control.sh setup
# bin/cronicle.js --echo --foreground --color --master