Use rollup to build the app #433

rchl · 2020-09-24T20:27:15Z

It's now an npm project with those advantages:

introduces a build step to allow for using modern code syntax
that will be automatically transpiled to support older browsers.
includes commit hook to automatically lint the code on committing
enables (selectively) type checking to catch issues early during development
allows to clean up the code and separate into smaller chunks that will
be combined into a single bundle
releases will be versioned so that it's possible to refer to concrete
versions and possibly include changelogs.
versioning also allows having URLs without random string attached
so that those can be cached by the browser

The end-user code is now generated in dist/ so that is what user has to
deploy rather than all files in the repo.

It's now an npm project with those advantages: - introduces a build step to allow for using modern code syntax that will be automatically transpiled to support older browsers. - includes commit hook to automatically lint the code on committing - enables (selectively) type checking to catch issues early during development - allows to clean up the code and separate into smaller chunks that will be combined into a single bundle - releases will be versioned so that it's possible to refer to concrete versions and possibly include changelogs. - versioning also allows to have non-random URLs for included scripts so that those can be cached by the browser The end-user code is now generated in dist/ so that is what user has to deploy rather than all files in the repo.

rchl · 2020-09-24T20:28:06Z

@resoai @akloeckner @cgarwood It's a pretty serious change. For those who are involved in the development or plan to be, I encourage you to try it and report issues, annoyances, or anything that might be wrong with this approach. We go forward with it only when everyone is happy. :)

BTW. The approach with document.write'ing the config file from the index.html is neither great nor very compatible with this solution. I mean, it works, but it required to do it in a sort of hacky way by processing globals/constants and the rest of the app in two input steps, and then document.write'ing the config in between.

A proper solution would be to make the config file be an angular service that could be loaded asynchronously but that would be a breaking change that would require all user configs to be adapted. So I don't really see this happening anytime soon.

Also, I did as minimal as possible changes to existing scripts. If this gets accepted and merged we can do further things with the scripts like:

using more modern JS
enabling more linting rules
structuring script files better
possibly splitting some code into smaller chunks (it all gets bundled together anyway)

akloeckner · 2020-09-24T23:48:03Z

Wow. Looks like a great deal of changes. I'll have to deep dive into npm and test with some of my devices... I'll report back!

akloeckner · 2020-09-25T11:17:04Z

So, I checkout out your branch and copied my config files to the dist folder. I receive the "invalid config" error... Is stuff like that not allowed anymore?

testEntity='sensor.aktuell_infizierte_in_koln';
//testEntity='input_select.mode';
//testEntity='input_number.wohnzimmer_ontime';

/**
 * Load JS file synchronously and evaluate
 */
function loadJS(url) {
   var xhttp = new XMLHttpRequest();
   var script = document.createElement( "script" );
   xhttp.open("GET", url+suffix, false);
   xhttp.send();
   script.text = xhttp.responseText;
   document.head.appendChild( script ).parentNode.removeChild( script );
}
loadJS('autoconfig.js');

/**
 * AUTOMATIC TILE SIZING
 *
 * Calculates tile size automatically for different screensizes.
 */

var groupWidth  =   2; // count of tiles horizontally
var groupHeight =   4; // count of tiles vertically
var tileMargin  =   6; // px between tiles
var groupMargin =  50; // px between groups
var headerSize  =  60; // px of group or page header
var maxTileSize = 120; // px

var tileSize = setTileSize();
window.addEventListener('resize', function () {CONFIG.tileSize = setTileSize();});

function setTileSize () {
   var width   = window.innerWidth  - headerSize - groupMargin - tileMargin * (groupWidth  - 1);
   var height  = window.innerHeight - headerSize               - tileMargin * (groupHeight - 1);
   var optimum = Math.min(width/groupWidth, height/groupHeight);
   return Math.min(maxTileSize, optimum).toFixed(1);
}

rchl · 2020-09-25T11:31:02Z

@akloeckner it should still work. You should check the browser console for the exact error.

When I've tried this code here I got an error about undefined suffix so I guess you were relying on the global one from the old index.html?

EDIT: BTW. Is there a reason you won't just put that code in the main config and avoid extra request?

akloeckner · 2020-09-25T13:20:26Z

Ok. I replaced the suffix. Now, it's complaining about angular not being found. Also window.angular doesn't get found. I'm using angular.equals and angular.element...

BTW: Yes. I just prefer my setup to be tidy. So, I moved all helper functions outside of config.js. I just couldn't find my way around in that humongous file anymore...

rchl · 2020-09-25T13:48:07Z

Right.

Due to this document.write nonsense, I'm creating two bundles, one with just essential global constants and utilities and another with the rest of the app. Config only has access to the former when it's evaluated and that former bundle doesn't have angular source yet.

Adding angular source to that first bundle would duplicate angular which means a much bigger bundle size and also some warnings from angular due to being evaluated twice.

I can't see an immediate clean solution for that but I can look more into it later if we are not fine with this being a breaking change.

akloeckner · 2020-09-25T14:10:11Z

I could get rid of element. But I need this in a few places:

var id = window.angular.equals(id0, {}) ? undefined : id0;

rchl · 2020-09-25T14:38:40Z

I could get rid of element. But I need this in a few places:
var id = window.angular.equals(id0, {}) ? undefined : id0;        

Such a simple check could probably easily be replaced with something like Object.keys(id0).length === 0 but still this is something that should probably be addressed.

akloeckner · 2020-09-25T14:59:49Z

I don't understand the process of bundling that you introduced. But would it help for this issue to have only one bundle? And what changes would be required on the user side? If the required changes are easily explainable, I'd say we do the restructure properly in the first place and put a migration section into the wiki.

rchl · 2020-09-25T15:14:30Z

There are two bundles because the config needs to be loaded between those two. Those two bundles are:

Constants and Global Utils
The whole rest including angular, dependencies, and TileBoard.

Config needs to be loaded between those two as it needs access to constants like TYPES.* and all the other.
At the same time, the TileBoard needs access right away to CONFIG from App.config and such.

Making one bundle could work if Config would utilize angular.service and angular.provider. The config would be structured something like this:

angular.module('App').provider('configProvider', function () { this.$get = function(Constants, Noty) { ... } })

I believe that should work. I haven't actually tried it yet.

Evgeny- · 2020-09-25T15:52:05Z

👍

Have you tried to make app initializing asynchronous? I'm not sure what was the original problem, probably was too lazy to make it. It should help to those who want to make module configs. Though, without rebuilding the app.

May be adding global require function could help solve that problem?

rchl · 2020-09-25T17:57:43Z

Your comment reminded me that there is a way to initialize angularjs manually. It actually works quite well with just a single bundle now. Thanks! :)

rchl · 2020-09-25T18:42:09Z

I've reverted some unnecessary style changes to make diffs easier to review.
Style issues will be handled in a separate PR.

…tion

rchl · 2020-09-25T20:21:18Z

In the latest commit, I have fixed issues with assets referenced from CSS not being included in the dist/ folder which meant that those wouldn't be available after deploying.

Also made production bundles have hashes in the filename. That means that on every change of some code, the hash will change which should eliminate issues with caching and not seeing the latest changes after updating to the new version. Also we don't have to bust cache on every request which allows the browser to cache those and load TB faster.

akloeckner · 2020-09-25T21:20:48Z

I replaced the angular.equals by your suggestion. No issue anymore. Now it complains about me using mergeObjects, which is apparently found. But inside of it window.angular.merge is not found...

Evgeny- · 2020-09-25T21:23:51Z

@resoai let's just merge it and solve all problems afterwards :)

resoai · 2020-09-25T21:28:05Z

@resoai let's just merge it and solve all problems afterwards :)

That is exactly our way! Since I still have absolutely no way to test it, I will leave it up to @rchl and @akloeckner to decide when it is time to merge.

rchl · 2020-09-25T21:29:15Z

@akloeckner Actually with my latest changes all of angular should be available in your config. So you can keep using it.
As for mergeObjects, I've pushed small change for that but it should have worked already. Maybe try again and post exact error?

akloeckner · 2020-09-25T22:22:06Z

Thanks! That seems to work now! (With original config, using angular.merge directly.)

When trying to start the development environment (impressive, I must say), I noticed a few thing to probably add to the CONTRIBUTING docs:

config.js must be copied to dist-dev (not dist)
http://<your-dev-server>:8080 needs to be added to HA's http: cors_allowed_origins:
in case you use https for HA, the default in config.example.js does not work anymore, because dist-dev is served on http

I have one open question still:

In my "production" environment I use multiple branches currently, which I merge together: master for sure, plus secret with my config, plus a few feature/fix branches for testing before I open PRs. As I understand, that will not be possible anymore, because merging does not execute the bundling step, right? (I guess, it boils down to the fact that I have local modifications of the code. An I'm sure, I'm not alone with this.) What would be the steps, I have to take? (Probably, I wouldn't want to override dist, because this would leave me with a dirty git repo. And I also wouldn't want to be running npm run dev all the time.) I suggest to put those steps in the contributing docs (or some more suitable place) as well.

As to the huge amount of changes, I'll have to trust you @rchl , since I have no idea of the magic you have done!

So, apart from these documentation changes, I suggest to merge the PR. Looks promising!

akloeckner · 2020-09-25T23:01:54Z

Another thought I just had: I have tileboard as a sub-repo in my HA config repo. So, instead of copying files from dist, I'll just open a different URL. The same might hold true for people simply checking out tileboard to their webserver.

So, it might help to put a minimal index.html in the tileboard root explaining that the app has moved. It would point people with those kinds of setup in the right direction.

rchl · 2020-09-26T14:38:57Z

Good foodback. Will try to address it later.

As for your development situation with different branches I'd have to understand that better. Would you be using dist-dev output or dist? With dist you would commit your modifications so changing branches shouldn't be an issue.

akloeckner · 2020-09-26T15:49:25Z

More observations:

somehow the history graphs don't have tooltips anymore.
some graphs fail with r is undefined
the endpoints of the graphs are strange.

akloeckner · 2020-09-26T16:06:32Z

As to different branches, i usually do something like this:

git checkout -b deploy master
git merge secret fix-camera-title cleanup-less

This leaves me with a working directory including all features I need/want. Will those merges work with dist? I have the impression, it will be a matter of luck, if merges will conflict, due to the toolchain inbetween. I don't mind changing my workflow. But I'd like it to be as easy (automated) as possible. I could extend my above script with some rollup command. Or I could set up an npm dev service.

Probably the branching thing is a specialty of mine. But others might have simple modifications/fixes they don't bother to put in a PR. Such as fixing the issue here #425 .

So, suppose the user has fixed that one line from #425 locally, what would be the steps to bring this change into dist? And will it survive a git pull?

rchl · 2020-09-26T20:32:24Z

Another thought I just had: I have tileboard as a sub-repo in my HA config repo. So, instead of copying files from dist, I'll just open a different URL. The same might hold true for people simply checking out tileboard to their webserver.

I've added an index.html in the root.

config.js must be copied to dist-dev (not dist)

Changed

http://<your-dev-server>:8080 needs to be added to HA's http: cors_allowed_origins:

Added a note about that. Unfortunately, that won't work for history API right now (home-assistant/core#39727).

in case you use https for HA, the default in config.example.js does not work anymore, because dist-dev is served on http

Added a note on setting that.

akloeckner · 2020-09-26T21:20:14Z

Created #434 for it.

My HA is runing for days. So this is probably not my very issue. (Still good, it's identified!) My debugging pointed me to some tooltip code. (tooltip is replaced by r in the bundled code, but the dev server still had tooltip.) So, I hope, this problem will be solved by you having forced the latest Chart.js version.

EDIT: From what I could alert out of the main.js, the response from API looks much like the responses for other graphs. But it's a huge object, so it's hard to tell.

Fixed all those issues by forcing the latest version of chart.js.

Could you commit that change , so I can test on my end again?

EDIT: If it's not fixed, I'll continue debugging.

Unfortunately, that won't work for history API right now (home-assistant/core#39727).

Well, actually, for me that does work, as I noted in #386. However, good it's in the docs now. I'd still suggest to also give a hint to the HA setting, because that's what I had to do to be able to authenticate with HA, even.

akloeckner · 2020-09-26T21:29:34Z

Could you commit that change , so I can test on my end again?

Sorry... I didn't see that commit. Tested and all seems to be fine now! Thanks!

EDIT: Tested in dev server. Very neat. But had to rebuild the dist using npm run build.

rchl · 2020-09-26T21:42:52Z

I'd still suggest to also give a hint to the HA setting, because that's what I had to do to be able to authenticate with HA, even.

Thanks for the remainder. Forgot about it. Now it's added.

EDIT: Tested in dev server. Very neat. But had to rebuild the dist using npm run build.

Yes, every change in the scripts or dependencies requires rebuild.

I think this should be solved by forcing build on every commit but since we've decided to investigate not committing the dist folder, I'll look into that instead.

akloeckner · 2020-09-26T21:53:08Z

CORS...

Thanks! Sorry for being pedantic: localhost will not always be the right host. Probably developers will notice. But since we're discussing it here... I'd phrase it:

Configure http.cors_allowed_origins setting in Home Assistant to allow your server (e.g. localhost) to communicate with the HA API:

http:
  cors_allowed_origins:
    - http://<your-server>:8080

every change in the scripts or dependencies requires rebuild.

Which takes 2m18s on my pi! 30s for the dev server. Maybe we can speed this up once we have the system in place. In my opinion that's anything but "iterate quickly"... Also, it seems to write lots of files. Not sure, what it will do to my SD card wear...

rchl · 2020-09-26T21:59:28Z

Which takes 2m18s on my pi! 30s for the dev server. Maybe we can speed this up once we have the system in place. In my opinion that's anything but "iterate quickly"... Also, it seems to write lots of files. Not sure, what it will do to my SD card wear...

Personally, I develop on my main Desktop and have a script that just pushes built files to PI. With a dev server I don't even have to push on each change as I can test locally.

You could speed up production build by removing terser plugin (appPlugins.push(terser()); line) which is for minifying the code. This is probably what makes the biggest difference. Or maybe try with treeshake: false and/or output.sourcemap: false in the config.

But yeah, it can be a bit heavy. There are pros and cons to everything...

akloeckner · 2020-09-26T22:02:33Z

Personally, I develop on my main Desktop

I should start that, too, maybe... :-) I'll try your options and see, what it does.

EDIT: 48s without terser and without sourcemaps (js and css). I couldn't find treeshake... Still not great, compared to simply reloading the page. ;-) But minifying is a good thing. And you listed a few strong advantages in the first post. So, let's not be stopped by that.

akloeckner · 2020-09-27T08:33:41Z

I now also got around to testing this with my old iPad, iOS 9.35. I had to create a long-lived token for it (see home-assistant/frontend#5376). However, the TileBoard does not load. It is stuck on the background color. Usually that is an indicator for some broken JS code, hindering the app to initialize.

TileBoard works as usual in the old structure.

rchl · 2020-09-27T08:41:47Z

Is it possible to get access to the console and see what it errors on exactly?
It's probably possible to configure builds process to support it.

akloeckner · 2020-09-27T08:56:10Z

I'm not sure, how to access the console. The internet seems to suggest that I need Safari on my Laptop and some USB cable. It might take some time to get that up and running...

Looking at this issue, however, the const seems to be unsupported by the old iOS (at least):
rollup/plugins#190

akloeckner · 2020-09-27T12:37:54Z

I'm not sure, how to access the console.

Hmm... as it seems, I need Safari for this to work.
https://www.kenst.com/2019/03/how-to-debug-problems-on-mobile-safari/

Unfortunately, Safari is said to be only available on Mac.
https://de.wikipedia.org/wiki/Safari_%28Browser%29

But I have no Mac. So, I guess I'm stuck...

rchl · 2020-09-27T13:58:17Z

I Will take care of the const problem and we'll see. The code is supposed to be transpiled for old browsers so there shouldn't be any consts in the production bundle

rchl · 2020-09-27T14:25:38Z

OK, pushed a change that actually transpiles the code for older browsers. Not sure if it will handle your old iOS by default. If not then we'll have to change config to target older browsers.

rchl · 2020-09-27T19:08:35Z

OK, some bigger changes in the latest commits:

dist directory is not committed (no built app is included)
both npm run build (production build) and npm run dev output to build directory (so not dist and dist-dev respectively)
New release will be created using npm version patch and that will trigger a github workflow which will automatically build, package and create a new release on https://github.com/resoai/TileBoard/releases. Then people can just download the zip file from there that includes only built code and nothing else.

akloeckner · 2020-09-27T21:04:02Z

I checked my old iOS. It works (npm run build)!

Since everything works now on my end, I'll be fine to merge as soon as you (and @resoai) are happy with the workflows!

rchl · 2020-09-28T08:05:53Z

I'm happy with the changes myself too so will merge later in the day.

rchl added 4 commits September 25, 2020 20:13

a single bundle with angular bootstrapped manually

9eae6da

Clean up package.json

80973c4

Revert some style changes

e628324

revert style

c337fe9

Fix references to assets in CSS and generate hashed bundles in produc…

c476c00

…tion

can now import angular directly

ba55277

rchl added 2 commits September 26, 2020 22:24

Add index.html in the root as a migration document

9d4586b

Update contributing instructions

cebafda

Add cors_allowed_origins instructions

3c710ee

update instructions

cd6d07e

transpile code using babel/preset-env

6e4bafb

rchl added 4 commits September 27, 2020 16:29

Don't commit compiled files

26d395b

Add build script and github workflow

c81f3c7

install deps first

aa57923

Create github release on pushing new version tag

3633cfb

rchl added 2 commits September 27, 2020 21:27

Add build:watch script for building production from watcher

5f269db

Minor style change

95ee27c

Update instructions in readme and index.html

0b91ad1

rchl merged commit c72f890 into master Sep 28, 2020

rchl deleted the feat/rollup branch September 28, 2020 14:37

Use rollup to build the app #433

Use rollup to build the app #433

Conversation

rchl commented Sep 24, 2020 • edited Loading

rchl commented Sep 24, 2020 • edited Loading

akloeckner commented Sep 24, 2020

akloeckner commented Sep 25, 2020

rchl commented Sep 25, 2020 • edited Loading

akloeckner commented Sep 25, 2020

rchl commented Sep 25, 2020

akloeckner commented Sep 25, 2020

rchl commented Sep 25, 2020

akloeckner commented Sep 25, 2020

rchl commented Sep 25, 2020

Evgeny- commented Sep 25, 2020

rchl commented Sep 25, 2020

rchl commented Sep 25, 2020

rchl commented Sep 25, 2020

akloeckner commented Sep 25, 2020

Evgeny- commented Sep 25, 2020

resoai commented Sep 25, 2020

rchl commented Sep 25, 2020

akloeckner commented Sep 25, 2020

akloeckner commented Sep 25, 2020

rchl commented Sep 26, 2020

akloeckner commented Sep 26, 2020

akloeckner commented Sep 26, 2020

rchl commented Sep 26, 2020

akloeckner commented Sep 26, 2020 • edited Loading

akloeckner commented Sep 26, 2020 • edited Loading

rchl commented Sep 26, 2020

akloeckner commented Sep 26, 2020

rchl commented Sep 26, 2020 • edited Loading

akloeckner commented Sep 26, 2020 • edited Loading

akloeckner commented Sep 27, 2020

rchl commented Sep 27, 2020

akloeckner commented Sep 27, 2020

akloeckner commented Sep 27, 2020

rchl commented Sep 27, 2020

rchl commented Sep 27, 2020

rchl commented Sep 27, 2020

akloeckner commented Sep 27, 2020

rchl commented Sep 28, 2020

rchl commented Sep 24, 2020 •

edited

Loading

rchl commented Sep 24, 2020 •

edited

Loading

rchl commented Sep 25, 2020 •

edited

Loading

akloeckner commented Sep 26, 2020 •

edited

Loading

akloeckner commented Sep 26, 2020 •

edited

Loading

rchl commented Sep 26, 2020 •

edited

Loading

akloeckner commented Sep 26, 2020 •

edited

Loading