Windows + meteor + sanjo:jasmine + karma: hot code updates prevent automatic test rerunning

It turns out that last time I was a bit wrong! Upon hot code update, not only PhantomJS jas problems. Whole Karma has, and has them regardless of the selected browser.

Velocity/Sanjo/Karma/(PhantomJS|Chrome|.+) hangs when Meteor auto-rebuilds your project

I must have been tired back then and I didn't notice that autoupdate breaks running tests in Chrome, too. Please see this issue for analysis on that. In short, I've learned that this in fact is Karma problem, or rather, Chokidar@Windows problem: when observed directory is removed or renamed, Chokidar stops observing it - and that's exactly what happens to whole build directory. Fortunatelly, after some work and a bit of hints from Sanjo about (re)starting Karma, I managed to work it around by:

• detecting when autoupdate is about to happen
• deterministically killing Karma in that case
• detecting when autoupdate finishes
• restarting Karma right after that

Here's a patch. Yay, now it really actually works with hot updates.

All details are described in the issue, but it's worth pointing out few things:

• since Karma dies just before autoupdate begans, all Chokidars file locks are freed - this means no EPERMs during rebuild, and meteor usually can just rename the dirs with no cp-r/rm-r fallbacks
• since Karma dies and rises, there's little point in watching any application files at all - change to client files will cause autoupdate (and restart Karma), change to server files will kill&restart whole app (and Karma too). The patch mentioned above turns these watches off, but leaves watches on test/spec files.
• Meteor raises two interesting 'events': 'message{refresh:client}' and 'onListening'

Message: refresh-client

reminder: hot code updates are either 'refreshable' or 'not-refreshable'. If you change only 'client' code, then it's the former. Clients will refresh and server won't notice. When you change any 'server' code (/server, /lib, ...), then it's the latter and whole app must be refreshed on both sides. By 'refreshed' I mean kill-everything-and-restart.

refresh:client is broadcasted by the application that is about to start an autoupdate cycle in 'refreshable' mode. Actually, there's a internal listener on that event which starts the autoupdate process. Listening to that event will run your code before the autoupdate starts, but only because that internal listener uses setTimeout(.., 100ms) to delay the process just a bit, so any custom listeners added must act .. quickly or assume running "in parallel" with the build.

During 'non-refreshable' mode does not raise that event, as the app is going to terminate and fully restart in a moment, what will cause all clients to refresh just like during any server reboot.

WebApp.onListening event

Basically, whatever listener is registered there, it's invoked when the hosted application is ready to serve requests. This surely means that any build processes have finished. It is invoked in all three cases:

• application's initial start
• after refreshable update finishes rebuilding
• not-refreshable update (because it's in fact 'initial start' anyways)

However, using this event needs some care. When raised on application's start, it is called before Velocity starts its startup tasks. During refreshable autoupdate it seems to be called last. Here in this case the handler wanted to start Karma after refreshable update, but the same handler is called during app's start when few-second-later the Velocity would start Karma too. That means that handlers may need to check at what stage they were invoked, just to make sure to not race with Velocity at application's start.

Another (small) catch with onListening is that it does not always run when serving starts. If the app is already running, registering handler to that event will not wait for the next update/restart, but instead will immediatelly that handler. Only handlers registered before the app is ready are delayed, stored and executed when its ready.

Another (big) catch is coming along with previous: I really said delayed, stored. Handlers registered when the app is already running are not stored and they are just run immediatelly, and they will not run upon next update/restart. Actually, those handlers registered early will not fire either. It turns out that the list of delayed handlers is cleared after the event occurs. List is cleared as handlers are invoked.

This causes interesting problem: It's tricky to permanently listen to onListening. You have to register the handler at the correct moment of time when the application is not considered to be running or else the handler will immediatelly fire. Fortunatelly, I noticed that one such points of time is the message-refresh-client. This notification is sent when preparing for a quick rebuild, so any onListening registered during message-refresh-client will be delayed and called when refresh cycle ends.

Any examples?

Just in case you missed the link, here's the patch that fixes re-running tests when hot code update happens. It uses these two events to do the job of restarting Karma. Oh, and also some small tweaks to sanjo-karma were needed to be able to actually stop Karma.