I recently tried using Meteor to build and test a simple application. Since I got used to Jasmine, I tried the mainstream plugin sanjo:jasmine for Velocity bundled with Meteor.
When having some unit-tests, autoupdates cause "Error EPERM, stat" and/or "Error EPERM, unlink"
I have to say, I'm not delighted with testing experience. At first glance it looks great. However during first few days I stumbled upon several bugs related to deleting directory trees in autobuilding/autorefreshing, very annoying, it literally killed (I mean, crashed) the server on each code update and forced me to rerun it manually - which invalidates the whole live/hotswapping idea. Ah yes, I forgot to say I'm on Windows. It probably works perfectly on Linux, where the filesystem behaves differently. Poor me.
I was able to trace and somewhat patch them up, but it doesn't 'feel stable'. I have observed that during autoupdate, even with those patches, the renameDirAlmostAtomically still falls back to cp-r/rm-r fallback, what probably shouldn't have place. Anyways, at least meteor stopped crashing due to EPERMs :)
You can find the rimraf patch here and the meteor patch here
Velocity/Sanjo/Karma by default opens a Chrome window to run the tests
Next thing that bothered me was the Chrome window that was repeatedly opened and closed by Karma on each autoupdate. I understand that this is necessary as the whole app is being killed including Karma and that Karma need to have an actual runner, but damn! when you work on a single monitor work station, you can't even get that Chrome minimizes because it will get soon killed and restarted and a new window will popup, cluttering your workspace again.
Tired by it I searched the web for some solution and I found that it's relatively easy to switch sanjo/jasmine to use PhantomJS instead of Chrome. Great, I knew PhantomJS back from working with Chutzpah. However, despite everybody talking how great is using hidden PhantomJS instead of windowed Chrome, the hints on how to do that varied. The most common were:
- set environment var
JASMINE_BROWSER=PhantomJS meteor - set environment var
JASMINE_BROWSER=PhantomJS meteor run - set environment var
JASMINE_BROWSER=PhantomJS meteor --test - set environment var
JASMINE_BROWSER=Chrome PhantomJS meteor run(sic! what a pity I didn't save the link where I saw that)
I wouldn't be writing about it if any of those worked. None was. After looking at the sources I noticed that this is almost directly passed to Karma, and that:
- this option takes the keyword like 'Chrome' or 'PhantomJS' only, no "commands" like
meteor --testare supported - this options does not support specifying multiple browsers, even though Karma could
So, at the time of writing, sanjo:jasmine ver 0.17.0, the only proper way of setting this variable is:
JASMINE_BROWSER=Chrome
JASMINE_BROWSER=PhantomJSotherwise it won't run anything, as sanjo:jasmine will not be able to pick a 'launcher' (karma-chrome-launcher, karma-phantomjs-launcher, etc) and Karma will start with no runners (so i.e. you will still be able to manually open localhost:9876 in some Chrome window).
So far so good. I stopped meteor from crashing, I switched it to PhantomJS, got rid of Chrome window! Did I mention that instead of a Chrome window I got now an empty console window?
Velocity/Sanjo/Karma/PhantomJS by default opens a console window to run the tests
I started to feel a little grumpy about that, but I kept digging and I've found that somewhere in sanjo's long-running-child-process there's a series of child_process.spawns that run a spawnscript with stdio bound to logfile, and that script in turn creates child PhantomJS process. That 'spawnscript' had an option detach: true set. That comes from NodeJS and is probably important on Linux to 'daemonize' a child, but on Windows the parent processes does not wait for their children anyways, there's no point in that option. I removed that option and wow, console window disappeared. Now I had the test process truly hidden.
You can find the patch here
Velocity/Sanjo/Karma/PhantomJS hangs when Meteor auto-rebuilds your project
How great was my surprise and disbelief when I noticed that the Velocity html reported stopped refreshing the test results ..but only sometimes:
1a. (re)started my meteor app, page got autoupdated, all tests were run
1b. edited specs, page was not updated (good!), testrunner noticed it, tests ran, results displayed (cool!)
1c. edited apps code, page was autoupdated (goodb!) ..but tests were **not** re-ran, old results displayed (wha?)
2a. (re)started my meteor app, page got autoupdated, all tests were run (ok, so nothing broken)
2b. edited apps code, page was updated (good!) ..but tests **not** ran, old results displayed (see the pattern)
3a. (re)started my meteor app, page got autoupdated, all tests were run, everything fine
3b. edited specs - everything ok, tests ran
3c. edited specs - everything ok, tests ran
3d. edited specs - everything ok, tests ran
3e. edited apps code - app updated, tests **not** ran (yup)
3f. edited apps code - app updated, tests **not** ran
3g. edited specs - everything ok, tests **not** ran (what?!)
3h. edited specs - everything ok, tests **not** ran (...)
4a. (re)started my meteor app, page got autoupdated, all tests were run, everything fineNoticeable time later wasted on tracing the problem proved that it happens only when using PhantomJS and only when the change to the application code causes restart. I'm pretty sure it's not a fault of PhantomJS. In .meteor/local/log/jasmine-client-unit.log I've found some errors regarding EPERMs and startup crashes mentioning that Package is not defined. Seems that PhantomJS was unable to reload some of the files, maybe there were not properly deleted or properly replaced. I wasn't able to trace it yet.
As I said, I noticed this problem occurrs only*) when PhantomJS is used. When Chrome is used with its irritating window popping up, everything works fine*). I switched back to using Chrome and focused on hiding the window. Unfortunatelly my time was getting shorter and shorter. I tried to find the place when Karma actually starts the child process so I could use some old tricks to send a hide-window message to it, but first I've found karma-chrome-launcher module that finds the Chrome executable and builds the command line options. Hey, maybe Chrome has some cool commandline switch to hide the window?
*) it turned out to be completely not true. see next post on that
Hiding the Chrome window
After some searching, I learned that most opinions tell it's not possible and that I have to use some third-party tools to send a message to its window to hide it (yeah, echo!). I ignored that and searched for the list of Chrome's commandline options. Here it is in raw form and here it is in a nice table.
As you can see there, there are some promising flags!
// Does not automatically open a browser window on startup (used when
// launching Chrome for the purpose of hosting background apps).
const char kNoStartupWindow[] = "no-startup-window";
// Causes Chrome to launch without opening any windows by default. Useful if
// one wishes to use Chrome as an ash server.
const char kSilentLaunch[] = "silent-launch";I managed to successfuly run Chrome with --no-startup-window and indeed it launched without any windows. It looked like it launched properly, it spawned all typical children, but the website I tried to make it load inside didn't seem to be actually visited. It maybe possible that this headless mode is only for running apps and not for visiting sites headless*), but it looks very promising as the normal worker tree is set up, just no windows. Yet my time was getting even shorter (probably the counter already has rolled into negative values) so I have not investigated further.
The second option --silent-launch made chrome process very silent. I didn't notice any children spawned at all and the process exited promptly. I doubt it'll be usable for this case.
After I failed my attempts with these options, I turned to less sophisticated ways. On the bottom of the list there are two options:
// Specify the initial window position: --window-position=x,y
const char kWindowPosition[] = "window-position";
// Specify the initial window size: --window-size=w,h
const char kWindowSize[] = "window-size";I edited karma-chrome-launcher\index.js to include options to move it completely out of the working area:
return [
'--user-data-dir=' + this._tempDir,
'--no-default-browser-check',
'--no-first-run',
'--disable-default-apps',
'--disable-popup-blocking',
'--disable-translate',
'--window-position=-800,0', // <-- added
'--window-size=800,600' // <-- added
].concat(flags, [url])and sure it's no true headless, but still the window is finally out of my sight and tests are properly re-ran whenever I edit either the app or specs. Yeay! I'll probably push a change proposal so those options can be turned on/off on demand.
Solution revisited
After searching a little more, I found out that karma already has a way of passing custom parameters to its browsers, and thanks to it the change was narrowed and moved from karma-chrome-launcher to sanjo:jasmine that was building the karma configuration file.
You can find the patch here and if using that, don't forget to remove JASMINE_BROSER env variable, or set it to HiddenChrome:
JASMINE_BROWSER=HiddenChromeHappy coding with new "HiddenChrome" browser launcher.
final notes
It feels dirty though. I'd prefer really-hiding that window and/or using PhantomJS.
..and that was only the client-unit tests. client-integration, server-integration and server-unit yet to be tried.. I mean, if server-unit module gets fixed, it's officially said to be unstable now and it's advised to use server-integration mode instead to run them.
No comments:
Post a Comment