javascript - Testing browser extensions -

i'm going write bunch of browser extensions (the same functionality each popular browser). hope, of code shared, i'm not sure yet. sure of extensions use native api. have not experience tdd/bdd, , thought it's time start folowing these ideas project.

the problem is, have no idea how handle it. should write different tests each browser? how far should go these tests? these extensions quite simple - data in local storage, refreshing page , listening through web sockets.

and observation why hard me - because there lot of behaviour, , not models, dependent on platform.

i practise 2 different ways of testing browser extensions:

• unit tests
• integration test

introduction

i use cross-browser youtube lyrics rob w extension example throughout answer. core of extension written in javascript , organized amd modules. build script generates extension files each browser. r.js, streamline inclusion of browser-specific modules, such 1 cross-origin http requests , persistent storage (for preferences), , module tons of polyfills ie.

the extension inserts panel lyrics played song on youtube, grooveshark , spotify. have no control on these third-party sites, need automated way verify extension still works well.

workflow

during development:

1. implement / edit feature, , write unit test if feature not trivial.
2. run unit tests see if broke. if wrong, go 1.
3. commit git.

before release:

1. run unit tests verify individual modules still working.
2. run integration tests verify extension whole still working.
3. bump versions, build extensions.
4. upload update official extension galleries , website (safari , ie extensions have hosted yourself) , commit git.

unit testing

i use mocha + expect.js write tests. don't test every method each module, ones matter. instance:

• the dom parsing method. dom parsing methods in wild (including jquery) flawed: external resources loaded , javascript executed.
  verify dom parsing method correctly parses dom without negative side effects.

• the preference module: verify data can saved , returned.

• my extension fetches lyrics external sources. these sources defined in separate modules. these definitions recognized , used infoprovider module, takes query, (black box), , outputs search results.

  • first test whether infoprovider module functions correctly.
  • then, each of 17 sources, pass pre-defined query source (with infoprovider) , verify results expected:
    • the query succeeds
    • the returned song title matches (by applying word similarity algorithm)
    • the length of returned lyrics fall inside expected range.

• whether ui not broken, e.g. clicking on close button.

these tests can run directly local server, or within browser extension. advantage of local server can edit test , refresh browser see results. if of these tests pass, run tests browser extension.
passing parameter debug build script, unit tests bundled extension.

running tests within web page not sufficient, because extension's environment may differ normal page. instance, in opera 12 extension, there's no global location object.

remark: don't include tests in release build. users don't take efforts report , investigate bugs, give low rating , "doesn't work". make sure extension functions without obvious bugs before shipping it.

summary

• view modules black boxes. don't care what's inside, long output matches expected or given input.
• start testing critical parts of extension.
• make sure tests can build , run easily, possibly in non-extension environment.
• don't forget run tests within extension's execution context, ensure there's no constraint or unexpected condition inside extension's context break code.

integration testing

i use selenium 2 test whether extension still works on youtube, grooveshark (3x) , spotify.

initially, used selenium ide record tests , see if worked. went well, until needed more flexibility: wanted conditionally run test depending on whether test account logged in or not. that's not possible default selenium ide (it's said possible flowcontrol plugin - haven't tried).

the selenium ide offers option export existing tests in other formats, including junit 4 tests (java). unfortunately, result wasn't satisfying. many commands not recognized.

so, abandoned selenium ide, , switched selenium.
note when search "selenium", find information selenium rc (selenium 1) , selenium webdriver (selenium 2). first old , deprecated, latter (selenium webdriver) should used new projects.

once discovered how documentation works, it's quite easy use.
prefer documentation @ project page, because it's concise (the wiki) , complete (the java docs).

if want started quickly, read getting started wiki page. if you've got spare time, through documentation @ seleniumhq, in particular selenium webdriver , webdriver: advanced usage.
selenium grid worth reading. feature allows distribute tests across different (virtual) machines. great if want test extension in ie8, 9 , 10, simultaneously (to run multiple versions of internet explorer, need virtualization).

automating tests nice. what's more nice? automating installation of extensions!
chromedriver , firefoxdriver support installation of extensions, seen in this example.

for safaridriver, i've written 2 classes install custom safari extension. i've published , sent in pr selenium, might available in future: https://github.com/seleniumhq/selenium/pull/87

the operadriver not support installation of custom extensions (technically, should possible though).
note advent of chromium-powered opera, old operadriver doesn't work more.

there's internet explorer driver, , 1 not allow 1 install custom extension. internet explorer doesn't have built-in support extensions. extensions installed through msi or exe installers, not integrated in internet explorer. so, in order automatically install extension in ie, need able silently run installer installs ie plugin. haven't tried yet.