My JavaScript book is out! Don't miss the opportunity to upgrade your beginner or average dev skills.

Wednesday, January 22, 2014

testardo - A Browser Agnostic JS Web Driver

you might think there are already so many solutions out there for testing browsers on real domains or real network interactions ... well, let me tell you why this is unfortunately not the reality until the point I show you ./testardo in action, bear with me please!

Selenium Ain't Portable

Despite being a wonderful project, Selenium WebDriver is great for Desktop but its Mobile Browser Version counter part is limited in terms of both supported platforms and supported browsers per platform.
In few words, Selenium is not suitable for all Mobile, Tablets, and Desktop browsers plus it requires knowledge of more than just JavaScript which is luckily, or unfortunately for others, the only programming language every browser understands today or did understand yesterday as well as the main one learned by Web Developers.

Simulators Are Liars

If you still trust simulators through your debugging tool, I've got a bad news for you: these are simply unreliable!
Starting from the real way events are dispatched and devices behaves compared with the browser engine you are testing on your Desktop, simulators might not be even able to give you a close scenario.
Look at this Chrome Bug to start with, for example, then realize also code behind your mobile browser is far from close to the one you are likely running on Desktop.
Not only unreliable then, simulators also give you a very fake idea of how performance will be, once landed on a real device.

Pay For Browser Testing ... No Thanks!

If testing is already something many companies and developers do not even invest some time that will inevitably pay back in both the short and the long term, imagine them investing time and money to have some test that will inevitably suffer cross browser and platform compatibility ... basically all available options I've considered and dropped these days.

Thanks Addy Anyway!

Addy Osmani has written excellent articles about testing, provided slides, examples, videos, but still there's not a complete solution.
While I really appreciate the work Addy is doing for the whole community about this field and for both client and server side world, I am pretty sure he would agree with me there's still not a simple to configure, simple to use, zero-stress config, and zero-spent money, testing solutions that satisfy the Web in its integrity, and for every old or modern device.

Episode IV - A New Hope

Considering Addy slides on Mobile Web Development Workflow, I will use same comparison points he used for those tools ... still talking about what testardo is, and what it can do at this stage.
  • [+] Free!
  • [+] Cross Platform
  • [+] Supports live reload
  • [+] Remote Inspect where supported
  • [+] Easily see how each change you save looks across devices
  • [+] Work with any modern AND not so modern Desktop and Mobile browser
  • [+] Does NOT Require you to install a package or extension
  • [+] Allows to match up scrolling and input
  • [+] Change urls and domain without needing to do that on devices
  • [-] Loads page into iframe
Long story short, almost every "plus" in Addy slides was actually a goal to reach with ./testardo, plus more:
  • [+] Loads external pages within a specific domain session
  • [+] Works and notifies even remotely
  • [+] If specified, it sends emails to the specified address and only once in order to not bother much if you are far away from "your free-time project" over the week end.
  • [+] It's entirely written in JavaScript and JavaScript is the only thing you need to know
  • [+] It allows you to inject before each test one or more external library as dependency (jQuery or others)
  • [+] It simulates as much as possible real DOM events triggered by users (note: at this time only the writing has been tested properly)
  • [+] It gives you an instant feedback with a green, yellow, or red with the eventual Error message notifying success, waiting for, or failure and the reason
  • [+] ... MOAR COMING ...
not convinced yet? Still no clue what is all this about?

testardo Live Demo

OK, the warning here is: take this video as a quick demo with a crappy setup as my home Wi-Fi router is, and considering testardo is at version 0.1.0 at the time I write this, thank you for your understanding :-)
So, here the video!

testardo - a browser and OS agnostic JS driver from Andrea Giammarchi on Vimeo.

Where, What, and How

So, everything is Open Source in Github and you are very welcome to contribute and help any way you can.
The single file you need is the built one but if you have node.js and npm installed, this is al you need to use testardo:
npm install -g testardo
After this, you should be able to run cross platform any test you want, where each test should be a module exports, similar to the one showed in this example.
I am trying to write a much better and complete documentation, right now you can find basically all you want to know and all you need inside the file itself, or simply running testardo without arguments.

Thank You All for your patience and Happy Testing!

12 comments:

Misha Reyzlin said...

Gonna try this one

Nimesh Gurung said...

Can you actually get hold of the specific DOM items and run test across all those multiple platforms. Because if you can then it's really awesome. I can see how that could save a lot of time.

Cristian Carlesso said...

What a good array of devices, if I were in your shoes I would have placed the devices in a triangle shaped position and wrote a test which causes devices to red/green somewhat at time with jingle bells tunes :D

Andrea Giammarchi said...

Nimesh you can run anything you want inside the test, including unit tests instead of integration tests, if needed.

You still need a server running in some port, but for this very specific reason, I am already considering a virtual-server mode that creates an empty page every time is pinged/reached to simplify the unit tests scenario/case.

Adam said...

This looks very promising. Are there plans to get it to work with PhantomJS ? It would be nice to be able to work in in with build tools like Jenkins.

Andrea Giammarchi said...

Adam there would be no value on running PhatnomJS against testardo, you can use Karma for that.

testardo is indeed not for unit tests but I won't exclude in the future the possibility to interact somehow with testardo through Jenkins.

Michal Kovář said...

Hi Andrea, Let me give you some friendly feedback.
Having played around with it I encountered the following problem. Testardo seems incapable of issuing requests other than GET, observed in server/main.js. There is no configuration option to sort it out. This is a problem since my server expects a POST request, causing the login to fail.

Andrea Giammarchi said...

I need to proxy all POST requests and it's something I've already planned to do via `sandbox.post(path, {prop}, onload)` so please stay tuned

Michal Kovář said...

Hi Andrea. Could you, please, give me some feedback regarding the proposed POST request functionality?

Andrea Giammarchi said...

I hope I'll find time to work on this the current week.

Michal Kovář said...

Hi Andrea, We are a company of 300 software developers in central Europe and I'd like you to know that there's much enthusiasm about the Testardo project among my colleagues. We would like to promote Testardo to be our main testing unit for all screens before they go to our clients.

In our company, much of the server-side communication happens on the POST basis so this has become the last bit where Testardo could improve.

Andrea, I don't mean to be pushy in any way. This is just to let you know that the project is receiving a lot of excitement around world.

Andrea Giammarchi said...

the POST landed in v 0.3.0 thanks for being patience