Browser Testing - Coverage

What would you consider as an adequate coverage for browser testing? For example, if you have data that says that 90% of the visitors to your site are using two browsers would you invest the time/money to test in the other browsers? I'll go ahead and tell you that I'm a believer in the 80/20 principle so you know where I'm coming from. What say you, oh keepers of knowledge?


As I work in a testing role I have never had to make the call, all I have had to do is present the facts that it will take x time to test 90% of browsers, y time for 95% and z time for 99%.

Having said that I would say potentially losing 10% of users is very high and will probably generate bad publicity for you company. As always it depends on what the website is trying to do. If it is a website for a local squash club giving out information only, you probably do not care about other browsers. If it is a commerce website then it is important to make it accessible to all - It is up to you/business to decided if the invested time/effort is worth either the extra commerce coming back or avoiding the bad publicity.

ou are looking to toss in a third browser, only, then I would go cross-platform and hit on Safari. MACs use a different color pallette (CYMK), so any hex colors you are defining, unless web safe, may appear differently in the MAC.

Keep in mind, also, that there is a lot of stuff happening in the background, on your own servers, which does not apply to the browser itself, so this could be a significant chunk cut away from your testing on other browsers.

Those are my three main browsers that I test when doing a web app. If I get more time then I will hit on Opera, Konqueror, and I'll also try to use mobile browsers where possible.

Also, remember that you have differnet versions within each that you will be hitting. So for IE, you might be looking at testing IE 5, 6, and 7 (with recent service packs for that version), Firefox 1.2, 2, and Safari 1 and 2.

If you're looking for 80/20 coverage, then you may only need to test IE and Firefox. That will probably do it. I have heard people say, though, that they prefer to test IE on PC and Safari on MAC when they are looking for a quick fix. This sounds good, in theory, because Safari is built on the Mozilla codebase, which Firefox is also built on, but they are completely different beasts. I wouldn't recommend it.

If you are adhering to 80/20, and your site isn't specifically geared toward MAC users, then I would stick with your IE and Firefox (both on PC) and leave the MAC users on the side of the road with their suitcases for the time being (sorry MAC lovers). However, if given any additional time, I would go straight to MAC. Also, note that Firefox is great at updating itself, and Firefox users are generally more conscious of updates, so you could even sacrifice testing previous versions of Firefox and test the most recent versions of both Firefox and Safari. That's cheating, but if you are forced to make that decision, that's where I'd go.

How to Test look and feel ?

Is a tool actually entirely necessary? All you actually need is to decide which platform is your baseline and then open the site using each of the browsers for comparison.
There are many ways in which you can do this, if you really have to do it. From your description, I am assuming that from look and feel you mean to say your web application is working properly on different browsers or not. Now if you want to make sure that all the objects/page-elements are available on the different browsers you can use tools like Selenium Remote Control ( Brief overview at testinggeek.com/seleniumrc.asp) but if you also want to find out that rendering is proper you need to find tools which support image comparison. But as Martin has suggested, do not do it if you do not do it. Hope it helps.

Well fisrt of all, you may want to consider some sort of virtualization (like using VMWare) so that you can set up for different screen resolutions and browser versions. Remember that while we are at IE7, IE6 is still very common and will probably still be in the majority, so this is another requirement.

I am actually using TestComplete right now myself and it allows you to take a snapshot of certain areas of the screen. It also allows you to do pixel comparisons, so you can confirm whether colors defined in your stylesheets, etc., are appearing the same across all browsers.

If you're looking for something to automate the confirmation of the layout design then this would be a good choice, and it's also cheap. They have like a 30-day trial too, so you can check and make sure it'll work for your application.