A few weeks ago, I finalised some new code for a new section of an existing website. I always code websites using web standards and always test on the latest versions of Chrome (v10), Safari(v5), Opera(v11), Firefox(v3.6) and Internet Explorer(v8). In the case of the latter two, I test both the current version and the upcoming version. So the site is also run through its paces on Firefox 4 and Internet Explorer 9. As for mobile, I test on iPhone and iPad too. As you can see, it's a pretty comprehensive test. However we began getting customer feedback that they couldn't log in to the new section of the site.
I looked through the logs and couldn't even see that they'd made an attempt to login. Very weird. I went back to the tests and everything worked. Then some users were able to log in. But the majority couldn't. We started to get some screenshots and feedback and discovered that it was a browser issue. Internet Explorer 7 to be exact. The jQuery login popup we had implemented didn't render correctly on the old Microsoft browser. Hence it didn't work, and no entry was made in the logs, as the customer wasn't even able to submit the form.
I had a look at the analytics data for this group of users and found that 89% of them were using Internet Explorer 7. A browser not in my test suite but used by the vast majority of our users (of this part of the site). The fix was easy once we knew the problem. And all customers can now log in.
Suffice to say that Internet Explorer 7 is now part of our test suite. We don't test that things "look" the same. We just test that the site "works". Internet Explorer 7 has 5.7% of the worldwide market share, and Internet Explorer 6 has 3.8%.
But the main lesson learned, is that when developing for a closed group (like an intranet or a internet site that is locked to certain users), make sure you know the browser statistics data. Not only will it ensure that what you're developing works, but it also provides the opportunity to give them a better experience through harnessing the power of that particular browser.