What could be simpler? Perhaps nothing. What could be more useless? Again, perhaps nothing.

You can only count what the vendor wants you to see

If Mozilla wanted to do better than Microsoft on this report, we would have an easy path: stop fixing and disclosing bugs that we find in-house. It is well known that Microsoft redacts release notes for service packs and bundles fixes, sometimes meaning that you get a single vulnerability “counted” for, say, seven defects repaired. Or maybe you don’t hear about it at all, because it was rolled into SP2 and they didn’t make any noise about it.

We count every defect distinctly. We count the ones that Mozilla developers find in-house. We count the things we do to mitigate defects in other pieces of software, including Windows itself and other third-party plugins. We count memory behaviour that we think might be exploitable, even if no exploit has ever been demonstrated and the issue in question was found in-house. We open our bugs up after we’ve shipped fixes, so that people don’t have to take our word for our severity ratings.

While Microsoft’s senior technical staff are trying to get severity ratings dialed down (unsuccessfully; kudos to MSRC for sticking to their guns), we are consistently rounding our severities up when there’s any doubt at all.

More fixes means less security?

Even if the scales were the same, and we were living in a parallel universe in which Microsoft even approached Mozilla’s standards of transparency and disclosure, the logic is just baffling: Jeff is saying that Mozilla’s products are less secure than Microsoft’s because Mozilla fixed more bugs. By that measure, IE4 is even more secure, because there were no security bugs fixed in that time frame; bravo to Microsoft for that!

I use Microsoft’s software products myself; I’m typing this on a machine that’s running Vista, in fact. Not only am I pretty upset that we see Microsoft referencing this report without disclosing that it was written by a Microsoft director of Security Strategy, but I’m also concerned for my own safety. Do people in charge of security strategy at Microsoft really believe that aggressively concealing the count of fixes that do make it out makes a product more secure? Shouldn’t they be trying to fix more bugs, rather than writing reports that would “punish” them for actively improving the security of their users rather than hoping that defects aren’t found by someone who they can’t keep quiet?

Microsoft should be embarrassed to be associated with this sort of ridiculous “analysis”. We don’t pretend that hiding the rate of fixes improves our users’ security in any way, and we never will. We’re transparent and aggressive in dealing with security issues, and 130 million Firefox users are safer for it every day.