The Washington Post announced today the availability of a new report by consulting firm SmartWare on Firefox 2 and Microsoft Internet Explorer 7 anti-phishing features comparison. The bottom line: “Firefox 2 Phishing Protection is more effective than the Microsoft Phishing Filter in Internet Explorer 7.” Numbers are shown below and based on 1040 tested known phishing web sites:

Some may remember a similar comparison by consulting firm 3Sharp commissioned by Microsoft, released on late September. Well, this time it’s not very clear yet how much Mozilla (the Foundation or the Corporation) was involved in the process but an acknowledgement at the end of the report to people and organizations involved in the analysis, suggests Mozilla as an entity and not just as a project was directly involved.

There are some differences however between this and 3Sharp’s report. First, and as I noted then, 27% of phishing sites were obtained from a Hotmail database. 3Sharp cofounder, Paul Robicheaux dismissed this as a source of bias “because the feed [3Sharp] used wasn’t incorporated in the data feeds that Microsoft uses for the Phishing Filter”. It would make no sense for Microsoft to exclude such valuable information, so I still remain uncertain.

In comparison, the SmartWare report used PhishTank a public database of phishing sites reported by people on all platforms, all email clients, all web browsers and all web mail providers. Plus, the whole process and results were audited by iSec Partners, a security consulting firm.

The question is: How can these two reports provide so different results? I believe both parts must provide more details on how those particular samples were selected, as anything other than random would be skewed on either side. SmartWare’s report also says nothing about false positives (fingering a good site as phishing). However, more details on SmartWare numbers, are expected soon.

Update: Paul Robicheaux comments in this report at his blog.