Firefox 3 ultimate feature: Performance

Along with the long list of new features and improvements we can find in Firefox 3 betas so far, there is a series of performance improvements coming to Firefox 3 Beta 4.

I’ve finally managed to run a set of tests to see how Firefox 3 performance compares to Firefox 2′s.

JavaScript

As you may know, JavaScript is progressively becoming more important as we, users come to expect web applications to behave more like their desktop equivalents. For web developers, the top tool for doing this is JavaScript, the language used to program the different web page elements and in some cases, query the server for small pieces of data.

For Firefox, it is even more critical since the whole interface (extensions included) runs on JavaScript: open a context menu, a menu, a tab or a window, and there’s JavaScript at work all around connecting the underlying pieces.

So you will be glad to know JavaScript performance in Firefox 3 has almost tripled compared to Firefox 2, doubled compared to Beta 3, and has become the fastest engine among all current browsers including Opera 9.5 beta and Safari 3.0.4 (also in beta). This is according to SunSpider, a JavaScript benchmark tool designed by WebKit (the project that makes Safari’s engine) to measure JavaScript performance.

SunSpider test results

It must be noted, that as any benchmark tool it is not perfect or free of criticism: an engine will most probably perform better in one test than another.

It is also said that an engine or application could be tweaked to perform better in a particular well known test. I think if the test is well designed and effectively represents what affects most users, then tweaking those particular factors only makes sense. What would be the point of fixing some obscure interfaces if it’s not what real world applications will benefit from?

I for one will be happy to learn about more and better JavaScript tests to benchmark Firefox 2 and 3.

Memory footprint

Firefox 2 has fame for consuming lots of memory and most annoyingly eating more of it when it is supposed to be sleeping.

So I designed a simple test with a set of 9 web pages on Windows XP SP2, with an empty profile and a blank page set as home page. Measures were taken with Windows Task Manager. The test covers a few simple web pages with simple browsing. Then more complex pages are added keeping the first pages opened. A couple of tabs are then closed to see how much memory is freed and finally the browser is left running for an hour to see if memory consumption remains flat.

Here are the steps in case you want to reproduce:

1. Visited The New York Times, clicked on Technology section, open a couple of articles

2. Visited Digg, clicked on Upcoming and clicked on pages 2 and 3

3. CNET News, clicked on each of the top three stories. Get back to the homepage afte reach

4. To cover international support, I visited the Russian and

5. Chinese Editions of Wikipedia. I then clicked on the first link in the left sidebar on each and then went back to the main page.

A. I took a first measure here.

6. Visited this YouTube video, view the video and one more from the right related videos sidebar.

7. On Gmail, I logged in, composed an email with 2 large images (about 2MB each), sent it to myself, and then opened the received email.

8. On Google Docs, I created a new text document, wrote a five word sentence, made it bold, centered it, added a small 20K GIF image and saved the document.

9. I visisted Netvibes, visit all the default tabs, change the page title and added a short item to the todo list

B. Second measure here.

10. Then closed Netvibes and You Tube and waited 20 seconds for Firefox to do its cleanup.

C. Third measure here.

12. The I left it opened for about one hour.

D. Last measure here.

Results:

Measure Firefox 2.0.0.12 Firefox 3 Beta 3
A. Regular web pages 73 MB 80 MB
B. Web 2.0 web sites 149 MB 135 MB
C. Close a couple of pages 142 MB 123 MB
D. After 1 hour with no activity 181 MB 123 MB

As you can see, Firefox 3 shows better memory management as it freed more memory when a couple of tabs were closed and it kept memory usage flat when not in use.

This test is far from comprehensive: depending on your browsing habits you may have much more or less tabs opened at the same time. Also, memory piles up as you close and open new tabs and windows, with more and different plugins. So this is just a reference of what some very casual browsing takes on each version and most importantly shows the many memory management improvements Firefox 3 will introduce.

Startup time

My poor man’s approach to measure how long it takes for Firefox to launch, consisted of a batch file that ran the time command and then the tested browser set with a simple home page that prints the current time with one line of JavaScript code. By subtracting these times I got the loading time with roughly to the second precision I need.

I restarted my laptop before each test to get a clear cache and good cold start times. Then closed and restarted Firefox to see how much it benefits from the OS caching.

In summary I got the same results: 10 seconds for a cold start, 4 seconds for a restart. There may be some milliseconds there but that’s beyond what a regular user can perceive.

More

Starting today, Firefox nightly Windows builds have the benefit of profile guided optimization (PGO). With PGO, after Firefox is compiled (written code is made ready for the computer as binary code), the just generated binary is ran while another application monitors how it behaves and creates a profile of what parts of Firefox are more used and in what order. Then the code is compiled again but this time guided by the produced profile generating a new optimized build.

Ted Mielczarek, who led the effort to PGO-enable the Mozilla build process, says a 10% performance gain can already be seen in some of Mozilla’s performance metrics. Linux builds may get PGO after Beta 4 and hopefully Mac OS X builds wil also benefit from it before the final release.

Another improvement came a few weeks ago as told by Vladimir Vukicevic thanks to the discovery of an undocumented Mac OS X interface that will now help Firefox do some work instead of just waiting for its turn to draw, which translates to more efficiency.

Firefox 3 Beta 4 should be out within the next couple of weeks.

78 Comments

  1. | Tech Freaks 4 You March 17, 2008 10:05 am 

    [...] from Computer World “According to Percy Cabello, on the Mozilla Links blog, Firefox 3.0 Beta 4 is 53 percent faster than Opera 9.5 beta, twice as fast as Apple’s Safari [...]

  2. Firefox 3 - cada vez mais rápido e melhor | Open2Tech March 17, 2008 7:10 pm 

    [...] O Firefox 3, ainda em suas versões beta (atualmente está no beta 4), já demonstra importantes melhorias no tocante à velocidade quando comparado com outros navegadores e também com suas versões anteriores. A versão 3 do navegador da Mozilla apresenta melhorias importantíssimas em diversos quesitos, até mesmo em relação ao consumo de memória, motivo de reclamações desde sua versão 1.x. Parece que finalmente o pessoal da Mozilla conseguiu “abrandar” um pouco a “fome” do navegador em sua versão 3, conforme teste efetuado pelo site Mozilla Links. [...]

  3. Firefox 3 Beta 5 is here, and reviewed : Mozilla Links April 2, 2008 1:47 pm 

    [...] it properly. This and other memory oriented tweaks, seem to have paid off so far: a set of tests I recently ran suggests a noticeable increase in memory management efficiency with more memory freed as tabs and [...]

  4. Firefox 3 beta 5 April 2, 2008 6:48 pm 

    [...] en particular (ojo que además yo tengo 64bits),…me he tirado a la piscina!!!, pero es que con los resultado de rendimiento que está dando no me he pedido resistir. Ya veremos a ver si me lo tengo que instalar. Por el [...]

  5. Mathew Hartnell April 24, 2008 11:57 am 

    Is there any reason you didn’t include Safari (which claims to be the fastest mainstream browser) in your test?

  6. Mozilla en la web 3.0 « Try Marketing May 14, 2008 12:05 pm 

    [...] encontrar los detalles de las pruebas de benchmark realizadas con SunSpider en el blog de Percy Cabello. Firefox beta 4 es tres veces mas velóz que su predecesor Firefox 2 y el más rapido de todos los [...]

  7. Firefox 3 RC 1 full review - Mozilla Links May 17, 2008 10:43 am 

    [...] it properly. This and other memory oriented tweaks, seem to have paid off so far: a set of tests I recently ran suggests a noticeable increase in memory management efficiency with more memory freed as tabs and [...]

  8. Firefox 3 is Arriving Shortly - Gear Up! | Techtracer May 25, 2008 1:42 pm 

    [...] the reports on comparisons of the performance of various browsers Firefox 3 has passed out with flying colors. Also the new version sports a more native look and feel of the underlying OS. This means, but [...]

  9. Firefox 3 RC 2 review - Mozilla Links June 4, 2008 10:19 pm 

    [...] it properly. This and other memory oriented tweaks, seem to have paid off so far: a set of tests I recently ran suggests a noticeable increase in memory management efficiency with more memory freed as tabs and [...]

  10. Firefox 3 RC 2 review - Mozilla Links June 4, 2008 10:19 pm 

    [...] it properly. This and other memory oriented tweaks, seem to have paid off so far: a set of tests I recently ran suggests a noticeable increase in memory management efficiency with more memory freed as tabs and [...]

  11. Uutisblogi - Mozilla.fi June 7, 2008 3:45 pm 

    [...] Merkittävästi nopeampi toiminnassa ja säästeliäämpi muistin kanssa: 3:ssa Firefoxin JavaScript-moottoria on viritetty huimasti. Mittausten mukaan moni toiminto onnistuu kertaluokkaa nopeammin verrattuna Firefox 2:een. Muistinkulutus on testeissä tippunut jopa kymmenyksiä verrattuna Firefox 2:een (molemmista englanniksi MozillaLinksissä). [...]

  12. mininglabs June 16, 2008 11:30 am 

    Firefox 3: an empirical performance study…

    In the vein of what’s been done for openoffice, the imminent release of firefox 3 gave us the idea to try to test the performance improvements in the version 3.0 compared to the various firefox browser versions. Since it’s our preferred …

  13. Kirk M June 28, 2008 9:47 am 

    Sudhir,

    On second thought here I see you already found that solution (I found your article by searching your site). Not only did you find it but you listed the steps verbatim in your post without proper attribution. This is in rather bad form.

    Your statement about Firefox CPU usage and lack of stability is completely unfounded. By reading your article that you to attempted to link to in your comment, you state that:

    “The Firefox process CPU usage keeps varying between 2-3% and 6-7%.”

    This amount of CPU usage is excellent for Firefox or any other piece of software during use so I don’t understand your complaint.

    Proper attribution and research are required before you make any blanket statements about any piece of software.

  14. Especial Firefox 3: más velocidad al mismo precio June 29, 2008 7:42 am 

    [...] basándonos en lo que nos cuentan distintas webs donde han hecho estudios basados en varias versiones de Firefox 3, lo que se puede concluir es que hay una mejora más que notable con respecto a la versión [...]

  15. Why Firefox 3 August 10, 2008 12:02 am 

    [...] the wrinkles, they have been ironed out in the new release. Studies conducted by various sources over recent months show massive performance gains over previous versions of Firefox as well [...]

  16. Z-Blog | Zemanta Ltd. August 26, 2008 12:02 pm 

    [...] If you are interested in technical side of this, check out this great article - Firefox 3 ultimate feature: Performance [...]

  17. rss blog » Blog Archive » Netvibes (heart) Firefox 3 August 29, 2008 9:15 am 

    [...] here) Firefox 3 is definitely faster and lighter than the current Firefox version. So far, tests using the SunSpider framework have shown Javascript, which Netvibes is intensively using, to be running up to 3 times faster [...]

  18. Phill Kenoyer February 20, 2009 3:06 am 

    I’m curious about these tests now. How about a new test using the following:

    Safari Webkit 3.2.1
    Latest Firefox 3
    Latest IE8

    I’m very curious how the latest webkit ranks.

Comments are closed.