Firefox 3 ultimate feature: Performance

Along with the long list of new features and improvements we can find in Firefox 3 betas so far, there is a series of performance improvements coming to Firefox 3 Beta 4.

I’ve finally managed to run a set of tests to see how Firefox 3 performance compares to Firefox 2’s.

JavaScript

As you may know, JavaScript is progressively becoming more important as we, users come to expect web applications to behave more like their desktop equivalents. For web developers, the top tool for doing this is JavaScript, the language used to program the different web page elements and in some cases, query the server for small pieces of data.

For Firefox, it is even more critical since the whole interface (extensions included) runs on JavaScript: open a context menu, a menu, a tab or a window, and there’s JavaScript at work all around connecting the underlying pieces.

So you will be glad to know JavaScript performance in Firefox 3 has almost tripled compared to Firefox 2, doubled compared to Beta 3, and has become the fastest engine among all current browsers including Opera 9.5 beta and Safari 3.0.4 (also in beta). This is according to SunSpider, a JavaScript benchmark tool designed by WebKit (the project that makes Safari’s engine) to measure JavaScript performance.

SunSpider test results

It must be noted, that as any benchmark tool it is not perfect or free of criticism: an engine will most probably perform better in one test than another.

It is also said that an engine or application could be tweaked to perform better in a particular well known test. I think if the test is well designed and effectively represents what affects most users, then tweaking those particular factors only makes sense. What would be the point of fixing some obscure interfaces if it’s not what real world applications will benefit from?

I for one will be happy to learn about more and better JavaScript tests to benchmark Firefox 2 and 3.

Memory footprint

Firefox 2 has fame for consuming lots of memory and most annoyingly eating more of it when it is supposed to be sleeping.

So I designed a simple test with a set of 9 web pages on Windows XP SP2, with an empty profile and a blank page set as home page. Measures were taken with Windows Task Manager. The test covers a few simple web pages with simple browsing. Then more complex pages are added keeping the first pages opened. A couple of tabs are then closed to see how much memory is freed and finally the browser is left running for an hour to see if memory consumption remains flat.

Here are the steps in case you want to reproduce:

1. Visited The New York Times, clicked on Technology section, open a couple of articles

2. Visited Digg, clicked on Upcoming and clicked on pages 2 and 3

3. CNET News, clicked on each of the top three stories. Get back to the homepage afte reach

4. To cover international support, I visited the Russian and

5. Chinese Editions of Wikipedia. I then clicked on the first link in the left sidebar on each and then went back to the main page.

A. I took a first measure here.

6. Visited this YouTube video, view the video and one more from the right related videos sidebar.

7. On Gmail, I logged in, composed an email with 2 large images (about 2MB each), sent it to myself, and then opened the received email.

8. On Google Docs, I created a new text document, wrote a five word sentence, made it bold, centered it, added a small 20K GIF image and saved the document.

9. I visisted Netvibes, visit all the default tabs, change the page title and added a short item to the todo list

B. Second measure here.

10. Then closed Netvibes and You Tube and waited 20 seconds for Firefox to do its cleanup.

C. Third measure here.

12. The I left it opened for about one hour.

D. Last measure here.

Results:

Measure Firefox 2.0.0.12 Firefox 3 Beta 3
A. Regular web pages 73 MB 80 MB
B. Web 2.0 web sites 149 MB 135 MB
C. Close a couple of pages 142 MB 123 MB
D. After 1 hour with no activity 181 MB 123 MB

As you can see, Firefox 3 shows better memory management as it freed more memory when a couple of tabs were closed and it kept memory usage flat when not in use.

This test is far from comprehensive: depending on your browsing habits you may have much more or less tabs opened at the same time. Also, memory piles up as you close and open new tabs and windows, with more and different plugins. So this is just a reference of what some very casual browsing takes on each version and most importantly shows the many memory management improvements Firefox 3 will introduce.

Startup time

My poor man’s approach to measure how long it takes for Firefox to launch, consisted of a batch file that ran the time command and then the tested browser set with a simple home page that prints the current time with one line of JavaScript code. By subtracting these times I got the loading time with roughly to the second precision I need.

I restarted my laptop before each test to get a clear cache and good cold start times. Then closed and restarted Firefox to see how much it benefits from the OS caching.

In summary I got the same results: 10 seconds for a cold start, 4 seconds for a restart. There may be some milliseconds there but that’s beyond what a regular user can perceive.

More

Starting today, Firefox nightly Windows builds have the benefit of profile guided optimization (PGO). With PGO, after Firefox is compiled (written code is made ready for the computer as binary code), the just generated binary is ran while another application monitors how it behaves and creates a profile of what parts of Firefox are more used and in what order. Then the code is compiled again but this time guided by the produced profile generating a new optimized build.

Ted Mielczarek, who led the effort to PGO-enable the Mozilla build process, says a 10% performance gain can already be seen in some of Mozilla’s performance metrics. Linux builds may get PGO after Beta 4 and hopefully Mac OS X builds wil also benefit from it before the final release.

Another improvement came a few weeks ago as told by Vladimir Vukicevic thanks to the discovery of an undocumented Mac OS X interface that will now help Firefox do some work instead of just waiting for its turn to draw, which translates to more efficiency.

Firefox 3 Beta 4 should be out within the next couple of weeks.

78 thoughts on “Firefox 3 ultimate feature: Performance”

  1. Pingback: Firefox 3 beta 5
  2. Pingback: mininglabs
  3. Sudhir,

    On second thought here I see you already found that solution (I found your article by searching your site). Not only did you find it but you listed the steps verbatim in your post without proper attribution. This is in rather bad form.

    Your statement about Firefox CPU usage and lack of stability is completely unfounded. By reading your article that you to attempted to link to in your comment, you state that:

    “The Firefox process CPU usage keeps varying between 2-3% and 6-7%.”

    This amount of CPU usage is excellent for Firefox or any other piece of software during use so I don’t understand your complaint.

    Proper attribution and research are required before you make any blanket statements about any piece of software.

  4. Pingback: Why Firefox 3
  5. I’m curious about these tests now. How about a new test using the following:

    Safari Webkit 3.2.1
    Latest Firefox 3
    Latest IE8

    I’m very curious how the latest webkit ranks.

Comments are closed.