Browser memory benchmark (plus lies and damn lies)

AutoItOpera Recent browser memory usage benchmark from Dot Net Perls is generating buzz (as such things do). I tried to stay away (as I do) but Martin’s post on discrepancies in it from incorrect measuring of multithreaded Chrome mentioned Opera memory consumption settings and got me interested.

Since original test used beta versions of browsers and I am not too good with C# in which code was provided – I coded my own version in AutoIt.

Disclaimer

Browser memory consumption is very vague metric because of differences between actively using memory and wasting it that are frankly impossible to determine on end-user distribution. Still it can be interesting insight in how browsers work and affected by different settings.

Counting memory

It is inconvenient to do task all in one program so first script waits for launch of specific process and then counts and logs its memory usage every three seconds.

$process=$CmdLine[1]
$log=@ScriptDir&"\"&$process&".log"

Do
	Sleep(100)
Until ProcessExists($process)

While ProcessExists($process)
	$memory=ProcessGetStats($process)
	$memory=$memory[0]
	FileWriteLine($log,$memory)
	Sleep(3000)
WEnd

Since it will be called from main script – it gets name of process to watch as command line parameter.

Opening sites

As original test suggested I took top 150 sites from list provided by Alexa. So much Google, so much porn. Ahem.

Opt("WinTitleMatchMode",2)
$list="d:\list.txt"
$browser="c:\FirefoxPortable\FirefoxPortable.exe"
$path="c:\FirefoxPortable\"
$process="firefox.exe"
$window="Firefox"

$list=FileRead($list)
$list=StringSplit($list,@CRLF,1)

Setting up some variables needed and reading text file with links into array.

ShellExecute(@ScriptDir&"\memory.exe",$process)
ShellExecute($browser,"",$path)
Sleep(1000*30)
$s=0
For $i=1 To 150
	 ShellExecute($list[$i])
	 Sleep(Random(1000,10000))
	 $s+=1
	 If $s=30 Then
		WinActivate($window)
		WinWaitActive($window)
		For $j=0 To 29
			Send("^w")
			Sleep(Random(100,1000))
		Next
		$s=0
	 EndIf
Next
Sleep(1000*30)
WinClose($window)
Exit

Launch memory script (compiled), then browser itself and let it settle in.

Then main loop starts that opens links in new tabs (with random 1-10s pause between them) and closes all tabs every thirty links.

Then there is cool off period and browser is closed.

Test bed

I was lazy to set up complete clean environment on virtual machine. So I settled for:

  • my home desktop – Pentium Dual Core 2.6GHz, 2Gb RAM;
  • Opera@USB 9.64, clean install:
    • one run with automatic (default) memory cache;
    • one run with 60Mb memory cache;
  • Firefox Portable 3.0.11. clean install.

I was mostly interested in different Opera settings so Firefox is just for company. Tests run full auto while I had productive Tetris rounds on my N810.

Results

Opera auto

Opera 60MB

Firefox

Average, MB

260,6

171,2

135,82

Maximum, MB

453,84

303,89

245,06

browser_memory_benchmark

Overall

Huge difference between Opera on automatic and manual settings clearly show that high memory consumption it displays is not because of wasted memory. By default Opera uses very aggressive memory cache to keep a lot of data in memory so it can be quickly recalled as needed.

Even on largest fixed setting memory consumption instantly falls close to Firefox levels.

Script https://www.rarst.net/script/memory.au3

Script https://www.rarst.net/script/browsers.au3

Logs https://www.rarst.net/script/browsers.ods

Related Posts

15 Comments

  • This is really interesting to see, seems that Firefox has improved some way from it’s overly bloated younger days when it was said to “leak” memory.
    Your comparison results aren’t too far away from dot net perls.
    This must have taken some time! Stumbled. Nice work.

  • @Jonny

    Yeah, Firefox v3 uses memory in much more sane ways.

    Actually I was surprised how fast I was done. :) Half an hour to write and debug script and under hour to run tests (I was running them sequentially, not in bulk like original test).

  • BTW, why don’t you try Firefox with Automatic RAM cache disabled or with different settings.

  • Why did not you try Firefox 3.5, after all dotnetprl… has compared diff browsers with Firefox 3.5 Rc2

  • @Abhinav

    As I said Firefox is for company. :) I am long time Opera user so that is my primary interest and browser I cover most here in my blog.

    On version – I took latests stable releases. Which 3.0.11 is and 3.5RC2 is not. I know it’s hip lately to use cutting edge releases but it is stable ones that are most used and worth using for security and such.

  • I try my self to make a very raw test for probe memory consuming in Opera 10B and Firefox 3, actually I open 15 very content loaded web page at once, wait to fully load and see how much memory consume each browser, as the result Opera round to 170 to 190 MB, Firefox round to 90 to 120 MB. Opera consume more memory that Firefox but in all the test process, Firefox was very much CPU time consuming, reaching 90% of CPU, Opera only sometimes reach 50% and almost all the time was low 20% of CPU time. Rarst, maybe you can make a CPU consume comparison among browser and share your result. Sorry for my terrible English. Take care.

  • @Chocobito

    CPU consumption is interesting but harder to measure correctly because it comes in short spikes. It would require much higher poll rate (three seconds won’t do), that may itself affect OS.

    There is also factor of storage that may affect CPU usage. Or so I think.

    I will try to add CPU monitoring to script (probably on weekend) and will see if it makes enough sense to update post. Thanks for suggestion and sharing your experience!

  • I’m not sure exactly how you’re script works but couldn’t you use Sysinternals / Microsofts Process Explorer to monitor cpu over the runs?

  • @Jonny

    That would be visual but not too correct of methodic. Process Explorer can be quite resource hungry itself and without logging capabilities unfit as emasurement tool.

    It’s not a problem to log CPU usage, it’s problem to do it rapidly, accurately and without influencing system performance which might shift the very CPU usage being measured.

  • Hi Rarst,

    I am here after a long time.

    Nice autoit script.

    Will test on my Lappy.

  • @Nihar

    Glad to see you. :) You was missing longer than usual.

  • hey have you guys tried the GreenBrowser? http://www.morequick.com/indexen.htm

    It has been almost always using half of the memory of Firefox according to my experience

  • @Ben

    As I understand that one runs on IE engine. Since IE is quite integrated into Windows it is considered hard to single out real memory usage for it.

  • […] I did my browser memory benchmarks I had coded in Ctrl+W to close tabs. Worked like a charm in Opera. Worked like a charm in Firefox, […]

  • Browsers are from bad to worse and nobody takes care of !!! Developers developing development of wasted resources and somewhat browsers are more heavy as some FEA FEM 3D CAD simulation software packages ! The developers of browsers are going MAD and intentionally or not working for IT hardware industry. But do not be fooled into that if you buy a 6 core AMD 3 GHz on AMD mobo with 16 GB RAM that that solve the problem ! On that comp Opera browser chokes just about in the same manor as on some low hardware spec ! Nothing changes the problems get only worse !!! All branded browsers are not immune to this just testing Comodo IceDragon and it just crashes few times ! just about nothing changes ! Chrome browsers crash very rare for me just happens once but RAM problems and problematic notorious pathogenic pathological java script miss use is outrageous ! Without java web/browsers becomes unusable with it huge and tremendous problems. All these browsers are just i like to call them ”10 page browsers” – its somewhat threshold to choked OS ! If you use flash then you have shot
    your system into knee ! That that was a bsod for MS in the past are today browsers ! The best cure for not to put your nerves goes sky high is not to use internet at all ! Im surprised that no one is suing developers for crappy work ! I suggest that all web content is running on servers and not on users computers this way no one will waist resources as they do to day ! And let them buy supercomputers to serve web no us to browse internet !

Comments are closed.