ruby Bench

Ruby Benchmark on Various Operating Systems

Operating Systems Tested

• NetBSD 5.0
• OpenBSD 4.5
• FreeBSD 8Beta2
• Linux Ubuntu 9
• OpenSolaris 2009/06
• DragonflyBSD 2.2.1

What was benchmarked?

• The Ruby Bench

Charts

• RubyBenchMarksOne is the initial run for 1.8.7 only on all six operating systems.
• RubyBenchMarksTwo is the initial run for 1.9.1
• RubyBenchMarksThree is the same as the first one. Ruby 1.8.7 run with FreeBSD 7.2 instead of with 8
• RubyBenchMarksFour is the same as the second one. Ruby 1.9 run with FreeBSD 7.2 instead of with 8

Details on Operating Systems tested:

All default installs plus the following changes.

FreeBSD:

• Rebuilt GENERIC without WITNESS/INVARIANTS
• FreeBSD retested with ln -s jk /etc/malloc.conf FreeBSDMalloc
• FreeBSD 7.2 vs 8.0 without invariants/witness FreeBSD7v8

OpenBSD

• single large root filesystem
• Softdeps was enabled

NetBSD

• NetBSD: had wapbl enabled for a single large / filesystem.

Ubuntu

• No changes from default install

OpenSolaris

• No changes from default install

DragonFlyBSD

• 2.4 was tried but install won't even boot without hitting db> on install. Even with safe mode or acpi disabled.

  mi_startup(7f5000,0,0,0,0) at mi_startup+0x92
  begin() at begin*0x42
  Debugger("panic")
  stopped at Debugger+0x34: movb  $0,in_Debugger,4260
  

• Hammer was chosen for filesystem format.
• Does not ship with MP capable kernel. Created one, but hangs during boot on this system.
• Should not be included given it was a UP kernel, but is here for completeness. Since 1.8.7 is not using native threads this should not be a problem.
• Not included in the 1.9 run since it uses native threads and thus would not be fit to compare with MP fast versions

-

Test System Specs:

• Test System: Dell PowerEdge 1750 Dual Core Xeon 2.8 Ghz 2 GB (DDR2 SDRAM) with HT disabled
• Each Operating system is installed on it's own 36GB drive.
• Each run was done after a cold boot

Ruby Details:

• All 1.8 tests results are from Ruby 1.8.7-p174 MRI compiled by hand with configure --prefix /tmp
• RubyGems 1.3.5 was used
• The mean value was plotted for each test. (each test is run five times)

Test Results:

• Raw data from the benchmarks can be taken from here

Chart Details:

• Charts are not shown for any of the tests where one or more OSs failed to complete them.
• Charts made from Gruff Ruby gem.

Anomalies:

• This chart has raised questions:
• 
• Debug from the bm_gc tests from netbsd
• Tprof output for 1.8 here
• Tprof output for 1.9 here
• Lockstat information for the 1.8.7 run here
• Lockstat information for the 1.9.1 run here
• Full ktrace of the single benchmark run. here.
• Code for bm_gc:

  Bench.run [500_000, 1_000_000, 3_000_000] do |n|
    a = []
      n.times { a << []} # use up some RAM
        n.times {[]}
  end
  

• Send suggestions/Hate mail to ober@mauthesis.com