IronPython Performance Report

An automated, lightweight comparison between 'IronPython 2.0.1' and 'IronPython 2.0'
on 02/12/2009 10:11:08.

Machine Setup

  • PC
    • Manufacturer - Dell Computer Corporation
    • Model - PowerEdge 860
    • Name - MERLIN-14
  • Software
    • Operating System - Microsoft Windows NT 6.0.6001 Service Pack 1
    • CLR Version - 2.0.50727.3053
  • CPU
    • Full Name - Intel(R) Pentium(R) D CPU 2.80GHz
    • Speed - 2800
    • Number of Cores - 2
  • RAM - 2146070528
  • Disk Drive - WDC WD1600JS-75NCB3 ATA Device

PyStone 1.1

Background
PyStone is an extremely high-level benchmark distributed with CPython. In
short, running pystone.py under various implementations of Python will emit
the number of PyStones (i.e., iterations) per second that can be run on a given
machine. A higher PyStone result indicates better overall performance of a Python
interpreter. The latest copy of PyStone can be found in lib\test\pystone.py
(relative to CPython installation locations).

Notes:
  • By default pystone.py runs 50,000 passes to determine PyStones per second. The results below were generated using this default
  • Due to site caching in the Dynamic Language Runtime, IronPython performs better with more PyStone passes than the default value

Results
Best IronPython 2.0 PyStone run (out of 5 attempts): 46464.2 PyStones/second
Best IronPython 2.0.1 PyStone run (out of 5 attempts): 51843.8 PyStones/second

PyBench 2.0

Background
PyBench is a collection of low-level benchmarks distributed with CPython. Whereas
PyStone can be used to discover the overall performance of the Python interpreter,
PyBench targets specific areas (e.g., the amount of time it takes to concatenate
two strings). You can find the latest version of PyBench on CPython's Subversion
source control system (e.g., http://svn.python.org/projects/python/trunk/Tools/pybench/).

Please note the following:
  • Much of the platform module is broken under IPy. As such, build numbers, build dates, etc had to be faked under IPy
  • The IPy garbage collector cannot be disabled. Although this puts IPy at a disadvantage, we kept it turned off for the CPy run
  • sys.setcheckinterval does not function properly under IronPython. This is another optimization CPython gets that IronPython doesn't
  • The default number of rounds and a warp factor of 1 were used

Results
(this=C:\SnapTemp\IronPython 2.0.1, other=C:\SnapTemp\IronPython 2.0)
Test min run-time this min run-time other min run-time diff avg run-time this avg run-time other avg run-time diff
BuiltinFunctionCalls: 278ms 466ms -40.3% 302ms 512ms -41.0%
BuiltinMethodLookup: 434ms 434ms -0.0% 461ms 492ms -6.3%
CompareFloats: 544ms 544ms -0.0% 562ms 581ms -3.2%
CompareFloatsIntegers: 91ms 357ms -74.5% 99ms 402ms -75.4%
CompareIntegers: 121ms 121ms -0.0% 143ms 146ms -2.1%
CompareInternedStrings: 223ms 207ms +7.5% 229ms 234ms -2.0%
CompareLongs: 1714ms 1760ms -2.7% 1787ms 1923ms -7.1%
CompareStrings: 352ms 367ms -4.1% 367ms 385ms -4.7%
CompareUnicode: 275ms 275ms -0.0% 285ms 308ms -7.6%
ConcatStrings: 4255ms 4286ms -0.7% 4528ms 4681ms -3.3%
ConcatUnicode: 2540ms 2509ms +1.2% 2749ms 2820ms -2.5%
CreateInstances: 762ms 761ms 0.1% 798ms 791ms 0.9%
CreateNewInstances: 1324ms 1355ms -2.3% 1345ms 1394ms -3.5%
CreateStringsWithConcat: 2115ms 2021ms 4.6% 2405ms 2338ms 2.9%
CreateUnicodeWithConcat: 840ms 824ms 1.9% 910ms 887ms 2.6%
DictCreation: 1230ms 1136ms 8.2% 1250ms 1208ms 3.5%
DictWithFloatKeys: 2147ms 6266ms -65.7% 2166ms 6339ms -65.8%
DictWithIntegerKeys: 1912ms 1865ms 2.5% 1929ms 1890ms 2.1%
DictWithStringKeys: 1974ms 1927ms 2.4% 1990ms 1963ms 1.3%
ForLoops: 390ms 406ms -3.8% 404ms 413ms -2.3%
IfThenElse: 322ms 307ms 5.1% 349ms 339ms 2.8%
ListSlicing: 3526ms 3479ms +1.3% 3596ms 3646ms -1.4%
NestedForLoops: 593ms 577ms +2.7% 594ms 596ms -0.3%
NormalClassAttribute: 4099ms 3897ms 5.2% 4115ms 3930ms 4.7%
NormalInstanceAttribute: 3101ms 3304ms -6.1% 3137ms 3356ms -6.5%
PythonFunctionCalls: 310ms 310ms -0.0% 324ms 334ms -2.8%
PythonMethodCalls: 2994ms 3072ms -2.5% 3030ms 3141ms -3.5%
Recursion: 652ms 668ms -2.3% 685ms 679ms +0.9%
SecondImport: 2214ms 2043ms 8.4% 2252ms 2076ms 8.5%
SecondPackageImport: 2292ms 2074ms 10.5% 2372ms 2129ms 11.4%
SecondSubmoduleImport: 3026ms 2854ms 6.0% 3107ms 2904ms 7.0%
SimpleComplexArithmetic: 622ms 512ms 21.3% 667ms 537ms 24.1%
SimpleDictManipulation: 2119ms 1931ms 9.7% 2189ms 1966ms 11.3%
SimpleFloatArithmetic: 402ms 402ms -0.0% 434ms 434ms -0.0%
SimpleIntFloatArithmetic: 464ms 480ms -3.2% 500ms 497ms +0.6%
SimpleIntegerArithmetic: 464ms 480ms -3.3% 475ms 505ms -5.9%
SimpleListManipulation: 1290ms 1290ms -0.0% 1310ms 1324ms -1.1%
SimpleLongArithmetic: 1012ms 1028ms -1.5% 1058ms 1064ms -0.6%
SmallLists: 1386ms 1386ms 0.0% 1441ms 1425ms 1.1%
SmallTuples: 1760ms 1775ms -0.9% 1817ms 1830ms -0.7%
SpecialClassAttribute: 4099ms 3632ms 12.9% 4181ms 3674ms 13.8%
SpecialInstanceAttribute: 3023ms 3211ms -5.8% 3078ms 3276ms -6.0%
StringMappings: 3024ms 2977ms 1.6% 3100ms 3050ms 1.7%
StringPredicates: 1051ms 1129ms -6.9% 1096ms 1161ms -5.6%
StringSlicing: 1367ms 1367ms +0.1% 1503ms 1524ms -1.4%
TryExcept: 26ms 26ms 0.0% 62ms 60ms 2.6%
TryRaiseExcept: 58357ms 58998ms -1.1% 58685ms 59230ms -0.9%
TupleSlicing: 1825ms 1825ms -0.0% 1852ms 1883ms -1.7%
UnicodeMappings: 2339ms 2339ms -0.0% 2381ms 2355ms +1.1%
UnicodePredicates: 1221ms 1268ms -3.7% 1255ms 1292ms -2.9%
UnicodeSlicing: 1165ms 1196ms -2.5% 1236ms 1278ms -3.4%
Totals: 133667ms 137725ms -2.9% 136591ms 141202ms -3.3%

Richards

Background
Much like PyStone, richards.py is a high-level benchmark intended to give the
Python user a single result indicating the overall "goodness" factor of a Python
interpreter. This benchmark is distributed with the PyPy interpreter, although
the original version was written in BCPL by Dr. Martin Richards at Cambridge
University. Lower "Average time per iteration" results indicate better performance.

Results
Average time per iteration for IronPython 2.0 (out of 5 attempts): 1291.7
Average time per iteration for IronPython 2.0.1 (out of 5 attempts): 1238.66

Last edited Feb 23, 2009 at 3:45 PM by dfugate, version 2

Comments

TraumaPony Feb 14, 2009 at 4:51 AM 
Wow, complex arithemetic got owned.