IronPython Performance Report

An automated, lightweight comparison between 'IronPython 2.0' and 'Python25'
on 12/09/2008 19:28:28.

Machine Setup

  • PC
    • Manufacturer - Dell Computer Corporation
    • Model - PowerEdge 860
    • Name - MERLIN-14
  • Software
    • Operating System - Microsoft Windows NT 6.0.6001 Service Pack 1
    • CLR Version - 2.0.50727.3053
  • CPU
    • Full Name - Intel(R) Pentium(R) D CPU 2.80GHz
    • Speed - 2800
    • Number of Cores - 2
  • RAM - 2146070528
  • Disk Drive - WDC WD1600JS-75NCB3 ATA Device

PyStone 1.1

Background
PyStone is an extremely high-level benchmark distributed with CPython. In
short, running pystone.py under various implementations of Python will emit
the number of PyStones (i.e., iterations) per second that can be run on a given
machine. A higher PyStone result indicates better overall performance of a Python
interpreter. The latest copy of PyStone can be found in lib\test\pystone.py
(relative to CPython installation locations).

Notes:
  • By default pystone.py runs 50,000 passes to determine PyStones per second. The results below were generated using this default
  • Due to site caching in the Dynamic Language Runtime, IronPython performs better with more PyStone passes than the default value

Results
Best CPy PyStone run (out of 5 attempts): 37982 PyStones/second
Best IPy PyStone run (out of 5 attempts): 46198 PyStones/second

PyBench 2.0

Background
PyBench is a collection of low-level benchmarks distributed with CPython. Whereas
PyStone can be used to discover the overall performance of the Python interpreter,
PyBench targets specific areas (e.g., the amount of time it takes to concatenate
two strings). You can find the latest version of PyBench on CPython's Subversion
source control system (e.g., http://svn.python.org/projects/python/trunk/Tools/pybench/).

Please note the following:
  • Much of the platform module is broken under IPy. As such, build numbers, build dates, etc had to be faked under IPy
  • The IPy garbage collector cannot be disabled. Although this puts IPy at a disadvantage, we kept it turned off for the CPy run
  • sys.setcheckinterval does not function properly under IronPython. This is another optimization CPython gets that IronPython doesn't
  • The default number of rounds were used along with a warp factor of 1
  • In the table below, positive(+) "run-time diffs" indicate better performance under Python25 and negative (-) "run-time_diff"s imply IronPython 2.0 performs better

Results
(this=C:\SnapTemp\perfresults\pybench.ipy.results, other=C:\SnapTemp\perfresults\pybench.cpy.results)
Test min run-time this min run-time other min run-time diff avg run-time this avg run-time other avg run-time diff
BuiltinFunctionCalls: 481ms 2054ms -76.6% 506ms 2085ms -75.7%
BuiltinMethodLookup: 450ms 1829ms -75.4% 477ms 2025ms -76.5%
CompareFloats: 543ms 1100ms -50.6% 562ms 1105ms -49.2%
CompareFloatsIntegers: 356ms 1369ms -74.0% 378ms 1380ms -72.6%
CompareIntegers: 136ms 955ms -85.8% 148ms 970ms -84.7%
CompareInternedStrings: 207ms 1016ms -79.7% 227ms 1023ms -77.8%
CompareLongs: 1760ms 1168ms 50.8% 1820ms 1178ms 54.5%
CompareStrings: 351ms 1225ms -71.3% 373ms 1320ms -71.8%
CompareUnicode: 259ms 1345ms -80.7% 275ms 1361ms -79.8%
ConcatStrings: 4364ms 3159ms 38.2% 4570ms 3283ms 39.2%
ConcatUnicode: 2665ms 2847ms -6.4% 2810ms 2955ms -4.9%
CreateInstances: 761ms 1876ms -59.4% 786ms 1903ms -58.7%
CreateNewInstances: 1370ms 1559ms -12.1% 1399ms 1580ms -11.5%
CreateStringsWithConcat: 2083ms 1320ms 57.9% 2322ms 1330ms 74.6%
CreateUnicodeWithConcat: 792ms 1970ms -59.8% 870ms 1988ms -56.2%
DictCreation: 1151ms 1215ms -5.3% 1192ms 1226ms -2.8%
DictWithFloatKeys: 6328ms 3198ms 97.9% 6352ms 3210ms 97.8%
DictWithIntegerKeys: 1865ms 960ms 94.3% 1891ms 967ms 95.5%
DictWithStringKeys: 1912ms 1053ms 81.5% 1934ms 1060ms 82.4%
ForLoops: 390ms 795ms -50.9% 406ms 801ms -49.4%
IfThenElse: 322ms 1025ms -68.6% 346ms 1034ms -66.6%
ListSlicing: 3494ms 1535ms 127.7% 3593ms 1546ms 132.3%
NestedForLoops: 577ms 1174ms -50.8% 593ms 1212ms -51.1%
NormalClassAttribute: 3912ms 1231ms 217.6% 3930ms 1242ms 216.4%
NormalInstanceAttribute: 3303ms 1158ms 185.3% 3335ms 1172ms 184.6%
PythonFunctionCalls: 310ms 1598ms -80.6% 328ms 1620ms -79.7%
PythonMethodCalls: 3088ms 1982ms 55.8% 3122ms 2007ms 55.5%
Recursion: 683ms 2117ms -67.8% 701ms 2128ms -67.1%
SecondImport: 2042ms 1491ms 37.0% 2070ms 1509ms 37.2%
SecondPackageImport: 2073ms 1562ms 32.7% 2108ms 1589ms 32.7%
SecondSubmoduleImport: 2869ms 2090ms 37.3% 2911ms 2115ms 37.7%
SimpleComplexArithmetic: 512ms 1415ms -63.8% 549ms 1424ms -61.5%
SimpleDictManipulation: 1962ms 1146ms 71.2% 1994ms 1158ms 72.2%
SimpleFloatArithmetic: 417ms 1341ms -68.9% 436ms 1357ms -67.9%
SimpleIntFloatArithmetic: 480ms 912ms -47.4% 486ms 922ms -47.3%
SimpleIntegerArithmetic: 479ms 915ms -47.6% 491ms 921ms -46.7%
SimpleListManipulation: 1290ms 944ms 36.6% 1312ms 953ms 37.7%
SimpleLongArithmetic: 1027ms 1269ms -19.1% 1060ms 1286ms -17.5%
SmallLists: 1416ms 1821ms -22.2% 1443ms 1849ms -22.0%
SmallTuples: 1775ms 1614ms 10.0% 1830ms 1645ms 11.2%
SpecialClassAttribute: 3647ms 1277ms 185.4% 3686ms 1293ms 185.0%
SpecialInstanceAttribute: 3225ms 2118ms 52.3% 3252ms 2134ms 52.4%
StringMappings: 2992ms 8079ms -63.0% 3042ms 8413ms -63.8%
StringPredicates: 1129ms 2848ms -60.4% 1165ms 2928ms -60.2%
StringSlicing: 1398ms 1688ms -17.2% 1526ms 1697ms -10.1%
TryExcept: 26ms 889ms -97.1% 60ms 894ms -93.3%
TryRaiseExcept: 59261ms 1294ms 4478.9% 59381ms 1305ms 4449.3%
TupleSlicing: 1825ms 1607ms 13.6% 1874ms 1654ms 13.3%
UnicodeMappings: 2339ms 1280ms 82.8% 2363ms 1286ms 83.8%
UnicodePredicates: 1283ms 1595ms -19.5% 1297ms 1622ms -20.0%
UnicodeSlicing: 1243ms 2190ms -43.3% 1289ms 2224ms -42.0%
Totals: 138625ms 85218ms 62.7% 140869ms 86892ms 62.1%

Richards

Background
Much like PyStone, richards.py is a high-level benchmark intended to give the
Python user a single result indicating the overall "goodness" factor of a Python
interpreter. This benchmark is distributed with the PyPy interpreter, although
the original version was written in BCPL by Dr. Martin Richards at Cambridge
University. Lower "Average time per iteration" results indicate better performance.

Results
Average time per iteration for CPy (out of 5 attempts): 833
Average time per iteration for IPy (out of 5 attempts): 1291

Last edited Feb 23, 2009 at 3:46 PM by dfugate, version 6

Comments

mycall May 18, 2009 at 8:21 PM 
http://www.smallshire.org.uk/sufficientlysmall/2009/05/17/the-performance-of-python-jython-and-ironpython/

kfarmer Dec 22, 2008 at 11:56 PM 
Please don't use "real world" as an adjective -- it's a useless affectation that says nothing about the implementation. I've seen shipping code where try/catch blocks were the abusive norm. Yes, that is a known-probably-bad pattern, but the code was certainly in the "real world".

Perhaps you mean to say "any well-designed application".

Eloff Dec 11, 2008 at 10:46 PM 
I'd like to point out that TryRaiseExcept is not critical to the performance of any real world application I'm aware of. If you factor that out, IronPython wins PyBench with a paltry -5.4%, so CPython's seemingly good score there doesn't mean much.