IronPython Performance Report

An automated, lightweight comparison between 'IronPython 2.6' and 'Python26'
on 03/24/2009 16:44:46.

Machine Setup

  • PC
    • Manufacturer - Dell Computer Corporation
    • Model - PowerEdge 860
    • Name - MERLIN-14
  • Software
    • Operating System - Microsoft Windows NT 6.0.6001 Service Pack 1
    • CLR Version - 2.0.50727.3053
  • CPU
    • Full Name - Intel(R) Pentium(R) D CPU 2.80GHz
    • Speed - 2800
    • Number of Cores - 2
  • RAM - 2146070528
  • Disk Drive - WDC WD1600JS-75NCB3 ATA Device

HelloWorld

Background
HelloWorld is a typical hello world program we've written and is used to evaluate
the startup time of a Python interpreter. Lower startup times indicate better
performance.

Notes:
  • No command-line flags are passed to the Python interpreters

Results
Best Python26 HelloWorld run (out of 5 attempts): 0.1050716 seconds
Best IronPython 2.6 HelloWorld run (out of 5 attempts): 2.6514158 seconds

PyStone 1.1

Background
PyStone is an extremely high-level benchmark distributed with CPython. In
short, running pystone.py under various implementations of Python will emit
the number of PyStones (i.e., iterations) per second that can be run on a given
machine. A higher PyStone result indicates better overall performance of a Python
interpreter. The latest copy of PyStone can be found in lib\test\pystone.py
(relative to CPython installation locations).

Notes:
  • By default pystone.py runs 50,000 passes to determine PyStones per second. The results below were generated using this default
  • Due to site caching in the Dynamic Language Runtime, IronPython performs better with more PyStone passes than the default value

Results
Best Python26 PyStone run (out of 5 attempts): 39097.1 PyStones/second
Best IronPython 2.6 PyStone run (out of 5 attempts): 25843 PyStones/second

PyBench 2.0

Background
PyBench is a collection of low-level benchmarks distributed with CPython. Whereas
PyStone can be used to discover the overall performance of the Python interpreter,
PyBench targets specific areas (e.g., the amount of time it takes to concatenate
two strings). You can find the latest version of PyBench on CPython's Subversion
source control system (e.g., http://svn.python.org/projects/python/trunk/Tools/pybench/).

Please note the following:
  • Much of the platform module is broken under IPy. As such, build numbers, build dates, etc had to be faked under IPy
  • The IPy garbage collector cannot be disabled. Although this puts IPy at a disadvantage, we kept it turned off for the CPy run
  • sys.setcheckinterval does not function properly under IronPython. This is another optimization CPython gets that IronPython doesn't
  • The default number of rounds and a warp factor of 1 were used

Results
(this=C:\SnapTemp\IronPython 2.6, other=C:\SnapTemp\Python26)
Test min run-time this min run-time other min run-time diff avg run-time this avg run-time other avg run-time diff
BuiltinFunctionCalls: 342ms 1690ms -79.8% 585ms 1731ms -66.2%
BuiltinMethodLookup: 637ms 1809ms -64.8% 887ms 1984ms -55.3%
CompareFloats: 590ms 1081ms -45.4% 917ms 1094ms -16.2%
CompareFloatsIntegers: 388ms 1397ms -72.2% 627ms 1414ms -55.7%
CompareIntegers: 215ms 849ms -74.7% 773ms 863ms -10.5%
CompareInternedStrings: 334ms 1218ms -72.6% 819ms 1242ms -34.1%
CompareLongs: 685ms 1052ms -34.9% 963ms 1070ms -10.1%
CompareStrings: 446ms 1268ms -64.8% 772ms 1359ms -43.2%
CompareUnicode: 339ms 1133ms -70.1% 579ms 1157ms -50.0%
ConcatStrings: 4131ms 2878ms 43.6% 4836ms 2972ms 62.7%
ConcatUnicode: 2604ms 2289ms 13.8% 2917ms 2395ms 21.8%
CreateInstances: 871ms 2011ms -56.7% 998ms 2065ms -51.7%
CreateNewInstances: 1261ms 1531ms -17.6% 1385ms 1570ms -11.8%
CreateStringsWithConcat: 2178ms 1104ms 97.2% 2799ms 1126ms 148.6%
CreateUnicodeWithConcat: 871ms 1250ms -30.3% 1043ms 1272ms -18.0%
DictCreation: 1698ms 1053ms 61.3% 2174ms 1075ms 102.2%
DictWithFloatKeys: 2429ms 2955ms -17.8% 2774ms 2991ms -7.3%
DictWithIntegerKeys: 2303ms 860ms 167.9% 2746ms 876ms 213.3%
DictWithStringKeys: 2444ms 1048ms 133.1% 2893ms 1063ms 172.1%
ForLoops: 390ms 654ms -40.3% 2952ms 663ms +345.2%
IfThenElse: 229ms 875ms -73.8% 788ms 894ms -11.9%
ListSlicing: 3510ms 1472ms 138.4% 3610ms 1506ms 139.7%
NestedForLoops: 437ms 929ms -53.0% 2601ms 963ms +170.1%
NormalClassAttribute: 3819ms 1093ms 249.3% 4246ms 1115ms 280.7%
NormalInstanceAttribute: 3101ms 967ms 220.6% 3527ms 989ms 256.8%
PythonFunctionCalls: 404ms 1602ms -74.8% 616ms 1651ms -62.7%
PythonMethodCalls: 3431ms 1861ms 84.4% 3593ms 1912ms 88.0%
Recursion: 902ms 2244ms -59.8% 963ms 2283ms -57.8%
SecondImport: 2526ms 1246ms 102.7% 2695ms 1282ms 110.1%
SecondPackageImport: 2604ms 1327ms 96.3% 2759ms 1358ms 103.1%
SecondSubmoduleImport: 3462ms 1682ms 105.9% 3615ms 1731ms 108.9%
SimpleComplexArithmetic: 668ms 1310ms -49.0% 1002ms 1334ms -24.9%
SimpleDictManipulation: 2399ms 1131ms 112.2% 2863ms 1147ms 149.6%
SimpleFloatArithmetic: 558ms 1092ms -48.9% 988ms 1271ms -22.2%
SimpleIntFloatArithmetic: 543ms 837ms -35.1% 970ms 850ms +14.1%
SimpleIntegerArithmetic: 512ms 743ms -31.1% 944ms 758ms +24.6%
SimpleListManipulation: 1416ms 916ms 54.6% 1854ms 944ms 96.4%
SimpleLongArithmetic: 1044ms 1196ms -12.8% 1264ms 1212ms +4.3%
SmallLists: 1480ms 1604ms -7.8% 1949ms 1641ms +18.8%
SmallTuples: 1870ms 1348ms 38.7% 2817ms 1383ms 103.7%
SpecialClassAttribute: 3756ms 1075ms 249.5% 4184ms 1097ms 281.3%
SpecialInstanceAttribute: 3101ms 1192ms 160.2% 3523ms 1212ms 190.7%
StringMappings: 3040ms 2306ms 31.8% 3203ms 2369ms 35.2%
StringPredicates: 1456ms 1805ms -19.3% 1831ms 1887ms -3.0%
StringSlicing: 1446ms 1521ms -5.0% 1830ms 1554ms +17.8%
TryExcept: 27ms 576ms -95.4% 105ms 585ms -82.1%
TryRaiseExcept: 53927ms 1092ms 4838.7% 54266ms 1119ms 4748.5%
TupleSlicing: 1903ms 1499ms 26.9% 2036ms 1522ms 33.8%
UnicodeMappings: 2012ms 945ms 112.9% 2048ms 971ms 110.9%
UnicodePredicates: 1548ms 1303ms 18.8% 1842ms 1369ms 34.6%
UnicodeSlicing: 1213ms 1648ms -26.4% 1551ms 1689ms -8.1%
Totals: 133500ms 69568ms 91.9% 154519ms 71581ms 115.9%

Richards

Background
Much like PyStone, richards.py is a high-level benchmark intended to give the
Python user a single result indicating the overall "goodness" factor of a Python
interpreter. This benchmark is distributed with the PyPy interpreter, although
the original version was written in BCPL by Dr. Martin Richards at Cambridge
University. Lower "Average time per iteration" results indicate better performance.

Results
Best average time per iteration for Python26 (out of 5 attempts): 610.5
Best average time per iteration for IronPython 2.6 (out of 5 attempts): 1397.3

Last edited Mar 25, 2009 at 3:48 PM by dfugate, version 1

Comments

mycall May 8, 2009 at 10:22 PM 
(this=C:\SnapTemp\IronPython 2.6, other=C:\SnapTemp\Python26) is confusing. It would make more sense if it said this:

(left side=C:\SnapTemp\IronPython 2.6, right side=C:\SnapTemp\Python26)