IronPython Performance Report

An automated, lightweight comparison between 'IronPython 2.6' and 'Python26'
on 05/20/2009 10:57:54.

Machine Setup

  • PC
    • Manufacturer - Dell Computer Corporation
    • Model - PowerEdge 860
    • Name - MERLIN-14
  • Software
    • Operating System - Microsoft Windows NT 6.0.6001 Service Pack 1
    • CLR Version - 2.0.50727.3053
  • CPU
    • Full Name - Intel(R) Pentium(R) D CPU 2.80GHz
    • Speed - 2800
    • Number of Cores - 2
  • RAM - 2146070528
  • Disk Drive - WDC WD1600JS-75NCB3 ATA Device

HelloWorld

Background
HelloWorld is a typical hello world program we've written and is used to evaluate
the startup time of a Python interpreter. Lower startup times indicate better
performance.

Notes:
  • No command-line flags are passed to the Python interpreters

Results
Best Python26 HelloWorld run (out of 5 attempts): 0.110854 seconds
Best IronPython 2.6 HelloWorld run (out of 5 attempts): 1.9379787 seconds

PyStone 1.1

Background
PyStone is an extremely high-level benchmark distributed with CPython. In
short, running pystone.py under various implementations of Python will emit
the number of PyStones (i.e., iterations) per second that can be run on a given
machine. A higher PyStone result indicates better overall performance of a Python
interpreter. The latest copy of PyStone can be found in lib\test\pystone.py
(relative to CPython installation locations).

Notes:
  • By default pystone.py runs 50,000 passes to determine PyStones per second. The results below were generated using this default
  • Due to site caching in the Dynamic Language Runtime, IronPython performs better with more PyStone passes than the default value

Results
Best Python26 PyStone run (out of 5 attempts): 39526.6 PyStones/second
Best IronPython 2.6 PyStone run (out of 5 attempts): 46858.4 PyStones/second

PyBench 2.0

Background
PyBench is a collection of low-level benchmarks distributed with CPython. Whereas
PyStone can be used to discover the overall performance of the Python interpreter,
PyBench targets specific areas (e.g., the amount of time it takes to concatenate
two strings). You can find the latest version of PyBench on CPython's Subversion
source control system (e.g., http://svn.python.org/projects/python/trunk/Tools/pybench/).

Please note the following:
  • Much of the platform module is broken under IPy. As such, build numbers, build dates, etc had to be faked under IPy
  • The IPy garbage collector cannot be disabled. Although this puts IPy at a disadvantage, we kept it turned off for the CPy run
  • sys.setcheckinterval does not function properly under IronPython. This is another optimization CPython gets that IronPython doesn't
  • The default number of rounds and a warp factor of 1 were used

Results
(this=C:\SnapTemp\IronPython 2.6, other=C:\SnapTemp\Python26)
Test min run-time this min run-time other min run-time diff avg run-time this avg run-time other avg run-time diff
BuiltinFunctionCalls: 279ms 1688ms -83.5% 303ms 1725ms -82.5%
BuiltinMethodLookup: 528ms 1816ms -70.9% 541ms 1922ms -71.9%
CompareFloats: 341ms 1077ms -68.4% 361ms 1086ms -66.8%
CompareFloatsIntegers: 122ms 1370ms -91.1% 140ms 1382ms -89.9%
CompareIntegers: 214ms 853ms -74.8% 235ms 870ms -73.0%
CompareInternedStrings: 349ms 1207ms -71.0% 357ms 1233ms -71.0%
CompareLongs: 638ms 1058ms -39.7% 657ms 1078ms -39.1%
CompareStrings: 446ms 1274ms -65.0% 462ms 1371ms -66.3%
CompareUnicode: 339ms 1140ms -70.3% 353ms 1156ms -69.5%
ComplexPythonFunctionCalls: 2368ms 2365ms +0.1% 2402ms 2413ms -0.4%
ConcatStrings: 4396ms 2912ms 51.0% 4746ms 2965ms 60.1%
ConcatUnicode: 2619ms 2363ms 10.9% 2797ms 2484ms 12.6%
CreateInstances: 918ms 2029ms -54.8% 954ms 2060ms -53.7%
CreateNewInstances: 1277ms 1509ms -15.4% 1313ms 1549ms -15.3%
CreateStringsWithConcat: 2022ms 1120ms 80.5% 2354ms 1146ms 105.5%
CreateUnicodeWithConcat: 809ms 1269ms -36.2% 943ms 1293ms -27.1%
DictCreation: 1589ms 1035ms 53.5% 1647ms 1065ms 54.6%
DictWithFloatKeys: 2242ms 2956ms -24.2% 2287ms 2987ms -23.4%
DictWithIntegerKeys: 2038ms 843ms 141.8% 2057ms 858ms 139.6%
DictWithStringKeys: 2147ms 1044ms 105.7% 2165ms 1066ms 103.2%
ForLoops: 390ms 661ms -41.0% 402ms 674ms -40.3%
IfThenElse: 214ms 870ms -75.4% 237ms 882ms -73.1%
ListSlicing: 3479ms 1477ms 135.6% 3551ms 1506ms 135.7%
NestedForLoops: 437ms 1027ms -57.5% 452ms 1060ms -57.3%
NormalClassAttribute: 3819ms 1134ms 236.8% 3849ms 1148ms 235.1%
NormalInstanceAttribute: 3117ms 1010ms 208.5% 3140ms 1031ms 204.7%
PythonFunctionCalls: 451ms 1620ms -72.2% 468ms 1646ms -71.6%
PythonMethodCalls: 3275ms 1835ms 78.5% 3310ms 1864ms 77.5%
Recursion: 886ms 2209ms -59.9% 902ms 2242ms -59.8%
SecondImport: 2870ms 1242ms 131.0% 2888ms 1273ms 126.8%
SecondPackageImport: 2760ms 1295ms 113.2% 2804ms 1331ms 110.7%
SecondSubmoduleImport: 3696ms 1696ms 117.9% 3742ms 1738ms 115.3%
SimpleComplexArithmetic: 544ms 1343ms -59.5% 595ms 1379ms -56.8%
SimpleDictManipulation: 2244ms 1095ms 104.9% 2275ms 1115ms 104.1%
SimpleFloatArithmetic: 480ms 1365ms -64.8% 515ms 1415ms -63.6%
SimpleIntFloatArithmetic: 512ms 823ms -37.9% 534ms 843ms -36.7%
SimpleIntegerArithmetic: 512ms 754ms -32.1% 523ms 765ms -31.6%
SimpleListManipulation: 1369ms 908ms 50.7% 1394ms 926ms 50.5%
SimpleLongArithmetic: 1012ms 1210ms -16.3% 1042ms 1228ms -15.1%
SmallLists: 1480ms 1621ms -8.7% 1505ms 1662ms -9.5%
SmallTuples: 1792ms 1349ms 32.8% 1846ms 1393ms 32.5%
SpecialClassAttribute: 4084ms 1096ms 272.7% 4128ms 1108ms 272.5%
SpecialInstanceAttribute: 3179ms 1227ms 159.1% 3204ms 1251ms 156.2%
StringMappings: 2993ms 2352ms 27.2% 3029ms 2391ms 26.7%
StringPredicates: 1299ms 1810ms -28.2% 1315ms 1844ms -28.7%
StringSlicing: 1399ms 1483ms -5.6% 1485ms 1496ms -0.7%
TryExcept: 27ms 578ms -95.3% 44ms 585ms -92.4%
TryFinally: 996ms 1735ms -42.6% 1024ms 1763ms -41.9%
TryRaiseExcept: 54224ms 1093ms 4860.1% 54357ms 1105ms 4817.0%
TupleSlicing: 1872ms 1503ms 24.6% 1927ms 1516ms 27.1%
UnicodeMappings: 1997ms 950ms 110.2% 2019ms 959ms 110.5%
UnicodePredicates: 1375ms 1304ms 5.4% 1388ms 1332ms 4.1%
UnicodeSlicing: 1197ms 1647ms -27.3% 1250ms 1694ms -26.2%
WithFinally: 731ms 2202ms -66.8% 761ms 2266ms -66.4%
WithRaiseExcept: 78766ms 1658ms 4651.7% 79053ms 1740ms 4442.7%
Totals: 215160ms 78108ms 175.5% 218028ms 79879ms 172.9%

Richards

Background
Much like PyStone, richards.py is a high-level benchmark intended to give the
Python user a single result indicating the overall "goodness" factor of a Python
interpreter. This benchmark is distributed with the PyPy interpreter, although
the original version was written in BCPL by Dr. Martin Richards at Cambridge
University. Lower "Average time per iteration" results indicate better performance.

Results
Best average time per iteration for Python26 (out of 5 attempts): 611.7
Best average time per iteration for IronPython 2.6 (out of 5 attempts): 1340.23

Last edited May 20, 2009 at 7:02 PM by dfugate, version 1

Comments

No comments yet.