IronPython Performance Report

An automated, lightweight comparison between 'IronPython 2.6' and 'Python26'
on 09/18/2009 10:26:48.

Machine Setup

  • PC
    • Manufacturer - Dell Computer Corporation
    • Model - PowerEdge 860
    • Name - MERLIN-14
  • Software
    • Operating System - Microsoft Windows NT 6.0.6001 Service Pack 1
    • CLR Version - 2.0.50727.3053
  • CPU
    • Full Name - Intel(R) Pentium(R) D CPU 2.80GHz
    • Speed - 2800
    • Number of Cores - 2
  • RAM - 2146070528
  • Disk Drive - WDC WD1600JS-75NCB3 ATA Device

HelloWorld

Background
HelloWorld is a typical hello world program we've written and is used to evaluate
the startup time of a Python interpreter. Lower startup times indicate better
performance.

Notes:
  • No command-line flags are passed to the Python interpreters

Results
Best Python26 HelloWorld run (out of 5 attempts): 0.121812 seconds
Best IronPython 2.6 HelloWorld run (out of 5 attempts): 2.1516179 seconds

PyStone 1.1

Background
PyStone is an extremely high-level benchmark distributed with CPython. In
short, running pystone.py under various implementations of Python will emit
the number of PyStones (i.e., iterations) per second that can be run on a given
machine. A higher PyStone result indicates better overall performance of a Python
interpreter. The latest copy of PyStone can be found in lib\test\pystone.py
(relative to CPython installation locations).

Notes:
  • By default pystone.py runs 50,000 passes to determine PyStones per second. The results below were generated using this default
  • Due to site caching in the Dynamic Language Runtime, IronPython performs better with more PyStone passes than the default value

Results
Best Python26 PyStone run (out of 5 attempts): 39562.9 PyStones/second
Best IronPython 2.6 PyStone run (out of 5 attempts): 55053.7 PyStones/second

PyBench 2.0

Background
PyBench is a collection of low-level benchmarks distributed with CPython. Whereas
PyStone can be used to discover the overall performance of the Python interpreter,
PyBench targets specific areas (e.g., the amount of time it takes to concatenate
two strings). You can find the latest version of PyBench on CPython's Subversion
source control system (e.g., http://svn.python.org/projects/python/trunk/Tools/pybench/).

Please note the following:
  • Much of the platform module is broken under IPy. As such, build numbers, build dates, etc had to be faked under IPy
  • The IPy garbage collector cannot be disabled. Although this puts IPy at a disadvantage, we kept it turned off for the CPy run
  • sys.setcheckinterval does not function properly under IronPython. This is another optimization CPython gets that IronPython doesn't
  • The default number of rounds and a warp factor of 1 were used

Results
(this=C:\SnapTemp\IronPython 2.6, other=C:\SnapTemp\Python26)
Test min run-time this min run-time other min run-time diff avg run-time this avg run-time other avg run-time diff
BuiltinFunctionCalls: 388ms 1726ms -77.5% 388ms 1726ms -77.5%
BuiltinMethodLookup: 606ms 2033ms -70.2% 606ms 2033ms -70.2%
CompareFloats: 28ms 1068ms -97.4% 28ms 1068ms -97.4%
CompareFloatsIntegers: 341ms 1411ms -75.8% 341ms 1411ms -75.8%
CompareIntegers: 27ms 852ms -96.9% 27ms 852ms -96.9%
CompareInternedStrings: 379ms 1205ms -68.5% 379ms 1205ms -68.5%
CompareLongs: 29ms 1010ms -97.1% 29ms 1010ms -97.1%
CompareStrings: 508ms 1266ms -59.9% 508ms 1266ms -59.9%
CompareUnicode: 416ms 1169ms -64.4% 416ms 1169ms -64.4%
ComplexPythonFunctionCalls: 2196ms 2379ms -7.7% 2196ms 2379ms -7.7%
ConcatStrings: 4442ms 3049ms 45.7% 4442ms 3049ms 45.7%
ConcatUnicode: 3149ms 2532ms 24.4% 3149ms 2532ms 24.4%
CreateInstances: 808ms 2054ms -60.7% 808ms 2054ms -60.7%
CreateNewInstances: 1168ms 1537ms -24.0% 1168ms 1537ms -24.0%
CreateStringsWithConcat: 2317ms 1248ms 85.6% 2317ms 1248ms 85.6%
CreateUnicodeWithConcat: 1168ms 1303ms -10.4% 1168ms 1303ms -10.4%
DictCreation: 2072ms 1053ms 96.9% 2072ms 1053ms 96.9%
DictWithFloatKeys: 2132ms 2979ms -28.4% 2132ms 2979ms -28.4%
DictWithIntegerKeys: 1834ms 859ms 113.5% 1834ms 859ms 113.5%
DictWithStringKeys: 1990ms 1046ms 90.1% 1990ms 1046ms 90.1%
ForLoops: 577ms 655ms -11.9% 577ms 655ms -11.9%
IfThenElse: 291ms 876ms -66.8% 291ms 876ms -66.8%
ListSlicing: 3697ms 1484ms 149.1% 3697ms 1484ms 149.1%
NestedForLoops: 640ms 997ms -35.8% 640ms 997ms -35.8%
NormalClassAttribute: 2976ms 1148ms 159.2% 2976ms 1148ms 159.2%
NormalInstanceAttribute: 3756ms 1013ms 270.6% 3756ms 1013ms 270.6%
PythonFunctionCalls: 388ms 1695ms -77.1% 388ms 1695ms -77.1%
PythonMethodCalls: 3525ms 1832ms 92.4% 3525ms 1832ms 92.4%
Recursion: 573ms 2277ms -74.8% 573ms 2277ms -74.8%
SecondImport: 934ms 1248ms -25.2% 934ms 1248ms -25.2%
SecondPackageImport: 1028ms 1334ms -22.9% 1028ms 1334ms -22.9%
SecondSubmoduleImport: 1496ms 1687ms -11.3% 1496ms 1687ms -11.3%
SimpleComplexArithmetic: 730ms 1352ms -46.0% 730ms 1352ms -46.0%
SimpleDictManipulation: 2196ms 1148ms 91.3% 2196ms 1148ms 91.3%
SimpleFloatArithmetic: 495ms 1072ms -53.8% 495ms 1072ms -53.8%
SimpleIntFloatArithmetic: 558ms 759ms -26.5% 558ms 759ms -26.5%
SimpleIntegerArithmetic: 558ms 745ms -25.1% 558ms 745ms -25.1%
SimpleListManipulation: 1337ms 934ms 43.2% 1337ms 934ms 43.2%
SimpleLongArithmetic: 1027ms 1199ms -14.3% 1027ms 1199ms -14.3%
SmallLists: 1432ms 1645ms -12.9% 1432ms 1645ms -12.9%
SmallTuples: 902ms 1366ms -34.0% 902ms 1366ms -34.0%
SpecialClassAttribute: 3179ms 1170ms 171.7% 3179ms 1170ms 171.7%
SpecialInstanceAttribute: 3818ms 1229ms 210.7% 3818ms 1229ms 210.7%
StringMappings: 3086ms 2277ms 35.5% 3086ms 2277ms 35.5%
StringPredicates: 1331ms 1784ms -25.4% 1331ms 1784ms -25.4%
StringSlicing: 1617ms 1509ms 7.2% 1617ms 1509ms 7.2%
TryExcept: 275ms 578ms -52.4% 275ms 578ms -52.4%
TryFinally: 1011ms 1763ms -42.7% 1011ms 1763ms -42.7%
TryRaiseExcept: 65595ms 1093ms 5902.5% 65595ms 1093ms 5902.5%
TupleSlicing: 2106ms 1505ms 40.0% 2106ms 1505ms 40.0%
UnicodeMappings: 2106ms 954ms 120.9% 2106ms 954ms 120.9%
UnicodePredicates: 1438ms 1296ms 11.0% 1438ms 1296ms 11.0%
UnicodeSlicing: 1119ms 1648ms -32.1% 1119ms 1648ms -32.1%
WithFinally: 653ms 2232ms -70.7% 653ms 2232ms -70.7%
WithRaiseExcept: 73597ms 1662ms 4327.3% 73597ms 1662ms 4327.3%
Totals: 216043ms 78944ms 173.7% 216043ms 78944ms 173.7%

Richards

Background
Much like PyStone, richards.py is a high-level benchmark intended to give the
Python user a single result indicating the overall "goodness" factor of a Python
interpreter. This benchmark is distributed with the PyPy interpreter, although
the original version was written in BCPL by Dr. Martin Richards at Cambridge
University. Lower "Average time per iteration" results indicate better performance.

Results
Best average time per iteration for Python26 (out of 5 attempts): 613
Best average time per iteration for IronPython 2.6 (out of 5 attempts): 1427.41

Last edited Sep 18, 2009 at 5:49 PM by dfugate, version 1

Comments

No comments yet.