IronPython Performance Report

An automated, lightweight comparison between 'IronPython 2.6' and 'Python26'
on 11/19/2009 12:25:30.

Machine Setup

  • PC
    • Manufacturer - Dell Computer Corporation
    • Model - PowerEdge 860
    • Name - MERLIN-14
  • Software
    • Operating System - Microsoft Windows NT 6.0.6002 Service Pack 2
    • CLR Version - 2.0.50727.4200
  • CPU
    • Full Name - Intel(R) Pentium(R) D CPU 2.80GHz
    • Speed - 2800
    • Number of Cores - 2
  • RAM - 2146070528
  • Disk Drive - WDC WD1600JS-75NCB3 ATA Device

HelloWorld

Background
HelloWorld is a typical hello world program we've written and is used to evaluate
the startup time of a Python interpreter. Lower startup times indicate better
performance.

Notes:
  • No command-line flags are passed to the Python interpreters

Results
Best Python26 HelloWorld run (out of 5 attempts): 0.0655611 seconds
Best IronPython 2.6 HelloWorld run (out of 5 attempts): 2.1076478 seconds

PyStone 1.1

Background
PyStone is an extremely high-level benchmark distributed with CPython. In
short, running pystone.py under various implementations of Python will emit
the number of PyStones (i.e., iterations) per second that can be run on a given
machine. A higher PyStone result indicates better overall performance of a Python
interpreter. The latest copy of PyStone can be found in lib\test\pystone.py
(relative to CPython installation locations).

Notes:
  • By default pystone.py runs 50,000 passes to determine PyStones per second. The results below were generated using this default
  • Due to site caching in the Dynamic Language Runtime, IronPython performs better with more PyStone passes than the default value

Results
Best Python26 PyStone run (out of 5 attempts): 39239.6 PyStones/second
Best IronPython 2.6 PyStone run (out of 5 attempts): 55765.7 PyStones/second

PyBench 2.0

Background
PyBench is a collection of low-level benchmarks distributed with CPython. Whereas
PyStone can be used to discover the overall performance of the Python interpreter,
PyBench targets specific areas (e.g., the amount of time it takes to concatenate
two strings). You can find the latest version of PyBench on CPython's Subversion
source control system (e.g., http://svn.python.org/projects/python/trunk/Tools/pybench/).

Please note the following:
  • Much of the platform module is broken under IPy. As such, build numbers, build dates, etc had to be faked under IPy
  • The IPy garbage collector cannot be disabled. Although this puts IPy at a disadvantage, we kept it turned off for the CPy run
  • sys.setcheckinterval does not function properly under IronPython. This is another optimization CPython gets that IronPython doesn't.
  • 1 rounds and a warp factor of 1 were used.

Results
(this=C:\SnapTemp\IronPython 2.6, other=C:\SnapTemp\Python26)
Test min run-time this min run-time other min run-time diff avg run-time this avg run-time other avg run-time diff
BuiltinFunctionCalls: 403ms 1737ms -76.8% 403ms 1737ms -76.8%
BuiltinMethodLookup: 606ms 1835ms -67.0% 606ms 1835ms -67.0%
CompareFloats: 28ms 1075ms -97.4% 28ms 1075ms -97.4%
CompareFloatsIntegers: 403ms 1354ms -70.2% 403ms 1354ms -70.2%
CompareIntegers: 27ms 853ms -96.9% 27ms 853ms -96.9%
CompareInternedStrings: 395ms 1237ms -68.1% 395ms 1237ms -68.1%
CompareLongs: 29ms 1071ms -97.3% 29ms 1071ms -97.3%
CompareStrings: 477ms 1428ms -66.6% 477ms 1428ms -66.6%
CompareUnicode: 400ms 1188ms -66.3% 400ms 1188ms -66.3%
ComplexPythonFunctionCalls: 2149ms 2574ms -16.5% 2149ms 2574ms -16.5%
ConcatStrings: 5129ms 3004ms 70.7% 5129ms 3004ms 70.7%
ConcatUnicode: 3040ms 2602ms 16.8% 3040ms 2602ms 16.8%
CreateInstances: 839ms 2035ms -58.8% 839ms 2035ms -58.8%
CreateNewInstances: 1121ms 1552ms -27.8% 1121ms 1552ms -27.8%
CreateStringsWithConcat: 2880ms 1113ms 158.8% 2880ms 1113ms 158.8%
CreateUnicodeWithConcat: 809ms 1250ms -35.3% 809ms 1250ms -35.3%
DictCreation: 2276ms 1052ms 116.3% 2276ms 1052ms 116.3%
DictWithFloatKeys: 2070ms 2956ms -30.0% 2070ms 2956ms -30.0%
DictWithIntegerKeys: 1818ms 851ms 113.8% 1818ms 851ms 113.8%
DictWithStringKeys: 1990ms 1043ms 90.7% 1990ms 1043ms 90.7%
ForLoops: 562ms 658ms -14.6% 562ms 658ms -14.6%
IfThenElse: 291ms 838ms -65.3% 291ms 838ms -65.3%
ListSlicing: 3713ms 1493ms 148.7% 3713ms 1493ms 148.7%
NestedForLoops: 640ms 1054ms -39.3% 640ms 1054ms -39.3%
NormalClassAttribute: 2961ms 1124ms 163.4% 2961ms 1124ms 163.4%
NormalInstanceAttribute: 3710ms 1030ms 260.3% 3710ms 1030ms 260.3%
PythonFunctionCalls: 325ms 1647ms -80.3% 325ms 1647ms -80.3%
PythonMethodCalls: 3556ms 1866ms 90.6% 3556ms 1866ms 90.6%
Recursion: 559ms 2214ms -74.8% 559ms 2214ms -74.8%
SecondImport: 966ms 1248ms -22.6% 966ms 1248ms -22.6%
SecondPackageImport: 1059ms 1313ms -19.3% 1059ms 1313ms -19.3%
SecondSubmoduleImport: 1496ms 1724ms -13.2% 1496ms 1724ms -13.2%
SimpleComplexArithmetic: 605ms 1324ms -54.3% 605ms 1324ms -54.3%
SimpleDictManipulation: 2072ms 1094ms 89.4% 2072ms 1094ms 89.4%
SimpleFloatArithmetic: 495ms 1475ms -66.4% 495ms 1475ms -66.4%
SimpleIntFloatArithmetic: 558ms 760ms -26.6% 558ms 760ms -26.6%
SimpleIntegerArithmetic: 542ms 757ms -28.4% 542ms 757ms -28.4%
SimpleListManipulation: 1384ms 911ms 51.9% 1384ms 911ms 51.9%
SimpleLongArithmetic: 1075ms 1197ms -10.2% 1075ms 1197ms -10.2%
SmallLists: 1464ms 1623ms -9.8% 1464ms 1623ms -9.8%
SmallTuples: 871ms 1429ms -39.1% 871ms 1429ms -39.1%
SpecialClassAttribute: 3210ms 1103ms 190.9% 3210ms 1103ms 190.9%
SpecialInstanceAttribute: 3865ms 1198ms 222.7% 3865ms 1198ms 222.7%
StringMappings: 3055ms 2353ms 29.8% 3055ms 2353ms 29.8%
StringPredicates: 1346ms 1798ms -25.1% 1346ms 1798ms -25.1%
StringSlicing: 1414ms 1522ms -7.1% 1414ms 1522ms -7.1%
TryExcept: 260ms 573ms -54.6% 260ms 573ms -54.6%
TryFinally: 949ms 1808ms -47.5% 949ms 1808ms -47.5%
TryRaiseExcept: 63930ms 1083ms 5805.3% 63930ms 1083ms 5805.3%
TupleSlicing: 2090ms 1517ms 37.8% 2090ms 1517ms 37.8%
UnicodeMappings: 2059ms 949ms 117.0% 2059ms 949ms 117.0%
UnicodePredicates: 1391ms 1297ms 7.2% 1391ms 1297ms 7.2%
UnicodeSlicing: 1399ms 1737ms -19.5% 1399ms 1737ms -19.5%
WithFinally: 652ms 2199ms -70.4% 652ms 2199ms -70.4%
WithRaiseExcept: 71699ms 1681ms 4164.4% 71699ms 1681ms 4164.4%
Totals: 213113ms 79407ms 168.4% 213113ms 79407ms 168.4%

Richards

Background
Much like PyStone, richards.py is a high-level benchmark intended to give the
Python user a single result indicating the overall "goodness" factor of a Python
interpreter. This benchmark is distributed with the PyPy interpreter, although
the original version was written in BCPL by Dr. Martin Richards at Cambridge
University. Lower "Average time per iteration" results indicate better performance.

Results
Best average time per iteration for Python26 (out of 5 attempts): 611.8
Best average time per iteration for IronPython 2.6 (out of 5 attempts): 1400.05

Last edited Dec 11, 2009 at 10:40 PM by dinov, version 1

Comments

fijal Mar 18, 2010 at 4:17 PM 
By "no command line arguments" you mean IronPython is not supporting frames, right?

lheneks Mar 18, 2010 at 4:16 PM 
Whats up with TryRaiseExcept: and WithRaiseExcept:? If those two are thrown out, the totals are almost identical.