IronPython Performance Report

An automated, lightweight comparison between 'IronPython 2.0.3' and 'Python25'
on 10/22/2009 20:03:28.

Machine Setup

  • PC
    • Manufacturer - Dell Computer Corporation
    • Model - PowerEdge 860
    • Name - MERLIN-14
  • Software
    • Operating System - Microsoft Windows NT 6.0.6001 Service Pack 1
    • CLR Version - 2.0.50727.3603
  • CPU
    • Full Name - Intel(R) Pentium(R) D CPU 2.80GHz
    • Speed - 2800
    • Number of Cores - 2
  • RAM - 2146070528
  • Disk Drive - WDC WD1600JS-75NCB3 ATA Device

HelloWorld

Background
HelloWorld is a typical hello world program we've written and is used to evaluate
the startup time of a Python interpreter. Lower startup times indicate better
performance.

Notes:
  • No command-line flags are passed to the Python interpreters

Results
Best Python25 HelloWorld run (out of 5 attempts): 0.055318 seconds
Best IronPython 2.0.3 HelloWorld run (out of 5 attempts): 3.9057397 seconds

PyStone 1.1

Background
PyStone is an extremely high-level benchmark distributed with CPython. In
short, running pystone.py under various implementations of Python will emit
the number of PyStones (i.e., iterations) per second that can be run on a given
machine. A higher PyStone result indicates better overall performance of a Python
interpreter. The latest copy of PyStone can be found in lib\test\pystone.py
(relative to CPython installation locations).

Notes:
  • By default pystone.py runs 50,000 passes to determine PyStones per second. The results below were generated using this default
  • Due to site caching in the Dynamic Language Runtime, IronPython performs better with more PyStone passes than the default value

Results
Best Python25 PyStone run (out of 5 attempts): 37961.2 PyStones/second
Best IronPython 2.0.3 PyStone run (out of 5 attempts): 50911.5 PyStones/second

PyBench 2.0

Background
PyBench is a collection of low-level benchmarks distributed with CPython. Whereas
PyStone can be used to discover the overall performance of the Python interpreter,
PyBench targets specific areas (e.g., the amount of time it takes to concatenate
two strings). You can find the latest version of PyBench on CPython's Subversion
source control system (e.g., http://svn.python.org/projects/python/trunk/Tools/pybench/).

Please note the following:
  • Much of the platform module is broken under IPy. As such, build numbers, build dates, etc had to be faked under IPy
  • The IPy garbage collector cannot be disabled. Although this puts IPy at a disadvantage, we kept it turned off for the CPy run
  • sys.setcheckinterval does not function properly under IronPython. This is another optimization CPython gets that IronPython doesn't
  • The default number of rounds and a warp factor of 1 were used

Results
(this=C:\SnapTemp\IronPython 2.0.3, other=C:\SnapTemp\Python25)
Test min run-time this min run-time other min run-time diff avg run-time this avg run-time other avg run-time diff
BuiltinFunctionCalls: 279ms 2034ms -86.3% 309ms 2087ms -85.2%
BuiltinMethodLookup: 434ms 1873ms -76.8% 455ms 2056ms -77.9%
CompareFloats: 543ms 1101ms -50.7% 555ms 1115ms -50.2%
CompareFloatsIntegers: 373ms 1418ms -73.7% 389ms 1444ms -73.1%
CompareIntegers: 121ms 940ms -87.1% 138ms 970ms -85.8%
CompareInternedStrings: 223ms 1015ms -78.0% 236ms 1036ms -77.3%
CompareLongs: 1714ms 1166ms 46.9% 1776ms 1186ms 49.7%
CompareStrings: 352ms 1205ms -70.8% 372ms 1330ms -72.0%
CompareUnicode: 275ms 1354ms -79.7% 283ms 1380ms -79.5%
ConcatStrings: 4304ms 3129ms 37.5% 4585ms 3283ms 39.6%
ConcatUnicode: 2542ms 2817ms -9.8% 2758ms 2885ms -4.4%
CreateInstances: 731ms 1933ms -62.2% 789ms 1948ms -59.5%
CreateNewInstances: 1324ms 1608ms -17.6% 1389ms 1616ms -14.0%
CreateStringsWithConcat: 1959ms 1322ms 48.2% 2304ms 1338ms 72.2%
CreateUnicodeWithConcat: 793ms 1927ms -58.8% 868ms 1948ms -55.4%
DictCreation: 1214ms 1218ms -0.3% 1258ms 1234ms +2.0%
DictWithFloatKeys: 2132ms 3175ms -32.9% 2171ms 3216ms -32.5%
DictWithIntegerKeys: 1912ms 949ms 101.4% 1935ms 969ms 99.8%
DictWithStringKeys: 1959ms 1065ms 84.0% 1985ms 1087ms 82.7%
ForLoops: 390ms 792ms -50.7% 406ms 801ms -49.4%
IfThenElse: 307ms 1027ms -70.1% 328ms 1041ms -68.5%
ListSlicing: 3479ms 1514ms 129.8% 3592ms 1533ms 134.4%
NestedForLoops: 577ms 1145ms -49.6% 591ms 1195ms -50.5%
NormalClassAttribute: 3742ms 1233ms 203.4% 3788ms 1258ms 201.2%
NormalInstanceAttribute: 3086ms 1165ms 164.9% 3133ms 1181ms 165.3%
PythonFunctionCalls: 310ms 1573ms -80.3% 332ms 1595ms -79.2%
PythonMethodCalls: 2901ms 1921ms 51.0% 2940ms 1937ms 51.8%
Recursion: 652ms 2098ms -68.9% 660ms 2107ms -68.7%
SecondImport: 2308ms 1485ms 55.4% 2341ms 1522ms 53.8%
SecondPackageImport: 2339ms 1599ms 46.3% 2389ms 1617ms 47.8%
SecondSubmoduleImport: 3228ms 2025ms 59.4% 3291ms 2059ms 59.9%
SimpleComplexArithmetic: 528ms 1392ms -62.1% 564ms 1417ms -60.2%
SimpleDictManipulation: 2119ms 1149ms 84.5% 2158ms 1168ms 84.7%
SimpleFloatArithmetic: 402ms 1310ms -69.3% 436ms 1343ms -67.5%
SimpleIntFloatArithmetic: 480ms 907ms -47.1% 486ms 915ms -46.9%
SimpleIntegerArithmetic: 464ms 909ms -48.9% 477ms 914ms -47.9%
SimpleListManipulation: 1290ms 944ms 36.7% 1309ms 952ms 37.6%
SimpleLongArithmetic: 1059ms 1257ms -15.7% 1100ms 1280ms -14.1%
SmallLists: 1370ms 1812ms -24.4% 1402ms 1835ms -23.6%
SmallTuples: 1791ms 1627ms 10.1% 1832ms 1665ms 10.0%
SpecialClassAttribute: 3663ms 1220ms 200.3% 3677ms 1242ms 196.1%
SpecialInstanceAttribute: 2992ms 2067ms 44.7% 3030ms 2101ms 44.2%
StringMappings: 2962ms 7961ms -62.8% 3026ms 8378ms -63.9%
StringPredicates: 1066ms 2818ms -62.2% 1091ms 2923ms -62.7%
StringSlicing: 1383ms 1694ms -18.4% 1519ms 1717ms -11.6%
TryExcept: 26ms 883ms -97.1% 60ms 896ms -93.3%
TryRaiseExcept: 58466ms 1279ms 4470.6% 58820ms 1311ms 4385.2%
TupleSlicing: 1825ms 1600ms 14.1% 1861ms 1623ms 14.7%
UnicodeMappings: 2324ms 1271ms 82.8% 2380ms 1288ms 84.7%
UnicodePredicates: 1221ms 1570ms -22.3% 1254ms 1616ms -22.4%
UnicodeSlicing: 1166ms 2154ms -45.9% 1245ms 2191ms -43.2%
Totals: 133099ms 84648ms 57.2% 136075ms 86748ms 56.9%

Richards

Background
Much like PyStone, richards.py is a high-level benchmark intended to give the
Python user a single result indicating the overall "goodness" factor of a Python
interpreter. This benchmark is distributed with the PyPy interpreter, although
the original version was written in BCPL by Dr. Martin Richards at Cambridge
University. Lower "Average time per iteration" results indicate better performance.

Results
Best average time per iteration for Python25 (out of 5 attempts): 819
Best average time per iteration for IronPython 2.0.3 (out of 5 attempts): 1224.56

Last edited Oct 23, 2009 at 3:56 AM by dfugate, version 1

Comments

No comments yet.