IronPython Performance Report

An automated, lightweight comparison between 'IronPython 2.6' and 'Python26'
on 07/21/2009 13:44:27.

Machine Setup

  • PC
    • Manufacturer - Dell Computer Corporation
    • Model - PowerEdge 860
    • Name - MERLIN-14
  • Software
    • Operating System - Microsoft Windows NT 6.0.6001 Service Pack 1
    • CLR Version - 2.0.50727.3053
  • CPU
    • Full Name - Intel(R) Pentium(R) D CPU 2.80GHz
    • Speed - 2800
    • Number of Cores - 2
  • RAM - 2146070528
  • Disk Drive - WDC WD1600JS-75NCB3 ATA Device

HelloWorld

Background
HelloWorld is a typical hello world program we've written and is used to evaluate
the startup time of a Python interpreter. Lower startup times indicate better
performance.

Notes:
  • No command-line flags are passed to the Python interpreters

Results
Best Python26 HelloWorld run (out of 5 attempts): 0.113501 seconds
Best IronPython 2.6 HelloWorld run (out of 5 attempts): 2.1976297 seconds

PyStone 1.1

Background
PyStone is an extremely high-level benchmark distributed with CPython. In
short, running pystone.py under various implementations of Python will emit
the number of PyStones (i.e., iterations) per second that can be run on a given
machine. A higher PyStone result indicates better overall performance of a Python
interpreter. The latest copy of PyStone can be found in lib\test\pystone.py
(relative to CPython installation locations).

Notes:
  • By default pystone.py runs 50,000 passes to determine PyStones per second. The results below were generated using this default
  • Due to site caching in the Dynamic Language Runtime, IronPython performs better with more PyStone passes than the default value

Results
Best Python26 PyStone run (out of 5 attempts): 39241.3 PyStones/second
Best IronPython 2.6 PyStone run (out of 5 attempts): 46625.5 PyStones/second

PyBench 2.0

Background
PyBench is a collection of low-level benchmarks distributed with CPython. Whereas
PyStone can be used to discover the overall performance of the Python interpreter,
PyBench targets specific areas (e.g., the amount of time it takes to concatenate
two strings). You can find the latest version of PyBench on CPython's Subversion
source control system (e.g., http://svn.python.org/projects/python/trunk/Tools/pybench/).

Please note the following:
  • Much of the platform module is broken under IPy. As such, build numbers, build dates, etc had to be faked under IPy
  • The IPy garbage collector cannot be disabled. Although this puts IPy at a disadvantage, we kept it turned off for the CPy run
  • sys.setcheckinterval does not function properly under IronPython. This is another optimization CPython gets that IronPython doesn't
  • The default number of rounds and a warp factor of 1 were used

Results
(this=C:\SnapTemp\IronPython 2.6, other=C:\SnapTemp\Python26)
Test min run-time this min run-time other min run-time diff avg run-time this avg run-time other avg run-time diff
BuiltinFunctionCalls: 264ms 1716ms -84.6% 289ms 1771ms -83.7%
BuiltinMethodLookup: 497ms 1805ms -72.5% 516ms 1862ms -72.3%
CompareFloats: 450ms 1174ms -61.7% 558ms 1190ms -53.1%
CompareFloatsIntegers: 123ms 1231ms -90.0% 140ms 1258ms -88.8%
CompareIntegers: 199ms 849ms -76.6% 222ms 859ms -74.1%
CompareInternedStrings: 349ms 1213ms -71.2% 365ms 1241ms -70.6%
CompareLongs: 637ms 1079ms -40.9% 668ms 1097ms -39.1%
CompareStrings: 446ms 1266ms -64.8% 466ms 1376ms -66.1%
CompareUnicode: 339ms 1083ms -68.8% 348ms 1133ms -69.3%
ComplexPythonFunctionCalls: 2399ms 2330ms 3.0% 2457ms 2382ms 3.2%
ConcatStrings: 4240ms 2897ms 46.3% 4526ms 2971ms 52.3%
ConcatUnicode: 2604ms 2311ms 12.7% 2760ms 2525ms 9.3%
CreateInstances: 871ms 2010ms -56.7% 915ms 2053ms -55.4%
CreateNewInstances: 1058ms 1494ms -29.2% 1085ms 1531ms -29.1%
CreateStringsWithConcat: 1976ms 1099ms 79.8% 2188ms 1122ms 95.1%
CreateUnicodeWithConcat: 778ms 1249ms -37.7% 888ms 1269ms -30.0%
DictCreation: 1464ms 1053ms 39.0% 1542ms 1070ms 44.2%
DictWithFloatKeys: 2351ms 2958ms -20.5% 2376ms 2994ms -20.6%
DictWithIntegerKeys: 2038ms 886ms 130.1% 2049ms 900ms 127.8%
DictWithStringKeys: 2147ms 1045ms 105.3% 2159ms 1066ms 102.5%
ForLoops: 390ms 655ms -40.4% 398ms 668ms -40.5%
IfThenElse: 214ms 855ms -75.0% 234ms 891ms -73.7%
ListSlicing: 3479ms 1452ms 139.5% 3532ms 1496ms 136.1%
NestedForLoops: 437ms 1102ms -60.4% 454ms 1122ms -59.5%
NormalClassAttribute: 5956ms 1129ms 427.3% 5994ms 1155ms 418.7%
NormalInstanceAttribute: 3647ms 1030ms 254.0% 3674ms 1049ms 250.4%
PythonFunctionCalls: 388ms 1607ms -75.8% 401ms 1657ms -75.8%
PythonMethodCalls: 3681ms 1828ms 101.4% 3721ms 1875ms 98.5%
Recursion: 761ms 2196ms -65.3% 763ms 2265ms -66.3%
SecondImport: 3494ms 1223ms 185.6% 3593ms 1252ms 187.1%
SecondPackageImport: 3494ms 1303ms 168.2% 3611ms 1326ms 172.3%
SecondSubmoduleImport: 4461ms 1683ms 165.1% 4558ms 1755ms 159.8%
SimpleComplexArithmetic: 653ms 1343ms -51.4% 683ms 1381ms -50.6%
SimpleDictManipulation: 2212ms 1110ms 99.3% 2254ms 1126ms 100.1%
SimpleFloatArithmetic: 480ms 1279ms -62.4% 496ms 1384ms -64.2%
SimpleIntFloatArithmetic: 527ms 743ms -29.0% 540ms 757ms -28.7%
SimpleIntegerArithmetic: 512ms 745ms -31.3% 530ms 760ms -30.2%
SimpleListManipulation: 1338ms 899ms 48.8% 1356ms 919ms 47.6%
SimpleLongArithmetic: 981ms 1207ms -18.7% 1008ms 1227ms -17.9%
SmallLists: 1433ms 1595ms -10.2% 1458ms 1631ms -10.6%
SmallTuples: 2182ms 1352ms 61.4% 2232ms 1400ms 59.4%
SpecialClassAttribute: 6268ms 1068ms 486.7% 6310ms 1099ms 474.3%
SpecialInstanceAttribute: 3617ms 1182ms 206.1% 3693ms 1206ms 206.1%
StringMappings: 2962ms 2303ms 28.6% 3013ms 2343ms 28.6%
StringPredicates: 1222ms 1805ms -32.3% 1259ms 1831ms -31.2%
StringSlicing: 1368ms 1498ms -8.7% 1462ms 1536ms -4.8%
TryExcept: 27ms 574ms -95.4% 44ms 585ms -92.5%
TryFinally: 965ms 1745ms -44.7% 988ms 1796ms -45.0%
TryRaiseExcept: 71039ms 1088ms 6426.6% 71257ms 1104ms 6353.2%
TupleSlicing: 1856ms 1513ms 22.7% 1894ms 1534ms 23.4%
UnicodeMappings: 2012ms 949ms 112.1% 2019ms 962ms 109.8%
UnicodePredicates: 1297ms 1296ms 0.0% 1322ms 1309ms 1.0%
UnicodeSlicing: 1197ms 1637ms -26.9% 1236ms 1668ms -25.9%
WithFinally: 653ms 2238ms -70.8% 675ms 2272ms -70.3%
WithRaiseExcept: 78652ms 1679ms 4583.8% 78780ms 1720ms 4480.7%
Totals: 239083ms 77658ms 207.9% 241957ms 79728ms 203.5%

Richards

Background
Much like PyStone, richards.py is a high-level benchmark intended to give the
Python user a single result indicating the overall "goodness" factor of a Python
interpreter. This benchmark is distributed with the PyPy interpreter, although
the original version was written in BCPL by Dr. Martin Richards at Cambridge
University. Lower "Average time per iteration" results indicate better performance.

Results
Best average time per iteration for Python26 (out of 5 attempts): 619.3
Best average time per iteration for IronPython 2.6 (out of 5 attempts): 1308.91

Last edited Jul 21, 2009 at 11:40 PM by dfugate, version 1

Comments

No comments yet.