IronPython Performance Report

An automated, lightweight comparison between 'IronPython 2.0.1' and 'Python25'
on 02/11/2009 16:37:21.

Machine Setup

  • PC
    • Manufacturer - Dell Computer Corporation
    • Model - PowerEdge 860
    • Name - MERLIN-14
  • Software
    • Operating System - Microsoft Windows NT 6.0.6001 Service Pack 1
    • CLR Version - 2.0.50727.3053
  • CPU
    • Full Name - Intel(R) Pentium(R) D CPU 2.80GHz
    • Speed - 2800
    • Number of Cores - 2
  • RAM - 2146070528
  • Disk Drive - WDC WD1600JS-75NCB3 ATA Device

PyStone 1.1

Background
PyStone is an extremely high-level benchmark distributed with CPython. In
short, running pystone.py under various implementations of Python will emit
the number of PyStones (i.e., iterations) per second that can be run on a given
machine. A higher PyStone result indicates better overall performance of a Python
interpreter. The latest copy of PyStone can be found in lib\test\pystone.py
(relative to CPython installation locations).

Notes:
  • By default pystone.py runs 50,000 passes to determine PyStones per second. The results below were generated using this default
  • Due to site caching in the Dynamic Language Runtime, IronPython performs better with more PyStone passes than the default value

Results
Best CPy PyStone run (out of 5 attempts): 38042.5 PyStones/second
Best IPy PyStone run (out of 5 attempts): 48516.4 PyStones/second

PyBench 2.0

Background
PyBench is a collection of low-level benchmarks distributed with CPython. Whereas
PyStone can be used to discover the overall performance of the Python interpreter,
PyBench targets specific areas (e.g., the amount of time it takes to concatenate
two strings). You can find the latest version of PyBench on CPython's Subversion
source control system (e.g., http://svn.python.org/projects/python/trunk/Tools/pybench/).

Please note the following:
  • Much of the platform module is broken under IPy. As such, build numbers, build dates, etc had to be faked under IPy
  • The IPy garbage collector cannot be disabled. Although this puts IPy at a disadvantage, we kept it turned off for the CPy run
  • sys.setcheckinterval does not function properly under IronPython. This is another optimization CPython gets that IronPython doesn't
  • The default number of rounds and a warp factor of 1 were used

Results
(this=C:\SnapTemp\perfresults\pybench.ipy.results, other=C:\SnapTemp\perfresults\pybench.cpy.results)
Test min run-time this min run-time other min run-time diff avg run-time this avg run-time other avg run-time diff
BuiltinFunctionCalls: 279ms 2044ms -86.3% 307ms 2067ms -85.1%
BuiltinMethodLookup: 434ms 1850ms -76.5% 461ms 1989ms -76.8%
CompareFloats: 590ms 1154ms -48.9% 605ms 1156ms -47.7%
CompareFloatsIntegers: 91ms 1299ms -93.0% 101ms 1312ms -92.3%
CompareIntegers: 121ms 948ms -87.2% 137ms 967ms -85.9%
CompareInternedStrings: 207ms 1017ms -79.6% 226ms 1019ms -77.8%
CompareLongs: 1714ms 1169ms 46.5% 1762ms 1172ms 50.3%
CompareStrings: 352ms 1222ms -71.2% 372ms 1299ms -71.4%
CompareUnicode: 275ms 1344ms -79.5% 282ms 1351ms -79.2%
ConcatStrings: 4193ms 3333ms 25.8% 4475ms 3350ms 33.6%
ConcatUnicode: 2525ms 3012ms -16.2% 2751ms 3041ms -9.5%
CreateInstances: 746ms 1854ms -59.7% 793ms 1933ms -59.0%
CreateNewInstances: 1277ms 1544ms -17.3% 1352ms 1606ms -15.8%
CreateStringsWithConcat: 2021ms 1318ms 53.4% 2411ms 1324ms 82.1%
CreateUnicodeWithConcat: 824ms 1936ms -57.4% 910ms 1953ms -53.4%
DictCreation: 1199ms 1182ms 1.5% 1257ms 1191ms 5.5%
DictWithFloatKeys: 2257ms 3191ms -29.3% 2269ms 3194ms -29.0%
DictWithIntegerKeys: 1912ms 943ms 102.8% 1934ms 950ms 103.6%
DictWithStringKeys: 1974ms 1050ms 88.0% 2002ms 1054ms 89.9%
ForLoops: 390ms 1098ms -64.5% 409ms 1105ms -63.0%
IfThenElse: 307ms 1027ms -70.2% 339ms 1040ms -67.4%
ListSlicing: 3463ms 1514ms 128.8% 3582ms 1529ms 134.2%
NestedForLoops: 624ms 1138ms -45.2% 633ms 1188ms -46.7%
NormalClassAttribute: 4115ms 1211ms 239.9% 4139ms 1237ms 234.5%
NormalInstanceAttribute: 3101ms 1167ms 165.8% 3147ms 1179ms 166.9%
PythonFunctionCalls: 310ms 1552ms -80.0% 326ms 1583ms -79.4%
PythonMethodCalls: 3010ms 1931ms 55.9% 3044ms 1939ms 57.0%
Recursion: 652ms 2083ms -68.7% 663ms 2092ms -68.3%
SecondImport: 2277ms 1506ms 51.2% 2292ms 1523ms 50.5%
SecondPackageImport: 2246ms 1586ms 41.6% 2274ms 1601ms 42.0%
SecondSubmoduleImport: 3057ms 2012ms 51.9% 3077ms 2037ms 51.1%
SimpleComplexArithmetic: 622ms 1415ms -56.1% 659ms 1422ms -53.7%
SimpleDictManipulation: 2134ms 1146ms 86.3% 2165ms 1152ms 88.0%
SimpleFloatArithmetic: 402ms 1221ms -67.1% 428ms 1333ms -67.9%
SimpleIntFloatArithmetic: 480ms 908ms -47.2% 488ms 909ms -46.4%
SimpleIntegerArithmetic: 464ms 911ms -49.0% 472ms 912ms -48.2%
SimpleListManipulation: 1290ms 935ms 38.0% 1312ms 941ms 39.4%
SimpleLongArithmetic: 1012ms 1263ms -19.8% 1044ms 1270ms -17.8%
SmallLists: 1386ms 1800ms -23.0% 1406ms 1810ms -22.3%
SmallTuples: 1760ms 1609ms 9.4% 1802ms 1642ms 9.8%
SpecialClassAttribute: 4115ms 1211ms 239.7% 4145ms 1222ms 239.2%
SpecialInstanceAttribute: 3023ms 2056ms 47.1% 3044ms 2068ms 47.2%
StringMappings: 2993ms 7939ms -62.3% 3058ms 8182ms -62.6%
StringPredicates: 1051ms 2827ms -62.8% 1069ms 2859ms -62.6%
StringSlicing: 1367ms 1657ms -17.5% 1483ms 1678ms -11.6%
TryExcept: 26ms 888ms -97.1% 63ms 890ms -92.9%
TryRaiseExcept: 58234ms 1286ms 4428.6% 58474ms 1298ms 4404.6%
TupleSlicing: 1825ms 1593ms 14.6% 1861ms 1607ms 15.8%
UnicodeMappings: 2339ms 1273ms 83.7% 2358ms 1279ms 84.4%
UnicodePredicates: 1221ms 1584ms -22.9% 1241ms 1593ms -22.1%
UnicodeSlicing: 1181ms 2172ms -45.6% 1237ms 2183ms -43.3%
Totals: 133469ms 84930ms 57.2% 136141ms 86234ms 57.9%

Richards

Background
Much like PyStone, richards.py is a high-level benchmark intended to give the
Python user a single result indicating the overall "goodness" factor of a Python
interpreter. This benchmark is distributed with the PyPy interpreter, although
the original version was written in BCPL by Dr. Martin Richards at Cambridge
University. Lower "Average time per iteration" results indicate better performance.

Results
Average time per iteration for CPy (out of 5 attempts): 822.2
Average time per iteration for IPy (out of 5 attempts): 1237.24

Last edited Feb 23, 2009 at 4:46 PM by dfugate, version 4

Comments

No comments yet.