IronPython Performance Report

An automated, lightweight comparison between 'IronPython 2.7' and 'Python27'
on 07/15/2010 10:56:38.

Machine Setup

  • PC
    • Manufacturer - Dell Computer Corporation
    • Model - PowerEdge 860
    • Name - MERLIN-14
  • Software
    • Operating System - Microsoft Windows NT 6.0.6002 Service Pack 2
    • CLR Version - 2.0.50727.4200
  • CPU
    • Full Name - Intel(R) Pentium(R) D CPU 2.80GHz
    • Speed - 2800
    • Number of Cores - 2
  • RAM - 2146070528
  • Disk Drive - WDC WD1600JS-75NCB3 ATA Device

HelloWorld

Background
HelloWorld is a typical hello world program we've written and is used to evaluate
the startup time of a Python interpreter. Lower startup times indicate better
performance.

Notes:
  • No command-line flags are passed to the Python interpreters

Results
Best Python27 HelloWorld run (out of 5 attempts): 0.0677239 seconds
Best IronPython 2.7 HelloWorld run (out of 5 attempts): 1.0527518 seconds

PyStone 1.1

Background
PyStone is an extremely high-level benchmark distributed with CPython. In
short, running pystone.py under various implementations of Python will emit
the number of PyStones (i.e., iterations) per second that can be run on a given
machine. A higher PyStone result indicates better overall performance of a Python
interpreter. The latest copy of PyStone can be found in lib\test\pystone.py
(relative to CPython installation locations).

Notes:
  • By default pystone.py runs 50,000 passes to determine PyStones per second. The results below were generated using this default
  • Due to site caching in the Dynamic Language Runtime, IronPython performs better with more PyStone passes than the default value

Results
Best Python27 PyStone run (out of 5 attempts): 41187 PyStones/second
Best IronPython 2.7 PyStone run (out of 5 attempts): 52101.1 PyStones/second

PyBench 2.0

Background
PyBench is a collection of low-level benchmarks distributed with CPython. Whereas
PyStone can be used to discover the overall performance of the Python interpreter,
PyBench targets specific areas (e.g., the amount of time it takes to concatenate
two strings). You can find the latest version of PyBench on CPython's Subversion
source control system (e.g., http://svn.python.org/projects/python/trunk/Tools/pybench/).

Please note the following:
  • Much of the platform module is broken under IPy. As such, build numbers, build dates, etc had to be faked under IPy
  • The IPy garbage collector cannot be disabled. Although this puts IPy at a disadvantage, we kept it turned off for the CPy run
  • sys.setcheckinterval does not function properly under IronPython. This is another optimization CPython gets that IronPython doesn't
  • The default number of rounds and a warp factor of 1 were used

Results
(this=C:\SnapTemp\IronPython 2.7, other=C:\SnapTemp\Python27)
Test min run-time this min run-time other min run-time diff avg run-time this avg run-time other avg run-time diff
BuiltinFunctionCalls: 391ms 1665ms -76.5% 405ms 1700ms -76.2%
BuiltinMethodLookup: 626ms 1835ms -65.9% 674ms 1946ms -65.4%
CompareFloats: 6ms 974ms -99.3% 8ms 983ms -99.2%
CompareFloatsIntegers: 487ms 1159ms -58.0% 501ms 1171ms -57.2%
CompareIntegers: 10ms 831ms -98.8% 12ms 835ms -98.6%
CompareInternedStrings: 596ms 1138ms -47.7% 610ms 1166ms -47.7%
CompareLongs: 6ms 924ms -99.4% 7ms 929ms -99.2%
CompareStrings: 724ms 1332ms -45.6% 728ms 1342ms -45.7%
CompareUnicode: 541ms 1118ms -51.6% 550ms 1129ms -51.3%
ComplexPythonFunctionCalls: 2231ms 1614ms 38.2% 2292ms 1624ms 41.1%
ConcatStrings: 4870ms 2903ms 67.8% 5734ms 3302ms 73.7%
ConcatUnicode: 2970ms 2343ms 26.8% 3471ms 2416ms 43.7%
CreateInstances: 918ms 1791ms -48.7% 1207ms 1807ms -33.2%
CreateNewInstances: 404ms 1328ms -69.6% 425ms 1346ms -68.4%
CreateStringsWithConcat: 2349ms 1197ms 96.2% 2680ms 1217ms 120.2%
CreateUnicodeWithConcat: 863ms 1199ms -28.0% 1051ms 1212ms -13.2%
DictCreation: 66ms 925ms -92.8% 82ms 930ms -91.2%
DictWithFloatKeys: 2020ms 3123ms -35.3% 2037ms 3133ms -35.0%
DictWithIntegerKeys: 1943ms 875ms 122.1% 1962ms 878ms 123.4%
DictWithStringKeys: 2107ms 996ms 111.6% 2134ms 1003ms 112.8%
ForLoops: 359ms 684ms -47.6% 375ms 687ms -45.5%
IfThenElse: 209ms 873ms -76.0% 229ms 881ms -74.0%
ListSlicing: 3755ms 1474ms 154.8% 3775ms 1493ms 152.8%
NestedForLoops: 477ms 994ms -52.0% 493ms 1006ms -51.0%
NormalClassAttribute: 3283ms 1052ms 212.0% 3337ms 1059ms 215.0%
NormalInstanceAttribute: 3631ms 871ms 316.9% 3652ms 878ms 315.7%
PythonFunctionCalls: 273ms 1589ms -82.8% 281ms 1617ms -82.6%
PythonMethodCalls: 3389ms 1875ms 80.7% 3420ms 1896ms 80.4%
Recursion: 553ms 2275ms -75.7% 558ms 2280ms -75.5%
SecondImport: 738ms 1205ms -38.8% 743ms 1217ms -39.0%
SecondPackageImport: 850ms 1260ms -32.6% 859ms 1270ms -32.3%
SecondSubmoduleImport: 1417ms 1679ms -15.6% 1435ms 1693ms -15.2%
SimpleComplexArithmetic: 759ms 1336ms -43.2% 789ms 1353ms -41.7%
SimpleDictManipulation: 2169ms 1083ms 100.3% 2211ms 1088ms 103.3%
SimpleFloatArithmetic: 481ms 1297ms -62.9% 494ms 1364ms -63.8%
SimpleIntFloatArithmetic: 479ms 757ms -36.7% 487ms 765ms -36.3%
SimpleIntegerArithmetic: 481ms 766ms -37.2% 492ms 771ms -36.2%
SimpleListManipulation: 1230ms 852ms 44.4% 1238ms 856ms 44.7%
SimpleLongArithmetic: 1346ms 1038ms 29.7% 1360ms 1050ms 29.4%
SmallLists: 2825ms 1528ms 84.9% 2860ms 1538ms 85.9%
SmallTuples: 1078ms 1437ms -24.9% 1112ms 1455ms -23.6%
SpecialClassAttribute: 3522ms 1016ms 246.7% 3560ms 1020ms 249.0%
SpecialInstanceAttribute: 3707ms 1170ms 216.8% 3733ms 1176ms 217.4%
StringMappings: 3377ms 2281ms 48.0% 3402ms 2292ms 48.5%
StringPredicates: 1341ms 1782ms -24.8% 1361ms 1818ms -25.1%
StringSlicing: 1827ms 1527ms 19.6% 1880ms 1542ms 21.9%
TryExcept: 157ms 585ms -73.2% 212ms 595ms -64.4%
TryFinally: 576ms 1795ms -67.9% 587ms 1821ms -67.8%
TryRaiseExcept: 1747ms 1077ms 62.2% 1774ms 1085ms 63.5%
TupleSlicing: 2942ms 1572ms 87.1% 3006ms 1578ms 90.5%
UnicodeMappings: 3639ms 941ms 286.9% 3658ms 944ms 287.3%
UnicodePredicates: 1444ms 1253ms 15.2% 1462ms 1267ms 15.5%
UnicodeSlicing: 1603ms 1621ms -1.1% 1660ms 1688ms -1.7%
WithFinally: 527ms 2258ms -76.7% 545ms 2279ms -76.1%
WithRaiseExcept: 43631ms 2621ms 1564.5% 43933ms 2637ms 1566.3%
Totals: 123951ms 76696ms 61.6% 127544ms 78027ms 63.5%

Richards

Background
Much like PyStone, richards.py is a high-level benchmark intended to give the
Python user a single result indicating the overall "goodness" factor of a Python
interpreter. This benchmark is distributed with the PyPy interpreter, although
the original version was written in BCPL by Dr. Martin Richards at Cambridge
University. Lower "Average time per iteration" results indicate better performance.

Results
Best average time per iteration for Python27 (out of 5 attempts): 589.5
Best average time per iteration for IronPython 2.7 (out of 5 attempts): 846.59

Last edited Jul 16, 2010 at 7:30 PM by dfugate, version 1

Comments

No comments yet.