IronPython Performance Report

An automated, lightweight comparison between 'IronPython 2.6B1' and 'IronPython 2.6A1'
on 05/20/2009 16:05:19.

Machine Setup

  • PC
    • Manufacturer - Dell Computer Corporation
    • Model - PowerEdge 860
    • Name - MERLIN-14
  • Software
    • Operating System - Microsoft Windows NT 6.0.6001 Service Pack 1
    • CLR Version - 2.0.50727.3053
  • CPU
    • Full Name - Intel(R) Pentium(R) D CPU 2.80GHz
    • Speed - 2800
    • Number of Cores - 2
  • RAM - 2146070528
  • Disk Drive - WDC WD1600JS-75NCB3 ATA Device

HelloWorld

Background
HelloWorld is a typical hello world program we've written and is used to evaluate
the startup time of a Python interpreter. Lower startup times indicate better
performance.

Notes:
  • No command-line flags are passed to the Python interpreters

Results
Best IronPython 2.6A1 HelloWorld run (out of 5 attempts): 2.7386741 seconds
Best IronPython 2.6B1 HelloWorld run (out of 5 attempts): 1.9468062 seconds

PyStone 1.1

Background
PyStone is an extremely high-level benchmark distributed with CPython. In
short, running pystone.py under various implementations of Python will emit
the number of PyStones (i.e., iterations) per second that can be run on a given
machine. A higher PyStone result indicates better overall performance of a Python
interpreter. The latest copy of PyStone can be found in lib\test\pystone.py
(relative to CPython installation locations).

Notes:
  • By default pystone.py runs 50,000 passes to determine PyStones per second. The results below were generated using this default
  • Due to site caching in the Dynamic Language Runtime, IronPython performs better with more PyStone passes than the default value

Results
Best IronPython 2.6A1 PyStone run (out of 5 attempts): 26282.6 PyStones/second
Best IronPython 2.6B1 PyStone run (out of 5 attempts): 46711.8 PyStones/second

PyBench 2.0

Background
PyBench is a collection of low-level benchmarks distributed with CPython. Whereas
PyStone can be used to discover the overall performance of the Python interpreter,
PyBench targets specific areas (e.g., the amount of time it takes to concatenate
two strings). You can find the latest version of PyBench on CPython's Subversion
source control system (e.g., http://svn.python.org/projects/python/trunk/Tools/pybench/).

Please note the following:
  • Much of the platform module is broken under IPy. As such, build numbers, build dates, etc had to be faked under IPy
  • The IPy garbage collector cannot be disabled. Although this puts IPy at a disadvantage, we kept it turned off for the CPy run
  • sys.setcheckinterval does not function properly under IronPython. This is another optimization CPython gets that IronPython doesn't
  • The default number of rounds and a warp factor of 1 were used

Results
(this=C:\SnapTemp\IronPython 2.6B1, other=C:\SnapTemp\IronPython 2.6A1)
Test min run-time this min run-time other min run-time diff avg run-time this avg run-time other avg run-time diff
BuiltinFunctionCalls: 279ms 373ms -25.1% 287ms 602ms -52.3%
BuiltinMethodLookup: 528ms 684ms -22.8% 537ms 940ms -42.8%
CompareFloats: 341ms 637ms -46.5% 363ms 946ms -61.7%
CompareFloatsIntegers: 122ms 123ms -0.6% 141ms 417ms -66.1%
CompareIntegers: 199ms 215ms -7.3% 218ms 764ms -71.5%
CompareInternedStrings: 349ms 349ms +0.0% 356ms 830ms -57.1%
CompareLongs: 638ms 746ms -14.5% 655ms 1012ms -35.2%
CompareStrings: 446ms 462ms -3.4% 466ms 778ms -40.1%
CompareUnicode: 339ms 339ms +0.0% 353ms 583ms -39.6%
ComplexPythonFunctionCalls: 2352ms 2150ms 9.4% 2402ms 2368ms 1.5%
ConcatStrings: 4303ms 4396ms -2.1% 4610ms 4998ms -7.8%
ConcatUnicode: 2541ms 2728ms -6.9% 2813ms 2964ms -5.1%
CreateInstances: 918ms 934ms -1.7% 952ms 1048ms -9.1%
CreateNewInstances: 1278ms 1261ms +1.3% 1326ms 1400ms -5.3%
CreateStringsWithConcat: 2006ms 2178ms -7.9% 2373ms 2783ms -14.7%
CreateUnicodeWithConcat: 809ms 887ms -8.8% 921ms 1113ms -17.2%
DictCreation: 1573ms 1636ms -3.8% 1644ms 2185ms -24.8%
DictWithFloatKeys: 2273ms 2399ms -5.2% 2285ms 2748ms -16.8%
DictWithIntegerKeys: 2053ms 2225ms -7.7% 2061ms 2651ms -22.3%
DictWithStringKeys: 2147ms 2350ms -8.6% 2174ms 2807ms -22.6%
ForLoops: 390ms 390ms +0.0% 398ms 3000ms -86.7%
IfThenElse: 215ms 229ms -6.5% 236ms 789ms -70.1%
ListSlicing: 3463ms 3510ms -1.3% 3557ms 3632ms -2.1%
NestedForLoops: 437ms 452ms -3.4% 457ms 2632ms -82.6%
NormalClassAttribute: 3803ms 3788ms +0.4% 3846ms 4214ms -8.7%
NormalInstanceAttribute: 3117ms 3164ms -1.5% 3154ms 3566ms -11.5%
PythonFunctionCalls: 451ms 404ms +11.6% 474ms 618ms -23.2%
PythonMethodCalls: 3291ms 3478ms -5.4% 3344ms 3668ms -8.8%
Recursion: 902ms 870ms +3.6% 908ms 948ms -4.3%
SecondImport: 2792ms 2464ms 13.3% 2857ms 2633ms 8.5%
SecondPackageImport: 2776ms 2511ms 10.6% 2818ms 2701ms 4.3%
SecondSubmoduleImport: 3759ms 3369ms 11.6% 3795ms 3583ms 5.9%
SimpleComplexArithmetic: 544ms 684ms -20.5% 587ms 1005ms -41.6%
SimpleDictManipulation: 2240ms 2415ms -7.2% 2276ms 2914ms -21.9%
SimpleFloatArithmetic: 480ms 496ms -3.1% 510ms 944ms -45.9%
SimpleIntFloatArithmetic: 512ms 527ms -3.0% 530ms 964ms -45.0%
SimpleIntegerArithmetic: 512ms 512ms -0.0% 524ms 942ms -44.4%
SimpleListManipulation: 1369ms 1431ms -4.4% 1394ms 1892ms -26.3%
SimpleLongArithmetic: 1012ms 1044ms -3.0% 1022ms 1279ms -20.1%
SmallLists: 1480ms 1573ms -5.9% 1495ms 2066ms -27.6%
SmallTuples: 1776ms 1870ms -5.0% 1807ms 2871ms -37.1%
SpecialClassAttribute: 4069ms 3772ms +7.9% 4106ms 4182ms -1.8%
SpecialInstanceAttribute: 3179ms 3132ms +1.5% 3201ms 3540ms -9.6%
StringMappings: 3008ms 2977ms +1.0% 3038ms 3244ms -6.3%
StringPredicates: 1299ms 1472ms -11.7% 1312ms 1845ms -28.9%
StringSlicing: 1415ms 1431ms -1.1% 1523ms 1838ms -17.1%
TryExcept: 27ms 27ms -2.9% 45ms 110ms -58.9%
TryFinally: 996ms 887ms +12.3% 1029ms 1182ms -12.9%
TryRaiseExcept: 54398ms 53802ms 1.1% 54723ms 54419ms 0.6%
TupleSlicing: 1872ms 1919ms -2.4% 1927ms 2069ms -6.9%
UnicodeMappings: 1996ms 2044ms -2.3% 2009ms 2086ms -3.7%
UnicodePredicates: 1375ms 1532ms -10.2% 1389ms 1861ms -25.4%
UnicodeSlicing: 1197ms 1260ms -5.0% 1230ms 1628ms -24.4%
WithFinally: 700ms 762ms -8.2% 729ms 1083ms -32.7%
WithRaiseExcept: 78825ms 62740ms 25.6% 78926ms 64881ms 21.6%
Totals: 215171ms 200010ms +7.6% 218116ms 224765ms -3.0%

Richards

Background
Much like PyStone, richards.py is a high-level benchmark intended to give the
Python user a single result indicating the overall "goodness" factor of a Python
interpreter. This benchmark is distributed with the PyPy interpreter, although
the original version was written in BCPL by Dr. Martin Richards at Cambridge
University. Lower "Average time per iteration" results indicate better performance.

Results
Best average time per iteration for IronPython 2.6A1 (out of 5 attempts): 1419.61
Best average time per iteration for IronPython 2.6B1 (out of 5 attempts): 1341.61

Last edited May 21, 2009 at 4:01 PM by dfugate, version 1

Comments

No comments yet.