The New IronPython Exception Model

With our next release we’re planning on offering a new exception model that attempts to fix the problems with our current model. This document discusses the problems with the current model and the current thinking for the new model.

One World or where we are…

In IronPython’s current exception implementation we have a unified exception type hierarchy. The Exception class that is visible Python programmers is an instance of PythonException. PythonException in turn derives from System.Exception and below PythonException there is the standard Python exception hierarchy. This results in a couple of perceptible differences between IronPython and CPython.

There’s a small difference in that our current exception hierarchy does not start at “Exception”. That’s not likely to break anyone anytime soon. There’s a more significant difference in that our exceptions currently are built-in types that can’t have arbitrary attributes added. And there’s also a difference in that we are currently limited in what Python programmers can throw; in CPython you can throw any object but we currently only allow you to throw Exceptions.

All of those aren’t very big problems and seem like they could mostly be tolerated or at the very least we could add some small workarounds to accommodate adding arbitrary attributes, hiding Exceptions base, or simply throwing arbitrary objects (as the CLI does allow this). But there are larger issues of what our interop story looks like with the rest of the CLI world. Before we look at where we’re going let’s look at what the interop story looks like today.

First let’s look at what it means to raise an exception from IronPython. When we encounter the raise statement we’ll end up calling one of two IronPython’s Ops.Raise method. Ultimately this method will either throw the exception. If the user passes in a pre-created instance (raise Exception()) we’ll just do throw of that object. If the user passed in a type we’ll go ahead and allocate the instance of that type and throw that. And as a special case today we allow throwing strings which we wrap up specially.

This exception is thrown using the CLI’s normal exception handling process. If user code has a try/except statement that handles the exception it will be caught and handled within the Python world. If running at the interpreter it will be caught by the interpreter and displayed to the user. And if called from some arbitrary CLI code we’ll leak the exception from Python into the CLI code as a “PythonException”, “PythonValueError”, “PythonTypeError”, etc…

Now let’s look at the interop story from the Python side. If you called some CLI code directly or if we failed to catch an error condition coming from some CLI code we call you’d see an unwrapped CLI exception. For example you might see an ArgumentNullException. But you’d only see it if you had imported System and explicitly done “except System.Exception:”! Otherwise there was no way for you to catch these exceptions.

Obviously this isn’t a very good story. If you’re a CLI developer you’re seeing Python exceptions that you don’t know how to handle leak into your code. If you’re a Python developer you’re seeing CLI exceptions leak into your code. Only if you’re living in a pure Python world does the story even begin to work – and only with the caveats previously mentioned.

!!Two Worlds or where we’re going…

What we want to achieve is a model where Python and CLI see a unified exception model. At the same time we want each environment to see its own exception model to its full fidelity.

The proposed exception model is one where we break apart the existing type hierarchy into two separate hierarchies. Under this model the Python Exception hierarchy will start at a base class “Exception” which will be an old-style class. There will be additional old-style classes that inherit from this (e.g. StandardError and further down ValueError) that fill out the rest of the exception type hierarchy. Because the Python exception hierarchy is represented by old-style classes these will never actually be thrown by IronPython. Because we keep the current Python exception hierarchy unchanged we’ll have all the standard old-style Python classes. If you write pure Python code with no reference to CLS libraries you’ll see exactly the current Python exception system.

On the other side of the world we have the CLI exception hierarchy. These are the exceptions that are understood by all languages that target the CLS. In order to maximize interoperability with other languages, IronPython will only throw these standard CLS exception objects. This means that other CLS languages will only see the exceptions they understand. Where the CLS exception hierarchy is not rich enough to capture the Python exception hierarchy we will derive new classes from the appropriate location in the CLS hierarchy.

When Worlds Collide…

So now if you’re a Python developer you see the Python exception hierarchy and that’s all you see and life is wonderful. Likewise if you’re a CLI developer calling Python code see exceptions that are just like you’d expect from any other CLI code. But what happens when the two worlds collide? This starts to happen when you have Python code that is directly calling other CLI code.

The core of this is: we respect except expression, …. If expression is a Python exception that is the exception we will catch and give you. If expression is a CLI exception that is also the exception we will catch and give you. That’s great if you’re working with some framework and want to catch a specific exception it throws.

A good example of this would be calling a file I/O API. Previously we’ve had bug reports where users would get a FileIOException instead of the more Python friendly IOError. Under the new model even if a CLI library throws a FileIOException or one of its subclasses the Python programmer will automatically see the IOError. Another similar example was when we failed to trap a bad argument to the intern function. Here we would throw ArgumentNullException but now this automatically gets translated into a TypeError.

Likewise we have a similar problem with Python code raises an exception. For example if Python code wanted to raise an EOFError CLS code would previously had no idea how to process this. Under this proposal CLS code will instead receive the EndOfStreamException is is expecting.

In general, Python exceptions and CLI exceptions should remain nicely separate because there are no overlapping names so it is always clear which exception you want to work with. The one potential troublemaker is Exception. If you do “from System import *” then exception suddenly has a new value. Therefore all of your exception Exception … statements will now behave differently – you’ll end up catching CLS exceptions instead of Python exceptions…

The one other tricky case is what should IronPython print when dumping a stack trace for an unhandled exception. In this case we have no clues to what world the user would like to see. Our current thinking is that we should display both sets of information and allow this to be user configurable. Another thought has been allowing a user-defined function that can be overridden to customize this functionality at runtime.

We’d love to hear what you think about the proposed change.

Implementation Details

The start of this is changing the exceptions that IronPython throws. Currently we throw our PythonException subclasses and now we’ll throw CLI classes. For example instead of throwing a ValueError exception we’ll throw an ArgumentException now. If a user calls raise we will likewise translate this into the closest CLI type and store the Python exception in the Data field of the exception.

The next change is in how IronPython catches exceptions. When an exception is caught the translation must be done. If the exception was raised from Python code the translation is easy – we merely need to extract the Python exception from the Data property. If the exception originated in CLI code (including the IronPython runtime) the exception will go through translation table. Ultimately we’ll create a new instance of the Python type that will be visible to the Python code.

If this exception is re-thrown then the original exception is pulled from the Python object. The original exception is also available for Python code to access if it wants to get additional information about the exception. This original exception value is also used in the event that the exception gets re-raised; it will be the exception value that gets thrown via the CLI.

Exception Hierarchy

Here’s the full proposed exception hierarchy mapping with a couple of notes on ones we’re still thinking about. Some of these exceptions map nicely to their CLS equivalents and in those cases we’ll use the CLS exceptions. In other cases CLS has no equivalent exceptions and in those cases we’ll define custom exceptions inside of IronpPython.

Exception System.Exception
SystemExit IP.O.SystemExit
StopIteration System.InvalidOperationException subtype
StandardError System.SystemException
KeyboardInterrupt IP.O.KeyboardInterruptException
ImportError IP.O.PythonImportError
EnvironmentError IP.O.PythonEnvironmentError
IOError System.IO.IOException
OSError S.R.InteropServices.ExternalException investigate where OSError comes from
WindowsError System.ComponentModel.Win32Exception
EOFError System.IO.EndOfStreamException
RuntimeError IP.O.RuntimeException
NotImplementedError System.NotImplementedException
NameError IP.O.NameException
UnboundLocalError IP.O.UnboundLocalException
AttributeError System.MissingMemberException
SyntaxError IP.O.SyntaxErrorException (System.Data has something close)
IndentationError IP.O.IndentationErrorException
TabError IP.O.TabErrorException
TypeError ArgumentTypeException
AssertionError IP.O.AssertionException
LookupError IP.O.LookupException
IndexError System.IndexOutOfRangeException
KeyError S.C.G.KeyNotFoundException
ArithmeticError System.ArithmeticException
OverflowError System.OverflowException
ZeroDivisionError System.DivideByZeroException
FloatingPointError IP.O.PythonFloatingPointError
ValueError ArgumentException
UnicodeError IP.O.UnicodeException
UnicodeEncodeError System.Text.EncoderFallbackException
UnicodeDecodeError System.Text.DecoderFallbackException
UnicodeTranslateError IP.O.UnicodeTranslateException
ReferenceError IP.O.ReferenceException
SystemError IP.O.PythonSystemError
MemoryError System.OutOfMemoryException
Warning System.ComponentModel.WarningException
UserWarning IP.O.PythonUserWarning
DeprecationWarning IP.O.PythonDeprecationWarning
PendingDeprecationWarning IP.O.PythonPendingDeprecationWarning
SyntaxWarning IP.O.PythonSyntaxWarning
OverflowWarning IP.O.PythonOverflowWarning
RuntimeWarning IP.O.PythonRuntimeWarning
FutureWarning IP.O.PythonFutureWarning

Last edited May 31, 2006 at 5:12 AM by dinov, version 1

Comments

No comments yet.