cysignals¶
This is the documentation for cysignals, a package to deal with interrupts and signal handling in Cython code.
When writing Cython code, special care must be
taken to ensure that the code can be interrupted with CTRL-C
.
Since Cython optimizes for speed, Cython normally does not check for
interrupts. For example, code like the following cannot be interrupted
in Cython:
while True:
pass
While this is running, pressing CTRL-C
has no effect. The only way
out is to kill the Python process. On certain systems, you can still
quit Python by typing CTRL-\
(sending a Quit signal) instead of
CTRL-C
.
The package cysignals provides functionality to deal with this,
see Interrupt handling.
Besides this, cysignals also provides Python functions/classes to deal with signals. These are not directly related to interrupts in Cython, but provide some supporting functionality beyond what Python provides, see Signal-related interfaces for Python code.
Interrupt/Signal Handling¶
Dealing with interrupts and other signals using sig_check
and sig_on
:
Interrupt handling¶
cysignals provides two related mechanisms to deal with interrupts:
Use sig_check() if you are writing mixed Cython/Python code. Typically this is code with (nested) loops where every individual statement takes little time.
Use sig_on() and sig_off() if you are calling external C libraries or inside pure Cython code (without any Python functions) where even an individual statement, like a library call, can take a long time.
The functions sig_check()
, sig_on()
and sig_off()
can be put in all
kinds of Cython functions: def
, cdef
or cpdef
. You cannot put them
in pure Python code (files with extension .py
).
Basic example¶
The sig_check()
in the loop below ensures that the loop can be
interrupted by CTRL-C
:
from cysignals.signals cimport sig_check
from libc.math cimport sin
def sine_sum(double x, long count):
cdef double s = 0
for i in range(count):
sig_check()
s += sin(i*x)
return s
See the example directory for this complete working example.
Note
Cython cdef
or cpdef
functions with a return type (like cdef int
myfunc():
) need to have an except value
to propagate exceptions. Remember this whenever you write sig_check()
or
sig_on()
inside such a function, otherwise you will see a message like
Exception KeyboardInterrupt: KeyboardInterrupt() in <function name>
ignored
.
Using sig_check()
¶
sig_check()
can be used to check for pending interrupts. If an interrupt
happens during the execution of C or Cython code, it will be caught by the next
sig_check()
, the next sig_on()
or possibly the next Python statement.
With the latter we mean that certain Python statements also check for
interrupts, an example of this is the print
statement. The following loop
can be interrupted:
>>> while True:
... print("Hello")
The typical use case for sig_check()
is within tight loops doing complicated
stuff (mixed Python and Cython code, potentially raising exceptions). It is
reasonably safe to use and gives a lot of control, because in your Cython code,
a KeyboardInterrupt
can only be raised during sig_check()
:
from cysignals.signals cimport sig_check
def sig_check_example():
for x in foo:
# (one loop iteration which does not take a long time)
sig_check()
This KeyboardInterrupt
is treated like any other Python exception and can be
handled as usual:
from cysignals.signals cimport sig_check
def catch_interrupts():
try:
while some_condition():
sig_check()
do_something()
except KeyboardInterrupt:
# (handle interrupt)
Of course, you can also put the try
/except
inside the loop in the
example above.
The function sig_check()
is an extremely fast inline function which should
have no measurable effect on performance.
Using sig_on()
and sig_off()
¶
Another mechanism for interrupt handling is the pair of functions sig_on()
and sig_off()
. It is more powerful than sig_check()
but also a lot more
dangerous. You should put sig_on()
before and sig_off()
after any
Cython code which could potentially take a long time. These two must always be
called in pairs, i.e. every sig_on()
must be matched by a closing
sig_off()
.
In practice your function will probably look like:
from cysignals.signals cimport sig_on, sig_off
def sig_example():
# (some harmless initialization)
sig_on()
# (a long computation here, potentially calling a C library)
sig_off()
# (some harmless post-processing)
return something
It is possible to put sig_on()
and sig_off()
in different functions,
provided that sig_off()
is called before the function which calls
sig_on()
returns. The reason is that sig_on()
is implemented using
setjmp()
, which requires that the stack frame is kept alive.
Therefore, the following code is invalid:
# INVALID code because we return from function foo()
# without calling sig_off() first.
cdef foo():
sig_on()
def f1():
foo()
sig_off()
But the following is valid since you cannot call foo
interactively:
from cysignals.signals cimport sig_on, sig_off
cdef int foo():
sig_off()
return 2+2
def f1():
sig_on()
return foo()
For clarity however, it is best to avoid this.
A common mistake is to put sig_off()
towards the end of a function (before
the return
) when the function has multiple return
statements. So make
sure there is a sig_off()
before every return
(and also before every
raise
).
Warning
The code inside sig_on()
should be pure C or Cython code. If you call
any Python code or manipulate any Python object (even something trivial like
x = []
), an interrupt can mess up Python’s internal state. When in
doubt, try to use sig_check() instead.
Also, when an interrupt occurs inside sig_on()
, code execution
immediately stops without cleaning up. For example, any memory allocated
inside sig_on()
is lost. See Signal handling without exceptions for ways to deal with
this.
When the user presses CTRL-C
inside sig_on()
, execution will jump back
to sig_on()
(the first one if there is a stack) and sig_on()
will raise
KeyboardInterrupt
. As with sig_check()
, this exception can be handled in
the usual way:
from cysignals.signals cimport sig_on, sig_off
def catch_interrupts():
try:
sig_on() # This must be INSIDE the try
# (some long computation)
sig_off()
except KeyboardInterrupt:
# (handle interrupt)
It is possible to stack sig_on()
and sig_off()
. If you do this, the
effect is exactly the same as if only the outer sig_on()
/sig_off()
was
there. The inner ones will just change a reference counter and otherwise do
nothing. Make sure that the number of sig_on()
calls equal the number of
sig_off()
calls:
from cysignals.signals cimport sig_on, sig_off
def f1():
sig_on()
x = f2()
sig_off()
cdef f2():
sig_on()
# ...
sig_off()
return ans
Extra care must be taken with exceptions raised inside sig_on()
. The problem
is that, if you do not do anything special, the sig_off()
will never be
called if there is an exception. If you need to raise an exception yourself,
call a sig_off()
before it:
from cysignals.signals cimport sig_on, sig_off
def raising_an_exception():
sig_on()
# (some long computation)
if (something_failed):
sig_off()
raise RuntimeError("something failed")
# (some more computation)
sig_off()
return something
Alternatively, you can use try
/finally
which will also catch exceptions
raised by subroutines inside the try
:
from cysignals.signals cimport sig_on, sig_off
def try_finally_example():
sig_on() # This must be OUTSIDE the try
try:
# (some long computation, potentially raising exceptions)
return something
finally:
sig_off()
If you also want to catch this exception, you need a nested try
:
from cysignals.signals cimport sig_on, sig_off
def try_finally_and_catch_example():
try:
sig_on()
try:
# (some long computation, potentially raising exceptions)
finally:
sig_off()
except Exception:
print("Trouble! Trouble!")
sig_on()
is implemented using the C library call setjmp()
which takes a
very small but still measurable amount of time. In very time-critical code, one
can conditionally call sig_on()
and sig_off()
:
from cysignals.signals cimport sig_on, sig_off
def conditional_sig_on_example(long n):
if n > 100:
sig_on()
# (do something depending on n)
if n > 100:
sig_off()
This should only be needed if both the check (n > 100
in the example) and
the code inside the sig_on()
block take very little time.
Handling other signals¶
Apart from handling interrupts, sig_on()
provides more general signal handling.
For example, it handles alarm()
time-outs by raising an
AlarmInterrupt
(inherited from KeyboardInterrupt
) exception.
If the code inside sig_on()
would generate a segmentation fault or call the
C function abort()
(or more generally, raise any of SIGSEGV, SIGILL,
SIGABRT, SIGFPE, SIGBUS), this is caught by the interrupt framework and an
exception is raised (RuntimeError
for SIGABRT, FloatingPointError
for
SIGFPE and the custom exception SignalError
, based on BaseException
,
otherwise):
from libc.stdlib cimport abort
from cysignals.signals cimport sig_on, sig_off
def abort_example():
sig_on()
abort()
sig_off()
>>> abort_example()
Traceback (most recent call last):
...
RuntimeError: Aborted
This exception can be handled by a try
/except
block as explained above.
A segmentation fault or abort()
unguarded by sig_on()
would simply
terminate the Python Interpreter. This applies only to sig_on()
, the
function sig_check()
only deals with interrupts and alarms.
Instead of sig_on()
, there is also a function sig_str(s)
, which takes a
C string s
as argument. It behaves the same as sig_on()
, except that the
string s
will be used as a string for the exception. sig_str(s)
should
still be closed by sig_off()
. Example Cython code:
from libc.stdlib cimport abort
from cysignals.signals cimport sig_str, sig_off
def abort_example_with_sig_str():
sig_str("custom error message")
abort()
sig_off()
Executing this gives:
>>> abort_example_with_sig_str()
Traceback (most recent call last):
...
RuntimeError: custom error message
With regard to ordinary interrupts (i.e. SIGINT), sig_str(s)
behaves the
same as sig_on()
: a simple KeyboardInterrupt
is raised.
Further topics in interrupt/signal handling¶
Testing interrupts¶
When writing documentation, one sometimes wants to check that certain
code can be interrupted in a clean way. The best way to do this is to
use cysignals.alarm()
.
The following is an example of a doctest demonstrating that the
SageMath function factor()
can be interrupted:
>>> from cysignals.alarm import alarm, AlarmInterrupt
>>> try:
... alarm(0.5)
... factor(10**1000 + 3)
... except AlarmInterrupt:
... print("alarm!")
alarm!
If you use the SageMath doctesting framework, you can instead doctest
the exception in the usual way (the Python doctest
module exits
whenever a KeyboardInterrupt
is raised in a doctest).
To avoid race conditions, make sure that the calls to alarm()
and
the function you want to test are in the same doctest:
>>> alarm(0.5); factor(10**1000 + 3)
Traceback (most recent call last):
...
AlarmInterrupt
Signal handling without exceptions¶
There are several more specialized functions for dealing with interrupts. As
mentioned above, sig_on()
makes no attempt to clean anything up (restore
state or freeing memory) when an interrupt occurs. In fact, it would be
impossible for sig_on()
to do that. If you want to add some cleanup code,
use sig_on_no_except()
for this. This function behaves exactly like
sig_on()
, except that any exception raised (like KeyboardInterrupt
or
RuntimeError
) is not yet passed to Python. Essentially, the exception is
there, but we prevent Cython from looking for the exception. Then
cython_check_exception()
can be used to make Cython look for the exception.
Normally, sig_on_no_except()
returns 1. If a signal was caught and an
exception raised, sig_on_no_except()
instead returns 0. The following
example shows how to use sig_on_no_except()
:
def no_except_example():
if not sig_on_no_except():
# (clean up messed up internal state)
# Make Cython realize that there is an exception.
# It will look like the exception was actually raised
# by cython_check_exception().
cython_check_exception()
# (some long computation, messing up internal state of objects)
sig_off()
There is also a function sig_str_no_except(s)
which is analogous to
sig_str(s)
.
Note
See the file src/cysignals/tests.pyx
for more examples of how to use the various sig_*()
functions.
Releasing the Global Interpreter Lock (GIL)¶
All the functions related to interrupt and signal handling do not require the
Python GIL
(if you don’t know what this means, you can safely ignore this section), they
are declared nogil
. This means that they can be used in Cython code inside
with nogil
blocks. If sig_on()
needs to raise an exception, the GIL is
temporarily acquired internally.
If you use C libraries without the GIL and you want to raise an exception before calling sig_error(), remember to acquire the GIL while raising the exception. Within Cython, you can use a with gil context.
Warning
The GIL should never be released or acquired inside a sig_on()
block. If
you want to use a with nogil
block, put both sig_on()
and
sig_off()
inside that block. When in doubt, choose to use
sig_check()
instead, which is always safe to use.
Debugging Python crashes¶
If cysignals
is imported, it sets up a hook which triggers when
Python crashes. For example, it would be triggered on a segmentation
fault outside a sig_on()
block.
When a crash happens, first a simple C backtrace is printed if supported by the C library on the system. Then GDB is run to print a much more complete backtrace (except on OS X, where running a debugger requires special privileges). For your convenience, these GDB backtraces are also saved to a logfile.
Finally, this familiar message is shown:
This probably occurred because a *compiled* module has a bug
in it and is not properly wrapped with sig_on(), sig_off().
Python will now terminate.
Environment variables¶
There are several environment variables which influence this:
- CYSIGNALS_CRASH_QUIET¶
If set, be completely quiet whenever a crash happens. No backtrace or other message is shown and GDB is not run.
- CYSIGNALS_CRASH_NDEBUG¶
If set, disable the GDB backtrace. The simple backtrace is still shown.
- CYSIGNALS_CRASH_LOGS¶
The directory where the logs of the crashes are stored. If this is empty, disable storing of crash logs. The default is
cysignals_crash_logs
in the current directory.
- CYSIGNALS_CRASH_DAYS¶
Automatically delete crash logs older than this many days in the directory where crash logs are stored. A negative value means that logs are never deleted. The default is 7 days if
CYSIGNALS_CRASH_LOGS
is unset and -1 days (never delete) otherwise.
Error handling¶
Defining error callbacks for external libraries using sig_error
:
Error handling in C libraries¶
Some C libraries can produce errors and use some sort of callback mechanism to report errors: an external error handling function needs to be set up which will be called by the C library if an error occurs.
The function sig_error()
can be used to deal with these errors. This
function may only be called within a sig_on()
block (otherwise the Python
interpreter will crash hard) after raising a Python exception. You need to use
the Python/C API for this
and call sig_error()
after calling some variant of PyErr_SetObject()
.
Even within Cython, you cannot use the raise
statement, because then the
sig_error()
will never be executed. The call to sig_error()
will use the
sig_on()
machinery such that the exception will be seen by sig_on()
.
A typical error handler implemented in Cython would look as follows:
from cysignals.signals cimport sig_error
from cpython.exc cimport PyErr_SetString
cdef void error_handler(char *msg):
PyErr_SetString(RuntimeError, msg)
sig_error()
Exceptions which are raised this way can be handled as usual by putting
the sig_on()
in a try
/except
block.
For example, the package cypari2
provides a wrapper around the number theory library PARI/GP.
The error handler
has a callback which turns errors from PARI/GP
into Python exceptions of type PariError
.
This can be handled as follows:
from cysignals.signals cimport sig_on, sig_off
def handle_pari_error():
try:
sig_on() # This must be INSIDE the try
# (call to PARI)
sig_off()
except PariError:
# (handle error)
SageMath uses this mechanism for libGAP, GLPK, NTL and PARI.
Links¶
cysignals on the Python package index: https://pypi.org/project/cysignals/
cysignals code repository and issue tracker on GitHub: https://github.com/sagemath/cysignals
cysignals documentation on Read the Docs: https://cysignals.readthedocs.io