pthread_setcancelstate.3, pthreads.7, signal-safety.7: Describe issues with cancellation points in signal handlers

In a recent conversation with Mathieu Desnoyers I was reminded
that we haven't written up anything about how deferred
cancellation and asynchronous signal handlers interact. Mathieu
ran into some of this behaviour and I promised to improve the
documentation in this area to point out the potential pitfall.

Thoughts?

8< --- 8< --- 8<
In pthread_setcancelstate.3, pthreads.7, and signal-safety.7 we
describe that if you have an asynchronous signal nesting over a
deferred cancellation region that any cancellation point in the
signal handler may trigger a cancellation that will behave
as-if it was an asynchronous cancellation. This asynchronous
cancellation may have unexpected effects on the consistency of
the application. Therefore care should be taken with asynchronous
signals and deferred cancellation.

Signed-off-by: Michael Kerrisk <mtk.manpages@gmail.com>

man3/pthread_setcancelstate.3

@@ -78,7 +78,10 @@ A cancellation request is deferred until the thread next calls
 a function that is a cancellation point (see
 .BR pthreads (7)).
 This is the default cancelability type in all new threads,
-including the initial thread.
+including the initial thread. Even with deferred cancellation a
+cancellation point in an asynchronous signal handler may still
+be acted upon and the effect is as-if it was an asynchronous
+cancellation.
 .TP
 .B PTHREAD_CANCEL_ASYNCHRONOUS
 The thread can be canceled at any time.

man7/pthreads.7

@@ -564,6 +564,15 @@ not specified in the standard as cancellation points.
 In particular, an implementation is likely to mark
 any nonstandard function that may block as a cancellation point.
 (This includes most functions that can touch files.)
+.in
+.PP
+It should be noted that even if an application is not using
+asynchronous cancellation, that calling a function from the above list
+from an asynchronous signal handler may cause the equivalent of
+asynchronous cancellation. The underlying user code may not expect
+asynchronous cancellation and the state of the user data may become
+inconsistent. Therefore signals should be used with caution when
+entering a region of deferred cancellation.
 .\" So, scanning "cancellation point" comments in the glibc 2.8 header
 .\" files, it looks as though at least the following nonstandard
 .\" functions are cancellation points:

man7/signal-safety.7

@@ -314,6 +314,11 @@ is likely to remove
 .BR fork (2)
 from the list of async-signal-safe functions.
 .\"
+.IP * 3
+Asynchronous signal handlers that call functions which are cancellation
+points and nest over regions of deferred cancellation may trigger
+cancellation whose behavior is as-if asynchronous cancellation had
+occurred and may cause application state to become inconsistent.
 .SS Deviations in the GNU C library
 The following known deviations from the standard occur in
 the GNU C library: