The following came up yesterday on the wget list.
The iconv.3 man page says
"... 2. The input byte sequence has been entirely converted,
i.e. *inbytesleft has gone down to 0."
and
"A different case is when inbuf is NULL or *inbuf is NULL,
but outbuf is not NULL and *outbuf is not NULL. In this case,
the iconv function attempts to set cd's conversion state to the
initial state and store a corresponding shift sequence at
*outbuf."
The POSIX page says
"For state-dependent encodings, the conversion descriptor cd is
placed into its initial shift state by a call for which inbuf
is a null pointer, or for which inbuf points to a null pointer.
When iconv() is called in this way, and if outbuf is not a null
pointer or a pointer to a null pointer, and outbytesleft points
to a positive value, iconv() shall place, into the output buffer,
the byte sequence to change the output buffer to its initial
shift state."
These texts are slightly misleading, in the sense that, in the
present implementation, iconv() may implement conversion
from an encoding that is not state-dependent in a way that
uses an artificial shift state to store lookahead bytes.
That means that after conversion, when *inbytesleft has gone
down to 0, it may be that contrary to what iconv.3 suggests not
all output has been stored, and a final flushing call is needed.
Maybe this violates POSIX.
A minimal warning is added by this patch.
--- man-pages-4.03/man3/iconv.3 2015-12-05 10:45:25.000000000 +0100
+++ ./iconv.3 2015-12-16 01:41:38.253049938 +0100
@@ -161,6 +161,11 @@
.SH CONFORMING TO
POSIX.1-2001, POSIX.1-2008.
.SH NOTES
+In each series of calls to
+.BR iconv (),
+the last should be one with \fIinbuf\fP or \fI*inbuf\fP equal to NULL,
+in order to flush out any partially converted input.
+
Although
.I inbuf
and
Signed-off-by: Michael Kerrisk <mtk.manpages@gmail.com>
Darren Hart pointed me to the comments from Rich Felker
that there are valid use cases for FUTEX_REQUEUE.
From: Rich Felker <dalias@libc.org>
Date: Wed, 29 Oct 2014 22:43:17 -0400
To: Darren Hart <dvhart@infradead.org>
Cc: GLIBC Devel <libc-alpha@sourceware.org>, ...
Subject: Re: Add futex wrapper to glibc?
On Wed, Oct 29, 2014 at 06:59:15PM -0700, Darren Hart wrote:
[...]
> I wonder though... can we not wrap FUTEX_REQUEUE? It's fundamentally
> broken. FUTEX_CMP_REQUEUE should *always* be used instead. The glibc
> wrapper is one way to encourage developers to do the right thing
> (don't expose the bad op in the header).
You're mistaken here. There are plenty of valid ways to use
FUTEX_REQUEUE - for example if the calling thread is requeuing the
target(s) to a lock that the calling thread owns. Just because it
doesn't meet the needs of the way glibc was using it internally
doesn't mean it's useless for other applications.
Reported-by: Darren Hart <dvhart@infradead.org>
Signed-off-by: Michael Kerrisk <mtk.manpages@gmail.com>
As commented on by Davidlohr Bueso:
This to me reads a bit too much into the kernel (fastpath,
refcnt, vmas). Why not just mention that it avoids overhead
in the kernel or something? I don't recall any manpage
mentioning such details, but I could be wrong.
Reported-by: Davidlohr Bueso <dave@stgolabs.net>
Signed-off-by: Michael Kerrisk <mtk.manpages@gmail.com>
Michael Kerrisk