After I wrote my original post about the fibrous thread pool, I recalled an additional issue that I needed to address during implementation.

Suppose I'm executing a function on a fiber that has been scheduled by the fibrous thread pool. If I call ReadFile, I need to perform a suspend operation because the I/O completion port will generate a completion packet corresponding to the read operation. What if ReadFile returns TRUE? We already know that the operation has completed successfully, so why should we bother suspending? In older versions of Windows, the completion event fires no matter what. This leads to some extra activity in the fibrous thread pool that is merely housekeeping to ensure that our completion events are in sync. Unfortunately this housekeeping is effectively redundant. What we really need is a way to suppress I/O completion callbacks when a file I/O operation returns TRUE without ERROR_IO_PENDING being set.

The Windows 6.x (i.e. Vista and Server 2008) versions provide the solution via a new API: SetFileCompletionNotificationModes. This function allows us to disable I/O completion notifications in instances where an I/O operation meets the above criteria.

For all the criticism that Vista has received, I've got to say that I'm actually quite pleased with many of the additions to the Win32 API. I only wish that they had been made available sooner!