New Page 1

Filtering Serial Port Modem Status
May 1, 2003
Walter Oney

Not long ago, I read a newsgroup post asking how to create a filter driver that would monitor changes in the RS-232 control lines for a serial port. This question is similar to other questions that crop up online from time to time that have in common the concept of slightly modifying the behavior of an existing function driver by installing an upper filter. For this month's The Architect, I thought it would be useful to describe how to build such a driver. As you'll see, a bit of experience with IRP handling techniques in the kernel is all that's required to do it.

The Basics of a Solution

Let's first consider the full scope of the problem. The online questioner wants to write an application that can learn when certain RS-232 lines change state on a serial port that's in use by some other application. The fact that some other application may have the port open means he could not simply write a user-mode application that would call CreateFile followed by SetCommMask and WaitCommEvent. Nor would it work to write a legacy device driver that would use IoGetDeviceObjectPointer to access a particular COM port in order to do the equivalent operations via IRP_MJ_DEVICE_CONTROL requests using the IOCTL_SERIAL_SET_WAIT_MASK and IOCTL_SERIAL_WAIT_ON_MASK requests. In either case, the serial port driver will reject an attempt to open a second handle. (In case you forgot, or never knew: IoGetDeviceObjectPointer calls ZwOpenFile internally, and that eventually leads into the same code path as a user-mode call to CreateFile.)

The only way I know of to legally send IRPs to the serial driver when someone already has a handle open is to be in the PnP stack above it. Hence, as the newsgroup poster had already concluded, solving the problem requires creating an upper filter driver.

Let's think next about how our own application is going to be able to talk to our filter driver in order to find out about changes to control lines. Plainly, we're going to want to call CreateFile, but with what filename? It won't work to specify the name of the COM port. After all, if that worked, we wouldn't be needing a filter driver at all. We might guess that we could name our Filter Device Object (FiDO) and create a symbolic link named, for example, \DosDevices\Fred. (For the record, my online correspondent was not named Fred.) Then our application would call CreateFile with a filename argument of \\.\Fred. This won't work either unless we do something pretty sleazy in the filter driver. True, the Object Manager will resolve the name \DosDevices\Fred to our FiDO, but it will then send an IRP_MJ_CREATE to the topmost driver in the PnP stack. We will see this IRP in our own DispatchCreate routine. We might, by some kludge, be able to realize that the application program used our symbolic link name in its call to CreateFile. Then we could conceivably complete the IRP_MJ_CREATE without regard to whether the function driver below us thinks a handle is open. We would then want to flag the FILE_OBJECT in some special way so that we'll realize later on that IRPs flowing into our dispatch routines are coming from our special application instead of a bona fide user of the COM port. We would presumably handle a few IRP_MJ_DEVICE_CONTROL requests for the "special" handle.

It's architecturally better (in the sense that it fits better with the way the rest of the system works) to create what I call an "Extra" Device Object (EDO) that hangs off to the side of the PnP stack. We'll name the EDO, and our application will talk to it instead of the real COM port. We'll tie the EDO and the FiDO together by means of pointers in the device extension structures. This is the same concept discussed in Programming the Microsoft Windows Driver Model Second Edition (Microsoft Press 2003) on pages 793-94 and illustrated by this figure (borrowed from the book):

Figure One
Driver Architecture with EDO

Now let's think about how our special application will work. Recall that we want to find out about changes in RS-232 signal lines. So, we might define a DeviceIoControl operation where the input data is a bit mask specifying the signals we're interested in and the output data is a bit mask indicating which signals have actually occurred. We would expect our filter driver to pend this request and complete it only when one of the selected signals changes state. The application might look like this, assuming that I gave the EDO a symbolic name of \DosDevices\Barney:

HANDLE hDevice = CreateFile("\\\\.\\Barney", . . .);
DWORD EventMask = EV_CTS | EV_DSR;
DWORD Events;
DWORD junk;
DeviceIoControl(hDevice, IOCTL_BARNEY_WAIT_ON_MASK, &EventMask, sizeof(EventMask), &Events, sizeof(Events), &junk, NULL);

This call will not return until one of the CTS or DSR signal lines, or possibly both of them, changes state.

On the filter driver side, we'll have a DispatchCreate routine that uses some common field in the device extension structure to distinguish between IRPs targeted at the EDO or at the FiDO. When we get an IRP for the FiDO, we would plan to send it down the stack. When we get an IRP for the EDO, it will have to be an IRP_MJ_CREATE, IRP_MJ_CLEANUP, IRP_MJ_CLOSE, or IRP_MJ_DEVICE_CONTROL whose IoControlCode is IOCTL_BARNEY_WAIT_ON_MASK. Otherwise, we'll fail the IRP with STATUS_INVALID_DEVICE_REQUEST. There are some difficult mechanics for handling IOCTL_BARNEY_WAIT_ON_MASK because of the need to install a cancel routine, but you can use a cancel safe queue to solve most of them (or else crib from the Pending IOCTL stuff described on pages 500-05 of my book).

Everything I've said so far is a pretty straightforward application of well-known filter driver programming techniques, but I haven't yet indicated how we're going to know when one of the signal lines changes state. That is the kind of problem I like to sink my teeth into when I'm writing drivers.

Monitoring the Signal Lines

One way to keep track of signal line changes is to constantly poll the modem status register on the port. This is actually a dreadful idea, as I'll show you in a minute, but let's first walk down the garden path far enough to see how you'd do it. First, let's make the rash assumption that we're dealing with a standard UART that has a MSR whose port address equals a base address (such as 0x3F8) plus 6. We've just ruled out using our filter driver with a USB-to-serial device or a multiport board, but never mind. As a minor concession to the principle of configurability, we can read out the base port address for the resource list that accompanies IRP_MN_START_DEVICE. Then we can call PsCreateSystemThread to create a polling thread that will sit there and call READ_PORT_UCHAR in some sort of timed loop. (Memo to self: look at Oney's POLLING sample to see the mechanics of a kernel polling thread.) Apart from complexity and lack of extensibility to other types of serial port, can you see the flaw here?

The flaw in the polling idea is that we're going to be in competition with the real serial port driver when we read the port's MSR. Reading the MSR will clear the bits that indicate changes and will also clear any pending modem status interrupt. We will pretty certainly cause the real port driver to miss interrupts and changes in modem status that have implications for flow control and other things.

So we're not going to read the modem status register ourselves. We have to ask the port driver to do it for us by sending it an IOCTL_SERIAL_WAIT_ON_MASK. But there's a hitch. Each serial port has just one event mask, which is set by an IOCTL_SERIAL_SET_WAIT_MASK. Furthermore, you're only allowed to have one IOCTL_SERIAL_WAIT_ON_MASK outstanding at one time. If the application that's really using our port happens to want to call SetCommMask or WaitCommEvent, it will be using those same IOCTLs.

Therefore, what we need to do is to virtualize the event mask for our port. Here's a sketch of how we'd do that:

We'll keep a shadow event mask in our FiDO's device extension. We'll change the shadow mask in response to IOCTL_SERIAL_SET_WAIT_MASK requests that come to the FiDO.
We'll keep another mask -- I'll call it the E-mask -- of events that the EDO user (our special application, in other words) is interested in. We'll change this mask in response to IOCTL_BARNEY_WAIT_ON_MASK requests that come to the EDO.
Each time either the shadow mask or the E-mask changes, we'll "or" them together and send an IOCTL_SERIAL_SET_WAIT_MASK down to the port driver. The port driver will, therefore, be looking for any of the changes that interest either our special application or the actual user of the port.
We'll normally pend an IOCTL_SERIAL_WAIT_ON_MASK or IOCTL_BARNEY_WAIT_ON_MASK that comes to the FiDO or the EDO, respectively. We'll need to very closely read the DDK documentation on IOCTL_SERIAL_WAIT_ON_MASK in order to handle it correctly in a variety of situations. This step is what it means to "virtualize" the WAIT_ON_MASK request.
We'll keep an IOCTL_SERIAL_WAIT_ON_MASK outstanding at all times except when the composite event mask is zero. Our completion routine for this IRP will reissue the IRP in order to keep it pending. Each time one of our WAIT_ON_MASK requests complete, we'll conceivably need to complete one or the other (or both) of a FiDO's IOCTL_SERIAL_WAIT_ON_MASK or an EDO's IOCTL_BARNEY_WAIT_ON_MASK.

I can see many details lurking in the cracks of this summary. Since this an architecture article, I can cheerfully shrug my shoulders and leave them for another day.

About the author:

Walter Oney is a freelance driver programmer, seminar leader, and author based in Boston, Massachusetts. You can reach him by e-mail at waltoney@oneysoft.com. Information about the Walter Oney Software seminar series, and other services, is available online at http://www.oneysoft.com.