Copyright © 2003 by Peter Jaquiery. All rights reserved
From WDM to WDF in five hours--this article explores the wonderfully unexciting adventure of writing the world's first WDF driver for real hardware. What was it like to port a simple bulk USB driver from WDM to WDF? Read on.
|
The technical details for future products as discussed at DevCon were provided under Non-Disclosure Agreement with Microsoft. This article discusses only technical information that you can learn from the publicly available betas provided through MSDN subscriptions |
The context and back story:
Our company makes a data acquisition system and software that is used in medical teaching and research labs to collect and analyze data. In early '99 we added a USB interface to our hardware and wrote a driver for it based on the BulkUSB sample provided with the Windows 98 DDK. A couple of years later, we got sick of the sudden disconnect dialog in Windows 2000. We updated the PnP handling in the driver to suppress the dialog. Early this year, we upgraded our hardware to the USB 2.0 specification. The original driver just kept right on working--well, almost anyway :-). Which all goes to say that the original driver was pretty simple and has been satisfyingly reliable.
In November of 2003, Microsoft held their first Windows Driver Developers Conference. I attended. During the conference, there were a number of sessions introducing the Windows Driver Framework (WDF), Microsoft's new driver framework (see FrameworkIntro.htm for an introduction to an earlier version of this technology). Toward the end of one of these sessions, I realized that there didn't seem to be much work required to get a driver as simple as ours going using WDF, so I stuck up my hand and asked, "Do you think it is possible to port our simple bulk USB driver from WDM to WDF during the six hour interactive session tomorrow?". The reply was along the lines of: "That would be pretty cool. See me after this session and we'll get it organized". So, there was the mission. The remainder of this article describes the journey that resulted.
Phase one, identify the problem and up the ante:
At the stage of development shown at the DDC, the WDF samples include an equivalent to the '98 DDK BulkUSB sample that our original driver was based on. The WDF sample actually handles isochronous and bulk transfer support; perhaps there wouldn't be enough sample code otherwise! Ultimately the isochronous code gets dumped for our driver, but not for a while yet.
The first step was to look through the WDF sample and identify the areas where changes needed to be made. At this point I had Eliyas Yakub (the author of the WDF samples) assisting me. In the first pass through it looked like the only changes would be in Queue.c (which provides the code for dispatch routines for create, close, device-control, read and write). The work entailed handling three IOCTLs that are used to get a few hardware specific vendor request actions performed. It about this point Eliyas had to head off to present a session, so I decided it was about time to get the WDF sample code building in our development environment.
Much to Microsoft's surprise and horror, we use Visual Studio to build our driver. At the conference I learned why this is a bad idea, but that is another story. However, the actual changes to the sample WDF driver looked so trivial that I decided that getting it building under Visual Studio would add enough challenge to make the whole mission just a tad more interesting. By the time Eliyas was finished with his session, and with the help of a couple of other people in the WDF group, I had the WDF sample compiling with a few link errors. To achieve that I needed to use the following include paths in the C/C++ properties for the Visual Studio project:
For the standard command line build all this is laid on. It is only deviant people (like me) who want to develop using the Visual Studio IDE and compiler need worry about this! [Ed: See Mark Roddy's summary of ways to use the Visual Studio IDE for driver projects at http://www.wd-3.com/archive/howbuild.htm] In similar fashion, you need to add a list of libraries to the linker properties for Visual Studio:
Again this is work that is only required because I am building using the Visual Studio IDE. Conventional builds using the command line tools do not require this extra futzing about!
Getting to the point where I had the unaltered WDF Bulk/Isochronous sample compiling took about one hour (not including time out at various points to listen in on Eliyas's session).
Phase two, Ye Olde Cutte and Paiste
Now that I had the WDF sample code compiling in my development system of choice, it was time to start the real work. The first part was just a copy and paste of the pertinent code from the original WDM driver into the appropriate places in the WDF driver. There were in effect two chunks of code copied from the old driver; the entries in the switch statement to dispatch the IOCTLs for IRP_MJ_DEVICE_CONTROL (generated by calls to DeviceIoControl()), and the functions that perform the work for the IOCTLs. A typical function from the original driver looks like this:
NTSTATUS GetPowerLabAppID
(PDEVICE_OBJECT DeviceObject, short int *appID)
/*
Routine Description:
Get a PowerLab App ID
Return Value:
STATUS_SUCCESS if successful,
STATUS_UNSUCCESSFUL otherwise
*/
{
PDEVICE_EXTENSION deviceExtension = DeviceObject->DeviceExtension;
NTSTATUS ntStatus = STATUS_SUCCESS;
PURB urb;
urb = BULKUSB_ExAllocatePool
(NonPagedPool, sizeof(struct _URB_CONTROL_VENDOR_OR_CLASS_REQUEST));
if (! urb)
{
BULKUSB_KdPrint (DBGLVL_DEFAULT, ("< GetPowerLabAppID can't allocate URB\r\n"));
return STATUS_INSUFFICIENT_RESOURCES;
}
UsbBuildVendorRequest
(
urb,
URB_FUNCTION_VENDOR_DEVICE,
(USHORT) sizeof(struct _URB_CONTROL_VENDOR_OR_CLASS_REQUEST),
USBD_TRANSFER_DIRECTION_IN,
0,
12,
0,
0, appID, 0, 2, // Bytes to transfer
0
);
ntStatus = BulkUsb_CallUSBD (DeviceObject, urb);
BULKUSB_ExFreePool (urb);
return ntStatus;
}
This is pretty standard stuff. Allocate some memory for an urb, initialise it, then call BulkUsb_CallUSBD to get the work done. Free up the memory, then return the status from BulkUsb_CallUSBD. [Ed: I would just declare the URB as an automatic variable, since the URB structure is small enough that we don't have to worry about a stack overflow. To each his own...]
The pertinent code from the IRP_MJ_DEVICE_CONTROL dispatch function looks like this (some code omitted):
NTSTATUS BulkUsb_ProcessIOCTL (IN PDEVICE_OBJECT DeviceObject, IN PIRP Irp)
{
PIO_STACK_LOCATION irpStack;
PVOID ioBuffer;
ULONG ioControlCode;
ULONG inputBufferLength;
ULONG outputBufferLength;
...
BulkUsb_IncrementIoCount (DeviceObject);
...
irpStack = IoGetCurrentIrpStackLocation (Irp);
Irp->IoStatus.Status = STATUS_SUCCESS;
Irp->IoStatus.Information = 0;
ioBuffer = Irp->AssociatedIrp.SystemBuffer;
inputBufferLength = irpStack->Parameters.DeviceIoControl.InputBufferLength;
outputBufferLength = irpStack->Parameters.DeviceIoControl.OutputBufferLength;
...
ioControlCode = irpStack->Parameters.DeviceIoControl.IoControlCode;
switch (ioControlCode)
{
...
case IOCTL_BULKUSB_GET_AppID:
{
// This api returns an Application ID from the PowerLab
//
// inputs - none
// outputs - AppID in IoStatus.Information
switch (ntStatus = GetPowerLabAppID (DeviceObject, ioBuffer))
{
case STATUS_SUCCESS:
Irp->IoStatus.Information = 2;
Irp->IoStatus.Status = STATUS_SUCCESS;
break;
default:
Irp->IoStatus.Information = 0;
Irp->IoStatus.Status = ntStatus;
ntStatus = STATUS_DEVICE_DATA_ERROR;
break;
}
break;
}
...
}
IoCompleteRequest (Irp, IO_NO_INCREMENT);
BulkUsb_DecrementIoCount (DeviceObject);
return ntStatus;
}
Again pretty standard stuff. Set up a few variables to get access to the IOCTL and the buffer passed in from the application. Get the work done by calling the handler function. Test the result and either set Irp->IoStatus.Information to the number of bytes transferred (on success), or set it to 0 and return a fail result. [Ed: I think it would be better to use an IO_REMOVE_LOCK instead of a home-grown locking mechanism like BULKUSB does. But the point here is to show how to take a WDM driver and adapt it quickly to use WDF.]
Phase three, code conversion:
On the face of it there is nothing particularly tricky in any of that, although a few comments on the parameters passed to UsbBuildVendorRequest would help understanding a little!
By this time Eliyas had returned and we started in on the conversion, mostly by employing my favourite coding technique: deleting lines of code. So, how did our first attempt at the WDF version look? The easiest bit was the switch statement entry in the IRP_MJ_DEVICE_CONTROL (DeviceIoControl initiated) dispatch routine. It became:
void UsbSamp_EvtIoDeviceControl
(
IN WDFQUEUE Queue,
IN WDFREQUEST Request,
IN ULONG OutputBufferLength,
IN ULONG InputBufferLength,
IN ULONG IoControlCode
)
{
WDFDEVICE device;
PVOID ioBuffer;
ULONG info;
NTSTATUS ntStatus;
PDEVICE_CONTEXT pDevContext;
info = 0;
device = WdfIoQueueGetDevice(Queue);
pDevContext = GetDeviceContext(device);
ntStatus = WdfRequestRetrieveBuffer(Request, &ioBuffer);
if(!NT_SUCCESS(ntStatus))
{
WdfRequestCompleteWithInformation(Request, ntStatus, 0);
return;
}
switch(IoControlCode)
{
...
case IOCTL_BULKUSB_GET_AppID:
// This api returns an Application ID from the PowerLab
//
// inputs - none
// outputs - AppID in IoStatus.Information
ntStatus = GetPowerLabAppID (pDevContext->WdfUsbDevice, Request, (PUSHORT)ioBuffer);
break;
...
default:
ntStatus = STATUS_INVALID_DEVICE_REQUEST;
break;
}
WdfRequestCompleteWithInformation(Request, ntStatus, info);
return;
}
Now that looks a bit cleaner, even with the extra error checking! Too clean you ask? Well, you may be right, but it wouldn't be an adventure without a few false paths would it?
GetPowerLabAppID took a few more iterations before we decided that we were happy with it. The result looked pretty much like this:
NTSTATUS GetPowerLabAppID (WDFUSBDEVICE WdfUsbDevice, WDFREQUEST Request, short int *appID)
/*
Routine Description:
Get a PowerLab App ID
Return Value:
STATUS_SUCCESS if successful,
STATUS_UNSUCCESSFUL otherwise
*/
{
NTSTATUS status = STATUS_SUCCESS; WDF_MEMORY_DESCRIPTOR memDesc;
WDF_MEMORY_DESCRIPTOR_INIT_BUFFER(&memDesc, (PVOID)appID, sizeof
(*appID));
status = WdfUsbDeviceSendVendor
(
WdfUsbDevice,
Request,
NULL,
&memDesc,
URB_FUNCTION_VENDOR_DEVICE,
USBD_TRANSFER_DIRECTION_IN,
0,
12,
0,
0,
0
);
return status;
}
If you glance back at the original version of this function and compare it with this version you will notice that WdfUsbDeviceSendVendor has wrapped up the memory management for the URB, its initialisation and the call down the stack into a single function call. In WDM the call down the stack involved an additional 50 line function (BulkUsb_CallUSBD)!
Some time during this process Eliyas got sick of using an unfamiliar editor on a portable and suggested that we move operations to his office. This we duly did. (Hey, I've been in the building where NT was created!) A few compile errors later (mostly fixing missing pointer casts) we had a first cut of the device driver. We are now well into our second hour working on the project.
Does it work? Does it even get loaded? Well, actually I can't really remember how well that first version worked. Blame it on end of (very full on) conference fogginess if you like. We did realise rather late in the piece that we needed to change the GUID that is registered for the device. That required adding the GUID to public.h and then using the device GUID in the call to WdfDeviceCreateDeviceInterface in the call back function UsbSamp_EvtDeviceAdd found in driver.c. With the GUID fixed our application loaded, found the hardware, and complained that the hardware wasn't playing the game properly.
At that point we decided to stop. Eliyas had family matters to attend to and we couldn't really make a lot more progress without being able to find out from our application why it was upset with the new hardware driver. I reckoned that we had about a 95% complete driver at that point, and actually we had! But you know what proportion of the time that last 5% takes don't you?
Phase four, shaking out the bugs:
That was pretty much the end of the conference. Any more progress was going to happen during long compiles and other such "spare moments" back at work. Without going into details, debugging happened, WDF documentation was read, and the first penny dropped. Nothing was being returned from GetPowerLabAppID. Specifically, 0 bytes were reported as transferred by the DeviceIoControl call in our application that requests an App ID from the hardware. A little more reading of the WDF documentation, another false trail or two, and the fix to the problem is apparent.
Refering to the code for UsbSamp_EvtIoDeviceControl above you will see it calls WdfRequestCompleteWithInformation just before exiting to set the status for the request and to set the value for info. For the DeviceIoControl call the info parameter is used to set the number of bytes actually returned. The fix for the vendor request is simply to pass the address of info into GetPowerLabAppID so that it can be passed into WdfUsbDeviceSendVendor to receive the count of bytes transferred. The changed code for GetPowerLabAppID becomes:
NTSTATUS GetPowerLabAppID
(PDEVICE_OBJECT DeviceObject, short int
*appID, ULONG *info) ... { ... UsbBuildVendorRequest
(
urb,
URB_FUNCTION_VENDOR_DEVICE,
(USHORT)
sizeof(struct
_URB_CONTROL_VENDOR_OR_CLASS_REQUEST),
USBD_TRANSFER_DIRECTION_IN,
0, 12, 0,
0,
appID,
0,
2,
// Bytes to transfer info // Gets bytes
transferred
);
...
}
Note that the call to WdfUsbDeviceSendVendor gets info as the last parameter rather than 0. Now you see why the first version looked too clean.
Yay, now our application gets the App ID it needs. But does everything burst into life? Don't be silly! We're only into hour three.
The next discovery was that the two bulk end points were not being opened. A little further investigation and I realised that the sample code uses end point numbers to identify the end points to be opened. We had changed that for our driver because our application doesn't know which end points needed to be used because that was somewhat dependent on the particular type of hardware connected. The application simply opens the endpoints by name - BulkIn and BulkOut - using CreateFile to get the work done. The CreateFile call ends up calling UsbSamp_EvtDeviceCreate:
NTSTATUS UsbSamp_EvtDeviceCreate
(
IN WDFDEVICE Device,
IN WDFFILEOBJECT FileObject,
IN PIO_SECURITY_CONTEXT SecurityContext,
IN ULONG Options,
IN USHORT FileAttributes,
IN USHORT ShareAccess,
IN ULONG EaLength
)
/*++
Routine Description:
Dispatch routine for create.
Arguments:
Device - handle to the device
Return Value:
NT status value
--*/
{
NTSTATUS ntStatus = STATUS_UNSUCCESSFUL;
PUNICODE_STRING fileName;
PFILE_CONTEXT pFileContext;
PDEVICE_CONTEXT pDevContext;
PPIPE_CONTEXT pipeContext;
PAGED_CODE();
if (FileObject == NULL)
return STATUS_INVALID_PARAMETER; // FsContext is Null for the device!
pDevContext = GetDeviceContext(Device);
pFileContext = GetFileContext(FileObject);
fileName = WdfFileObjectGetFileName(FileObject);
if (0 == fileName->Length)
ntStatus = STATUS_SUCCESS; // Opening a device
else
{// Opening a pipe
WDFUSBPIPE pipe = NULL;
ntStatus = FindPipeFromName (pDevContext, fileName, &pipe);
if (NT_SUCCESS(ntStatus) && pipe != NULL)
{// found a match
pFileContext->Pipe = pipe;
pipeContext = GetPipeContext(pipe);
pipeContext->PipeOpen = TRUE;
WdfUsbPipeSetNoMaximumPacketSizeCheck(pipe);
ntStatus = STATUS_SUCCESS;
}
else
ntStatus = STATUS_INVALID_DEVICE_REQUEST;
}
if (NT_SUCCESS(ntStatus))
// increment OpenHandleCounts
InterlockedIncrement(&pDevContext->OpenHandleCount);
return ntStatus;
}
Which in turn calls FindPipeFromName to match a pipe to the file name. The fix was to rework FindPipeFromName to recognise BulkIn or BulkOut, rather than a pipe number, and to find a suitable pipe endpoint. The first step is simply to match the file name and set a flag appropriately:
NTSTATUS FindPipeFromName
(
IN PDEVICE_CONTEXT DeviceContext,
IN PUNICODE_STRING FileName,
WDFUSBPIPE *pipe
)
/*++
Routine Description:
This routine will pass the string pipe name and
fetch the pipe handle.
Arguments:
DeviceContext - pointer to Device Context
FileName - string pipe name
pipe - the returned pipe
Return Value:
The device extension maintains a pipe context for
the pipes on the USB device
--*/
{
NTSTATUS ntStatus = STATUS_INSUFFICIENT_RESOURCES; // init status to bad
*pipe = NULL; // Set it bad to start with
if (FileName->Length != 0)
{// Get pipe to open
enum {NoSearch, SearchIn, SearchOut} mode = NoSearch;
UNICODE_STRING inStr;
UNICODE_STRING outStr;
ULONG i;
ULONG numPipes;
RtlInitUnicodeString (&inStr, L"\\BulkIn");
RtlInitUnicodeString (&outStr, L"\\BulkOut");
if (RtlEqualUnicodeString (FileName, &inStr, 1))
mode = SearchIn;
else if (RtlEqualUnicodeString (FileName, &outStr, 1))
mode = SearchOut;
else
{
mode = NoSearch;
ntStatus = STATUS_NO_SUCH_FILE;
}
The next step scans through the pipes looking for a bulk endpoint in the right direction. This code uses WdfCollectionGetCount to find out how many endpoints there are in the collection. WdfUsbPipeGetType is used to check that the endpoint is a bulk endpoint. The two routines WdfUsbPipeIsOutEndpoint and WdfUsbPipeIsInEndpoint are used to check the direction of the endpoint. WdfCollectionGetItem retrieves the endpoint to return.
numPipes = WdfCollectionGetCount (DeviceContext->PipesCollection);
for (i = 0; mode != NoSearch && i < numPipes; i++)
{
if (WdfUsbPipeGetType (*pipe) != UsbdPipeTypeBulk)
continue; // Wrong type
else if (mode == SearchOut && ! WdfUsbPipeIsOutEndpoint (*pipe))
continue; // Wrong direction
else if (mode == SearchIn && ! WdfUsbPipeIsInEndpoint (*pipe))
continue; // Wrong direction
*pipe = WdfCollectionGetItem (DeviceContext->PipesCollection, i);
ntStatus = STATUS_SUCCESS;
break; // Found pipe, done
}
}
return ntStatus;
}
So, how are we going now? Well, actually, pretty good. We are now into hour four and our application is finally starting to talk to the hardware. It's not just that the bulk pipes are open, there is meaningful communication going on here! But not total communication. Now our application is reporting that longer messages are not getting through.
This one turns out to be pretty easy to debug and fix. The firmware in the hardware expects to be able to transfer blocks of data in messages up to 32,000 bytes in size (a legacy of Mac Pascal). The sample driver code uses 256 bytes as the maximum transfer size. The fix is simply to change the two constants MAX_TRANSFER_SIZE and TEST_BOARD_TRANSFER_BUFFER_SIZE in private.h to a more appropriate size. (32 * 1024) is a nice round number and should do the job just fine.
Does it do the job just fine? Yep, it does and that brings up the five hours. Our application works happily with the new driver and has been doing so for a couple of weeks now, at least on my system.
So really, how does WDF stack up?
Really? Pretty well actually. It did really and truly only take about five hours to get our simple "bulk transfer and a few vendor request" driver up and going starting from the bulk/iso sample provided with WDF. There is a bit of overhead - our driver ended up at about 150K where the original WDM driver is about 17K. Although I've not tested it thoroughly, my impression is that the WDF driver performs about the same as the WDM driver. The WDF driver has about 81K of source code compared to the WDM driver with about 169K. [Ed: The Microsoft development lead says that WDF will eventually be distributed as a DLL, which will greatly reduce the size of the module.]
Peter Jaquiery is a software engineer for ADInstruments Ltd. He lives in Dunedin, New Zealand. You can reach him at peter@adi.co.nz.