TM ® The NT Insider The only publication dedicated entirely to Windows ® system software development A publication by OSR Open Systems Resources, Inc. Not endorsed by or associated with Microsoft Corporation. January — February 2009 Volume 16 Issue 1 Common Topics Deferred Procedure Call Details D eferred Procedure Calls (DPCs) are a commonly used feature of Windows. Their uses are wide and varied, but they are most commonly used for what we typically refer to as "ISR completion" and are the underlying technology of timers in Windows. If they're so commonly used, then why are we bothering to write an entire article on them? Well, what we've found is that most people don't really understand the underlying implementation details of how DPCs work. And, as it turns out, a solid understanding is important in choosing the options available to you when creating DPCs and is also a life saver in some debug scenarios. Introduction This article is not meant to be a comprehensive review of why or how DPCs are used. It is assumed that the reader already knows what a DPC is or, even better, has used them in a driver. If you do not fall into this category, information at that level is readily available on MSDN. In addition, Threaded DPCs, which are a special type of DPC available on Windows Vista and later, will not be covered in any detail. As a basis of our discussion, let's briefly review some basic DPC concepts. A working definition of DPCs is that they are a method by which a driver can request a callback to an arbitrary thread context at IRQL DISPATCH_LEVEL. The DPC object itself is nothing more than a data structure with a LIST_ENTRY, a callback pointer, some (Continued on page 12) Happy 2009? W ell, that remains to seen for many folks, regardless of the industry. Profits are down, job losses are up, belts are being tightened...all is not right in the world. For those of you who are not aware, OSR provides job forums for both those companies looking to hire permanent, experienced driver developers, and of course, those looking to be hired by said companies. Check out Driver Jobs in the left-hand navigation menu at OSR Online (www.osronline.com). If you’re still gainfully employed, great. If you’re not, rest assured The NT Insider will still keep coming your way. Why Take An OSR Seminar? Whether you want to learn kernel debugging techniques, or you crave in-depth kernel device driver or file systems know- how, OSR has a specific solution tailored precisely to the type of training you need. Who are your instructors? Make no mistake about it, when you take an OSR seminar, you’ll be learning from the best. First of all, OSR isn’t a “training company.” We are consultants. Senior developers. Development leads on actual, cutting-edge projects developing system software. As your instructors, they bring you real-world answers to questions that textbooks – or other seminars – simply can ’t answer. We train more engineers in the design, development and debugging of Windows kernel-mode software than any other company. Period. Check out the latest seminar schedule on the back of this issue, or visit us on the web at www.osr.com. Interested in a customized, on-site presentation for your team? Drop us an email ([email protected]) and we’ll work with you to custom tailor one of our presentations for your needs. We can even schedule an extra day of consultancy to answer specific questions relative to your current project!
Transcript
Page 1
TM ®
The NT Insider The only publication dedicated entirely to Windows® system software development
A publication by OSR Open Systems Resources, Inc. Not endorsed by or associated with Microsoft Corporation.
January — February 2009 Volume 16 Issue 1
Common Topics Deferred Procedure Call Details
D eferred Procedure Calls (DPCs) are a commonly used feature of
Windows. Their uses are wide and varied, but they are most commonly used for what we typically refer to as "ISR completion" and are the underlying technology of timers in Windows. If they're so commonly used, then why are we bothering to write an entire article on them? Well, what we've found is that most people don't really understand the underlying implementation details of how DPCs work. And, as it turns out, a solid understanding is important in choosing the options available to you when creating DPCs and is also a life saver in some debug scenarios. Introduction This article is not meant to be a
comprehensive review of why or how DPCs are used. It is assumed that the reader already knows what a DPC is or, even better, has used them in a driver. If you do not fall into this category, information at that level is readily available on MSDN. In addition, Threaded DPCs, which are a special type of DPC available on Windows Vista and later, will not be covered in any detail. As a basis of our discussion, let's briefly review some basic DPC concepts. A working definition of DPCs is that they are a method by which a driver can request a callback to an arbitrary thread context at IRQL DISPATCH_LEVEL. The DPC object itself is nothing more than a data structure with a LIST_ENTRY, a callback pointer, some
(Continued on page 12)
Happy 2009?
W ell, that remains to seen for many folks, regardless of the industry.
Profits are down, job losses are up, belts are being tightened...all is not right in the world. For those of you who are not aware, OSR provides job forums for both those companies looking to hire permanent, experienced driver developers, and of course, those looking to be hired by said companies. Check out Driver Jobs in the left-hand navigation menu at OSR Online (www.osronline.com). If you’re still gainfully employed, great. If you’re not, rest assured The NT Insider will still keep coming your way.
Why Take An OSR Seminar?
Whether you want to learn kernel debugging techniques, or you crave in-depth kernel device driver or file systems know-how, OSR has a specific solution tailored precisely to the type of training you need. Who are your instructors? Make no mistake about it, when you take an OSR seminar, you’ll be learning from the best. First of all, OSR isn’t a “training company.” We are consultants. Senior developers. Development leads on actual, cutting-edge projects developing system software. As your instructors, they bring you real-world answers to questions that textbooks – or other seminars – simply can ’t answer. We train more engineers in the design, development and debugging of Windows kernel-mode software than any other company. Period. Check out the latest seminar schedule on the back of this issue, or visit us on the web at www.osr.com. Interested in a customized, on-site presentation for your team? Drop us an email ([email protected]) and we’ll work with you to custom tailor one of our presentations for your needs. We can even schedule an extra day of consultancy to answer specific questions relative to your current project!
Page 2
The NT Insider™
Published by OSR Open Systems Resources, Inc. 105 Route 101A, Suite 19 Amherst, New Hampshire USA 03031 (603) 595-6500
http://www.osr.com Consulting Partners W. Anthony Mason Peter G. Viscarola Executive Editor Daniel D. Root Contributing Editors Mark J. Cariddi Scott J. Noone OSR Associate Staff Consultant At Large Hector J. Rodriguez Send Stuff To Us: email: [email protected] fax: (603) 595-6503
Stuff Our Lawyers Make Us Say All trademarks mentioned in this publication are the property of their respective owners. “OSR”, “The NT Insider”, “OSR Online” and the OSR corporate logo are trademarks or registered trademarks of OSR Open Systems Resources, Inc. We really try very hard to be sure that the information we publish in The NT Insider is accurate. Sometimes we may screw up. We’ll appreciate it if you call this to our attention, if you do it gently. OSR expressly disclaims any warranty for the material presented herein. This material is presented “as is” without warranty of any kind, either expressed or implied, including, without limitation, the implied warranties of merchantability or fitness for a particular purpose. The entire risk arising from the use of this material remains with you. OSR’s entire liability and your exclusive remedy shall not exceed the price paid for this material. In no event shall OSR or its suppliers be liable for any damages whatsoever. It is the official policy of OSR Open Systems Resources, Inc. to safeguard and protect as its own, the confidential and proprietary information of its clients, partners, and others. OSR will not knowingly divulge trade secret or proprietary information of any party without prior written permission. All information contained in The NT Insider has been learned or deduced from public sources...often using a lot of sweat and sometimes even a good deal of ingenuity. Neither OSR nor The NT Insider is in any way endorsed by Microsoft Corporation. And we like it that way, thank you very much.
Inside This Issue:
Common Topics — Deferred Procedure Call Details 1
New WDK for Win7: TRY IT NOW! 3
Peter Pontificates: Computer Programming Has Ruined My Brain 4
Filtering File Systems — Ten Things You Should Know 6
Safety First — Using the Windows String Safe Functions 8
How to Fetch Checked Components for Vista/Server 2008 18
Who Was at the DDC? Well, We Were, But… 19
OSR Seminar Schedule
Mini-filter driver development—Boston, MA
WDF driver development—Portland, OR Kernel Debugging — Portland, OR
WDM driver development—Boston, MA File systems development—Boston, MA
See Back Cover for Dates
Page 3
New WDK for Win7: TRY IT NOW
I f you haven't done so, do make some time to install, examine, and actually play with the Beta version of the Win7 WDK. You can download it right now from Microsoft Connect (http://connect.microsoft.com). If you're not already
on the WDK Beta list, you can use invitation ID OSR-GMVB-T9HX to join (enter this invitation ID in the " Were you invited to join Connect?" box on the right of the home page). We know you're almost certainly busy doing work that's not related to Windows 7. So are we. But, here's the thing: Your chance to influence what's in the Win7 WDK is now, not months from now when you've started to work on Win7 stuff. You might be surprised at some of the things you find. Some good, some that may make you wonder. For example:
There's a new Office Automated Code Review tool that will automatically run PreFast on your driver for you.
The Windows 2000 Build Environment has ceased to be. It is no longer. It has gone to meet its maker.
There have been some changes to the SOURCES file syntax The legacy file system filter driver samples are gone
These are just a few things that we noticed. Are these things good? Well, here at OSR we're not particularly happy about the disappearance of the Win2K Build Environment, for example. But the point isn't what we think (we've already provided the WDK team with our thoughts)... The point is what do you think about these things.? And what will you notice that you like or dislike, that you find useful or problematic, about the Beta version of the Win7 WDK? If you don't install it, you won't know. So, go get the WDK, install it, and give it a run-through. And provide feedback to the WDK team about your experience. You complain that Microsoft doesn't listen to you? Well, we've heard the WDK team ask for feedback repeatedly -- both publically and privately -- on the Win7 WDK Beta. The best way to provide feedback is via the Win7 WDK feedback form on Microsoft Connect. This form is specifically (and only) for WDK bugs -- not Win7 O/S bugs. The WDK team has told us that this is the "express pipeline" directly to them for Win7 WDK Beta comments and bug reports. Alternatively, you can send feedback by email to [email protected].
Design & Code Reviews Have a great product design, but looking for extra security to validate internal operations before bringing it to your board of directors? Or perhaps you’re in the late stages of development of your driver and are looking to have an expert pour over the code to ensure stability and robustness before release to your client base. A small investment in time and money in either service can “save face” in front of those who will be contributing to your bottom line. OSR has worked with both startups and multi-national behemoths. Consider what a team of internals, device driver and file system experts can do for you. Contact OSR Sales — [email protected].
House 26 Jan 8:00PM FoxHD As an aside, please note that this was not a TiVo. TiVo is great. TiVo is the highest form of its art. There's nothing wrong with anything that TiVo does. There is probably not even anything wrong with anything made in Alviso California, in fact. I was using a different, low-rent, knock-off, ghetto, poseur TiVo wanna-be DVR. Obviously. What drove me nuts -- insane, in fact -- about the results display was that it showed the title of the show on each line... but failed to show the episode name. I already knew the name of the damn show. I was searching by title, right? Showing the title was useless. My wife was sitting with me:
Me: Ar rgh. Why are they showing me the Title field? Karen: Huh? It' s the title of the show. Me: I know that. But why are they showing it to me? On the search results page. Why didn't they show me the Episode name? I mean, heck... we've seen "House's Head" like three times... but we missed "Wilson's Heart." Karen: Cuz that's what they show. So you' ll know the title. So… Me: I searched by title. I know the title. I' ll tell you why they show it. It's because the dev was too lazy to code a separate page for each search result type, that's why. If she had coded a search results
(Continued on page 5)
J ust a confirmation ladies and gentlemen: this is Alaska Airlines flight 15 non-stop to Seattle. Please move down
the aisle to your seats as quickly as possible, so we can be underway without delay. Please remember that the primary storage location for your luggage is under the seat in front of you. As this is a very fully flight… I hadn't really been paying attention to the background blather, but my mind just sort of locked-in on that last statement. Did she say "a very full flight"? Huh? I turn to my seat mate, a skinny dude with glasses trying very hard to make sure everyone knows he is a medical doctor, because of his superior manner, the non-stop use of his cell phone to talk about his patients in a loud voice, and his constant waving of a copy of Journal of Clinical Oncology.
Me: Did she say " very full flight" ? Dr Boy: (looking over his Dolce and Gabbana glasses) Excuse me? Me: Very full. Did the flight attendant just say it was a "very full flight"? Because, well I don't really understand how that can be, given that "full" is a boolean. It means "filled to capacity." The flight can be FULL or NOT FULL. It can't be VERY FULL. Dr Boy: Uh, uhm... I'm not sure (waves Oncology Journal)... Stewardess! (orders a scotch, while ostentatiously making another phone call)
I can't stand stuff like this. Stuff like people acting highly superior on planes (jet travel is soooo glamorous these days, don't you think?) of course, but that's not what I specifically mean in the context of this article. I mean booleans being used as floating points. If full is a boolean, it can only be TRUE or FALSE (or true or false if you're with me on that C# thing from last year, but I digress). It can't have any other value. So how could the plane be very full? As I considered this, and noticed the stares I was still getting from Dr Boy out of the corner of his eye, I started to worry about my health just a bit. After I thought about it for a while, I came to a realization: Computer programming has ruined my brain. I suppose I knew this even before I had the above described revelation on the tarmac. It was just the other day that I was looking for scheduled episodes of a particular TV show with my DVR that I was driven into a fit of pique. I did "Search for shows" and then "By title." I entered "House" (the show title). The search results display page had three columns that read as follows: House 21 Jan 8:00PM USAHD House 21 Jan 9:00PM USAHD House 23 Jan 2:00AM HDNet House 23 Jan 9:00PM USAHD
Peter Pontificates: Computer Programming Has Ruined My Brain
Learn KMDF from Experts
Why wouldn’t you? If you’ve got a new device you need to support on Windows, you should be considering the advantages of writing a KMDF driver. Hands on experience with labs that utilize OSR’s USB FX2 device make learning easy -and you get to walk away with the hardware! Contact our seminar coordinator at [email protected].
Page 5
page for "search by title" that showed the episode name, and a separate search results page for "search by actor" or whatever that showed the show title, that would have been more work. Seriously, this is just a bunch of HTML and how much time would it have taken? Probably less time than it will now take me to select each of the results from the search results page and see what the episode title is. Karen: You just click on it.. Me: I know I just click on it. But why was it written this way? It's freakin' idiotic. If one of the devs who works with me did that, they'd be fired. It's either lazy or it's stupid… Karen: I'm going to get a cup of tea. Let me know when you're done talking to the dog about how the TiVo works and there's something for us to watch. Me: Ar r rgh... It' s not a TiVo, it' s a cheap-ass off-brand TiVo clone… Karen: (from the kitchen now, with the teapot) uh huh, yeah, TiVo, yup. Come here Toby!
She was familiar with this type of tirade. It was similar to the one I loosed when I didn't like the fact that her new car navigation system required you to go 7 menus deep in order to set what I figured should be a common parameter. That tirade contained a long soliloquy starting with "Who writes software like this?" and ending with"...would be fired" and "... either lazy or stupid." If I didn't know how to program, I don't think these things would bother me. I'd probably just accept "that's the way that the DVR works" and probably be amazed that the damn thing knew when every episode of House was scheduled for the next two weeks. I'd be blissful in my ignorance. But, computer programming having ruined my brain, I couldn't
(Continued from page 4)
Computer Programming... accept the bad design choices. The really awful implementation decisions. The just plain dumb code. I wish that I could report that the spongiform that is computer programming had only affected my brain in the use of consumer electronics products. Sadly, this is not the case. It's affected just about everything I do. Recently, my dog got sick. On one of my visits to the vet, I made him sit down and put the various treatment options into "if... then... else..." statements. He thought I was, shall we say, rather unusual . I thought he had a very disorganized way of thinking. How does somebody work in an office that doesn't even have a white board! It's ridiculous. I wanted to find another vet -- not really trusting anyone who doesn't have an immediate grasp of C syntax or Expo markers -- but Karen talked me out of it. Anyhow, at least now I realized my malady. And as I sat on the plane, getting ready to take off, I rolled the whole thing over in my mind. Maybe it wasn't so bad to be infected by computer programming. At least it kept me from speaking loudly on my cell phone and waving an oncology journal at the flight attendant. Hmmmm... I wonder who wrote the software that does the avionics displays?
Peter Pontificates is a regular opinion column by OSR consulting partner, Peter Viscarola. Peter doesn’t care if you agree or disagree, but you do have the opportunity to see your comments in a future issue. Send your own comments, rants, or distortions of fact to: [email protected]
OSR Software Development—Experience, Expertise and Guarantee In times like these, you can’t afford to hire a fly-by-night Windows driver developer. The money you think you’ll save in hiring cheap help by-the-hour, will disappear once you realize this trial and error method of development has turned your time and materials project into a lengthy “mopping up” exercise...long after your contract programmer is gone. Consider the advantages of working with OSR. If we can be of value-add to your project, we’ll tell you—if we can’t we’ll tell you that too. You deserve (and should demand) definitive expertise. You shouldn't pay for inexperienced devs to attempt to develop your solution. What you need is fixed-price solutions with guaranteed results. Contact the OSR Sales team at [email protected] to discuss your next project.
Page 6
another driver, it will become the problem of that other driver writer. Remember, when someone types “!analyze –v”, it will blame the first non-Microsoft driver on the stack. Hence, it’s better to not be on the stack, at least from a support overhead burden. Validate Parameters Sure, we all know that it isn’t necessary to validate buffers that are passed to us by other kernel mode components, but only Microsoft has the luxury of this type of magic thinking. For the rest of us, it really is vital to keep in mind that the system is not static and we are constantly forced to “figure out” why things are not working. Keep in mind when validating any information:
Buffers are not static. You do not own the only reference to that buffer and they can (and do) change. If you want to rely upon the contents of buffers, you must capture that information (if it changes inside a private captured buffer, it’s called “memory corruption” and not a buffer handling error!)
With Windows Vista, MDLs that arrive in your filter can include “dummy pages”. These pages can create potential confusion for any filter that tries to modify the data (e.g., an encryption filter) because that buffer’s content is not static. This merely reinforces our observation before – if you want static content, capture it and operate on it separately from the original.
If you decide to switch around the MDLs in an I/O request, don’t forget to also update the UserBuffer field (the file systems rely upon this being “correct” – which means it must be the same virtual address that is in the MDL passed). Beware the Failure Cases Over the years, we have consistently observed that developers are, as a group, remarkably optimistic. This shows in their coding, which tends to assume that everything is going to work properly. As it turns out, however, this optimism is unwarranted – and the source of frequent bugs. Further, as your driver is more widely deployed, you will observe a greater number of these failure scenarios. Indeed, in our experience, even mature code bases will experience “we’ve never seen this failure case before” scenarios. Naturally, the fact that it is almost impossible to test all of these failure scenarios further complicates writing code to handle those cases. Nevertheless, it is important for file system filter driver writers (both legacy and mini-filter writers) to think about error cases and handle them.
(Continued on page 7)
R egardless of whether you are still supporting legacy file system filter drivers, or maintaining or building new mini-
filter drivers, there are some common techniques (ok, there are only nine…) you can use to improve the robustness of your filter. Many of these are pragmatic – they are not things you will read in the documentation, they come from experience working with filters for many, many years. Never Trust Buffers Filter drivers are forced to endure a remarkably hostile environment, both from their neighbors (other filter drivers) as well as those that are inherently untrusted (any user mode code). A single mistake in a file system filter driver can create a potential “time bomb” waiting to happen. User Buffers are the most commonly identified “risk factor” when it comes to buffer management. Despite that, we still routinely see filters that fail to take the two necessary steps for protecting against user buffers. First, you must ensure that a user buffer really is within the user portion of the address space. This is performed by probing the address, normally using ProbeForRead or ProbeForWrite. Probing a user buffer only ensures it is in the user’s address space, however. If it is invalid at that point in time the probe will work, but this call does nothing to ensure the address remains valid. Thus, the second key part of buffer handling is to ensure any access to it is protected with a structured exception handler. Microsoft provides their own custom handler and it is invoked using special extensions implemented by the C compiler used in the WDK environment. This is done using __try and __except. Both of these steps are essential to ensuring correct user buffer handling. In addition, file system filter drivers must deal with the pragmatic issue that they may be handed invalid kernel data structures. The harsh reality is that it is not uncommon for other kernel components to pass along invalid structures. Mini-filters are somewhat protected against this, since invalid structures passed into filter manager will cause filter manager to crash, not the mini-filter. Despite this, if you are the only mini-filter on the system it won’t crash without your filter and it will crash with your filter (and we all know that this means it must be “your fault”). Thus, what we generally suggest– as much as possible – is to carefully validate your own data structures. While not guaranteed, we’ve used this technique to detect erroneous situations numerous times. In our experience it is better to use “deprecated” functions like MmIsNonPagedSystem AddressValid (which is of dubious value) to add even a modest amount of robustness to our code. This is pragmatic – if the system crashes in our driver, we will be blamed for the crash. If we can instead return an error and it crashes in
Filtering File Systems Ten Things You Should Know
Page 7
In general, we’ve noticed a few classes of such failures:
Allocation failures. These are quite common, but something often overlooked is that handling allocation failures at the point you need the allocated object is frequently “too late” to handle the error gracefully. Generally, we suggest moving the allocation to an earlier point in the code – keeping in mind that freeing an object won’t fail. So, in a mini-filter (for example) we allocate our data structure for our context tracking information in the pre-create path, even though we won’t use it until the post-create path. If we don’t need it, we free it. In the pre-create path, however, we can gracefully return an error. In post-create, we are forced to try and cancel the file open (which is far more complicated and subject to errors).
Connection failures. While we’ll mention redirectors “later”, we have noticed that timeouts and disconnections are common enough failures that just aren’t handled. If you use a service, you have to consider what your driver will do when it fails, not to mention how it fails. Nothing is worse for a user than the frustration of having an operation (in your filter) hang forever waiting on a service that has entered an infinite loop.
Unwinding failures. These are perhaps the most insidious of the error handling cases. You’ve done your job and checked for a specific error case. But you have already done operations that need to be undone. Thus, your error handling code must then attempt to undo something you’ve already done. It could be as simple as changing the read-only bit on a file, or as complicated as changing its ACL. Some later step fails and you attempt to set it back – but that attempt to unwind itself fails.
Of course, our point is not to provide some complete list, but rather to point out that when writing code, and particularly when code reviewing it, try looking through it and ignoring the “everything works right” cases and look at the “everything comes unraveled” cases. After all, the “everything works right” case is by far the most likely case to be tested and fixed quickly. Avoid Changing File Sizes The most difficult filters to build and maintain are those that change the size of the underlying file and then try to mask this behavior from the applications. There are very good reasons for masking these changes from applications – after all, applications rely upon the size information in the directory entry being correct. For example the “verify” option for xcopy (“/v”) actually looks at the size in the directory entry, not in the file itself. Exacerbating this issue is that directories can be surprisingly
(Continued from page 6)
Filtering...
“File and Folder” Encryption Several commercially shipping products are a testament to the success of OSR’s most recent development toolkit, the Data Modification Kit. With the hassle of developing transparent file encryption solutions on the rise, why not work with a codebase, a support team and a reputation to implement your encryption or other data-modifying file system solution? Check out the DMK at www.osr.com, and/or contact OSR to discuss the DMK and your needs to determine if you can take advantage of this unique solution. Phone: +1 603.595.6500 Email: [email protected]
large (we’ve received bug reports against directories with over 700,000 entries in a single directory – so large that the owner of that large directory could not enumerate it with Explorer and instead only enumerated it from the command line!) If you have to actually open the file in order to retrieve the file size, it will make directory enumeration dramatically slower. In our experience, unless you implement caching, the cost of “correcting directory entry sizes” has an amazing impact – it effectively makes doing almost anything with high access directories impossible (e.g., “\Windows”). Underlying this is the fact that application programs are sloppy – they use information that is demonstrably not updated (NTFS, for example, only updates the size in the link used, so this size from the directory entry trick can be spoofed by using a link to change the file size). We have seen people try to use “naming tricks” to resolve this issue. Fair warning: whenever you think you have a clever solution in a filter driver, ask yourself “and how will this work when someone else implements a filter driver that plays the same/similar trick.” If the answer is “it does not work” then you don’t have a viable solution. Name prefix/suffix tricks fall into this category. The best thing to do is to never change size. If you must change the size of the file, the next best trick is to do this only on file systems that support both sparseness and alternate data streams – then you can keep a dummy default stream (“::$DATA” on NTFS) that is the correct size, with the modified data in an alternate data stream. Unfortunately, that often is not a viable solution because it is restricted to only some file systems. In that case, be prepared for building a very complicated filter driver. FSD/MM/CC Interactions Are Complicated We often observe that interactions in this business between the components are horribly complicated (See Figure 1). File system filter drivers sit in the midst of this complex
(Continued on page 15)
Page 8
All the string safe functions return an NTSTATUS, with the only possible success code being STATUS_SUCCESS (there are, of course, several possible error codes).
Each string handling function has both a Cch and Cb variant available for use.
Most functions have an extended "Ex" version which provides extended functionality.
String Safe Character Count Functions Table 1 lists the character count string safe functions provided in "strsafe.h" and the c-runtime functions (CRT) functions they are intended to replace.
String Safe Byte Count Functions Table 2 lists the byte count string safe functions provided in "strsafe.h" and the c-runtime functions (CRT) functions they are intended to replace. Unicode Strings At this point I probably have given you the impression that the string safe functions are only intended to replace c-runtime functions. That is not true. The string safe functions include many functions, of both Cch and Cb variants, to help you with Unicode strings also. The list below contains the functions provided:
U nless you have had your head in a hole for the last couple of years, you have undoubtedly heard about Microsoft
fixing security holes in Windows kernel mode code. Many of the problems they encountered had to do with string handling functions, for example strcat, strcpy, and their related c-runtime library functions (CRT) . The weakness in these functions is that no buffer lengths are specified, so it is very easy for a user to copy a long string into a short buffer and thus corrupt memory or worse, cause some unhandled exception. Buffer overruns have even been used as security attacks. To fix this class of problem Microsoft created a safer set of functions called the "String Safe" Functions. These relatively recent functions (XP SP1), to no one's surprise, add parameters to describe the length of the string buffers, to reduce the chance that buffers are overflowed When Microsoft created the string safe functions they created two sets of functions. One set uses byte counts to describe buffer lengths, while the other set of functions uses character counts to describe the buffer lengths. Byte Count related functions always contain "Cb" (for count in bytes) in the name, for example RtlStringCbCat. The character count related functions always contain "Cch" (for count in character s) in the name, for example RtlStringCchCat. The type of function to use is entirely up to you. Now, it is nice to know that Windows provides these functions, and some of the advantages are probably already evident to you. However, let's review some of the major benefits to using these newer functions: Since you are providing the correct size of the destination buffer this ensures that the function will not write past the end of the input buffer. All string safe functions guarantee that the output buffers are null terminated, even if the operation truncates the intended result.
Safety First Using the Windows String Safe Functions
Example Let us assume for the moment that you need to write a function to generate the full name of a file from an input full directory path and the handle to a file. Furthermore, we will also assume that we will never see a long path name, because everyone must be like us and hate to have long file names. Thus the function that you may write to handle this could be as shown in Figure 1. If you analyze this code, I am sure that you can see a great many faults that could occur in this code. Most of them center around overrunning the dirname buffer since we forgot to look at the string lengths of the buffers we were concatenating from. A better solution to this function could be either Figure 2 or Figure 3 (P. 10), one illustrating the use of the
(Continued from page 8)
String Safe... Cch version of the string safe functions (Figure 2)and the other using the Cb version (Figure 3). As you notice with the Cch and Cb versions of the function, the main difference is that the size of the destination buffer is specified in each call. This provides the string safe functions with the ability to ensure that the destination buffer is not overrun. In addition, the functions return a status so that we can be sure to terminate function processing if an error occurs. Building Your Code In order to build your code correctly, you may want to know about the compile time variables that you can define to allow you to optimize how you use the string safe functions. STRSAFE_LIB If you want to use the string safe functions in library form then define STRSAFE_LIB before including "strsafe.h" in your code. In addition, you will need to insure that you link your code against "$(SDK_LIB_PATH)\strsafe.lib". Why do this? Well, instead of including the string safe routines that you use in your binary, this allows your driver to dynamically bind to the string safe functions when it loads,
wchar* GenerateFullPath(wchar* fulldirPath,HANDLE File) { static wchar dirname[100]; // assume you think all paths are small wchar* fileName = NULL; ASSERT(fulldirPath); ASSERT(File); fileName = GetFileName(File); // Get the name of the input file strcpy(dirname,fulldirPath); // copy the input dir name strcat(dirname,L"\\"); // append a \ to the name strcat(dirname,fileName); // concatenate the filename return dirname; // return the full path to the caller. }
Table 2
Figure 1
Page 10
hopefully allowing you to be running against the latest most improved implementations of these functions. This can be both good and bad.
Example: #define STRSAFE_LIB #include <strsafe.h>
If you don't define STRSAFE_LIB in your code, the current
(Continued from page 9)
String Safe... revision of the string safe functions will be compiled as part of your driver code (i.e., the implementations of the routines will be compiled as part of "strsafe.h"), and you do not need to link against strsafe.lib STRSAFE_NO_DEPRECATE Normally when you include "strsafe.h" all older functions (c-runtime functions) that are replaced by "strsafe.h" defined functions are deprecated, i.e. you are given warnings that that the functions are not to be used and you are urged to use their string safe equivalents. If for some unknown reason you have to use one of these functions, you should define STRSAFE_NO_DEPRECATE. This definition will suppress the warnings and allow you to compile your driver.
(Continued on page 11)
wchar* GenerateFullPath(wchar* fulldirPath,HANDLE File) { static wchar dirname[MAX_PATH]; wchar* fileName = NULL; NTSTATUS status; ASSERT(fulldirPath); ASSERT(File); ASSERT(MAX_PATH <= STRSAFE_MAX_CCH); fileName = GetFileName(File); // Get the name of the input file status = RtlStringCchCopyW(dirname,MAX_PATH,fulldirPath); if(!NT_SUCCESS(status) { return NULL; } status = RtlStringCchCatW(dirname,MAX_PATH,L"\\"); if(!NT_SUCCESS(status) { return NULL; } status = RtlStringCchCatW(dirname,MAX_PATH,fileName); if(!NT_SUCCESS(status) { return NULL; } return dirname; // return the full path to the caller. }
Figure 2 — Cch String Safe Usage
wchar* GenerateFullPath(wchar* fulldirPath,HANDLE File) { static wchar dirname[MAX_PATH]; wchar* fileName = NULL; NTSTATUS status; ASSERT(fulldirPath); ASSERT(File); ASSERT(MAX_PATH <= STRSAFE_MAX_CCH); fileName = GetFileName(File); // Get the name of the input file status = RtlStringCbCopyW(dirname,MAX_PATH*sizeof(WCHAR),fulldirPath); if(!NT_SUCCESS(status) { return NULL; } status = RtlStringCbCatW(dirname,MAX_PATH*sizeof(WCHAR),L"\\"); if(!NT_SUCCESS(status) { return NULL; } status = RtlStringCbCatW(dirname,MAX_PATH*sizeof(WCHAR),fileName); if(!NT_SUCCESS(status) { return NULL; } return dirname; // return the full path to the caller. }
STRSAFE_NO_CB_FUNCTIONS If for some reason you want to limit your code to only use character count related string safe functions, then you should define STRSAFE_NO_CB_FUNCTIONS. This definition will then limit your code to the use of the RtlStringCchxxx functions.
STRSAFE_NO_CCH_FUNCTIONS Similarly, the STRSAFE_NO_CCH_FUNCTIONS define allows you to limit your code to only use byte count related string functions. Using this definition will then limit your code to use of the RtlStringCbxxx functions.
Summary While it may seem like a hassle to learn a new set of functions
to replace the old c-runtime functions that we have grown to
love, it really is in your best interest to change. The string
safe functions provided in "strsafe.h" provide you with a
robust set of functions which will make your driver safer from
buffer overruns and silly string mistakes that we all tend to
make.
Corporate, Onsite Training Save Money, Travel Hassles, Gain Customized Expert Instruction
If you’re not aware of the value and benefit of bringing OSR to your location for seminars, we’re obviously not doing our job (blame Dan!). With tightened training budgets and/or travel restrictions combined with increased travel costs, OSR’s private, on-site training solutions can be your way to convince management that they can “have their cake and eat it too”. We can: Prepare and present a one-off, private, on-site seminar for your team to address a specific area of
deficiency or to prepare them for an upcoming project Design and deliver a series of offerings with a roadmap catered to a new group of recent hires or within
an existing group. Work with your internal training organization/HR department to offer monthly or quarterly seminars to
your division or engineering departments company-wide. With any option, OSR’s seminar staff, in conjunction with the instructor, will work with you in advance to help craft a training solution that will maximize technical value and ensure you’re getting the most from your training budget. To take advantage of our expertise in Windows internals, and in instructional design, contact an OSR seminar consultant at +1.603.595.6500 or by email at [email protected].
Page 12
context for the callback, and a bit of control data: typedef struct _KDPC { UCHAR Type; UCHAR Importance; USHORT Number; LIST_ENTRY DpcListEntry; PKDEFERRED_ROUTINE DeferredRoutine; PVOID DeferredContext; PVOID SystemArgument1; PVOID SystemArgument2; __volatile PVOID DpcData; } KDPC, *PKDPC, *PRKDPC;
You initialize a DPC Object with KeInitializeDpc and queue the DPC Object with KeInsertQueueDpc. Drivers that use DPCs to perform more extensive work than is appropriate for an Interrupt Service Routine typically use the DPC Object that's embedded in the Device Object, and cause this DPC Object to be queued by calling the function IoRequestDpc (which internally calls KeInsertQueueDpc). Once queued, at some point in the future your DPC routine is invoked from an arbitrary thread context at IRQL DISPATCH_LEVEL. With that basic info in hand, we can now cover the gory details of both the queuing and delivery mechanisms that are used for DPCs. That will lead us to discussing what options we have for controlling the behavior of DPCs and what impact those options have. DPC Queuing As mentioned previously, DPCs are queued (directly or indirectly) via the KeInsertQueueDpc DDI: NTKERNELAPI BOOLEAN KeInsertQueueDpc ( __inout PRKDPC Dpc, __in_opt PVOID SystemArgument1, __in_opt PVOID SystemArgument2 );
DPCs are actually queued to a particular processor, which is accomplished by linking the DPC Object into the DPC List that’s located in the Processor Control Block (PRCB) of the target processor. Determining the processor to which the DPC Object is queued is fairly easy for the O/S. By default, the DPC is queued to the processor from which KeInsertQueueDpc is called (the " cur rent processor" ). However, a driver writer can indicate that a given processor be used for a particular DPC object, using the function KeSetTargetProcessorDpc. Viewing the DPC List on a particular processor is easy using WinDBG. While the DPC List is actually contained within the PRCB, the PRCB is an extension of the Processor Control Region (PCR). By viewing the PCR with the !pcr command we can see any DPCs currently on the queue for that processor: 0: kd> !pcr 0 KPCR for Processor 0 at ffdff000:
(Continued from page 1)
DPCs... Major 1 Minor 1 NtTib.ExceptionList: 8054f624 NtTib.StackBase: 805504f0 NtTib.StackLimit: 8054d700 NtTib.SubSystemTib: 00000000 NtTib.Version: 00000000 NtTib.UserPointer: 00000000 NtTib.SelfTib: 00000000
One aspect of DPCs to note is that once a DPC Object has been queued to a processor, subsequent attempts to queue the same DPC Object are ignored until the DPC Object has been dequeued (by Windows for execution of its callback). This is what the BOOLEAN return value of KeInsertQueueDpc indicates: TRUE means that Windows queued the DPC to the target processor and FALSE means that the DPC Object is already queued to some processor. This makes sense from a programming perspective, as the DPC data structure only has a single LIST_ENTRY field and thus can only appear on a single queue at a time. What About Priority? Where the DPC is placed on the target processor’s DPC List is an interesting question. Whether a DPC Object is inserted at the beggining or end of the target processor’s DPC List is one aspect of the priority feature of DPCs. You can set the importance of a given DPC Object by using the function KeSetImportanceDpc. This DDI lets you indicate that the DPC object is low, medium, or high importance. Also, in Vista and later you can set the importance to “medium high.” Low, medium, and medium high importance DPCs are placed at the end of the DPC queue, while high importance DPCs are placed at the front of the queue. You may ask yourself at this point, "then what's the difference between low, medium, and medium high?" We'll answer that question shortly. The DISPATCH_LEVEL Software Interrupt Once the DPC has been queued to the target processor, a DISPATCH_LEVEL software interrupt is typically generated on the processor. The choice of whether or not to request the DISPATCH_LEVEL software interrupt when the DPC Object is queued is largely based on four factors: the importance of the DPC, the target processor of the DPC, the depth of the DPC List on the target processor, and "drain rate" of the DPC List on the target processor.
(Continued on page 13)
Page 13
If the target processor for the DPC Object is the current processor, the DISPATCH_LEVEL software interrupt is requested if the DPC Object is of any importance other than low. For low importance DPCs, the software interrupt is only requested if the O/S believes that the processor is not servicing DPCs fast enough, either because the DPC queue has become large or is not draining at a sufficiently fast rate. If either of those are true, the interrupt is requested even if the DPC is low importance. If the target processor for the DPC Object is not the current processor, the decision process is different. Because requesting an interrupt on the other processor will involve a costly Inter Processor Interrupt (IPI), the situations under which it is requested are restricted. Prior to Vista, the IPI request would only be made if the DPC was high importance or if the DPC queue on the target processor had become too deep. Vista added the medium high importance DPCs to the check and went one step further to cut down the number of IPIs by requiring the target processor to be idle for the DISPATCH_LEVEL software interrupt to be requested (See Table 1 for a high-level breakdown). DPC Delivery Once the DPC has been queued to the processor, at some point it must be dequeued and the callback executed. Remember that there were two scenarios that occurred after the DPC was queued to the processor, either the DISPATCH_LEVEL software interrupt was requested or it was not. Delivery from the Software Interrupt Service Routine To keep things relatively simple, we’ll restrict our discussion here to the case of queuing the DPC Object to the current processor. Let's start with the case in which the IRQL DISPATCH_LEVEL software interrupt was requested. At the time KeInsertQueueDpc was called, there are two situations the system could be in: The first would be running at an IRQL < DISPATCH_LEVEL, in which case the
DPCs... DISPATCH_LEVEL interrupt would be delivered immediately. The second case would be if the current processor is at IRQL >= DISPATCH_LEVEL, in which case the interrupt would remain pending until the IRQL was about to return to an IRQL < DISPATCH_LEVEL. In either case, once the service routine for the DISPATCH_LEVEL interrupt begins executing, it checks to see if any DPCs are queued to the current processor. If the DPC queue is not empty, Windows will loop and entirely drain the DPC List before returning from the service routine. Before draining the DPC List, Windows wants to ensure that it has a fresh execution stack for the DPC routines to run on. This will presumably cut down the incidents of stack overflows in the case where the current stack does not have much space remaining. Thus, every PRCB also contains a pointer to a previously allocated DPC stack that Windows switches to before calling any DPCs: 0: kd> dt nt!_KPRCB DpcStack +0x868 DpcStack : Ptr32 Void
We can see evidence of the switch in the debugger if we set a breakpoint in a DPC routine. Here we chose a DPC from the ATAPI driver: 0: kd> bp atapi!IdePortCompletionDpc 0: kd> g Breakpoint 1 hit atapi!IdePortCompletionDpc: f9806990 8bff mov edi,edi 0: kd> k ChildEBP RetAddr f9dc7fcc 80544e5f atapi!IdePortCompletionDpc f9dc7ff4 805449cb nt!KiRetireDpcList+0x61 f9dc7ff8 f9a2b9e0 nt!KiDispatchInterrupt+0x2b WARNING: Frame IP not in any known module. Following frames may be wrong. 805449cb 00000000 0xf9a2b9e0
Notice the strange call stack - it seems to disappear after the call to KiDispatchInterrupt. The problem is that WinDBG has ceased to be able to unwind the stack due to the stack switch, and the call stack that we see here is the call stack for the DPC stack. If we try to match the EBP addresses shown with the stack limits of the current stack we will see the discrepancy:
MediumHigh Other than current END YES (Vista: if target idle)
High Current HEAD YES
High Other than current HEAD YES (Vista: if target idle)
Table 1
Page 14
stack. Because the idle loop uses so little thread stack itself, there is not much use in going through the effort of swapping stacks in this case.
Conclusion Hopefully this cleared up a few misconceptions about DPCs
and how they are handled by the system. 0: kd> !thread THREAD 81964770 Cid 028c.02b8 Teb: 7ffd8000 Win32Thread: e1873008 RUNNING on processor 0 IRP List: 8195b870: (0006,0190) Flags: 00000970 Mdl: 00000000 819128b0: (0006,0190) Flags: 00000970 Mdl: 00000000 Not impersonating DeviceMap e1001980 Owning Process 818d5978 Image: csrss.exe Attached Process N/A Image: N/A Wait Start TickCount 6779 Ticks: 0 Context Switch Count 4104 LargeStack UserTime 00:00:00.000 KernelTime 00:00:00.265 Start Address 0x75b67cd7 Stack Init f9a2c000 Current f9a2ba58 Base f9a2c000 Limit f9a29000 Call 0
Note that the EBP addresses do not fall within the base and limit of the current thread's stack. Using the techniques outlined in last issue's Debugging Techniques: Take One...Give One article (September-October 2008), you can actually get the original call stack of the interrupted thread back. This can be useful as it sometimes can provide key information as to what kicked off a chain of events on that processor. Once the O/S has switched stacks, it begins dequeuing DPC objects and then executing the call backs. Thus, while your DPC routine is executing, it is no longer on any DPC queue and may be queued again. Delivery from the Idle Thread But what about those low importance DPCs or targeted DPCs that didn't request the DISPATCH_LEVEL software interrupt? Who processes those? Well, there are actually two ways in which they'll be processed. Either another DPC will come along that will request the DISPATCH_LEVEL interrupt and the DPC will be picked up on the subsequent drain, or the idle loop will come along and notice that the DPC queue is not empty. Part of the idle loop's work is to check the DPC queue and determine if it is empty or not. If it finds that the queue is not empty, it begins draining the queue by dequeuing entries and calling the callbacks. We can see this in a different call stack but using the same DPC routine as the previous example: Breakpoint 1 hit atapi!IdePortCompletionDpc: f9806990 8bff mov edi,edi 0: kd> k ChildEBP RetAddr 80550428 80544e5f atapi!IdePortCompletionDpc 80550450 80544d44 nt!KiRetireDpcList+0x61 80550454 00000000 nt!KiIdleLoop+0x28
The difference here is that in this case the stack is not switched, thus the DPCs actually executed on the idle thread's
(Continued from page 13)
DPCs...
Kernel Debugging for Windows
You’ve seen our articles where we delve into analyses of various crash dumps or system hangs to determine root cause. Want to learn the tools and techniques yourself? Consider attendance at OSR’s Kernel Debugging for Windows seminar. This hands-on training not only provides fundamentals you need to debug Windows kernel-mode software, but spends considerable time showing you the valued techniques that are used and espoused internally at OSR. The next offering of this limited-attendance seminar is to be held in Portland, Oregon from 6-10 April! Act soon, to save your seat! For more information, visit us at www.osr.com or call us at +1.603.595.6500, or email us at [email protected].
Page 15
is that filters are often written to target NTFS first and only once working against NTFS are they moved to other file system. Sadly, this approach fails horribly when it comes to the network file systems. Filters that “work fine” on NTFS frequently exhibit serious problems when moved to the Windows redirector. Indeed, over the years, we’ve observed that it is often better to start with the network case and then move to NTFS – its generally a far less painful move than vice versa, particularly because project management often assumes that “once it works on NTFS we’re almost done.” Some of the issues we have seen when it comes to filtering network redirectors over the years include:
Naming. Nothing in the local file system arena prepares you for the hideous complexity of naming in the network. Effectively, file names may be aliased due to hard links (which at least exist for physical file systems) but also due to overlapping shares not to mention special names sent by DFS to the redirector.
Context. On a local file system, it is rather easy to track unique instances of a given stream (“stream” is
(Continued on page 16)
environment and must be written to handle the ramifications of those interactions. The simplest way to handle these interactions is to simply ignore them – in other words, don’t get involved in the data flow through the system. That is definitely the simplest solution. Many filters must become involved in this in at least some fashion, however. Filters that are involved in I/O operations need to understand these interactions. Still, it is beyond the scope of this article to describe these complexities. The point, however, is that in order to build a robust file system filter driver, you really do need to understand the file system. Redirectors Do Not Behave Like Local File Systems Some filters can restrict themselves simply to a single file system (e.g,. NTFS) or perhaps to a class of file systems (physical file systems) but many filters must handle the network as well. Over the years, the pattern we have observed
(Continued from page 7)
Filtering...
Figure 1—A Complex Environment to Exist In
Page 16
that is still in use by their customer base) and the desires of Microsoft (which is to move the platform forward so their customers will buy an upgrade). The harsh reality is that we have a huge number of versions of Windows to support. Microsoft wants everyone to move to mini-filters (and has clearly stated they will enforce this at some point, which seems as if it will just force the legacy support to go “underground”). Each new platform requires additional resources to test and fix bugs – there are always changes in behavior, often for very specialized usage cases (transactions come to mind) that can directly impact on your own development (transactions matter to filters because filters see the “forward” operations but see no I/O if the transaction is aborted – even though the state of the underlying system changes). Testing is Difficult We can give you a basic list of the “tools of the trade” but the bottom line is that no matter how much testing you perform, you will never be able to cover every case. In all fairness, debugging a problem that manifests itself via an application – perhaps a common one like “Explorer” (common but still amazingly frustrating) or perhaps something far less common, like “Avid Newscutter Nitris DX” – is always going to be rather challenging to debug because it often requires that you determine why the application fails – you know it is related to something you are doing, but you don’t know what it is. Microsoft does not provide us with a conformance test. The closest thing we have (the ifskit tests) have mysterious failure modes and are often as inscrutable to understand as any failing application. Indeed, we generally do not investigate “SEV2” issues because we’ve found they generally are of the “oh, we don’t work the same way that NTFS does in this case” category. Another tool we often point out to people is the “FileTest” utility. It allows you to exercise uncommon operations like “open by file ID” that can be difficult to simulate with an off-the-shelf program, but need to work properly within your filter driver. I/O Stress can be useful for finding issues in your I/O handling paths. Driver Verifier is a must – if you aren’t using it during development and part of your testing, you aren’t doing a good job. However, you must also test without Driver Verifier enabled. We’ve even seen cases where Driver Verifier has tests that only trigger on single processor computers! At OSR, we have our own testing framework – it’s a scripting language that is built against the native API and allows us to trivially write scripts that can test the native windows API. If you restrict yourself to the Win32 API, your tests will definitely be deficient, since it is a subset of the functionality in the native API. Finally, don’t forget to test via the network – but this time, with the file SERVER running over your filter. Again, you will see different behavior. For example, the SL_INDEX_SPECIFIED option that is used for directory enumeration is difficult to trigger –but does show up with
(Continued on page 17)
the NTFS terminology. On FAT, there is only a single stream within a file, but in NTFS there can be many streams within a single file). This is typically done using the FsContext parameter of the file object. Legacy filters may do this directly (via a lookup table keyed from this value) or indirectly (via a “filter context” that is attached to a list within the FsContext structure). Mini-filters rely upon Filter Manager; Filter Manager in turn relies upon filter contexts. The Windows redirector, however, does not necessarily use the same context even for two file objects pointing to the same file. This works correctly for them, but it violates this basic assumption in many filter drivers. Further complicating this, the network file systems may coalesce file objects together – changing their FsContext values when it detects two instances of the same file. Filters, once again, must be able to handle this particular case.
Relative open “issue”. IoCreateFileSpecifyDevice ObjectHint (in at least some still in-use versions) has an issue when performing a relative open over the Windows redirector. In our experience, it is necessary to always construct “fully qualified path names” and send these into this API – or any API (e.g., FltCreateFile and its progeny) that relies upon it.
Let’s not lose sight of the fact that there are also two other network redirectors that ship with at least one version of Windows (“in the box”) – WebDAV (the http protocol for “distributed authoring and versioning”) and NFS (which is shipped as part of the Vista and Server 2008 distributions, although versions were previously available as “Services for Unix”). Maybe You Need to Support Systems Older Than Windows 7 As difficult as it is for those in Redmond to believe, there are still a few stragglers left using Windows versions prior to Windows 7 (please don’t miss the sarcasm, though the point is still valid). There are various reasons for this, but the biggest driver really is the cost associated with upgrading the systems. Particularly when those systems are performing their primary function properly, there is no compelling reason. People that do not need transactions, bit locker, resizable thumbnails, fancy graphical features and the like and instead are focused on executing custom software solutions, running embedded devices, etc. are very resistant to upgrading their systems. In spite of this resistance, they do expect their anti-virus products, quota managers, content control managers, backup components, encryption products, etc. to work on their version of Windows. Microsoft does not receive revenue from old versions of Windows, but those of us building filter drivers receive revenue from building products for those older versions. This creates a constant tension between the general development community (that needs support for any version
(Continued from page 15)
Filtering...
Page 17
some versions of the Windows file server software. Naturally, this is exercised when you access files over the network. Oh, and lest you missed this fact, you now (beginning with Server 2003) need to test between machines – loopback mode has been optimized out so that it bypasses the normal client/server model (allegedly there is a registry parameter to disable this functionality, but usually we find it easier to simply tell test organizations to use two computers, since that’s going to be the most common configuration “in the field”). Don’t Forget Plugfest For anyone working with file system filter drivers should always plan on attending the periodic 'Plugfest' that Microsoft sponsors to aid interoperability. While the opportunity to do direct developer-to-developer interop testing is invaluable, the contacts - both Microsoft and third party vendors - are often more valuable. The events are sponsored by the file systems filtering team at Microsoft and are done directly at the Microsoft main campus. These events are publicized in a variety of ways, including via OSR's well regarded NTFSD mailing list, a highly technical discussion group talking about the development of file systems and file system filter drivers for Windows." Conclusions File system filter drivers are the most complicated type of filter driver to write on Windows; deceptively, they span a very broad range, from “simplistic” (a surprising number of examples fall into this category) to “more complicated than a file system driver”. There are many reasons for this and we’ve tried to touch on some of them. But if you are building a file system filter driver because you think it will be “simpler” than a file system driver, you are likely to be unpleasantly surprised. If you can achieve your functionality without a filter driver, we would suggest doing it in that way. A real filter driver is truly a challenge to build.
Filtering...
File Systems Development Kit
For more than ten years, OSR’s File System Development Kit has been used to develop stable, highly functional, performant and commercially successful file system solutions in the Windows enterprise. Local, remote, embedded, pseudo, layered—whatever type of file system your project requires, the OSR FSDK has been put to the test. What more can you ask for but a hardened solution, that is maintained and fully-supported by industry-recognized experts? We provide a straight-forward interface for you to develop against, while managing the complex interactions with Windows. This means you can utilize your engineering resources to implement your product’s value-add, without having to worry about the myriad of edge cases that you don’t even know about. Want to know more? Check out the interface document available for the FSDK toolkit at www.osr.com. Call or email us with questions. Send your requirements for us to review under NDA. Remember we’re here to help — were not in the business of licensing one of our toolkits to someone with a, “good luck and have a nice day”. We are here to make our clients successful. If this technology and OSR’s reputation is something you think you can take advantage of, we look forward to hearing from you. Contact us at [email protected].
Page 18
How to Fetch... Checked Components for Vista/Server 2008
and installed, you now have to puzzle over which three little files Don was referring to. Read the awful WAIK docs. Zero in on imagex and the clutter of command line options that go with it. That would be one file of the requisite three. Then infer that wimfltr.sys and wimfltr.inf are the other two useful files. Note that the WAIK does not contain any reference I can find regarding how one ought to install wimfltr.sys, although it does note that it is required every now and then. 5. Decide that imagex rivals bcdedit in command line option fatigue. Google how other people are actually using this thing. End up here: http://www.windows-now.com/blogs/robert/archive/2007/11/04/mounting-wim-images-from-windows-explorer.aspx and from there here: http://www.autoitscript.com/gimagex/. 6. Download gimagex from that last link. 7. Install wimfltr (right click on the inf file, select install.) No need to reboot! 8. Use gimagex to mount \sources\install.wim from the checked debug build. 9. Browse around like you did back in the good old XP/W2K3 days and find the driver images you were looking for to begin with and copy them somewhere safe. Heck, while you are there, copy all the drivers from the image to someplace reasonable. Now all I have to do is remember what it was I was doing to begin with that made me decide that using a debug inbox driver would be a good idea.
OSR would like to thank Mark Roddy for this contribution, originally posted to the OSR-hosted NTDEV mailing list. Just because this was such a royal PITA for what used to be such a trivial task, I thought I would share how I wasted an entire day with the rest of you, and perhaps spare some unfortunate the grief I went through. Task: obtain the checked version of an inbox driver for Vista or later OS release in order to install that driver for debugging purposes. Estimated time to complete: one to two hours, based on prior experience with other OS platforms. Most of that time would be spent downloading an OS image if a DVD or CD were not available. Actual time to complete: approximately eight hours and one headache. 1. Search MSDN for a complete checked distribution of the OS (Vista SP1 in this case). MSDN, which had a frightfully bad archive catalog system, now has managed to re-implement that as a hideously and frightfully bad catalog system. As far as I can tell, without an MSDN subscription you cannot find a post-RTM version of the checked distribution legitimately. Note that if you just want the hal and kernel, they are in the WDK. 2. Download the image - approximately 2.5GB. Take a nap. Run some errands. 3. Mount the image either by burning it to DVD or using any of the useful image mounting programs out there. Spend (waste) quite a while searching the image for drivers and come up with nothing. Google this puzzle and discover of course that Don Burn has been here before and noted that now you need to download the WAIK: Microsoft definitely changed things. What you need to do is download the Windows Automated Installation Kit at http://www.microsoft.com/downloads/details.aspx?familyid=C7D4BC6D-15F3-4284-9123-679830D629F2&displaylang=en. This provides a file system that will allow you to open the files on the DVD's and extract the individual files you need. Unfortunately the kit is almost 1GB and you need three small files from it. http://www.tech-archive.net/Archive/Development/microsoft.public.win32.programmer. kernel/2007-09/msg00073.html. 4. Download the WAIK - another hour of my life lost. I did have a nice camembert on rye sandwich. Once downloaded
Who Was at the DDC? Well, We Were, But... N othing like hanging around in building 33 on the
Microsoft Redmond campus waiting for...um, the sky to fall? Anyway, this is where we found ourselves during a break from presentations at the Windows Driver Development Conference (DDC) when we ran into a few colleagues from the driver development community. Fortunately, someone even had a camera to capture the Kodak moment! To the right (left to right) you’ll see: OSR’s Peter Viscarola; DDK MVP Maxim Shatskih; Jake Oshins, Microsoft Hyper-V I/O Architect. Below, Peter and Jake are joined (left to right) by DDK MVP Martin O’Brien; OSR’s Scott Noone; OSR’s Mark Cariddi. A hearty thanks to Maxim for the pics, and to these folks and others who hung with us for some couch time during the conference.
Page 20
PRESORTED FIRST CLASS MAIL
U.S. POSTAGE PAID
PERMIT # 456 MANCHESTER, NH
OSR Open Systems Resources, Inc. 105 State Route 101A, Suite 19 Amherst, New Hampshire 03031 USA (603)595-6500 ♦ Fax (603)595-6503
First Class RETURN SERVICE REQUESTED
The NT Insider™ is a subscription-based publication
New OSR Seminar Schedule!
Course outlines, pricing, and how to register at: www.osr.com/schedule
Seminar Dates Location
Developing File System Mini-Filters (Lab) 23-27 February Boston, MA
Writing WDF Drivers (Lab) 30 March—3 April Portland, OR
Kernel Debugging (Lab) 6-10 April Portland, OR
Writing WDM Drivers (Lab) 27 April—1 May Boston, MA
