MCCI Blog

Useful Tips

Tips and useful step-by-step procedures. Not USB specific!

Using Microsoft's VHD version of Windows Server 2015 R2

Posted by Terry Moore
Terry Moore
 
User is currently offline
on Sunday, 06 September 2015
in Useful Tips

Microsoft has a pre-installed VHD version of Windows Server 2012 R2 available for download. 

The reference can be found here: https://technet.microsoft.com/library/dn303418.aspx, under the section titled "Installing versions distributed as VHDs".  The VHD can be downloaded from here: http://www.microsoft.com/en-us/evalcenter/evaluate-windows-server-2012-r2#evaluation_521.

I can confirm that the download evaluation image works well, and is much faster to set up than installing from distribution ISOs. But I could not make their instructions work for me, and a quick search of the web indicated that others were puzzled.

Here are the original instructions:

To use the VHD distribution, you must have a computer running Windows Server 2008 R2, Windows Server 2012, or Windows Server 2012 R2. The Hyper-V server role must be installed.

To install the VHD

  1. Download the VHD file.
  2. Start Hyper-V Manager. On the Action menu, select Import Virtual Machine.
  3. Navigate to the directory that the virtual machine file was extracted to and select the directory (not the directory where the VHD file is located).
  4. Select the Copy the virtual machine option.
  5. Confirm that the import was successful by checking Hyper-V Manager.
  6. Configure the network adapter for the resulting virtual machine: right-click the virtual machine and select Settings. In the left pane, click Network Adapter. In the menu that appears, select one of the network adapters of the virtualization server, and then click OK.
  7. Start the virtual machine.

These instructions don't quite work. 

First of all, the VHD is not a VM, so you can't copy or import it.

Second, it's not clear (from web searches) that this will work at all under Server 2008 R2 -- there were hotfixes suggested.

Luckily, I have Windows 10 Pro, and this works fine as a host. (In fact, it appears that Windows 8.1 Enterprise also works; I used that for some of the screenshots. But I did most of this on Windows 10, and that's what I'd recommend to others.)

Here's the procedure I followed.

To prepare the Windows 10 Pro system

  1. Make sure you have Windows 10 Pro installed. Press [Windows}R and type "winver", then press enter. In the resulting text box, confirm that you see the text, "The Windows 10 Pro operating system and its user interface are protected..." If you see Windows 10 Home, you'll need to upgrade.
  2. Right click on Start, and select "Programs and Features"
  3. Select "Turn Windows features on or off" (on left)
  4. Scroll down to Hyper-V, and make sure the top level checkbox and subordinate checkboxes are all ticked. If not, check them, and go through the installation process. A reboot will be needed at this point.
  5. Once you've confirmed that Hyper-V is installed, press [Windows]R and type "mmc", then press enter. You'll probably get a UAC pop-up ("do you want to allow this app to make changes to your PC?"); if you do, click [Yes].
  6. Select File>Add/Remove Snap-In...
  7. Scroll down to Hyper-V Manager, click "Add". There typically will be a pregnant pause, followed by a progress bar; then Hyper-V Manager will appear under "Selected snap-ins" on the right. Click OK.  (The process of doing this creates an applet; from this point on, you can click start and type "Hyper" and "Hyper-V Manager" will show up.
  8. You'll now see Hyper-V Manager under Console Root:

  9. Click on "Hyper-V Manager" in the left pane, expand things if necessary, and you'll see something like the following:

  10. You probably want to configure one or more networks for use by Hyper-V. To do this, make sure your machine is selected in the left panel, then on the Action menu select Virtual Switch Manager... I created an External switch, Connection type  "External network", and selected my Ethernet adapter in the drop-down list. (I've also successfully selected a Wi-Fi adapter at this step.) Very important: be sure to select "[x] Allow management operating system to share this network adapter".  Click OK.
  11. You're now ready to proceed.

To install the VHD

  1. Download the VHD file. 
  2. If needed, copy the VHD file to a suitable location. On my system, Hyper-V looks for VHD files in C:\Users\Public\Documents\Hyper-V\Virtual hard disks, so I put it there.
  3. Start Hyper-V Manager, and make sure your machine is selected in the left panel. On the Action menu, select New... > Virtual Machine.
  4. Go through the steps of creating a new virtual machine. On the following menu, make sure you select "Generation 1 Virtual Machine". Click Next.

  5. "Assign Memory": Specify 2GB of RAM. I didn't select "[ ] Use Dynamic Memory for this virtual machine". Click Next.
  6. "Configure Networking": you can do this later -- just click Next
  7. "Connect Virtual Hard Disk" -- I selected "Attach a virtual hard disk later". Click Next.

  8. "Completing the New Virtual Machine Wizard" -- review the options, and click "Finish" to create the VM.
  9. Verify that the VM was successfully created by checking Hyper-V Manager
  10. Configure the downloaded VHD file as the disk for the resulting virtual machine: right-click the virtual machine and select Settings. In the left pane, click IDE Controller 0. In the right pane, select "Hard Drive", then Add. In the menu that appears, under Virtual Hard Disk, select Browse..., and navigate to the VHD image you downloaded. Select the file and click OK.
  11. Configure the network adapter for the resulting virtual machine: right-click the virtual machine and select Settings. In the left pane, click Network Adapter. In the menu that appears, select one of the network adapters of the virtualization server, and then click OK.
  12. Start the virtual machine.

Was this post useful to you? Want to see more on some topic? Leave a comment, or find me on Twitter (@TmmMcci) or LinkedIn (http://www.linkedin.com/in/terrillmoore/en).

 

 

Hits: 2603 0 Comments
0 votes

Cygwin home directories

Posted by Terry Moore
Terry Moore
 
User is currently offline
on Saturday, 04 July 2015
in Useful Tips

I recently installed some new tools from a third party, and had to update my Cygwin installation.  There is no official version of the overall release (see the FAQ); but I was running a release from November 2014, or so, and I updated today (July 4).

After the update, my Cygwin home directory had changed, and wasn't what I wanted.

Summary: if your Cygwin home directory is wrong after an update, you need to look at /etc/nsswitch.conf, not /etc/passwd.

Here are the details.

After I updated, my home directory was a different place than before. I recognized this immediately because the prompt changed from

bash:redacted::~ $

to

terry@redacted ~
$

And of course, all the other settings from my .bashrc were not taking effect.

In my setup, for historical reasons, my home directory is "c:\usr\terry" -- I've been running this setup since Windows NT days, and it's been nice not having a name that changed based on the Windows version in use. I use the same home directory for Cygwin and Windows.

Also for historical reasons, I have let Windows continue to assign whatever it felt like for a home directory. On my current Win8.1 machine, that's "c:\users\terry".

In the old version of Cygwin, this was accomplished by editing the /etc/passwd file (as on Unix). My passwd file contains the following line:

terry:unused:1000:513:U-REDACTEDterry,S-REDACTED-1000:/cygdrive/c/usr/terry:/bin/bash

In previous versions of Cygwin, this was all that was needed; when I launched a Cygwin window, bash found my .bashrc file in c:usrterry, and things proceeded as I expected.

After I updated to the version including Cygwin 2.0.4, this no longer worked. Cygwin was not honoring the contents of /etc/passwd.

I'll omit a discussion of the things I tried. It turns out that, by default, the newer versions of Cygwin no longer use /etc/passwd.  Instead, there's a new mechanism, /etc/nsswitch.conf, based on a similar mechanism from Linux. This page has all the details.  Search for nsswitch.conf, and don't worry too much about all the AD and SAM stuff; I was able to ignore it.

In order to get my home directory setup working again, I added the following to /etc/nsswitch.conf:

db_home: /cygdrive/c/usr/%U

%U is expanded to the current user's Windows login name (terry in my case).

The following setting is similar and useful, but it don't do what I want.

db_home: windows

It doesn't do what I want, because it uses the Windows home directory, which is c:\users\terry. This causes my home directory to be /cygdrive/c/Users/terry, which isn't the place I want. (Something like this might be good for other people, however, which is why I mention it.)

(Updated 2015-09-06: add some missing backslashes, make one-page article.)

Was this post useful to you? Want to see more on some topic? Leave a comment, or find me on Twitter (@TmmMcci) or LinkedIn (http://www.linkedin.com/in/terrillmoore/en).

Hits: 2875 0 Comments
0 votes

Building one driver for Win7, Win8, Win8.1

Posted by Terry Moore
Terry Moore
 
User is currently offline
on Wednesday, 18 March 2015
in Useful Tips

Among other things, MCCI creates Windows kernel-mode drivers for internal product use and for licensing to customers to bundle with their products.

We want to run the same binaries on all compatible platforms. So (for example) we build one driver for x86 that will run on all supported operating systems.

This article summarizes our experience in doing this, and offers some concrete tips on how to do things (and how to solve problems).

We'll start with some reference material.

You have two choices, either limit yourself to the features of the oldest operating system, or dynamically determine what version you're running on, and dynamically import symbols using MmGetSystemRoutineAddress().

MCCI needs to do the latter (use the newer features on newer OS versions, and fall back to a reasonable substitute on older operating systems).

Microsoft has specific recommendations about what to do, posted in two MSDN articles:  https://msdn.microsoft.com/en-us/office/ff554887 (writing), and https://msdn.microsoft.com/en-us/office/jj572863 (building). It is odd to find a recommendation about kernel mode drivers in the "Office Dev Center", but let us not digress. The salient points for our use case are:

  1. Use the most recent applicable WDK (if you want to target 7 through 8.1, use WDK 8.1)
  2. Use RtlIsNtDdiVersionAvailable() to query whether a given feature is available, and then use MmGetSystemRoutineAddress() to get the function name if so.
  3. Use the Microsoft C "typedef typeExp ( functionName ) extension. This extension is somewhat like gcc "typedef typeof(function) typename"; it creates a type from a function, including all the SAL annotations and attributes like calling sequence (CDECL, STDCALL, FASTCALL, etc.)  You can directly create a pointer type from the function, but MCCI normally prefers to create a function type, as this lets us also define substitute functions that can be used when needed.
  4. Don't directly call functions that are not present in all systems of interest.
  5. Link with $(DDK_LIB_PATH)\BufferOverflowK.lib if you need to support Win7 and earlier. (The default if you're targeting Win8.1 willl be $(DDK_LIB_PATH)\BufferOverflowFastFailK.lib -- this is not what you want. Drivers built with the Win8.1 WDK will immediately bug-check on Win7.

This advice is all good -- but not sufficient.  (More about that below.)  Let's start by amplifying some of the above points.

Dynamically Importing APIs

It's very important to check RtlIsNtDdiVersionAvailable() before calling MmGetSystemRoutineAddress() -- MmGetSystemRoutineAddress() doesn't necessarily return NULL on platforms where the function is not supported.

Defining types for your imported functions

Because of calling-sequence issues, it's very important to use the typedef trick. But bear in mind that, at least with some Microsoft compilers, it doesn't work right for FASTCALL functions on X86. MCCI's workaround in that case is to cut and paste the exact definition from the WDK header file (normally wdm.h) and put it in our header file.

Selecting the right Buffer Overflow library

Linking with BufferOverflowK.lib is critical. But you have to make sure that you add this in the right place, because it uses the variable $(DDK_LIB_PATH), which must be already defined.

Find the following place in your .vcxproj file:

The above line will end up assigning a value to DDK_LIB_PATH.

After that line, insert the following:



  $(DDK_LIB_PATH)\BufferOverflowK.lib

The label "KernelBufferLib" is arbitrary.

If you put the property group in the wrong place, it won't have effect, and the WDK will silently use BufferOverflowFastFailK.lib -- which is definitely not what you want.

It's a very good idea to look at your msbuild.log and examine the link.exe invocation -- make sure that the library you wanted was linked. I just search for BufferOverflow in the log, and make sure that it's the library I want.

Don't directly call functions that are not present (KeInitializeSpinLock as an example)

This one is trickier than it looks. Microsoft sometimes changes function calling sequence, or changes from inline to calling a function exported by the kernel. In these cases, the MSDN documentation doesn't help you -- the API is present, but the way it compiles is different.

For example, the API KeInitializeSpinLock() is present in all versions of Windows. However.... In Windows 7 and earlier, it was implemented as an inline function. The kernel (ntkern.sys) didn't export a function named KeInitializeSpinLock.  A driver built with the Win7 WDK will not import KeInitializeSpinLock(); on Win8.1, it will use the inline instead of calling the kernel-exported function. No great loss, as the kernel function does the same thing, plus possibly inserting barriers and/or doing more Driver Verifier checks. (To be honest, I haven't looked into why they made the change.)

The tricky part is building the same driver with the Win8.1 WDK. It will run fine on Windows 8.1, but on Win7, the system won't load it. (You'll get a Code 39 error in device manager, and a message about the driver possibly being corrupt.)

To a developer, Code 39 almost always means "oops, I'm using an export that doesn't exist on this system". Finding the missing export can be tricky. I use depends.exe/depends.dll/depends.chm from the WDK 8.1 tools tree. (There's a website that claims to have it, but there wasn't enough authentication, so I didn't want to download it.) I put the three files on a thumb drive and carry it to the Win 7 system, and run the tool.

"depends.exe" is useful, but itself is a bit tricky. For me, it always complains that it can't find wmilib.sys. I have learned to ignore that as a false positive. The key thing is select each imported DLL, then click on the PI^ button in the top-right box. This will bring any unsatisfied imports to the top of the list, as in the following screenshot.

In fact, on Win7, you'll see that KeInitializeSpinLock appears in red. This tells you (1) Win7 doesn't export KeInitializeSpinLock(), and (2) Win8.1 does.  (You know it does, because your driver loads on Win8.1.) This should cause you to immediately compare the Win7 WDK wdm.h definition of KeInitializeSpinLock with the Win8.1 equivalent. You will immediately see that on Win7, KeInitializeSpinLock is always a FORCEINLINE function; whereas on Win8.1, it may be an external function. You will see some complicated conditional compiles, and (if you're like me) you'll conclude that trying to force the header file to generate the inline when targeting Win8.1 would be a mistake -- you don't know what else will happen. Finally (again, if you're like me), you will realize that there probably is an important reason to use the kernel's KeInitializeSpinLock if it's available.

So we have to do the run-time import MmGetSystemRoutineAddress() trick.

How to do this?

We have to start by defining a symbolic type based on KeInitializeSpinLock(). If you don't need to worry about compiling on older WDKs (no legacy code or systems, lucky you), you can probably just do this:

typedef MYDRIVER_KeInitializeSpinLock_t (KeInitializeSpinLock);

[MCCI has 20 years of customers who need to be supported, which means that I can't simply mandate that the Win7 WDK will go away; and we use common code; so I'm not as lucky as you. But we will discuss the solution to Win7 / Win8.1 WDK source compatibility in another post, if at all.]

You'll need someplace to store the pointer; you don't want to look this up every time.

MYDRIVER_KeInitializeSpinLock_t *MYDRIVER_gpKeInitializeSpinLock;

You'll need a function to use in case you're on an old system and there's no kernel export to call:

MYDRIVER_KeInitializeSpinLock_t KeInitializeSpinLock_DownLevel;

VOID KeInitializeSpinLock_DownLevel(PKSPIN_LOCK pSpinLock) {
    *pSpinLock = 0;
}

At driver initialization, you need to check the system version and import the name:

if (RtlIsNtDdiVersionAvailable(NTDDI_WIN8)) {
   UNICODE_STRING fName;
   RtlInitUnicodeString(&fName, L"KeInitializeSpinLock");
   pKeInitializeSpinLock = (MYDRIVER_KeInitializeSpinLock_t *)MmGetSystemRoutineAddress(&fName);
} else {
   pKeInializeSpinLock = KeInitializeSpinLock_DownLevel;
}

Then you must find all calls to KeInitializeSpinLock() in your code, and change (for example):

KeInitializeSpinLock(&pDriverExtension->MyLock);

to

(*MYDRIVER_gpKeInitializeSpinLock)(&pDriverExtension->MyLock);

MCCI driver code isn't permitted to use global variables at all; so instead of using a global, MCCI would put it in a "driver object extension", but the principle is the same.

By the way, the "MYDRIVER_" prefix is whatever prefix you use in your driver to distinguish your symbols from Microsoft's symbols. (You do use a prefix, don't you?)

Was this post useful to you? Want to see more on some topic? Leave a comment, or find me on Twitter (@TmmMcci) or LinkedIn (http://www.linkedin.com/in/terrillmoore/en).

Tags: Untagged
Hits: 2362 0 Comments

Creating a .lib file for libclang on Windows

Posted by Terry Moore
Terry Moore
 
User is currently offline
on Thursday, 06 November 2014
in Useful Tips

I'm doing some code refactoring work. For a variety of reasons, I want to try libclang, the C interface for clang/llvm. I want to get to doing my tests immediately, and not waste time building it.

I downloaded the LLVM 3.5 distribution for Visual Studio.

Immediate roadblock: they ship "libclang.dll" (the code I want to use, in compiled form), but no "libclang.lib" (which the Visual Studio linker wants for linking my C apps to libclang).

Some searching found this very useful page: Create .lib file from .dll

However, it was manual; I'll never edit by hand if I can write a script. So here are my steps. Obviously, this can be used for any .dll, if it uses C APIs. (Equally obviously, you'll need a header file to compile your C, but ... I have that.)

  1. Start a command window from Visual C. I do this to get all the paths set up correctly for the VC tools. I don't like having my path set statically -- I use many different compilers, and several different versions of Visual C side by side.

  2. In the VC command window, use dumpbin.exe to generate a listing of the exports.
    dumpbin libclang.dll /exports >clangdefs.txt
  3. Then in a cygwin window:
    awk 'BEGIN { print "EXPORTS" }
        /ordinal hint/,/Summary/ {
            if ( $1 ~ /^[0-9]+$/ ) print $1,$4
        }' clangdefs.txt >libclang.def
  4. Finally, back in the VC window (from steps 1 & 2):
    lib /def:libclang.def /out:libclang.lib
    Microsoft (R) Library Manager Version 10.00.40219.01
    Copyright (C) Microsoft Corporation.  All rights reserved.

    LINK : warning LNK4068: /MACHINE not specified; defaulting to X86
       Creating library liblang.lib and object liblang.exp

 Of course, this can all be combined into a single script.

Thanks to Adrian for posting the instructions. You can find terse instructions in MSDN (http://support.microsoft.com/kb/131313), but I certainly agree with Adrian that his instructions are much more to the point.

 

Tags: Untagged
Hits: 2075 0 Comments

Decoding Arbitrary USB Protocols Captured with Total Phase Beagle

Posted by Terry Moore
Terry Moore
 
User is currently offline
on Monday, 16 June 2014
in Useful Tips

One of the most useful tools for USB development is a protocol analyzer, such as the Total Phase Beagle 480. These tools not only capture data, but include class protocol decoders that interpret the trace at a higher level of abstraction.

Sometimes, however, the analyzer software doesn't have the ability to decode the protocol of your device. MCCI often has to capture USB data for new protocols, or for vendor-specific protocols. In this case, the analyzer software doesn't have the appropriate class decoder. In this case, we typically write tools to parse the output from the analyzer and display the data in terms of the target protocol.

MCCI's Yogender Saroha recently wrote a case study of how to do this for Remote NDIS and the Beagle 480, which Total Phase has published on their website, and sent out with their June newsletter. You can access the case study here. The case study includes a link to the complete source code distribution. The tool that Yogi built integrates nicely with the scripting facilities of the Total Phase Data Studio, and works on Windows, OS X, and Linux. You can readily modify the code to parse other protocols (and you'll get an idea of how MCCI does things).

If you find the tool useful, please let us know!

Tags: Useful tips
Hits: 1509 0 Comments
0 votes

Seeing recent mail in Thunderbird

Posted by Terry Moore
Terry Moore
 
User is currently offline
on Thursday, 04 July 2013
in Useful Tips

Thanks go to Balaji Sivan (profile here) who followed up my recent post Outlook tip: Getting just the messages received in the last 2 days with a comment showing how to do something similar in Thunderbird.

I think his comment deserves its own post, so without further ado...

Hits: 1471 3 Comments

Outlook tip: Getting just the messages received in the last 2 days

Posted by Terry Moore
Terry Moore
 
User is currently offline
on Wednesday, 03 July 2013
in Useful Tips

In this post, I describe how to use Microsoft Outlook search folders to show you all and only the messages received in the last two days.

MCCI runs by email.  Even for our web apps like Jira and Crucible, most of us get our information via email. It's simple, it's efficient, and there's only one place to look.

This is great in theory, and works well with mailers like Eudora, but it's not a workflow that Outlook accommodates directly. With Eudora, you could bring everything into your inbox, read the message, then apply filters to move the message to the right folder. That's not convenient with Outlook. Instead, you set up filters so that incoming mail gets delivered to the right folder immediately upon arriving at your system.

Outlook's pre-filtering works pretty well, unless you have a lot of folders. Then it becomes difficult to keep track of the "recent" mail. You can do a quick search for unread messages, but then as soon as you open a message, it drops off the list. That irks me when I'm reading messages to get a picture of what's going on (what my friends with military experience call "building situational awareness"). I like to scan messages quickly, going to the next message unless something demands immediate attention. Sometimes, it's not until after I've moved on that I realize that I need to act on a message two or three back in the list.  Unfortunately, with an "unread message" quick search, you can't back up -- you've read  the older message, so it's no longer in the unread-message search list. Since mail is pre-filtered and dropped into folders, I'm often not sure where to look. After a few painful searches trying to find a message that was filed in a different place than I expected, I stopped using this technique.

The simple approach, and the one I used until I found this approach, is to simply scan over your folders in the pane on the left (the Navigation pane). The ones with unread messages will be in bold. However this is tedious and error prone if you have a lot of folders; and if the folder contains subfolders, the parent folder doesn't get bolded, only the subfolder with the unread message. If the parent folder is closed, you don't see that there's a message waiting at all.

The right approach is to have all the messages for the last day or two show up in a search folder. At midnight every day, the messages from the day before yesterday fall out of the folder. Even with lots of mail and lots of folders, I've found I can cut my mail scan time substantially.

Setting up a "Recent Mail" search folder takes multiple steps, but it's well worth the effort. I showed this to one of my colleagues, who showed it to another; it seems to be useful no matter your work style. It's useful enough (and popular enough within MCCI) that I thought I'd share instructions on how to set it up. Keep reading to get the gory details.

(If you're looking for equivalent instructions for Thunderbird, check this post: Seeing recent mail in Thunderbird.)

Hits: 11404 2 Comments
0 votes
Legal and Copyright Information