Getting symbols for an alternate version of NTDLL.DLL

[Bob Altman]

Hi all,

I'm using Visual Studio 2005 to debug a process crash dump. The dump file
was generated on a computer running Windows XP SP2. I'm debugging it on a
computer running XP SP3. I've defined the _NT_SYMBOL_PATH environment
variable as:

symsrv*symsrv.dll*E:\DebugSymbols\*http://msdl.microsoft.com/download/symbols

When I load the dump file into the VS debugger and look at the loaded
modules I see that symbols were not loaded for ntdll.dll. The "Symbol
Status" is "No matching binary found". I also see that it is looking for a
different version of ntdll.dll than the version in my System32 folder -- not
surprising since the debug PC has different service packs and updates than
the application PC.

So, what's the easiest way to get symbols for the DLLs whose versions differ
from (or which may be completely missing from) the debug PC's DLLs?

Note that we have a bunch of application PCs, each with its own collection
of updates and service packs, none of which are convenient for doing
debugging. So the standard procedure is to capture a crash dump and take it
to a different PC for analysis. So I have the general problem of setting up
the debug PC to make it as easy and automatic as possible to get correct
symbols for the dump file being analyzed.

TIA - Bob

[Hongye Sun [MSFT]]

Hi Bob,

Thanks for your post.

When you debug a dump, the machine on which you debug must have access to
the binaries for the program. Visual Studio will search for the missing
binaries in a certain pattern, if all the search pathes do not contain
matched binaries, the mesage "No matching binary found" appears in the
Modules window.

How does the Visual Studio search for binaries?
It is documented at Dump Module Loading
(http://msdn.microsoft.com/en-us/library/da0e4t52.aspx).

For example, following binaries and dlls are loaded when you create a dump:
D:\qa\exmpl\exmpl.exe
D:\qa\exmpl\dll.dll
F:\windows\system32\ntdll.dlll
F:\windows\system32\kernel32.dll

The dump is located at:
C:\qa\dumps\exmpl.dmp

Visual Studio searches ntdll.dll in the following locations:
C:\qa\dumps\ntdll.dll
C:\qa\dumps\system32\ntdll.dll
C:\qa\dumps\windows\system32\ntdll.dll
F:\windows\system32\ntdll.dll

Solution:
Based on how Visual Studio searches for binaries, the missing binary issue
can be resolved by placing required binaries into the same folder as the
dump file. In the example above, the dll should be placed at
C:\qa\dumps\ntdll.dll. Whenever the dump file is copied to different
machines, the binaries should be copied as well.

In addition, your guess is right. The ntdll.dll files in XP SP2 and XP SP3
are different. They are:
xp sp3 ntdll: 5.1.2600.5512
xp sp2 ntdll: 5.1.2600.2180

Please have a try of the solution above and let us know if the issue has
resolved. Thanks.

Have a nice weekend.

Regards,
Hongye Sun (***@online.microsoft.com, remove 'online.')
Microsoft Online Community Support

Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
***@microsoft.com.
 
This posting is provided "AS IS" with no warranties, and confers no rights.

[Bob Altman]

Post by Hongye Sun [MSFT]
Based on how Visual Studio searches for binaries, the missing binary issue
can be resolved by placing required binaries into the same folder as the
dump file. In the example above, the dll should be placed at
C:\qa\dumps\ntdll.dll. Whenever the dump file is copied to different
machines, the binaries should be copied as well.
In addition, your guess is right. The ntdll.dll files in XP SP2 and XP SP3
xp sp3 ntdll: 5.1.2600.5512
xp sp2 ntdll: 5.1.2600.2180

So what you are saying is that I need to identify all of the missing or
mismatched system DLLs, then go back to the application PC and make copies
of them, and drop them into the folder that contains the dump file. This
either requires two trips to the app computer (one to get the crash dump and
a second to get the missing or mismatched system DLLs once I've loaded the
crash dump into the debugger) or a home-brew way to keep multiple versions
of system DLLs in a handy place to save myself the trip back to the app PC.

Here's the part I don't understand: I am debugging an image of memory that
was taken from the app computer. The debugger knows the names and versions
of the DLLs that are loaded into that memory image. Isn't there some way to
just give the debugger the PDB file associated with one of those DLLs, even
though the actual DLL isn't available? And, if so, is there a way to fetch
that version of that PDB file from the Microsoft symbol server?

Bob

[Bob Altman]

Post by Bob Altman
Here's the part I don't understand: I am debugging an image of memory
that was taken from the app computer. The debugger knows the names and
versions of the DLLs that are loaded into that memory image. Isn't there
some way to just give the debugger the PDB file associated with one of
those DLLs, even though the actual DLL isn't available?

And I guess the answer (based on reading "between the lines" of the KB
article referenced earlier in this thread) is a big fat "no". VS apparently
wants the DLL file even though it has the memory image.

[Bob Altman]

Post by Bob Altman

Post by Bob Altman
Here's the part I don't understand: I am debugging an image of memory
that was taken from the app computer. The debugger knows the names and
versions of the DLLs that are loaded into that memory image. Isn't there
some way to just give the debugger the PDB file associated with one of
those DLLs, even though the actual DLL isn't available?

And I guess the answer (based on reading "between the lines" of the KB
article referenced earlier in this thread) is a big fat "no". VS
apparently wants the DLL file even though it has the memory image.

And here is what a different KB article
(http://msdn.microsoft.com/en-us/library/htzy3t6f.aspx) has to say on the
issue:

+++
When you debug a dump, the machine on which you debug must have access to
the PDB files and the binaries for the program. Visual Studio can cope with
missing binaries for some modules, however, as long as it has enough modules
to generate valid call stacks. The message "No matching binary found"
appears in the Modules window. For more information, see Dump Module
Loading.

If you load a minidump saved with the heap, Visual Studio can load the
symbols even if the application binary is not found. Minidumps saved without
the heap need to find the binary to load symbols. They are considerably
smaller, however, which may be important if you have storage or bandwidth
limits.
---

So it looks like I may be able to resolve symbols without having to manage
copies of system DLLs if I can figure out how to tell adplus to "create a
minidump saved with the heap".

[Hongye Sun [MSFT]]

Hi Bob,

Thanks for your reply.

After performing a complete research on this issue, we found that it is
possible for you to load the symbols without binaries in VS 2005. VS 2005
has already supported to download binaries from symbol server. The key
point is that you must specify symbol path at Visual Studio. Following are
the steps:
1. Open VS 2005
2. Open Menu [Tools] -> [Options] -> [Debugging] -> [Symbols]
3. Add link: http://msdl.microsoft.com/download/symbols into Symbol file
locations
4. Set E:\DebugSymbols\ as cached directory

After that VS 2005 will automatically download the ntdll.dll from symbol
server and load its symbol.

Please have a try and let us know the result. Thanks for your cooperation.

Regards,
Hongye Sun (***@online.microsoft.com, remove 'online.')
Microsoft Online Community Support

Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
***@microsoft.com.
 
This posting is provided "AS IS" with no warranties, and confers no rights.

[Bob Altman]

Post by Hongye Sun [MSFT]
Hi Bob,
Thanks for your reply.
After performing a complete research on this issue, we found that it is
possible for you to load the symbols without binaries in VS 2005. VS 2005
has already supported to download binaries from symbol server. The key
point is that you must specify symbol path at Visual Studio. Following are
1. Open VS 2005
2. Open Menu [Tools] -> [Options] -> [Debugging] -> [Symbols]
3. Add link: http://msdl.microsoft.com/download/symbols into Symbol file
locations
4. Set E:\DebugSymbols\ as cached directory
After that VS 2005 will automatically download the ntdll.dll from symbol
server and load its symbols.

Much to my surprise, that works. Thanks!

The reason I am surprised is because after I created the _NT_SYMBOL_PATH
environment variable, I started seeing a couple of new behaviors that led me
to believe that everything was wired up correctly: (1) I was able to
automatically get symbols for system DLLs as long as those DLLs could be
found on my system, and (2) I started getting a pop-up dialog box
occasionally (not always) when I launch the VS debugger asking me to accept
some license terms, presumably from the symbol server.

So I assumed that defining _NT_SYMBOL_PATH is equivalent to putting the same
information into Tools->Options->Debugging->Symbols. I guess it's not quite
the same, at least as far as giving VS the ability to fetch missing or
mismatched binaries from the symbol server.

Is my understanding correct? Is this close enough to a bug for me to post
it to Microsoft Connect?

Bob

[Hongye Sun [MSFT]]

Hi Bob,

Thanks for your reply.

I think your understanding is right. It should be an issue in Visual Studio
2005.
In addition, we also tested it in latest version of WinDBG and Visual
Studio 2008 SP1. The result is:

1. WinDBG
After configuring environment variable _NT_SYMBOL_PATH, it will
automatically download missing DLL.

2. Visual Studio 2008 SP1
Without any configuration or environment variable, it can download missing
DLL if user manually select context menu [Load Symbol From] | [Symbol
Server] in the call stack window. It will automatically search
http://referencesource.microsoft.com/ and http://msdl.microsoft.com/. The
DLL and Symbol file will be saved in
C:\Users\<UserName>\AppData\Local\Temp\MicrosoftPublicSymbols by default.

It seems that the issue is no longer existing in Visual Studio 2008 SP1 any
more.

Hope this information helps.

Regards,
Hongye Sun (***@online.microsoft.com, remove 'online.')
Microsoft Online Community Support

Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
***@microsoft.com.
 
This posting is provided "AS IS" with no warranties, and confers no rights.

[Bob Altman]

Post by Hongye Sun [MSFT]
It seems that the issue is no longer existing in Visual Studio 2008 SP1 any
more.

Thanks Hongye. It was a pleasure doing business with you!

Bob

[Hongye Sun [MSFT]]

Thanks for your reply, Bob. It is also my pleasure.

Thanks for using Microsoft Newsgroup Service.

Regards,
Hongye Sun (***@online.microsoft.com, remove 'online.')
Microsoft Online Community Support

Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
***@microsoft.com.
 
This posting is provided "AS IS" with no warranties, and confers no rights.