Symbol Servers are one of the best things that have ever happened in Windows debugging. Having a central store for all PDB files makes the debugging experience so much better because you’ll always get exact right symbols with essentially no effort on your part. This means perfect call stacks on native code and if you’ve done your job correctly by using the Source Server indexing tools, you’ll have the exact version of the source code used for the build automatically show up in the debugger. As the Symbol and Source Servers are trivial to set up, whenever someone tells me they are having trouble getting symbols and source, I do have to wonder if they are serious about developing software.
Probably the most common question I get about the debugger is “Why does the debugger hang on me when I start debugging or occasionally in the middle of debugging?” As wonderful as the Symbol and Source Servers are, I hate to say this, but they are at the root of the problem. In fact, I have a very special extra sensory perception (ESP) skill. Even if I’m not in the office, I can always tell when a company’s Source Server has gone down. It’s all those emails going back and forth saying “Hey, what happened? This morning I was debugging just fine, but every time I start this afternoon, the debugger hangs. Visual Studio’s got a bug.” While I’m proud of my ESP skills, I wish they would tell me the winning lottery numbers instead.
As the Symbol and Source Server are utilizing network file shares all it takes is the server going down or an invalid share specified in either the _NT_SYMBOL_PATH environment variable or Visual Studio Options dialog. With the network timeout blocking the user interface, you have two choices. Either wait it out or kill the debugger with Task Manager. Obviously, the first trouble shooting step is to see if you can ping the Symbol Servers you’ve told the debugger to use. If there’s no response, you’ve solved the problem. With a down Symbol Server, I temporarily work around the problem by starting a Command Prompt or PowerShell window, disable the _NT_SYMBOL_PATH environment variable and start DEVENV.EXE (Visual Studio) or WINDBG.EXE from that shell. When Visual Studio starts, I go into the Options dialog and disable the symbol server there as well.
The other debugger slowdown you run into is when you first start debugging on a new operating system installation, after a major gaggle of updates on Patch Tuesday, or your company’s Symbol Server is based in another country. In all those cases, the debugger slowdown is because of downloading all the symbols into your local cache. As some of these PDB files can be large and networks spotty, it’s a pain.
Fortunately, there’s an easy remedy in the SYMCHK.EXE program that comes with the Debugging Tools for Windows (AKA WinDBG). Instead of having the debugger download the PDB files, the trick is to fill your cache with them before you need them. Of course, if you like having that extra-long debugging session once a month for an extended coffee break you may want to forget all about SYMCHK.EXE.
To make SYMCHK.EXE easier to run always ensure you have the _NT_SYMBOL_PATH environment variable set so you don’t have to specify giant sets of command line parameters. (Previously on my blog, I’ve written PowerShell scripts to make this setup easy for VS 2008 and VS 2010.) To fill your local symbol cache, run the following two commands to get the .NET and operating systems DLLs:
symchk -r C:windowsMicrosoft.NET.dll
symchk -r C:windowsSystem32.dll
Once you get your cache filled, you can keep running those two commands so they pick up any changed files. So I don’t have to remember to run the above commands after, on my machines I set up a Scheduled Task to run the two commands in a batch file every Wednesday morning at 5:00 AM. Obviously, you can run SYMCHK.EXE against your product’s installation directories to get those builds into your cache as well.
Now debugger hangs and stutters should be a thing of the past for you. Less time waiting on the debugger means faster debugging, which means more time writing code. As always if you have any questions about SYMCHK.EXE or symbols in general, let me know in the comments.
Oh! I’m feeling an ESP moment… Numerous of you are thinking “This is all great but what if the debugger hangs on just one or two symbols every time? Is there a way to tell the debugger to skip loading those symbols?” I knew you were going to ask that question, so I wrote about it back on May 29, 2009. How’s that for preparation!?
Cloud management is difficult to do manually, especially if you work with multiple cloud…
Azure’s scalable infrastructure is often cited as one of the primary reasons why it's the…
https://www.youtube.com/watch?v=wDzCN0d8SeA Watch our "Unlocking the Power of AI in your Software Development Life Cycle (SDLC)"…
FinOps is a strategic approach to managing cloud costs. It combines financial management best practices…
Using Kubernetes with Azure combines the power of Kubernetes container orchestration and the cloud capabilities…
In the intricate landscape of modern business, compliance is both a cornerstone of operational integrity…
View Comments
Hi,
I'm using the SYMCHK.EXE too, but found out, the it doesn't check if the pdb is already present in the Symbol-Cache and downloads it again. This causes a lot of traffic.
André
PingBack from http://topsy.com/trackback?utm_source=pingback&utm_campaign=L2&url=https://training.atmosera.com/CS/blogs/jrobbins/archive/2010/06/02/the-secret-to-avoiding-debugger-slowdowns.aspx
when I run symchk as you specified for MS public symbols, all files give me a FAILED - xxx.pdb mismatched or not found message
I used your script to configure the symbol servers first.
André,
I think you're using a very old version of SYMCHK.EXE. The version that I'm using from WinDBG 6.11 skips downloading PDB files if they are already in the cache.
Steve,
After running my scripts to configure the symbol server, did you log out and log back in? You're getting that error because the _NT_SYMBOL_PATH environment variable is not set.
Hope it helps!
-John Robbins
John,
I figured out that it wouldn't work if _NT_SYMBOL_PATH contained more than one http server, so I setup a script to set that envvar to just the msdl site (leaving off the referencesource site), run symchk, then set the envvar back to both servers.
Is there any reason why you setup your scheduled task to scan both locations that you showed instead of just C:windows*.dll since the Microsoft.NET location is within that recursive search? Just curious...
-Steve
The other day I wrote about using SYMCHK.EXE from the Debugging Tools for Windows (WinDBG) install to
Steve,
Good catch on the two commands. :) I meant to say C:WindowsSystem32 so fixed the blog entry.
I've got my _NT_SYMBOL_PATH set with both the reference source and MSDL locations and it works. However, I've got both separated out in the environment variable:
SRV*C:SYMBOLSPUBLICMicrosoftPublicSymbols*http://referencesource.microsoft.com/symbols;SRV*C:SYMBOLSPUBLICMicroso
ftPublicSymbols*http://msdl.microsoft.com/download/symbols
- John Robbins
John Robbins a publier 2 posts à propos de l'utilisation de SymChk (Symchk.exe, livré avec les Debugging
Remote Debugging question:Any information you can provide me is very much appreciated. I am remote debugging with visual studio 2008 a target machine from my workstation. Both machines are domain accounts running XP pro. I log into both with my domain account.Twice after severe crashes my Visual Studio 2008 app no longer recognizes the target.I need to have I.T. remotely log on to my target machine change the domain name with their admin rights and then I'm good to go.I think there probably is a visual studio file that contains some corrupted information that perhaps I need to delete to solve the problem in lieu of call I.T. (who are often MIA). Thanks in advance for any information you can provide.Rick EisPS. Tell JR that the CLR via C# is excellent
I also get complete failures as I run this too:
SYMCHK: AcSignExt.dll FAILED - AcSignExt.pdb mismatched or not found
SYMCHK: AcSignIcon.dll FAILED - AcSignIcon.pdb mismatched or not found
SYMCHK: atmfd.dll FAILED - atmfd.pdb mismatched or not found
SYMCHK: AUDIOKSE.dll FAILED - audiokse.pdb mismatched or not found
SYMCHK: blackbox.dll FAILED - blackbox_notestroot.pdb mismatched or not found
SYMCHK: ccmcore.dll FAILED - ccmcore.pdb mismatched or not found
...
So as a workaround, I also downloaded a complete set of symbols from the windows debugging website:
http://www.microsoft.com/whdc/devtools/debugging/symbolpkg.mspx#d