Thanks for all the comments and emails about my blog entry, PDB Files: What Every Developer Must Know. There was lots of nice feedback, but more importantly great questions! Instead of responding in the blog comments to these questions as I normally do, a couple of the questions were important enough that I wanted to make full blog entries out of them. Please continue to ask any questions you have about PDB files as I am taking it as my personal mission to ensure that everyone developing for Windows knows exactly how to use PDB files!
The first question I want to cover is from Eric Hill:
Thanks for the deep dive on PDB files. You’re exactly right, all the information you have provided probably exists somewhere, but not all together.
<
p style=”margin-left:36pt”>I suspect the information you’ve provided explains a problem I had recently doing remote debugging I copied my private build to another machine, with PDB file right there with it, but when I remote-debugged, VS could not find any of my breakpoints. Just on a hunch, I moved the executable files on the remote machine such the path was the same as where the files were on the build machine, and that fixed the problem. But that doesn’t seem like a very reasonable requirement. What approach would you take to remove this restriction for remote debugging?
For the most part, remote debugging works well. However, the one thing I haven’t found discussed in the Remote Debugging documentation is where the PDB files are loaded, which is the crux of Eric’s issue.(For the rest of this article, I’m assuming you’ve read all the remote debugging documentation.) To make everything clear, let me define two terms; the remote machine is where MSVSMON.EXE and your application execute and the local machine is where the Visual Studio IDE is running. The key trick to know about remote debugging is for .NET binaries, the PDB files are loaded on the remote machine and for native binaries, the PDB files are loaded on the local machine.
From Eric’s question, I’m betting he was doing native remote debugging because when he moved the binaries to the same path on both machines, everything worked like expected because it was Visual Studio found the binary on the local machine and in turn, the PDB file as I described in my previous blog entry.
If you want to verify where the PDB files load, here’s how I verified the loading. Set up remote debugging for a console application using your favorite language. Put the project in the same drive and directory on both the local and remote machine. Start single stepping with Visual Studio and stop at the entry point of the application. Run Process Explorer on both machines and in each press CTRL+F to search for handles and DLL strings. Enter “PDB” and click the Search button. Here’s a screen shot from the local machine where I was remote debugging to a native C++ application:
With Visual Studio doing the symbol loading for native remote debugging, both local build and public build symbols load where you expect them to load. It’s a little more interesting for .NET binaries for remote debugging. For your private builds, you’ll copy the binary and matching PDB file to the remote machine where the loading occurs. What about the public build PDB files for .NET binaries?
As all .NET PDB files are loaded on the remote machine, the public build PDB files are loaded there as well. In order to get them you have to set the _NT_SYMBOL_PATH environment variable to point to your symbol server. If you don’t want to do that manually for forty test systems, you can use the PowerShell script I wrote to automate the process. Once you have the _NT_SYMBOL_PATH set on the remote machine, when you start remote debugging, Visual Studio will let you know that it will be downloading public symbols to the remote machine:
With remote debugging, if both machines have the local build binaries in the same drive and directories, life is easy because the symbol loading occurs without you having to worry about the type of development you’re doing. Eric asks if that’s a reasonable requirement. Well, I don’t have a real answer other than to say that’s just kind of how it is, especially for native code. I wanted to provide the background as to how remote debugging worked so you would understand the limitations and ramifications.
Please keep the questions about PDB files coming!
Personal note: I have to let everyone know that Eric Hill, who asked the above question, had a huge influence on my career. Way back in 1992/1993 he was punished for some horrible past life transgression by being stuck with me as a lab partner for CSC 311 (Computer Organization and Logic) at North Carolina State University. That was the killer death class in the CS curriculum that everyone dreaded as it was electrical engineering for CS majors. Back in those days, you were handed a breadboard, an Intel 8051CPU, a bunch of wires and you truly built a computer, none of that easy simulation garbage they have today. Eric’s brilliant (he got an A+) and pulled my stupid butt through the whole class (and I was thrilled with my C-). Oh, he also kept me from electrocuting myself a couple of times as well. When it came time for the end of year project, where we had to do something real, like control hardware devices, with our hand built computer we didn’t want to have to unplug all those wires we had carefully gotten working. Eric had the brilliant idea of doing a PROM-based debugger as our project. It was all software and much easier than anything else we could have done. As you can guess, all my interest in debuggers stem from that wonderful project. Thanks, Eric!
Cloud management is difficult to do manually, especially if you work with multiple cloud…
Azure’s scalable infrastructure is often cited as one of the primary reasons why it's the…
https://www.youtube.com/watch?v=wDzCN0d8SeA Watch our "Unlocking the Power of AI in your Software Development Life Cycle (SDLC)"…
FinOps is a strategic approach to managing cloud costs. It combines financial management best practices…
Using Kubernetes with Azure combines the power of Kubernetes container orchestration and the cloud capabilities…
In the intricate landscape of modern business, compliance is both a cornerstone of operational integrity…
View Comments
John,
Thanks for the great post once again. I was looking forward to your debugging virtual training, but unfortunately it was cancelled.
The company I work for is pushing pack against building release mode binaries with debug information generated, one of the reasons I signed up for the class :). They are afraid performance will be affected.
My question is what are the best command line arguments for generating symbols in release mode? Also is there somewhere I can reference to show that there should be no performance hits.
Thanks
Hey, John,
Little did you know, 15 years ago I had a feeling I would be needing to understand VS Remote Debugging in about 15 years, so I befriended you and got you hooked on debuggers, so that when the time came, you would be able to answer my questions. Pretty good foresight on my part, don't you think?
I'm afraid you lose your bet, but you had no chance, because my main exe is mixed-mode (C++/CLI rocks!), and then it loads a purely managed (C#) assembly for some stuff. Here is what I found:
Test 1: Path to the exe is the same on both machines. PDB files are beside the executable on both machines.
Results: EXE listed twice in modules window, once for native, once for managed. Both say "symbols loaded" from the common path. C# dll also says "symbols loaded" from the common path. ProcExp shows no hits for PDB files on the local machine, 4 hits on the remote machine: 1 hit for myapp.exe, 3 hits for msvsmon.exe. Breakpoints in both exe and C# dll are found. Nirvana.
Test 2: Path to exe is the same on both machines. PDB files deleted from remote machine.
Results: EXE listed twice in modules window, native says symbols found, managed not. Oddly, procexp on local machine shows no hits for myapp.pdb. procexp on remote machine shows a couple PDBs, but not mine. Breakpoints in native C++ code are found, breakpoints in managed C++ and C# are not found.
Test 3: Path to the executable is different on the remote machine. PDB files are beside the executable on both machines.
Result: Modules window only shows the exe once (native), and the C# DLL does not show up at all (!). exe says symbols are loaded from the local path. ProcExp on local machine still shows no hits for .pdb files. ProcExp on remote machine shows no PDB hits either. Breakpoints in native C++ are found, breakpoints in managed C++ and C# are not found.
Test 3 is the one that puzzles me -- if the PDB files are right there next to the exe on the remote machine, why can't the debugger find them. I'm sure there is a good technical reason.
The requirement, apparently for managed debugging, that the path to the exe be the same on local and remote machines is really not that big a deal, for me at least. All of the machines are in my office; I can put the files where I need to.
Many thanks, and thanks for the post about disabling the loading of symbols for a DLL. I have a situation where that will be helpful.
Take care,
Eric
Hi John,
I find it rather strange that the PDB files for .Net applications should be on the remote machine, not the local machine.
After our release builds, we store all PDB and EXE files on a symbol server. That way we can easily debug crash dump (.DMP) files sent to us by customers.
Sometimes we use remote debugging to solve a problem at a customer (using VPN over the Internet), and for native applications, the debugger nicely gets the .PDB file from the symbol server.
However, if we would introduce .Net in our applications, would this mean that we have to send the .PDB file to our customer?
This will not only be a rather slow task (our PDB files can easily be 200 MB), but also this means that we have to give all of our debug information in the hands of our customer.
Aren't PDB files meant to be 'sensitive'; something that you want to keep within the walls of your company?
What do you think?
Thanks,
Patrick
Now that Devscovery is over, it's time to get back to answering questions about one of my favorite subjects:
Thanks for your post. I was also doing the mistake of Native debugging and that got resolved after reading your article.
.. but it isn't :(Visual Studio's remote debugger is a really neat bit of kit that lets people with a...
A handy trick I use is to keep all the source code on my local machine on it's own drive partition. I use S: (S for Source).
Right click on this drive and make it available as a network Share (with appropriate permissions).
On the remote machine set-up a mapped network folder, and map it to the same drive letter - S: again.
Now, you don't have to mess about manually copying pdb's to the remote computer because the same pdb file can be found using the exact same path on both the local and remote computer.
Hy,
Thanks for this very interesting post. I'm getting trouble with remote debugging a native application through a VPN.
The remote debugger show that i'm connected but doesn't answer back.
Do you know what can i do ?
Regards,