For reasons I haven’t discovered yet (see the update below) trying to remote debug an actual Debug build doesn’t work. You need to use a Release build. Of course that means you need to set up your release builds for debugging. The key thing you don’t want to do is define the _Debug symbol. Start by opening the project properties tab. Select Release from the Configuration drop down on the Build tab. Make sure Optimize Code is unchecked. Then use the Advanced… button to open the Advanced Build Settings dialog and make sure to select full from the Debug Info: drop down menu. All the relevant steps are shown in the figure below.
Turns out the reason you can't remote debug a DEBUG build has to do with Visual C++ and the redistributable libraries. (The C# GUI linked to a C++/CLI .DLL that I built). Turns out with the debug flag set, the linker links in MSVCR100D.DLL and MSVCP100D.DLL, when the flag isn't set it uses the files without the "D" suffix. There are two ways around this and both are thoroughly addressed in the answer to my stackoverflow question.