1

Whelp, I avoided it as long as possible, but the time has finally come to make some changes to some ancient VB6 code. I'm a recent college grad so I was hoping to stay away from this code that is almost as old as I am.

The DLL is compiled in a local installation of VB6 and deployed via Component Services. That component is then called by VBScript in a Classic ASP application running on a windows 2003 Server.

I've tried just about everything to debug this code using the VB6 debugger, but I can't get it to work. I've compiled the DLL to the same location that Component Services has it deployed to (a random folder on a non-system drive), and set binary compatibility to that same DLL. Each class file has instancing set to MultiUse and MTSTransactionMode set to '1 - NoTransactions.' The component works fine until I attach the debugger, then I get 'ActiveX component can't create object' when trying to instantiate the object. If I detach the debugger, it works again.

I noticed that in Component Services when I attach the debugger, the DLL location changes to C:\Program Files\Microsoft Visual Studio\VB98\VB6DEBUG.DLL, which I assume is just VB6 intercepting calls to the component. When I detach the debugger, it changes back. I should also note that the component is configured to use a specific user identity separate from my logged-in user account, however everyone has been granted access to the component via a user role. The component activation type is also 'Server Application,' however setting it to 'Library Activate' doesn't seem to make a difference.

I've also tried using the VC++6 IDE, and VS2010, but had no luck. I was able to debug the dllhost.exe in VS2010 and was able to hit some breakpoints, but the symbols weren't loaded properly. I tried compiling the DLL with debugging symbols and loading those to the VS2010 environment via the Debug settings menu, however that didn't resolve the issue.

I also didn't find the component services JIT Debugger to be very useful. I'd much prefer to just be able to debug in VB6.

What amazes me most is that this code is thousands of lines and has been maintained without the use of any debugger whatsoever. My young, spoiled self isn't ready to accept that this code can't be debugged.

thomasrea0113
  • 360
  • 1
  • 9
  • 1
    I'd check what user is used to run IIS service – Iłya Bursov Sep 24 '18 at 19:26
  • 1
    @Iłya The site app pool is running under the network service account. What difference would that make? I changed it to the same account that is used to run the component service, however I have the same issue. – thomasrea0113 Sep 24 '18 at 19:34
  • Can you confirm - are you trying to debug this on your server, away from the VB IDE, as I am not sure that will work. My live environment is quite close to yours, Windows Server, IIS, Classic ASP, COM component VB6 DLL's running under a specific user identity and in a non-system folder. We can't debug there. However, we can debug on the dev environment where the vb6 IDE is installed - open the project in the IDE, compile it, set breakpoints and F5. Run the IIS side and the VB IDE will flash when the breakpoint fires. – Vanquished Wombat Sep 25 '18 at 06:59
  • You cannot debug this VB6 DLL while it's intercepted by COM+. Just leave it out of Component Service (must delete VB6 DLL app from COM+ console), compile DLL w/ PDBs (and No Optimizations) and use VS2010/2015 to debug both ASP Script and Native on the w3wp.exe IIS (Express) worker process. If you need to debug object pooling or any other COM+ feature used by the DLL you'll have to resort to printf debugging AFAIK. – wqw Sep 25 '18 at 10:11
  • @VanquishedWombat The non-system drive is attached to the IIS server, which is also where I build, compile, and attemjpt to debug the DLL. This is a separate debug environment. I've taken the exact steps you've outline, and VB6 correctly intercepts the calls and funnels them through the VB6 Debug DLL (as evident by view the DLL source property of the COM Component). However, when connecting to the ASP application, the COM object can't be created. – thomasrea0113 Sep 25 '18 at 13:06
  • @wqw I'm confused... If I don't register the COM component, then there will be nothing to attach to in VS2010. 2015 won't install on a 2003 server. As I said, I was able to get that far in 2010, however the symbols weren't loaded properly so debugging wasn't very useful. Also, can you clarify why you think the COM application can't be debugged in VB6? This article states otherwise: https://learn.microsoft.com/en-us/windows/desktop/cossdk/debugging-in-the-visual-basic-ide. I've already shown that VB6 attempts to intercept calls, it just fails to create the object. – thomasrea0113 Sep 25 '18 at 13:11
  • @Developer4993 I remember having problem with debugging when VS runs under your account and iis under system/network account, or when your account is not in admin group – Iłya Bursov Sep 25 '18 at 14:04
  • @IłyaBursov I'm not having issues with VS, I'm having issues with VB6 IDE. VB6 Components cannot be debugged in VS unless you attach to the dllhost.exe which runs the compile DLL. I tried setting the App Pool to use my user account just for testing purposes, and I have the same issue. – thomasrea0113 Sep 25 '18 at 15:26
  • @Developer4993 - sounds like you have a more recent setup than mine. I develop & debug VB6 dll's in the VB6 IDE itself. If the dll you are trying to debug is already installed under com then you will likely need to recompile your debug version with a different name and mod the IIS-side code to refer to the new id otherwise your com version is probably getting found first is my hunch. You are also likely to hit permission issues as other users have pointed out. Sorry can't be more precise. – Vanquished Wombat Sep 25 '18 at 17:03
  • 1
    @Developer4993 COM registration is different than COM+ registration. The second one is done by creating so called COM+ Apps in Component Services console. These intercept regular COM registration and spawn in-proc servers in surrogate hosts. Remove VB6 DLL from COM+ to be able to debug. – wqw Sep 25 '18 at 21:40
  • My experience follows @wqw. Once the system has run the deployed dll it is hard to get it to prefer to run the debugging version in the VB6 IDE. To do the latter I have had to change the dll id slightly whilst debugging (and referring code in the IIS app), then when debugging is complete reverse the name chamge, compile and deploy to component services. – Vanquished Wombat Sep 27 '18 at 06:38
  • [This post](http://www.vbforums.com/showthread.php?866841-VB6-IIS-Configuration-for-RoundTrip-Debugging-of-http-RPCs) might be useful for debugging VB6 COM DLLs (but before getting under COM+ registration) in Classic ASP. – wqw Sep 30 '18 at 09:26

0 Answers0