There is no security issue that I can think of. There is most certainly a performance issue, the Debug build of your assemblies contains an attribute (DebuggableAttribute) that will always prevent the jitter optimizer from optimizing the code. This can make a great deal of difference on the perf of the running program. Optimizations performed by the jitter are documented in this answer.
You could have a problem with memory consumption. The garbage collector will operate differently, keeping local variables alive until the end of the method body. This is a corner case and such a problem should have been diagnosed while testing the app, assuming you used realistic data.
Specific to VB.NET, shipping the Debug build can very easily cause your program to crash with an OutOfMemoryException when it is run on your user's machine without a debugger attached. It fails due to a leak on WeakReferences, used by Edit+Continue to keep track of classes that have an event handler with the WithEvents keyword.
If you don't have a need for the perf enhancements produced by the jitter optimizer and don't ship VB.NET assemblies then there isn't much to worry about.