You can do side by side registration free COM using Delphi, I believe you need to write a manifest file like this, this is an inexact and incomplete example, but if you follow the information available around various places, and substitute your own class and interfaces, I believe this shows the minimal amount of entries you need to do:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">
<assemblyIdentity version="14.0.3615.26342" processorArchitecture="*" name="MyApp" type="win32"></assemblyIdentity>
<description>my app description</description>
<dependency>
<dependentAssembly>
<assemblyIdentity type="win32" name="Microsoft.Windows.Common-Controls" version="6.0.0.0" processorArchitecture="*" publicKeyToken="6595b64144ccf1df" language="*"></assemblyIdentity>
</dependentAssembly>
</dependency>
<file name="MyComObject.DLL" hashalg="SHA1">
<comClass clsid="{AA67839B-8AF0-4651-BDEE-96F01E44A682}" tlbid="{2E23AF44-33A0-48AD-88A9-948B004E0982}" description="MyClass"></comClass>
<typelib tlbid="{EEEEEEE4-33A0-48AD-88A9-948B004E0982}" version="1.0" helpdir="" flags="FLAGS"></typelib>
</file>
<comInterfaceExternalProxyStub name="IMyThing" iid="{AAAAAAAA-4584-4AEE-9FA0-667460953082}" tlbid="{2E23AF44-33A0-48AD-88A9-948B004E0982}" proxyStubClsid32="{AAAAAAAA-0000-0000-C000-000000000046}"></comInterfaceExternalProxyStub>
</assembly>
You can find some more help here:
http://www.mazecomputer.com/sxs/help/inside2.htm
As for using the manifest in Delphi, you must include it into an RC file, and then link that RC file to your EXE, and do NOT check the "Enable Windows Themes" checkbox in the project options (Delphi 2007 or later) as that would override the manifest you are trying to link here. THe part above that references "Microsoft.Windows.Common-Controls" is the way you make your Delphi/VCL app theme aware, so if you didn't want that, take that whole XML dependency section out.