Disclaimer: I am one of the founders of the Rubberduck project
EDIT 2020-06: Rubberduck does not require administrative privileges to install anymore, and add-in registration occurs during installation. If the installation was made for "all users", the add-in needs to be registered for each user manually - the installer provides a start menu shortcut for this now, but this answer still accurately describes what's going on under the hood, and how to manually make it work.
</meta>
Rubberduck is a VBE add-in, not an Excel or COM add-in; if it's properly registered as a VBE add-in you'll find it in the VBE's Add-In Manager, from the Add-Ins menu:

note: avoid loading/unloading Rubberduck from that window - there are known teardown issues (most of which are fixed in next release 2.0.12) that don't play well with the VBE's add-in manager.
The VBE looks for available add-ins under the HKCU hive, which is for the current user - if you're not the admin user that ran the installer, the VBE doesn't know there's a new add-in because there's nothing under HKCU that says so.
Per the installing instructions in the project's wiki:
Installing for non-admin users
The Rubberduck installer requires administrative privileges, notably to register the COM types that enable the unit testing feature. Because VBE add-ins need to be registered under the HKCU registry hive, the installer will create the Addins key for the admin user that is running the installer.
To register the add-in for a non-admin user, you need to run the following PowerShell commands:
Set-ExecutionPolicy -ExecutionPolicy RemoteSigned
New-Item -Path 'HKCU:\Software\Microsoft\VBA\VBE\6.0\AddIns64\Rubberduck.Extension' -Force
New-ItemProperty -Path 'HKCU:\Software\Microsoft\VBA\VBE\6.0\AddIns64\Rubberduck.Extension' -Name Description -PropertyType String -Value 'Rubberduck'
New-ItemProperty -Path 'HKCU:\Software\Microsoft\VBA\VBE\6.0\AddIns64\Rubberduck.Extension' -Name FriendlyName -PropertyType String -Value 'Rubberduck'
New-ItemProperty -Path 'HKCU:\Software\Microsoft\VBA\VBE\6.0\AddIns64\Rubberduck.Extension' -Name LoadBehavior -PropertyType DWord -Value 3
This will safely create the required registry keys under HKCU, where the VBE is looking for its add-ins: HKCU:\Software\Microsoft\VBA\VBE\6.0\AddIns64
. Note that 32-bit VBA hosts will be looking for add-ins under the \6.0\AddIns
key.
Alternatively, you can create the missing keys manually in the registry editor:
