I checked Excel Application not Closing from Outlook and tried the solution from the voted answer.
To make sure it was nothing else I was doing I wrote the simplest macro for Outlook to only open and close Excel. It's not the end of the world as this background process consumes almost nothing from memory, but it only closes when Outlook (who called it) closes.
I placed this in the "ThisOutlookSession":
Public Sub Test()
MsgBox "Trying"
Dim exApp As Excel.Application
Set exApp = Excel.Application
exApp.EnableEvents = False 'Originally I didn't placed this, but I saw it on the other posts
exApp.DisplayAlerts = False 'Originally I didn't placed this, but I saw it on the other posts
exApp.Visible = False 'Originally I didn't placed this, but I saw it on the other posts
exApp.Quit
Set exApp = Nothing
MsgBox "Finished"
End Sub
Tried on Office 2010 and 2016 with same results.