I ran into the same problem and tried various solutions without success. I got closer when I started releasing all of the COM objects I saved as variables, not just the ones for the workbook, worksheet, and Excel application.
For example, take the following example code:
$Excel = New-Object -ComObject Excel.Application
$Excel.Visible = $False
$Workbook = $Excel.Workbooks.Open("C:\Temp\test.xlsx")
$Worksheet = $Workbook.Worksheets.Item(1)
$UsedRange = $Worksheet.UsedRange
$Range = $Worksheet.Range("A1:B10")
$Workbook.Close()
$Excel.Quit()
[void][System.Runtime.Interopservices.Marshal]::ReleaseComObject($Range)
[void][System.Runtime.Interopservices.Marshal]::ReleaseComObject($UsedRange)
[void][System.Runtime.Interopservices.Marshal]::ReleaseComObject($Worksheet)
[void][System.Runtime.Interopservices.Marshal]::ReleaseComObject($Workbook)
[void][System.Runtime.Interopservices.Marshal]::ReleaseComObject($Excel)
[GC]::Collect()
If you were to take out just one of the ReleaseComObject statements, the Excel process would remain open. In my code I release all the ones like ranges, tables, etc. first and then I do the worksheet, workbook, and finally the Excel application itself. Then because that only seemed to work like 90% of the time, I added the garbage collection command at the end and finally had a solution that seems to work every time without having to kill the process.
Note: My system is Windows 8.1 with PowerShell v5 and Office 2013.