Since VSCode 1.72 you can right-click an icon in the tab bar and choose to hide it from the context menu:

If you like the tab bar buttons in general and just dislike their position (because it makes editor tabs jump around), you can move them down to the breadcrumb bar, where they are smaller and do not take up that much space:

This is done using the Customize UI extension with the following configuration:
"customizeUI.stylesheet": {
".tabs-and-actions-container": {
"background-color": "inherit",
},
".tabs-and-actions-container .editor-actions": {
"position": "absolute",
"top": "100%",
"right": "0px",
"height": "22px !important",
"z-index": "1",
"background-color": "inherit",
},
".tabs-and-actions-container .editor-actions .action-item": {
"margin-right": "3px !important",
},
".tabs-and-actions-container .editor-actions .action-item a": {
"font-size": "13px",
},
".tabs-and-actions-container .editor-actions .action-item .codicon": {
"width": "13px",
"height": "13px",
},
".tabs-and-actions-container .tab:last-child": {
"margin-right": "0 !important",
},
".title.tabs.show-file-icons": {
"overflow": "unset !important",
},
}
This solution is theme-independent, so it should work for all colour combinations. The background color for the buttons is always the same as the background color of the tab bar. If you use only one static theme, you could hard-code the background-color
for the .tabs-and-actions-container .editor-actions
selector to the exact color of the breadcrumb bar for a more seamless design. However, this does not work when switching themes.
The only drawback to this solution is that the buttons overflow the rightmost breadcrumb information, but I'm fine with that.