If you code it properly, yes it could be WCAG conformant.
It's a little tricky because if the hidden columns are truly hidden, then a screen reader won't know there are extra columns. When they navigate to the table, they'll hear "table with 5 columns and X rows" but the table might really have 20 columns. So you might need some invisible text just for the screen reader that tells the user that there are really more columns than currently displayed. Perhaps use the <caption>
element with a class of "sr-only".
<table>
<caption class="sr-only">This table has X columns but Y of them are hidden. Select the "Show more columns" button to unhide the columns</caption>
The "sr-only" class is not a specially defined class but it's a common name to use for a class that creates text that is visibly hidden but still available to screen readers. See What is sr-only in Bootstrap 3?. You can call the class whatever you want.
The '...' buttons would need to have an appropriate screen reader label (accessible name). Visually, you can show '...' but it'll need something more descriptive for the screen reader user.
<button aria-label="Show more columns">...</button>
Does the '...' button unhide all the hidden columns or just some of them? If only some of them, then the user has to click on '...' again to unhide some more? If so, then the label for '...' would have to indicate that.
When the columns are unhidden, that makes the table very wide so you'll have a horizontal scrollbar on the table?