I've reproduced your problem in my side. Here's my experience.
After assigning powerapp license(I use O365 E5 here) to guest account and share the app to this account, I can visit the app but can't see the data in the table. I assume that it resulted from the connector hasn't been shared, but it's true that this connector can't be shared because of no 'share button'.
Then I tried to add access policy to my guest account with these two sql:
CREATE USER [tiny-wa_outlook.com#EXT#@xx.onmicrosoft.com] FROM EXTERNAL PROVIDER;
GRANT SELECT ON dbo.tinyTest TO [tiny-wa_outlook.com#EXT#@xx.onmicrosoft.com];
Pls note here, I used the User Principal Name here(can find the principal name in azure ad->users), I also tested to use 'tiny-wa@outlook.com' in the sql but after executing successfully, it didn't work.
So I think you can try to use the sql above to add your guests accounts and make them can access the powerapp.
Here's some related docs:
create contained users mapped to azure ad identities
Share app resources
add table permission to user
==========================UPDATE==========================
Share my experience of this troubleshot.
First I need to create a power app but I found that after creating the connector with sql server azure ad authentication, it can't connect to the sql server, the error message is like 'Credentials are missing or not valid. inner exception: The credentials provided for the SQL source are invalid', solution is add my admin account as sql server instance Active Directory Admin.
Then I choose a table and successfully create a sample app. With the creating account, I can visit the app but other accounts can't. Here needs to share the app and it's related connectors to other users. But other accounts still can't reach the app because of no license. Because sql server connector is premium connector, so I assign O365 E3 license here. I met an error when assign license, the user's 'Usage location' can't be null or it can't assign license in M365 admin center.
Then I met similar error with Op, the difference is that both member account and guest account can't see the data in app. I try to find the way to share the connector to these uses but failed, I haven't made sure if those connectors without share button can be shared to others. So I have no options to study if this kind of connectors are authenticated in other ways so they don't need to be shared at all.
Next actions is using the account which used to create the sql server and database to sign in database and execute the sqls above.

Then the member account can see data in the power app while the guest account can't see. The reason is I used xx@outlook.com as the parameter in the sql, when I used xx_outlook.com#EXT#@xx.onmicrosoft.com, it worked finally.
Hope this can also help you.
===================================================
For creating my demo app: First, I'm sure my environment isn't a sandbox(the environment in the screenshot below). And I think it's easy to create a demo app, and my app is simple, just choose to create an app from data and then select sql server as the connector, next I choose auzre ad auth and click the connector, enter server name and database name then choose a table, after that my app has created. That table has one row of data so when I signed in the app with the creating account, I see it in the screen while other accounts(member or guests) can't.
My sql server instance and database are created long time ago, but I'm sure I followed this tutorial to create them.
