2

How do I use a column value as the database name to JOIN two tables from those two different databases?

I have already successfully joined two tables between two databases with a statically defined (second) database name:

SELECT * 
FROM db1.table_a AS ta
INNER JOIN db2.table_b AS tb on (ta.db_table_name = b.user_id) 

However where db2.table_b is in that query I need to somehow have the db2 instead be a value from the first table in the first database; the table name will be statically defined. All of the kind-of-related threads were totally useless and wildly convoluted.


Details: there is one common database and all of the other databases represent the same application but for different accounts. In order for all of the users on all of the different accounts to be able to interact with each other (e.g. database_2.accounts.user.43 (DB->Table->Column->ID (43)) the common database (db1 above) must not only store the id of the user but also the name of the database that must be joined.

To help visualize things:

  • Database: common
  • Database: db2

SELECT id, database_name 
FROM common.table_a AS ct 
INNER JOIN [database_name].table_b AS dn ON (ct.user_id = [database_name].users.id)

Visually the data returned should look something like this:

+----------+------------+----------+
| database | account_id | username |
+----------+------------+----------+
| db1      | 1          | John     |
+----------+------------+----------+
| db2      | 1          | Sally    |
+----------+------------+----------+
| db3      | 43         | John     |
+----------+------------+----------+
| db4      | 1          | Sally    |
+----------+------------+----------+

Then the HTML output should look something like this:

  • Comment from John from db1.
  • Comment from Sally from db2.
  • Comment from John from db3.
  • Comment from Sally from db4.

I can worry about ensuring visually that John from db1 and John from db3 (and Sally from db2 and Sally from db4) all four of which are different people in real life are represented as so. It's the dynamic aspect of selecting them based on the value of the column's value that contains the database name to be used to JOIN is all that matters.

John
  • 1
  • 13
  • 98
  • 177
  • I don't completely follow, but this sounds like you will need to use some dynamic SQL for this. – Tim Biegeleisen Jan 02 '18 at 14:03
  • @TimBiegeleisen Each database is for it's own application though on occasion all those databases and their respective users utilize a common application. In order to display who they are the name of the database that needs to be joined is stored in the common (`db1` in this case) database so the system will know what the second database that will `JOIN` for each row. – John Jan 02 '18 at 14:05
  • So db1.table_a contains a column which indicates which db table_b lives in. With that kind of design the solution is going to be 'convoluted'. Perhaps you could clarify with some sample data. – P.Salmon Jan 02 '18 at 14:06
  • @P.Salmon Updated. – John Jan 02 '18 at 14:11
  • Create a view. See here for details: https://stackoverflow.com/questions/10694601/creating-view-across-different-databases – Ghigo Jan 02 '18 at 14:24
  • @Ghigo That question asks *how* to create a view, not *what* a view is. It seems that creating a view in MySQL is like setting a function, it's static and does not change. This database JOIN must be dynamic; different users from different databases must be pulled in a single query. – John Jan 02 '18 at 14:28
  • Usually databases are separated for reasons. Otherwise there's a design problem. And your problem looks like there's a mess somewhere. Try to think to another solution: write a webservice for each application with separate database and extract data with it. – Ghigo Jan 02 '18 at 14:37
  • @Ghigo While not exactly a useless "it can't be done" response I can assure you that the overall database structure is exactly as it should be. It is essentially an account neutral application that is accessible for all application accounts. – John Jan 02 '18 at 14:45
  • sql expects you to know the structure of your data (the "s" in "sql"). If you don't, you will not get a nice and clean solution, but need some of the mentioned workarounds (dynamic sql, ...). For a more "sql" solution, you could e.g. design your model to have just one database and a column in every table to identify its owner (it can literally look like the result of your visualized query). You may also have individual userdbs (with basically their private data), and one common database (with known name!) where you would have all shared data that other users need to read (+modify?). – Solarflare Jan 02 '18 at 15:57

2 Answers2

1

Do you have hundreds of databases? That would be a 'bad' design. To discuss further, please explain why you have so many.

If you don't have many databases, but you need to dynamically pick which db, again, poor design; let's discuss further.

If you must do one of those, the hide it in Stored Routines (as P.Salmon almost suggested; his code needs some polishing) or in an application library (PHP, Java, whatever).

Otherwise, wherever you can say table_a, you can replace that with db1.table_a. In fact, you can see MySQl doing that: EXPLAIN EXTENDED SELECT ...; SHOW WARNINGS; Example:

mysql> EXPLAIN EXTENDED SELECT province FROM canada; SHOW WARNINGS;
+----+-------------+--------+-------+---------------+----------+---------+------+------+----------+-------------+
| id | select_type | table  | type  | possible_keys | key      | key_len | ref  | rows | filtered | Extra       |
+----+-------------+--------+-------+---------------+----------+---------+------+------+----------+-------------+
|  1 | SIMPLE      | canada | index | NULL          | province | 105     | NULL | 5484 |   100.00 | Using index |
+----+-------------+--------+-------+---------------+----------+---------+------+------+----------+-------------+
1 row in set, 1 warning (0.01 sec)

+-------+------+---------------------------------------------------------------------------------------+
| Level | Code | Message                                                                               |
+-------+------+---------------------------------------------------------------------------------------+
| Note  | 1003 | /* select#1 */ select `world`.`canada`.`province` AS `province` from `world`.`canada` |
+-------+------+---------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

In this example, the table canada was replaced by world.canada because world was the database.

Rick James
  • 135,179
  • 13
  • 127
  • 222
0

All things being equal (and I bet they aren't) and assuming all schemas/dbs are on the same server you should be able to construct a simple dynamic sql statement.

so given

+----+------+
| id | name |
+----+------+
|  1 | aaa  |
|  2 | bbb  |
+----+------+
2 rows in set (0.00 sec)

Where name serves as proxy for db name we can first select all distinct names and create a sql statement unioning all the tables from all the dbs. Something like this.

SET @SQL =
( 
SELECT  GROUP_CONCAT(GCSTRING)
FROM
(
SELECT 'A' AS GC,CONCAT('SELECT ID,NAME FROM USERS U1 ',JSTRING,' ',DBNAME,' AS  ',NAMEALIAS,'  ON ',NAMEPREFIX,'.',USTRING) GCSTRING
FROM
(
SELECT DISTINCT 'JOIN ' AS JSTRING,NAME DBNAME , 
        NAME AS NAMEALIAS, NAME AS NAMEPREFIX, 'TABLEB.USER_ID = UI.NAME UNION' USTRING 
FROM USERS
) S
) T
GROUP BY GC
)
;

SET @SQL = REPLACE(@SQL,',',' ');
SET @SQL = SUBSTRING(@SQL,1,LENGTH(@SQL) - 5);
SET @SQL = CONCAT(@SQL,';');
SELECT @SQL

+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| @SQL                                                                                                                                                                    |
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| SELECT ID NAME FROM USERS U1 JOIN  aaa AS  aaa  ON aaa.TABLEB.USER_ID = UI.NAME UNION SELECT ID NAME FROM USERS U1 JOIN  bbb AS  bbb  ON bbb.TABLEB.USER_ID = UI.NAME ; |
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)
P.Salmon
  • 17,104
  • 2
  • 12
  • 19
  • Whoa, okay, going over it with the time I have left right now. If I don't get this figured out today I'll be on it first thing tomorrow. – John Jan 02 '18 at 15:49
  • I'm having trouble finding the documentation (even after -mssql and other search flags thrown in) for things like `SET @SQL` and the column and table names are all uppercase in your example. Any chance some insight in to what is going on in the code above please? I've never worked with anything like that. Do I use one or multiple PHP MySQLi `mysqli_query` commands? – John Jan 02 '18 at 16:08
  • Sorry John there was no mention of php in the question and it was not tagged as such. I don't have php at all so I cannot help you there, answer is pure mysql/mariadb, Principal would be the same for php I think. – P.Salmon Jan 02 '18 at 16:16
  • All the databases are on the same server from the same MariaDB service and they're all using the same everything. You don't need to know PHP; my question in that regard is if everything in that giant block of code is a single SQL query or multiple SQL queries? If it's multiple queries then I am guessing the semi-colon is the deliminator? If there are multiple queries then is this something that only stays active during the connection or does it persist like a user-defined function? – John Jan 02 '18 at 16:20
  • Also could you please link to the documentation page for the `SET @SQL =` bit please? I'm having a hell of a time trying to track down the relevant page in the manual and I'm *really* trying here... – John Jan 02 '18 at 16:24
  • Yes multiple queries seperated by ;. No stored program is created - it's just a sql script that you would run on demand ,nothing to stop you pushing this in to a procedure - a case of try it. – P.Salmon Jan 02 '18 at 16:24
  • https://dev.mysql.com/doc/refman/5.7/en/user-variables.html , https://stackoverflow.com/questions/1009954/mysql-variable-vs-variable-whats-the-difference you might also need this https://dev.mysql.com/doc/refman/5.7/en/sql-syntax-prepared-statements.html – P.Salmon Jan 02 '18 at 16:25
  • @John - "mssql" is a different product. – Rick James Jan 02 '18 at 16:33
  • 1
    @P.Salmon - I think you need to do prepare+execute+deallocate, not simply `SELECT @sql`. – Rick James Jan 02 '18 at 16:34
  • @Rick James absolutely I have given the op a link to the documentation. – P.Salmon Jan 02 '18 at 16:37
  • @John - You should not try to execute multiple queries in "one giant block". Instead, learn about stored routines. – Rick James Jan 02 '18 at 16:38