Whats the best way to retrieve array data from MySql

Question

I'm storing a object / data structure like this inside a MySql (actually a MariaDb) database:

{
  idx: 7,
  a: "content A",
  b: "content B",
  c: ["entry c1", "entry c2", "entry c3"]
}

And to store it I'm using 2 tables, very similar to the method described in this answer: https://stackoverflow.com/a/17371729/3958875

i.e.

Table 1:

+-----+---+---+
| idx | a | b |
+-----+---+---+

Table 2:

+------------+-------+
| owning_obj | entry |
+------------+-------+

And then made a view that joins them together, so I get this:

+-----+------------+------------+-----------+
| idx | a          | b          | c         |
+-----+------------+------------+-----------+
|   7 | content A1 | content B1 | entry c11 |
|   7 | content A1 | content B1 | entry c21 |
|   7 | content A1 | content B1 | entry c31 |
|   8 | content A2 | content B2 | entry c12 |
|   8 | content A2 | content B2 | entry c22 |
|   8 | content A2 | content B2 | entry c32 |
+-----+------------+------------+-----------+

My question is what is the best way I can get it back to my object form? (e.g. I want an array of the object type specified above of all entries with idx between 5 and 20)

There are 2 ways I can think of, but both seem to be not very efficient.

Firstly we can just send this whole table back to the server, and it can make a hashmap with the keys being the primary key or some other unique index, and collect up the different c columns, and rebuild it that way, but that means it has to send a lot of duplicate data, and take a bit more memory and processing time to rebuild on the server. This method also won't be very pleasant to scale if we have multiple arrays, or have arrays within arrays.

Second method would be to do multiple queries, filter Table 1 and get back the list of idx's you want, and then for each idx, send a query for Table 2 where owning_obj = current idx. This would mean sending a whole lot more queries.

Neither of these options seems very good, so I'm wondering if there is a better way. Currently I'm thinking it can be something that utilizes JSON_OBJECT(), but I'm not sure how.

This seems like a common situation, but I can't seem to find the exact wording to search for to get the answer.

PS: The server interfacing with MySql/MariaDb is written in Rust, don't think this is relevant in this question though

If it was PHP I'd do something like `SELECT idx, a, b, GROUP_CONCAT(c) FROM view GROUP BY idx` and then `$obj = $result->fetch_object(); $obj->c = explode($obj->c);` — Nick, Dec 03 '18 at 08:14
@Barmar I already addressed the duplicate this in the question, I am asking whether there is a better way than retrieving a massive amount of duplicate data before using hashmap to seperate it out. — 8176135, Dec 03 '18 at 08:23

Barmar · Accepted Answer · 2018-12-03T08:34:19.357

You can use GROUP_CONCAT to combine all the c values into a comma-separated string.

SELECT t1.idx, t1.a, t1.b, GROUP_CONCAT(entry) AS c
FROM table1 AS t1
LEFT JOIN table2 AS t2 ON t1.idx = t2.owning_obj
GROUP BY t1.idx

Then explode the string in PHP:

$result_array = [];
while ($row = $result->fetch_assoc()) {
    $row['c'] = explode(',', $row['c']);
    $result_array[] = $row;
}

However, if the entries can be long, make sure you increase group_concat_max_len.

If you're using MySQL 8.0 you can also use JSON_ARRAYAGG(). This will create a JSON array of the entry values, which you can convert to a PHP array using json_decode(). This is a little safer, since GROUP_CONCAT() will mess up if any of the values contain comma. You can change the separator, but you need a separator that will never be in any values. Unfortunately, this isn't in MariaDB.

Whats the best way to retrieve array data from MySql

1 Answers1