Sorting with many to many relationship

Question

I have a 3 tables person, person_speaks_language and language.

• person has 80 records
• language has 2 records

I have the following records

• the first 10 persons speaks one language
• the first 70 persons (include the first group) speaks 2 languages
• the last 10 persons dont speaks any language

Following with the example I want sort the persons by language, How I can do it correctly.

I'm trying to use the the following SQL but seems quite strange

SELECT "person".*
FROM "person"
  LEFT JOIN "person_speaks_language" ON "person"."id" = "person_speaks_language"."person_id"
  LEFT JOIN "language" ON "person_speaks_language"."language_id" = "language"."id"
ORDER BY "language"."name"
  ASC

dataset

71,Catherine,Porter,male,NULL
72,Isabelle,Sharp,male,NULL
73,Scott,Chandler,male,NULL
74,Jean,Graham,male,NULL
75,Marc,Kennedy,male,NULL
76,Marion,Weaver,male,NULL
77,Melvin,Fitzgerald,male,NULL
78,Catherine,Guerrero,male,NULL
79,Linnie,Strickland,male,NULL
80,Ann,Henderson,male,NULL
11,Daniel,Boyd,female,English
12,Ora,Beck,female,English
13,Hulda,Lloyd,female,English
14,Jessie,McBride,female,English
15,Marguerite,Andrews,female,English
16,Maurice,Hamilton,female,English
17,Cecilia,Rhodes,female,English
18,Owen,Powers,female,English
19,Ivan,Butler,female,English
20,Rose,Bishop,female,English
21,Franklin,Mann,female,English
22,Martha,Hogan,female,English
23,Francis,Oliver,female,English
24,Catherine,Carlson,female,English
25,Rose,Sanchez,female,English
26,Danny,Bryant,female,English
27,Jim,Christensen,female,English
28,Eric,Banks,female,English
29,Tony,Dennis,female,English
30,Roy,Hoffman,female,English
31,Edgar,Hunter,female,English
32,Matilda,Gordon,female,English
33,Randall,Cruz,female,English
34,Allen,Brewer,female,English
35,Iva,Pittman,female,English
36,Garrett,Holland,female,English
37,Johnny,Russell,female,English
38,Nina,Richards,female,English
39,Mary,Ballard,female,English
40,Adrian,Sparks,female,English
41,Evelyn,Santos,female,English
42,Bess,Jackson,female,English
43,Nicholas,Love,female,English
44,Fred,Perkins,female,English
45,Cynthia,Dunn,female,English
46,Alan,Lamb,female,English
47,Ricardo,Sims,female,English
48,Rosie,Rogers,female,English
49,Susan,Sutton,female,English
50,Mary,Boone,female,English
51,Francis,Marshall,male,English
52,Carl,Olson,male,English
53,Mario,Becker,male,English
54,May,Hunt,male,English
55,Sophie,Neal,male,English
56,Frederick,Houston,male,English
57,Edwin,Allison,male,English
58,Florence,Wheeler,male,English
59,Julia,Rogers,male,English
60,Janie,Morgan,male,English
61,Louis,Hubbard,male,English
62,Lida,Wolfe,male,English
63,Alfred,Summers,male,English
64,Lina,Shaw,male,English
65,Landon,Carroll,male,English
66,Lilly,Harper,male,English
67,Lela,Gordon,male,English
68,Nina,Perry,male,English
69,Dean,Perez,male,English
70,Bertie,Hill,male,English
1,Nelle,Gill,female,Spanish
2,Lula,Wright,female,Spanish
3,Anthony,Jensen,female,Spanish
4,Rodney,Alvarez,female,Spanish
5,Scott,Holmes,female,Spanish
6,Daisy,Aguilar,female,Spanish
7,Elijah,Olson,female,Spanish
8,Alma,Henderson,female,Spanish
9,Willie,Barrett,female,Spanish
10,Ada,Huff,female,Spanish
11,Daniel,Boyd,female,Spanish
12,Ora,Beck,female,Spanish
13,Hulda,Lloyd,female,Spanish
14,Jessie,McBride,female,Spanish
15,Marguerite,Andrews,female,Spanish
16,Maurice,Hamilton,female,Spanish
17,Cecilia,Rhodes,female,Spanish
18,Owen,Powers,female,Spanish
19,Ivan,Butler,female,Spanish
20,Rose,Bishop,female,Spanish
21,Franklin,Mann,female,Spanish
22,Martha,Hogan,female,Spanish
23,Francis,Oliver,female,Spanish
24,Catherine,Carlson,female,Spanish
25,Rose,Sanchez,female,Spanish
26,Danny,Bryant,female,Spanish
27,Jim,Christensen,female,Spanish
28,Eric,Banks,female,Spanish
29,Tony,Dennis,female,Spanish
30,Roy,Hoffman,female,Spanish
31,Edgar,Hunter,female,Spanish
32,Matilda,Gordon,female,Spanish
33,Randall,Cruz,female,Spanish
34,Allen,Brewer,female,Spanish
35,Iva,Pittman,female,Spanish
36,Garrett,Holland,female,Spanish
37,Johnny,Russell,female,Spanish
38,Nina,Richards,female,Spanish
39,Mary,Ballard,female,Spanish
40,Adrian,Sparks,female,Spanish
41,Evelyn,Santos,female,Spanish
42,Bess,Jackson,female,Spanish
43,Nicholas,Love,female,Spanish
44,Fred,Perkins,female,Spanish
45,Cynthia,Dunn,female,Spanish
46,Alan,Lamb,female,Spanish
47,Ricardo,Sims,female,Spanish
48,Rosie,Rogers,female,Spanish
49,Susan,Sutton,female,Spanish
50,Mary,Boone,female,Spanish
51,Francis,Marshall,male,Spanish
52,Carl,Olson,male,Spanish
53,Mario,Becker,male,Spanish
54,May,Hunt,male,Spanish
55,Sophie,Neal,male,Spanish
56,Frederick,Houston,male,Spanish
57,Edwin,Allison,male,Spanish
58,Florence,Wheeler,male,Spanish
59,Julia,Rogers,male,Spanish
60,Janie,Morgan,male,Spanish
61,Louis,Hubbard,male,Spanish
62,Lida,Wolfe,male,Spanish
63,Alfred,Summers,male,Spanish
64,Lina,Shaw,male,Spanish
65,Landon,Carroll,male,Spanish
66,Lilly,Harper,male,Spanish
67,Lela,Gordon,male,Spanish
68,Nina,Perry,male,Spanish
69,Dean,Perez,male,Spanish
70,Bertie,Hill,male,Spanish

Update

the expect results are: each person must be appears only one time using the language order

For explain the case further, I'll take a new and small dataset, using only the person id and the language name

1,English
2,English
3,English
4,English
19,English
1,Spanish
2,Spanish
3,Spanish
4,Spanish
5,Spanish
14,Spanish
15,Spanish
16,Spanish
19,Spanish
21,Spanish
25,Spanish

I'm using the same order but if I use a limit for example LIMIT 8 the results will be

1,English
2,English
3,English
4,English
19,English
1,Spanish
2,Spanish
3,Spanish

And the expected result is

1,English
2,English
3,English
4,English
19,English
5,Spanish
14,Spanish
15,Spanish

What I'm trying to do

What I'm trying to do is sorting, paginating and filtering a list of X that may have a many-to-many relationship with Y, in this case X is a person and Y is the language. I need do it in a general way. I found a trouble if I want ordering the list by some Y properties.

The list will show in this way:

firstname, lastname, gender  , languages
Daniel   , Boyd    , female  , English Spanish
Ora      , Beck    , female  , English
Anthony  , Jensen  , female  , Spanish
....

I only need return a array with the IDs in the correct order

this is the main reason I need that the results only appears the person one time is because the ORM (that I'm using) try to hydrate each result and if I paginate the results using offset and limit. the results maybe aren't the expected. I'm doing assumptions many to many relationships

I can't use the string_agg or group_concat because I dont know the real data, I dont know if are integers or strings

Answer 1

If you want each person to appear only once, then you need to aggregate by that person. If you then want the list of languages, you need to combine them in some way, concatenation comes to mind.

The use of double quotes suggests Postgres or Oracle to me. Here is Postgres syntax for this:

SELECT p.id, string_agg(l.name) as languages
FROM person p LEFT JOIN 
     person_speaks_language psl
     ON p.id = psl.person_id LEFT JOIN
     language l
     ON psl.language_id = l.id
GROUP BY p.id
ORDER BY COUNT(l.name) DESC, languages;

Similar functionality to string_agg() exists in most databases.

Answer 2

There is nothing wrong with Bertie Hill appearing in two rows, with one language each, that is the Tabular View of Data per the Relational Model. There are no dependencies on data values or number of data values. It is completely correct and un-confused.

But here, the requirement is confused, because you really want three separate lists:

• speaks one language
• speaks two languages [or the number of languages currently in the language file]
• speaks no language [on file] ) ...

But you want those three lists in one list.

Concatenating data values is never, ever a good idea. It is a breach of rudimentary standards, specifically 1NF. It may be common, but it is a gross error. It may be taught by the so-called "theoreticians", but it remains a gross error. Even in a result set, yes.

1. It creates confusion, such as I have detailed at the top.

2. With concatenated strings, as the number of languages changes, the width of that concatenated field will grow, and eventually exceed space, wherever it appears (eg. the width of the field on the screen).

Just two of the many reasons why it is incorrect, not expandable, sub-standard.

By the way, in your "dataset" (it isn't the result set produced by your code), the sexes appear to be nicely mixed up.

Therefore the answer, and the only correct one, even if it isn't popular, is that your code is correct (it can be cleaned it up, sure), and you have to educate the user re the dangers of sub-standard code or reports.

• You can sort by person.name (rather than by language.name) and then write smarter SQL such that (eg) the person.name is not repeated on the second and subsequent row for persons who speak more than one language, etc. That is just pretty printing.

The non-answer, for those who insist on sub-standard code that will break one day when, is Gordon's response.

Response to Comments

In the Relational Model:

• There is no order to the rows, that is deemed a physical or implementation aspect, which we have no control over, and which changes anyway, and which we are warned not to rely upon. If order is sought in the output result set, then we must us ORDER BY, that is its purpose in life.

• The data has meaning, and that meaning is carried in Relational Keys. Meaning cannot be carried in surrogates (ie. ID columns).

Limiting myself to the files (they are not tables) that you have given, there is no such thing in the data as:

• the first 10 persons who speaks one language

Obtaining persons who speak one language is simple, I believe you already understand that:

  SELECT  person.first_name,
          person.last_name
      FROM person P,
      (SELECT person_id
          FROM person_speaks_language
          GROUP BY person_id
          HAVING COUNT(*) = 1          -- change this for 2 languages, etc
          ) AS PL
      WHERE P.person_id = PL.person_id

But "first" ? "first" by what criteria ? Record creation date ?

      ORDER BY date_created            -- if it exists in the data

Record ID does not give first anything: as records are added and deleted, any "order" that may exist initially is completely lost.

You cannot extract meaning out of, or assign meaning to something that, by definition, has no meaning. If the Record ID is relevant, ie. you are going to use it for some purpose, then it is not a Record ID, name the field for what it actually is.

I fail to see, I do not understand, the relevance of the difference between the "dataset" and the updated "small dataset". The "dataset" size is irrelevant, the field headings are irrelevant, what the result set means, is relevant.

The problem is not some "limitation" in the Relational Model, the problem is (a) your fixed view of data values, and (b) your lack of understanding about what the Relational Model is, what it does, understanding of which makes this whole question disappear, and we are left with a simple SQL (as tagged) "how to" question. Eg. If I had a Relational Database, with persons and languages, with no ID columns, there is nothing that I cannot do with it, no report that I cannot produce from it, from the data.

Please try to use an example that conveys the meaning in the data, in what you are trying to do.

the expect results are: each person must be appear only one time

They already appear only once (for each language)

using the language order

Well, there is no order in the language file. We can give it some order, whatever order is meaning-ful, to you, in the result set, based on the data. Eg. language.name. Of course, many persons speak each language, so what order would you like within language.name? How about last_name, first_name. The Record IDs are meaningless to the user, so I won't display them in the result set. NULL is also meaningless, and ambiguous, so I will make the meaning here explicit. This is pretty much what you have, tidied up:

    SELECT  [language] = CASE name
                WHEN NULL THEN "[None]"
                ELSE name
                END, 
            last_name, 
            first_name
        FROM person P
            LEFT JOIN person_speaks_language PL
                ON P.id = PL.person_id
            LEFT JOIN language L
                ON PL.language_id = L.id
        ORDER BY name, 
                 last_name, 
                 first_name

But then you have:

And the expected result is

The example data of which contradicts your textual descriptions:

the expect results are: each person must be appear only one time using the language order

So now, if I ignore the text, and examine the example data re what you want

• (which is a horrible thing to do, because I am joining you in the incorrect activity of focussing on the data values, rather than understanding the meaning),

it appears you want the person to appear only once, full stop, regardless of how many languages they speak. Your example data is meaningless, so I cannot be asked to reproduce it. See if this has some meaning.

    SELECT  last_name,
            first_name,
            [language] = (                   -- correlated subquery
        SELECT TOP 1                         -- get the "first" language
            CASE name                        -- make meaning of null explicit
                WHEN NULL THEN "[None]"
                ELSE name
                END
            FROM person_speaks_language PL
                JOIN language L
                    ON PL.language_id = L.id
            WHERE P.id = PL.person_id        -- the subject person
            ORDER BY name                    -- id would be meaningless
            )
        FROM person P                        -- vector for person, once
        ORDER BY last_name,
                 first_name

Now if you wanted only persons who speak a language (on file):

    SELECT  last_name,
            first_name,
            [language] = (                     -- correlated subquery
        SELECT TOP 1                           -- get the "first" language
                name
            FROM person_speaks_language PL
                JOIN language L
                    ON PL.language_id = L.id
            WHERE P.id = PL.person_id          -- the subject person
            ORDER BY name                      -- id would be meaningless
            )
        FROM person P,
            (
            SELECT  DISTINCT person_id         -- just one occ, thanks
                FROM person_speaks_language PL -- vector for speakers
                ) AS PL_1
        WHERE P.id = PL_1.person_id            -- join them to person fields

There, not an outer join anywhere to be seen, in either solution. LEFT or RIGHT will confuse you. Do not attempt to "get everything", so that you can "see" the data values, and then mangle, hack and chop away at the result set, in order to get what you want from that. No, forget about the data values and get only what you want from the record filing system.

Response to Update

I was trying to explain the case with a data set, I think I made things tougher than they actually were

Yes, you did. Reviewing the update then ...

1. The short answer is, get rid of the ORM. There is nothing in it of value:

   • you can access the RDB from the queries that populate your objects directly. The way we did for decades before the flatulent beast came along. Especially if you understand and implement Open Architecture Standards.

   • Further, as evidenced, it creates masses of problems. Here, you are trying to work around the insane restrictions of the ORM.

   • Pagination is a straight-forward issue, if you have your data Normalised, and Relational Keys.

2. The long answer is ... please read this Answer. I trust you will understand that the approach you take to designing your app components, your design of windows, will change. All your queries will be simplified, you get only what you require for the specific window or object.

   The problem may well disappear entirely (except for possibly the pagination, you might need a method).

Then please think about those architectural issues carefully, and make specific comments of questions.