List folder contents and only show unique value

Question

Is there a way to use the 'ls' command then then pipe the output to only show the unique item?

Here is the example:

ls /dev/disk/by-id


ata-ST500DM002-1BD142_S2AFE0JP
ata-ST500DM002-1BD142_W2AEDMQK
ata-ST500DM002-1BD142_W2AEDMQK-part1
ata-ST500DM002-1BD142_W2AEDMQK-part2
ata-ST500DM002-1BD142_W2AEDMQK-part3
ata-WDC_WD5000AAKX-60U6AA0_WD-WCC2EEV65804
ata-WDC_WD5000AAKX-60U6AA0_WD-WCC2EEV65804-part1
ata-WDC_WD5000AAKX-60U6AA0_WD-WCC2EEV65804-part9
ata-WDC_WD5000AAKX-60U6AA0_WD-WCC2EKK60289
ata-WDC_WD5000AAKX-60U6AA0_WD-WCC2EKK60289-part1
ata-WDC_WD5000AAKX-60U6AA0_WD-WCC2EKK60289-part9
ata-WDC_WD5000AAKX-60U6AA0_WD-WCC2ET092491
ata-WDC_WD5000AAKX-60U6AA0_WD-WCC2ET092491-part1
ata-WDC_WD5000AAKX-60U6AA0_WD-WCC2ET092491-part9
ata-WDC_WD5003AZEX-00MK2A0_WD-WCC3F2HAD1XC
ata-WDC_WD5003AZEX-00MK2A0_WD-WCC3F2HAD1XC-part1
ata-WDC_WD5003AZEX-00MK2A0_WD-WCC3F2HAD1XC-part9

As you can see in the output all but one of the items has the serial number and a -part# to it, but the serial ontop:

'ata-ST500DM002-1BD142_S2AFE0JP'

Does not. What I am trying to do is get the output to only show me the resits that do not have any other duplicate serial entries.The out out would be just the unique serial number.

ata-ST500DM002-1BD142_S2AFE0JP

Thank you.

Does this answer your question? [How to print only the unique lines in BASH?](https://stackoverflow.com/questions/23740545/how-to-print-only-the-unique-lines-in-bash) — Daemon Painter, Oct 19 '20 at 17:38
btw: why this is not matching your parameters? ata-ST500DM002-1BD142_W2AEDMQK — Daemon Painter, Oct 19 '20 at 17:39
You can collect names in an array using a glob like `files=( /dev/disk/by-id/* )`. You can modify the contents of an array using parameter expansion, like `serials=( "${files[@]#ata-}" ) serials=( "${files[@]%-part*}" )`. You can output such an array using `printf '%s\n' "${serials[@]}"`. You can make a list unique using `sort -u`. — kojiro, Oct 19 '20 at 17:40
Please note: [Why *not* parse `ls`?](http://unix.stackexchange.com/questions/128985/why-not-parse-ls) — Cyrus, Oct 19 '20 at 17:40
There are other lines in this output that do not have `part` in them. What **precisely** is different about the first line vs the others? — dawg, Oct 19 '20 at 17:41
What for example is the difference between `ata-ST500DM002-1BD142_S2AFE0JP` and `ata-ST500DM002-1BD142_W2AEDMQK`? — dawg, Oct 19 '20 at 17:43
@dawg the different is that ata-ST500DM002-1BD142_W2AEDMQK, has the -part# after it, I am trying to only output any serual that does not have the part# — Vlad, Oct 19 '20 at 17:45

score 1 · Accepted Answer · answered Oct 19 '20 at 17:52

1

Assuming these entries don't contain newlines, you may use:

printf '%s\n' /dev/disk/by-id/* |
awk '!/-part[0-9]*$/{arr[$0]; next} { sub(/-[^-]+$/, ""); delete arr[$0] } 
END { for (i in arr) print i }'

ata-ST500DM002-1BD142_S2AFE0JP

answered Oct 19 '20 at 17:52

anubhava

761,203
64
569
643

Rachid K. · Answer 2 · 2020-10-19T17:54:16.137

0

Use grep with a regular expression : ls | grep -Ev 'part.*$'

$ ls test
ata-WDC_WD5000AAKX-60U6AA0_WD-WCC2EEV65804        ata-WDC_WD5000AAKX-60U6AA0_WD-WCC2EEV65804-part9  ata-WDC_WD5000AAKX-60U6AA0_WD-WCC2ET092491-part1
ata-WDC_WD5000AAKX-60U6AA0_WD-WCC2EEV65804-part1  ata-WDC_WD5000AAKX-60U6AA0_WD-WCC2ET092491        ata-WDC_WD5000AAKX-60U6AA0_WD-WCC2ET092491-part9
$ ls test | grep -Ev 'part.*$'
ata-WDC_WD5000AAKX-60U6AA0_WD-WCC2EEV65804
ata-WDC_WD5000AAKX-60U6AA0_WD-WCC2ET092491

edited Oct 19 '20 at 17:54

answered Oct 19 '20 at 17:42

Rachid K.

4,490
3
11
30

Getting this error: grep: invalid option -- 'p' – Vlad Oct 19 '20 at 17:45
I fixed the answer as the "-" was considered as an option. – Rachid K. Oct 19 '20 at 17:48
Not working, I still get the serial numbers, (serial)- - part# – Vlad Oct 19 '20 at 17:50
Don't forget the dot right before the *. I just fixed it. – Rachid K. Oct 19 '20 at 17:51
It is close, but also if the serial had an -part# associated with it, to eliminate that as well from the output – Vlad Oct 19 '20 at 17:54
2

Generally [don't use `ls` in scripts.](http://mywiki.wooledge.org/ParsingLs) – tripleee Oct 19 '20 at 17:55
@tripleee, if not ls,then what command to output the contents? – Vlad Oct 19 '20 at 17:57
Read the link; that's why I provided it. – tripleee Oct 19 '20 at 17:57

List folder contents and only show unique value

2 Answers2