Remove duplicates in a glossary source column while merging different meanings in target column

Question

I am stuck with finding a Regex to accomplish a task, so please help, I am sure you can rock with a great solution. I build glossaries, and sometimes I get repeated source terms that are exactly the same, but have different targets. See below as an example:

Absolute potential  الجهد المطلق
Absolute potential  جهد مطلق
Absolute potential  جهد مطلقفرق الجهد المطلق بين الفلز والمحلول
**Absolute power    سلطة استبدادية
Absolute power  سلطة مطلقة
Absolute power  قدرة مطلقة**
Absolute power consumption  استهلاك الطاقة الفعلي
Absolute pressure   الضغط المطلق
Absolute prices أسعار مطلقة
Absolute priority   أولوية مطلقة
Absolute priority   الأولوية المُطلقة
Absolute priority   اولوية / اسبقية

Those are tab delimited files. I am looking for a way to look for any repeated Source Term, such as Absolute Power, because it is the same across all 3 lines, and replace with just one entry, just one Absolute Power, while having all the target meanings merged and separated by a pipe character. So the entry would look like this:

**Absolute power    سلطة استبدادية | سلطة مطلقة | قدرة مطلقة**

So I am looking for a Regex to automatically do this task please. So a Term, followed by a Tab character, followed by the merged Arabic entries separated by Pipe Characters, across the whole large glossary text file. That would really make my day. Sincerely, Sam

Answer 1

Here is an example of how you can do it by using a script assuming the input file glossary.txt is UTF8 encoded:

use feature qw(say);
use strict;
use warnings;
use open qw(:std IN :encoding(utf-8) OUT :utf8); 

my $fn = 'glossary.txt';
open ( my $fh, '<', $fn ) or die "Could not open file '$fn': $!";
my %target;
my @order;
while( my $line = <$fh> ) {
    chomp $line;
    my ($subject, $target) = split "\t", $line;
    if (exists $target{$subject}) {
        push @{ $target{$subject} }, $target;
    }
    else {
        $target{$subject} = [ $target ];
        push @order, $subject;
    }
}
close $fh;

for my $subject (@order) {
    say $subject . "\t" . join " | ", @{$target{$subject}};
}