Replace each file path in a file with the path following symbolic links

Question

I am working with a Jenkins build of a project using Typescript. It uses "jest" to generate a "lcov.info" file containing code coverage data. The workspace has some symbolic links. The resulting paths in the "lcov.info" file represent the "source" of the link, not the destination. This should be fine, assuming everything in the toolchain understands symbolic links.

Unfortunately, I'm also working with a somewhat older version of SonarQube, which is using a somewhat older version of the SonarTS plugin, which does NOT understand symbolic links, so all the paths are not found.

I have no idea whether the bug in SonarTS has been fixed in a future version. Waiting for an upgrade is not possible.

I had written a simple-minded sed script for this before, when an assumption that I had made about the actual absolute path following the symlink was correct. That assumption is now not correct, so I have to build a more sophisticated bash command line in the Jenkins pipeline script.

My first attempt looks something like this:

sh "cat coverage/lcov.info | sed -e 's/^SF:(.*)/SF:`ls -L \\1`/g' > /tmp/lcov.info"

My intention was to replace the file path with the "ls -L" result, which is following the symbolic link. This does not work, because the ... script gets the string "\1", not what I want. I can understand why this is, because the backtick expression is probably built before the regexp is built.

I'd prefer to not build a multi-line script and store it in the build, but compose it inline.

I'm guessing that it might be more practical to do this with awk, but it's been a while since I've awk-ed something non-trivial.

Update:

Although I've found a gnarly bash solution for this, I find that perl solution attractive, because it is simpler. However, I can't actually get it to work.

I created a directory called "dir1", and put an empty file in there called "stuff". I created a symlink named "dir2" that points to "dir1". I created a file named "junk1" with the following contents:

SF:dir2/stuff
SF:dir2/stuff

Also note:

% ls -lt dir2
lrwxrwxrwx 1 dk068x dk068x 4 Dec 16 13:22 dir2 -> dir1/

I then ran the following and got the indicated results:

% cat junk1 | perl -pe 's/^(SF:)(.*)/ $1 . readlink $2 /ge'
SF:
SF:

I tested this on both Cygwin and an Ubuntu VM.

Answer 1

haven't had time with a thorough answer yet, but i'd imagine something like using mawk/gawk to open to the file, the

OFS = FS = "^SF:"

create a composite command with the name, run it

while (cmd | getline ) { } close(cmd)

the command being print those original link names, residing in $2 now , pipe to

gnu parallel -N 1 -j 8 --timeout 1800 -k readlink -f 

the -k to ensure output order is identical to that of input. readpath also works. then substitute the output of command's NR to your input file's NR. i set a large timeout as a worst case scenario capture.

Much better done in batch, so if the file not too gigantic, read it all in first, and do all this in END { } , and print out results to the file you've mentioned in /tmp folder. The POSIX shell isn't explicitly called but it's done implicitly via awk. i can hash it out to code later.

Answer 2

If Perl is your option, would you please try:

perl -pe 's/^(SF:)(.*)/ $1 . readlink $2 /ge' < coverage/lcov.info > /tmp/lcov.info

• The readlink function returns the value of a symbolic link.
• The e option to the s/pattern/replacement/ operator enables the replacement to be an expression of Perl.

[Update]

Now the script analyze the path described in the info file to find a symlink substring recursively.

#!/bin/bash

perl -lne '
    use strict;
    use warnings;

    my $col1;                   # 1st column of the info file
    my $col2;                   # 2nd column of the info file
    my $real_path;              # resolved path (final)
    my $link_substr;            # substring of $path which is a symlink
    my $link_result;            # output of readlink

    # return substring which is a symlink
    sub checklink() {
        my $path = shift;
        while (1) {
            return $path if -l $path;   # $path is a symlink
            return "" if $path !~ m#/#; # return "" if $path contains no symlink
            $path =~ s#/[^/]*$##;       # remove the rightmost "/" and following substring
        }
    }
    if (/^(SF:)(.*)/) {
        $col1 = $1; $col2 = $2;
        $link_substr = &checklink($col2);
        if ($link_substr ne "") {
            $link_result = readlink($link_substr);
            $link_result =~ s#/$##;     # remove trailing "/" if any
            $real_path = $link_result . substr($col2, length($link_substr));
        } else {
            $real_path = $col2;
        }
        print $col1, $real_path;
    }
' coverage/lcov.info > /tmp/lcov.info