For each line L of a file file1.txt, remove the line of the file composer.json that contains L

Question

I have a file file1.txt that contains almost forty lines, each line contains a package name. I have another file composer.json that contains hundreds of packages. I need to remove all packages of my file1.txt from composer.json

file1.txt

remote5/package40
remote1/package1
remote4/package3
...
remote1/package2

Current composer.json

{
    "name": "name/project_name",
    "description": "sf project blabla",
    ...
    "config": {
        "installer-paths: {
            "remote34/package345",
            "remote21/package345",
            "remote123/package345",
            "remote384/package345",
            "remote444/package345",
            ...
            "remote1/package1",
            "remote1/package2",
            "remote4/package3",
            ...
            "remote5/package40",
         }
     }
}

As you can see, the files are different, so I can't use this solution Remove Lines from File which appear in another File

Wanted composer.json

{
    "name": "name/project_name",
    "description": "sf project blabla",
    ...
    "config": {
        "installer-paths: {
            "remote34/package345",
            "remote21/package345",
            "remote123/package345",
            "remote384/package345",
            "remote444/package345",
            ...
         }
     }
}

Here is the command I've been trying so far

for i in `cat file1.txt`; do echo $i && sed -i '/$i/d' composer.json; done

This doesn't change composer.json. However the command below works

sed -i '/remote4\/package3/d' composer.json

UPDATE

grep -Fvxf file1.txt composer.json

The output is exactly composer.json's content

Possible duplicate of [Remove Lines from File which appear in another File](https://stackoverflow.com/questions/4366533/remove-lines-from-file-which-appear-in-another-file) — Sundeep, Oct 25 '17 at 15:44
you can still re-purpose on of the answers(https://stackoverflow.com/a/32267899/4082052) to `grep -vFf file1.txt composer.json` — Sundeep, Oct 25 '17 at 16:02
Omit the `x` from the `grep -Fvf file1.txt compser.json` - it means exact match. Your JSON lines have leading blanks and quotes around the values, so they aren't exact matches. The `-F` means 'no metacharacters' (which means a different form of 'exact match', and the one you want). — Jonathan Leffler, Oct 25 '17 at 16:17
`"installer-paths` - the right double quote is missed. Also this `{ "remote34/package345", "remote21/package345", "remote123/package345", "remote384/package345", "remote444/package345", ... }` is invalid JSON notation. Update your `composer.json` to make it valid JSON — RomanPerekhrest, Oct 25 '17 at 16:34
@smarber, actually, I'm already observing the expected result on my local machine. It only remains for you to provide a proper/valid `composer.json` — RomanPerekhrest, Oct 25 '17 at 17:01
in `grep` the use of `-x` is looking for exact line match, which you don't have due to various formatting differences... — karakfa, Oct 25 '17 at 18:32

score 1 · Accepted Answer · answered Oct 25 '17 at 18:31

1

with grep

grep -Fvf <(sed 's/.*/"&",/' file1.txt) composer.json

decorate your search keys with the json format and look for fixed string match.

answered Oct 25 '17 at 18:31

karakfa

66,216
7
41
56

This works like a charm! However, could you explain the **<(....)** or provide a reference because I'm not familiar with ? – smarber Oct 26 '17 at 08:30
It's called process substitution. – karakfa Oct 26 '17 at 09:56

score 0 · Answer 2 · answered Oct 25 '17 at 17:10

0

Apparently, your composer.json is not valid. With your data, you can try

awk 'NR==FNR{a[$0];next} {b=$0;sub("[^\"]*\"","");sub("\"[^\"]*$","");if(!($0 in a))print b}' file1.txt composer.json

answered Oct 25 '17 at 17:10

ctac_

2,413
2
7
17

For each line L of a file file1.txt, remove the line of the file composer.json that contains L

2 Answers2