How to lint all the files recursive while printing out only files that have an error?

Question

I want to lint all the files in the current (recursive) directory while printing out only files that have an error, and assign a variable to 1 to be used after the linting is finished.

#!/bin/bash

lint_failed=0
find . -path ./vendor -prune -o -name '*.php' | parallel -j 4 sh -c 'php -l {} || echo -e "[FAIL] {}" && lint_failed=1';

if [ "$lint_failed" -eq "1" ]; then
    exit 1
fi

Example:

[FAIL] ./app/Model/Example.php

The above code doesn't find any errors, but if I run php -l ./app/Model/Example.php an error is returned.

Is https://github.com/JakubOnderka/PHP-Parallel-Lint missing anything from your needs? It doesn't appear to do a simple list of fails, but you may be able to parse it out of the Json output with a tool like `jq` — Alister Bulman, Jan 03 '18 at 23:18

astrangeloop · Accepted Answer · 2018-01-08T16:25:20.273

5

The parallel command already does what you want: it exits 0 if all jobs exit 0, and it exits non-zero if any one job exits non-zero. parallel's exit options are configurable, see the EXIT STATUS section of man parallel for details.

In your script, the use of || echo obscures the exit status of the jobs, but you can expose this again doing something like this (tested bash 4.4.7 on ubuntu):

#!/bin/bash

php_lint_file()
{
    local php_file="$1"
    php -l "$php_file" &> /dev/null
    if [ "$?" -ne 0 ]
    then
        echo -e "[FAIL] $php_file"
        return 1
    fi
}

export -f php_lint_file

find . -path ./vendor -prune -o -name '*.php' | parallel -j 4 php_lint_file {}

if [ "$?" -ne 0 ]
then
    exit 1
fi

edited Jan 08 '18 at 16:25

answered Jan 03 '18 at 23:04

astrangeloop

1,430
2
13
11

I had to set this aside for a while and am just coming back to it. It's close but I found that `$?` increments on the amount of errors linting finds (If 3 files fail then `$?` is 3. I can get around this with `if [ "$?" -ne 0 ]; then exit 1 fi` at the end of the script. – Michael Delle Jan 08 '18 at 16:22
@MichaelDelle excellent, I've updated my answer to reflect this. – astrangeloop Jan 08 '18 at 16:25
Great! I'll put this into my CI pipeline later and test it more. Thank you! :) – Michael Delle Jan 08 '18 at 16:55
Found another problem: It's creating so many processes. https://pastebin.com/raw/gnj2mdhs – Michael Delle Jan 08 '18 at 19:27
I would expect 2 processes per `parallel` job (one for the script and one for `php -l`) so 8 in total for the example script, but no more than that. Do you have multiple CI build steps happening at the same time? – astrangeloop Jan 08 '18 at 20:58
No, this was on dev, I just didn't notice it before. It's starting 13 processes :( 4 of them are for `php -l` – Michael Delle Jan 08 '18 at 21:20
This doesn't work on MacOS due to the `parallel` command being missing. I don't care about running this in parallel, is there a more robust version of this? – Tom J Nowell Apr 12 '19 at 13:33

score 2 · Answer 2 · answered Jan 19 '18 at 21:26

You can use PHP Parallel Lint tool which checks the syntax of PHP files faster and with a fancier output by running parallel jobs while printing out only files with the errors.

Example usage:

./bin/parallel-lint --exclude app --exclude vendor .

Or using Ant's build.xml:

<condition property="parallel-lint" value="${basedir}/bin/parallel-lint.bat" else="${basedir}/bin/parallel-lint">
    <os family="windows"/>
</condition>

<target name="parallel-lint" description="Run PHP parallel lint">
    <exec executable="${parallel-lint}" failonerror="true">
        <arg line="--exclude" />
        <arg path="${basedir}/app/" />
        <arg line="--exclude" />
        <arg path="${basedir}/vendor/" />
        <arg path="${basedir}" />
    </exec>
</target>

How to lint all the files recursive while printing out only files that have an error?

2 Answers2