Do i still include dependencies in Pom.xml if installed locally?

Question

I originally used a library which is located in Central Maven repository. So I just included the dependencies in my pom file in IntelliJ.

Now I need to modify the source code a bit and then I had to re-compile the external library and installed locally into my .m2 repository. In this case, do I need to remove the dependencies section from my Pom in order to use my locally installed jar file which was compiled by myself?

The script to compile and generate the jar file. I run this script which comes with the library source code to compile and install locally.

#!/bin/bash -e

# helper function that ensures cmd returns 0 exit code
function checkexit {
    "$@"
    local status=$?
    if [ $status -ne 0 ]; then
        echo "error with $1" >&2
        exit 1
    fi
    return $status
}

function maybeUpdateRepo {
    if [ "$UPDATE_REPOS" == "true" ]; then
        # are there uncommited changes in the repo?
        if ! (git diff-index --quiet HEAD --) then
           echo "Some uncommited changes found in this repo! Stashing..."
           git stash
        fi
        git pull
    fi
}

# check incoming parameters
while [[ $# -gt 1 ]]
do
key="$1"
#Build type (release/debug), packaging type, chip: cpu,gpu,lib type (static/dynamic)
case $key in
    -c|--chip)
    CHIP="$2"
    shift # past argument
    ;;
    -cc|--compute)
    COMPUTE="$2"
    shift # past argument
    ;;
    -a|--march)
    NATIVE="$2"
    shift # past argument
    ;;
     -l|--libtype)
    LIBTYPE="$2"
    shift # past argument
    ;;
     --scalav)
    SCALAV="$2"
    shift # past argument
    ;;
    -s|--shallow)
    SHALLOW="YES"
    ;;
    -r|--repostrategy)
    REPO_STRATEGY="$2"
    shift # past argument
    ;;
    --deploy)
    DEPLOY="YES"
    ;;
    --testnd4j)
    TEST_ND4J="YES"
    ;;
    --testdatavec)
    TEST_DATAVEC="YES"
    ;;
    --testdl4j)
    TEST_DL4J="YES"
    ;;
    --skiplibnd4j)
    SKIP_LIBND4J="YES"
    ;;
    --skipnd4j)
    SKIP_ND4J="YES"
    ;;
    --skipdatavec)
    SKIP_DATAVEC="YES"
    ;;
    --skipdl4j)
    SKIP_DL4J="YES"
    ;;
    --mvnopts)
    MVN_OPTS="$2"
    shift
    ;;
    *)
            # unknown option
    ;;
esac
shift # past argument or value
done

# default for chip
if [[ -z "$CHIP" ]]; then
    # test for cuda libraries
    if (type ldconfig &> /dev/null); then
       if (ldconfig -p | grep -q libcuda\.so); then
           CHIP="cuda"
       else
           CHIP="cpu"
        fi
    fi
fi

# adjust scala versions
if [ "$SCALAV" == "2.10" ]; then
  SCALA="2.10.6"
fi
# adjust scala versions
if [ "$SCALAV" == "2.11" ]; then
  SCALA="2.11.7"
fi

# set git cloning to a shallow depth if the option says so
if [[ -z "$SHALLOW" ]]; then
    GIT_CLONE="git clone"
else
    GIT_CLONE="git clone --depth 1"
fi

# set git cloning to a shallow depth if the option says so
if [[ -z "$DEPLOY" ]]; then
    MVN_GOAL="install"
else
    MVN_GOAL="deploy"
fi

# Report argument values
echo CHIP          = "${CHIP}"
echo COMPUTE       = "${COMPUTE}"
echo DEPLOY        = "${DEPLOY}"
echo NATIVE        = "${NATIVE}"
echo LIBTYPE       = "${LIBTYPE}"
echo SCALAV        = "${SCALAV}"
echo SHALLOW       = "${SHALLOW}"
echo REPO_STRATEGY = "${REPO_STRATEGY}"
echo TEST_ND4J     = "${TEST_ND4J}"
echo TEST_DATAVEC  = "${TEST_DATAVEC}"
echo TEST_DL4J     = "${TEST_DL4J}"
echo SKIP_LIBND4J  = "${SKIP_LIBND4J}"
echo SKIP_ND4J     = "${SKIP_ND4J}"
echo SKIP_DATAVEC  = "${SKIP_DATAVEC}"
echo SKIP_DL4J     = "${SKIP_DL4J}"
echo MVN_OPTS      = "${MVN_OPTS}"

###########################
# Script execution starts #
###########################

pushd ..

# removes lingering snapshot artifacts from existing maven cache to ensure a
# clean build
JAVA_PROJECTS="nd4j datavec deeplearning4j"
for dirName in $JAVA_PROJECTS; do
    if [ -d "$dirName" ]; then
        pushd "$dirName"
        mvn dependency:purge-local-repository -DreResolve=false -DactTransitively=false
        popd
    fi
done

# What to do with existing repos
case $REPO_STRATEGY in
    "delete")
        DELETE_REPOS="true"
        UPDATE_REPOS=""
        ;;
    "update")
        UPDATE_REPOS="true"
        DELETE_REPOS=""
        ;;
    *) # unknown : do nothing
        UPDATE_REPOS=""
        DELETE_REPOS=""
        ;;
esac

# removes any existing repositories to ensure a clean build
if [[ ! (-z "$DELETE_REPOS") ]]; then
    PROJECTS="libnd4j nd4j datavec deeplearning4j"
    for dirName in $PROJECTS; do
        find . -maxdepth 1 -iname "$dirName" -exec rm -rf "{}" \;
    done
fi

# compile libnd4j
if [[ -z "$SKIP_LIBND4J" ]]; then
    if [[ ! (-z "$DELETE_REPOS") || ! (-d libnd4j) ]]; then
        checkexit $GIT_CLONE https://github.com/deeplearning4j/libnd4j.git
    fi
    pushd libnd4j
    maybeUpdateRepo
    if [[ -z "$NATIVE" ]]; then
        checkexit bash buildnativeoperations.sh "$@" -a native
    else
        checkexit bash buildnativeoperations.sh "$@"
    fi

    if [ "$CHIP" == "cuda" ]; then
        if [[ -z "$COMPUTE" ]]; then
            checkexit bash buildnativeoperations.sh -c cuda
        else
            checkexit bash buildnativeoperations.sh -c cuda -cc "$COMPUTE"
        fi
    fi
    LIBND4J_HOME=$(pwd)
    export LIBND4J_HOME
    popd
fi

# build and install nd4j to maven locally
if [[ -z "$SKIP_ND4J" ]]; then
    if [[ ! (-z "$DELETE_REPOS") || ! (-d nd4j) ]]; then
        checkexit $GIT_CLONE https://github.com/deeplearning4j/nd4j.git
    fi
    if [[ -z "$TEST_ND4J" ]]; then
        ND4J_OPTIONS="-DskipTests"
    else
        ND4J_OPTIONS=""
    fi
    pushd nd4j
    maybeUpdateRepo
    if [ "$CHIP" == "cpu" ]; then
        checkexit bash buildmultiplescalaversions.sh clean $MVN_GOAL -Dmaven.javadoc.skip=true -pl '!nd4j-backends/nd4j-backend-impls/nd4j-cuda,!nd4j-backends/nd4j-backend-impls/nd4j-cuda-platform,!nd4j-backends/nd4j-tests' $ND4J_OPTIONS $MVN_OPTS
    else
        checkexit bash buildmultiplescalaversions.sh clean $MVN_GOAL -Dmaven.javadoc.skip=true $ND4J_OPTIONS $MVN_OPTS
    fi
    popd
fi

# build and install datavec
if [[ -z "$SKIP_DATAVEC" ]]; then
    if [[ ! (-z "$DELETE_REPOS") || ! (-d datavec) ]]; then
        checkexit $GIT_CLONE https://github.com/deeplearning4j/datavec.git
    fi
    if [[ -z "$TEST_DATAVEC" ]]; then
        DATAVEC_OPTIONS="-DskipTests"
    else
        if [ "$CHIP" == "cuda" ]; then
            DATAVEC_OPTIONS="-Ptest-nd4j-cuda-8.0"
        else
            DATAVEC_OPTIONS="-Ptest-nd4j-native"
        fi
    fi
    pushd datavec
    maybeUpdateRepo
    if [ "$SCALAV" == "" ]; then
        checkexit bash buildmultiplescalaversions.sh clean $MVN_GOAL -Dmaven.javadoc.skip=true $DATAVEC_OPTIONS $MVN_OPTS
    else
        checkexit mvn clean $MVN_GOAL -Dmaven.javadoc.skip=true -Dscala.binary.version="$SCALAV" -Dscala.version="$SCALA" $DATAVEC_OPTIONS $MVN_OPTS
    fi
    popd
fi

# build and install deeplearning4j
if [[ -z "$SKIP_DL4J" ]]; then
    if [[ ! (-z "$DELETE_REPOS") ||  ! (-d deeplearning4j) ]]; then
        checkexit $GIT_CLONE https://github.com/deeplearning4j/deeplearning4j.git
    fi
    if [[ -z "$TEST_DL4J" ]] ; then
        DL4J_OPTIONS="-DskipTests"
    else
        if [ "$CHIP" == "cuda" ]; then
            DL4J_OPTIONS="-Ptest-nd4j-cuda-8.0"
        else
            DL4J_OPTIONS="-Ptest-nd4j-native"
        fi
    fi
    pushd deeplearning4j
    maybeUpdateRepo
    if [ $DELETE_REPOS == "true" ]; then
        # reset the working diectory to the latest version of the tracking branch
        git remote update
        TRACKING_BRANCH=$(git rev-parse --abbrev-ref --symbolic-full-name '@{u}')
        git reset --hard "$TRACKING_BRANCH"
    fi
    if [ "$SCALAV" == "" ]; then
        if [ "$CHIP" == "cpu" ]; then
            checkexit bash buildmultiplescalaversions.sh clean $MVN_GOAL -Dmaven.javadoc.skip=true -pl '!./deeplearning4j-cuda/' $DL4J_OPTIONS $MVN_OPTS
        else
            checkexit bash buildmultiplescalaversions.sh clean $MVN_GOAL -Dmaven.javadoc.skip=true $DL4J_OPTIONS $MVN_OPTS
        fi
    else
        checkexit mvn clean $MVN_GOAL -Dmaven.javadoc.skip=true -Dscala.binary.version="$SCALAV" -Dscala.version="$SCALA" $DL4J_OPTIONS $MVN_OPTS
    fi
    popd
fi

popd

How could Maven know that you intend to use that version of that jar in your app? — JB Nizet, Oct 28 '17 at 05:13
hopefully you gave your compiled version a different version number and told your POM to use that number instead? — blurfus, Oct 28 '17 at 05:14

Naman · Accepted Answer · 2017-10-28T05:27:53.673

1

I would suggest a better practice as follows along with your work line:

Now I need to modify the source code a bit and then I had to re-compile the external library and installed locally into my .m2 repository.

You must update it version from x.y.z to x.y.z+1-SNAPSHOT to mark its change and then publish the artifact. Let's say currently to your local itself.

Note - SNAPSHOT since this seems to be under development, or else you can mark the change to x.y.z+1 release version as well.

In this case, do I need to remove the dependencies section from my Pom in order to use my locally installed jar file which was compiled by myself?

Now in your original project when you were using the dependency as

<groupId>somegroup</groupId>
<artifactId>someartifactId</artifactId>
<version>x.y.z</version>

just update the version to

<version>x.y.z+1-SNAPSHOT</version>

Edit : This could be useful to understand the approach, What exactly is a Maven Snapshot and why do we need it?

edited Oct 28 '17 at 05:27

answered Oct 28 '17 at 05:13

Naman

27,789
26
218
353

" you can mark the change to x.y.z+1 release version as well." How to achieve this? The "...SNAPSHOT.jar" name was automatically generated by mvn. – user697911 Oct 28 '17 at 05:21
@user697911 Well if you don't append `-SNAPSHOT` to your version number that's a released version. Once deployed, shall not be overridden then. So for example `guava 23.0` is a released version. But there could have been a `23.0-SNAPSHOT` which even when shared and deployed is tentative to change in its implementation. – Naman Oct 28 '17 at 05:23
I added the script and how to modify this script to generate the modified jar file name? – user697911 Oct 28 '17 at 05:28
How to modify this script to generate "x.y.z+1-SNAPSHOT" – user697911 Oct 28 '17 at 05:29
@user697911 Honestly, I won't understand the script as much as you do. And I don't know why it wasn't there in the initial question. But assuming that you understand the script and if there is a way you do what you specified in your question previously(the quoted text in the answer) then I am sure you can update the relevance sections. – Naman Oct 28 '17 at 05:32
I didn't include it in the original question because I don't know how to approach the problem. I don't know how the "snapshot" was added to the jar. There should be somewhere I can' add something to change the standard name. – user697911 Oct 28 '17 at 05:36
@user697911 Well you shall probably ask the owner of the script ideally. intelliJ and maven(tags used) has less to do in such a case. Isn't it? IMHO, it doesn't make sense for me to understand some random script and figure out what it does. – Naman Oct 28 '17 at 05:38

Do i still include dependencies in Pom.xml if installed locally?

1 Answers1