1

I am trying to use Scala/Spark from IntelliJ in Windows 7, but it IntelliJ (and SBT command line) fails to download files. I am behind a proxy server.

        [info] Updating {file:/D:/Users/user1/IdeaProjects/untitled17/project/}untitled17-build...
        [warn]  [FAILED     ] org.apache.logging.log4j#log4j-core;2.8.1!log4j-core.jar(test-jar): typesafe-ivy-releases: unable to get resource for org.apache.logging.log4j#log4j-core;2.8.1: res=https://repo.typesafe.com/typesafe/ivy-releases/org.apache.logging.log4j/log4j-core/2.8.1/test-jars/log4j-core-tests.jar: java.io.IOException: Failed to authenticate with proxy (452ms)
        [warn]  [FAILED     ] org.apache.logging.log4j#log4j-core;2.8.1!log4j-core.jar(test-jar): sbt-plugin-releases: unable to get resource for org.apache.logging.log4j#log4j-core;2.8.1: res=https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/org.apache.logging.log4j/log4j-core/2.8.1/test-jars/log4j-core-tests.jar: java.io.IOException: Failed to authenticate with proxy (157ms)
        [warn]  [FAILED     ] org.apache.logging.log4j#log4j-core;2.8.1!log4j-core.jar(test-jar): public: unable to get resource for org/apache/logging/log4j#log4j-core;2.8.1: res=https://repo1.maven.org/maven2/org/apache/logging/log4j/log4j-core/2.8.1/log4j-core-2.8.1-tests.jar: java.io.IOException: Failed to authenticate with proxy (115ms)
        [warn]  Detected merged artifact: [FAILED     ] org.apache.logging.log4j#log4j-core;2.8.1!log4j-core.jar(test-jar):  (0ms).
        [warn] ==== typesafe-ivy-releases: tried
        [warn] ==== sbt-plugin-releases: tried
        [warn]   https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/org.apache.logging.log4j/log4j-core/2.8.1/test-jars/log4j-core-tests.jar
        [warn] ==== local: tried
        [warn]   d:\Users\user1\.ivy2\local\org.apache.logging.log4j\log4j-core\2.8.1\test-jars\log4j-core-tests.jar
        [warn] ==== public: tried
        [warn]   https://repo1.maven.org/maven2/org/apache/logging/log4j/log4j-core/2.8.1/log4j-core-2.8.1-tests.jar
        [warn] ==== local-preloaded-ivy: tried
        [warn]   d:\Users\user1\.sbt\preloaded\org.apache.logging.log4j\log4j-core\2.8.1\test-jars\log4j-core-tests.jar
        [warn] ==== local-preloaded: tried
        [warn]   file:/d:/Users/user1/.sbt/preloaded/org/apache/logging/log4j/log4j-core/2.8.1/log4j-core-2.8.1-tests.jar
        [warn]  ::::::::::::::::::::::::::::::::::::::::::::::
        [warn]  ::              FAILED DOWNLOADS            ::
        [warn]  :: ^ see resolution messages for details  ^ ::
        [warn]  ::::::::::::::::::::::::::::::::::::::::::::::
        [warn]  :: org.apache.logging.log4j#log4j-core;2.8.1!log4j-core.jar(test-jar)
        [warn]  ::::::::::::::::::::::::::::::::::::::::::::::
        [error] sbt.librarymanagement.ResolveException: download failed: org.apache.logging.log4j#log4j-core;2.8.1!log4j-core.jar(test-jar)

Similar problems were already reported here.

Already tried:

  • SBT versions 0.13.16, 1.0.3 and 1.1.1

  • setting proxy properties JAVA_OPTS, SBT_OPTS, sbtconfigtxt

        -Dhttp.proxyHost=*
        -Dhttp.proxyPort=*
        -Dhttp.proxyUser=*
        -Dhttp.proxyPassword=*
        -Dhttps.proxyHost=*
        -Dhttps.proxyPort=*
        -Dhttps.proxyUser=*
        -Dhttps.proxyPassword=*
    

    without success

  • verified the issue in SBT:

        d:\Users\user1>sbt.bat
        [info] Loading project definition from D:\Users\user1\project
        [info] Updating {file:/D:/Users/user1/project/}user1-build...
        [warn]  [FAILED     ] org.apache.logging.log4j#log4j-core;2.8.1!log4j-core.jar(t
        est-jar): typesafe-ivy-releases: unable to get resource for org.apache.logging.l
        og4j#log4j-core;2.8.1: res=https://repo.typesafe.com/typesafe/ivy-releases/org.a
        pache.logging.log4j/log4j-core/2.8.1/test-jars/log4j-core-tests.jar: java.io.IOE
        xception: Failed to authenticate with proxy (353ms)
        [warn]  [FAILED     ] org.apache.logging.log4j#log4j-core;2.8.1!log4j-core.jar(t
        est-jar): sbt-plugin-releases: unable to get resource for org.apache.logging.log
        4j#log4j-core;2.8.1: res=https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases
        /org.apache.logging.log4j/log4j-core/2.8.1/test-jars/log4j-core-tests.jar: java.
        io.IOException: Failed to authenticate with proxy (8ms)
        [warn]  [FAILED     ] org.apache.logging.log4j#log4j-core;2.8.1!log4j-core.jar(t
        est-jar): public: unable to get resource for org/apache/logging/log4j#log4j-core
        ;2.8.1: res=https://repo1.maven.org/maven2/org/apache/logging/log4j/log4j-core/2
        .8.1/log4j-core-2.8.1-tests.jar: java.io.IOException: Failed to authenticate wit
        h proxy (8ms)
        [warn]  Detected merged artifact: [FAILED     ] org.apache.logging.log4j#log4j-c
        ore;2.8.1!log4j-core.jar(test-jar):  (0ms).
        [warn] ==== typesafe-ivy-releases: tried
        [warn] ==== sbt-plugin-releases: tried
        [warn]   https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/org.apache.logg
        ing.log4j/log4j-core/2.8.1/test-jars/log4j-core-tests.jar
        [warn] ==== local: tried
        [warn]   d:\Users\user1\.ivy2\local\org.apache.logging.log4j\log4j-core\2.8.1\
        test-jars\log4j-core-tests.jar
        [warn] ==== public: tried
        [warn]   https://repo1.maven.org/maven2/org/apache/logging/log4j/log4j-core/2.8.
        1/log4j-core-2.8.1-tests.jar
        [warn] ==== local-preloaded-ivy: tried
        [warn]   d:\Users\user1\.sbt\preloaded\org.apache.logging.log4j\log4j-core\2.8
        .1\test-jars\log4j-core-tests.jar
        [warn] ==== local-preloaded: tried
        [warn]   file:/d:/Users/user1/.sbt/preloaded/org/apache/logging/log4j/log4j-co
        re/2.8.1/log4j-core-2.8.1-tests.jar
        [warn]  ::::::::::::::::::::::::::::::::::::::::::::::
        [warn]  ::              FAILED DOWNLOADS            ::
        [warn]  :: ^ see resolution messages for details  ^ ::
        [warn]  ::::::::::::::::::::::::::::::::::::::::::::::
        [warn]  :: org.apache.logging.log4j#log4j-core;2.8.1!log4j-core.jar(test-jar)
        [warn]  ::::::::::::::::::::::::::::::::::::::::::::::
        [error] sbt.librarymanagement.ResolveException: download failed: org.apache.logg
        ing.log4j#log4j-core;2.8.1!log4j-core.jar(test-jar)
        [error]         at sbt.internal.librarymanagement.IvyActions$.resolveAndRetrieve
        (IvyActions.scala:331)
        [error]         at sbt.internal.librarymanagement.IvyActions$.$anonfun$updateEit
        her$1(IvyActions.scala:205)
        [error]         at sbt.internal.librarymanagement.IvySbt$Module.$anonfun$withMod
        ule$1(Ivy.scala:229)
        [error]         at sbt.internal.librarymanagement.IvySbt.$anonfun$withIvy$1(Ivy.
        scala:190)
        [error]         at sbt.internal.librarymanagement.IvySbt.sbt$internal$libraryman
        agement$IvySbt$$action$1(Ivy.scala:70)
        [error]         at sbt.internal.librarymanagement.IvySbt$$anon$3.call(Ivy.scala:
        77)
        [error]         at xsbt.boot.Locks$GlobalLock.withChannel$1(Locks.scala:95)
        [error]         at xsbt.boot.Locks$GlobalLock.xsbt$boot$Locks$GlobalLock$$withCh
        annelRetries$1(Locks.scala:80)
        [error]         at xsbt.boot.Locks$GlobalLock$$anonfun$withFileLock$1.apply(Lock
        s.scala:99)
        [error]         at xsbt.boot.Using$.withResource(Using.scala:10)
        [error]         at xsbt.boot.Using$.apply(Using.scala:9)
        [error]         at xsbt.boot.Locks$GlobalLock.ignoringDeadlockAvoided(Locks.scal
        a:60)
        [error]         at xsbt.boot.Locks$GlobalLock.withLock(Locks.scala:50)
        [error]         at xsbt.boot.Locks$.apply0(Locks.scala:31)
        [error]         at xsbt.boot.Locks$.apply(Locks.scala:28)
        [error]         at sbt.internal.librarymanagement.IvySbt.withDefaultLogger(Ivy.s
        cala:77)
        [error]         at sbt.internal.librarymanagement.IvySbt.withIvy(Ivy.scala:185)
        [error]         at sbt.internal.librarymanagement.IvySbt.withIvy(Ivy.scala:182)
        [error]         at sbt.internal.librarymanagement.IvySbt$Module.withModule(Ivy.s
        cala:228)
        [error]         at sbt.internal.librarymanagement.IvyActions$.updateEither(IvyAc
        tions.scala:190)
        [error]         at sbt.librarymanagement.ivy.IvyDependencyResolution.update(IvyD
        ependencyResolution.scala:20)
        [error]         at sbt.librarymanagement.DependencyResolution.update(DependencyR
        esolution.scala:56)
        [error]         at sbt.internal.LibraryManagement$.resolve$1(LibraryManagement.s
        cala:38)
        [error]         at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$12(Libr
        aryManagement.scala:91)
        [error]         at sbt.util.Tracked$.$anonfun$lastOutput$1(Tracked.scala:68)
        [error]         at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$19(Libr
        aryManagement.scala:104)
        [error]         at scala.util.control.Exception$Catch.apply(Exception.scala:224)
    
        [error]         at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$11(Libr
        aryManagement.scala:104)
        [error]         at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$11$adap
        ted(LibraryManagement.scala:87)
        [error]         at sbt.util.Tracked$.$anonfun$inputChanged$1(Tracked.scala:149)
        [error]         at sbt.internal.LibraryManagement$.cachedUpdate(LibraryManagemen
        t.scala:118)
        [error]         at sbt.Classpaths$.$anonfun$updateTask$5(Defaults.scala:2353)
        [error]         at scala.Function1.$anonfun$compose$1(Function1.scala:44)
        [error]         at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFuncti
        ons.scala:42)
        [error]         at sbt.std.Transform$$anon$4.work(System.scala:64)
        [error]         at sbt.Execute.$anonfun$submit$2(Execute.scala:257)
        [error]         at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.sc
        ala:16)
        [error]         at sbt.Execute.work(Execute.scala:266)
        [error]         at sbt.Execute.$anonfun$submit$1(Execute.scala:257)
        [error]         at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(Con
        currentRestrictions.scala:167)
        [error]         at sbt.CompletionService$$anon$2.call(CompletionService.scala:32
        )
        [error]         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        [error]         at java.util.concurrent.Executors$RunnableAdapter.call(Executors
        .java:511)
        [error]         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        [error]         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolE
        xecutor.java:1149)
        [error]         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPool
        Executor.java:624)
        [error]         at java.lang.Thread.run(Thread.java:748)
        Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore? [error] (*:update)
         sbt.librarymanagement.ResolveException: download failed: org.apache.logging.log
        4j#log4j-core;2.8.1!log4j-core.jar(test-jar)
    
peleitor
  • 459
  • 7
  • 24

2 Answers2

4

If you're convinced that this is a proxy issue (and not, say, a temporary network connection problem), then here are your alternatives...

As you mention, you can define a JVM_OPTS environment variable specifying JVM proxy configuration options. Just to be clear, it's best if this is defined globally (via the Environment Variables dialog in the Windows System applet, since it looks like you're using Windows). If you're not sure how to do this, let me know. It's important to realize that your existing processes will not pick up changes made to global environment variables - the simplest solution is just to close and restart them.

Another alternative is to put the JVM options into a .jvmopts file in the root folder of your SBT project. (It's a good idea to put project-specific JVM options in this file, but not such a good idea to put machine- or site-specific options - such as proxy settings - into such a file. Clearly, if you need login credentials to configure your proxy, don't go this route!) If you pick this option, you need one setting on each line of the file. SBT will read any such settings each time it runs.

Either way, it's not clear from your question whether you have specified the JVM proxy settings correctly, since * is not a valid value for any of them. (If you've just used * to replace your actual settings, fair enough.) You'll know best what your proxy settings should be, but - if you have setup system proxy settings for your machine in Windows - often the simplest way to configure Java applications to use a proxy is to use the following setting:

JAVA_OPTS=-Djava.net.useSystemProxies=true

or create a .jvmopts file with these contents:

-Djava.net.useSystemProxies=true

If you haven't set up system proxy information, you'll need to be more explicit:

JAVA_OPTS=-Dhttp.proxyHost=my.proxyhost.com -Dhttp.proxyPort=80 -Dhttps.proxyHost=my.proxyhost.com -Dhttps.proxyPort=443

or, in .jvmopts:

-Dhttp.proxyHost=my.proxyhost.com
-Dhttp.proxyPort=80
-Dhttps.proxyHost=my.proxyhost.com
-Dhttps.proxyPort=443

Note also that if your proxy uses SOCKS instead of HTTP, then the relevant settings will be:

JAVA_OPTS=-DsocksProxyHost=my.proxyhost.com -DsocksProxyPort=8080

or, in .jvmopts:

-DsocksProxyHost=my.proxyhost.com
-DsocksProxyPort=8080

Obviously, you'll need to replace these host addresses and port numbers with those for your proxy. (If your domain authenticates you to the proxy server, you shouldn't need to specify username and password settings.)

That should ensure SBT works from the command line, and for the occasions that you execute SBT tasks from within IntelliJ (as a run configuration).

However, IntelliJ also has it's own proxy settings, which it uses when running SBT internally. You'll need to configure this too. You can access the IntelliJ proxy settings via the File/Settings... menu option, then looking under Appearance & Behavior/System Settings/HTTP Proxy (or type 'proxy' into the setting search box):

enter image description here

Let me know how you get on...

Mike Allen
  • 8,139
  • 2
  • 24
  • 46
-1

For me, proxy settings (no matter if they were set at Intellij, SBT.conf or environment variables), did not work.

​A couple of considerations that solved this issue (for me at least): ​

​- if you use SBT 0.13.16 (not newer that that)

​- set Use Auto Import

​ ​Then, no "FAILED DOWNLOADS" messages appear.

peleitor
  • 459
  • 7
  • 24