Difficulty building Pig using Ant

Question

I am a pig newbie and have been trying to get it running on my Windows 7 machine. My hadoop (2.4.0) works fine on my local machine, but whenever I try to run a simple Pig script, it throws up errors whenever I try to dump the contents of a file.

Going off of a solution for a similar issue, found here, I have tried to rebuild my Pig (0.12.1) using Ant. Whenever I run

ant clean jar-withouthadoop -Dhadoopversion=23

I get the following errors:

Buildfile: C:\pig-0.12.1-src\build.xml
......
[org.apache.ftpserver#ftpserver-core;1.0.0] in [hadoop20]
[ivy:resolve]   org.apache.mina#mina-core;2.0.0-M2 by [org.apache.mina#mina-core;2.0.0-M5] in [hadoop20]
---------------------------------------------------------------------
|                  |            modules            ||   artifacts   |
|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
|      master      |   0   |   0   |   0   |   0   ||   0   |   0   |
|      default     |  101  |   32  |   0   |   3   ||  101  |   0   |
|      runtime     |  101  |   32  |   0   |   3   ||  101  |   0   |
|      compile     |   89  |   29  |   0   |   2   ||   90  |   0   |
|       test       |  101  |   32  |   0   |   3   ||  101  |   0   |
|      javadoc     |  101  |   32  |   0   |   3   ||  101  |   0   |
|   releaseaudit   |   3   |   2   |   0   |   0   ||   3   |   0   |
|       jdiff      |   3   |   0   |   0   |   0   ||   3   |   0   |
|    checkstyle    |   10  |   3   |   0   |   0   ||   10  |   0   |
|     buildJar     |  101  |   32  |   0   |   3   ||  101  |   0   |
|     hadoop20     |   48  |   30  |   0   |   9   ||   39  |   0   |
|     hadoop23     |   40  |   13  |   0   |   0   ||   42  |   0   |
|      hbase94     |   1   |   0   |   0   |   0   ||   2   |   0   |
|      hbase95     |   7   |   0   |   0   |   0   ||   13  |   0   |
---------------------------------------------------------------------

ivy-compile:
[ivy:retrieve] :: retrieving :: org.apache.pig#pig
[ivy:retrieve]  confs: [compile]
[ivy:retrieve]  90 artifacts copied, 0 already retrieved (80170kB/407ms)
[ivy:cachepath] DEPRECATED: 'ivy.conf.file' is deprecated, use 'ivy.settings.file' instead
[ivy:cachepath] :: loading settings :: file = C:\pig-0.12.1-src\ivy\ivysettings.xml

init:
    [mkdir] Created dir: C:\pig-0.12.1-src\src-gen\org\apache\pig\impl\logicalLayer\parser
    [mkdir] Created dir: C:\pig-0.12.1-src\src-gen\org\apache\pig\tools\pigscript\parser
    [mkdir] Created dir: C:\pig-0.12.1-src\src-gen\org\apache\pig\tools\parameters
    [mkdir] Created dir: C:\pig-0.12.1-src\build\classes
    [mkdir] Created dir: C:\pig-0.12.1-src\build\test\classes
    [mkdir] Created dir: C:\pig-0.12.1-src\test\org\apache\pig\test\utils\dotGraph\parser
    [mkdir] Created dir: C:\pig-0.12.1-src\src-gen\org\apache\pig\data\parser
    [move] Moving 1 file to C:\pig-0.12.1-src\build\ivy\lib\Pig

cc-compile:
   [javacc] Java Compiler Compiler Version 4.2 (Parser Generator)
   [javacc] (type "javacc" with no arguments for help)
   [javacc] Reading from file C:\pig-0.12.1-
    ....
   [javacc] Parser generated successfully.

prepare:
    [mkdir] Created dir: C:\pig-0.12.1-src\src-gen\org\apache\pig\parser

genLexer:
     [java] error(1):  cannot write file : java.io.FileNotFoundException: C:\pig-0.12.1-src\src-gen\org\apache\pig\parser\C:\QueryLexer.java (The filename, directory name, or volume label syntax is incorrect)
     [java] java.io.FileOutputStream.open(Native Method)
     [java] java.io.FileOutputStream.<init>(FileOutputStream.java:221)
     [java] java.io.FileOutputStream.<init>(FileOutputStream.java:171)
     [java] java.io.FileWriter.<init>(FileWriter.java:90)
     [java] org.antlr.Tool.getOutputFile(Tool.java:871)
     [java] org.antlr.codegen.CodeGenerator.write(CodeGenerator.java:1262)
     [java] org.antlr.codegen.Target.genRecognizerFile(Target.java:94)
     [java] org.antlr.codegen.CodeGenerator.genRecognizer(CodeGenerator.java:460)
     [java] org.antlr.Tool.generateRecognizer(Tool.java:655)
     [java] org.antlr.Tool.process(Tool.java:468)
     [java] org.antlr.Tool.main(Tool.java:93)
BUILD FAILED
C:\pig-0.12.1-src\build.xml:495 Java returned: 1

What am I doing wrong here? Am I missing files, or is this configuration issue in build.xml?

EDIT: I have run this command from both the cmd and cygwin terminal. They both output the same error.

EDIT2: I'm building with Ant (1.9.4)

FINAL EDIT: Turns out I needed to move the pig src archive (pig-0.12.1-src.tar.gz) to the cygwin home directory, unzip it, cd into the unzipped archive and run the command.

Did you try using the same command from `cygwin` ? – merours Jun 17 '14 at 13:48 — merours, Jun 17 '14 at 13:48
I did, generates the same error. – Phreakradio Jun 17 '14 at 14:11 — Phreakradio, Jun 17 '14 at 14:11

merours · Accepted Answer · 2015-05-19T07:11:07.723

I had the same problem. I solved it by compiling pig via cygwin in my cygwin home.

My ivy settings were as such :

<ivysettings>

 <!--
   Licensed to the Apache Software Foundation (ASF) under one or more
   contributor license agreements.  See the NOTICE file distributed with
   this work for additional information regarding copyright ownership.
   The ASF licenses this file to You under the Apache License, Version 2.0
   (the "License"); you may not use this file except in compliance with
   the License.  You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.
-->

 <!--
  see http://www.jayasoft.org/ivy/doc/configuration
  -->
  <!-- you can override this property to use mirrors
          http://repo1.maven.org/maven2/
          http://mirrors.dotsrc.org/maven2
          http://ftp.ggi-project.org/pub/packages/maven2
          http://mirrors.sunsite.dk/maven2
          http://public.planetmirror.com/pub/maven2
          http://ibiblio.lsu.edu/main/pub/packages/maven2
          http://www.ibiblio.net/pub/packages/maven2
  -->
  <property name="repo.maven.org" value="${mvnrepo}" override="true"/>
  <property name="repo.jboss.org" value="http://repository.jboss.com/nexus/content/groups/public/" override="false"/>
  <property name="repo.apache.snapshots" value="http://repository.apache.org/content/groups/snapshots-group/" override="false"/>
  <property name="repo.dir" value="${user.home}/.m2/repository" override="false"/>
  <property name="maven2.pattern" value="[organisation]/[module]/[revision]/[module]-[revision](-[classifier])"/>
  <property name="maven2.pattern.ext" value="${maven2.pattern}.[ext]"/>
  <property name="snapshot.pattern" value="[organisation]/[module]/[revision]/[artifact]-[revision](-[classifier]).[ext]"/>
  <property name="resolvers" value="default" override="false"/>
  <property name="force-resolve" value="false" override="false"/>
  <!-- pull in the local repository -->
  <include url="${ivy.default.conf.dir}/ivyconf-local.xml"/>
  <settings defaultResolver="${resolvers}"/>
  <resolvers>
    <ibiblio name="maven2" root="${repo.maven.org}" pattern="${maven2.pattern.ext}" m2compatible="true"/>
    <ibiblio name="jboss-maven2" root="${repo.jboss.org}" pattern="${maven2.pattern.ext}" m2compatible="true"/>
    <ibiblio name="apache-snapshots" root="${repo.apache.snapshots}" pattern="${snapshot.pattern}"
         checkmodified="true" changingPattern=".*SNAPSHOT" m2compatible="true"/>
    <filesystem name="fs" m2compatible="true" checkconsistency="false" force="${force-resolve}"
         checkmodified="true" changingPattern=".*SNAPSHOT">
      <artifact pattern="${repo.dir}/${maven2.pattern.ext}"/>
      <ivy pattern="${repo.dir}/[organisation]/[module]/[revision]/[module]-[revision].pom"/>
    </filesystem>
    <chain name="internal" checkmodified="true">
      <resolver ref="fs"/>
    </chain>
    <chain name="external" dual="true">
      <resolver ref="maven2"/>
      <resolver ref="jboss-maven2"/>
      <resolver ref="apache-snapshots"/>
    </chain>
    <chain name="default" dual="true" checkmodified="true">
      <resolver ref="internal"/>
      <resolver ref="external"/>
    </chain>
  </resolvers>
  <modules>
    <module organisation="org.apache.pig" name=".*" resolver="internal"/>
  </modules>
</ivysettings>

can you share your ivy.settings? – Capacytron May 15 '15 at 08:26 — Capacytron, May 15 '15 at 08:26

Difficulty building Pig using Ant

1 Answers1