1

i ran into an issue when trying to print out a list of file dependencies.

About the program:

  1. Scans given *.c files for dependencies, more specifically looks for "#include "%"
  2. Find those files and scans them for their dependencies recursively
  3. All information is stored in a ConcurrentHashMap(key:String value:Linked List of Strings) theTable, where Linked List of Strings contains the list of dependencies.
  4. After processing a certain file, i end up with the following hash table:
chuckfinley
  • 2,577
  • 10
  • 32
  • 42
  • Why do you need a ConcurrentHashMap here? – isnot2bad Nov 27 '13 at 11:58
  • Because i will need to implement concurrency after i'm done with the single threaded version. – chuckfinley Nov 27 '13 at 12:01
  • in most cases StackOverflowError related to recursion means that your stop-condition is just wrong – Maciej Dobrowolski Nov 27 '13 at 12:03
  • 2
    If you are trying to create a Makefile, do not do it the above way! How will you know if a specific include is not within `#if 0 .. #include .. #endif` ? Instead, let your compiler figure out the includes. For GCC, there's -E -MM. I do not know about others. – UltraInstinct Nov 27 '13 at 12:06
  • @Thrustmaster that's my task ;) – chuckfinley Nov 27 '13 at 12:07
  • 2
    @martynas, I knew because I was doing the exact same thing few months back. :) I wanted an extremely simple quick and dirty system that can create a Makefile. [And I implemented it in Python here](https://bitbucket.org/thrustmaster/simple-makefile-creator/src/16f3132bee5d46be0261f7160a7cc5947ca134f9/makefile-creator?at=master) – UltraInstinct Nov 27 '13 at 12:09
  • possible duplicate of [What is a stack overflow error?](http://stackoverflow.com/questions/214741/what-is-a-stack-overflow-error) –  Mar 27 '14 at 05:47

5 Answers5

4

If I understand what the map output is saying, there is a cycle ( a #include loop ) in your header files.

i_50.h=[i_35.h, i_28.h, i_45.h, i_44.h, i_46.h],
....
i_35.h=[i_50.h, i_51.h]

That means that your dependencies are a graph and not a DAG. And that in turn means that a simple recursive walk will not work.

By the looks of it, you are attempting to do a graph walk, but for some reason your cycle detection / avoidance is not working, and your algorithm goes into "infinite" recursion.


After looking at the code, I think I can see where the problems are. In the first method, you check that a dependency has already been printed, and then set the entry in the alreadyPrinted map to say it has. But you then proceed to print it irrespective. Then in the second method you are (inexplicably) creating a new alreadyPrinted map each time you recurse to the first method. In other words, the logic of your cycle avoidance is broken.


Rather than fixing your code for you, I'm going to suggest that you go to your favourite "data structures and algorithms" text book and look up "graph traversal" in the index. Alternatively, here's an page I found in some online lecture notes:

There's also stuff on graph traversal in Wikipedia, and other places. Google for "java recursive graph traversal", and try and find something that makes sense to you.

The general algorithm is something like this:

    traverse(Node node):
        traverse_0(node, new Set<Node>())

    traverse_0(Node node, Set<Node> visited):
        if (visited.contains(node)) 
            return
        visited.add(node)
        for (Node child: node.children) 
            traverse_o(child, visited)
Stephen C
  • 698,415
  • 94
  • 811
  • 1,216
1

The only place where you test if a dependency has already been printed, is the first for loop. You should check in the second for loop too!

for (String d : dependencies) {
    if (!alreadyPrinted.containsKey(d)) {
        LinkedList<String> key = theTable.get(d);           
        if (key != null)            
            output += printDependencies(theTable, key, alreadyPrinted);
    }
}
isnot2bad
  • 24,105
  • 2
  • 29
  • 50
1

It is fairly easy to see that your algorithm recurses, as soon as some dependency looks like:

item:  ...., item, ....

(I hear you say: "that cannot happen, because ...". Yet, the SO shows that either it did happen, or your stack is too small.)

By the way, you maintain the map "already printed", but it is nowhere used? This hints to a flaw in your implementation.

Ingo
  • 36,037
  • 5
  • 53
  • 100
1

As you are maintaining some state (alreadyPrinted and output) I would recommend moving the state to instance-variabes and to use an object and no class-methods.

AlexS
  • 5,295
  • 3
  • 38
  • 54
1

The problem was that my Graph traversal had cycles which I was not handling. The working code is provided below.

private static String printDependencies(ConcurrentHashMap<String, LinkedList<String>> theTable, LinkedList<String> dependencies, ConcurrentHashMap<String, Boolean> alreadyPrinted) {

    String output = "";

    for (String d : dependencies) {
        boolean isPrinted = alreadyPrinted.containsKey(d);
        if (!isPrinted) {
            output += " " + d;
            alreadyPrinted.put(d, true);
        }           
    }

    for (String d : dependencies) {
        LinkedList<String> key = theTable.get(d);
        if (key != null) {
            LinkedList<String> unvisited = new LinkedList<String>();
            for (String filename : key)
                if (!alreadyPrinted.containsKey(filename))
                    unvisited.add(filename);
            if (unvisited != null)            
                output += printDependencies(theTable, unvisited, alreadyPrinted);               
        }
    }

    return output;
}

private static void printDependencies(ConcurrentHashMap<String, LinkedList<String>> theTable, ConcurrentLinkedQueue<String> toProcess) {
    String output = ""; 

    for (String current : toProcess) {
        ConcurrentHashMap<String, Boolean> alreadyPrinted = new ConcurrentHashMap<String, Boolean>(); // Keeps track of dependencies already printed
        output += current + ":" + printDependencies(theTable, theTable.get(current), alreadyPrinted) + "\n";
    }

    System.out.println(output);     
}
chuckfinley
  • 2,577
  • 10
  • 32
  • 42