3

Update: Note that as it stands this question does not make much sense, since the callstack will grow independently of the type T. The callstack solely depends on the number of nodes in the list being processed - the data contained in those nodes has no effect on the size of the callstack.


Let's say I have a recursive java implementation of a "remove-node-from-list" method which looks something like this:

// head
public void remove(T data) {
    if (data == null)
        throw new IllegalArgumentException();
    head = remove(head, data);
}

// other nodes of list
private ListNode remove(ListNode current, T data) {
    if (current == null)
        return null;
    if (current.data.compareTo(data) == 0)
        return current.next;
    current.next = remove(current.next, data);
    return current;
} 

Let's further assume that the type T is int

My question is: How large will the callstack get (in the worst-case) compared to the size of the list I'm processing?

What I've thought so far: From what I've seen Java 8 does not optimize recursions (right? my whole analysis assumes this to be true). Hence we will have a callstack growing linearly in with the size of the list.

More precisely: this method remove(ListNode current, T data) will basically put one node reference (32-Bits on 32-Bit arch) and an int (also 32-bit) on the callstack for each node in my list.

Now, since a node in the list also consists of one reference and one int a node has actually pretty much the same size as one of those recursive calls. Actually one of those calls might even use more space (the return address also has to be pushed onto the stack,right? or can this be optimized away somehow?).

So if I'm correct, then the callstack will basically get as large as the list itself in the worstcase (when the element to be removed is the last in the list)! Is this really the case? If I am correct then this would imply a factor of 1 or even larger with regard to the callstack required to process the list of ints! As a consequence, when processing large lists (actually not even that large), say 1M one will get a stackoverflow if one does run it with the default settings for stacksize (0.5M)

I'm aware of the fact that for larger data-types, the factor will be less, but still what I would stand firm on in any case is basically this: If there is no optimization going on, then the callstack will grow linearly with the size of the list - thus one should prefer the iterative over the recursive procedure.

In general I'd say, that the callstack will grow linearly roughly with a factor (size_of_all_arguments_in_call + size_of_return_address)/size_of_a_single_listelement which in the case of a list of ints is pretty much the same, right? Please don't get me wrong with the size_of_all_arguments: I know that we only pass the value of the references to the objects in those calls and not the whole objects themselves. Hence, for large objects, the factor might not be so dramatic but I'd still say one shouldn't accept that unnecessary linear space-overhead. However, for a list of ints the factor would be approx. 1.

Is this analysis correct or am I missing something?

Background: The person I was discussing this with, was trying to convince me that the callstack in java will not grow so dramatically and pointed out I was missing something obvious (unfortunately failed to tell me what and to me it really is not obvious at all) - since I don't know what it is I am missing I'm eager to learn ;)

Related:

The main thing missing in the related questions for me is how much space will the callstack actually use

Community
  • 1
  • 1
dingalapadum
  • 2,077
  • 2
  • 21
  • 31
  • I'm not sure if the JVM accounts for anything, but you could see this post and the linked duplicates http://stackoverflow.com/questions/17250883/is-it-possible-to-determine-how-much-space-is-available-on-the-stack – OneCricketeer May 15 '16 at 15:04
  • Found the Java question http://stackoverflow.com/questions/20030120/java-default-stack-size – OneCricketeer May 15 '16 at 15:07
  • I think you also need to consider the frame size for this: http://www.herongyang.com/JVM/Stack-Overflow-and-JVM-Stack-Frame.html – Lee May 15 '16 at 15:18
  • @Lee Also considering the frame size would make the factor *worse*, right? The frame size is pretty much what I was referring to with "pushing the return address onto the stack". But yes, this also should be taken into account in a rigouros analysis. – dingalapadum May 15 '16 at 15:21
  • @cricket_007 Unfortunately I fail to see anything answering my question in the posts you linked - for me this isn't about increasing the size of the stack or about garbage-collection. – dingalapadum May 15 '16 at 15:22
  • Well if by worse you mean more accurate then yes it would. You can adjust the frame size with some JVM options. – Lee May 15 '16 at 15:23
  • I was directing you to 'how large is the stack' because if you were to get a stackoverflow then you would obviously hit the maximum size. As far as "how much space does a recursive call occupy", then not much because it is only a pointer and a return address. The methods and variables that are declared between the ends of that method probably take more space – OneCricketeer May 15 '16 at 15:26
  • @Lee By worse, I meant the factor gets *higher*, i.e. it is even *less* advisable to use the recursion. But, yes, it would be more accurate! – dingalapadum May 15 '16 at 15:52
  • Yes. Unless the compiler does a very tricky optimization that very few compilers even attempt - removing non-tail recursion - the stack grows proportionally to the input length. I'd never use the recursive remove in production code. – Gene May 15 '16 at 16:03
  • The HotSpot optimizer will perform method inlining for recursive calls just like for other method calls, which will greatly reduce the required stack size, as discussed in [“Why is the max recursion depth I can reach non-deterministic?”](http://stackoverflow.com/a/27047776/2711488). However, even the reduced stack requirement scales *linearly* with you recursion depth/ list size so the general problem only becomes smaller und less deterministic, but not solved. – Holger May 20 '16 at 12:47

1 Answers1

0

How much stack does a recursion use (removing node from list)?

That's a difficult question to answer without profiling the code or having specifics on a particular JVM. That's because the Java Virtual Machine Specification (JVMS) leaves many details to the implementor.

JVMS Chapter 2: Implementation details that are not part of the Java Virtual Machine's specification would unnecessarily constrain the creativity of implementors. For example, the memory layout of run-time data areas, the garbage-collection algorithm used, and any internal optimization of the Java Virtual Machine instructions (for example, translating them into machine code) are left to the discretion of the implementor.

Part of those details are the stack and stack frame implementations. And since it seems you're actually asking about the relation between the stack created by the recursive remove and the list the algorithm is processing, we have to consider that the stack frame may not even be as big on the stack as you think (or propose in your question).

JVMS (§2.5.2): Because the Java Virtual Machine stack is never manipulated directly except to push and pop frames, frames may be heap allocated.

That means there could just be a single reference put onto the stack for each stack frame. If so, that would make the relation between the stack and the list it's processing truly insignificant for the example you provide. Let's look at some numbers:

According to Algorithms (Sedgewick, Wayne):

  • An Integer is 24 bytes
    Overhead + instance variable (int) + padding
    = 16 + 4 + 4 = 24 bytes

  • A non-static recursively defined Node class nested in the associated List is 40 bytes Overhead + references + additional overhead (reference to List class)
    = 16 + 8 + 8 + 8 = 40 bytes

Therefore, we can see there's 64 bytes per entry in the list. So, if there's an implementation where the frames are heap allocated the relation between the stack and the list you're processing would be 8/64 = 1/8 bytes.

It's more difficult to determine the size of the stack when the stack frames are allocated on the stack. In fact, we need further implementation details to determine it.

JVMS (§2.6. Frames): The sizes of the local variable array and the operand stack are determined at compile-time and are supplied along with the code for the method associated with the frame (§4.7.3). Thus the size of the frame data structure depends only on the implementation of the Java Virtual Machine, and the memory for these structures can be allocated simultaneously on method invocation.

Even so, I'd say, given your example, it's not likely the stack will grow at, or near, 1:1 in relation to a list (talking about total size in memory here). Sure, you could present a specific problem that would validate your argument but it would likely be trivial and likely not with the example you've provided.

ChiefTwoPencils
  • 13,548
  • 8
  • 49
  • 75