2

The following looks very unusual :

?- findall(X, member(X, [1, 2, 3]), X).
X = [1, 2, 3].

The trace even more so

?- trace, findall(X, member(X, [1, 2, 3]), X).
^  Call: (11) findall(_100058, member(_100058, [1, 2, 3]), _100058) ? creep
^  Exit: (11) findall([1, 2, 3], user:member([1, 2, 3], [1, 2, 3]), [1, 2, 3]) ? creep
X = [1, 2, 3]

Thinking in terms of semantics of findall this makes little sense. What is going on?

rajashekar
  • 3,460
  • 11
  • 27
  • 1
    Good find. In particular `findall([1,2,3],member([1,2,3],[1,2,3]),[1,2,3]).` fails. The above makes only sense if the variant `X` and the bag `X` are not the same `X` although the trace indicates otherwise. Breakage due to the imperative program underneath. This probably warrants a fix (if it doesn't involve expensive checks) – David Tonhofer Nov 06 '20 at 14:30
  • Ad edit: this does not depend on the specific implementation. It is like that everywhere, and also in the standard. – false Nov 07 '20 at 17:15
  • @false : ISO standard says that this behavior is normal? Is this not a quirk of implementing `findall` in a specific way? – rajashekar Nov 08 '20 at 01:34
  • 1
    @rajashekar: yes it does. And I have not seen any other interpretation of this highly procedural built-in anywhere anytime. Remember the pure part of Prolog is based on first order logic. – false Nov 08 '20 at 11:54

2 Answers2

3

To expand on my comments, maybe this might help:

?- findall(X, member(X, [1, 2, 3]), Xs).
Xs = [1, 2, 3].

If you look closely, you will see that Prolog (SWI, in this case) did not print a substitution for X. This means that X is not bound when the query succeeds. Indeed:

?- findall(X, member(X, [1, 2, 3]), Xs), var(X).
Xs = [1, 2, 3].

This does not mean that X is never bound while the query executes:

?- findall(X, ( member(X, [1, 2, 3]), writeln(X) ), Xs), var(X).
1
2
3
Xs = [1, 2, 3].

But after all solutions have been generated, X is unbound and can be bound to some other value -- such as the list of solutions. This will work in any standard conforming Prolog, as the standard says explicitly that findall only tries to unify its third argument after it has created the list of solutions. It even contains an example with sharing between the template and the list of instantiations:

findall(X, (X=1;X=2), [X, Y]).
   Succeeds, unifying X with 1, and Y with 2.

So how does this binding and unbinding work? With a failure-driven loop, as quoted in rajashekar's answer from the SWI-Prolog implementation. In general, succeeding predicates bind some variables. When at some later point something fails (or, equivalently, the user presses ; when prompted by the toplevel), backtracking takes place: It unbinds variables to allow them to take new values, then retries some goal.

What goes on inside findall is the same as goes on when you write the following:

?- ( member(X, [1, 2, 3]), writeln(X), false ; true ), var(X).
1
2
3
true.

So while findall is very impure, it is not so impure as to be completely un-Prolog-like. In fact, we can write our own:

:- dynamic my_findall_bag/1.

my_findall(Template, Goal, Instances) :-
    % initialization
    retractall(my_findall_bag(_)),
    asserta(my_findall_bag([])),
    
    % collect solutions
    (   call(Goal),
        copy_term(Template, NewSolution),
        retract(my_findall_bag(PreviousSolutions)),
        asserta(my_findall_bag([NewSolution | PreviousSolutions])),
        % failure-driven loop: after saving the solution, force Goal to
        % generate a new one
        false
    ;   true ),

    % cleanup and finish; the saved solutions are in reversed order (newest
    % first), so reverse them
    retract(my_findall_bag(AllSavedSolutions)),
    reverse(AllSavedSolutions, Instances).

This behaves as expected:

?- my_findall(X, member(X, [1, 2, 3]), Xs).
Xs = [1, 2, 3].

Or even:

?- my_findall(X, member(X, [1, 2, 3]), X).
X = [1, 2, 3].

A minor problem with this is that the instantiation of Goal should be checked. A major problem with this is that all my_findall calls share the same bag, so calling my_findall from inside a my_findall (or in parallel) will make you unhappy. This could be fixed using some sort of gensym mechanism to give each my_findall run its unique key into the database.

As for the trace output, it is an unfortunate consequence of wanting to express "your goal succeeded with such-and-such bindings" on one line. At the point of success, it is true that findall(X, ..., X) succeeded, and it is true that X = [1, 2, 3], and hence it is true that the successful instance of the goal is findall([1, 2, 3], ..., [1, 2, 3]).

Consider:

forty_two(FortyTwo) :-
    var(FortyTwo),
    FortyTwo = 42.

my_call(Goal) :-
    format('about to call ~w~n', [Goal]),
    call(Goal),
    format('success: ~w~n', [Goal]).

For example:

?- my_call(forty_two(X)).
about to call forty_two(_2320)
success: forty_two(42)
X = 42.

So forty_two(42) is a succeeding instance of forty_two(X). Even though forty_two(42) does not succeed:

?- forty_two(42).
false.

It is logical that printing the term forty_two(X) in an environment with X = 42 prints forty_two(42). I think the problem is that this logical behavior sticks out as strange among all the non-logical stuff going on here.

Isabelle Newbie
  • 9,258
  • 1
  • 20
  • 32
  • Thank you for a prolog implementation of `findall` it was illuminating. – rajashekar Nov 07 '20 at 04:58
  • copy_term/2 is redundant asserta/1 will do a copy for you. And therefore you already see the problem of the algorithm, its quadratic. Also there could be problems with findall/3 inside findall/3. –  Nov 08 '20 at 20:46
  • The problem with `findall` inside `findall` is mentioned in the answer as a major problem. You're right about the extra copy (and `retract` will do one more, no?). The implementation is meant to be short and illustrative and has served that purpose. The quadratic behavior is from using a list, one could do it linearly by `assertz`ing each answer, then collecting those into a list. That's slightly more code. – Isabelle Newbie Nov 08 '20 at 21:59
1

I did some code diving to try and figure out what is going on. In swi-prolog listing(findall, [source(true)]). gives the following code :

findall(Templ, Goal, List) :-
    findall(Templ, Goal, List, []).

findall(Templ, Goal, List, Tail) :-
    setup_call_cleanup(
        '$new_findall_bag',
        findall_loop(Templ, Goal, List, Tail),
        '$destroy_findall_bag').

findall_loop in the appropriate file is as follows :

findall_loop(Templ, Goal, List, Tail) :-
    (   Goal,
        '$add_findall_bag'(Templ)   % fails
    ;   '$collect_findall_bag'(List, Tail)
    ).

After consulting the C source files, I found out that findall/4 is setting up a global variable in C-source ('$new_findall_bag') and findall_loop/4 is pushing the Templ to it when the Goal succeeds (with '$add_findall_bag'(Templ)). When the Goal fails Templ is uninstantiated and hence the final clause '$collect_findall_bag'(List, Tail) succeeds even when List and Templ are the same variable.

We can see in trace that Templ is usuall uninstantiated.

?- trace, findall(X, member(X, [1, 2, 3]), Xs).
^  Call: (11) findall(_28906, member(_28906, [1, 2, 3]), _28916) ? creep
^  Exit: (11) findall(_28906, user:member(_28906, [1, 2, 3]), [1, 2, 3]) ? creep
Xs = [1, 2, 3].

So the process of finding all instantiations of Templ so that the Goal succeeds is separate from the process of collecting all those instantiations into the variable List and hence we can use the same variable without causing and error. But the semantics of writing such a clause is not making much sense to me.

EDIT: Similar situation occurs in gprolog, where the process of collecting solutions and that of retriving them are separate. Relevant Yap code also looks quite similar, but i was not able to install it to check.

rajashekar
  • 3,460
  • 11
  • 27
  • I just could not figure out how `findall_loop` is actually looping over all solutions even after trying the C source. – rajashekar Nov 06 '20 at 14:15
  • 1
    That loop is not in the C source, it's in the snippet you posted above: `findall_loop` calls `Goal` and adds the answer to the bag. *Then it fails inside `$add_findall_bag`.* This forces it to try `Goal` again, which will generate a new answer, which it adds using `$add_findall_bag`, which then fails again, etc. This is a "normal" failure-driven loop, except that the failure is hidden inside another predicate. Fortunately there is the comment, at least. – Isabelle Newbie Nov 06 '20 at 15:43
  • @IsabelleNewbie : Thank you, I have been trying to find a loop in `add_finall_bag` function in the source code. Now that I check again, it always returns `FALSE` when used as a unary predicate. This is wonderful. – rajashekar Nov 06 '20 at 15:49
  • BTW this repeated failure is also the reason why you can reuse `X`: Every time the `Goal` succeeds, it will bind `X`. But then the failure will unbind `X` again so that the next alternative for `Goal` can bind it again. If `Goal` terminates, this sequence will always end in a failure of `Goal`, so `X` will not be bound after all solutions have been collected. So it can be bound to something else! Like the result of the `findall`. No, this doesn't make "logical" sense, it's very procedural. – Isabelle Newbie Nov 06 '20 at 15:56
  • How do you handle findall/3 inside findall/3 ? Does '$new_findall_bag' use its own thread local stack? Why not some '$new_findall_bag'/1 that returns a reference to the bag? –  Nov 08 '20 at 20:51
  • @MostowskiCollapse : Yes, `'$new_findall_bag'` does use its own stack. Since the reference is not a logical term, i.e., it's value changes and makes the code dynamic, I think it was a design decision to just use them in `C` and not reference them in `prolog`. If you look at @IsabelleNewbie's answer then you can see how `findall` is implemented using dynamic code. – rajashekar Nov 09 '20 at 02:45
  • @MostowskiCollapse : `findall` inside `findall` is handled using stack similar to how recursive functions use stack. If if a findall finishes execution then the stack frame(here bag) is freed or else a new one is put on top. – rajashekar Nov 09 '20 at 04:17