0

To insert an item at position 0 in scheme I can do the following:

(define 1-to-3 (cons 1 (cons 2 (cons 3 nil))))
(cons 100 1-to-3)
; (100 1 2 3)

Is there a built-in way to insert an element at the end of a list (i.e., append it to the list)?

Will Ness
  • 70,110
  • 9
  • 98
  • 181
David542
  • 104,438
  • 178
  • 489
  • 842
  • 1
    just a comment: If you need to regularly add to the end of a data structure, a list will not serve you well. – tf3 May 24 '21 at 10:03
  • @tf3 what's a better lisp-data-structure in that case? Or do you just mean using something like a singly-linked-list instead (if so, how would that work in Lisp?) – David542 May 24 '21 at 17:55
  • 2
    Isn’t a lisp list really a singly linked list, that’s why we go the whole length of the list to get to the end? You need a queue data structure, which is just a list but wherein you also have a pair containing references to the first and the last element of that list. So you can reach the end of the list in constant time, and *mutate* the cdr of the last pair to point to a new pair. (You won’t be mutating anything in chapter 2, chapter 3 introduces mutation) – tf3 May 24 '21 at 19:34
  • cf. https://stackoverflow.com/questions/13255731/tail-recursive-function-appending-element-to-list – Will Ness May 27 '21 at 16:36

4 Answers4

2

Use append

There is no built-in way to add an item to the end of a list in standard Scheme. If this is something that you really need to do, but infrequently, you can use append:

;;; Using append: linear in the length of the list `xs`.
(define (append-item xs x)
  (append xs (list x)))
> (append-item '(1 2 3 4) 5)
(1 2 3 4 5)

Pick a Better Data Structure

If you need to add a lot of items to the ends of lists, append will become expensive since it has linear time complexity. The specific use case will guide your choices here; if the goal is to build a list from the end instead of from the front then it may be best to cons up the list and reverse it, as discussed by @Sylwester. If instead you need to be able to add and remove elements from both ends of a list, you might consider a double-ended queue. You can roll your own, or you may be able to use a preexisting library such as SRFI 117 or SRFI 134.

There is no way to get the last pair of a list without either walking the list, maintaining an index, or maintaining a pointer to the last pair of the list in Scheme. You can walk a list directly (but have linear time complexity); you can get constant time complexity by maintaining state (e.g., an index or tail pointer). When you start down this path, you will likely want to create some sort of data structure that abstracts the details. A deque is one example of such a data structure, but as @WillNess has observed, that may be more data structure than you need. A simpler data structure could be created, but the details would depend on an actual use case.

Mutation on a List

You can use mutation to add an element to the end of a list, but this is not idiomatic in Scheme, and it is probably a bad idea, anyway. (Although you might find mutation used in an implementation of a deque or similar data structure). You could mutate the last pair in the input list, using set-cdr! to attach a list containing the new element, like this:

;;; Using `set-cdr!`: this is still linear in the length of `xs`, since it
;;; requires `length`.
(define (append-item! xs x)
  (set-cdr! (list-tail xs (- (length xs) 1)) (list x))
  xs)
> (append-item! (list 1 2 3 4) 5)
(1 2 3 4 5)
> (append-item! (list 1 2 3 4) 5)
(1 2 3 4 5)
> (define xs (list 1 2 3 4))
> (append-item! xs 5)
(1 2 3 4 5)
> xs
(1 2 3 4 5)

This is a bad idea because: 1) you should not attempt to modify list literals in Scheme, which means that you have to pay attention to the provenance of lists given to append-item!, and 2) it is linear in the length of its input list anyway.

ad absurdum
  • 19,498
  • 5
  • 37
  • 60
  • thanks, this is what I was looking for. How does that approach compare to the example procedure I tried to write? – David542 May 24 '21 at 04:19
  • Both are linear in the length of the input. The version using `append` is easier to write, hence less error-prone; if you did not need it frequently you might just code it up inline. I would expect that the `append` version would perform better, at least by some constant factor, in a quality Scheme implementation since built in procedures are usually pretty optimal. – ad absurdum May 24 '21 at 04:32
  • Also: the version that you wrote isn't tail recursive. Some implementations may do tail call elimination for your code, i.e., [tail recursion modulo cons](https://jamesrwilcox.com/tail-mod-cons.html), but standard Scheme only requires tail call elimination for calls in tail position. – ad absurdum May 24 '21 at 05:25
  • @adabsurdum "you should not attempt to modify list literals in Scheme" - can you please elaborate on why list literals should not be mutated? How is mutating a list literal different from mutating a list defined as (define items (list 1 2 3)) ? – tf3 May 24 '21 at 09:47
  • @tf3 -- quoted lists are constants, and may share memory locations, or may reside in read-only memory. Lists created with `list` are freshly allocated. Given `(define xs '(1 2 3))` and `(define ys '(1 2 3))`, attempts to mutate `xs` _may_ have unintended consequences for `ys`. Different implementations may behave differently. – ad absurdum May 24 '21 at 12:18
  • @adabsurdum what I have learnt (till now) is that quoted lists treat their contents as symbols, and it's the individual symbols that are stored in locations that are shared between different lists, i.e. the lists simply point to these locations. Maybe it happens in some implementations, that not just the symbol but the whole list is treated as a symbol. It doesn't in MIT scheme. In it, I would have to do something like (define xs '(1 2 3)) and (define ys xs) for such a behaviour. – tf3 May 24 '21 at 13:56
  • 1
    @tf3 -- as I said, different implementations may behave differently. Better to code to the standard. The standard says that it is an error to attempt to modify the value of a literal expression; implementations may or may not signal that error, and implementations may have extensions allowing for some situations that are errors in the standard. Further, the question is not tagged with a specific implementation, so standard Scheme is assumed. – ad absurdum May 24 '21 at 14:12
  • 1
    @tf3 -- "_it's the individual symbols that are stored in locations that are shared between different lists_": that is what, e.g. `(list 'a 'b 'c)` does, but `(quote (a b c))` is not required to create a fresh list. In R7RS `quote` is described in 4.1.2. Literal Expressions: "_This notation is used to include literal constants in Scheme code.... it is an error to attempt to alter a constant (i.e. the value of a literal expression)...._" – ad absurdum May 24 '21 at 14:26
  • @adabsurdum agree, the standard is pretty explicit "You’ll get an error if you try to change, for example, a quoted pair with set-car! or a literal string with string-set!" , while in MIT scheme, even something like (set-car! '(a b) 'c) runs fine. – tf3 May 24 '21 at 14:37
  • 1
    @tf3 -- I wouldn't rely on that, even in MIT Scheme. Implementations are not required to detect the error, but the behavior is unspecified when attempting to modify a literal constant. As far as I can tell, the MIT Scheme Reference Manual makes no special claims about this; it may not work in some circumstances, or it may change over time with newer versions of the compiler. – ad absurdum May 24 '21 at 15:06
  • mutating the last cons cell is O(1). finding the last cons cell of a list is O(n). solution: maintain the "pointer to" the last cons cell together with the list, and use it (and update it) to append to the list's end. thus, O(1) append. and if [`set-cdr!` is available](https://groups.csail.mit.edu/mac/ftpdir/scheme-reports/r5rs-html/r5rs_8.html#SEC59), we can just as well use it. – Will Ness May 27 '21 at 15:45
  • as for TRMC, we already have [the tag](https://stackoverflow.com/tags/tailrecursion-modulo-cons/info) here. – Will Ness May 27 '21 at 16:09
  • @WillNess -- "_maintain the 'pointer to' the last cons cell ...._" That is why I mentioned using a deque; you can't append an element to the end of a list in Scheme in O(1) without some additional structure such as you suggest. Using `set-cdr!` is fine, so long as you don't try to use it on a list literal. I did not realize that we had a [tag:tailrecursion-modulo-cons] tag! – ad absurdum May 27 '21 at 18:02
  • 1
    @adabsurdum deque is much more than is needed here (deque needs a doubly-linked list to be implemented in the most efficient manner). the pair of `(first-cell, last-cell)` is a _queue_. the list literal thing is a non-issue, we'd just need to always make a copy of it, on introduction. :) (I dislike the page you linked, for its calling that paper "a random paper from the 70s"). nothing random about it or its authors. :) – Will Ness May 27 '21 at 18:36
  • @WillNess -- if you want to be able to _remove_ the last element from the list in O(1), you need to maintain a pointer to the item preceding the last element of the list, too, which is why I was thinking of a deque. It all depends on what OP _actually_ wants to do. – ad absurdum May 27 '21 at 20:10
  • I'm not sure what you mean by "_literal thing is a non-issue_"; of course you can copy the list literal, but if you copy it every time you call your appending procedure, then you have O(n) appending. Calling it on introduction could mean that you require the programmer to think of it (fine), or maybe you create another data structure, say using a closure, to capture the copied list? – ad absurdum May 27 '21 at 20:11
  • to *remove* from the end would make it a *deque* indeed. :) --- no, I just mean `(define x (copy-list '(1 2 3)))`. – Will Ness May 27 '21 at 20:16
  • @WillNess -- Oh, I see. Well, that caveat is exactly what I meant by "_you have to pay attention to the provenance of lists given to append-item!_, " i.e., you must know where the input list came from. Looking back at my answer, the final sentence about "_no way to get the last pair ... without walking the list_" was too broad; I'm not sure why I wrote that since I already had mentioned deques with tail pointers in mind. I'll do some updating. – ad absurdum May 27 '21 at 20:24
  • what I meant was exactly to counter this, since if we follow the discipline of _always_ `copy-list`'ing (or what's the Scheme equivalent) the literals on variables introduction, we don't have to concern ourselves with that. – Will Ness May 28 '21 at 08:21
  • @WillNess -- I am not sure that I can agree with you on this. For a one-off appending operation, none of this matters: use `append`. When you need to do this a lot, exposing mutation to the programmer adds complications. That may be fine, but this depends on external factors, e.g., use case, programmer expectations. When `append` is too expensive, I think that it is generally better to choose a better data model (which may involve mutation behind the scenes). It doesn't have to be a deque, but any data structure will bring its own trade-offs. – ad absurdum May 28 '21 at 11:46
  • "exposing" etc. yes, that's (among other reasons) why SICP advocates for hiding the details behind the interface functions. – Will Ness May 28 '21 at 12:05
1

Appending a single element to a list is linear in the list size, so it is not a "common" operation.

One can use last and (setf car) though:

(defparameter *l* (list 1 2 3))
==> *L*
* (setf (cdr (last *l*)) (list 4))
==> (4)
* *l*
==> (1 2 3 4)

If you really want to append at the end, you might want to use vector-push-extend instead (which works with extensible arrays rather than lists).

sds
  • 58,617
  • 29
  • 161
  • 278
1

The reason is that a list is a singly linked list. Such structures can remove or add to the beginning in O(1), while doing the same to the end means recreating the list or at least iterating it and mutate the tail.

If you are builing a list it is much better to build it in reverse and then do a single reverse on the result. eg.

(define (add-1 lst)
  (define (helper lst acc)
    (if (null? lst)
        (reverse acc)
        (helper (cdr lst) (cons (+ 1 (car lst)) acc))))

  (helper lst '()))

Now the end result of this is O(n) while if I instead used (append acc (list (+ 1 (car lst)))) and not use reverse the result will be the same, but it would create a new fresh list with 0...n elements and all but the last one will need garbage collection. The time complexity is O(n^2) because append is O(n) and it is done for each element. As the lists grow larger the append version will perform very poorly very quickly.

Sylwester
  • 47,942
  • 4
  • 47
  • 79
  • this is extremely confusing/confused. mutating the tail is wholly separate from finding it. `append ... (list ...)` should be faster and easier on GC than the `helper-reversing` version. _both_ are O(n). repeated use of _both_ is quadratic. – Will Ness May 27 '21 at 16:41
  • @WillNess `append`-ing a one element list to the end in each iteration makes it quadratic. TRMC is O(n), but it being `sicp` I'm guessing mutation is not the order of the day at this point. – Sylwester May 27 '21 at 23:45
  • the question doesn't ask about nor even mentions anything about repeated iterations. it asks about appending one element to the end of a list. – Will Ness May 28 '21 at 09:36
0

Here is an example way that you could implement this:

(define (list-append lst elem)
  (if (null? lst)
      (cons elem nil) ; for empty element, extend by the list of elem
      (cons (car lst) (list-append (cdr lst) elem))))

(list-append (list-append 1-to-3 77) 200)
; (1 2 3 77 200)
David542
  • 104,438
  • 178
  • 489
  • 842
  • in Racket this is actually a fine code to write. the built-in `append` might perform better than this, when called as `(append lst (list elem))`, or it might not, depending on details of implementation. re the tail-mutating code like in my answer on the Q&A entry I linked to in one of my comments here (and in sds's answers in both entries (there and here), though in Common Lisp), I remember someone doing the tests and it was twice faster than the next fastest version (which could be just like the one here, IIRC). – Will Ness May 28 '21 at 08:14