Using RxJS to build data from multiple API calls

Question

I'm trying to get a better understanding of how to use RxJS Operators to solve a specific problem I'm having. I actually have two problems but they're similar enough.

I'm grabbing a bunch of documents from an API endpoint /api/v3/folders/${folderId}/documents and I've setup services with functions to do that, and handle all of the authentication etc.

However, this array of document objects does not have the description attribute. In order to get the description I need to call /api/v3/documents/${documentId}/ on each document from the previous call. My document interface looks like this:

export interface Document {
  id: number;
  name: string;
  description: string;
}

I'm thinking I need to use mergeMap to wait and get the documents some how to add in the description onto my Document interface and return the whole thing, however I'm having trouble getting the end result.

getDocuments(folderId: number) {
    return this.docService.getDocuments(folderId, this.cookie).pipe(
      map(document => {
        document; // not entirely sure what to do here, does this 'document' carry over to mergeMap?
      }),
      mergeMap(document => {
        this.docService.getDocument(documentId, this.cookie)
      }) // possibly another map here to finalize the array? 
    ).subscribe(res => console.log(res));
  }

This may seem like a bit of a duplicate but any post I've found hasn't 100% cleared things up for me.

Any help in understanding how to properly use the data in the second call and wrap it all together is much much appreciated. Thank you.

Thanks to @BizzyBob, here's the final solution with an edit and explanation:

  getDocuments(folderId: number) {
    const headers = new HttpHeaders({
      'Content-Type': 'application/json',
      'Context': this.cookie$
    });
    return this.docService.getDocuments(folderId, this.cookie$).pipe(
      mergeMap(documents => from(documents)),
      mergeMap(document => this.http.get<IDocument>(
        this.lcmsService.getBaseUrl() + `/api/v3/documents/${document.id}`,
        { headers: headers }
      ).pipe(
        map(completeDoc => ({...document, description: completeDoc.description}))
      )),
      toArray()
    ).subscribe(docs => {
      this.documents = docs;
      console.log(this.documents);
    }
    )
  }

For some reason I was unable to pipe() from the second service subscription so I ended up having to make the http.get call there. The error was "Cannot use pipe() on type subscription" which is a bit confusing, since I'm using pipe() on the first subscription. I applied this change to my service function that updates a behavior subject and it's working perfect. Thanks!

Does this answer your question? [Need Rxjs example of loading and enriching list of items in single pipe](https://stackoverflow.com/questions/59936346/need-rxjs-example-of-loading-and-enriching-list-of-items-in-single-pipe) — frido, Oct 09 '20 at 18:51
Yeah, it's the same question. Note that you have to use the `forkJoin` approach if you want to guarantee that the order of the items in the final `docs` array is the same as the order of the items in the array emitted by `this.docService.getDocuments(folderId, this.cookie$)`. — frido, Oct 12 '20 at 14:39
Ah, thanks! I'm about to do some more thorough testing with the actual data. Hopefully everything works as well. Thanks again! — qbert, Oct 12 '20 at 15:33

score 2 · Accepted Answer · answered Oct 10 '20 at 05:00

There are a couple different ways to compose data from multiple api calls.

We could:

use from to emit each item into the stream individually
use mergeMap to subscribe to the secondary service call
use toArray to emit an array once all the individual calls complete

  getDocuments() {
    return this.docService.getDocuments().pipe(
        mergeMap(basicDocs => from(basicDocs)),
        mergeMap(basicDoc => this.docService.getDocument(basicDoc.id).pipe(
          map(fullDoc => ({...basicDoc, description: fullDoc.description}))
        )),
        toArray()
    );
  }

We could also utilize forkJoin:

  getDocuments() {
    return this.docService.getDocuments().pipe(
      switchMap(basicDocs => forkJoin(
        basicDocs.map(doc => this.docService.getDocument(doc.id))
      ).pipe(
        map(fullDocs => fullDocs.map((fullDoc, i) => ({...basicDocs[i], description: fullDoc.description})))
      )),
    );
  }

Here's a working StackBlitz

Also, you may consider defining your stream as a variable documents$ as opposed to a method getDocuments().

  documents$ = this.docService.getDocuments().pipe(
    mergeMap(basicDocs => from(basicDocs))
    mergeMap(basicDoc => this.docService.getDocument(basicDoc.id).pipe(
      map(fullDoc => ({...basicDoc, description: fullDoc.description}))
    ))
    toArray()
  );

Thank you _so_ much! Your explanation of the various parts really helped me. @coderrr22 got me on the right track but your answer ultimately solidified things for me. I was never utlizing `from()` and I was never getting the second `pipe()` and `map()` right. Really appreciate it. Will edit my question with the finalized solution. — qbert, Oct 12 '20 at 12:51
Also, if you have time, could you explain why I was unable to use `pipe()` on the second subscription without making the `get` call rather than using the method from the service? — qbert, Oct 12 '20 at 13:23
The reason you couldn’t access data from the first call in the second call was that they weren’t part of the same closure. Adding an additional ‘.pipe()’, puts the results of both calls in the same scope so so you have access to both of them. Check out this answer (and the question) for more details: https://stackoverflow.com/a/63462962/1858357 — BizzyBob, Oct 12 '20 at 13:44

coderrr22 · Answer 2 · 2020-10-09T19:44:26.317

Since you must call the document/{id} endpoint to get the description, For each document You will eventually make rest calls as the nunber of documents you have..

mergeMap returns an observable and subscribes to it on outer observable emit. (Keeps the inner subscriptions live in contrast of switchMap) https://www.learnrxjs.io/learn-rxjs/operators/transformation/mergemap

Rxjs from takes an array and returns an observable, by emitting every item of the array in sequence. https://www.learnrxjs.io/learn-rxjs/operators/creation/from

So we can use both in order to achieve your goal like so :

(Sorry for the bad formatting im using my phone)

I assumed that docService.getDocuments returns an array of documents.

getDocuments(folderId: number) { 
    return this.docService.getDocuments(folderId, this.cookie).pipe(
      mergeMap((allDocumentsArray)=> {
             return from(allDocumentsArray).pipe(
               mergeMap((document) =>        
                   this.docservice.getDocument(document).pipe(
                        map(doc) => {...document,description})),//maps the document with document.description
     toArray(),//combines the returned values to an array
    ).subscribe(res => console.log(res));

I recommend reading about mergemap in their documentation. And test this before using cause i havent tested.

Thank you, your comment got me going on the right track! – qbert Oct 12 '20 at 12:49 — qbert, Oct 12 '20 at 12:49

Using RxJS to build data from multiple API calls

2 Answers2

Linked