Angular 2 recursive http.gets of unknown depth

Question

I've been using a pattern like the below to chain together http.gets in Angular2 to retrieve information from a hierarchical structure, two layers deep, of folders (all pseudotypescript):

myObservable = this.myService.getSubFolders(topFolderUrl)
    .switchMap(subFolders=> this.myService.getInfoFromSubFolders(subFolders))
    .map(subFolders=> => {
        ...do stuff with subFolders...
        return subFolders;
        }
    );

Where myService looks something like this:

getSubFolders(topFolderUrl): Observable<Folder[]> {
    return this.http.get(topFolderUrl)
        .map(res => {
            let body = res.json();
            let foldersToReturn: Folder[] = [];
            for (let subfolder of body.subFolders) {
                let tempFolder = new Folder;
                tempFolder.url = subfolder.url;
                tempFolder.otherProps = subfolder.otherPropValue;
            }
        return foldersToReturn;
        }
    .catch(this.handleError);
}

getInfoFromSubFolders(subFolders:Folder[]): Observable<Folder[]> {
    let calls: any[]  = [];

    for (let folder of subFolders:Folder){
        calls.push(
            this.http.get(folder.url)
            );

    var subject = new Subject<Folder[]>();       //see: http://stackoverflow.com/a/38668416/2235210 for why Subject

    Observable.forkJoin(calls).subscribe((res: any) => {
        let foundFolder = subFolders.find(folder=> {
                return response.url.indexOf(folder.url)!==-1;
            });
        for (let response of res){
            let bodyAsJson = JSON.parse(response._body);
            foundFolder.otherProps = bodyAsJson.otherPropValue; 
        }
    subject.next(subFolders);
    });
return subject;
}

I then subscribe to myObservable using an | async pipe in my template. Object in myObservable ends up something like:

{
  "url": "topFolderUrl", 
  "otherProps": "otherPropsValue", 
  "subFolders": [
    {
      "url": "subFolder1Url",
      "otherProps": "otherPropsValue"
    },
    {
      "url": "subFolder2Url",
      "otherProps": "otherPropsValue",
    }
  ]
}

However, this relies on this structure of folders being exactly two layers deep - no more, no less I have two related questions:

1. How would I refactor this to allow me to recursively work my way down a series of folder n layers deep - I can only request one layer at a time - i.e each subFolder has "subFolders": [] and so on.
2. How would I allow the system to cope if there were no subFolders? ie not call getInfoFromSubFolders in the .switchMap

I have a feeling this this is a very common scenario so it is likely to be useful to many people.

Any pointers gratefully received

Answer 1

I think that the expand operator could help you here since it tackles recursion in such use cases.

See this question for more details:

• Make a second http call and use the result in same Observable

Answer 2

I came across this post when breaking my head to implement an other problem to do with recursivity and Observables. Fortunately, I am able now to answer your question.

The problem to resolve Starting from a folder provided as an argument, you would like to visit all sub-folders. For every sub-folder, visit all sub-folders and repeat this process without a limit in depth.

To answer your question about refactoring and model, this problem is a typical tree traversal for which options are available for data structures.

You did not provide much detail about your Folder but let’s make some assumptions. Your class has the following properties:

• parent: Folder
• children: Folder[]

You currently have a service that returns children for a given folder. Let’s assume the method signature is:

myObservable: Observable<Folder[]> = this.myService.getSubFolders(aFolder);

An approach using recursivity

For implementing a recursive approach it helps to create a context that will contain the (under construction) result and any other properties needed for the state of the recursion: FolderTraversalContext .

processSubFolders(ctx: FolderTraversalContext): Observable<FolderTraversalContext> {
  return this.getSubFolders(ctx.folder)
    .map( folders: Folder[] => {
      folders.forEach( f => {
        f.parent = folder;
        ctx.toVisit.push(f);
      });
      folder.children = folders; 

      //Prepare for next cycle
      ctx.complete = ctx.toVisit.length() === 0;
      if (! ctx.complete) {
        ctx.folder = ctx.toVisit[0];
        ctx.toVisit.splice(0,1);
      }
      return ctx;
    }
}

Use of expand operator

The method that implements the recursion in a reactive way looks like this then:

readAllSubfolders(topFolder: Folder): Observable<Folder> {
  const ctx = new FolderTraversalContext();
  ctx.topFolder = topFolder;
  ctx.complete = false;
  ctx.toVisit = [];
  ctx.folder = topFolder;

  return Observable.of(ctx)
    .expand( ctx => {
      return (ctx.completed) ? Observable.empty() : this.processSubFolders(ctx);
    })
    .map(ctx => ctx.topFolder);
}

Allez!