4

I have been trying to learn NodeJS for quite some time now. All the books and tutorials seems to follow similar pattern of structuring their code. Example -

const express = require('express');

const app = express();
app.set('view engine','hbs');

app.get('/', (req, res) =>{
    res.render('index');
});

app.get('/getName', (req, res) =>{
    // Mock DB call to fetch Name
    res.render('displayName');
});


app.listen(3000, () => {
    console.log("Started server on port : 3000");
});

As you can see above, the /getName controller is performing DB call as well as returning the view. So the business logic as well as the CRUD operation is being done at the same place.

I come from the world of JAVA and there we do it slightly differently. For example, a Spring Boot application would contain the following structure -

  • DTO
  • Repository
  • Service
  • Controller

So, controller classes are the actual endpoints which do not perform any business logic but call the underlying service class to handle all that. The service classes implement the business logic and persist/fetch data required for it with the help of the repository classes. The repository on the other hand handles the CRUD operations.

This seemed like a sane way to develop software. Given that each class has their defined roles, it becomes really easy to handle any change.

I understand the NodeJs is a dynamic but -

1. Is there a way to separate out the functionality like we do in Spring ? If not,

2. How to structure large projects having multiple endpoints and DB transactions.

Regards


EDIT -

Consider the following scenario -

I have a requirement, where I need to fetch a list of users from the database whose status is True ( assume status is a boolean field in the model ).

In Java -

@Service
public class UserService {

    @Autowired
    UserDetailRepository userDetailRepository;

    @Override
    public UserDetail getUserDetail (boolean status) {
        UserDetail userDetail  = UserDetailRepository .findUserDetailByStatus(status);
        return userDetail  ;
    }

Controller.java -

@GetMapping("/getUserDetails")
        public ArrayList<UserDetail> getUserDetail(){

        return UserService.getUserDetail(true);
}

Now, if the requirement changes and there needs to be a new endpoint that returns only top 10 user details whose status is true. In that case, we can add a new controller and just limit the returned results to 10. We can make use of the same business logic/ service class.

Controller.java

@GetMapping("/getUserDetailsTop10")
            public ArrayList<UserDetail> getUserDetail(){

            List<UserDetails> users = UserService.getUserDetail(true);
            // Filter the list to send top 10
            // return users .
    }

If I have to implement the same use case in NodeJS, I'll have to write the business logic to fetch the user twice -

const express = require('express');

const app = express();
app.set('view engine','hbs');

app.get('/getUserDetails', (req, res) =>{
    // Call DB and get users whose status is True
    res.send(userdetails);
});

app.get('/getUserDetailsTop10', (req, res) =>{
    // Call DB and get users whose status is True
    // Filter the returned results and limit the result to 10 values only
    res.send(userdetails);
});


app.listen(3000, () => {
    console.log("Started server on port : 3000");
});

At best, I can abstract away this logic into a function, that will return a list of users with status True but then again this approach is not very scalable. There has to be a complete separation of Business logic from controller.

Boudhayan Dev
  • 970
  • 4
  • 14
  • 42
  • Please take the [tour] (you get a badge!), have a look around, and read through the [help], in particular [*How do I ask a good question?*](/help/how-to-ask), [*What types of questions should I avoid asking?*](/help/dont-ask), and [*What topics can I ask about here?*](/help/on-topic) This question is **far** to broad and open-ended for SO's Q&A format. – T.J. Crowder Apr 09 '19 at 16:30
  • Hi @T.J.Crowder. I can re-format the question if its needed. But I believe I am quite specific in my question. I am asking how can I refactor the above code by using layered architecture. If it is not possible, that's fine but if in case it is, I would like to know how. – Boudhayan Dev Apr 09 '19 at 16:33
  • 2
    The format of the question isn't the problem, the content of the question is. There are hundreds of different possible equally-valid answers. The Node.js ecosystem is rich and varied, there's no one way you do this. Please do see the links above, in particular [this one](/help/dont-ask). It's not that it's not a perfectly reasonable question to ask, it's just not suited to the format of *this* site. – T.J. Crowder Apr 09 '19 at 16:37
  • Although I'd personally love to see some discussions on this topic, this question is indeed quite broad, that could need a series of blogs to properly answer. Paradigm introduce by framework like spring boot is more or less language agnostic. You might as well ask, how is spring boot designed so to make such layered architecture possible, in java, without even mentioning js. Knowing that answer, I could probably write a similar framework from scratch in js then. – hackape Apr 09 '19 at 16:48
  • @hackape makes sense. But in that case, how do you handle large NodeJS projects ? Or NodeJs not used at the same scale as that of Java ? – Boudhayan Dev Apr 09 '19 at 16:52
  • Not like it's a bad question through. @TJCrowder Just curious, where do you suggest to be a proper place to raise a question like this one? – hackape Apr 09 '19 at 16:53
  • 1
    similar question is discussed to some extent here https://stackoverflow.com/questions/5178334/folder-structure-for-a-node-js-project – 1565986223 Apr 09 '19 at 16:54
  • Hi @naga-elixir-jar, This is more of code organization rather than logic separation. The top comment suggests we move our ORM models to a separate folder and express endpoints to controller. But the original problem still persists. The Business logic to handle the DB call is still being performed in the controller. – Boudhayan Dev Apr 09 '19 at 17:02
  • 1
    Hmm, I'm not the best guy to answer your question. Not that experienced in backend js, but I'll try. Js community embrace things that's light-weighted. We got runtime that has native support for first class function, event loop and such. This way we tend to naturally seek solution in patterns like pipeline chaining, observable stream, pub/sub. We too have separation of concern, but just not like java style. – hackape Apr 09 '19 at 17:03
  • Take express.js app for example. You can have multiple layers of middlewares handling one request. You can delegate to external service in middleware. You could say that our style is more functional-ish, as opposed to object oriented. – hackape Apr 09 '19 at 17:07
  • @hackape . I made myself familiar with those patterns but somehow I still can't understand how Node handles code re-use. For ex- in Java, Say I have a service class which returns Users whose status is True. And in one controller class, I will be returning that. Now, if need arises to include another controller class that needs to return users whose status is True but only limit the returned values to top 5, then that controller ca re-use the existing service. I don't know how this can be done in Node though !! Only way is to create a function which can be referenced by multiple endpoints. – Boudhayan Dev Apr 09 '19 at 17:11
  • Found similar discussion on SO - https://stackoverflow.com/questions/41875617/building-enterprise-app-with-node-express – Boudhayan Dev Apr 09 '19 at 17:17
  • ^ you know what, this is a concrete question to ask. Why not rephrase your question and add some java code example, then we discuss what's the mainstream js way to tackle the code re-use problem. – hackape Apr 09 '19 at 17:18
  • @hackape Edited. Please take a look. – Boudhayan Dev Apr 09 '19 at 17:34
  • 1
    "*Only way is to create a function which can be referenced by multiple endpoints.*" - yes, that's precisely what you would do. (When need arises, otherwise YAGNI). You can put such functions in their own modules, files, or even write them as classes if you prefer that. I don't understand why you think this is not sufficient or scalable, or why you don't consider it to separate concerns. – Bergi Apr 09 '19 at 17:56
  • @Bergi The reason is simple. Either I will have all or none. I do not want to hand-pick functions for pieces of logic that I feel could be re-used in the future. Either that should be the norm i.e write the business logic in functions and call them from the controllers or we do not have them at all and continue with the existing model. Selecting pieces of code here and there and wrapping them into functions does not actually seem very nice. There has to be a standard way to do this. – Boudhayan Dev Apr 10 '19 at 02:41
  • @BouhayanDev one of the thing that throw me away in java is, you're *forced* to create a class before using a function. I don't get the point why *grouping* functions within class suddenly make it more reusable. I'm happy with scattered functions loosely grouped by well-named files, aka modules in js. Don't take me wrong, it's just I genuinely don't see the difference. What's the fundamental upside of using a class as a "foldler" against modules. – hackape Apr 10 '19 at 03:49
  • However I do see upside of this function-all-over-the-place approach: I get to compose/composite them freely w/o worrying about their owning class. To me that seems much more reusable. – hackape Apr 10 '19 at 03:52
  • @hackape It just derives from the OOPS concept where the reasoning goes something like this - functions are analogous to behaviors whereas classes are analogous to real-life entities (ex TV, projector etc). Now, does it make sense to allow functionality without defining the container/owner of the functionality ? It doesn't. Hence this bundling of functions in class. However, I will agree with you, functionality wise there does not seem to be an issue whether I use module/class to hold my functions – Boudhayan Dev Apr 10 '19 at 03:52
  • @BoudhayanDev my day job is authoring large scale web app, like WebIDE. I also find it more natural in many scenarios to use class, esp when exposing APIs to other parties. This upside roots in 2 reasons, a) state persistency, b) classes are indexed/symbolised/searchable in IDE, while files are not. – hackape Apr 10 '19 at 04:12
  • 2
    I think you are feeling conceptual discomfort facing multi-paradigm goodness in js land. Java does have some burden going functional programming, but not in js. FP or OOP, I just go with whichever fits, depending on use case. – hackape Apr 10 '19 at 04:17
  • Yes that is the case here. i was thinking of replicating an existing Spring application using Node. But seems like Node was coupling all the layers into one single file. But I'll try destructuring it. – Boudhayan Dev Apr 10 '19 at 04:42
  • @BoudhayanDev "*Either I will have all or none.*" - well if you want, you can of course always put everything in separate functions. It's just a convention, and you are encouraged to do what you feel comfortable with. The advantage of a loose style (instead of being forced by a framework to follow a particular model) is that it scales much better from simple and inlined to complex and separated. You only use what you need. – Bergi Apr 10 '19 at 08:06

4 Answers4

3

Not really an answer... but some thoughts, as I come from C# and started as a NodeJs developer 3 years ago.

Dependency Injection

This is (sadly) rarely used in many NodeJs projects that I see. There are some npm modules that provide this feature, but none of them have convinced me with a proper approach and API. Dependency Injection is still perfectly possible, only, a bit uglier because of some boilerplate code you have to write. So sadly no easy @autowire approaches. So yes you can do Dependency Injection like with Spring, but not as convenient. But let my opinion not stop you from researching Dependency Injection libraries for node.

Architecture

You can still use the concepts of DTOs, Repositories, Services and Controllers. Only for some (odd) reason, the majority of tutorials and guides out there forget about common sense architecture and just throw everything within a single controller. Don't let them seduce you in forgetting about the concepts you learned in Java. Not that OOP and Java doesn't have flaws, but there is a difference in "writing JavaScript style code" and "let's forget about proper architecture all together".

Do note that you might see some more "functional programming" patterns rise in the NodeJs community, which is not bad (but neither is OOP).

How to structure large projects having multiple endpoints and DB transactions.

Ignore the bad examples you see out there, just follow your gut feeling from your Java experience how to structure your code, with the right layers, responsibilities and Design Patterns. Only keep in mind that you don't have interfaces and there is no easy alternative to that. So you'll have to learn to work your way around that, but you can still write elegant code without them.

Middleware

Middleware is a common concept for NodeJs applications, they are similary-ish to the proxy/aspect oriented programming style. You might find example projects with excessive amount of middleware, don't let yourself be tempted by that either. Middleware is good for auth/serialisation/headers/security but not for business logic. I've seen middleware hell and middleware driven development, it ain't pretty.

Repository

A lot of people use directly the MongoDb/Mongoose/SQL client without using the adapter pattern like Repository, leaking MongoDb queries all over their solution.

My advice for you would be, just use the same structure and approach you are familiar with, but with the tools that JavaScript gives you. I genuinely love JavaScript, but it makes me cringe when I look at the lack of design and architecture that I see in the mainstream resources, sure small projects and PoC's don't need extensive design, but often when project grow, they don't clean up. How you structure a project should be agnostic to the language and framework you are writing in.

Happy coding

Segers-Ian
  • 1,027
  • 10
  • 21
  • Hi @Ian Segers , totally agree with you. I could go ahead and abstract the business logic in a separate module and then call from the controllers and that would work. But I would love to see NodeJs tutorials actually enforcing this. However, I am not sure how would I separate out the DB queries from the separate module (service functions). I would love to see a well structured NodeJS project that follows these coding standard. – Boudhayan Dev Apr 10 '19 at 03:22
1

UserService.js

import { userDetailRepository } from '../models/user';

class UserService {
    getUserDetail (status) {
        const userDetail  = UserDetailRepository.findUserDetailByStatus(status);
        return userDetail;
    }
}

export const userService = new UserService();

UserController.js

import { userService } from '../services/user';

@RouterAdapter
class UserController {
  @GetMapping("/getUserDetails")
  getUserDetail() {
    return userService.getUserDetail(true);
  }
  @GetMapping("/getUserDetailsTop10")
  getUserDetailsTop10() {
    return userService.getUserDetail(true).slice(10);
  }
}

export const userController = new UserController();

app.js

import * as express from 'express';
import { userController } from './controllers/user';

const app = express();
app.set('view engine','hbs');

app.use(userController);

app.listen(3000, () => {
    console.log("Started server on port : 3000");
});

I intentionally "warp" my js code to match with java style so perhaps you feel at home. Personally I don't write js this way, I prefer using function way more than class.

I did not include the implementation of two decorators (@RouterAdapter, @GetMapping) I used, that's a detail not related to the discussion. I can assure you this is doable in js. The only thing missing is that js doesn't have runtime method overload, so I have to name 2 diff controller methods.

The point I'm making here is that design patterns are mostly language agnostic. If you're familiar with the lang, you can always find a way to handle separation of concern well.

Now I intentionally use class in above example, but a function acting as a UserService doesn't make it less reusable. I don't see why you think "it doesn't scale".

hackape
  • 18,643
  • 2
  • 29
  • 57
  • This would definitely work. But yes, I'll agree with you here that using classes to structure code in Node seems an ugly version of Java :p What I would like to know, is this really how large projects in Node are handled ? Or am I wrong in assuming the importance of layered architecture. Also, by 'it doesn't scale' in my earlier comment I meant that, for me to separate out the logic from controller, I will need to write all that logic in a service class/function. But this abstraction needs to be done for all cases where there is a business logic. We can't cherry-pick function as we want – Boudhayan Dev Apr 10 '19 at 03:29
  • Also, what are your thoughts regarding the DB related ops ? Like if I have to persist/fetch something from DB, I will invoke the sql client connector and execute the query in NodeJs. This piece of logic can go directly in the service class. Is there a way to abstract this as well ? So, I am assuming to mock the repository behavior, we can create another module that basically has all the CRUD operations defined for a particular Entity in the form of functions. And these functions are then invoked in service layer. Then there would be clear separation of - Controller -> Service -> Repo ? – Boudhayan Dev Apr 10 '19 at 03:32
  • Lastly, why don't NodeJs tutorials/resources talk about this ? Is it because Node isn't used for large scale projects where such problems would be encountered or there is an alternative strategy to handle such problems ? – Boudhayan Dev Apr 10 '19 at 03:35
  • @BoudhayanDev 1. Hopefully answered in the comments under the question, don't see why cherry-pick fn unacceptable, maybe you just want to pass a bunch of related functions all at once, in that case you can wrap them in a plain object. – hackape Apr 10 '19 at 04:27
  • @BoudhayanDev 2. I personally agree w/ your "assumption", but again, im more a frontend guy, not best guy to answer. – hackape Apr 10 '19 at 04:29
  • @BoudhayanDev 3. 2 reasons, 1st maybe they did talk about these, like in the other SO post you found, but you just don't recognize cus them looks unlike spring boot. 2nd such "rigid" and commonly agreed approach doesn't exist in js, in java this is prominent largely because spring boot is so dominant, and plus it's battery included. Express.js is dominated but no battery included, so everyone brings their own battery. Because the ease of multi-paradigm in js. People tend to use whatever solves their problem. Spring boot-ish setup for small project is simple overkill. – hackape Apr 10 '19 at 04:36
  • And btw you're wrong about nodejs not for large-scale. taobao.com (Chinese amazon) use nodejs extensively. Imagine their PV and complexity. – hackape Apr 10 '19 at 04:42
  • Like I said, I do not have much experience in Node so cannot say whether it is the right tool for big projects but having read your answers I agree that Node/Express has its own way to handle things. This reminds of another framework that I tried out couple of years back - called Flask (in Python). This pattern of not enforcing strict responsibilities to classes/functions seems to be common in all dynamically typed languages. I guess I'll just adapt to the Node way of doing things rather than trying to complicate it by following Java's code structuring. – Boudhayan Dev Apr 10 '19 at 04:48
  • @Boudhayan Dev yeah sure. It's always good to try out new stuff, broadens your horizon. – hackape Apr 10 '19 at 04:56
  • Yes, just do it nodejs way. By interpolating another layer of complexity with the node apps to follow some other architectural principle is not justified imho. You can ofc abstract out per-Entity service layer classes, but due to what dynamic nature of language offers you, and what the driver you are using for DB is, you can also get fair amount of metadata for generating these classes. You should take a look at knex.js and see what that can do for you ;) – bigkahunaburger Apr 18 '19 at 11:02
1

An idea:

const express = require('express');

const app = express();
const { userDetails } = require('./usersBusinessLogic.js')
app.set('view engine','hbs');

app.get('/getUserDetails', async (req, res) =>{
  const limit = parseInt(req.query.limit || '0')
  if (IsNaN(limit)) {
    res.status(400)
    return res.end('limit expected to be a number')
  }
  const detailsResponse = await userDetails({ limit })
  res.send();
});

And then you write your logic as function userDetails({ limit, ...rest } = { limit: 0 }).

In another file i.e. usersBusinessLogic.js:

async function userDetails({ limit, ...restOfTheOptions } = { limit: 0 }) {
  console.log({ limit, restOfTheOptions }) // TODO logic here.
  // const result = await db.find('users', {...})
  // return serialize(result)
}

// function serialize(result) {
//  return JSON.stringify(result)
// }

module.exports = {
  userDetails
}

After which you can call GET /userDetails or GET /userDetails?limit=10.

Let me know what do you think.

bigkahunaburger
  • 426
  • 5
  • 10
  • Hi. This approach would work. At least we have separated out the BL from controller. But what would you suggest to further split out the service module ? So, in my opinion it is always better to keep the DB related queries in a separate module/class. For ex - getUsers, saveUsers, getUserByID etc are all DB (ORM or otherwise) queries. So, if I am separating this out, does it make sense to introduce another module to abstract these queries ? Because I am not sure if using modules is the right way to do it mainly because I am new to JS. But I would love to know your thoughts. – Boudhayan Dev Apr 10 '19 at 03:38
  • It's not particularly tied to modules the thing that you are trying to achieve. It's related to your architectural decisions. In my example, I have put ```await db.find('users', {...})``` in comment, but haven't said where it comes from. Certainly at the top of your module, you would have to ```require('./db')```, and in db.js module, implement your logic against database. Would it be implementing ORM or just using driver directly, it's up to you. But you would have to expose something out for BL. Hope it got that cleared out. Feel free to ask more. – bigkahunaburger Apr 18 '19 at 10:56
1

<TL;DR> You can do it with express-beans.

You can generate a project with npm create express-beans-server and try by yourself. </TL;DR>

I know, it's an old question but it is worth to respond anyway because I was used to SpringBoot and then I wrote a lot of microservices in express.js. Express is a very well designed framework because it is simple and performant but if you have to write multiple endpoints and logic can be a real mess to organize your code. Fortunately in the near future decorators will be officially part of ECMAScript standard and I wrote a simple library to wrap up an express app and use decorators like @Bean to declare a Singleton just like SpringBoot.

Just write an ExampleRouter.ts (it is typescript but it can be used also in js files):

import { Request, Response } from 'express';
import { InjectBean, Route, RouterBean } from 'express-beans';
import ExampleService from '../services/ExampleService';

@RouterBean('/example')
export default class ExampleRouter {
  @InjectBean(ExampleService)
  private exampleService!: ExampleService;

  @Route('GET', '/hello')
  getHello(_req: Request, res: Response) {
    res.end(this.exampleService.example());
  }
}

Then an ExampleService.ts as follows:

import { Bean } from 'express-beans';

@Bean
export default class ExampleService {
  private msg: string;

  constructor() {
    this.msg = 'hello world!';
  }

  example() {
    return this.msg;
  }
}

et voilà!

All information and documentation (and code) is available on GitHub with MIT license