XGBoost uses the method of additive training in which it models the residual of the previous model.
This is sequential though, how does it to parallel computing then?
Asked
Active
Viewed 2.1k times
54
-
Thanks for asking--I had the same question. – nkhuyu Apr 28 '16 at 20:37
-
1A nice blog in [here](http://www.parallelr.com/parallel-computation-with-r-and-xgboost/) to cover parallel and XGboost. – Patric Jan 28 '17 at 14:38
1 Answers
64
Xgboost doesn't run multiple trees in parallel like you noted, you need predictions after each tree to update gradients.
Rather it does the parallelization WITHIN a single tree my using openMP to create branches independently.
To observe this,build a giant dataset and run with n_rounds=1. You will see all your cores firing on one tree. This is why it's so fast- well engineered.
T. Scharf
- 4,644
- 25
- 27
-
1@T. Scharf But how does this work for multiple nodes i.e. multiple computers rather than 1 computer with multiple cores ? The amount of communication to sync the tree within a tree would be immense – AbdealiLoKo Nov 19 '16 at 13:32
-
2Yea no getting around this fact.. @AbdealiJK but if your data is so big you need to distribute it this is the price you pay – T. Scharf Nov 20 '16 at 15:27
-
1