Selenium performance issue(1000+ instances)

Asked Oct 18 '18 at 17:50

Active Oct 18 '18 at 18:17

Viewed 1,148 times

I need to run 1000 selenium instances for my project.

Every chrome session eats up 300-500mb memories, that means 300-500gb memories in total. My plan is to buy 3 servers with 128gb memories to run it.

I have already bought a second-hand server(cpu: Xeon E5670x2, ddr3 144gb) for the testing. and set 35 nodes in the server, set maxinstances to 10 for every node. so the plan is to bring up 350 instances in this server.

The problem now is the CPU already reached to 100% when the server runs about 100 instances.

I have already tried below things:

1, chrome headless

2, disable images

3,
optionChrome.AddArgument("--disable-dev-shm-usage"); optionChrome.AddArgument("--no-sandbox"); optionChrome.AddArgument("--headless"); optionChrome.AddArgument("--disable-gpu"); optionChrome.AddArgument("--disable-infobars");

What can I do next to improve the performance?

asked Oct 18 '18 at 17:50

ktoya

1

Why not just rent cycles in the cloud? There are several options that already have Selenium all set up for you. – SiKing Oct 18 '18 at 19:09
Checkout this discussion [Limit chrome headless CPU and memory usage](https://stackoverflow.com/questions/50701824/limit-chrome-headless-cpu-and-memory-usage) – undetected Selenium Oct 18 '18 at 19:27
Do you need all 1000 of them to run perellel? – Moshe Slavin Oct 18 '18 at 19:57

Selenium performance issue(1000+ instances)

0 Answers0