3

Given: Each call to a BE module takes several seconds even with a SSD drive. (A well configured setup runs below 1 second for general BE tasks.)

  • What are likely bottlenecks?
  • How to check for them?
  • What options to speed up?

On purpose I don't give a special configuration, but ask for a general checklist, so that the answer is suitable for many people as first entry point.

Blcknx
  • 1,921
  • 1
  • 12
  • 38
  • 1
    Without more details I can only give general answers: run in production mode, add more memory, add faster/more CPU cores, optimize PHP/webserver/database, add caching server – Rudy Gnodde Dec 18 '18 at 10:54
  • I am asking for general answers applicable to a large audience. – Blcknx Dec 18 '18 at 10:54

4 Answers4

2

General tips on performance tuning for TYPO3 can be found here: https://wiki.typo3.org/Performance_tuning

However, in my experience most general performance problems are due to one of a few reasons:

  1. Bad/no caching. Usually this is a problem with one or more extensions (partly) disabling cache. Try disabling all third party extensions and enabling them one by one to see which causes the site to slow down the most. $GLOBALS['TSFE']->set_no_cache() will disable all cache, so you could search for that. USER_INT and COA_INT in TypoScript also disable cache for anything that's configured inside there.
  2. A lot of data. Check the database for any tables containing a lot of data. How many constitutes "a lot", depends on a lot of factors, but generally anything below a million records shouldn't be too much of a problem unless for example you do queries with things like LIKE '%...%' on fields containing a lot of data.
  3. Not enough resources on the server. To fix this, add more memory and/or CPU cores to the server. Or if it's a shared server, reduce the number of sites running on it.
  4. Heavy traffic. No matter how many resources a server has, it will always have a limit to the number of requests it can process in a given time. If this is your problem you will have to look into load balancing and caching servers. If you don't (normally) have a lot of visitors, high traffic can still be caused by robots crawling your site too quickly. These are usually easy to block on IP address in your firewall or webserver configuration.

A slow backend on a server without any other traffic (you're the only one who can access it) rules out 1 (can only cause a slow backend if users are accessing the frontend and causing a high server load) and 4 (no other traffic).

Rudy Gnodde
  • 4,286
  • 2
  • 12
  • 34
2

one further aspect you could inspect: in the user record a lot of things are stored, for example the settings you used in the log module.
one setting which could consume a lot of memory (and time to serialize and deserialize) is the state of the pagetree (which pages are expanded/ which are not).

Cleaning the user settings could make the backend faster for this user.
If you have a large page tree and the user has to navigate through many pages the effect will stall. another draw back: you loose all settings as there still is no selective cleaning.

Bernd Wilke πφ
  • 10,390
  • 1
  • 19
  • 38
2

Cannot comment here but need to say: The TSFE-Object does absolutely nothing in the TYPO3 Backend. The Backend is always uncached. The TYPO3-Backend is a standalone module to edit and maintenance the frontend output. There are tons of Google search results that will ignore this fact.

Possible performance bottlenecks are poor written extensions that do rendering or data processing. Hooks to core functions are usually no big deal but rendering of many elements for edit forms (especially in TYPO3s Fluid Template Engine) can cause performance problems.

The Extbase-DBAL-Layer can also cause massive performance problems. The reason is the database model does not know indexes. It' simple but stupid. A SQL-Join on a big table of 2000 records+ will delay the output perceptibly, depending on the data model.

Also TYPO3 Backend does not really depend on the Typoscript-Configuration but in effect to control some output or loaded by extensions, the full parsing of the *.ts files is needed. And this parser is very slow.

If you want to speed things up you need to know what goes wrong. The only way to debug this behaviour is to inspect the runtime with a PHP profiling tool like xdebug because the TYPO3 Framework is very complex. It's using some kind of Doctrine Framework and will load tons of files, by every request. Thus a good configured OpCache is a must.

Most reason the whole thing is slow is because it is poor written. You can confirm that fact by inspecting the runtime.

Coderoy
  • 21
  • 1
  • 1
    I like to bust the myth, that the TypoScript parser is slow. Due to this myth I wrote a new https://github.com/elmar-hinz/TX.tsp. When I actually measured the speed of the current parser, my results told me that it is quite fast itself. So I did not follow this any further. IMHO what is slow, is the PHP code that is executed after TypoScript has been parsed, i.g. Extbase. It's a pity that the TypoScript parser is accused for slowness that is not caused by it. – Blcknx Dec 29 '18 at 13:16
  • Thank you for registering to answer my question. Good start. – Blcknx Dec 29 '18 at 14:18
  • @Blcknx I did mention the parser as whole parsing process. It may be fast for standalone parts but this does not count if the already processed resources will not be stored for fast access later. There is not even a framework cache for the TypoScript by default, it is parsed by every request instead. Yes, there is a little serialization but its nearly useless because it only stores the TypoScript serialized again, not parsed and processed fragments of them. – Coderoy Jan 01 '19 at 15:43
  • Also serialization beats parsing of PHP scripts but Fluid still using PHP files as cache. That are things I do not understand. TYPO3 could be very fast but developers are unable to profile there classes and workflow right. Optimizing is the key to success here. I did a quick overview over your code and saw some regex parts. Regex is not fast at all for such a parser. Using tokenizer would be much faster if you are using them right. Things like calls to count() within a loop are also typical areas to optimize. Every ms is important in a single threaded application. – Coderoy Jan 01 '19 at 15:44
  • To understand why so many obvious developments are so slow, you have to understand the mentality of the core team. It just does not scale. It is not capable to really integrate all the knowledge and power that is around in the community. It's not a question of technology. It's a mere question of social dynamics. In this social dynamics also is the answer, why the community is almost limited to Germany. – Blcknx Jan 01 '19 at 21:26
  • As long as the TYPO3 response needs over 1000 ms there are parts, that are more urgent to improve than the TS-parser that is just taking a few ms. The first step would be to profile, where most of the time is lost. That's why I did not try to optimise the TS-parser by tokenisation. – Blcknx Jan 01 '19 at 21:33
  • As for the Regex. Even Flex uses them for tokanisation. See https://en.wikipedia.org/wiki/Flex_(lexical_analyser_generator). They are not slow in in general. They can get slow, if applied poorly. You may take my code and try to figure out, if it is possible to tokenise TypoScript without and if that is faster. Also see: https://stackoverflow.com/questions/5892115/whats-the-time-complexity-of-average-regex-algorithms – Blcknx Jan 01 '19 at 22:00
  • As long as it works I will not say anything against useful regex but it is slower than dumb string functions in php. It also depends on what you want to achieve with. I personally would not use it for a complex script parser because the regex may not good enough to catch valid inputs. Of course, if you already tokenizing with your own logic and then use it to validate the strings, it will make a huge difference. I fully agree with the social dynamic of the TYPO3 core team. Every developer that finds a bug or wants to implement a better solution has to deal with it :\ – Coderoy Jan 02 '19 at 23:57
1

In addition to what already has been said, put the runtime environment onto your checklist:

Memory:

If heavy IDE and other tools are open at the same time, available memory can become an issue. To check the memory profile, you may start a tool that monitors the memory usage of the machine.

If virtualization is used, check the memory assigned to the box. Try if assigning more memory improves behaviour.

If required and possible spend more memory to your machine. This should not be a bugfix to poorly written code. Bad code can blow up any size of memory.

File access:

TYPO3 reads and writes thousands of files. If you work with a contemporary SSD, this is surprisingly fast. I did measure this. Loading all class files of TYPO3 takes just a fraction of a second.

However this may look different if you do not work with a standard setup. Many factors may slow you down:

  • USB-Sticks as storage.
  • Memory cards as storage.
  • All kind of external storage may be limited due to slow drivers.
  • Virtualization can become an issue. Again it's a question of drivers.

In doubt test and store your files and DB on a different drive to compere the behaviour.

Routing

The database itself may be fast. A bad routing of your request may still slow you down. Think of firewalls, proxies etc. even on your local machine and specially if virtualisation is used.

Database connection:

I fast database connection is crucial. If the database access is slow TYPO3 can't be fast.

Especially due to Extbase TYPO3 often queries much more data than really required and more often than really required, because a lot of relations are resolved in the PHP layer instead of the DB layer itself. Loading data structures like the root line may cause a lot of ping-pong between the PHP and the DB layer.

I can't give advice, how to measure your DB-connection. You have to as your admin for that. What you always can do is to test and compare with another DB from a completely different environment.

The speed of the database may depend on the type of the database itself. Typically you use MySQL/Maria-DB which should be fast. It also depends on the factors mentioned above, memory, file access and routing.

Strategy:

Even without being and admin and knowing all performance tools, you can always exchange parts of your system and check if matters improve. By this approach you can localise the culprit without being an expert. Once having spotted the culprit, Google may help you to get more information.

When it comes to a clean and performant setup of routing or virtualisation it's still the best idea to ask an experienced admin.

Summary

This is all in addition to what others have already pointed to.

What would be really helpful would be a BE-Plugin, that analyses and measures the environment. May there are some out there I don't know.

Blcknx
  • 1,921
  • 1
  • 12
  • 38