0

There are many articles and books on problems in HPC, but I feel like I am missing on the diagnose of scaling and efficiency issues. For example, I am reading a books called "Introduction to High Performance Computing for Scientists and Engineers" by Horst Simon where he discusses a wide variety of problems and solutions such as,

  • Cache misses
  • Load Imbalance
  • Poor Vectorization of code
  • etc.

But if I were handed a piece of code even remotely complex (ie more than nested for-loops) I would have a very hard time discovering what the bottleneck was or proving that the code had reached the limits of a given piece of hardware.

In analog with medicine, I can currently list out a bunch of possible diseases that make people "less efficient", but this is hardly useful. I need to figure out how to diagnose my "patients" and then prescribe a "cure".

Could I please be referred to literature that teaches how to diagnosis of HPC problems (efficiency, scalability, etc)? Almost a step-by-step guide. Like put stethoscope of chest, then listen, ...

  • Interesting topic but the question as posed is [way too broad for StackOverflow](http://stackoverflow.com/help/on-topic). – Paul R Jan 26 '16 at 16:57
  • May I ask where I can ask such questions? In the MathOverflow reference requests seemed reasonably typical. Arguably, references will enable people to solve more problems on their own. I am not debating the rules though. Is there a research level StackOverflow equivalent? –  Jan 26 '16 at 17:02
  • My best guess would be http://programmers.stackexchange.com, which covers programming concepts and methodologies. – Paul R Jan 26 '16 at 17:04
  • 2
    @PaulR - reference requests like this are explicitly off-topic on Programmers. Please do not recommend other SE sites unless you are familiar with their quality guidelines. –  Jan 26 '16 at 17:06
  • @GlenH7: OK - my bad - I was looking at the online help for programmers.SE and it seemed like a reasonable fit - perhaps you could make an alternate suggestion ? – Paul R Jan 26 '16 at 17:07
  • Is there a site that can provide reference requests? Pretty much all of the questions I currently have are reference requests. –  Jan 26 '16 at 17:08
  • 1
    @PaulR - To my knowledge, reference requests are off-topic for pretty much _all_ of StackExchange. There's just too many problems with maintaining questions of that sort over the long term. –  Jan 26 '16 at 17:08
  • I am pretty sure it is allowed on MathOverflow as there is even a tag for it. Plus I have done it a couple times without getting in trouble –  Jan 26 '16 at 17:09
  • @aidan.plenert.macdonald - MathOverflow is the exception to the rule in this case. SO, Progs, CS, TCS, etc... do not allow those types of questions. –  Jan 26 '16 at 17:10
  • Hmm. So no exception for computational questions? I guess I can see the problems that would arise. Do you know of any other services (not StackExchange) that offer this? –  Jan 26 '16 at 17:11
  • @aidan.plenert.macdonald - While somewhat Progs specific, [this answer](http://meta.programmers.stackexchange.com/a/6487/53019) goes into detail as to why those questions don't work. As far as other services, I don't know of any. You're welcome to pose this question in [The Whiteboard chat](http://chat.stackexchange.com/rooms/21/the-whiteboard) and you might get some recommendations, but I don't recall if there are any HPC experts who frequent that room. –  Jan 26 '16 at 17:14
  • 1
    You could try posting on the [HPC subreddit on Reddit](https://www.reddit.com/r/HPC/) - it's reasonably active - people there may also be able to suggest other forums where questions about HPC performance optimisation etc might be more welcome. – Paul R Jan 26 '16 at 17:18
  • 1
    @Paul R Exactly what I needed. Thank you. I will let you guys close this if you deem appropriate. –  Jan 26 '16 at 17:24

1 Answers1

0

This question is two questions: one is how do I find bottlenecks, the other is how do I know the limits of my hardware and if I am at them.

The first is that you must run the code inside a profiler. Any profiler with a "top down" view of your code according to time is showing you the bottlenecks.

Try the profilers suggested here (answer applies to c++ and Fortran): Good profiler for Fortran and MPI - both Allinea MAP and HPC Toolkit have the sort of presentation you need. (NB I work for Allinea).

The second question is the most "open" part. That one needs your book or optimization guide. However, a good start is to see how much vectorization you have (Some of the profiler examples can show this) as this is where the most compute power can be found.

The bigger question is what the theoretical limit of your problem is - eg. Some problems are not amenable to vectorization, some have memory access needs that can never be cache friendly, some have communication needs that are simple whereas others require costly regular global updates.

Community
  • 1
  • 1
David
  • 756
  • 5
  • 10