The key here is of course that if you use "intrusive" solutions, that is, you store the links for a linked list inside each node, vs. having a separate node structure that holds a pointer/reference to some other location that holds the data itself, there will be some performance impact [and that can work in both directions - not necessarily that the intrusive solution is always best].
The TYPICAL case where people get it wrong is when they compare storing an allocated object in a container of some sort, and then comparing that to a solution that stores a copy of the object in the container. If that container is one that needs to "grow" by re-allocating, then it will cause performance problems. But that's just "wrong choice if thing to store" or "wrong kind of container for this type of object storage".
For most cases, the implementation of HOW you do something (e.g. is it a tree or a vector) is where you get the 100x improvement. The write something specific for this type of storage will give you 5-20% improvement, depending on how well the original code and how well the optimized code is written. Now, assuming it's not part of the inner details of your rendering loop in a game, or the sheduler or memory allocation code in the OS kernel, 5% is not worth having, and for the vast majority of code, 20% improvement is probably not much value either [for one part of the system functionality - if you are storing stuff in a container, hopefully you do more with it than just plain store it and work your way back and forth over the container content, right?]
Before spending any time optimizing code, there should be some decent analysis of where the code is spending the time. Then you can attack making it better. The first step in that is to look at "What is the behaviour of the algorithm here?" Are we spending O(2^n) on sorting something, when we could use a different method that is much faster? Are we spending time inserting/removing things from a large array, when a list, stack or queue would perhaps be a better choice? Are we writing small chunks of data to disk, when we could write a smaller number of large blocks?
These sort of questions will give you the 5, 10, 100x improvements that make REAL difference. Of course, shaving 5% of something is worth doing. Sometimes shaving 0.5% off is worth doing, if it's what makes your game run at 200fps instead of 100fps, because it is that half a percent that trips you over the frame switch rate. But in general, if it doesn't at least double the speed, you probably should spend the time elsewhere.