Short answer to the question "why matplotlib.collections is faster?":
Because collections are designed to be faster while individual line segments are designed to be more flexible.
From the module documentation:
Classes for the efficient drawing of large collections of objects that
share most properties, e.g., a large number of line segments or
polygons.
The classes are not meant to be as flexible as their single element
counterparts (e.g., you may not be able to select all line styles) but
they are meant to be fast for common use cases (e.g., a large set of
solid line segemnts)
I am not familiar enough with matplotlib internals to explain how the implementations differ in detail. If I had to guess, I would say that the collection knows it is only going to draw lines and so can set up line drawing and then draw all the lines in one go. In contrast, individual lines don't know if the next thing that is drawn has the same properties or even if it is also a line, so they have to clean up after drawing and the next line needs to set up the state again. But this is just speculation.