The first paragraph is taken out of the flow by being floated, while the second paragraph remains in flow, so the box of the second paragraph is positioned as though the first paragraph were not there.
However, the text in the second paragraph is pushed down by the first paragraph so that it remains visible. The technical reason for this is that the text is inline content, which flows along a set of line boxes. From the spec:
Since a float is not in the flow, non-positioned block boxes created before and after the float box flow vertically as if the float did not exist. However, the current and subsequent line boxes created next to the float are shortened as necessary to make room for the margin box of the float.
[...]
If a shortened line box is too small to contain any content, then the line box is shifted downward (and its width recomputed) until either some content fits or there are no more floats present.
Since the text is pushed down, this causes the boundaries of the second paragraph box to extend downwards as well, to accommodate the new position of the text.
As mentioned in other answers, this behavior is typically prevented by clearing the float:
<p style="float:left; border:red solid 1px;">
first paragraph. first paragraph. first paragraph. first paragraph. first paragraph. first paragraph. first paragraph. first paragraph. first paragraph. first paragraph. first paragraph. first paragraph. first paragraph.
</p>
<p style="clear:both; float:none; border:green solid 1px;">
second paragraph. second paragraph. second paragraph. second paragraph. second paragraph. second paragraph. second paragraph. second paragraph. second paragraph.</p>