Grid with Flexbox, remove gaps between columns

Question

I have this HTML&CSS.

HTML:

 <div class="cont">
  <div class="first">
    <div class="children">First First</div>
    <div class="children">First Second</div>
  </div>
  <div class="second">
    <div class="children">Second First</div>
    <div class="children">Second second</div>
    <div class="children">Second third</div>
  </div>
  <div class="third">
    <div class="children">Third First</div>
  </div>
  <div class="fourth">
    <div class="children"></div>
  </div>
</div>

and CSS:

.cont{
    display: flex;
    flex-wrap: wrap;
    max-width: 1100px;
    width: 100%;
    margin: 0 auto;
}
.children{
    height: 200px;
    margin: 10px 0;
    display: block;
    background: red;
}
.cont > div{
    padding: 0 10px;
    width: 40%;
}
.first {
    background: green;
}
.second{
    background: brown;
}
.third{
    background: blue;
}

You can see it here: CODEPEN.

It is parent container with display:flex and columns .first, .second, .third, etc. Each column is with different background and has elements with fixed height, just for the example.

The problem is that, there is a gap, between different column. For the example, between first and third columns. I see that, somehow first and second columns are with same height (first column is with background:green).

How can I remove that gap and is it possible only with CSS ?

I have tried many things with align-items, align-content, 'display:flex' for these columns, etc.