You escape a double-quote with another double-quote, both in VB and in a connection string, and you can wrap connection string field values in double-quotes. That means that, to write your connection string literally, you would do this:
Dim ConnectionString As String = "Server=DT2719MOD;Database=abs2;User Id=TestUserLogon;Password=""ilo;veac'h""""alle;nge"""
The password gets wrapped in double-quotes in the connection string so that the semicolon is interpreted literally and they need to be escaped in VB. The double-quote in the password needs to be escaped in the connection string and then both of those need to be escaped in VB too.
Note that, as suggested in the comments, a connection string builder will do that for you, e.g.
Dim builder As New SqlConnectionStringBuilder With {
.DataSource = "DT2719MOD",
.InitialCatalog = "abs2",
.UserID = "TestUserLogon",
.Password = "ilo;veac'h""alle;nge"}
Dim connection As New SqlConnection(builder.ConnectionString)
In that case, the only thing you need to do is escape the double-quote in the password once for VB.
For the record, you could have used the Server Explorer to generate a connection string for you too.