For those who wonder what InstantRun is (first setting in the picture):

And yes disabling that setting fixes the issue.
Edit 01/19/16
'com.android.tools.build:gradle:2.0.0-alpha6' was released three days ago. Using that new version, the "Split lib_main was defined multiple times" error is gone and installation of the app succeeds but now I get a runtime error:
java.lang.RuntimeException: Unable to instantiate application
com.android.tools.fd.runtime.BootstrapApplication:
java.lang.ClassNotFoundException: Didn't find class
"com.android.tools.fd.runtime.BootstrapApplication" on path:
DexPathList[[zip file "/data/app/com.onegravity.k10.pro2-2/base.apk",
zip file "/data/app/com.onegravity.k10.pro2-/split_lib_main.apk"]
Caused by: java.lang.ClassNotFoundException: Didn't find class
"com.android.tools.fd.runtime.BootstrapApplication" on path:
DexPathList[[zip file "/data/app/com.onegravity.k10.pro2-2/base.apk",
zip file "/data/app/com.onegravity.k10.pro2-2/split_lib_main.apk"]
Suppressed: java.io.IOException: No original dex files found for dex location
/data/app/com.onegravity.k10.pro2-2/base.apk
Suppressed: java.lang.ClassNotFoundException:
com.android.tools.fd.runtime.BootstrapApplication
Turning off instant run eliminates that error.
Let's wait for com.android.tools.build:gradle:2.0.0-alpha7...
Edit 01/25/16
'com.android.tools.build:gradle:2.0.0-alpha7' was released two days ago.
Getting the same exception as with 2.0.0-alpha6