I am having a hard time configuring zypper clients (SLES and OpenSUSE) to work with our RPM Repository in JFrog due to some "authentication" issues. At first, I thought it was just an issue with zypper and opensuse so I first focused on our efforts to make things work with zypper. This was the previous question.
But these authentication-related errors come up every now and then with yum clients as well (centOS), although not that often.
So I tried doing man-in-the-middle to see what's happening under the hood, and this is the sequence with zypper:
Scenario 1 - successful authentication
The following is the sequence of a zypper refresh --repo myrepo:
(1) zypper sends an HTTP HEAD request with the base64-encoded username::
HEAD /myapp-release-rpm/stable/myapp/2019/3/repodata/repomd.xml HTTP/1.1
Host: rpm.example.com
Authorization: Basic dXNlcm5hbWU6
User-Agent: ZYpp 17.19.0 (curl 7.60.0) openSUSE-Leap-15.1-x86_64
Accept: */*
Connection: close
(2) jfrog responds with HTTP 401 Unauthorized with the WWW-Authenticate header:
HTTP/1.1 401 Unauthorized
Date: Thu, 28 May 2020 08:20:04 GMT
Content-Type: application/json;charset=ISO-8859-1
Connection: close
Server: Artifactory/7.4.3 70403900
X-Artifactory-Id: 2148103ba10eacbb:-16f1c4c1:172093a231a:-8000
X-Artifactory-Node-Id: artifactory-server
WWW-Authenticate: Basic realm="Artifactory Realm"
(3) zypper sends another HTTP HEAD request, this time with the base64-encoded username:password:
HEAD /myapp-release-rpm/stable/myapp/2019/3/repodata/repomd.xml HTTP/1.1
Host: rpm.example.com
Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ=
User-Agent: ZYpp 17.19.0 (curl 7.60.0) openSUSE-Leap-15.1-x86_64
Accept: */*
Connection: close
(4) jfrog finally responds with an HTTP 200.
HTTP/1.1 200 OK
Date: Thu, 28 May 2020 08:20:04 GMT
Content-Type: application/xml
Content-Length: 1394
Connection: close
Server: Artifactory/7.4.3 70403900
X-Artifactory-Id: 2148103ba10eacbb:-16f1c4c1:172093a231a:-8000
X-Artifactory-Node-Id: artifactory-server
Last-Modified: Fri, 08 May 2020 10:25:19 GMT
Accept-Ranges: bytes
X-Artifactory-Filename: repomd.xml
Cache-Control: no-store
These are logged by Artifactory:
artifactory-request.log:
2020-05-28T08:20:34.566Z [5f78297c2aeabaa8] [DENIED LOGIN] for client : username / 213.1.1.1.
2020-05-28T08:20:34.870Z [570978212a5318e3] [ACCEPTED DOWNLOAD] myapp-release-rpm-cache:stable/myapp/2019/3/repodata/repomd.xml for client : username / 213.1.1.1.
artifactory-access.log:
2020-05-28T08:20:34.566Z|5f78297c2aeabaa8|213.2.2.2|non_authenticated_user|HEAD|/myapp-release-rpm/stable/myapp/2019/3/repodata/repomd.xml|401|-1|0|8|ZYpp 17.19.0 (curl 7.60.0) openSUSE-Leap-15.1-x86_64
2020-05-28T08:20:34.721Z|8018b7cbc9c424e8|213.2.2.2|username|HEAD|/myapp-release-rpm/stable/myapp/2019/3/repodata/repomd.xml|200|-1|1394|3|ZYpp 17.19.0 (curl 7.60.0) openSUSE-Leap-15.1-x86_64
2020-05-28T08:20:34.870Z|570978212a5318e3|213.2.2.2|username|GET|/myapp-release-rpm/stable/myapp/2019/3/repodata/repomd.xml|200|-1|1394|2|ZYpp 17.19.0 (curl 7.60.0) openSUSE-Leap-15.1-x86_64
...
So basically, zypper comes in with a HEAD request, JFrog says "you're not authenticated", zypper responds back and tries to authenticate, and finally JFrog authenticates zypper. Makes sense so far.
Scenario 2 - 403 forbidden
Do some work, then run the same zypper refresh --repo myrepo command after a few minutes or so, and here is the result:
(1) zypper sends an HTTP HEAD request with the base64-encoded username::
HEAD /myapp-release-rpm/stable/myapp/2019/3/repodata/repomd.xml HTTP/1.1
Host: rpm.example.com
Authorization: Basic dXNlcm5hbWU6
User-Agent: ZYpp 17.19.0 (curl 7.60.0) openSUSE-Leap-15.1-x86_64
Accept: */*
Connection: close
(2) jfrog responds with HTTP 401 Unauthorized with the WWW-Authenticate header:
HTTP/1.1 401 Unauthorized
Date: Thu, 28 May 2020 08:30:44 GMT
Content-Type: application/json;charset=ISO-8859-1
Connection: close
Server: Artifactory/7.4.3 70403900
X-Artifactory-Id: 2148103ba10eacbb:-16f1c4c1:172093a231a:-8000
X-Artifactory-Node-Id: artifactory-server
WWW-Authenticate: Basic realm="Artifactory Realm"
(3) zypper sends another HTTP HEAD request, this time with the base64-encoded username:password:
HEAD /myapp-release-rpm/stable/myapp/2019/3/repodata/repomd.xml HTTP/1.1
Host: rpm.example.com
Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ=
User-Agent: ZYpp 17.19.0 (curl 7.60.0) openSUSE-Leap-15.1-x86_64
Accept: */*
Connection: close
(4) this time, jfrog responds with 403 Forbidden instead of 200 OK.
HTTP/1.1 403 Forbidden
Date: Thu, 28 May 2020 08:30:44 GMT
Content-Type: application/json;charset=ISO-8859-1
Connection: close
Server: Artifactory/7.4.3 70403900
X-Artifactory-Id: 2148103ba10eacbb:-16f1c4c1:172093a231a:-8000
X-Artifactory-Node-Id: artifactory-server
WWW-Authenticate: Basic realm="Artifactory Realm"
artifactory-request.log:
2020-05-28T08:30:44.496Z [46c81a2450623166] [DENIED LOGIN] for client : username / 213.1.1.1.
2020-05-28T08:30:44.630Z [769ed41c652daa7a] [DENIED LOGIN] for client : username / 213.1.1.1.
artifactory-access.log:
2020-05-28T08:30:44.496Z|46c81a2450623166|213.2.2.2|non_authenticated_user|HEAD|/myapp-release-rpm/stable/myapp/2019/3/repodata/repomd.xml|401|-1|0|9|ZYpp 17.19.0 (curl 7.60.0) openSUSE-Leap-15.1-x86_64
2020-05-28T08:30:44.630Z|769ed41c652daa7a|213.2.2.2|non_authenticated_user|HEAD|/myapp-release-rpm/stable/myapp/2019/3/repodata/repomd.xml|403|-1|0|1|ZYpp 17.19.0 (curl 7.60.0) openSUSE-Leap-15.1-x86_64
Notice that zypper sends the same Authorization header value when asked to authenticate, but on the second scenario, JFrog fails to authenticate the request.
Did anybody have this same issue with JFrog before? We are guessing this is an issue with JFrog 7 since ours was just recently upgraded, but there is no way for us to verify this. And unfortunately for us, we are on the paid plan which doesn't even have a support license.
Any suggestions and comments will be very much appreciated.
Additional information
It might be worth-noting that yum clients (centos and rhel) do not have this exact issue with JFrog Artifactory, and it's most likely because yum always sends the username and password hash in the HTTP header, avoiding the 401 Unauthorized re-authentication sequence.
More details
I submitted a bug report to JFrog last May, and so far did not get any update.
