Implementing SCRAM-SHA1 client, getting it wrong somewhere

Question

Note: I've already read the very good answer to this question, but it doesn't answer my issues.

I'm attempting to implement SCRAM-SHA1 authentication standard, as specified by RFC 5802, in Common Lisp. I am running into issues when it comes to generating the client final response message.

This is the code of the function (the rest of the functions are available here) -- this is an attempt to implement the algorithm as described on page 7 of the RFC:

(defun gen-client-final-message
    (&key password client-nonce client-initial-message server-response)
  (check-type client-nonce string)
  (check-type client-initial-message string)
  (check-type server-response string)
  (check-type password string)
  "Takes a password, the initial client nonce, the initial client message & the server response.
   Generates the final client message, and returns it along with the server signature."
  (progn
    (if (eq nil (parse-server-nonce :nonce client-nonce :response server-response)) NIL)
    (let* ((final-message-bare (format nil "c=biws,r=~a" (parse-server-nonce :nonce client-nonce
                                                                             :response server-response)))
           (salted-password    (ironclad:pbkdf2-hash-password
                                 (ironclad:ascii-string-to-byte-array password)
                                 :salt       (ironclad:ascii-string-to-byte-array
                                               (parse-server-salt :response server-response))
                                 :digest     :sha1
                                 :iterations (parse-server-iterations :response server-response)))
           (client-key         (gen-hmac-digest :key salted-password
                                                :message (ironclad:ascii-string-to-byte-array "Client Key")))
           (stored-key         (gen-sha1-digest :key client-key))
           (auth-message       (format nil "~a,~a,~a"
                                       client-initial-message
                                       server-response
                                       final-message-bare))
           (client-signature   (gen-hmac-digest :key stored-key
                                                :message (ironclad:ascii-string-to-byte-array auth-message)))
           (client-proof       (integer->bit-vector (logxor (ironclad:octets-to-integer client-key)
                                                            (ironclad:octets-to-integer client-signature))))
           (server-key         (gen-hmac-digest :key salted-password
                                                :message (ironclad:ascii-string-to-byte-array "Server Key")))
           (server-signature   (gen-hmac-digest :key server-key
                                                :message (ironclad:ascii-string-to-byte-array auth-message)))
           (final-message      (format nil "~a,p=~a"
                                       final-message-bare
                                       (base64-encode (write-to-string client-proof)))))
      (pairlis '(final-message
                 final-message-bare
                 salted-password
                 client-key
                 stored-key
                 auth-message
                 client-signature
                 client-proof
                 server-key
                 server-signature)
               (list final-message
                     final-message-bare
                     salted-password
                     client-key
                     stored-key
                     auth-message
                     client-signature
                     client-proof
                     server-key
                     server-signature)))))

The example conversation in the RFC uses the username user and the password pencil:

C: n,,n=user,r=fyko+d2lbbFgONRv9qkxdawL
S: r=fyko+d2lbbFgONRv9qkxdawL3rfcNHYJY1ZVvWVs7j,s=QSXCR+Q6sek8bf92,
   i=4096
C: c=biws,r=fyko+d2lbbFgONRv9qkxdawL3rfcNHYJY1ZVvWVs7j,
   p=v0X8v3Bz2T0CJGbJQyF0X+HI4Ts=
S: v=rmF9pqV8S7suAoZWja4dJRkFsKQ=

Taking the same server response (r=fyko+d2lbbFgONRv9qkxdawL3rfcNHYJY1ZVvWVs7j,s=QSXCR+Q6sek8bf92,i=4096) and feeding it into my function, I get:

* (cl-scram:gen-client-final-message :password "pencil" :client-nonce "fyko+d2lbbFgONRv9qkxdawL" :client-initial-message "n,,n=user,r=fyko+d2lbbFgONRv9qkxdawL" :server-response "r=fyko+d2lbbFgONRv9qkxdawL3rfcNHYJY1ZVvWVs7j,s=QSXCR+Q6sek8bf92,i=4096")

((CL-SCRAM::SERVER-SIGNATURE
  . #(33 115 21 228 67 190 35 238 223 122 117 125 222 242 209 136 175 228 67
      151))
 (CL-SCRAM::SERVER-KEY
  . #(15 224 146 88 179 172 133 43 165 2 204 98 186 144 62 170 205 191 125 49))
 (CL-SCRAM::CLIENT-PROOF
  . #*1100100111101011000000111010100000010101011001000101011100110001111100001100100010001101001000110101001010101010001011111000100011100001001110100001001110000)
 (CL-SCRAM::CLIENT-SIGNATURE
  . #(251 9 164 14 244 111 236 112 227 116 148 143 243 255 231 75 58 114 21
      88))
 (CL-SCRAM::AUTH-MESSAGE
  . "n,,n=user,r=fyko+d2lbbFgONRv9qkxdawL,r=fyko+d2lbbFgONRv9qkxdawL3rfcNHYJY1ZVvWVs7j,s=QSXCR+Q6sek8bf92,i=4096,c=biws,r=fyko+d2lbbFgONRv9qkxdawL3rfcNHYJY1ZVvWVs7j")
 (CL-SCRAM::STORED-KEY
  . #(233 217 70 96 195 157 101 195 143 186 217 28 53 143 20 218 14 239 43
      214))
 (CL-SCRAM::CLIENT-KEY
  . #(226 52 196 123 246 195 102 150 221 109 133 43 153 170 162 186 38 85 87
      40))
 (CL-SCRAM::SALTED-PASSWORD
  . #(29 150 238 58 82 155 90 95 158 71 192 31 34 154 44 184 166 225 95 125))
 (CL-SCRAM::FINAL-MESSAGE-BARE
  . "c=biws,r=fyko+d2lbbFgONRv9qkxdawL3rfcNHYJY1ZVvWVs7j")
 (CL-SCRAM::FINAL-MESSAGE
  . "c=biws,r=fyko+d2lbbFgONRv9qkxdawL3rfcNHYJY1ZVvWVs7j,p=IyoxMTAwMTAwMTExMTAxMDExMDAwMDAwMTExMDEwMTAwMDAwMDEwMTAxMDExMDAxMDAwMTAxMDExMTAwMTEwMDAxMTExMTAwMDAxMTAwMTAwMDEwMDAxMTAxMDAxMDAwMTEwMTAxMDAxMDEwMTAxMDEwMDAxMDExMTExMDAwMTAwMDExMTAwMDAxMDAxMTEwMTAwMDAxMDAxMTEwMDAw"))

As you can see, my client-proof (the p= part of the final-message) is wildly different to the one in the example.

I added all of the intermediate variables to the return in case anyone here can see what's going wrong. Unfortunately, there are no examples which show the intermediate variable values, so I can't compare what I'm getting to the alternatives.

Style: Can you explain what the IF does? What is the PROGN for? The documentation string comes first and the CHECK-TYPE calls are a part of the body. — Rainer Joswig, Sep 09 '15 at 09:03
@RainerJoswig Sure, the IF checks if the server nonce begins with the client nonce. If that's nil, the function will return nil instead of going through and creating a pointless message to return. I don't need the progn, that's a relic from when the if was on the outside of the statement. — Matt Prelude, Sep 09 '15 at 18:19
How should that work? The message will be returned anyway... The result of the IF is not used and there is no useful control flow. — Rainer Joswig, Sep 09 '15 at 18:21
Thanks for reminding me about the PROGN. Have committed and pushed improvements. :) — Matt Prelude, Sep 09 '15 at 18:34

score 6 · Accepted Answer · edited Oct 07 '21 at 05:54

The intermediate values for the sample in the RFC 5802: Salted Challenge Response Authentication Mechanism (SCRAM) SASL and GSS-API Mechanisms are on the bottom of this answer.

Your p value is way too long; you are probably encoding the bits as string instead of bytes. You should loop over the byte blocks and XOR each unsigned byte separately. Converting to integer, then to bit string, then back to octet string is going to fail because it will probably remove the most significant zero bits. Once you've got the XOR'ed octet string you can base 64 encode it.

Furthermore, you need to remove n,, from the start of your AuthMessage, as specified in the RFC.

For future developers, without further ado, the intermediate values:

In base 64:

SaltedPassword: HZbuOlKbWl+eR8AfIposuKbhX30=
ClientKey: 4jTEe/bDZpbdbYUrmaqiuiZVVyg=
StoredKey: 6dlGYMOdZcOPutkcNY8U2g7vK9Y=
AuthMessage: n=user,r=fyko+d2lbbFgONRv9qkxdawL,r=fyko+d2lbbFgONRv9qkxdawL3rfcNHYJY1ZVvWVs7j,s=QSXCR+Q6sek8bf92,i=4096,c=biws,r=fyko+d2lbbFgONRv9qkxdawL3rfcNHYJY1ZVvWVs7j
ClientSignature: XXE4xIawv6vfSePi2ovW5cedthM=
ClientProof: v0X8v3Bz2T0CJGbJQyF0X+HI4Ts=

Using decimal arrays:

SaltedPassword: 29 150 238 58 82 155 90 95 158 71 192 31 34 154 44 184 166 225 95 125
ClientKey: 226 52 196 123 246 195 102 150 221 109 133 43 153 170 162 186 38 85 87 40
StoredKey: 233 217 70 96 195 157 101 195 143 186 217 28 53 143 20 218 14 239 43 214
AuthMessage: n=user,r=fyko+d2lbbFgONRv9qkxdawL,r=fyko+d2lbbFgONRv9qkxdawL3rfcNHYJY1ZVvWVs7j,s=QSXCR+Q6sek8bf92,i=4096,c=biws,r=fyko+d2lbbFgONRv9qkxdawL3rfcNHYJY1ZVvWVs7j
ClientSignature: 93 113 56 196 134 176 191 171 223 73 227 226 218 139 214 229 199 157 182 19
ClientProof: 191 69 252 191 112 115 217 61 2 36 102 201 67 33 116 95 225 200 225 59

Feel free to add the comments to your answer, to help the next poor sod trying to implement this standard. — Matt Prelude, Sep 09 '15 at 09:38
If you happen to know more about mongo-specific stuff: http://stackoverflow.com/questions/32486579/implementing-mongodb-sasl-authentication-in-cl-mongo — Matt Prelude, Sep 09 '15 at 18:19
I know absolutely nothing of mongo *or* lisp *or* SASL. When I saw your question I just implemented the RFC in Java (without an actual client or server of course) :) — Maarten Bodewes, Sep 09 '15 at 21:16
Hi, for a worked authentication conversation with MongoDB see (my) article [MongoDB SCRAM-SHA-1 over SASL](http://www.fantomfactory.org/articles/mongodb-scramsha1-over-sasl#.VjZpGRY0xiA) — Steve Eynon, Nov 01 '15 at 19:49
Great, thanks for sharing! I was assuming that something of this would be given back to the community, which was one of the reasons for going the extra mile. — Maarten Bodewes, Nov 01 '15 at 20:05

Implementing SCRAM-SHA1 client, getting it wrong somewhere

1 Answers1