3

We have a OpenSSL running on our embedded system, which is running ECOS OS. We are now upgrading our OpenSSL to 1.0.2 version. We have successfully ported and compiled the OpenSSL library. But when when we try to connect our device using SSL (via https), handshake fails with bad record mac alert always. We have enabled OpenSSL debug option, but unable to identify why its failing.

Have someone ported latest OpenSSL code to ECOS? Do we need to take of any special compilation flags with latest OpenSSL code for ECOS?

For reference, here is the relevant part of ssl3_get_record:

mac = rr->data + rr->length;
i=s->method->ssl3_enc->mac(s,md,0 /* not send */);
if (i < 0 || CRYPTO_memcmp(md, mac, (size_t)mac_size) != 0)
    {
    al=SSL_AD_BAD_RECORD_MAC;
    SSLerr(SSL_F_SSL3_GET_RECORD,SSL_R_DECRYPTION_FAILED_OR_BAD_RECORD_MAC);
    goto f_err;
    }
jww
  • 97,681
  • 90
  • 411
  • 885
Vikash Jain
  • 81
  • 8
  • Thanks for the info, tested with 1.0.2 also, still getting same issue. Note that our ECOS running on embedded system has no file system. So wandering if we are missing some compilation flags? – Vikash Jain Jun 15 '15 at 12:38
  • I'm not sure if you are missing any `$cflags` or `$ldflags` because you did not provide them :) What's your [Configure triplet](https://wiki.openssl.org/index.php/Compilation_and_Installation) (I presume you added a custom one)? And what were the [other options](https://wiki.openssl.org/index.php/Compilation_and_Installation#Configure_Options) used to configure the library? Does your ROM have room for the self tests? Even if its a one-off build to ensure the library builds and executes correctly.... – jww Jun 15 '15 at 12:53
  • Also, did you see [Andrew Lunn of Ascom has made available a port of the OpenSSL library....](http://ecos.sourceware.org/contrib.html) on the eCos site? It may have the configure triplet with the custom `cflags` and `$ldflags` settings. – jww Jun 15 '15 at 13:04
  • We are already using andrew's port which is age old 1.9.6 b. Due to recent vulnerabilities in SSL, we have to upgrade our SSL to support higher version .i.e. TLS1.1 and TLS1.2. – Vikash Jain Jun 17 '15 at 02:32
  • Oh, good job on upgrading :) What is the `Configure` triplet he is using? Its likely custom, so just use it. (And then donate the code back to the eCos community to help others). – jww Jun 17 '15 at 06:17

1 Answers1

1

After debugging we found that the random library (RAND) was failing for ECOS. There were lot of places in OpenSSL where it checks for random_bytes return type. Due to this failure, pre-master key decryption was failing. And incoming packets were not decrypted properly. Hence a BAD Mac records error was seen.

We also checked with our old ported code (0.9.6), RAND library was failing there also, but there we no return check for random_bytes and pseudo_rand_bytes. As a fix we made RAND to return success every time, and we can see SSL session being established fine with OpenSSL 1.0.2 version.

Vikash Jain
  • 81
  • 8
  • *"As a fix we made RAND to return success every time..."* - maybe I'm parsing this incorrectly, but this sounds like a very bad idea... – jww Jun 19 '15 at 16:26
  • Rephrasing, when i say RAND library is not working, we can see random bytes being generated, but due to some entropy mismatch it always return failure. Same was not working in old ECOS OpenSSL port also. Will update the answer once we know why the failure is happening. – Vikash Jain Jun 19 '15 at 16:42
  • You are in an uncomfortable spot. I *think* one of the better things you can do is to modify OpenSSL and incorporate ***Hedging***. For the papers, see [When Good Randomness Goes Bad: Virtual Machine Reset Vulnerabilities and Hedging Deployed Cryptography](http://www.isoc.org/isoc/conferences/ndss/10/pdf/15.pdf) and [When Virtual is Harder than Real: Resource Allocation Challenges in Virtual Machine Based IT Environments](http://static.usenix.org/event/hotos05/final_papers/full_papers/garfinkel/garfinkel.pdf). – jww Jun 19 '15 at 18:12
  • Hedging follows Gutmann's philosophy of *["mix every entropy source you can get your hands on into your PRNG, including less-than-perfect ones..."](http://lists.randombit.net/pipermail/cryptography/2013-July/004746.html)*. It will surely help you avoid the failure you are experiencing. – jww Jun 19 '15 at 18:21
  • Thanks alot Jeff, I will surely try this. – Vikash Jain Jun 22 '15 at 12:03