ESP32 arduino x509 format is invalid loading from SPIFFS

Question

I'm trying to fetch 3 keys from SPIFFS that I stored from a string to a file (when fetched from an API endpoint).

I am able to read the files using the SPIFFS library and print the values out in the serial console. When I check the contents, they are just fine. When I use the contents in the secureClient.setCACert() function, they "break" and throw an exception; E][ssl_client.cpp:36] _handle_error(): [start_ssl_client():138]: (-8576) X509 - The CRT/CRL/CSR format is invalid, e.g. different type expected

My code looks like the following:

#include <WiFiClientSecure.h>
#include <FS.h>
#include <SPIFFS.h>


WiFiClientSecure net = WiFiClientSecure();

String read2String(const char * path) {
    String output = "";
    
    File file = SPIFFS.open(path, "r");
    if (!file || file.isDirectory()) {
        return "";
    }
    
    while (file.available()) {
      char c = file.read();
      output.concat(c);
    }
    file.close();
    
    return output;
}

void readFilesToVariables() {
  String awsRootCa = read2String("/AmazonRootCA1.pem");
  String privateKey = read2String("/private.pem.key");
  String certificate = read2String("/certificate.pem.crt");

  net.setCACert(awsRootCa.c_str());
  net.setCertificate(certificate.c_str());
  net.setPrivateKey(privateKey.c_str());
}

void setup() {
  Serial.begin(115200);
  SPIFFS.begin();

  readFilesToVariables();

  mqttClient.begin(AWS_IOT_ENDPOINT, AWS_IOT_ENDPOINT_PORT, net); // crashes here (I think)

...
}

I have on the mqttClient.begin(...) line (the mqtt client and all the variables except for the net var) a couple of variables that are defined in my script but not shared here.

Anyone any idea what I'm doing wrong here?

the exact same code works when I define the certificates and private key using the following methodology:

static const char AWS_CERT_CA[] PROGMEM = R"EOF(
-----BEGIN CERTIFICATE-----
certificate_content_here
-----END CERTIFICATE-----
)EOF";

What am I doing wrong?

Lele · Answer 1 · 2022-09-14T14:40:19.020

The same thing drove me crazy for the last few days. Then I found a solution, but only supposed the problem. I think that this issue has something to do with memory handling and where and how you store the certificates after you read it from flash.

Here's how I read the certificates. Using calloc I allocate the memory and never free it, so that the certificate will always be available to the MQTT client

char* getFileAsString(String path) {
  File file = SPIFFS.open(path);
  if (!file) {
    Serial.println("Failed to open " + path);
    return NULL;
  }
  char* buffer = (char*)calloc(file.size(), sizeof(char));
  file.readBytes(buffer, file.size());
  file.close();
  return buffer;
}

Then I just use it in the config struct

const esp_mqtt_client_config_t mqttConf = {
  .uri = MQTT_BROKER,
  .client_id = MQTT_CLIENT_ID,
  .cert_pem = getFileAsString("/caFile"),
  .client_cert_pem = getFileAsString("/certificateFile"),
  .client_key_pem = getFileAsString("/privateFile"),
};

Note: I did't put the *_len attributes, because I stored my certificates with a leadinig null character:

dd if=/dev/zero bs=1 count=1 >> [my cert name].pem

You can use regular certificates, but you must set the certificates length or adding 1 byte to the calloc instruction (calloc will set all the allocated memory to 0x00, so there will be a leading null character):

char* buffer = (char*)calloc(file.size() + 1, sizeof(char));

ESP32 arduino x509 format is invalid loading from SPIFFS

1 Answers1