Load SSL certificate from memory using cURL and CyaSSL

The cacertinmem sample in cURL shows you how to do this with OpenSSL, but it's slightly different if you use CyaSSL, and actually simpler.

From the CyaSSL manual:

To use the extension define the constant NO_FILESYSTEM and the following functions will be made available:
int CyaSSL_CTX_load_verify_buffer(...)
int CyaSSL_CTX_use_certificate_buffer(...)
int CyaSSL_CTX_use_PrivateKey_buffer(...)
int CyaSSL_CTX_use_certificate_chain_buffer(...)
Use these functions exactly like their counterparts that are named fileinstead of buffer. 
And instead of providing a filename provide a memory buffer

Use the NO_FILESYSTEM preprocessor definition when building CyaSSL, cURL, as well as in your own project.

In my case, I wanted to load a bundle of CA certificates, and would therefore use CyaSSL_CTX_load_verify_buffer() (please turn to the CyaSSL API reference).

In addition, you must be able to use the SSL callback function which isn't supported in cURL (version 7.29.0) when using SSL libraries other than OpenSSL. To solve this problem, read my article about the SSL context function with cURL and CyaSSL.

Bundle of CA root certificates

Useful when you don't know which SSL providers were used. I got it from curl.haxx.se.

I compressed the CA certificates with GZip before embedding them. Poco (C++ framework) took care of decompresson.

Reduction: 115 KB / ~45% (245 KB down to 130 KB).

Probably sensible enough if GZip (or whatever you want to use) is used in your application already elsewhere (thinking about increased code size).

main.cpp

#include "CaCert.h"

#include <curl/curl.h>
#include <cyassl/ssl.h>

#include <string>

// ...

CURLcode sslContextCallback(void* ctx) {
    // Load CA certificates from memory
    CaCert cert;
    const std::string& certData = cert.GetData();
    CyaSSL_CTX_load_verify_buffer(
        reinterpret_cast<CYASSL_CTX*>(ctx),
        reinterpret_cast<const unsigned char*>(certData.c_str()),
        certData.size(),
        SSL_FILETYPE_PEM);
    return CURLE_OK;
}

// ...
curl_easy_setopt(curl, CURLOPT_SSL_CTX_FUNCTION, *sslContextCallback);

CaCert.h

#ifndef __CaCert__
#define __CaCert__

#include <string>

class CaCert {
public:
    const std::string& GetData();
private:
    static const unsigned char m_cacert_pem_gz[];
    std::string m_cacert_pem;
};

#endif // guard

CaCert.cpp

#include "CaCert.h"
#include "cacert.pem.gz.h"
#include <Poco/InflatingStream.h>
#include <sstream>

const std::string& CaCert::GetData() {
    if (m_cacert_pem.empty()) {
        std::string gzstr(reinterpret_cast<const char*>(&m_cacert_pem_gz[0]), sizeof(m_cacert_pem_gz));
        std::istringstream istr(gzstr);
        Poco::InflatingInputStream inflater(istr, Poco::InflatingStreamBuf::STREAM_GZIP);
        std::stringstream ostr;
        ostr << inflater.rdbuf();
        m_cacert_pem = ostr.str();
    }

    return m_cacert_pem;
}

cacert.pem.gz.h

This file was generated using Hex Workshop and modified slightly.

// Generated by BreakPoint Software's Hex Workshop v6.6.1.5158
//   http://www.hexworkshop.com
//   http://www.bpsoft.com
//
//  Source File: C:\code\curltest\cacert.pem.gz
//         Time: 17.02.2013 04:55
// Orig. Offset: 0 / 0x00000000
//       Length: 133503 / 0x0002097F (bytes)
const unsigned char CaCert::m_cacert_pem_gz[133503] = { /*...*/ };

Reduce size and memory footprint

We don't always need all of the CA root certificates. We can remove all of the certificates we don't need, and any comments. In my case, I needed only StartCom certificates.

I skipped compression since the file was only 7.37 KB.

Reduction: 236.63 KB / ~96.9%.

main.cpp

#include "StartComCaCert.h"

#include <curl/curl.h>
#include <cyassl/ssl.h>

#include <string>

// ...

CURLcode sslContextCallback(void* ctx) {
    // Load CA certificates from memory
    StartComCaCert cert;
    const unsigned char* certData = cert.GetData();
    CyaSSL_CTX_load_verify_buffer(
        reinterpret_cast<CYASSL_CTX*>(ctx),
        certData,
        cert.GetSize(),
        SSL_FILETYPE_PEM);
    return CURLE_OK;
}

// ...
curl_easy_setopt(curl, CURLOPT_SSL_CTX_FUNCTION, *sslContextCallback);

StartComCaCert.h

#ifndef __StartComCaCert__
#define __StartComCaCert__

#include <string>

class StartComCaCert {
public:
    const unsigned char* GetData() const;
    unsigned int GetSize() const;
private:
    static const unsigned char m_cacert_StartCom_pem[];
};

#endif // guard

StartComCaCert.cpp

#include "StartComCaCert.h"
#include "cacert_StartCom.pem.h"

const unsigned char* StartComCaCert::GetData() const {
    return &m_cacert_StartCom_pem[0];
}

unsigned int StartComCaCert::GetSize() const {
    return sizeof(m_cacert_StartCom_pem);
}

cacert_StartCom.pem.h

This file was also generated.

// Generated by BreakPoint Software's Hex Workshop v6.6.1.5158
//   http://www.hexworkshop.com
//   http://www.bpsoft.com
//
//  Source File: C:\code\curltest\curltest\cacert_StartCom.pem
//         Time: 18.02.2013 01:23
// Orig. Offset: 0 / 0x00000000
//       Length: 7547 / 0x00001D7B (bytes)
const unsigned char StartComCaCert::m_cacert_StartCom_pem[7547] = { /*...*/ }

It'll be ridiculous to continue, but let's do it!

Looking at the certificates as they are stored in PEM format, our precious bytes are obviously wasted while it tries to look pretty to humans.

While keeping the base-64-encoded data (which also wastes space), let's go ahead and write a small Python script to minify the file.

minify_cert_pem.py

It removes everything at the beginning of the file (comments, newlines), removes repeated equal signs, and newlines mixed in the base-64-encoded data.

import re

def stripNewlines(match):
    return match.group(1) + match.group(2) + match.group(3).replace("\n", "") + match.group(4)

inFile = open("cacert_StartCom.pem", "r")
caCertData = inFile.read()

# Strip comments and everything else in the beginning of the file
result = re.sub(r"(?s)^.*?\n(\w)", r"\1", caCertData)
# Strip the newlines we can strip
result = re.sub(r"(?s)(=)+\s*(\s-----BEGIN CERTIFICATE-----\s)(.*?)(\s-----END CERTIFICATE-----\s)\s*", stripNewlines, result)
# Strip remaining newlines at the end
result = result.rstrip()
outFile = open("cacert_StartCom.min.pem", "w")
outFile.write(result)

Reduction: 175 bytes / ~2.32%.

Just for the sake of it, minify the original cacert.pem

Reduction: 7.53 KB / 3%.
Comparing old and new GZipped file: 5.98 KB / 4.59% reduction.
Total reduction: 121 KB / 49.32%.