Pages

Friday, October 18, 2013

Error: The target "_WPPCopyWebApplication" does not exist in the project

This applies to both Visual Studio 2010 and Team Foundation Server (TFS) 2010.

In my case, I had two projects of the follow types that needed to be published on the file system after build:

  • ASP.NET MVC
  • WCF Web Service Library

Problem

Build succeeds in the development environment, but fails on the CI/build server, giving the following error message:

The target "_WPPCopyWebApplication" does not exist in the project

Cause

Certain files do not exist unless you've installed Visual Studio on your build server.

The "good practice"-way to solve it

You can install Visual Studio on your build server to fix this easily. If you worry about maintaining files, I recommend simply installing Visual Studio and forgetting about it.

Now for the other method

On your development computer with Visual Studio installed, open the folder %ProgramFiles(x86)%\MSBuild\Microsoft\VisualStudio\v10.0.

For web development, two folders are required:

  • Web
  • WebApplications

Copy these folders to the same place on the build server.

Your awesome programs should be building and publishing properly now, but read on if the problem persists.

Files copied but still doesn't work?

In your project file (*.csproj), make sure to import the right .target file before trying to use "_WPPCopyWebApplication".

Here's example; this code automatically publishes a web application/service on the file system, after being buildt:

  <Import Project="$(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v10.0\WebApplications\Microsoft.WebApplication.targets" />
  <Target Name="AfterBuild">
    <MSBuild Condition="'$(hasBuilt)' == ''" Projects="$(ProjectPath)" Properties="hasBuilt=true;Platform=$(Platform);Configuration=$(Configuration);WebProjectOutputDir=published" Targets="ResolveReferences;_WPPCopyWebApplication" />
  </Target>

In case you wonder where to put this, then somewhere at the bottom should be fine.

Check list
  1. Import Microsoft.WebApplication.targets, not Microsoft.Web.Publishing.targets. The latter will be imported automatically, so specify only the first one.
  2. Use $(MSBuildExtensionsPath32), not $(MSBuildExtensionsPath). The files don't exist under $(MSBuildExtensionsPath).

Decent reasons to only copy the files

  • Visual Studio 2013 was released just about a day ago, and I don't think we'll see (m)any updates for VS 2010.
  • The difference in size is several gigabytes (VS 2010) vs less than one megabyte (copy the files).

Conclusion

There are at least two ways to solve the problem; the right one depends on your needs.

If you install Visual Studio, the pro is that you never have to think about the files again ‒ not even when updates are released. The obvious con to copy only the necessities, is potential maintenance.

With all this in mind, I choose to copy just the files. They take less than 1 MB compared to several GB, and I think it's fair considering the age of Visual Studio 2010. If the problem persists with VS 2012 and 2013, I'm not certain that I would make the same decision at time of writing.

Wednesday, July 17, 2013

CyaSSL and 4096-bit certificates

As of version 2.6.0, CyaSSL now uses the fastmath library by default (versus the big integer library) when building with the ./configure system.

One of the less portable aspects of fastmath is the need for fixed buffers to reduce dynamic memory use. By default, these buffers allow a 2048 bit X 2048 bit multiply into a 4096 bit buffer. Since most sites are using 2048 bit RSA keys this is fine. But for those sites/users that have a 4096 bit RSA key the fastmath buffer size needs to be increased to 8192. Since your certs use 4096 bit RSA keys, you'll need to increase the size by modifying the define

FP_MAX_BITS

in <cyassl_root>/cyassl/ctaocrypt/tfm.h, and setting it to 8192.

Sunday, February 17, 2013

Load SSL certificate from memory using cURL and CyaSSL

The cacertinmem sample in cURL shows you how to do this with OpenSSL, but it's slightly different if you use CyaSSL, and actually simpler.

From the CyaSSL manual:

To use the extension define the constant NO_FILESYSTEM and the following functions will be made available:
int CyaSSL_CTX_load_verify_buffer(...)
int CyaSSL_CTX_use_certificate_buffer(...)
int CyaSSL_CTX_use_PrivateKey_buffer(...)
int CyaSSL_CTX_use_certificate_chain_buffer(...)
Use these functions exactly like their counterparts that are named fileinstead of buffer. 
And instead of providing a filename provide a memory buffer

Use the NO_FILESYSTEM preprocessor definition when building CyaSSL, cURL, as well as in your own project.

In my case, I wanted to load a bundle of CA certificates, and would therefore use CyaSSL_CTX_load_verify_buffer() (please turn to the CyaSSL API reference).

In addition, you must be able to use the SSL callback function which isn't supported in cURL (version 7.29.0) when using SSL libraries other than OpenSSL. To solve this problem, read my article about the SSL context function with cURL and CyaSSL.

Bundle of CA root certificates

Useful when you don't know which SSL providers were used. I got it from curl.haxx.se.

I compressed the CA certificates with GZip before embedding them. Poco (C++ framework) took care of decompresson.

Reduction: 115 KB / ~45% (245 KB down to 130 KB).

Probably sensible enough if GZip (or whatever you want to use) is used in your application already elsewhere (thinking about increased code size).

main.cpp
#include "CaCert.h"

#include <curl/curl.h>
#include <cyassl/ssl.h>

#include <string>

// ...

CURLcode sslContextCallback(void* ctx) {
    // Load CA certificates from memory
    CaCert cert;
    const std::string& certData = cert.GetData();
    CyaSSL_CTX_load_verify_buffer(
        reinterpret_cast<CYASSL_CTX*>(ctx),
        reinterpret_cast<const unsigned char*>(certData.c_str()),
        certData.size(),
        SSL_FILETYPE_PEM);
    return CURLE_OK;
}

// ...
curl_easy_setopt(curl, CURLOPT_SSL_CTX_FUNCTION, *sslContextCallback);
CaCert.h
#ifndef __CaCert__
#define __CaCert__

#include <string>

class CaCert {
public:
    const std::string& GetData();
private:
    static const unsigned char m_cacert_pem_gz[];
    std::string m_cacert_pem;
};

#endif // guard
CaCert.cpp
#include "CaCert.h"
#include "cacert.pem.gz.h"
#include <Poco/InflatingStream.h>
#include <sstream>

const std::string& CaCert::GetData() {
    if (m_cacert_pem.empty()) {
        std::string gzstr(reinterpret_cast<const char*>(&m_cacert_pem_gz[0]), sizeof(m_cacert_pem_gz));
        std::istringstream istr(gzstr);
        Poco::InflatingInputStream inflater(istr, Poco::InflatingStreamBuf::STREAM_GZIP);
        std::stringstream ostr;
        ostr << inflater.rdbuf();
        m_cacert_pem = ostr.str();
    }

    return m_cacert_pem;
}
cacert.pem.gz.h

This file was generated using Hex Workshop and modified slightly.

// Generated by BreakPoint Software's Hex Workshop v6.6.1.5158
//   http://www.hexworkshop.com
//   http://www.bpsoft.com
//
//  Source File: C:\code\curltest\cacert.pem.gz
//         Time: 17.02.2013 04:55
// Orig. Offset: 0 / 0x00000000
//       Length: 133503 / 0x0002097F (bytes)
const unsigned char CaCert::m_cacert_pem_gz[133503] = { /*...*/ };

Reduce size and memory footprint

We don't always need all of the CA root certificates. We can remove all of the certificates we don't need, and any comments. In my case, I needed only StartCom certificates.

I skipped compression since the file was only 7.37 KB.

Reduction: 236.63 KB / ~96.9%.

main.cpp
#include "StartComCaCert.h"

#include <curl/curl.h>
#include <cyassl/ssl.h>

#include <string>

// ...

CURLcode sslContextCallback(void* ctx) {
    // Load CA certificates from memory
    StartComCaCert cert;
    const unsigned char* certData = cert.GetData();
    CyaSSL_CTX_load_verify_buffer(
        reinterpret_cast<CYASSL_CTX*>(ctx),
        certData,
        cert.GetSize(),
        SSL_FILETYPE_PEM);
    return CURLE_OK;
}

// ...
curl_easy_setopt(curl, CURLOPT_SSL_CTX_FUNCTION, *sslContextCallback);
StartComCaCert.h
#ifndef __StartComCaCert__
#define __StartComCaCert__

#include <string>

class StartComCaCert {
public:
    const unsigned char* GetData() const;
    unsigned int GetSize() const;
private:
    static const unsigned char m_cacert_StartCom_pem[];
};

#endif // guard
StartComCaCert.cpp
#include "StartComCaCert.h"
#include "cacert_StartCom.pem.h"

const unsigned char* StartComCaCert::GetData() const {
    return &m_cacert_StartCom_pem[0];
}

unsigned int StartComCaCert::GetSize() const {
    return sizeof(m_cacert_StartCom_pem);
}
cacert_StartCom.pem.h

This file was also generated.

// Generated by BreakPoint Software's Hex Workshop v6.6.1.5158
//   http://www.hexworkshop.com
//   http://www.bpsoft.com
//
//  Source File: C:\code\curltest\curltest\cacert_StartCom.pem
//         Time: 18.02.2013 01:23
// Orig. Offset: 0 / 0x00000000
//       Length: 7547 / 0x00001D7B (bytes)
const unsigned char StartComCaCert::m_cacert_StartCom_pem[7547] = { /*...*/ }

It'll be ridiculous to continue, but let's do it!

Looking at the certificates as they are stored in PEM format, our precious bytes are obviously wasted while it tries to look pretty to humans.

While keeping the base-64-encoded data (which also wastes space), let's go ahead and write a small Python script to minify the file.

minify_cert_pem.py

It removes everything at the beginning of the file (comments, newlines), removes repeated equal signs, and newlines mixed in the base-64-encoded data.

import re

def stripNewlines(match):
    return match.group(1) + match.group(2) + match.group(3).replace("\n", "") + match.group(4)

inFile = open("cacert_StartCom.pem", "r")
caCertData = inFile.read()

# Strip comments and everything else in the beginning of the file
result = re.sub(r"(?s)^.*?\n(\w)", r"\1", caCertData)
# Strip the newlines we can strip
result = re.sub(r"(?s)(=)+\s*(\s-----BEGIN CERTIFICATE-----\s)(.*?)(\s-----END CERTIFICATE-----\s)\s*", stripNewlines, result)
# Strip remaining newlines at the end
result = result.rstrip()
outFile = open("cacert_StartCom.min.pem", "w")
outFile.write(result)

Reduction: 175 bytes / ~2.32%.

Just for the sake of it, minify the original cacert.pem

Reduction: 7.53 KB / 3%.
Comparing old and new GZipped file: 5.98 KB / 4.59% reduction.
Total reduction: 121 KB / 49.32%.

SSL context function with cURL and CyaSSL

For some reason, cURL disables this functionality when using SSL libraries other than OpenSSL. To solve the problem, we need to dive into the cURL source code for a bit.

This code applies for cURL version 7.29.0.

Source file: cyassl.c, cyassl_connect_step1()

Declare and define variable at the top:

CURLcode retcode = CURLE_OK;

The rest of the code:

#ifndef NO_FILESYSTEM
// ...
#else
/* give application a chance to interfere with SSL set up. */
if(data->set.ssl.fsslctx) {
    retcode = (*data->set.ssl.fsslctx)(data, conssl->ctx, data->set.ssl.fsslctxp);
    if(retcode) {
        failf(data,"error signaled by ssl ctx callback");
        return retcode;
    }
}
#endif /* NO_FILESYSTEM */

Most of this was copied from ssluse.c.

Source file: url.c, Curl_setopt()

Look for the following code.

#ifdef USE_SSLEAY
    /* since these two options are only possible to use on an OpenSSL-
       powered libcurl we #ifdef them on this condition so that libcurls
       built against other SSL libs will return a proper error when trying
       to set this option! */
  case CURLOPT_SSL_CTX_FUNCTION:
    /*
     * Set a SSL_CTX callback
     */
    data->set.ssl.fsslctx = va_arg(param, curl_ssl_ctx_callback);
    break;
  case CURLOPT_SSL_CTX_DATA:
    /*
     * Set a SSL_CTX callback parameter pointer
     */
    data->set.ssl.fsslctxp = va_arg(param, void *);
    break;
  case CURLOPT_CERTINFO:
    data->set.ssl.certinfo = (0 != va_arg(param, long))?TRUE:FALSE;
    break;
#endif

Change the #ifdef:

#if defined(USE_SSLEAY) || defined(USE_CYASSL)

Project files: *.vcxproj

Add preprocessor definition if needed: USE_CYASSL

Using it

CURLcode sslContextCallback(CURL* curl, void* ctx, void* param) {
    return CURLE_OK;
}

// ...
curl_easy_setopt(curl, CURLOPT_SSL_CTX_FUNCTION, *sslContextCallback);

SSL context function with cCURLpp

The function signature for cURL can't be used with cURLpp.

Expected code (not working)

If you're familiar with cURL, you would probably try this first.

CURLcode sslContextCallback(CURL* curl, void* ctx, void* param) {
    return CURLE_OK;
}

// ...
using namespace curlpp;
Easy easy;
// ...
easy.setOpt(new options::SslCtxFunction(sslContextCallback)); // CURLOPT_SSL_CTX_FUNCTION in cURL

This would give the following error (VC++):

error C2198: 'CURLcode (__cdecl *)(CURL *,void *,void *)' : too few arguments for call

Code for cURLpp

CURLcode sslContextCallback(void* ctx) {
    return CURLE_OK;
}

// ...
using namespace curlpp;
Easy easy;
// ...
easy.setOpt(new options::SslCtxFunction(sslContextCallback)); // CURLOPT_SSL_CTX_FUNCTION in cURL

I'm not sure at this time how to get the optional extra data, but for now, I don't need it either.