Basic internet requests

Introduction

The last few days in the office I've been hearing tons of curses regarding how cumbersome it is to do some internet requests. Basically, a colleague wrote some Qt code wanting to do some simple GET. QNetworkAccessManager, QNetworkReply, signals, slots, deleteLater(), some 3-7 classes were involved - a little bit too much. So I grabbed some simple classes I was using on a personal project and threw them to Vlad (the screaming colleague, not the impaler) and voila - problem solved.

Background

Familiarity with Wininet is useful and very basic C++ classes usage (wstring, stream). For understanding the Justin.TV REST API - please go to the documentation page here.

Starting: From backwards

We'll start here with what we want to do. In this case, I'm writing a stream player and I'm implementing justin.tv support - I'm making the stream/summary justin.tv API call.

int __cdecl 
wmain(int argc, wchar_t** argv) {
    // get justin.tv stream summary
    jtv_test_streamsummary();
    
    return 0;
    UNREFERENCED_PARAMETER(argc);
    UNREFERENCED_PARAMETER(argv);
}

The stream/summary call is calling in turn a "justin.tv" API call - implemented in the jtv_api class (see below).

void jtv_test_streamsummary() {
    std::wstring data;
    if(jtv_api::streamsummary(data, L"", L"", L"")) {
        std::wcout << data << std::endl;
    }
}

The jtv_api class (trimmed version for this purpose) implements a public call streamsearch; I won't get into the details of the jtv REST API - stream/search returns XML or JSON with aggregate information about all live channels. (The HTTP call composing is minimal - simply appends to the base bath /api/stream/summary the arguments of this call - channel, category, and language which filters the result to the specified values, if any - so don't expect this to work on special non-escaped cases or so. Remember - it is a sample.)

class jtv_api {
    private:
        jtv_api() {
            }
        ~jtv_api() {
            }
    private:
        jtv_api(const jtv_api&);
        jtv_api& operator=(const jtv_api&);
public:
    static bool streamsummary(std::wstring& data, 
        const std::wstring& channel, 
        const std::wstring& category, 
        const std::wstring& language
    ) {
        //    compose request relative url
        std::wstring rpath = L"/api/stream/summary";
        rpath += L".xml";
        //    add query part
        std::wstring qry = L"";
        if(!channel.empty()) {
            if(!qry.empty()) {
                qry += L"&"; }
            qry += L"channel=";
            qry += channel;
        }
        if(!category.empty()) {
            if(!qry.empty()) {
                qry += L"&"; }
            qry += L"category=";
            qry += category;
        }
        if(!language.empty()) {
            if(!qry.empty()) {
                qry += L"&"; }
            qry += L"language=";
            qry += language;
        } 
        //    append query part, if any
        if(!qry.empty()) {
            rpath += L"?";
            rpath += qry;
        }
        return _execute_request(data, rpath);
    }

In turn, all the public calls - the public call, in this version :) - routes to the _execute_request private helper which does all the HTTP and converts the byte response to std::wstring using the _response_to_string helper:

private:
    static bool _response_to_string(std::wstring& data, const vector_t<BYTE>& response) {
        int cv = ::MultiByteToWideChar(CP_UTF8, 0, 
           (LPCSTR)response.operator const BYTE *(), response.size(), NULL, 0);
        if(cv != 0) {
            wchar_t* wz = new wchar_t[cv];
            if(!wz) {
                return false; }
            memset(wz, 0, cv * sizeof(wchar_t));
            int cv2 = ::MultiByteToWideChar(CP_UTF8, 0, 
               (LPCSTR)response.operator const BYTE *(), response.size(), wz, cv);
            data = wz;
            delete []wz;
            return true;
            UNREFERENCED_PARAMETER(cv2);
        }
        return false;
    }

Our point of interest is the _execute_request call:

static bool _execute_request(std::wstring& data, 
    const std::wstring& request, const std::wstring& verb = L"GET") {
/*1*/    internet_api api;
/*2*/    internet_session session(&api); 
    if(!session.open()) {
        return false; }
/*3*/    internet_connection connection(&api, &session, L"api.justin.tv"); 
    if(!connection.open()) {
        return false; }
    //    do request
/*4*/    internet_request req(&api, &connection); 
    if(!req.open(verb.c_str(), request.c_str())) {
        return false; }
    std::wstring status;
    vector_t<BYTE> response;
    if(!req.get_response(status, response)) {
        return false; }
    if(!_response_to_string(data, response)) {
        return false; }
    return true;
}

which condenses all the internet work. These calls use the internet classes as described below (all classes except internet_api are derived from the internet_handle HINTERNET wrapper class).

1. internet_api api object: nothing more than a fancy dynamic wrapper over wininet.dll used calls: InternetConnectW, InternetOpenW, InternetCloseHandle, etc.; loads wininet.dll, resolves all APIs using GetProcAddress, and is passed to all the other classes needing API calls.
2. instantiates and opens the session object, which calls InternetOpenW;

class internet_session
    : public internet_handle {
public:
    internet_session(internet_api* api) 
        : internet_handle(api) {
        }
    virtual ~internet_session() {
        }
private:
    internet_session(const internet_session&);
    internet_session& operator=(const internet_session&);
public:
    virtual bool open() {
        HINTERNET h = _api->api_InternetOpenW(
            NULL, 
            INTERNET_OPEN_TYPE_PRECONFIG, 
            NULL, NULL, 0);
        if(h == NULL) {
            return false; }
        attach(h);
        return true; }
    };

3. instantiates the connection object, which calls InternetConnectW to connect to the host:

class internet_connection
    : public internet_handle {
private:
    pointer_t<internet_session, false> _session;
    std::wstring                       _host;    
public:
    internet_connection(internet_api* api, 
             internet_session *session, const std::wstring& host) 
        : internet_handle(api)
        , _session(session)
        , _host(host) {
        }
    virtual ~internet_connection() {
        }
private:
    internet_connection(const internet_connection&);
    internet_connection& operator=(const internet_connection&);
public:
    const std::wstring& host() const {
        return _host; }
    virtual bool open() {
        if(!_session) {
            return false; }
        HINTERNET h = _api->api_InternetConnectW(
            *_session, _host.c_str(), 
            INTERNET_INVALID_PORT_NUMBER, NULL, NULL, 
            INTERNET_SERVICE_HTTP, 
            0, 0);
        if(h == NULL) {
            return false; }
        attach(h);
        return true; }
};

4. instantiates the internet_request req object, which calls HttpOpenRequestW to perform the request open (using the default verb GET), gets the request response (HttpSendRequestW), status, and data (HttpQueryInfoW, InternetReadFile), and converts the response to std::wstring.

class internet_request
    : public internet_handle {
private:
    pointer_t<internet_connection, false> _connection;
public:
    internet_request(internet_api* api, internet_connection *connection) 
        : internet_handle(api)
        , _connection(connection) {
        }
    virtual ~internet_request() {
        }
private:
    internet_request(const internet_request&);
    internet_request& operator=(const internet_request&);
public:
    virtual bool open(LPCWSTR verb, LPCWSTR object) {
        if(!_connection) {
            return false; }
        HINTERNET h = _api->api_HttpOpenRequestW(
            *_connection, 
            verb, 
            object, 
            HTTP_VERSIONW, 
            NULL, NULL, 
            INTERNET_FLAG_KEEP_CONNECTION    
            |   INTERNET_FLAG_NO_UI
            |   INTERNET_FLAG_KEEP_CONNECTION    
            |   INTERNET_FLAG_PRAGMA_NOCACHE
            |   INTERNET_FLAG_DONT_CACHE
            |   INTERNET_FLAG_RELOAD, 0);
        if(h == NULL) {
            return false; }
        attach(h);
        return true; }

public:
    bool get_response(std::wstring& status, 
        vector_t<BYTE>& response) {
        status = L"";

        if(!_api->api_HttpSendRequestW(_h, NULL, 0, NULL, 0)) {
            return false; }

        WCHAR sz[128] = L"";
        DWORD buflen = _countof(sz);
        if(!_api->api_HttpQueryInfoW(
            _h, HTTP_QUERY_STATUS_CODE, 
            (LPVOID)sz, &buflen, NULL)) {
            return false; }
        status = sz;

        //    read response
        DWORD bufrd = 0;
        do {
            BYTE respp[8192 + 1];
            memset(respp, 0, _countof(respp));
            bufrd = 0;
            if(!_api->api_InternetReadFile(
                _h, (LPVOID)respp, _countof(respp) - 1, &bufrd)) {
                return false; }
            if(bufrd == 0) {
                break; }
            for(DWORD d = 0; d < bufrd; d++) {
                response += respp[d]; }
        } while(bufrd != 0);
        response += (BYTE)'\0';
        response += (BYTE)'\0';
        return true; }
    };

Everything is encapsulated - in this example, in the _execute_request(data, rpath) call. That was the initial purpose - to have a simple way to do HTTP client requests using only simple Win32 calls and minimalistic classes.

(A note on the support classes, pointer_t and vector_t - they have for sure better equivalents on STL and boost universe for sure, but for my project, minimal dependencies (minus Operating System and the VC runtime) is a must and they serve their purpose for now.)

History

• Version 1.0 - 24 July, 2011.