IBM Cloudant HTTP API in a Nutshell

Introduction

This article is the second one of five, focused on practical examples of how to start with IBM Cloudant and how to use it in real life IoT solutions. The first article demonstrated the basic CRUD operation that you need to start working with the database engine. This post will delve deeper and discuss further details. We will try to use different technologies to manage data via Cloudant API.

The source code of the examples will be provided for download to help readers to better understand the concepts and the implementation details.

Introduction to Cloudant REST API

If you are building apps or websites you've probably heard of REST API, but you may not be sure what these things are, whether you should use them, or how to get started. If this sounds familiar, then this part of the article is for you.

It is good to clarify: Why REST API? This tutorial is focused on Cloudant HTTP API.

Are REST and HTTP the same thing?

No, they are not. HTTP stands for HyperText Transfer Protocol and is a way to transfer files. This protocol is used to link pages of hypertext in what we call the world-wide-web. However, there are other transfer protocols available like FTP and gopher, though they are less popular.

REpresentational State Transfer, or REST, is an architectural pattern. It sets some rules and constrants that ensure a scalable, relianble and easily extensible system. REST by itself is not a new invention, but it's the documentation on such systems like the world-wide-web.

REST vs HTTP

The most RESTful APIs use HTTP as a transport layer, but it is not mandatory. It is possible to implement a REST API using different protocol. It's perfectly possible to use other transfer protocols like SNMP, SMTP and others to use, and your API could still very well be a RESTful API. Actually in our case IBM Cloudant HTTP API is a REST API. We will you both expressions in this article.

Background

API: what and why?

The term API stands for 'Application Programming Interface.' In the world of web development the term 'API' is synonymous with online web services which client apps can use to retrieve and update data. These online services have had several names/formats over the years such as SOAP. However, the current popular choice is to create a REST (or RESTful) API.

We will discuss why REST is now preferred later, but first let's examine why you would even bother with an API.

Modern applications have multi-tired architecture, which include several kind of web apps and mobile apps on various platforms. The schema below demonstrates the basic architecture of these applications without API. In this case, each client app has its own embedded business logic.

No API Architecture

Client applications have their own business logic (these applications contain both: the presentation layer and the business logic layer). This approach makes the client applications more complex and it needs more effort to maintain these applications. Support of these kinds of solutions will be very expensive and difficult.

If we look at the same architecture with a central API which hold all the business logic

Introduction to the HTTP API Architecture

This section is a short introduction to HTTP API Architecture. This information is from (https://blogs.msdn.microsoft.com/martinkearn/2015/01/05/introduction-to-rest-and-net-web-api/ )

Each application uses the same API to get, update and manipulate data.

HTTP is a request and response system, including a client and the back-end endpoint. The client sends a request to an endpoint and the endpoint responds.

There are several key elements of HTTP that you should consider:

• Resources
• Request Verbs
• Request Headers
• Request Body
• Response Body
• Response Status codes

In the context of a REST API, resources typically represent your data entities. The verb that is sent with the request informs the API what to do with the resource, for example a GET request gets data about an entity, POST requests create a new entity.

The table below shows some typical requests you can see in an ecommerce API:

API Reference

The official API reference is available here: https://docs.cloudant.com/api.html

This tutorial covers practical examples how to use the Cloudant HTTP and also including quotes from official documentation. It covers several parts, related to the API.

• HTTP Headers and HTTP Status Codes
• Authentication
• Databases
• Documents
• Query

Other parts will be implicity referred in the next parts of this training.

Cloudant HTTP API

All details about HTTP API you can find in the Cloudant documentation: ( https://docs.cloudant.com/http.html ) Here we will simply mention just the most important parts of this API:

HTTP Headers

The supported HTTP request headers include:

• Accept
• Content-Type
• If-None-Match

Accept

The Accept header is used to specify the allowed data types that can be used as a type of the server response. It specifies the format can be a list of one or more MIME types, separated by colons. The use of the Accept header in Cloudant queries is not required, but is highly recommended. It helps to ensure that the data returned can be processed by the client.

Content-Type

The Content-Type header specifies the content type of the returned information within the request.

Content-Encoding

This header specifies the encoding of the request body.

HTTP STATUS CODES

With the interface to Cloudant working through HTTP error codes and statuses are reported using a combination of the HTTP status code number and the corresponding data in the body of the response data.

• 401 - Unauthorized
• 403 - Not Found
• 405 - Resource Not Allowed
• 406 - Not Acceptable
• 409 - Conflict
• 412 - Precondition Failed
• 415 - Bad Content Type

Authentication

There are two ways that clients can provide credentials (authenticate) with Cloudant: Basic authentication and Cookie authentication. Basic authentication is authentication per request. Cookie authentication is similar to having a key (based on the cookie) and it is up to the client to decide when to leave. The key is a cookie named AuthSession. More details about the authentication you can find in the Cloudant documentation: (https://docs.cloudant.com/authentication.html)

Database

There are several main operations over databases:

• Create a new database
• Read information about a specified database
• Delete a specified database
• Get all databases from a specific account
• Get all the documents in a database

When you have a REST request against a specified database, the required URL is
https://$USERNAME.cloudant.com/$DATABASE

Create a new database

Creating a new database needs a PUT request and deleting a database needs POST request. All other requests are GET request.

List all databases in an account

This functionality can be realized using a GET request against https://$USERNAME.cloudant.com/_all_dbs.

List all documents in a database

The information in this section is from Cloudant documentation: (https://docs.cloudant.com/database.html)

To list all the documents in a database, make a GET request against

https://$USERNAME.cloudant.com/$DATABASE/_all_docs

The _all_docs endpoint accepts these query arguments:

Getting all documents in a database:

GET /_all_dbs HTTP/1.1

Getting all documents in a database that match at least one of the specified keys:

GET /_all_docs?keys=["somekey","someotherkey"] HTTP/1.1

Documents

You can have CRUD operations with documents via Cloudant API. Documents are JSON objects. Documents are containers for your data and are the basis of the Cloudant database. All documents must have two fields: a unique _id field, and a _rev field. Field names beginning with the underscore character (_) are reserved in Cloudant.

Below are several examples how to work with documents.

CREATE

To create a document, make a POST request with the document's JSON content to
https://$USERNAME.cloudant.com/$DATABASE.

Creating a document:

POST /$DATABASE HTTP/1.1
Content-Type: application/json
    {
      "_id": "phone",
      "brand": "Nokia",
      "prices": {
        "530": 49.99,
        "630": 74.99,
        "950": 450.99
    }
}

Example response:

{   
	  "ok":true,   
	  "id":"phone",   
	  "rev":"1-2902191555"  
}

READ

To retrieve a document make a GET request to https://$USERNAME.cloudant.com/$DATABASE/$DOCUMENT_ID. If you do not know the _id for a particular document you can query the database for all documents.

Query Parameters

This is a list of parameters you can add to the URL in the usual way, e.g. /db/doc?attachments=true&conflicts=true. All parameters are optional.

Reading a document:

GET /$DATABASE/$DOCUMENT_ID HTTP/1.1

Example response:

{
  "_id": "phone",
  "_rev": "1-2902191555",
  "brand": "Nokia",
  "prices": {
        "530": 49.99,
        "630": 74.99,
        "950": 450.99
  }
}

More details about the Cloudant domuments API you can find here: (https://docs.cloudant.com/document.html)

Query

Cloudant Queries are features providing a declarative JSON querying syntax for Cloudant databases. There are two different types of queries in the context of IBM Cloudant: MapReduce Views (where the index type is JSON), and Search Indexes (where the index type is text). This section mentions just the most important things, related to these queries. More details you can find in the next two articles of this tutorial.

You have the following operations with queries:

• CREATING AN INDEX
• DELETING AN INDEX
• LIST INDEXES
• FIND DOCUMENTS

SELECTORS

Selectors are elements, related to the querying, The selector syntax requires you to specify one or more fields, and the corresponding values required for those fields.

Example of creating a new index for the field called region:

{   
	  "index": {   
	      "fields": ["region"]   
	  },   
	  "name" : "region-index",   
	  "type" : "json"  
} 

More details about Cloudant Queries you can find here: (https://docs.cloudant.com/cloudant_query.html)

Tools for REST API Requests

cURL

cURL is a helper application that allows from the command line to make a data transfer operations computer using different protocols. cURL is available for almost all existing platforms

You can download the latest versions of cURL from this link: https://curl.haxx.se/

Fiddler

Fiddler is a tool that captures HTTP and HTTPS traffic and logs it for the user to review. This application can also be used to modify ("fiddle with") HTTP traffic for troubleshooting purposes as it is being sent or received. By default, traffic from HTTP(S) stack is automatically directed to the proxy at runtime, but any web application can be configured to route its traffic through Fiddler. Fidler is written by Eric Lawrence. The first version is released in 2003.

From 2012 Fiddler is a product of Telerik ( now Progress Software ).

You can download Fiddler here: https://www.telerik.com/download/fiddler

If you prefer to use browser extensions there are several extensions for Chrome and Firefox. Below are mentioned some of the more popular extensions:

Chrome REST extensions

• Postman: http://www.getpostman.com/
• Insomnia REST Client: http://insomnia.rest/
• Advanced REST client: https://chromerestclient.appspot.com/

Firefox add-ons:

• REST Easy: https://addons.mozilla.org/en-US/firefox/addon/rest-easy/
• RESTClient: https://addons.mozilla.org/en-US/firefox/addon/restclient/
• RESTer: https://addons.mozilla.org/en-US/firefox/addon/rester/

Sample screen from Postman:

Using the Code

There are many libraries that you can use for different platforms with REST services. We will consider implementations on .Net and JavaScript/jQuery, that can be used.

.NET Implementation of REST Client for Cloudant

The simplest way to implement a REST client for IBM Cloudant is to use some of the REST client libraries for .NET.

Probably the most popular framework is RestSharp http://restsharp.org/. You can download the source code from GitHub or just to use the NuGet manager to refer the libraries.

You can see below a sample code that gets database information of the specific database. Two different requests executions are demonstrated - sync and async:

class Program
    {
        static void Main(string[] args)
        {
            var client = new RestClient();
            client.BaseUrl = new Uri("https://mikeamm.cloudant.com/pouchrecords");
            client.Authenticator = new HttpBasicAuthenticator("username", "password");

            var request = new RestRequest(Method.GET);

            // easily add HTTP Headers
            request.AddHeader("Content-Type", "application/json");

            //unfinite loop until esc key is pressed
            Console.WriteLine("Press ESC to stop");
            do
             {

                    IRestResponse syncresponse = client.Execute(request);
                Console.WriteLine(syncresponse);

                // easy async support
                client.ExecuteAsync(request, response => {
                    Console.WriteLine(response.Content);
                });
            } while (Console.ReadKey(true).Key != ConsoleKey.Escape);

        }
    }

The sample response is shown in the screenshot below:

If you don't want to go deep in details you can just use the System.Net.Http.dll from the previous article for IBM Cloudant.

JavaScript and jQuery with Cloudant REST API

You can manage the REST API using the XMLHttpRequest Object (JavaScript) or the jQuery.ajax() method with jQuery. The example below demonstrates how to use XMLHttpRequest Object:

function createRequest() {
  var result = null;
  if (window.XMLHttpRequest) {
    // FireFox, Safari, etc.
    result = new XMLHttpRequest();
    if (typeof xmlhttp.overrideMimeType != 'undefined') {
      result.overrideMimeType('text/xml'); // Or anything else
    }
  }
  else if (window.ActiveXObject) {
    // MSIE
    result = new ActiveXObject("Microsoft.XMLHTTP");
  } 
  else {
    // No known mechanism -- consider aborting the application
  }
  return result;

An XMLHttpRequest object allows you to send a request, either GET or POST; however, it does not immediately return a value.

var req = createRequest(); // defined above
// Create the callback:
req.onreadystatechange = function() {
  if (req.readyState != 4) return; // Not there yet
  if (req.status != 200) {
    // Handle request failure here...
    return;
  }
  // Request successful, read the response
  var resp = req.responseText;
  // ... and use it as needed by your app.
}

Sending the Request

Once we have created the request object and set up its callback function, it is time to issue the request:

req.open("GET", url, true);
req.send();

How to consume a RESTful service using jQuery

It is an actual case how to consume a RESTful service using jQuery - the most poular JavaScript framework. In jQuery there is the jQuery.ajax() function, which can be used for communication with REST services.

The example code below is copied from this post: (http://www.lm-tech.it/Blog/post/2013/05/08/How-to-consume-a-RESTful-service-using-jQuery.aspx)

jQuery.ajax({
    type: "GET|POST|DELETE|PUT",
    url: url,
    data: data,
    dataType:"text|html|json|jsonp|script|xml"
    success: success_callback,
    error: error_callback
});

How to use this in the modern WEB applications? Actually it is a relatively big effort to implement all queries using a REST interface, though it is possible to use some libraries or PouchDB.

PouchDB

PouchDB is an lightweight open-source JavaScript database that is designed to run well within the browser. The implementation of PouchDB is inspired by Apache CouchDB. This solution was created to help web developers build applications that work as well offline as they do online. PouchDB enables WEB applications to store data locally while offline. The synchronization with CouchDB and compatible servers is relaized when the application is back online. More details you can find at the PouchDB website (https://pouchdb.com/ )

IBM Cloudant is based on CouchDB. The good thing is that PouchDB is also compatible with Cloudant.

You need to consider some details if you need to sync databases: You need to get data from and to Cloudant, syncing with your PouchDB client.

PouchrecordsObj = function (databasename, remoteorigin) 
{
    
 	// define proeprties 
 	this.opts = 
	{
      
	continuous: true
    
	};
    
	this.pdb = new PouchDB(databasename);
    
	this.remote = remoteorigin + '/'+databasename, this.opts;
 
	his.pdb.replicate.from(this.remote, this.opts);


};

Bi-directional replication is implemented with these functions:

this.pdb.replicate.to(this.remote, this.opts);
    
this.pdb.replicate.from(this.remote, this.opts);

The demonstrated client below allows to work online and offline with Cloudant. If an Internet connection is not available, data is stored in the local storage of the browser and when the connection is restored the local data is synchronized with the Cloudant database.

Below you can see some screen from the sample PouchDB client.

Source code:

Examples are available in GitHub:

• Cloudant .NET Rest client ( using RestSharp )
• Cloudant IoT simulated device
• Sample PouchDemo client, synchronizing data about IoT Reports with Cloudant database

The IoT client simulated device is the extended version of this one from the previous article "Getting Started with Cloudant" .

Both IoT client and the WEB Pouch client are parts of the IoT solution, discussed in the next articles.

Summary

This tutorial is on how to start using the Cloudant HTTP API. It covers basics of REST services, different ways to send REST requests using:

• Developer tools like Fiddler REST browser extensions
• RestSharp for .NET
• JavaScript and jQuery for WEB applications

There are also examples how to use PouchDB for CRUD operations in WEB clients, that will simplify a lot the implementation of Cloudant WEB clients.