Horizontal scaling application using Proxy & Service Discovery

• Sample Application for Proxy based Horizontal scaling - 1.8 KB

Introduction

In my earlier article, we discussed about horizontal scaling of application. But major focus here was to expand application processing (middle layer) and database layer as per our growing needs. In this article we will try and touch how we can expand application interactive / UI layer.  I will be writing code in javascript and will be using nodejs module to demonstrate implementation. I am assuming here you have some basic understanding of nodejs & Javascript.

I would like to get feedback on this article. Please feel free to share your comments or you can also email me at shettyashwin@outlook.com. If you like this article, please don't forget to rate it.

Background

When we start developing an application we generally see typical way user interaction handled by the application. Browser or app makes a call to application UI / API. Hosted application returns the data traversing through its layer as requested by the client. This is a typical Monolithic design. Now let’s start increasing complexity to this and see how the application starts behaving. Now we expect large no of user visiting our application due to fflash sales which are planned and also few urgent patch we needs to be developed without effecting client interaction. Now the bigger challenge here is to deploy new changes without affecting the user interaction and also to balance out the load coming on our website.

Our Approach

To overcome such issue we will have to think on two important things.

1. How we can scale up our application to process temporary load
2. How are we going make our deployment without effecting the client interaction

Now let’s take this issue one by one, let’s focus on scalability first. Instead of making the application directly accessible to end user lets route the request first. By route I mean some type of proxy service which route your application request to available application node. Let see how we can achieve this using nodejs.  

Create an empty folder and name it ProxyServer. Open a command prompt and navigate inside proxyserver folder. Once you are at proxyserver folder root, execute following command for getting dependencies which we require.

npm install http

npm install http-proxy

After successfully execution of above command, you should be able to see node_modules folder inside ProxyServer folder. Both modules will be available inside this folder. Http-proxy is the module which we will be using to route our request to available node. Now it’s time to write some code. Add a new file app.js at ProxyServer root. Open app.js in text editor or editor which you are comfortable with and add below line of code.

var http = require('http');
var httpProxy = require('http-proxy');

//load both the module which we will require

var proxy = httpProxy.createProxyServer();
// Create an instance of proxy server which will route your request.

http.createServer(function (req, res) {
    proxy.proxyRequest(req, res, { target : 'http://localhost:8080' });
    //This will route your request to another instance of application which is running on localhost 8080
}).listen(3000);

Get back to command prompt and execute 

node app.js

Once the above mentioned line is successfully executed, all the request coming at port 3000 will be routed to localhost :8080 where our actual application is running. Now instead of routing your application to single instance at port 8080 (which we have hard coded) you can create collection of available instance and then route this to anyone using round robin.

Now our first problem is partially fixed. Why I am saying “partially fixed” is because we have hard coded the URL where request is been routed. This url instance may be running or may not be running when user makes a request. Also if we are adding additional node it will be difficult to update the list without stopping the ProxyServer instance. To resolve this issue we will have to add one additional module which will work has a mediator between proxy and application node. Return back to command window and stop the running instance of ProxyServer. Time to install our new module, execute following command to get seaport module. 

npm install seaport -g

Make sure you are installing seaport globally. This is what seaport has to say about its module

Quote:

Seaport stores (host,port) combos (and other metadata) for you so you won't need to spend so much effort keeping configuration files current as your architecture grows to span many processes on many machines. Just register your services with seaport and then query seaport to see where your services are running. 

Reference : https://github.com/substack/seaport

Ok now let’s back to code changes

var http = require('http');
var httpProxy = require('http-proxy');
var seaport = require('seaport');

//load both the module which we will require

var proxy = httpProxy.createProxyServer();

// Create an instance of proxy server which will route your request.

var ports = seaport.connect('localhost', 9090);
// Connecting to seaport instance which is available at port 9090 

var i = -1;

http.createServer(function (req, res) {
    var addresses = ports.query('application-server');
    
    // if there are not workers, give an error
    if (!addresses.length) {
        console.log("Returned service unavailable, since no service was available");
        res.writeHead(503, {'Content-Type' : 'application/json'});
        res.end('Service unavailable');
        return;
    }
    
    console.log("Address detail");
    //console.log(addresses);
  
    i = (i + 1) % addresses.length;
    console.log("Route to Instance : " + addresses[i].port); 
    // Displays which instance the request was routed 

    proxy.proxyRequest(req, res, { target : addresses[i] });

}).listen(3000);

What we have done here is instead of hard coding the URL, we are now depending on seaport for getting us the list of node which are available and then routing the request to available once. We will also fetch the list of service which have been registered with seaport under "Application-Server" pool. This is important when we have multiple type of services. Seaport module will also monitor the instance which are connect to it. If one of the node crashes it will removed the node detail from the Seaport registry 

Application node will have to register themselves with seaport to be part of processing request. Yes this solves our second problem of deploying fix without impacting. Once we have deployed our new changes, newly deployed node will register itself with seaport and start processing newly request which is coming in. Slowly we will remove all the older instances by unregistering itself from seaport. In this article I have not covered unregistering logic but it is fairly simple. Details for same are also available at here

Ok let’s come back and complete few changes which are necessary. Now in application node we will have to register itself to seaport. Below is the logic to achieve the same. Create a new folder in parallel to ProxyServer and name it Server. Install http, mongoose and seaport modules as we did earlier in this folder.

var http = require('http');

var seaport = require('seaport');
var ports = seaport.connect('localhost', 9090);
//connect to seaport instance available at port localhost port 9090
console.log("Starting basic");

var mongoose = require('mongoose');
mongoose.connect('mongodb://localhost/pathDB'); 
//mongodb Server and Database information. pathDB is the name of the database

//Your schema defination. PaymentModes is the table / collection name inside PathDb database
var paymentModeSchema = mongoose.model('PaymentModes', {name: String, description: String});

// Create a basic server that responds to any request with the pi estimation
var server = http.createServer(function (req, res) {
     paymentModeSchema.find({},function(err, docs){
        res.writeHead(200, {'Content-Type' : 'application/json'});
        res.end(docs.toString());
    });
});

console.log("Starting hosting");
var hostingDetail = ports.register('application-server');
//register with Seaport instanse available at port 9090

// Listen to a specified port, or default to 8000
server.listen(hostingDetail);
console.log("hosting port : " + hostingDetail);

console.log("hosting completed");

Before running this module make sure you have installed http, mongoose and seaport modules. In code sample Mongoose module is only used to fetch data from mongoDb database. You can change the return logic and remove dependence on mongoose. Also seaport instance needs to be running. Below are the command which will help you install http, mongoose & seaport module, and also to start instance of seaport.

npm install npm 
npm install mongoose
npm install seaport
seaport listen 9090

Time to run proxyserver and application server. Open two different command prompt and navigate to root path of Proxyserver and server folder. In command prompt run node app.js. Please make sure you have three commnd prompt window running each of the below application.

1. Seaport
2. ProxyServer
3. Server

If you now hit http://localhost:3000/ in your browser, in proxyserver window you should be able to see which application node received the request.

I have attached my code set with this article, after downloading do not forget to install http, http-proxy & seaport modules in proxyserver and http, mongoose & seaport in server folder. Also the code will need mongodb instance running.

Reference

https://github.com/substack/seaport

https://www.npmjs.com/package/http-proxy