My task is to show Huge data on UI. DataBase is returning around 25 sets and 5 sets are having around 10000 rows and rest are having around 1000 rows, so total around 80k rows. I was trying it in demo application where I've created very basic WebApi, in which I am just getting data from DataBase and filling my class using SqlDaraReader(See below code).
public HttpResponseMessage Get()
{
HugeDataSetClass obj = new HugeDataSetClass();
string connectionString = string.Empty;
connectionString = System.Configuration.ConfigurationManager.ConnectionStrings["DefaultConnection"].ConnectionString;
string jsonResult;
using (SqlConnection connection = new SqlConnection(connectionString))
{
connection.Open();
using (SqlCommand command = new SqlCommand("sp_testHugeData", connection))
{
command.CommandType = CommandType.StoredProcedure;
using (SqlDataReader reader = command.ExecuteReader())
{
obj.obj1 = new BindingList<HugeDataModel>();
while (reader.Read())
{
HugeDataModel obj111 = new HugeDataModel();
obj111.id = reader.GetInt32(0);
obj111.name1 = reader.GetString(1); obj111.name2 = reader.GetString(2); obj111.name3 = reader.GetString(3);
...
...
obj111.name89 = reader.GetString(89); obj111.name90 = reader.GetString(90);
obj.obj1.Add(obj111);
}
...
...
reader.NextResult();
obj.obj29 = new BindingList<HugeDataModel>();
while (reader.Read())
{
HugeDataModel obj1 = new HugeDataModel();
obj1.id = reader.GetInt32(0);
obj1.name1 = reader.GetString(1); obj1.name2 = reader.GetString(2); obj1.name3 = reader.GetString(3);
...
...
obj1.name89 = reader.GetString(89); obj1.name90 = reader.GetString(90);
obj.obj29.Add(obj1);
}
}
}
}
jsonResult = Newtonsoft.Json.JsonConvert.SerializeObject(obj);
var response = Request.CreateResponse(HttpStatusCode.OK);
response.Content = new StringContent(jsonResult, Encoding.UTF8, "application/json");
return response;
}
In above code I am getting "OutOfMemory" exception while Serializing obj. If I comment this serialization code and response thing and return object of HugeDataSetClass then on browser(I've tried all the possible browsers) I get the data but before displaying it browser gets crashed. Below is my UI(AngularJs) code:
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>Test Page To Display Huge Data</title>
<style type="text/css">
div {
width: 100%;
margin: 5px;
}
table {
border: 2px solid red;
}
</style>
<script src="Scripts/jquery-1.8.2.min.js"></script>
<script src="Scripts/angular.min.js"></script>
<script type="text/javascript">
var app = angular.module("hugeDataApp", []);
app.controller("hugeDataController", function ($scope, $http) {
$scope.GetHugeData = function () {
$http.get("http://localhost:60533/api/HugeData").then(
function (response) {
$scope.HugeDataSets = response.data;
}
);
};
});
</script>
</head>
<body ng-app="hugeDataApp" ng-controller="hugeDataController">
<input type="button" id="btnLoad" value="Load" ng-click="GetHugeData()" />
<br />
<div id="div1">
<table>
<tr ng-repeat="x in HugeDataSets.obj1">
<td>{{x.id}}</td>
<td>{{x.name1}}</td>
<td>{{x.name2}}</td>
<td>{{x.name3}}</td>
...
...
<td>{{x.name89}}</td>
<td>{{x.name90}}</td>
</tr>
</table>
</div>
<div id="div29" style="width: 100%">
<table>
<tr ng-repeat="x in HugeDataSets.obj29">
<td>{{x.id}}</td>
<td>{{x.name1}}</td>
<td>{{x.name2}}</td>
<td>{{x.name3}}</td>
...
...
<td>{{x.name89}}</td>
<td>{{x.name90}}</td>
</tr>
</table>
</div>
</body>
</html>
So, if I show small amount of data say 1000 rows(worst case) and keep the original huge set at client side for further computation, will it work? And suppose I can keep huge data at client side but it's not allowing me to serialize it in WebApi, or I should not at all serialize it?
Submit an article or tip