Click here to Skip to main content
15,867,330 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
See more:
I am using MVC4 .NET framework 4.5 with jQuery. Backend is in Hadoop. Business users want the single page UI to be extremely fast and fluid. No lags.

I have to display 5,00,000 (5 hundred thousand/5 lac) records (Show names) in searchable Multi select drop down list.

Please suggest how can I achieve fast and responsive UI with that volume of data so that user can search for record quickly and select multiple show names in no time. It is a business critical application and cannot afford lags. Thank you.

What I have tried:

1) Till now I have tried caching all the Show names in ASP.NET cache when Application loads in IIS and then throw it as JSON array on UI. But the multi select is unable to handle that volume of data (I am using multi select drop down control from data-behaviors/examples/js/jquery-multiselect at master · Marak/data-behaviors · GitHub[^] ).

I tried to load the drop down list from JSON array as user searches. This is close but still not very fluid.

2) Async load data in drop down while search. But this approach lags.

3) Load everything at ones in drop down. Doesn't work.
Posted
Updated 22-Jan-17 14:06pm
v3

There is no such thing as "no lag" if you insist on trying to transfer 500,000 records to a client. Even less so if you try to HTML format that much data.

You have to only display what the user can actual see. This means paging. Showing only one page of data at a time, whatever will fit on screen. If you're going to do filtering, you have to do the filtering in the database so it only transfers the number of filtered records that will fit in a page. Get the database to do the heavy lifting of filtering and paging as much as possible. That will remove the most "lag".

You're not going to eliminate the lag completely. You can only do what you can to minimize it.
 
Share this answer
 
Comments
Wild-Programmer 22-Jan-17 19:53pm    
The problem is user wants to search any random item and at the same time wants to see all the data. They will scroll through the list and select whatever they find useful or type in keywords if they know the item.
Dave Kreskowiak 22-Jan-17 20:07pm    
It's physically impossible to show 500,000 records on any monitor smaller than a football pitch.

You MUST do paging, even with scrolling. There is no other way to do this without lag.
[no name] 22-Jan-17 20:18pm    
Your "user" has no idea what they are asking for. It's up to you to set their expectations not blindly do whatever they think they want.
Quote:
Business users want the single page UI to be extremely fast and fluid. No lags.
I have to display 5,00,000 (5 hundred thousand/5 lac) records (Show names) in searchable Multi select drop down list.

The 2 sentences are contradictory, you can't have both at same time.
A drop down list of 500k lines will be also impractical to use. A drop down list with 500 lines is already difficult to use.
The multi select is a bad idea with 500k lines.
Quote:
What I have tried:
There is numerous ways to fail o, what you tried, It will be difficult to check what you done without code.

I would try to use a simple field with a drop down loaded as user type and a separate grid that hold all the selections.
 
Share this answer
 
Sometimes you have to "educate" the user/client about what is possible, and clarify with/for them what functionality they actually need.

I cannot imagine anyone scrolling around in a list-UI with that many records: surely there is some organizing/sorting principle by which you can categorize the data into clusters; or the user's needs can be met by a search facility with wild-card features.

If possible, meet the client and observe what they are doing now, and try to get a sense of what capabilities they need.
 
Share this answer
 
Comments
Wild-Programmer 22-Jan-17 20:46pm    
You are right, probably I need to talk to business users. We are not magicians, we have limitations sometimes.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900