Verify your request with LINQPad:
http://www.linqpad.net/[
^].
Using that tool, you will have an idea how your request is translated into SQL and the time it takes.
Does the data come from a single table or multiple tables? If it come from multiple tables, the first thing to check would be to see if lot of round-trip are made to the server.
And effectively for large tables, you have to uses paging or virtual loading of the data.