Let see if I understand you correctly. You have an image classifier built on matlab code and it has been trained on some historical data set. This trained classifier has been compiled into an exe. The role of c# code here is to send new image to this exe classifier which in turn classifies this image and returns the result (say 'sad' or 'happy') to your c# code. It is working but takes too long. Am I correct? How do you know the problem lies within c# code and not the matlab code. Have you tried classifying the same new image using the trained classifier in the MATLAB environment to compare their response time?
If you are still interested in 'other efficient method', there are three ways to integrate matlab code into a c#, briefly:
1. COM
2. .NET assembly
3. C Shared library
Read the details and demo from the MATLAB Central
Integrating MATLAB with C#.