Click here to Skip to main content
15,903,201 members
Articles / All Topics

Nice Idea, Shame It Doesn’t Work

Rate me:
Please Sign up or sign in to vote.
5.00/5 (2 votes)
13 Aug 2014CPOL2 min read 7.8K   1   2
Nice idea, shame it does not work

Over the last few days I’ve been messing around with Montecarlo algos and distributed processing. It’s struck me that despite many attempts at parallelising on a single computer in .NET, I still can’t easily beat the performance of native C.

So if I want to do better, then I should start thinking about using multiple machines and setting up a grid. But the options here seem expensive. I could buy physical hardware at £500-2000 a go or use a professional cloud service like Azure (where prices can rocket quickly) or Amazon (prices are obscure) or Google (need a PhD in pricing).

Instead, I’ve been checking out Slicify an online exchange for buying and selling compute power – a crowd sourced cloud if you will. If I could get reasonable compute power at <£40 a month as they advertise, then it might be worth a look.

As with most of the other services (except Google App Engine), Slicify uses Virtual Machine technology and more specifically sellers host an instance of Ubuntu Linux on their machine and share it with the buyer. Cool idea, but.

The problems are listed below:

  1. The download required is huge including a full instance of a VM takes 20-30 mins on my 100 Mbps line. Tiring.
  2. After installing, reinstalling, configuring and reconfiguring as per their instructions, I am not able to get Slicify to work. It just sits there attempting to launch a VM instance and erroring out. Exasperating.
  3. When you attempt to report problems, you have to copy Slicifys error log to the clipboard. After 1 day, mine was over 23k lines of text so a copy and paste locked my 4 core 16Gb system. Very shoddy.
  4. No power management or monitoring means that I can’t track whether the extra electricity bill is smaller than the amount I sell it for.

I’ve turned Slicify off as in it’s current state, it is unusable. I have no idea how easy it would be to build distributed apps with C#. I suspect that it won’t be easy even with Java. But it seems strange to have to use an entire OS where all I want is a sand boxed .NET CLR or Java VM.

Reading about Google App Engine which (on the face of it), hosts just a runtime looks promising but of the major technologies seems only to support Java.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


Written By
Technical Lead Alpha Integralis Limited
United Kingdom United Kingdom
CatchExAs aka Nick Wilton runs Alpha Integralis, a software development consultancy to companies in the City of London.

Main interests include exploring Algo-Trading, Quant Development and Physics using C# and Java.

www.nickwilton.info/Blog

Comments and Discussions

 
QuestionHave been doing distributed processing with Go Pin
CarlMCook15-Aug-14 20:02
CarlMCook15-Aug-14 20:02 
QuestionI have the same point of view but... Pin
Alexey KK15-Aug-14 8:54
professionalAlexey KK15-Aug-14 8:54 
Good day!
There is known fact that c# real numbers library is very fast, dynamic collections are well realised and difficult to overbet them on c++. There are some articles from Richter something like C# or CLR for c /c++ programmers concerning this subject. I have read some tests when c++ programmers tried to surpass but couldnt realise anything better than Microsoft programmersSmile | :) How they wanted to do it?
The main difference between c# and c++ it starting speed - for C# compilation is needed in runtime. But there are some tricks to "warm it up". First run with 0 arguments, Garbage.Collect() ( if needed and that's all).
As for C# compiled code speed: for example i ve taken one example from codeproject with 14 version of square root computing - sqrt in C# gave me better result than imported dll compiled under C++ and having only one assembler inlining.

As for algorithm realisation - i think many depends on how comfortable you feel.
In your previuos Monte Carlo article - trust me you could optimize it in 20-30 times in sequential mode. System programmer of high level could add some additional improvements to my version. Certainly code would be not as smooth as it looking now.
Unfortunately in my real time application i cant avoid data series computing on every new data coming - i use adaptive boosting and real time background optimization. But when i can avoid dataseries computing i use addition of new indicator calculation - it may give about 1000-1200 times speeding upSmile | :) for each indicator (filter).
As for clouds and other tricks - there are some NVidia cards with shared memory - didnt test yet, previuos versions without shared memory with dataseries exceeding 100.000 - 200.000 of values may be computed faster on CUDA. There is an article concerning CudaFy.Net. Author shows an example on I5 Acer or Dell NoteBook with mid level Nvidia (800 USD price) boosting (30-40times).
Alexey


modified 18-Aug-14 9:53am.

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.