|
tijujohn83 wrote: Parallel processing is supposed to increase performance, not decrease.
Am I missing something???
Yup, you're missing the piece of information that states that threads need to be created before they execute code. Look at it as if each thread was an application; each app needs be started, needs resources, needs handling.. that's all overhead.
Threads are used to divide the work over multiple cores, if there's much work to do (anything that takes longer than a second, according to MS design guidelines)
Try copying large files from the methods, you'll find that the second thread can work on the second core, without having to wait for thread 1 I are Troll
|
|
|
|
|
hmm. you are right. I was just trying out the parallel loops in the Task process library in .net 4
and while i was trying out different things, i ran the test again, and this time the parallel version run much faster.
|
|
|
|
|
You've run into the little problem of overhead and you've approached your problem incorrectly.
First, finding the primes of the first 100000 integers takes far less time than setting up the range partitioner (by default on IList and arrays) and grabbing an extra thread out of the pool to run your code.
You've also parallelized the outside loop that runs two (3 in this case) copies of the 1 through 100000 range instead of parallelizing the range itself. What you've done is really just forced this into a Task situtation and used Parallel to do it. This is not what Parallel was designed for.
It should look more like:
var primes = from c in Enumerable.Range(1, 10000000).AsParallel Select IsPrime(c)
Sorry, if this doesn't really compile, I'm translating from VB.NET to C# having never done LINQ in C#.
|
|
|
|
|
thanks for the comments. i'll try this out.
i have one doubt,
the two streams would be like
1,5,9,13,17,...
3,7,11,15,19...
how is this different from parallelizing the range.
|
|
|
|
|
Using your code as a base, this is how I would rewrite it. Note I threw together the custom IEnumerator<int> for an enumerable ForStep. I didn't see a "steppable" version in Enumerable or Parallel anywhere, so I threw together my own. Just a reminder, these LINQ queries are not executed until the variables Prime1 and Prime3 are used.
using System;
using System.Collections;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Diagnostics;
namespace IsPrime
{
class Program
{
static void Main(string[] args)
{
Stopwatch sw = new Stopwatch();
var Primes1 = from x in EnumerableForEach.ForEach(1, 10000000, 4).AsParallel()
where IsPrime(x)
select x;
var Primes3 = from x in EnumerableForEach.ForEach(3, 10000000, 4).AsParallel()
where IsPrime(x)
select x;
sw.Start();
Console.WriteLine("Primes (1, step 4) collection size: {0}", Primes1.Count());
sw.Stop();
Console.WriteLine("Time: {0}\n", sw.Elapsed);
sw = Stopwatch.StartNew();
Console.WriteLine("Primes (3, step 4) collection size: {0}", Primes3.Count());
sw.Stop();
Console.WriteLine("Time: {0}", sw.Elapsed);
}
static bool IsPrime(int x)
{
if (x < 2) return false;
double sqrRoot = Math.Ceiling(Math.Sqrt(x));
bool isPrime = true;
for (int i = 2; i < sqrRoot; i++)
{
if (x % i == 0)
{
isPrime = false;
break;
}
}
return isPrime;
}
}
}
public class EnumerableForEach
{
public static IEnumerable<int> ForEach(int start, int limit, int step)
{
return new EnumerableForEach.ForEachIterator(start, limit, step);
}
private sealed class ForEachIterator: IEnumerable<int>, IEnumerator<int>, IDisposable
{
int _start;
int _limit;
int _step;
int _current;
public ForEachIterator(int start, int limit, int step)
{
_start = start;
_limit = limit;
_step = step;
_current = start - step;
}
public IEnumerator<int> GetEnumerator()
{
return this;
}
IEnumerator IEnumerable.GetEnumerator()
{
return this;
}
public int Current()
{
return _current;
}
public void Dispose()
{
}
object IEnumerator.Current
{
get { return _current; }
}
public bool MoveNext()
{
_current += _step;
return (_current < _limit);
}
public void Reset()
{
throw new NotImplementedException();
}
int IEnumerator<int>.Current
{
get { return _current; }
}
}
}
The results I got on a 2 core VM (VMWare) running Windows Server 2003, through 10,000,000 instead of 100,000. Remove the .AsParallel() lines to force these LINQ's to run single threaded.
Primes (1, step 4) collection size: 332625
Time: 00:00:05.4500960
Primes (3, step 4) collection size: 332398
Time: 00:00:05.1003926
Press any key to continue . . .
|
|
|
|
|
Hi Tiju,
I can't speak to you specific question but I'm about to embark on a parallel project myself. I'm studying the Intel Concurrent Collections for C++. MSVC 2010 has some new stuff to support this also but my understanding is much of it is based on Intel technology.
Good Luck!
Rob.
|
|
|
|
|
Why are you using doubles to calculate prime numbers when primes are all integers?
If it's just so you can use the square root - there is no rule saying you can't cast that back to an int, and even better, it's possible to calculate the integer square root.
And you don't even need the square root (which is slow to calculate) anyway, you could compare the square of i with x (which is obviously equivalent to comparing i with the square root of x ) - better yet, you don't even need to do the multiplication (you can use repeated adding of odd numbers)
Also, after testing whether x is even (if ((x&1)==0) ) you never need to put an even number into the trial division ever again since if i divides x and i is even, 2 also divides x .
And you're already applying parallelisation? It's a good thing this is just a test..
|
|
|
|
|
There is more to it.
All primes except 2 and 3 are of the form multiple-of-6 plus or minus 1, so the two threads should be devoted to those two series, i.e. the increment needs to be 6, not 4 so multiples of 3 aren't even considered (I agree, they would fall out pretty quickly).
And then all that fancy enumerator stuff is costing cycles; its overhead well exceeds its functionality. A cleanly coded single thread can completely outpace what has been shown. I have a primes workbench where I experiment a lot, it currently locates all primes up to 10 million in 0.25 seconds; and up to 1 billion in 6.8 seconds.
Eventually I will write a little article on it!
|
|
|
|
|
Or you could use one of those fancy sieves.. but then you're changing the whole algorithm of course (cheat)
But yea write that article, you've been talking about for over half a year (IIRC)
|
|
|
|
|
I'm still researching, and making progress...
|
|
|
|
|
Fair enough, but it's the only article I'm really looking forward to reading
|
|
|
|
|
That is too bad, as I am planning some articles on other topics too...
|
|
|
|
|
|
I have a whole list of topics, all in various stages.
I plan to publish around ten articles on CP this year.
The one I like most myself is about the dynamics of ThreadPool. It will have some nice graphs!
And the biggest undertaking may be a series about P/Invoke.
|
|
|
|
|
That sounds pretty interesting as well.. ok.. maybe I'm looking forward to more than 1 article
|
|
|
|
|
Yeah, I just threw it together based on his code. It was by no means meant to be a tutorial on the most efficient way to do this. It just demonstrated a concept is all. Also, the performance numbers on that machine, well, suck, because it is a virtual running on a host that sucks at running VM's. I'm not allowed to put VS2010 on a real machine around here yet.
|
|
|
|
|
Fair enough. The concept and your code is interesting, it is just the example is a bit unfortunate.
I am looking forward to the official release of .NET 4.0 so I can investigate how it works, and how it compares to the manual approach.
|
|
|
|
|
I've been playing with the RC on that VM for about a week now. It's really cool to finally get to toy around with Tasks and Parallel. But, truthfully, the custom enumerator I threw together came from toying around with Enumberable.Range. It only uses integer and the results it returns can be no longer than integers. So, I tried throwing a custom Range method that supported Long's together and that's what I adapted to quickly come up with that example. I took his point to be that he was trying to figure out how the PTL worked. So...
I'm very interested in seeing an article on the Prime Sieve you've got. It would be interesting to see if it could be adapted to use or work with PTL.
|
|
|
|
|
It probably will be a few more months before my article on primes gets finished.
I can tell you right now the parallel stuff will not be useful for it, multi-threading hardly helps at all here. The first step in my analysis is proving calculations are so fast (assuming a sieve) the job is bandwidth limited, not compute bound. So I optimized it with banding (locality of reference, cache efficiency), all on a single thread. Only then I set out to figure a way to get extra cores involved, and there hardly is any. IIRC a second thread yields no more than a few percents of extra speed, taking care of some initialization, as running the sieve itself is an inherently sequential operation.
|
|
|
|
|
Hi,
i need to get only title, author and lenght of mp3 file, how i can do this??
please help me!
|
|
|
|
|
You need to read id3 tags from the file.
|
|
|
|
|
|
A little researc using this information[^] should speed you on your way. txtspeak is the realm of 9 year old children, not developers. Christian Graus
|
|
|
|
|
|
Bangalore, March 11, 2010: Darwin’s reference to biological symbiosis involves two or more species that commit to a relationship and through their combined efforts and mutual collaborative interactions, both stand to gain success. In a similar vein, there can be no doubt that one of the hottest spaces in enterprise software today is collaboration. It’s no surprise collaboration is getting a lot of interest. The old processes for capturing, authoring and sharing information are stale and inefficient. As such, there is lot of room for achieving productivity improvements through improved user experience. This has been true for all software, but especially so in the enterprise software space where collaboration is essential for daily operation, where every ounce of productivity translates into big dollars.
When we refer to collaboration on a business activity, there are generally two buckets for these processes. The first are activities that would fit within the process of innovation. This might include creating new products, models or business processes. In other words, we are talking about the process of creation and innovation by teams. The second bucket includes activities around executing the business processes. Execution includes the hundreds of processes that run each day to operate the business. These are tactical in nature, and traditionally have key performance indicators (KPIs) measuring the success of a given process.
In this day and age, fast is synonymous with instantaneous. Increasingly, there is less time to dial up a fax machine or wait for a file to slowly attach to e-mail (then wait yet again during sending). This is especially true for entrepreneurs, whose small businesses are built on every advantageous edge that makes them competitive with larger corporations as well as one another. For small business, everything must be real-time – not just information anymore, but also changes. Small business needs collaborative tools that can keep up.
Enter web-based collaborative tools, center stage. These business implementations are the only means fast enough to match the high-speed demands of the Internet-age industry. They achieve instantaneous cooperation despite distance, cost, and other past inhibitive considerations. In support of a powerful storage hub, they allow you to securely store your information, make real-time changes to it, share it openly or discretely, and, most importantly, collaborate on it with family, friends, groups, departments, business teams, clients, vendors, or anyone.
Fast isn't the only measure, though. These tools must be easy, with user-friendly interfaces and intuitive commands. Complicated FTP programs are a thing of the past, and e-mail is too limited (and sometimes too flaky) to fully meet a company's diverse needs. Tools must be cheap. The idea is productivity, and expensive software or services prohibit that goal. Web-based tools fill the demands traditionally met by established application suites, but without their hefty price tags attached. Most of all, though, tools must be reliable. Web-based collaboration does not depend on one machine (PC or Mac) or one server, but reliably and securely functions outside of these constraints in the cloud, so to speak. The information your enterprise team works on in the office all day should still be accessible at your home computer (so you can continue work with them at night). And with the extensive controls built in to web-based tools, even the most sensitive materials remain completely confidential despite their availability within user-defined limitations. Simply put, all-in-all web-based tools make collaboration simple.
In the pre-computer age (before 1970), collaboration was a manual process. This was done mainly in face-to-face meetings and phone calls. The technologies used included calculators, blackboards, telephones, postal mail and pencil on paper. While computers might be part of the process, it was mainframe host oriented that offered no interaction by the participants. This was labor intensive with long cycle times for even simple tasks. In this era, teams needed to be in the same location, or endure significant travel.
When the Internet entered the picture (1980s), it provided a basis for the team to communicate, mainly via email. This allowed teams to collaborate while geographically separated, which further improved the efficiency of the process. Since collaboration allowed teams to extend beyond a single location, they could now include more talented people in other geographies. Other technology advances such as overnight mail, video conferencing, faxing and conference calling further improved the process.
Another emerging area of how web collaboration is impacting execution is through business intelligence networks. These are like Facebook or Twitter on steroids. An example of this would be collecting sales and market intelligence through an internal network – very similar to a social network. The key difference is that in these applications the participants are internal communities, and the network has intelligence. Information is assembled and analytics push back conclusions based on all the inputs. This allows the community to respond quickly to new conditions. In other words, the decision latency is reduced closer to the dream of “real time business.”
Clearly, web collaboration has enabled business execution to be better, cheaper and faster than in the past.
Saltmarch Media's annual Great Indian Developer Awards honors software products across 12 categories,based on their productivity, innovation excellence, universal usefulness, simplicity, functionality and most importantly on the ground feedback from India’s software developer ecosystem. In the Collaboration Tools Category, the final shortlist consists of Citrix’s popular GoToMeeting, Microsoft’s Office, WebEx WebOffice, Microsoft’s Office SharePoint server and Microsoft’s Exchange Server. Oracle’s Collaboration Suite was the first collaboration tool to win this award in 2008 followed by Adobe Acrobat Connect Professional in the 2009 chapter of the same awards. Voting is open from http://www.developersummit.com/gida3_llist.
If there is a particular development environment that you personally endorse to your colleagues or you evangelize about them at the first opportunity you get, here is your chance to vote for it (voting closes April 10 2010) and see it win this prestigious award. Visit the 2010 Great Indian Developer Awards website and cast your vote. It counts!
A Saltmarch Media Press Release
E: info@saltmarch.com
Ph: +91 80 4005 1000
|
|
|
|
|