Concurrent Vector and Algorithm Help.

See more:

C++/CLI

I am currently working on a personal project that uses the following threads:

1 Thread has to read files and split them into chunks and adds these chunks to a queue. This thread ends when there are no more files to be read.

N threads will then use the queue and pop off the chunk on top and proceed to do what ever task is needed with that chunk, this can range depending on the file. They only access the queue when they need data after they have finished their processing.

A total of N + 1 threads.

(This is just a framework for other personal projects)

The first thread is very simple as all it does is read and push data to the queue, but when I start adding more than one extra thread I run into a problem, and I have not done a lot with more than two threads working from the same data source.

I am not really sure what the best method would be to protect my queue from being corrupt from having more than one thread access it. I thought of wrapping the queue class and adding locks to the pops and pushes of the std::queue or is there another library that would provide a thread safe way of doing this. I have looked into some solutions online, but I have only done a little bit of multithreaded programming.

Would a different method of spreading the work be better? Or will having locks, semaphores or critical_section, on queue->pop() and queue->push() be sufficient for my purpose?