Multithreaded socket programing

Question

0.00/5 (No votes)

See more:

There are 60 or more linux client machines, each having a dedicated directory containing 100 or more data files. Each file is of size 1GB. All these files are in dedicated paths and must be transferred to server everyday 12AM.

I need a solution which can transfer these data file to server automatically every day at 12AM.

I need a idea how to resolve the problem. Please guide me how can I do this?

What I have tried:

I wrote a multithreaded client ser ver application where the serve accepts Multiple clients and transfer a simple file from client to server.

Posted 14-Jul-22 9:05am

Member 10244582

Updated 15-Jul-22 10:48am

Add a Solution

Comments

Greg Utas 14-Jul-22 16:40pm

So how did your multithreaded server work out? That's how I would also do it, so I'm not sure what you're asking.

Dave Kreskowiak 14-Jul-22 18:04pm

Why would a CRON job not handle this? It would launch a series of copy commands to copy the files to a network share on your server.

Not a single line of code required.

k5054 14-Jul-22 18:29pm

Lets take a quick look at your requirements:
60 + clients
100 + files @ 1 GB each
I make that over 6 TB of data to transfer. With a 1 GB NIC, That's going to take about 90 minutes to transfer, assuming there's nothing else using bandwidth at the same time.
First question: Can the data be compressed for transfer?
Second question: Would some sort of network filesystem (e.g NFS or Samba) be an acceptable solution? That would mean that rather than uploading the files to the server, the files would always be available to the server. So, rather than updating the server once per day, updates would be live, as and when they occur on the client.

[no name] 14-Jul-22 19:24pm

Hard to picture any client generating 100 GB of data per day. That's like 4 days of HD video ... per day.

steveb 16-Jul-22 12:22pm

Stock bid ask recorded data easily goes to 1 TB per day file just for one security

1 solution

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

steveb · Answer 1 · 2022-07-15T10:48:00

You do not need multithreaded socket programming for that. All you need to do is to map the network drive of the remote computer and copy files over with a batch command. And if you want to automate this use operating system's scheduler that will execute the batch file at the specified time