Multithreaded socket programing

Question

0.00/5 (No votes)

See more:

There are 60 or more linux client machines, each having a dedicated directory containing 100 or more data files. Each file is of size 1GB. All these files are in dedicated paths and must be transferred to server everyday 12AM.

I need a solution which can transfer these data file to server automatically every day at 12AM.

I need a idea how to resolve the problem. Please guide me how can I do this?

What I have tried:

I wrote a multithreaded client ser ver application where the serve accepts Multiple clients and transfer a simple file from client to server.

Posted 14-Jul-22 9:05am

Member 10244582

Updated 15-Jul-22 10:48am

Add a Solution

Comments

Greg Utas 14-Jul-22 16:40pm

So how did your multithreaded server work out? That's how I would also do it, so I'm not sure what you're asking.

Dave Kreskowiak 14-Jul-22 18:04pm

Why would a CRON job not handle this? It would launch a series of copy commands to copy the files to a network share on your server.

Not a single line of code required.

k5054 14-Jul-22 18:29pm

Lets take a quick look at your requirements:
60 + clients
100 + files @ 1 GB each
I make that over 6 TB of data to transfer. With a 1 GB NIC, That's going to take about 90 minutes to transfer, assuming there's nothing else using bandwidth at the same time.
First question: Can the data be compressed for transfer?
Second question: Would some sort of network filesystem (e.g NFS or Samba) be an acceptable solution? That would mean that rather than uploading the files to the server, the files would always be available to the server. So, rather than updating the server once per day, updates would be live, as and when they occur on the client.

[no name] 14-Jul-22 19:24pm

Hard to picture any client generating 100 GB of data per day. That's like 4 days of HD video ... per day.

steveb 16-Jul-22 12:22pm

Stock bid ask recorded data easily goes to 1 TB per day file just for one security

1 solution

Add a Solution

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

steveb · Answer 1 · 2022-07-15T10:48:00

You do not need multithreaded socket programming for that. All you need to do is to map the network drive of the remote computer and copy files over with a batch command. And if you want to automate this use operating system's scheduler that will execute the batch file at the specified time

Multithreaded socket programing

1 solution

Solution 1

Add your solution here

Preview 0

Multithreaded socket programing

1 solution

Solution 1

Add your solution here

Preview 0

Existing Members

...or Join us