Concurrent download of a file with threads

Question

Hi I've some problems with a concurrent download&sendind of a file. this is my code:

in main()

while(1){
    while(i < wrapper->request_get->maxconnect && nconn < 5){
         wrapper->request_get->temp = i;
         i++;nconn++;
         if((pth=pthread_create(&id[i],NULL,thread_func,wrapper))!=0){
         ...
         ...
      }

so I need to download a generic file with max 5 connection at once. variables "i" and "nconn" are global and start from value 0.

in thread_func():

    void *thread_func(void* args){
    pthread_mutex_lock(&mutex);
    j++;//it's global set to -1
    struct RequestWrapper *wr = args;
    ...
    ...//I set the range of data to request.
    ...//for example,thread #1 needs bytes from 1 to 100;#2 from 101 to 200 etc...
    ...//I do that with the value of "j".

    handlerRequest(wr)//this function asks(using a connect) to server a file's single part and save it
                      //into a buffer of struct wr.

    if(array[j]!=1){//if is not it my turn to send, wait
           pthread_cond_wait(&cond, &mutex);}

    SendData(wr)//send data conteined into a buffer of wr.
    array[j+1]=1;//I enable the next thread to send
    pthread_mutex_unlock(&mutex);
    pthread_cond_broadcast(&cond);//unlock other threads
nconn--;
pthread_exit(NULL);                 
}

unfortunately, in this way I've a sequential download and sending of data.

thread#1 download---send---thread#2 download---send---etc etc

how can I download the five parts of file at the same time and then send them in a orderly manner as soon as ready? I'm newbie about threads, so I think that some unknown, for me, functions can help me but which?ps: I cannot use signal...

It's not clear exactly what you're trying to do. Are you trying to download over HTTP using five separate HTTP connections? — David Schwartz, Jan 27 '12 at 01:28
sorry, I'm connected to a server with a connect, so using tcp. and I need five different connection,so 5 different connect. — rschirin, Jan 27 '12 at 01:43
So, how do you plan to tell the server to send different parts of the file on each of the five connections? Using HTTP? Or some other way? — David Schwartz, Jan 27 '12 at 01:59
server does accept in a while(1), checks the range of my request(FROM nbyte-TO nbyte)and send data with send syscall. into each of 5 threads, my program does a recv syscall.so everything uses tcp protocol — rschirin, Jan 27 '12 at 02:28
And the server can send to more than one connection at a time? — David Schwartz, Jan 27 '12 at 02:51

score 0 · Answer 1 · edited Jun 19 '18 at 15:42

Ok I'm assuming the http part is working just fine and the server can accept multiple requests and from-to parts is supported. Now just entering your thread I see you are holding on a single mutex object :o . This itself makes the whole operation sequential (one handlerRequest at a time) because only one thread is executing at one time which is what you don't want. The whole block of code does not need thread safety, what do you say?

You need to create 5 threads with 5 thread objects to wait on, I don't know what is used for this purpose in Linux, but in Windows you would use WaitForMultipleObjects, maybe check this:

WaitForSingleObject and WaitForMultipleObjects equivalent in linux

Usually when a file is to be downloaded in parts, you request the file size from the server, allocate a file on disk of returned length, open multiple file pointers (in your case 5) starting from different positions, each thread will be using its respective file pointer to write to that file. Once the 5 mutex (or whatever correct thread objects) have been released it would indicate that 5 threads have finished working and then the SendData can do its thing.

I hope I'm not totally wrong here hehe :)

I'm under linux. as I said, in that way it works in a sequential way. nut that is not a great way. — rschirin, Jan 27 '12 at 18:22
that link can give you a hint on what apis to explore cuz you can ignore the windows part. — user734028, Jan 27 '12 at 18:24

Concurrent download of a file with threads

1 Answers1