1

I have created a sorting algorithms that uses multithreaded quicksorting. The input data is created randomly. 100million integers are randomized and inserted into an array and then they are sorted. the thing is, I want to test the rate at which the algorithm's complexity grows when more data is input.

The problem: 100million looks like it is the upper limit of how many numbers I can generate and/or insert them in an array. The questions: how to increase the stack/heap size to allow more numbers to be used. I plan on doing something like 100billion. P.S RAM is not a problem.

The part where the number of generated integers is defines, has this written on the row: "//number of generated integers".

Thanks.

The code:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
#include <sys/time.h>
#include <pthread.h>
struct Params
{
    int *start;
    size_t len;
       int depth;
};
pthread_mutex_t mtx = PTHREAD_MUTEX_INITIALIZER;
void *merge_sort_thread(void *pv);
void merge(int *start, int *mid, int *end)
{
    int *res = malloc((end - start)*sizeof(*res));
    int *lhs = start, *rhs = mid, *dst = res;
    while (lhs != mid && rhs != end)
        *dst++ = (*lhs < *rhs) ? *lhs++ : *rhs++;
    while (lhs != mid)
        *dst++ = *lhs++;
    // copy results
    memcpy(start, res, (rhs - start) * sizeof *res);
    free(res);
}
// initialization
   void merge_sort_mt(int *start, size_t len, int depth)
{
    if (len < 2)
        return;
    if (depth <= 0 || len < 4)
    {
          merge_sort_mt(start, len/2, 0);
        merge_sort_mt(start+len/2, len-len/2, 0);
    }
    else
    {
        struct Params params = { start, len/2, depth/2 };
        pthread_t thrd;
        pthread_mutex_lock(&mtx);
        printf("Starting subthread...\n");
        pthread_mutex_unlock(&mtx);
        pthread_create(&thrd, NULL, merge_sort_thread, &params);
        merge_sort_mt(start+len/2, len-len/2, depth/2);
        pthread_join(thrd, NULL);
        pthread_mutex_lock(&mtx);
        printf("Finished subthread.\n");
        pthread_mutex_unlock(&mtx);
    }
    merge(start, start+len/2, start+len);
}
void *merge_sort_thread(void *pv)
{
    struct Params *params = pv;
    merge_sort_mt(params->start, params->len, params->depth);
    return pv;
}
void merge_sort(int *start, size_t len)
{
    merge_sort_mt(start, len, 8);
}
int main()
{
clock_t start, stop;
    static const unsigned int N = 100000000; //number of generated integers
    int *data = malloc(N * sizeof(*data));
    unsigned int i;
    srand((unsigned)time(0));
    for (i=0; i<N; ++i)
    {
    data[i] = rand() % 500;
  //  printf("%4d ", data[i]);
   //     if ((i+1)%8 == 0)
          // printf("\n");
    }
  //  printf("\n");
   start = clock();
    merge_sort(data, N);
    for (i=0; i<N; ++i)
    {
       // printf("%4d ", data[i]);
     //   if ((i+1)%8 == 0)
        //    printf("\n");
    }
   stop = clock();
       printf("Elapsed: %f seconds\n", (double)(stop - start) / CLOCKS_PER_SEC);
   // printf("\n");
    free(data);
    getchar();
    getchar();
    getchar();
    getchar();
    getchar();
    return 0;
}
trincot
  • 317,000
  • 35
  • 244
  • 286
John Abbrams
  • 47
  • 10
  • What kind of array are you working with where you're able to handle 100 billion values? – Wai Ha Lee Feb 07 '19 at 21:14
  • @WaiHaLee it works with 100million, but I would like to make it work for 100billion – John Abbrams Feb 07 '19 at 21:15
  • 1
    @JohnAbbrams My guess is that the OS doesn't let you allocate all that memory at once and the `malloc()` call just fails on OS level. You can try allocating it in chunks. I don't have a terabyte of RAM to verify thou, good luck. EDIT: [This is what you are probably looking for](https://stackoverflow.com/a/3486163/6119495). – Somrlik Feb 08 '19 at 04:44
  • What's overflowing? It looks like you're creating a whole lot of threads., all of which are running concurrently, and all of which end up allocating memory. You should add a memory use statistic that you update and display when you allocate or deallocate memory. You'll need to make it global, and protect it with a mutex. I think you'll find that you're using a whole lot more memory than you think. Also, 100 billion integers will require 400 *gigabytes*. Are you sure memory isn't a problem? – Jim Mischel Feb 11 '19 at 12:19
  • 1
    Also, it's odd that you're using a multithreaded merge sort on an array of 100 million numbers in the range 0 to 500. A simple single-threaded bucket sort is going to be a whole lot faster and require a whole lot less memory. – Jim Mischel Feb 11 '19 at 12:21
  • Possible duplicate of [How big can a malloc be in C?](https://stackoverflow.com/questions/3463207/how-big-can-a-malloc-be-in-c) – ejderuby Aug 23 '19 at 18:50

0 Answers0