Writing structs to a text file

Question

So i've recently gotten the hang of structs and am finally comfortable with Arrays, now I must tackle the files my professor wants me to be able to store 1million structs (payroll records) and im pretty sure an array that big would crash my program. So I am confused on how I can back up the structs into a txt file and upload them back into my program when started. Would the text come out as a mess? Here is the main part of my program I'll need to put into a file. It basically makes the payroll record using the struct. So i was just wondering any one could give me tips on sending this into a text file

void payRollAdd(PAYROLL employee[], long int *pCounter){    
    int userinput;
    do{
        printf("Enter Employee Name:\n");
        fgets(employee[*pCounter].name, 40, stdin);
        printf("Please Enter the paydate(mmddyyyy):\n");
        scanf_s("%02d%02d%d", &employee[*pCounter].payDate.month, &employee[*pCounter].payDate.day, &employee[*pCounter].payDate.year);
        printf("Enter Employee Age:\n");
        scanf_s("%d", &employee[*pCounter].age);
        printf("Enter Employee Hours Worked\n");
        scanf_s("%f", &employee[*pCounter].hrsWorked);
        printf("Enter Employee Hourly Wage:\n");
        scanf_s("%f", &employee[*pCounter].hrlyWage);
        printf("  Employee Data \n");
        printf("Name: %s\n", employee[*pCounter].name);
        printf("Age:  %d\n", employee[*pCounter].age);
        printf("Pay Date: %02d/%02d/%d:\n", employee[*pCounter].payDate.month, employee[*pCounter].payDate.day, employee[*pCounter].payDate.year);
        printf("Hours Worked: %0.2f\n", employee[*pCounter].hrsWorked);
        printf("Hourly Wage: $%0.2f\n", employee[*pCounter].hrlyWage);


        if (employee[*pCounter].hrsWorked > 40.00){
            float temp = 40.00;
            employee[*pCounter].regPay = temp * employee[*pCounter].hrlyWage;
            printf("Regular pay: $%0.2f\n", employee[*pCounter].regPay);
        }
        else
        {
            employee[*pCounter].regPay = employee[*pCounter].hrsWorked * employee[*pCounter].hrlyWage;
            printf("Regular pay: $%0.2f\n", employee[*pCounter].regPay);
        }
        if (employee[*pCounter].hrsWorked > 40.00){
            float temp2 = 0.00;
            temp2 = employee[*pCounter].hrsWorked;
            temp2 -= 40.00;
            employee[*pCounter].otPay = temp2 * `(employee[*pCounter].hrlyWage + (employee[*pCounter].hrlyWage / 2.00));`
            printf("OT pay: $%0.2f\n", employee[*pCounter].otPay);
        }
        else
        {
            employee[*pCounter].otPay = 0;
            printf("OT pay: $0\n");
        }
        employee[*pCounter].totalPay = employee[*pCounter].regPay + employee[*pCounter].otPay;
        printf("Total Pay: $%0.2f\n", employee[*pCounter].totalPay);

Answer 1

You can use a combination of fseek, fwrite and fread to read and write and find the structures in the file:

fseek(fp,0L,SEEK_END);
fwrite(&employee,sizeof(employee),1,fp);

You can do something similar to read the record in:

fseek(fp,sizeof(employee) * recno,SEEK_SET);
fread(&employee,sizeof(employee),1,fp);

Check the return codes!

Answer 2

Most do { } statements can be written as while { } statements, if you use flag variables, which makes the code easier to read, as one can see the condition at the top of the loop.

You can use open() and write(), and read() to store the data. Alternatively you can use fopen(), fclose(), fprintf(), fgets() etc... But then you have to be meticulous in handling of EOL markers and null terminators, etc... The man pages highlight the details you need to pay close attention to regarding delimiters and record sizes.

With open(), read(), write(), close(), as long as there's a fixed record size (struct size in this case) you don't need to deal with record delimiters. As long as the file system has the space there should be no problem writing out a million records. Reading back all the data is fine up to the limits of available memory. If you have more data than you can practically manage in memory you need to read it in chunks and then over-write the memory with new data. Some sort of paging mechanism or something. There's no other choice. You either have the disk space and memory or you don't, and if you don't you have to get creative and find a strategy for you to quantize the reading or writing and translating it or interacting with the user.

If you were printing out a payroll you could just keep printing to a printer as you read data in a loop and wouldn't have to keep more than one record (struct's worth) of data at a time in memory. In real life huge files are usually indexed somehow and keyed so that limits the amount of it need to be loaded at once. It can be done in C of course, but as it gets more complex it's obviously easier to use a database.

As long as you don't overrun a fixed size array or memory you won't get garbage. If you need to grow your array dynamically, use pointers and use malloc() or calloc() to allocate one record's worth at a time and create pointers in your struct (often named 'prev' and 'next' by convention), (which are pointers to the struct's type), to point to the next entry and previous entry as you allocate them. It's called a linked list and there should be plenty of examples online for how to create one.

HTH.

PS: fseek() advice is good too, but I don't think that was your question (e.g. how to randomly access data)

Answer 3

I'm assuming you're wanting to write the PAYROLL record array into a file. Some tips:

1. Use a new line for each record. That way, if you want to edit the file manually you can do so. Also you will be able to read it by fgets()'ing each line.
2. Use a separator and tokens: e.g. you can have a line such as Employee name####10.00(pay rate)#### details. When you are reading, use strtok to split the string into its component parts and then use sscanf to extract detail from them.