0

Solution: Allocated memory is freed up after a program exits. Have to read+write from disk back into a linked list, and then rewrite to update the database! Thank you everyone =)

Hello, I've basically been working on this database program for the past few nights but I just continuously reach dead ends. The assignment is due today so if you could help me out, it would be very much appreciated. =T

The database is implemented with a Linked List and consists of a few files: sdbm.c, sdbm.h, new.c, get.c, insert.c, put.c, and remove.c. sdbm.c holds the methods for the database based on the sdbm.h interface, and the other files contain main methods that use the methods from sdbm.

The first problem comes with the insert program, which seems to work fine when I try to add in a key and value pair ... that is, until we try to call the insert program again. The memory allocated seems to have disappeared! I've been researching, trying to figure out why even though I have malloced, why does it disappear after the insert program exits. Here is some code:

  • The node structure + global variable:
struct dbase_Node {
  char *keyValue;
  char *element;
  struct dbase_Node *next;
};

typedef struct dbase_Node Node;

Node *head;

========

  • The insert method
static bool sdbm_insert_back(Node **headRef, const char *key, const char *value)
{
  Node *new = (Node *)malloc(sizeof(Node));
  if (new == NULL)
    return false;
  else {
    new->keyValue = malloc(strlen(key));
    new->element = malloc(strlen(value));
    strcpy(new->keyValue, key);
    strcpy(new->element, value);

    new->next = *headRef;
    *headRef = new;
    return true;
  }
}
  • The sync method
bool sdbm_sync()
{
  if (!isOpen()) { return false; }

  if (fopen(databaseName, "w" ) == NULL) {
    error = SDBM_FOPEN_FAILED;
    return false;
  }

  Node *current = head;

  while (current != NULL) {
    fprintf(database, "Key: %s\n", current->keyValue);
    fprintf(database, "Value: %s\n", current->element);
    current = current->next;
  }
  return true;
}

I run the following:

./new [database] <-- works fine ./insert [database] [key] [value] <--seems to work fine

And then after I try to insert more, the already added nodes have disappeared ...

  • 2
    There is no such thing as anything 'urgent' on a Q&A site. –  Mar 04 '11 at 08:55
  • 1
    Not again! Your `malloc()` doesn't allocate space for the null character. I have no idea if that causes the problem you observe, but this causes undefined behavior, since your `strcpy()` writes beyond the buffer end and this could lead to anything. – sharptooth Mar 04 '11 at 08:57
  • I apologize; I understand it's just that I've been banging my head on this program for the past few nights and I've run out of time. It would be great if anyone can point me in the right direction because I've exhausted all that I can think of ... – user644371 Mar 04 '11 at 08:57
  • @user644371: Instead of banging your head around that program, you should bang your head around your understanding of the process life cycle. http://en.wikipedia.org/wiki/Process_states – datenwolf Mar 04 '11 at 09:01
  • 1
    Maybe I'm missing something but... allocated memory is released when the process terminates, i.e. it doesn't persist across invocations. You have to write the data to disk if you want it to persist! – CAFxX Mar 04 '11 at 09:05
  • Hi, I wondered if you be interested in aodbm ( sf.net/projects/aodbm/ ), since you're writing what looks like a dbm style DBMS. – dan_waterworth Mar 04 '11 at 09:09
  • @datenwolf: I will do that ... teacher lectures are not too informative x.x – user644371 Mar 04 '11 at 09:11
  • @CAFxX: Does it? Is that the only option? – user644371 Mar 04 '11 at 09:13
  • @dan_waterworth: I'm not sure but basically our teacher threw us a database assignment. Thanks for the link though – user644371 Mar 04 '11 at 09:14
  • @user644371, would you also post the function that populates the linked list from disk. – dan_waterworth Mar 04 '11 at 09:25
  • @dan_waterworth: Ah you see ... that was the problem. datenwolf and CAFxX and a few others pointed out that the allocated memory frees up after the processes and I did not realize that. Going to implement that right now ... >.> – user644371 Mar 04 '11 at 09:29
  • On StackOverflow we click the accept button on the answer which helped most. This both marks the question as solved and give's the most helpful person credit. – datenwolf Mar 04 '11 at 09:54
  • @user644371 yes, it's the only option. I'd suggest also using mmap() but it's definitely better if you at least get the basics first. – CAFxX Mar 04 '11 at 10:14

5 Answers5

5
new->keyValue = malloc(strlen(key));
new->element = malloc(strlen(value));
strcpy(new->keyValue, key);
strcpy(new->element, value);

This causes an off-by-one buffer overflow. You need to allocate space for the terminating \0, so use strlen(key) + 1 and strlen(value) + 1.

Or even better, use strdup():

new->keyValue = strdup(key);
new->element = strdup(value);

The problem of your data being lost after the program exists is very simple but not simple to fix: You are not storing it anywhere but in memory. And memory is automatically free'd when the program exits. So you'll have to store it e.g. in a file.

ThiefMaster
  • 310,957
  • 84
  • 592
  • 636
  • +1: `strdup` isn't ISO standard so it may not be available everywhere but it's trivially easy to write: http://stackoverflow.com/questions/252782/strdup-what-does-it-do-in-c/252802#252802 – paxdiablo Mar 04 '11 at 09:02
  • Thank you for pointing this out. Yet the nodes still reset after running the program. =\ – user644371 Mar 04 '11 at 09:05
  • While your comments point out some problem, it's not the cause for the OPs problem. He's performing the two different operations in two different processed. – datenwolf Mar 04 '11 at 09:09
  • Thank you. =) Now I understand; I had assumed that malloc() was a permanent memory holder, but now that I think about it, that doesn't even make sense! Thank you, and hopefully, I'll be able to finish this. – user644371 Mar 04 '11 at 09:35
1

This line in your question:

./new [database] <-- works fine ./insert [database] [key] [value] <--seems to work fine

implies you are running the program once to create a database, then again to create the key/value pairs. Are you anywhere persisting your data to disk?

JeremyP
  • 84,577
  • 15
  • 123
  • 161
  • Yes! After the nodes have been added, the sync() (synchronize) method copies it into a file. It seems that after they have been written, the nodes disappear/reset. – user644371 Mar 04 '11 at 09:02
  • @user644371: Allocated memory can't be passed via files. Storing the value of a pointer in a file won't carry the memory that was allocated. – datenwolf Mar 04 '11 at 09:10
  • @datenwolf: He's actually attempting to write the strings to a file. It wont work, of course, because of the off-by-one malloc error spotted by ThiefMaster. – JeremyP Mar 04 '11 at 09:15
  • @datenwolf: Is there anything I can do to save this whole linked-list implementation? It seems that the only thing I can do is write the memory to a disk .. but that would defeat the entire purpose of the database because we also need to sync() the data in some data structure onto the disk. =/ – user644371 Mar 04 '11 at 09:17
  • @JeremyP: No, the strings are being written just fine. I see the Key/Value pair being written to the file but the data structure containing the data vanishes – user644371 Mar 04 '11 at 09:19
  • @user644371: The data structure is just a linked list of key value pairs. It's implied by the ordering of the pairs in the data file. When you read the data back on the next invocation, you need to reconstruct the linked list as you go along reading the pairs. – JeremyP Mar 04 '11 at 09:23
1

Do I read that right? For each of those operations you're starting a new process? If so there's your problem: Once a process terminates all the memory it allocated is freed.

What you need is persistent storage. In the usual operating systems process memory is not persistent (there are some OS in which allocated memory is persistent, but still bound to a certain process, so even in such a OS this kind of program won't work).

datenwolf
  • 159,371
  • 13
  • 185
  • 298
  • That makes sense then! Unfortunately, I have to start new processes every time because that is how my teacher has laid out the assignment. Is there a way I can implement persistent storage or do I need to start over with an array or something similar > – user644371 Mar 04 '11 at 09:07
  • You need to flatten out the data, i.e. write the data in the list to a file. Then in the insert program you read the file, insert/add the new key->value pairs, write to a new file, close it and rename the new file to the old file's name. The interesting part is, how one implements such a scheme. Daniel Bernstein implemented a very efficient key->value DB system called 'cdb' a few years ago, maybe you've a look at it. But don't cheat! – datenwolf Mar 04 '11 at 09:16
  • I see! Perhaps the linkedlist implementation wasn't such a bad choice after all then because it is unbounded. Thanks, maybe after a go at this again, I'll take a look at it. And I won't! =) – user644371 Mar 04 '11 at 09:22
  • I think I should add, that writing structs to a file by performing a simple write(fd, &data, sizeof(data)); is just plain wrong. Whenever a student hands me in code with this kind of file processing I give negative marks, so even if the rest of the program is correct, improper file interaction may reduce the points. For a simple reason: Doing it in that lazy way is the number one reason, for programs being attackable. If e.g. a webbrowser would read images from the internet that way => boom, instant machine infection. – datenwolf Mar 04 '11 at 09:22
  • I appreciate the information. I actually would love to make this database more efficient and neat! However, my teacher basically 'threw us into the fray' and does not even expect us to finish. We've barely covered pointers, and much less memory allocation, but I'm trying my best to figure things out on my own! – user644371 Mar 04 '11 at 09:26
  • Now a suggestion: A very simple method for flatting out that structure would be writing it as a series of netstrings: http://cr.yp.to/proto/netstrings.txt Two consecutive netstrings form a key->value pair, so you can populate that linked list until you hit the EOF. – datenwolf Mar 04 '11 at 09:27
  • Oh before I forget: In the case of a flat netstring file, adding new list entries can be done by simply appending to the file's end, so no need to read back for insertion, so you even don't need an interim file, just open write-append and write new entries. – datenwolf Mar 04 '11 at 09:33
  • I think that would solve my problem of how to determine where the key/value ends! Thank you! I just might be able to do this. – user644371 Mar 04 '11 at 09:33
0

I feel the issue is in the else part, u need to check if it's the first node in the List if so allocate memory, store into head. Next time u call it if the list is not null then link head->next to ur newly created one.

NirmalGeo
  • 773
  • 4
  • 12
-1

Suggestion: in sdbm_sync close the file after writing the data.

Also in sdbm_sync where is database defined? Is the fopen line missing somthing? ???

bool sdbm_sync()
{
  if (!isOpen()) { return false; }
  /* ******** MISSING DATABASE ? ? ? ? ?
  FILE *database;
  if ((database = fopen(databaseName, "w" )) == NULL) {
  ******** */
  if (fopen(databaseName, "w" ) == NULL) {
    error = SDBM_FOPEN_FAILED;
    return false;
  }

  Node *current = head;
  while (current != NULL) {
    fprintf(database, "Key: %s\n", current->keyValue);
    fprintf(database, "Value: %s\n", current->element);
    current = current->next;
  }
  if (!fclose(database)) /* unexpected error closing database */;
  return true;
}
pmg
  • 106,608
  • 13
  • 126
  • 198