Delete lines in SPIFFS (ESP8266/Arduino)

Question

I am trying to create a function that will delete a line from a text file stored in SPIFFS on an ESP8266. I have tried to follow the "space overwrite" method found here https://forum.arduino.cc/index.php?topic=344736.msg2380000#msg2380000, but I just end up with a line of spaces.

My understanding is that it should work as below, with - representing the replacement spaces. The file in question only has \n at the line end.

Before
======
This is a line of text\n
This is another line of text\n
And here is another

Replace line 2 with spaces
==========================
This is a line of text\n
----------------------------\n
And here is another

Desired result
==============
This is a line of text\n
And here is another

What I actually end up with is the following.

Actual result
=============
This is a line of text\n
                            \n
And here is another

The file size remains the same also.

Here is the deleteLine function I am using.

bool deleteLine (char* path, uint32_t lineNum) {
  if (!SPIFFS.begin()) {
    Serial.println("SPIFFS failed!");
    SPIFFS.end();
    return false;
  }
  File file = SPIFFS.open(path, "r+");
  if (!file) {
    Serial.println("File open failed!");
    file.close();
    SPIFFS.end();
    return false;
  }
  uint32_t size = file.size();
  file.seek(0);
  uint32_t currentLine = 0;
  bool lineFound = false;

  for (uint32_t i = 0; i < size; ++i) {
    if (currentLine == lineNum) {
      lineFound = true;
      for (uint32_t j = i; j < size; ++j) {
        uint32_t currentLocation = j;
        char currentChar = (char)file.read();
        if (currentChar != '\n') {
          file.seek(j);
          file.write(' ');
        } else {
          break;
        }
      }
      break;
    }
    char currentChar = (char)file.read();
    if (currentChar == '\n') {
      ++currentLine;
    }
  }
  file.close();
  SPIFFS.end();
  if (lineFound) {
    return true;
  } else {
    return false;
  }
}

Am I doing something stupid here?

I know that what I could do is create a new file, copy the original file across omitting the line, however, I am working with 2x files around 1MB each, and would need an extra 1MB free space for the temporary file which is not ideal.

I'm also interested in whether there is any way to truncate a file. If there is, I could just run through the file, replacing characters to essentially remove the desired line, then add an end of file character, or truncate to get rid of the junk at the end of the file.

I don't think there is a solution. Even if you want edit a file, you have to remove the old one and write the new one, if they don't have the same size. If you can split your big file in small files with just 16kb, you could load the entire file to RAM as string and save it again once you removed all lines. — Adriano, Dec 18 '19 at 12:40
the only way to remove some part in the middle of the file is to read the file and write the new content in a new file. then delete the old file and rename the new file. this applies to most computer file systems too. — Juraj, Dec 18 '19 at 16:32

score 0 · Answer 1 · answered Mar 22 '20 at 11:35

What you want to do is directly write at a low-level to the file system which is done in the provided link (So on micro controllers with open source HW/SW you have more options than just file read and write like in a PC based OS).

  for(uint8_t i=0;i<32;i++) ch[i]=' ';

try replacing with

 for(uint8_t i=0;i<32;i++) ch[i]='X';

and you will see it works (at least it does for me). I work with a dynamic line length = char count from previous \n to the \n of the line to overwrite. As a safe guard I check for the file size. And once in a while I compact the file by renaming the old file to bak, deleting all "empty" lines by ignoring them while writing to a new file to the SPIFFS. Works flawless for a large configuration file 64kb max.

Delete lines in SPIFFS (ESP8266/Arduino)

1 Answers1