Append to file using flysystem in ILIAS

Question

I am currently writing a logger writing to rather large log files in ILIAS for some plugin. I am using the neat new filesystem. Since the new log messages need to append to a file, I can not simply use put or update, since they seem to always truncate the logfile. Flysystem seems not to support an easy append command, so one way I found that is working is the following:

$old_data = "";
if ($DIC->filesystem()->storage()->has($this->path)) {
    $old_data = $DIC->filesystem()->storage()->read($this->path);
}
$DIC->filesystem()->storage()->put($this->path, $old_data.$string);

However, this seems like very expensive regarding IO if massive amounts of data are appended to the log. I guess, this is best done using streams. I the docs (https://github.com/ILIAS-eLearning/ILIAS/tree/release_5-3/src/Filesystem) I found the following:

$webDataRoot = $DIC->filesystem()->web();
$fileStream = $webDataRoot->readStream('relative/path/to/file');

//seek at the end of the stream
$fileStream->seek($fileStream->getSize() - 1);
//append something
$fileStream->write("something");

However, with this I am getting the Can not write to a non-writable stream exception. It seems, that I would need to open the stream like so:

$resource = fopen('myPath', 'rw');

However this is specifically discouraged in the docs. What is the best way to tackle this by making use of the filesystem in ILIAS?

Answer 1

Problem

The Filesystem abstraction of ILIAS leverages flysystem to manipulate the filesystem. However as you already pointed out flysystem provides no easy way to append data to files. The main reason seems to be the fact that many of the filesystem adapters which are used to write the file to the storage backend wont support the append operation because of the underlying technology, for example the S3 V3 storage adapter which re-uploads the whole file.

Unfortunately the documentation located at (ILIAS-eLearning) is wrong, because the streams returned by flysystem are read only due to the fact that some filesystem adapters only "emulate" the behaviour of a filestream. The stream points to a local copy or in-memory stream instead of the "real" file located at the storage backend, see Flysystem AWS S3 Adapter.

Conclusion

In conclusion there is no correct way to append data to an existing file with the current ILIAS filesystem implementation, because flyway provides no such operation. In addition, some storage backends will still not support such operations even if flyway would provide it. A good example is the AWS S3 storage backend see the answer of the question 41783903.

Possible workaround for your use case

If you are able to use multiple log files, use some sort of log file rotation to reduce your IO, which is not a solution but helps to keep the load at least at a consistent level. For example, write to a new log file if the size would exceed 20MB and keep the last 20 log files.