flock-ing a C++ ifstream on Linux (GCC 4.6)

Question

context

I'm slowly writing a specialized web server application in C++ (using the C onion http server library and the JSONCPP library for JSON serialization, if that matters)., for a Linux system with GCC 4.6 compiler (I don't care about portability to non Linux systems, or to GCC before 4.5 or to Clang before 3.0).

I decided to keep the user "database" (there will be very few users, probably one or two, so performance is not a concern, and O(n) access time is acceptable) in JSON format, probably as a small array of JSON objects like

 { "_user" : "basile" ;
   "_crypasswd" : "XYZABC123" ; 
   "_email" : "basile@starynkevitch.net" ;
   "firstname" : "Basile" ;
   "lastname" : "Starynkevitch" ;
   "privileges" : "all" ;
 }

with the convention (à la .htpasswd) that the _crypasswd field is the crypt(3) "encryption" of the user password, salted by the _user name;

The reason I want to describe users by Json objects is that my application might add (not replace) some JSON fields (like e.g. privileges above) in such Json objects describing users. I'm using JsonCpp as a Json parsing library for C++. This library wants an ifstream to be parsed.

So I am reading my password file with

extern char* iaca_passwd_path; // the path of the password file
std::ifstream jsinpass(iaca_passwd_path);
Json::Value jpassarr;
Json::Reader reader;
reader.parse(jsinpass,jpassarr,true);
jsinpass.close();
assert (jpassarr.isArray());
for (int ix=0; ix<nbu; ix++) {
  const Json::Value&jcuruser= jpassarr[ix];
  assert(jcuruser.isObject());
  if (jcuruser["_user"].compare(user) == 0) {
    std::string crypasswd = jcuruser["_crypasswd"].asString();
    if (crypasswd.compare(crypted_password(user,password)) == 0) {
         // good user
    }
  }
}

---

question

Obviously, I want to flock or lockf the password file, to ensure that only one process is reading or writing it. To call these functions, I need to get the file descriptor (in Unix parlance) of the ifstream jsinpass. But Google gives me mostly Kreckel's fileno (which I find complete, but a bit insane) to get the file descriptor of an std::ifstream and I am not sure that the constructor won't pre-read some of it. Hence my question:

how can I lock a C++ ifstream (Linux, GCC 4.6) ?

(Or do you find some other way to tackle that issue?)

Thanks

Answer 1

My solution to this problem is derived from this answer: https://stackoverflow.com/a/19749019/5899976

I've only tested it with GCC 4.8.5.

#include <cstring>  // for strerror()
#include <iostream> // for std::cerr
#include <fstream>
#include <ext/stdio_filebuf.h>

extern "C" {
#include <errno.h>
#include <sys/file.h>  // for flock()
}

    // Atomically increments a persistent counter, stored in /tmp/counter.txt
int increment_counter()
{
    std::fstream file( "/tmp/counter.txt" );
    if (!file) file.open( "/tmp/counter.txt", std::fstream::out );

    int fd = static_cast< __gnu_cxx::stdio_filebuf< char > * const >( file.rdbuf() )->fd();
    if (flock( fd, LOCK_EX ))
    {
        std::cerr << "Failed to lock file: " << strerror( errno ) << "\n";
    }

    int value = 0;
    file >> value;
    file.clear();   // clear eof bit.
    file.seekp( 0 );
    file << ++value;

    return value;

    // When 'file' goes out of scope, it's closed.  Moreover, since flock() is
    //  tied to the file descriptor, it gets released when the file is closed.
}

Answer 2

A deficiency with the filestream API is that you cannot (at least not easily) access the file descriptor of an fstream (see here and here, for example). This is because there is no requirement that fstream is implemented in terms of FILE* or file descriptors (though in practice it always is). This is also required for using pipes as C++ streams.

Therefore the 'canonical' answer (as implied in the comments to the question) is:

create a stream buffer (derived from std::basic_streambuf) that uses Posix and C stdio I/O functions (i.e open etc) and thus gives access to the file descriptor.

Create your own 'LockableFileStream' (derived from std::basic_iostream) using your stdio based stream buffer instead of std::streambuf.

You may now have a fstream like class from which you may gain access to the file descriptor and thus use fcntl (or lockf) as appropriate.

There are a few libraries which provide this out of the box.

I had thought this was addressed partly now that we've reached C++17 but I can't find the link so I must have dreamed it.

Answer 3

You might want to use a separate lockfile rather than trying to get the descriptor from the ifstream. It's much easier to implement, and you could probably wrap the ifstream in a class that automates this.

If you want to ensure atomic open/lock, You might want to construct a stream using the method suggested in this SO answer, following open and flock

Answer 4

Is the traditional unix-y solution of relying on the atomicity of rename() unacceptable?

I mean, unless your JSON serialization format supports in-place update (with a transaction log or whatever), then updating your password database entails rewriting the entire file, doesn't it? So you might as well write it to a temporary file, then rename it over the real name, thus ensuring that readers read a consistent entry? (Of course, in order for this to work each reader must open() the file each time it wants to access a DB entry, leaving the file open doesn't cut it)