C++ Tweetnacl hash a file without read whole file to memory

Question

I'm using tweetnacl to generate sha512 hashes of strings and file. For strings it works quite well but i have no idea how to do it with files.

The signature of the function ist

extern "C" int crypto_hash(u8 *out, const u8 *m, u64 n);

where u8 is of type unsigned char and u64 is of unsigend long long. For string a can use it like that

string s("Hello");
unsigned char h[64];

crypto_hash(h, (unsigned char *)s.c_str(), s.size());

This works great for a string and small files but if i want to create a hash for a big file, it is not viable and uses to much memory. I searching for a solution to read the file byte by byte and pass it as unsigend char pointer to that function. Has anyone a idea how to achieve that?

P.S Sorry for the poor English. p.s.s I use tweetnacl because of the small size and i need only the hashing function.

Is this what you're looking for?http://stackoverflow.com/questions/1220046/how-to-get-the-md5-hash-of-a-file-in-c — UKMonkey, Oct 12 '16 at 11:29
The tweetnacl source code probably contains what you are looking for. — Jabberwocky, Oct 12 '16 at 11:44
@MichaelWalz Unfortunately the documentation is really rare and although i dug into the source file, i couldn't find anything. But i admit i'm a c++ beginner and so not everything is clear to me in the source file. — snmed, Oct 12 '16 at 12:25

score 3 · Answer 1 · answered Oct 12 '16 at 11:49

Probably the easiest way is to use a memory-mapped file. This lets you open a file and map it into virtual memory, then you can treat the file on disk as if it is in memory, and the OS will load pages as required.

So in your case, open your file and use mmap() to map it into memory. Then you can pass the pointer into your crypto_hash() function and let the OS do the work.

Note that there are caveats to do with how large the file is wrt virtual memory.

For various platforms:

It looks promising to me, i will check it out this evening and accept the answer if it works for me. — snmed, Oct 12 '16 at 12:33

Adrian Colomitchi · Accepted Answer · 2016-10-12T12:32:32.683

2

I'd suggest you to use a different implementation, one which you can incrementally feed in chunks.

This one for example. As the licence is bsd and the code is C with no dependencies, you can copy/paste only the 3 functions that you need without bringing a whole lib (no matter how small) into your project.

The life-cycle goes like:

sha256_init(&ctx)
repeatedly read blocks from file and feed them into sha256_update(&ctx, buff, buffLen)
when EOF, get your digest using sha256_final(&ctx, digestHere)

edited Oct 12 '16 at 12:32

answered Oct 12 '16 at 12:16

Adrian Colomitchi

3,974
1
14
23

Thx this is probably the better solution, especially it's also really small and i doesn't need to map any file to memory. Strange i didn't find it while searching for a small library... :-( – snmed Oct 12 '16 at 12:48
@ConfoederatioHelvetica "Strange i didn't find it while searching for a small library..." I used "sha512 stream c++" with google. The first result was [this](http://www.zedwood.com/article/cpp-sha512-function) C++ incarnation, which has in the license.txt the link that I posted. – Adrian Colomitchi Oct 12 '16 at 12:53
I have to admit i searched only with "c++ sha512 library", now it seems obvious to me, to search for stream and sha512. sorry for my omissons. – snmed Oct 12 '16 at 13:33

C++ Tweetnacl hash a file without read whole file to memory

2 Answers2