"Which explains what I am seeing, but I have a couple of questions:"
"Is there a way to detect if something was already mapped to certain address? without accessing /proc/maps?"
Yes, use mmap without MAP_FIXED.
"Is there a way to force mmap to fail in the case of finding overlapping pages?"
Apparently not, but simply use munmap after the mmap if mmap returns a mapping at other than the requested address.
When used without MAP_FIXED, mmap on both linux and Mac OS X (and I suspect elsewhere also) obeys the address parameter iff no existing mapping in the range [address, address + length) exists. So if mmap answers a mapping at a different address to the one you supply you can infer there already exists a mapping in that range and you need to use a different range. Since mmap will typically answer a mapping at a very high address when it ignores the address parameter, simply unmap the region using munmap, and try again at a different address.
Using mincore to check for use of an address range is not only a waste of time (one has to probe a page at a time), it may not work. Older linux kernels will only fail mincore appropriately for file mappings. They won't answer anything at all for MAP_ANON mappings. But as I've pointed out, all you need is mmap and munmap.
I've just been through this exercise in implementing a memory manager for a Smalltalk VM. I use sbrk(0) to find out the first address at which I can map the first segment, and then use mmap and an increment of 1Mb to search for room for subsequent segments:
static long pageSize = 0;
static unsigned long pageMask = 0;
#define roundDownToPage(v) ((v)&pageMask)
#define roundUpToPage(v) (((v)+pageSize-1)&pageMask)
void *
sqAllocateMemory(usqInt minHeapSize, usqInt desiredHeapSize)
{
char *hint, *address, *alloc;
unsigned long alignment, allocBytes;
if (pageSize) {
fprintf(stderr, "sqAllocateMemory: already called\n");
exit(1);
}
pageSize = getpagesize();
pageMask = ~(pageSize - 1);
hint = sbrk(0); /* the first unmapped address above existing data */
alignment = max(pageSize,1024*1024);
address = (char *)(((usqInt)hint + alignment - 1) & ~(alignment - 1));
alloc = sqAllocateMemorySegmentOfSizeAboveAllocatedSizeInto
(roundUpToPage(desiredHeapSize), address, &allocBytes);
if (!alloc) {
fprintf(stderr, "sqAllocateMemory: initial alloc failed!\n");
exit(errno);
}
return (usqInt)alloc;
}
/* Allocate a region of memory of at least size bytes, at or above minAddress.
* If the attempt fails, answer null. If the attempt succeeds, answer the
* start of the region and assign its size through allocatedSizePointer.
*/
void *
sqAllocateMemorySegmentOfSizeAboveAllocatedSizeInto(sqInt size, void *minAddress, sqInt *allocatedSizePointer)
{
char *address, *alloc;
long bytes, delta;
address = (char *)roundUpToPage((unsigned long)minAddress);
bytes = roundUpToPage(size);
delta = max(pageSize,1024*1024);
while ((unsigned long)(address + bytes) > (unsigned long)address) {
alloc = mmap(address, bytes, PROT_READ | PROT_WRITE,
MAP_ANON | MAP_PRIVATE, -1, 0);
if (alloc == MAP_FAILED) {
perror("sqAllocateMemorySegmentOfSizeAboveAllocatedSizeInto mmap");
return 0;
}
/* is the mapping both at or above address and not too far above address? */
if (alloc >= address && alloc <= address + delta) {
*allocatedSizePointer = bytes;
return alloc;
}
/* mmap answered a mapping well away from where Spur prefers. Discard
* the mapping and try again delta higher.
*/
if (munmap(alloc, bytes) != 0)
perror("sqAllocateMemorySegment... munmap");
address += delta;
}
return 0;
}
This appears to work well, allocating memory at ascending addresses while skipping over any existing mappings.
HTH
