If as you say the system is configured with no swap space, then the amount of free memory can be calculating by adding the "MemFree", "Buffers" and "Cached" values from /proc/meminfo.
This is exactly what the command 'free -m' shows under 'free' in the '-/+ buffers/cache' line.
In Python, I would implement this as follows:
with open('/proc/meminfo', 'rt') as f:
vals = {}
for i in f.read().splitlines():
try:
name, val = i.split(':')
vals[name.strip()] = int(val.split()[0])
except:
pass
memfree = vals['MemFree'] + vals['Buffers'] + vals['Cached']
This would give a value in kilobytes.
As others have said, malloc is not likely to return null ever. Linux will overallocate and when you start using a page that really cannot be found, the OOM killer will kick in.