private final Map<String, Path> relativePathsByKey = new ConcurrentHashMap<>();
private byte[] readValue(final String key) throws IOException {
Path relativePath = relativePathsByKey.get(key);
final byte[] returnValue;
if (relativePath != null) {
returnValue = readValue(getCacheFileFullPath(relativePath, key));
} else {
relativePath = createRelativeFilePath(key);
Path fullPath = getCacheFileFullPath(relativePath, key);
if (fullPath.toFile().exists()) {
returnValue = readValue(fullPath);
relativePathsByKey.put(key, relativePath);
} else {
returnValue = null;
}
}
return returnValue;
}
private Path createRelativeFilePath(final String key) {
final MLFileCacheKey fileCacheKey =
new MLFileCacheKey(
key, offHeapCacheConfig.getLevels(), offHeapCacheConfig.getSegmentSize());
final StringBuilder sb = new StringBuilder();
final String[] segments = fileCacheKey.getKeySegments();
for (int i = 0; i < segments.length; i++) {
final int hash = segments[i].hashCode();
sb.append('d')
.append(i)
.append('_')
.append(Math.abs(hash % offHeapCacheConfig.getDirectoriesPerLevel()))
.append(File.separator);
}
return Paths.get(sb.toString().intern());
}
I keep the file paths in a ConcurrentHashMap for quick access to files. I took two heap dumps at different times. The results are shown below. What I found is that the size (memory) of the map is totally not making sense when compared to each other.
The second last column from the right is the retained size in bytes as shown by yourkit heap dump:
When the map had ~10M entries - retained size = 1.7GB
When it had ~20M entries - retained size = 6.4GB. It is not making sense at all to me when compared to the above picture.
When I went into each node of the map I see that each node in the map is having a list of next nodes:
node from the map having ~20M entries and the node's retained size = 2,368 bytes
node from the map with ~10M entries and the node's retained size is much less 1,216 bytes:
Then I went ahead and checked the retained objects of one of the "next" nodes in both dumps:
The node from the dump with ~20M entries is referencing two extra objects that are taking up the extra space:
From the dump with ~20M entries:
One from the dump with ~10M entries:
I'm now confused as to whether it is the GC that's lagging or I took the two dumps at times that did not line up with the way the map internally organizes itself in Java.
