Stack Overflow
Stack Overflow

Questions tagged [mappedbytebuffer]

A direct byte buffer whose content is a memory-mapped region of a file.

A direct byte buffer whose content is a memory-mapped region of a file.

Part of java.nio package from version 1.4

Docs: https://docs.oracle.com/javase/7/docs/api/java/nio/MappedByteBuffer.html

61 questions
10
votes
3 answers

Using files for for shared memory IPC

In my application, there is one process which writes data to a file, and then, in response to receiving a request, will send (some) of that data via the network to the requesting process. The basis of this question is to see if we can speed up…
dan.m was user2321368
  • 1,600
  • 9
  • 18
9
votes
4 answers

How to properly close MappedByteBuffer?

This is the code I'm running: import java.io.RandomAccessFile; import java.nio.MappedByteBuffer; import java.nio.channels.FileChannel; public class Main { public static void main(String[] args) throws Exception { String filePath =…
ZhekaKozlov
  • 36,558
  • 20
  • 126
  • 155
8
votes
2 answers

How do I avoid mapFailed() error when writing to large file on system with limited memory

I have just encountered an error in my opensrc library code that allocates a large buffer for making modifications to a large flac file, the error only occurs on an old PC machine with 3Gb of memory using Java 1.8.0_74 25.74-b02 32bit Originally I…
Paul Taylor
  • 13,411
  • 42
  • 184
  • 351
7
votes
0 answers

How do memory-mapped files work in a multi-process scenario?

This question has been confusing me for days: Assume that I have two processes (p_write and p_read) which run on the same machine. Process p_write is for writing/updating the mmap file. Process p_read is for consuming the mmap file, or in other…
Lubor
  • 989
  • 3
  • 10
  • 33
5
votes
0 answers

Read huge file ~14GB file using MappedByteBuffer

I am trying to solve producer consumer problem which is I/O intensive. Producer constants appends data to a file and consumer reads from this growing file. File size is usually in GB (around 10GB) . Initially I tried BufferedOutputStream and…
Srujan Kumar Gulla
  • 5,721
  • 9
  • 48
  • 78
5
votes
1 answer

Why is this "line count" program slow in Java? Using MappedByteBuffer

To try MappedByteBuffer (memory mapped file in Java), I wrote a simple wc -l (text file line count) demo: int wordCount(String fileName) throws IOException { FileChannel fc = new RandomAccessFile(new File(fileName), "r").getChannel(); …
cidermole
  • 5,662
  • 1
  • 15
  • 21
4
votes
1 answer

Multi-threaded ByteBuffers slower than sequential?

I have a huge byte array that needs to be processed. In theory, it should be possible to slice the work into even pieces and assign them to different threads to increase performance on a multi-core machine. I allocated a ByteBuffer for each thread…
BullyWiiPlaza
  • 17,329
  • 10
  • 113
  • 185
4
votes
2 answers

java Memory mapped Files multithreading read / write

I have 2 threads that concurrently access the same large file(.txt). 1st Thread is reading from the File. 2nd Thread is writing to the File. Both threads access the same block e.g. (start:0, blocksize:10), but with different channel & Buffer…
Rami.Q
  • 2,486
  • 2
  • 19
  • 30
3
votes
2 answers

How mappingsize or limit affects MappedByteBuffer performance?

I am working with large files and I am using MappedByteBuffer to read and write operations. I have a little lack of knowledge so I am wondering somethings about it. MappedByteBuffer buf = raf.getChannel().map(FileChannel.MapMode.READ_WRITE, offset,…
cgrgcn
  • 361
  • 2
  • 6
  • 24
3
votes
0 answers

How does MappedByteBuffer get garbage-collected?

From the Java docs, The contents of direct buffers may reside outside of the normal garbage-collected heap, and so their impact upon the memory footprint of an application might not be obvious Also from the Java docs, MappedByteBuffer: A direct…
Viplav Kumar
  • 85
  • 5
3
votes
3 answers

Converting Bitmap to ByteBuffer (float) in Tensorflow-lite Android

In tensorflow-lite android demo code for image classification, the images are first converted to ByteBuffer format for better performance.This conversion from bitmap to floating point format and the subsequent conversion to byte buffer seems to be…
anilsathyan7
  • 1,423
  • 17
  • 25
3
votes
2 answers

Read large file multithreaded

I am implementing a class that should receive a large text file. I want to split it in chunks and each chunk to be hold by a different thread that will count the frequency of each character in this chunk. I expect with starting more threads to get…
barni
  • 49
  • 1
  • 2
  • 7
3
votes
4 answers

The best way to read a huge file (for example a very large text document)

I'm new to java ...in my current project I need to read and write a very huge text file (1 GB - 5 GB) ... first i used this classes : BufferedReader and BufferedWriter public static String read(String dir) { BufferedReader br; String result…
AvB
  • 171
  • 1
  • 2
  • 12
2
votes
0 answers

Cannot delete temporary file after writing to it via a MappedByteBuffer

When I run this JUnit test, it fails because file.delete() returns false: @Test public void testDeleteTempFile() throws Exception { File file = Files.createTempFile("_file_del_test", ".txt").toFile(); try (RandomAccessFile f = new…
Michael
  • 4,722
  • 6
  • 37
  • 58
2
votes
0 answers

File memory mapping in java is useless if memory have to be copied to primitives to be manipulated later

I am using MappedByteBuffer to load a file I wrote into memory. This is supposed to be a fast way of loading a file. My file contains a bunch of ints and doubles. There are a lot of tutorials online that show you how to write and then read the file.…
bcsta
  • 1,963
  • 3
  • 22
  • 61
1
2 3 4 5