3

I allocated a very large byte array for threads to read and write bytes in the array. Write operation is to assign the new byte directly to the value in the array like byte[i] = byte2;

I can ensured that there will be no conflict for those operation (same section in the byte array will not write by 2 different threads. when a section in reading, no write operation will perform by other threads). My concern is will the modification to array been instantly available to other threads. I know there might be memory fence, which other threads may still read the old values in the array.

If the problem exists, how to avoid? will volatile byte[] store; works in this circumstance?

Shisoft
  • 4,197
  • 7
  • 44
  • 61

2 Answers2

4

will the modification to array been instantly available to other threads.

not without a write fence for the writer and a read fence for the reader.

I know there might be memory fence, which other threads may still read the old values in the array.

You will need to use one of those for the reader and writer (not one per byte, one per operation)

will volatile byte[] store; works in this circumstance?

It won't do what you want, though it will slow down your application. When you do

store[i] = b;

This added a read fence as you are reading the store reference to the byte[].

The best solution is to use Unsafe to have complete control over these read write operations. See the code for AtomicIntegerArray.

However, without going down that road you can do a dummy operation like this which is not as efficient/elegant, but a lot simpler to implement.

private final AtomicBoolean fence = new AtomicBoolean();

public void readFence() {
    fence.get();
}

public void writeFence() {
    fence.set(true);
}

For this to work, you must do a writeFence() after all writes, and a readFence() before all reads.

If you are going down the more low level route you might find using off heap memory an advantage. Off heap memory can store very large (TBs) amounts of data without impacting the GC.

Peter Lawrey
  • 525,659
  • 79
  • 751
  • 1,130
  • Why will volatile not work in his case? It will make sure all threads always have the proper vaule, no? Nvm, I found this: http://stackoverflow.com/questions/2236184/how-to-declare-array-elements-volatile-in-java – Christophe De Troyer Feb 05 '16 at 13:30
  • 1
    @ChristopheDeTroyer correct, all threads will read the right value of the `byte[]` *reference* using a read fence. There is no write fence and the *contents* of the byte[] might no be visible. `volatile` like `final` only applies to the reference to the array, not operations on the array. e.g. `final bytes[] store = { .. }` you can change the contents. You could do a dummy `store = store;` to trigger a write fence, but I find this more hacky than what proposed. The other downside is there is a memory barrier on *every operation* which can slow down the copies by an order of magnitude. – Peter Lawrey Feb 05 '16 at 13:33
  • Thank you, I think I can find my answer in the `AtomicIntegerArray`. I still don't understand what is the `fence` in your example do to the store to make the write fence. – Shisoft Feb 05 '16 at 13:37
  • @Shisoft when you do `fence.set(true)` this uses a write barrier which will provide guarantees for all and any writes which occurred before it. When you do any thread safe read e.g. `fence.get()` this will ensure that every read after that will see the latest data at point. (even if it has nothing to do with it conceptually) – Peter Lawrey Feb 05 '16 at 13:41
  • Another short question, is `AtomicIntegerArray` already off heap? It looks likely because off heap also use unsafe to do the job. – Shisoft Feb 05 '16 at 13:46
  • @Shisoft it's not off heap but it could be modified. If you would like to go low level have a look at Chronicle-Bytes https://github.com/OpenHFT/Chronicle-Bytes which supports on heap arrays and thread safe off heap blocks of any size. Note: the off heap block can be persisted to a file and shared between JVMs on the same machine. – Peter Lawrey Feb 05 '16 at 14:59
  • 1
    I know that you know, but I think it'd still be interesting to mention the old "assign volatile array to itself" trick. That's still the only portable way (what with unsafe being an implementation detail theoretically) if you don't want to use the AtomicArray classes. – Voo Feb 05 '16 at 17:21
  • I am reviewing unsafe right now, are putXXX and getXXX methods the solution to avoid thread cache or memory fence for my case? – Shisoft Feb 06 '16 at 07:03
  • @Shisoft they allow you to access memory without fences and you can selectively use a fence with the `xxxVolatileXxx` and `putOrderedXxx` methods. – Peter Lawrey Feb 06 '16 at 17:53
  • 1
    I have already migrate my application to use unsafe for backend store. It seems more easy to use than my byte array solution since it has internal encoder and decoder for primary types and able to break the 2G size limit. – Shisoft Feb 06 '16 at 18:21
  • If you use https://github.com/OpenHFT/Chronicle-Bytes supports bounds checks so you don't corrupt memory but has thread safe operations and is not limited in size either. It supports memory mapped files which means your data can be immediately available on restart. It also support text and complex binary encodings. E.g. MappedBytes is an unbounded Bytes which grows as required (and the underlying file as well) – Peter Lawrey Feb 06 '16 at 18:25
-2

You can use SynchronizedList for this Example

byte[] c = new byte[];
List list = Collections.synchronizedList(Arrays.asList(c));

It will be thread-safely

Cortwave
  • 4,747
  • 2
  • 25
  • 42