issue with caching and threading

Question

In this app I am working on, I am caching content such as audio from online. At Certain points in the app I run a batch download on content the app predicts the user will be needing soon. All of this is done through my own cache system and in background threads. The basic jist of it is that I send my cache the url, which it hashes to a file name, then we search the correct location for the file, if it is there we return the location of the file we found, if not we download the file, and then return the location of the file. When I do batch operations I throw them in a queue so that they do not clog up the threadpool and make it impossible to have other higher priority async tasks done. This way since the queue runs one new async task on completion of the previous one if a device cant run multiple tasks then the one I need done now will be next in line.

Here is the issue.

If I am currently writing a certain audio file to the sd card, if I request it as the queue is running it, what will happen? Can two threads try to write the same file at once?

Or will it find the file, and return a half-created and corrupt file?

how might I avoid this?

I have thought of possibly running everything through a priority queue, however with devices that are capable of running multiple async threads this will create a even tighter bottle neck.

edit:

my app covers android 2.3-current

according to this thread Running multiple AsyncTasks at the same time -- not possible?

it seems that depending on the os/device version that tasks may be ran in parallel (all at once to a limit) or one-at-a-time.

xD which is why im trying to manage them myself.

Answer 1

Depending on your operating system you might be able to write to the same file concurrently, windows for example will not let 2 processes lock the same file. But writing at the same time will give you no guarantee that what you are writing will be in a useful order, you almost certainly corrupt the file.

Probably the best thing to do is have a central writer for the device like you mentioned, or you can have a locking scheme where the multiple writers wait nicely for the other to finish.

Answer 2

You can use a ConcurrentHashMap to avoid the problem of multiple threads writing to the same file.

ConcurrentHashMap<String, Boolean> map = new ConcurrentHashMap<>();

public void writeToFile(String fileName, byte[] data) {
    // putIfAbsent returns the value previously associated with fileName, or null
    if(map.putIfAbsent(fileName, Boolean.TRUE) == null) {
        try(FileOutputStream stream = new FileOutputStream(fileName)){
            stream.write(data);
        } finally {
            map.remove(filename);
        }
    }
}

So long as all threads use the putIfAbsent guard, this will prevent multiple threads from writing to the same filename.