7

I'd like to set up a blocking file read in Java. That is, have a file such that when wrapped by FileInputStream and any read() method is call, the call blocks.

I can't think of an easy OS-independent way - on Unix-like OSes I could try to create a FIFO using mkfifo and read from that file. A possible work around would be to just create a very large file and read from that - the read is unlikely to complete before I capture the stack, but it's ugly and slow (and indeed reads can still be incredibly fast when cached).

The corresponding socket read() case is trivial to set up - create a socket yourself and read from it, and you can have deterministic blocking.

The purpose is to examine stack of the method to determine what the top frames are in such a case. Imagine I have a component which periodically samples the stacks traces of all running threads and then tries to categorize what that thread is doing at the moment. One thing it could be doing is file IO. So I need to know what the "top of stack" looks like during file IO. I have already determined that by experimentation (simply read a file in a variety of ways and sample the stack), but I want to write a test that will fail if this ever changes.

The natural way to write such a test is to kick off a thread which does a file read, then examine the top frame(s). To do this reliably, I want a blocking read (or else the thread may finish its read before the stack trace is taken, etc).

BeeOnRope
  • 60,350
  • 16
  • 207
  • 386
  • Could You explain the blocking? Shoud it block for any Java application, or only within one instance of Your app? – maslan Jan 20 '15 at 23:50
  • I will update the main question with details. – BeeOnRope Jan 21 '15 at 00:22
  • I can't think of any way to do what you want on Windows (which is your problem with OS independence, I think). From your description, I don't think grabbing `FileInputStream.getChannel()` and continually resetting the cursor will work as it will change the stack trace for a while. If you think it might - I can expand in an answer. Similarly, I guess extending FileInputStream won't work as it is the FileInputStream.read* methods you're trying to "fingerprint" if I understand correctly. – J Richard Snape Jan 28 '15 at 11:51
  • Do you have the ability to use your own FileInputStream? Then you could fetch the stack and validate it before the file is actually read. It also sounds like you want to prevent file access in an unwanted context. That is usually the job of a SecurityManager. – M.P. Korstanje Jan 30 '15 at 13:41
  • I don't want to prevent file access in an unwanted context. I want to be able to thread dump (externally) a running thread, and determine "oh, it's in file IO". I could use my own `FIS`, sure, but the problem is I can't take the dump from the thread that is doing the read. By definition, when the _native_ `read0` method (or whatever) is at the top of the stack, I'm not in control and can't dump that thread. I could do it right before, sure, but that does get me the correct top frame or two. – BeeOnRope Jan 30 '15 at 23:55
  • Ah. I understand what you want now. In that case. No. Not OS independently anyway. – M.P. Korstanje Feb 01 '15 at 21:32

6 Answers6

6

To get a guaranteed blocked I/O, read from a console, e.g. /dev/console on Linux or CON on Windows.

To make this platform-independent, you may hack the FileDescriptor of FileInputStream:

    // Open a dummy FileInputStream
    File f = File.createTempFile("dummy", ".tmp");
    f.deleteOnExit();
    FileInputStream fis = new FileInputStream(f);

    // Replace FileInputStream's descriptor with stdin
    Field fd = FileInputStream.class.getDeclaredField("fd");
    fd.setAccessible(true);
    fd.set(fis, FileDescriptor.in);

    System.out.println("Reading...");
    fis.read();
    System.out.println("Complete");

UPDATE

I've realized you don't even need a method to block. In order just to get a proper stacktrace you may invoke read() on an invalid FileInputStream:

    FileInputStream fis = new FileInputStream(new FileDescriptor());
    fis.read(); // This will throw IOException exactly with the right stacktrace

If you still need a blocking read(), named pipes is the way to go: run mkfifo using Runtime.exec on POSIX systems or create \\.\PIPE\MyPipeName on Windows.

apangin
  • 92,924
  • 10
  • 193
  • 247
  • That also crossed my mind (although not the trick to reflectively swap the fd), but is there any other blocking stream to use other than stdin? The problem is there may actually be some input on stdin, and I should not consume it. – BeeOnRope Jan 29 '15 at 00:55
  • The idea to use an invalid filedesc is pretty great, thanks! Awarding you the bounty... – BeeOnRope Feb 02 '15 at 19:38
1

I don't know of anyway to make a File in a OS-Independent way that will always block when read.

If I were trying to find the stack trace when a specific function were called, I would run the program under a debugger and set a break point on that function. Although, method breakpoints will slow down your program and give you different results than you would normally get if timing is important.

If you have access to the source code of the progream, you could make a fake FileInputStream that extends the real one but always blocks on a read. All you need to do is to switch out the import statements throughout the code. However, this won't capture places where you are not able to switch out import statements and it could be a pain if there is a lot of code.

If you want to use your own FileInputStream without changing the program source code or compiling, you can make a custom class loader that loads your custom FileInputStream class instead of the real one. You can specify which class loader to use on the command line by: 

java -Djava.system.class.loader=com.test.MyClassLoader xxx

Now that I think about it, I have an even better idea, instead of making a custom FileInputStream that blocks on read(), make a custom FileInputStream that prints out the stack traces on read(). The custom class can then call the real version of read(). This way you will get all of the stack traces for all calls.

OfNothing
  • 371
  • 1
  • 7
  • The point is I want to see what the stack trace is when the read is occurring. I can't set a breakpoint because I don't know exactly what the method is (and indeed it's a native method likely which is difficult to breakpoint on). I need this for an automated test that verifies that the top frames are "as expected" when doing File IO. – BeeOnRope Jan 21 '15 at 00:19
  • @BeeOnRope So you want the read to hang then you are going to send a signal to the JVM to dump the stack trace to make sure everything is correct? It is possible to put java breakpoints onto methods built into the JVM. Here I think you want to FileInputStream.read(). I am here looking at the source code for that for java7. FileInputStream.read() calls the native function: read0() Because it sounds like you are doing testing, You may simply want to implement it in an OS-dependent way, for example on your *Nix test platform, make a fifo, on your windows platform make a ... *cough* something. – OfNothing Jan 21 '15 at 00:38
  • Yes, although I'm not using a signal, but simply something like `ThreadMXBean.getThreadInfo()` can return the stack for any thread. This is for automated testing, so breakpoints don't apply. The other problem is the whole point that I'm trying to test that the method *is in fact* FileInputStream.read(). – BeeOnRope Jan 21 '15 at 04:20
1

From my understanding you want to write a test which inspects the stack trace of FileInputStream.read() method. What about descendants of FileInputStream if they override the read() method?

If you don't need to inspect the descendants, I think you can use the JVM Tool Interface by inserting a break point at runtime in the desired method, and in the event processing of this event (break point) - dump the stack trace. After the dump is completed you remove the break point and continue the execution. (This all occurs in runtime using this API, no black magic :) )

Genry
  • 1,358
  • 2
  • 23
  • 39
0

You could have a separate thread watch for changes to the file's access time and generate a jvm thread dump when that happens. As to generating the thread dump in code I haven't tried but looks like that's answered here: Generate a Java thread dump without restarting.

I don't know how well this will work with the timing between your threads but I imagine this should come pretty close. I'm also not 100% on the OS independence of this solution as I haven't tested it, but it should work for most modern-ish systems. See the javadocs on the java.nio.file.attribute.BasicFileAttributes to see what will return if it's not supported.

Community
  • 1
  • 1
Foosh
  • 1,195
  • 12
  • 16
  • I think there is some misunderstanding - I want to, in a test scenario, set up a deterministic blocking read so I can take a take a dump of that thread and check that the frames I expect are at the top of stack. Dumping the stack itself is easy, e.g., with [ThreadMXBean.getThreadInfo()](http://docs.oracle.com/javase/7/docs/api/java/lang/management/ThreadMXBean.html#getThreadInfo%28long,%20int%29). – BeeOnRope Jan 28 '15 at 00:23
  • I understood the what, I was trying to answer the how to get the same result. The file's access time would be updated when the read begins so if you generated your dump at that point you would get the top of your thread stack as desired, in code so you could operate on it in your unit test as necessary. No need to block if you still get the data you desire. There are ways to block but it's messy and requires native interactions and using the "fake" FileInputStream mentioned by @OfNothing – Foosh Jan 28 '15 at 01:30
  • I don't see any reason to believe that if I take a dump at the moment the timestamp changes I'll get the IO calls at the top of stack. I'd rather assume the IO is done at that point. – BeeOnRope Jan 28 '15 at 02:59
  • The access time is updated when the file is opened for reading, not when the io operation completes as is the case for create and modify. This is simple enough to test empirically with a `tail -f` command on a *nix system. – Foosh Jan 28 '15 at 19:29
0

One trick is : if it is possible to modify your API to return a Reader instead of a File, then you can wrap a String with a custom StringReader (class SlowAsRubyStringReader extends Reader, say) that overrides the various int read() methods with a Thread.sleep(500) before it does the real work. Only during testing, of course.

@see http://docs.oracle.com/javase/7/docs/api/java/io/StringReader.html

I think there is a larger issue here, not just Files : you want to inspect the context in which an API is getting called during your test cases is it not? That is, you want to be able to examine the stack and say, "aha! I caught you calling the MudFactory API from the JustTookABath object, OUTRAGEOUS!". If this is the case, then you may have to delve into dynamic proxies, which will allow you to hijack function calls or use aspect-oriented programming, which allows you to do the same, but in a more systematic way. See http://en.wikipedia.org/wiki/Pointcut

vijucat
  • 2,059
  • 1
  • 15
  • 17
-1

read() dives quickly into native code so yes probably need to go native to block at that level. Alternatively you may want to consider logging a stack trace at the point in your code before or after read().

Something like:

log ( ExceptionUtils.getStackTrace(new Exception()) );

ExceptionUtils doco is here: https://commons.apache.org/proper/commons-lang/javadocs/api-3.1/org/apache/commons/lang3/exception/ExceptionUtils.html

David Soroko
  • 8,521
  • 2
  • 39
  • 51
  • Logging a trace before or after isn't the point - I need to get the frames that will be on the top of the stack when the read is actually occurring (i.e., the top frame is calling into the native read method). I don't follow why I necessarily need to go native to set up a blocking read. Certainly with sockets it is easy to set up in Java even though at the end of the chain there are of course native methods. – BeeOnRope Jan 21 '15 at 00:18
  • ExceptionUtils gives you stack frames as well. By looking at the source you can complete the picture. – David Soroko Jan 21 '15 at 00:31
  • I know very well how to get a stack trace of the current thread. What I need is that some thread, call it `T1` goes into a blocking read, at which point I will, on another thread `T2`, get the stack trace for `T1`, to see what frames are on the top of stack when a read is occurring. I can't very well do that from `T1` which by definition is (a) blocked and (b) in native code I can't modify. – BeeOnRope Jan 21 '15 at 04:22
  • T1 could put the captured frames in some buffer and let T2 examine the data when it comes along – David Soroko Jan 21 '15 at 08:13
  • That's impossible because `T1` is blocked in a native method. By definition I can't ask it to take a stack trace of itself at that point... – BeeOnRope Jan 21 '15 at 08:19
  • Sure, I was suggesting that T1 buffers the stack data just before entering read() – David Soroko Jan 21 '15 at 08:39
  • ... but that would defeat the whole purpose of the question. At any point I can write Java code (at which point I could "buffer" the stack) I know exactly what the stack will be. What I need to know the stack up to and including the native read method, which happens after we leave the Java code I can write. – BeeOnRope Jan 21 '15 at 09:01