1

I've written a real-time application for Android that receives hundreds of input events per second and spits out a few output events per second. The input events drive state machines to produce the output events. I write both the input and output events to a file for each run. I want to use the files of runs that I deem valid for regression testing. I'll feed the recorded input events to the system and look for the expected output events.

My system receives events via a looper/handler and uses timers, but it doesn't otherwise depend on the Android API except for android.util.log for debugging purposes, which I can easily wrap.

Which desktop framework will give me the APIs I need? Will a JUnit test suffice? Does this need to be an instrumented test? Do I actually need to run some sort of mocker library? This is my first Android app and I'm still getting my bearings.

Ideally, I'd write a command line tool in Java so I can vary the parameters that govern testing. I doubt the kind of testing I have in mind is suitable for any generic test framework, as some intelligence is required for deciding when to attempt to match actual results with expected results and when to halt the test run. I suspect I'm just looking for suitable desktop libraries.

I think I see how to write another Android app that would do the job, but that looks a lot more complicated than doing it from a shell, and it would be a lot less convenient to use as well. My backup plan is to test my system below the looper API (my library's public API) by using regular Java thread messaging. (I want to feed events to the system at their original timing.)

Update: As I search around for an answer, it's become clear that I could write a JUnit test that runs all my real-time tests, but complaints of slow emulation concern me. Timing is crucial during these tests. On the other hand, the desktop computer could run background processes that interfere with timing, so maybe I do have to create an test harness that runs on the device. Hoping for advice...

Joe Lapp
  • 2,435
  • 3
  • 30
  • 42
  • 1
    Is there anything in your app that would need to be tested from device? If not, you can just write JUnit tests that run on the JVM. Since you're not using the Android API, you wouldn't need to spin up an emulator and do any instrumentation testing. – nicobatu Jan 13 '17 at 00:38
  • Thanks! I'm mainly worried about thread messaging with the looper. Everything else I can stub if necessary. – Joe Lapp Jan 13 '17 at 00:48
  • To clarify, this real-time system exposes an API for application use. I want to test through that API. If I have to work around the Looper, I'm not testing through the API that must be guaranteed to work. Not ideal. – Joe Lapp Jan 13 '17 at 02:57
  • [Robolectric](http://robolectric.org/) emulates the Looper and Handler, but [apparently doesn't work for custom instances](https://github.com/robolectric/robolectric/issues/1993). I'll look at writing my own Looper/Handler emulation. – Joe Lapp Jan 13 '17 at 17:59
  • 1
    Why isn't testing on an actual device against your API an option? From what I understand, you want to feed your system a known input payload, watch for expected output, and make sure the whole thing works under a specific interval of time. This is a case where UI testing + perf counters and data input initialization all need to occur to verify success. No? – Paul Bruce Jan 16 '17 at 18:12
  • Good question. It is an option, but less favorable to me. There is no feedback loop between inputs and outputs, except through a relatively-slow user, so output processing can take its time. I just need to test logic. I hope to be able to run multiple tests simultaneously on a fast computer, because serially running all tests will take a long time. Also, I anticipate less work creating a flexible command line tool than a flexible Android UI. (I'm translating user gestures into user requests.) – Joe Lapp Jan 16 '17 at 22:49
  • Also, as an update, I plan to switch my code over to use `android.os.HandlerThread`, so I can stub that class rather than the looper, that latter of which must somehow hang data on whichever thread it's run from. I'm reorganizing code for this now. – Joe Lapp Jan 16 '17 at 22:51
  • On the other hand, it's possible for the delay between input and output to become unacceptable to the user, or for output to backlog to unacceptable levels on certain devices. You're right, I would need to run tests for this on the device itself. So I will need the ability to run at least load tests on the device. Hmm... Have to think how this changes things for me. Thanks! – Joe Lapp Jan 16 '17 at 23:26
  • Also as you point out, I'll need some end-to-end testing on the device. Okay, so I must support replay on the device regardless. Hmm... – Joe Lapp Jan 16 '17 at 23:45
  • Update: I've written a library of classes for emulating the Looper in real-time testing on the desktop. It works, but I'm having to refactor an interface to allow for realistic real-time testing on the desktop, so I don't yet know whether I've addressed all real-time-related issues that might pop up. – Joe Lapp Jan 27 '17 at 04:12

0 Answers0