if local variables cant be tested then what other ways can the variable values be checked

Question

This is going to be a long question, driven by the thirst of wanting to know how something is working against the conventional methodologies.

I came across a very interesting application codeacademy, that was actually testing the local variables within a main method. Here is a screenshot of the page, that got me into thinking how this can be possible. found some similar questions in stackoverflow.

Is it possible to obtain variables in main method from junit test?

How to use BCEL or ASM to get the value of a local declared variable in JAVA?

I am not satisfied by knowing that it cant be done, what i want to know, is that, is there a way, like the java compiler api or some other, that can point me into knowing how such an application made is possible.

The value of local variables *don't matter* in tests, because they don't exist once the method exits. You should only care about things that are observable outside the lifetime of the method, e.g. return values, exceptions, member variables, invocations of methods on dependencies. With that said, you might care about their values while you are debugging; and this is what a debugger is for. — Andy Turner, Dec 01 '17 at 12:35
Please avoid using screen shots. You can put code as **text** into your question; and you could simply quote the instructions as code as well. Images dont work for screen readers - and there is nothing in your question that requires the usage of images! — GhostCat, Dec 01 '17 at 12:35
@GhostCat i wanted to post the screenshot to show, that the instruction set in the web app was actually checking a local variable value — isnvi23h4, Dec 01 '17 at 12:37
I don't think they are running your program just reading it as text and applying checks whether the variables contain certain values. My guess would be they are using regex. — Yasser Hussain, Dec 01 '17 at 12:38
It might be using the debugging interface. When debugging in e.g. Eclipse, you also see the local variables. Doesn't mean I'd support the idea of unit-testing internals of a method - not at all! — Ralf Kleberhoff, Dec 01 '17 at 12:38
*This is going to be a long question,* Really? When will you expand it to be long? This is pretty short, and barely expands on what the question title is asking. ([This](https://stackoverflow.com/questions/46012574/per-element-atomicity-of-vector-load-store-and-gather-scatter) is a medium-to-long question.) — Peter Cordes, Dec 01 '17 at 12:45
So what happens on the codeacademy site, if you do `myNumber = 21 + 21;`, will it accept it? And how about `myNumber = (myNumber = 21) << 1;`? And `myNumber = 21; myNumber <<= 1;` ? ... all of them set `myNumber` to value `42`, but if the course check is string-based, it will surely not cover all the more tricky variants. — Ped7g, Dec 01 '17 at 12:50
@Ped7g thanks for that helpful comment, ran the code `myNumber = 21 + 21` and it failed to recognize the correct variable value. — isnvi23h4, Dec 01 '17 at 13:11
Then, it’s parsing the text rather than the compiled code, as `myNumber = 21 + 21` compiles to exactly the same bytecode as `myNumber = 42`. — Holger, Dec 01 '17 at 13:12
Maybe even `myNumber = 0x2A;` would break it? Which is even funnier, as that doesn't even require additional compile-time evaluation from compiler, it is literally **same** as `myNumber = 42;` in almost every possible sense, except using different base for number formatting. ( @souparnomajumder sorry for bothering you, but I don't want to register on that site just to satisfy my curiosity) — Ped7g, Dec 01 '17 at 13:34

score 2 · Accepted Answer · answered Dec 01 '17 at 14:01

Testing local variables in implementation-validation (like unit testing, or QA automated testing) is generally bad practice.

Local variables depend on the particular implementation, and particular implementation should be hidden behind reasonably abstract API, to allow the developers to replace the implementation in the future - if they get some better idea - without affecting consumers of the result (as long, as the API is so good, that it doesn't need any change).

With some very complex implementations/algorithms a developer indeed may be interested to verify particular intermediate/inner results of the complex algorithm, to make the development of the implementation itself easier. At that point it makes sense to create internal inner-API providing reasonably abstract intermediate results, even if they are hard-bonded to particular algorithm, and test that inner-API in unit tests. But in case of the algorithm replacement you have to accept change of the inner-API, and all the inner unit tests. There should be still reasonable high-level abstract API which is unaffected by the inner changes.

A need to have tests on the level of local variables should indicate some deeper problem of the code base.

Your particular use-case of Java tutorial is quite unrelated to real Java code development, as the purpose of that lecture is completely different.

As was shown by the simple myNumber = 21 + 21; test, the validation of lecture is purely compare-text based, using probably some regex on the source code inputted by the student. Not even checking the resulting bytecode, as that one is identical with myNumber = 42;.

If you are working on some lecturing system, using some kind of custom virtual machine and debug interface may work, but in case of simple lectures even text-compare solution may be enough.

Usually when the student is advanced enough to solve some task, you can start to use input/output from/to stdin/stdout to create automated tests to validate student solution with set of known input/output tests, like some sites like https://www.hackerrank.com/ do, or various programming contests. At that point you don't need access to anything, nor local variables, nor API unit testing, you just redirect stdin to feed solution with desired input, and catch stdout to compare it with designed output.

But lecture validation is like completely unrelated to unit testing.

And too much stringent testing for expected result may be even counterproductive in the lecturing process!

Imagine you task the student to write code which will output sum of squares from 1 to N, and you will accept only:

    int sum = 0;
    for (int i = 1; i <= N; ++i) {
        sum += i * i;
    }

(verified on bytecode level, in a way that names of variables and ++ suffix/prefix doesn't matter)

Now some student will try to submit this:

    int sum = 0, square = 0, sq_delta = -1;
    for (int i = 1; i <= N; ++i) {
        sum += (square += (sq_delta += 2));
    }

And it will fail... (even if his solution would be absolutely superior around 1985 to the variant with actual multiplication) ... sad story. :)

score 0 · Answer 2 · answered Dec 01 '17 at 12:50

0

Probably we are over-thinking this.

When I got your question, you are wondering how the online tool used here can know about the content of variables inside some main() method.

Thing is: this isn't necessarily a Java feature.

Keep in mind: it is their web application. They can do whatever they implement in there. In other words: of course you can use a Java compiler to generate an AST (abstract syntax tree) representation of any piece of Java code. And of course, that AST contains all the information that is present in the corresponding source code.

answered Dec 01 '17 at 12:50

GhostCat

137,827
25
176
248

For this trivial code, it would even be possible to compile using a conventional Java compiler (`javac` or `ecj`) with debug information enabled and follow the code flow to predict the variable’s final values. It doesn’t contain anything that can’t be predicted by static code analysis. Of course, that’s also the reason why this code doesn’t deserve unit testing. Unit testing starts, where this predictability ends… – Holger Dec 01 '17 at 13:11
I don't fully agree. Unit testing has several functions: A) when doing TDD, they start as runable spec of your requirement B) they are supposed to help finding bugs/regressions C) most code should be *predictable* ... or do you write code of which you don't know what it will be doing? In that sense: there is a lot of sense in writing unit tests in situations when you exactly know what the code under test is doing. But you want an automated test of knowledge. It doesn't help at all to *know* what it will be doing. – GhostCat Dec 01 '17 at 15:12
1

Maybe you misunderstood me. In this specific example, it is possible to predict the values of the variables without the need to pass in or assume any runtime input. Code that will always evaluate to the same result can be replaced by the constant result and constants leave no doubt about always evaluating to the specified value. For any other nontrivial code, analyzing it with assumed input to predict its result is not different to executing the code. – Holger Dec 01 '17 at 21:55

if local variables cant be tested then what other ways can the variable values be checked

2 Answers2