4

I'm trying to find a way to make the Flutter unit test failures messages more useful to me.

Right now on a simple test that doesn't do anything it useful will produce a stack trace of 50+ lines of things that are all useless to me. I don't care what the Flutter framework is doing to get my test to run, I care about what I can control, in this example the top 2 lines.

Currently it does something like this:

══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
The following TestFailure object was thrown running a test:
  Expected: <true>
  Actual: <false>

When the exception was thrown, this was the stack:
#4      main.<anonymous closure> (file:///D:/disclaimer_screen_test.dart:57:5)
<asynchronous suspension>
#5      main.<anonymous closure> (file:///D:/disclaimer_screen_test.dart)
#6      testWidgets.<anonymous closure>.<anonymous closure> (package:flutter_test/src/widget_tester.dart:146:29)
<asynchronous suspension>
#7      testWidgets.<anonymous closure>.<anonymous closure> (package:flutter_test/src/widget_tester.dart)
#8      TestWidgetsFlutterBinding._runTestBody (package:flutter_test/src/binding.dart:784:19)
<asynchronous suspension>
#11     TestWidgetsFlutterBinding._runTest (package:flutter_test/src/binding.dart:764:14)
#12     AutomatedTestWidgetsFlutterBinding.runTest.<anonymous closure> (package:flutter_test/src/binding.dart:1173:24)
#13     FakeAsync.run.<anonymous closure>.<anonymous closure> (package:fake_async/fake_async.dart:178:54)
#18     withClock (package:clock/src/default.dart:48:10)
#19     FakeAsync.run.<anonymous closure> (package:fake_async/fake_async.dart:178:22)
#24     FakeAsync.run (package:fake_async/fake_async.dart:178:7)
#25     AutomatedTestWidgetsFlutterBinding.runTest (package:flutter_test/src/binding.dart:1170:15)
#26     testWidgets.<anonymous closure> (package:flutter_test/src/widget_tester.dart:138:24)
#27     Declarer.test.<anonymous closure>.<anonymous closure> (package:test_api/src/backend/declarer.dart:175:19)
<asynchronous suspension>
#28     Declarer.test.<anonymous closure>.<anonymous closure> (package:test_api/src/backend/declarer.dart)
#33     Declarer.test.<anonymous closure> (package:test_api/src/backend/declarer.dart:173:13)
#34     Invoker.waitForOutstandingCallbacks.<anonymous closure> (package:test_api/src/backend/invoker.dart:231:15)
#39     Invoker.waitForOutstandingCallbacks (package:test_api/src/backend/invoker.dart:228:5)
#40     Invoker._onRun.<anonymous closure>.<anonymous closure>.<anonymous closure> (package:test_api/src/backend/invoker.dart:383:17)
<asynchronous suspension>
#41     Invoker._onRun.<anonymous closure>.<anonymous closure>.<anonymous closure> (package:test_api/src/backend/invoker.dart)
#46     Invoker._onRun.<anonymous closure>.<anonymous closure> (package:test_api/src/backend/invoker.dart:370:9)
#47     Invoker._guardIfGuarded (package:test_api/src/backend/invoker.dart:415:15)
#48     Invoker._onRun.<anonymous closure> (package:test_api/src/backend/invoker.dart:369:7)
#55     Invoker._onRun (package:test_api/src/backend/invoker.dart:368:11)
#56     LiveTestController.run (package:test_api/src/backend/live_test_controller.dart:153:11)
#57     RemoteListener._runLiveTest.<anonymous closure> (package:test_api/src/remote_listener.dart:256:16)
#62     RemoteListener._runLiveTest (package:test_api/src/remote_listener.dart:255:5)
#63     RemoteListener._serializeTest.<anonymous closure> (package:test_api/src/remote_listener.dart:208:7)
#81     _GuaranteeSink.add (package:stream_channel/src/guarantee_channel.dart:125:12)
#82     new _MultiChannel.<anonymous closure> (package:stream_channel/src/multi_channel.dart:159:31)
#86     CastStreamSubscription._onData (dart:_internal/async_cast.dart:85:11)
#120    new _WebSocketImpl._fromSocket.<anonymous closure> (dart:_http/websocket_impl.dart:1145:21)
#128    _WebSocketProtocolTransformer._messageFrameEnd (dart:_http/websocket_impl.dart:338:23)
#129    _WebSocketProtocolTransformer.add (dart:_http/websocket_impl.dart:232:46)
#139    _Socket._onData (dart:io-patch/socket_patch.dart:2044:41)
#148    new _RawSocket.<anonymous closure> (dart:io-patch/socket_patch.dart:1580:33)
#149    _NativeSocket.issueReadEvent.issue (dart:io-patch/socket_patch.dart:1076:14)
(elided 111 frames from dart:async and package:stack_trace)

This was caught by the test expectation on the following line:
  file:///D:/disclaimer_screen_test.dart line 57
The test description was:
  someScreen Widget Test
════════════════════════════════════════════════════════════════════════════════════════════════════
Test failed. See exception logs above.
The test description was: someScreen Widget Test

✖ someScreen Widget Test
Exited (1)

Ideally I would like to limit or remove the stack trace altogether so that it looks something like this:

══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
The following TestFailure object was thrown running a test:
  Expected: <true>
  Actual: <false>

This was caught by the test expectation on the following line:
  file:///D:/disclaimer_screen_test.dart line 57
The test description was:
  someScreen Widget Test
════════════════════════════════════════════════════════════════════════════════════════════════════
Test failed. See exception logs above.
The test description was: someScreen Widget Test

✖ someScreen Widget Test
Exited (1)

This way I get all the relevant information to me without all the noise of things I cannot control and quite honestly are not broken. I trust that the Framework and 3rd party packages are working as expected, if they aren't I'm doing a different investigation.

I found the defaultStackFilter method but that appears to be an internal Flutter method rather than something I can control, perhaps I just don't understand it.

Is there a way to change the stack output as a whole or perhaps a way to disable it in the unit testing environment?

Thank you in advance!

Michael Woolsey
  • 803
  • 8
  • 15

2 Answers2

2

To elaborate on Randal Schwartz's answer, here is the full code I used:

import 'package:stack_trace/stack_trace.dart';

void main() async {
  WidgetsFlutterBinding.ensureInitialized();
  FlutterError.onError = (FlutterErrorDetails details) {
    print("Error :  ${details.exception}");
    print(Trace.from(details.stack!).terse);
  };
  Chain.capture(() async {
    runApp(MyApp()); // starting point of app
  }, onError: (error, stackTrace) {
    print("Async Error :  $error");
    print(stackTrace.terse);
  });
}

This incorporates another answer here which explains how to intercept both sync and async errors that occur within the Flutter context.

This gives us full control of how errors are displayed. I appreciate this particularly for the async case, where previously the stack traces did not contain any code which was written by me. Now it properly points to the originating network call that I made.

This approach seems like it should be standard practice, as it drastically reduces the amount of time and energy searching for the cause of errors.

Brandon
  • 917
  • 10
  • 11
1

Yes... the stack_trace package in the pub has this:

You can further clean up the stack trace using Trace.terse. This folds together multiple stack frames from the Dart core libraries, so that only the core library method that was directly called from user code is visible.

Very handy.

Randal Schwartz
  • 39,428
  • 4
  • 43
  • 70