Write and debug tests easily, built on integration_test

Last update: May 23, 2022

flutter_convenient_test: Write and debug tests easily, built on integration_test

Flutter Package CI Post-release Codacy Badge

Quick demo

full_video.mov

Have questions?

Though used in production environment in my own 200kloc Flutter app, this package - especially the doc - is surely not yet perfect. Just create an issue and I usually reply quite quickly.

Features

Action history

See what actions are taken in the tests (in the left panel)


Time travel with screenshots

Tap an action to see its screenshots


Rapidly re-execute

Edit code, save, run - within seconds


Videos recorded

Watch what has happened in full detail (in right panel)

P.S. Highlighted action is in sync with the playing video. Tap action to seek video.


Being interactive

Temporarily play with the app, interactively. (Flutter originally cannot interact with app in test mode)


integration_test is still there

You can still use everything from integration_test, mockito, test, etc. This package is not a reinvented wheel, and has exposed the underlying integration_test to you.

If you want to migrate to this package from raw integration_test, have a look at Getting Started section below.

Flaky tests awareness

Flaky is flaky, and we are aware of it. It is neither failed nor success, so you will not be alarmed by false positives, and not completely ignore it.

Simpler and shorter code

  • No manual pump
  • No manual wait and retry
await t.get(find.byTooltip('Fetch Data From Backend')).tap();
// OK even if "fetch data" needs undeterministic time interval. Will automatically pump, wait and retry.
await t.get(find.text('New Data')).should(findsOneWidget);

More in quickstart and tutorials below.

Visually see target regions

Useful when replaying videos and screenshots

  • Marks have colored borders
  • Gestures have visual feedbacks

CI / headless mode

This tool can be run without GUI and only produce log data, making it suitable to be run in a CI. Just run flutter run integration_test/main_test.dart --dart-define CONVENIENT_TEST_CI_MODE=true.

If you want to examine the details with GUI, just open the generated artifact in the GUI using Open File button.

Run single test/group

Tap "Run" icon button after each test or group to re-run only that test/group, without running anything else.

Raw logs

Tap "Raw Logs" in the right panel to see raw outputs of a test.

Tutorial: Run examples

  1. Clone this repository and enter the packages/convenient_test/example folder.
  2. Run the example app (e.g. using iOS simulator) via flutter run /path/to/flutter_convenient_test/packages/convenient_test/example/integration_test/main_test.dart --host-vmservice-port 9753 --disable-service-auth-codes --dart-define CONVENIENT_TEST_APP_CODE_DIR=/path/to/this/app. Can also be run via VSCode or Android Studio with similar commands.
  3. Run the GUI located in packages/convenient_test_manager. It is nothing but a normal Flutter Windows/MacOS/Linux app, so run it follow Flutter official doc. Or run via flutter profile mode to speed up. Or execute cd macos && fastlane build and use the generated (release-version) application.
  4. Enjoy the GUI!

Getting started

  1. In pubspec.yaml, add convenient_test: ^1.0.0 to the dependencies section, and convenient_test_dev: ^1.0.0 to the dev_dependencies section. As normal, we need to flutter pub get.
  2. Create integration_test/main_test.dart file in your app. Fill it like void main() => convenientTestMain(MyConvenientTestSlot(), () { ... the normal test code you write });. See the example package for demonstration.
  3. Run your app (e.g. using iOS simulator) via flutter run /path/to/your/app/integration_test/main_test.dart --host-vmservice-port 9753 --disable-service-auth-codes --dart-define CONVENIENT_TEST_APP_CODE_DIR=/path/to/this/app. Can also be run via VSCode or Android Studio with similar commands.
  4. Run the GUI located in packages/convenient_test_manager. It is nothing but a normal Flutter Windows/MacOS/Linux app, so run it follow Flutter official doc. Or run via flutter profile mode to speed up. Or execute cd macos && fastlane build and use the generated (release-version) application.
  5. Enjoy the GUI!

Thanks for testing frameworks in JavaScript, especially Cypress, for giving inspirations!

GitHub

https://github.com/fzyzcjy/flutter_convenient_test
You might also like...

This repository is meant to save all the code I may write about this course.

COD3R - Aprenda Flutter & Dart e Construa APPs iOS e Android 💡 Idea: This repository is meant to save all the code and projects I may write with this

Mar 9, 2022

The classic to-do application where a user can write down all the things he wants to accomplish. Android only.

The classic to-do application where a user can write down all the things he wants to accomplish. Android only.

todo-app The classic to-do application where a user can write down all the things he wants to accomplish. Android only. Table of Contents todo-app Tab

Mar 3, 2022

Ozzie is your testing friend. Ozzie will take an screenshot during integration tests whenever you need. Ozzie will capture performance reports for you.

Ozzie is your testing friend. Ozzie will take an screenshot during integration tests whenever you need. Ozzie will capture performance reports for you.

ozzie.flutter Ozzie is your testing friend. Ozzie will take an screenshot during integration tests whenever you need. Ozzie will capture performance r

Nov 8, 2021

Fake Firebase Performance for use during Flutter unit & widget tests.

Fake Firebase Performance Fakes to write unit tests for apps using Firebase Performance monitoring. Instantiate a FakeFirebasePerformance, then pass i

Apr 16, 2022

Quizlet-Flutter - A quiz app tests the analyzing Information on Financial Statements (True or False)

Quizlet-Flutter - A quiz app tests the analyzing Information on Financial Statements (True or False)

Quizlet-Flutter This is a simple Quizlet mobile application testing your financi

Jan 19, 2022

Integration Test Preview allows tests on multiple screen sizes in a single e2e test run.

Integration Test Preview allows tests on multiple screen sizes in a single e2e test run.

Integration Test Preview has pre-configured methods that allow for faster test deployment for end to end (e2e) test coverage (using Android and iOS pl

Apr 17, 2022

A most easily usable cookie management library in Dart. With SweetCookieJar, you can easily manage cookie on your application.

A most easily usable cookie management library in Dart. With SweetCookieJar, you can easily manage cookie on your application.

A most easily usable cookie management library in Dart! 1. About 1.1. Introduction 1.1.1. Install Library 1.1.2. Import It 1.1.3. Use SweetCookieJar 1

May 19, 2022

A most easily usable cache management library in Dart. With CacheStorage, you can easily manage cache on your application.

A most easily usable cache management library in Dart! 1. About 1.1. Introduction 1.1.1. Install Library 1.1.2. Import It 1.1.3. Use CacheStorage 1.2.

Dec 13, 2021

A most easily usable RESAS API wrapper in Dart. With this library, you can easily integrate your application with the RESAS API.

A most easily usable RESAS API wrapper library in Dart! 1. About 1.1. What Is RESAS? 1.2. Introduction 1.2.1. Install Library 1.2.2. Import It 1.2.3.

Apr 7, 2022
Comments
  • 1. Unhandled Exception: gRPC Error (on Windows)

    Hey there!

    To give you some context. when i run the service worker in my app with this command, and the manager tries to fetch the tests i get: I/flutter (12691): 2022-05-22T08:50:04.966581Z|info|ManagerRpcService|create ciMode=false E/flutter (12691): [ERROR:flutter/lib/ui/ui_dart_state.cc(198)] Unhandled Exception: gRPC Error (code: 14, codeName: UNAVAILABLE, message: Error connecting: SocketException: Connection refused (OS Error: Connection refused, errno = 111), address = 127.0.0.1, port = 51318, details: null, rawResponse: null, trailers: {}) E/flutter (12691):

    The manager itself has this in the log (seems fine): ` An Observatory debugger and profiler on Windows is available at: http://127.0.0.1:57699/io_NKYGf7zM=/ flutter: 2022-05-22T08:49:17.031247Z|info|_ServiceConnectionManager|vmServiceOpened flutter: 2022-05-22T08:49:17.141247Z|info|_ServiceConnectionManager|handleServiceEvent kind=ServiceRegistered service=reloadSources method=s0.reloadSources flutter: 2022-05-22T08:49:17.142247Z|info|_ServiceConnectionManager|handleServiceEvent kind=ServiceRegistered service=hotRestart method=s0.hotRestart flutter: 2022-05-22T08:49:17.142247Z|info|_ServiceConnectionManager|handleServiceEvent kind=ServiceRegistered service=flutterVersion method=s0.flutterVersion
    flutter: 2022-05-22T08:49:17.142247Z|info|_ServiceConnectionManager|handleServiceEvent kind=ServiceRegistered service=compileExpression method=s0.compileExpression flutter: 2022-05-22T08:49:17.143248Z|info|_ServiceConnectionManager|handleServiceEvent kind=ServiceRegistered service=flutterMemoryInfo method=s0.flutterMemoryInfo flutter: 2022-05-22T08:49:17.143248Z|info|_ServiceConnectionManager|handleServiceEvent kind=ServiceRegistered service=flutterGetSkSL method=s0.flutterGetSkSL
    The Flutter DevTools debugger and profiler on Windows is available at: http://127.0.0.1:9103?uri=http://127.0.0.1:57699/io_NKYGf7zM=/ flutter: 2022-05-22T08:49:22.956742Z|info|VmServiceWrapperService|hotRestart start flutter: 2022-05-22T08:49:25.328493Z|info|VmServiceWrapperService|hotRestart end resp={type: Success} flutter: 2022-05-22T08:49:54.811098Z|info|VmServiceWrapperService|hotRestart start flutter: 2022-05-22T08:49:57.576298Z|info|VmServiceWrapperService|hotRestart end resp={type: Success} flutter: 2022-05-22T08:50:03.464899Z|info|VmServiceWrapperService|hotRestart start flutter: 2022-05-22T08:50:05.847344Z|info|VmServiceWrapperService|hotRestart end resp={type: Success}

    `

    I run everthing from comandline to avoid getting stuff from intellij or something. this is my consts.dart

    const kConvenientTestManagerHost = 'localhost';
    const kConvenientTestManagerPort = 3579;
    
    const kWorkerVmServiceHost = '127.0.0.1';
    // TODO temporarily hardcode this
    const kWorkerVmServicePort = 9753;
    
    const kReportFileExtension = 'bin';
    

    Thank you!

    Reviewed by MalteBoehm at 2022-05-22 08:55
  • 2. getNavContext with GoRouter

    Hey there, i try to get it to work with some packages i really like.

    How would you get the "getNavContext" if you were using GoRouter?

    I´m not sure how to get the currentContext from this.

    class MyApp extends ConsumerWidget {
      @override
      Widget build(BuildContext context, ref) {
        return MaterialApp.router(
          routerDelegate: goRouter.routerDelegate,
          routeInformationParser: goRouter.routeInformationParser,
          debugShowCheckedModeBanner: false,
          theme: ThemeData(
            inputDecorationTheme: const InputDecorationTheme(
              labelStyle: TextStyle(color: Colors.blueAccent),
            ),
            elevatedButtonTheme: ElevatedButtonThemeData(
              style: ElevatedButton.styleFrom(
                primary: Colors.black, // background (button) color
                onPrimary: Colors.white, // foreground (text) color
              ),
            ),
          ),
        );
      }
    } ```
    
    
    Thank you, and thanks for this package :) by far best testing experience
    Reviewed by MalteBoehm at 2022-05-21 19:52
  • 3. Add `Manager` app to the github release

    Is your feature request related to a problem? Please describe. It is not comfortable to clone project to run manager app.

    Describe the solution you'd like You can:

    • add manager's source code to the release to make it easier to run it.
    • add a compiled version of application for your os (mac os) as well.
    • create a codemagic pipeline which will build manager for all desctop platforms and then send it somewhere.

    I understand that it is not a trivial task but this enhancement will make using one of the main feature easier. Also I can try to help you with ci setup if you want :)

    Reviewed by Maksimka101 at 2022-05-20 10:53

Related

:bug: Flutter debug helper widget with common and custom actions
:bug: Flutter debug helper widget with common and custom actions

Debug Friend Flutter debug helper widget with common and custom actions This helps you reduce the development and testing time of new features Show so

Apr 3, 2022
A Dart Build Plugin that uploads debug symbols for Android, iOS/macOS and source maps for Web to Sentry via sentry-cli

Sentry Dart Plugin A Dart Build Plugin that uploads debug symbols for Android, iOS/macOS and source maps for Web to Sentry via sentry-cli. For doing i

Apr 26, 2022
UIWidget is a Unity Package which helps developers to create, debug and deploy efficient, cross-platform Apps.
UIWidget is a Unity Package which helps developers to create, debug and deploy efficient, cross-platform Apps.

⚠️ The main repository of UIWidgets has been moved to https://github.com/Unity-Technologies/com.unity.uiwidgets. Please visit the new site if you have

May 13, 2022
Android Debug Drawer for faster development
Android Debug Drawer for faster development

Android Debug Drawer Faster development with Debug Drawer Features DeviceModule - common information about your device BuildModule - app build informa

May 20, 2022
Adds a side menu in all screens with debug information
Adds a side menu in all screens with debug information

Adds a side menu in all screens with debug information. You can decide which information to show and create new modules to include more information.

May 19, 2022
Write iOS&Android Code using Dart. This package liberates you from redundant glue code and low performance of Flutter Channel.
Write iOS&Android Code using Dart. This package liberates you from redundant glue code and low performance of Flutter Channel.

Dart_Native Dart_Native operates as both a code generator tool and a bridge to communicate between Dart and native APIs. Replaces the low-performing F

May 19, 2022
Klutter plugin makes it possible to write a Flutter plugin for both Android and iOS using Kotlin only.

The Klutter Framework makes it possible to write a Flutter plugin for both Android and iOS using Kotlin Multiplatform. Instead of writing platform spe

May 22, 2022
A Todo Notes Application developed with flutter, with basic functionalities to write quick Notes.
A Todo Notes Application developed with flutter, with basic functionalities to write quick Notes.

Notes Application - Flutter A Todo Notes Application developed with flutter, with basic functionalities to write quick Notes. NOTES PASSWORD-PROTECTED

May 14, 2022
The v2ex client write in flutter.
The v2ex client write in flutter.

Language: English | 中文简体 V2LF V2LF is a v2ex unofficial app. 'V2LF' means 'way to love flutter'. The original intention of developing this app is to l

May 22, 2022
This is an open source Tips & Tricks for Flutter, that helps flutter Developers write simple but powerful dart codes.

Showcasing-flutter - This is an open source Tips & Tricks for Flutter, that helps flutter Developers write simple but powerful dart codes.

Jan 4, 2022