A CLI tool to verify the test coverage of a pull request only, ignoring the rest of the project

This is a CI tool that analyzes the coverage rate of a pull request, ignoring the lines that were not changed in the PR.

Motivation

The coverage rate threshold on CI tools is a common approach to encourage developers to write tests and keep improving the whole project's quality. Unfortunately, judging a pull request coverage by analyzing the coverage of the entire project is not always fair, especially on big refactor tasks, witch may naturally decrease the coverage rate.

This package tries a different approach to analyse the test coverage. We only analyse lines that have been added in the pull request. The coverage rate will be calculated by dividing the number of uncovered new lines by the number of new lines.

You can set thresholds to make tests fail on a CI. This package can also print those lines that were not covered, making it easier to identify the missing tests.

Installing

Add this line to your package's pubspec.yaml under the dev_dependencies section:

dev_dependencies:
  pull_request_coverage:

You should specify the version to avoid breaking changes

Usage

This package uses two information to generate its report:

• A lcov.info file, generated by the flutter test --coverage command
• A diff between the current branch and the main one, generated by the git diff command

Generating the lcov.info file

There is a known issue with the flutter test --coverage command. It may not report untested files. There is a workaround for it, described in this issue

Run the following command to generate the coverage/lcov.info file:

flutter test --coverage

Running pull_request_coverage

To check the PR's code, pull_request_coverage needs a diff between its branch and the target one. The diff is read from the STDIN input.

You can feed the STDIN using bash's | operator, like this:

git diff repository/main | flutter pub run pull_request_coverage

Output example:

Uncovered lines are printed in red, starting with → symbol.

Example with thresholds set and not showing uncovered lines:

git diff repository/main | flutter pub run pull_request_coverage --minimum-coverage 95 --maximum-uncovered-lines 5 --hide-uncovered-lines

Exit code

Parameters

--lcov-file

• default : coverage/lcov.info

The path to the lcov.info file generated by the flutter test --coverage command.

--exclude-suffix

• default: .g.dart,.pb.dart,.pbenum.dart,.pbserver.dart,.pbjson.dart

Exclude all file paths that start with those suffixes, separated by commas

--exclude-prefix

Exclude all paths that start with those prefixes, separated by commas

--minimum-coverage

Fail the test if the coverage rate is below this value

--maximum-uncovered-lines

Fail the test if the the number of uncovered lines is less than this value

--report-fully-covered-files

• default: true

The file path of each fully covered file will be printed, as a celebrating message =)

--show-uncovered-code

• default: true

The source code of the uncovered lines will be printed, with a red font color, to make it easier to identify the missing tests. If this parameter is set to false, only the file path will be shown on the log.

--use-colorful-font

• default : true

pull_request_coverage uses a colorful font to highlight uncovered lines. You can disable this by setting this parameter to false