StringScanner lost data with the code below:

var content = 'xxxx[fn=4][fn=37][fn=36][fn=12][fn=26]xxxx';

var _scanner = StringScanner(content);
    while (!_scanner.isDone) {
      if (_scanner.scan(RegExp('\\[fn=(\\d+)]'))) {
        print(content.substring(_scanner.lastMatch!.start + 1, _scanner.lastMatch!.end - 1));
      }
      if (!_scanner.isDone) {
        _scanner.position++;
      }
    }

the result is :

I/flutter (25050): fn=4
I/flutter (25050): fn=36
I/flutter (25050): fn=26

but If I use this method directly：

var allMatches = RegExp('\\[fn=(\\d+)]').allMatches(content);
    for (var element in allMatches) {
      print('--${element.start}');
    }

I got the right answer：

I/flutter (25050): [fn=4]
I/flutter (25050): [fn=37]
I/flutter (25050): [fn=36]
I/flutter (25050): [fn=12]
I/flutter (25050): [fn=26]

something lost when I use StringScanner.

<img src="https://avatars.githubusercontent.com/u/17034?v=3" align="left" width="96" height="96"hspace="10"> Issue by kevmoo Originally opened as dart-lang/sdk#18469

It was fixed in https://code.google.com/p/dart/source/detail?r=35368

Need to verify this fix is sufficient.

...and bump the min-SDK to 1.4

The following example demonstrates the issue:

import 'package:string_scanner/string_scanner.dart';

void main() {
  final text = '\u{12345}';
  final scanner = new SpanScanner(text);
  final start = scanner.state;

  while (!scanner.isDone) {
    scanner.readChar();
  }

  print(scanner.spanFrom(start));
}

This code throws:

RangeError: End 2 must not be greater than the number of characters in the file, 1.

I believe the issue is that package:string_scanner operates on code units, whereas package:source_span operates on code points (runes), resulting in this index mismatch for surrogate pairs.

I found a workaround by decoding the string first:

import 'package:source_span/source_span.dart';
import 'package:string_scanner/string_scanner.dart';

void main() {
  final text = '\u{12345}';
  final file = new SourceFile.decoded(text.codeUnits);
  final scanner = new SpanScanner.within(file.span(0));
  final start = scanner.state;

  while (!scanner.isDone) {
    scanner.readChar();
  }

  print(scanner.spanFrom(start));
}

Is the code unit and code point distinction here intentional? If so is there a better way to ensure proper UTF-16 support? The workaround seems reasonable, although having to provide a span rather than the decoded file itself is clunky. Would adding a constructor to SpanScanner which accepts a decoded file or list of code units be a suitable solution? Or could SpanScanner optionally operate on code points instead of code units?

Use a 2.12.0 lower bound since pub does not understand allowed experiments for earlier versions.

Use a 3.0.0 upper bound to avoid a warning in pub and to give some flexibility in publishing for stable.

master contains an @alwaysThrows fix for StringScanner.error (https://github.com/dart-lang/string_scanner/commit/97cdbe21c5c3d4844a0732042b26d37b3032768d). Unfortunately, the CHANGELOG was updated to say 1.1.0, but the pubspec.yaml was updated to say 1.0.4. I think 1.0.4 is correct.

Bumps actions/checkout from 3.1.0 to 3.2.0.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Bumps actions/checkout from 2 to 3.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Bumps dart-lang/setup-dart from 0.3 to 1.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

This reverts commit 7e41395d327e93b432f380aaf4c9668f0d952ef3.

Since this has not been published or rolled through to the SDK or Flutter we will hold it back to reduce risk.

I have solved TODO of duplicated code in line_scanner.dart and eager_span_scanner.dart. But you should refactor my changes carefully because I am beginner in Dart.

Please consider adding these methods

  /// Scans until [delimiter] matches [string] at the current position.
  ///
  /// Returns `true` if something is consumed, `false` otherwise.
  bool scanUntil(Pattern delimiter) {
    final oldPosition = position;
    while (!isDone && !matches(delimiter)) readChar();
    return oldPosition < position;
  }

  /// Scans until any of [delimiters] matches [string] at the current position.
  ///
  /// Returns `true` if something is consumed, `false` otherwise.
  scanUntilAny(List<Pattern> delimiters) {
    final oldPosition = position;
    while (!isDone && !delimiters.any(matches)) readChar();
    return oldPosition < position;
  }

here is my code

class _MyHomePageState extends State<MyHomePage> {
  StringScanner _scanner = StringScanner(testStr);
  @override
  Widget build(BuildContext context) {
    print('$_scanner');
    print('${_scanner.scan(RegExp(r'/\*(.|\n)*\*/'))}');
    return Container(
      child: Text('test'),
    );
  }
}

It works great on the emulator, but when I package the release version, it will cause a stack overflow.

Complete code here：Github code

and I build release apk

apk