The ROHD Verification Framework is a hardware verification framework built upon ROHD for building testbenches.

Overview

Tests Docs License Contributor Covenant

ROHD Verification Framework

The ROHD Verification Framework (ROHD-VF) is a verification framework built upon the Rapid Open Hardware Development (ROHD) framework. It enables testbench organization in a way similar to UVM. A key motivation behind it is that hardware testbenches are really just software, and verification engineers should be empowered to write them as great software. The ROHD Verification Framework enables development of a testbench in a modern programming language, taking advantage of recent innovations in the software industry.

With ROHD and ROHD-VF, your testbench and hardware execute natively in Dart in a single fully-debuggable process. There is no black-box vendor simulator to interact with, just execute your software. You can leverage the cosimulation functionality of ROHD to build ROHD-VF testbenches for designs that include (or are entirely) written in other languages (e.g. SystemVerilog).

The ROHD Verification Framework does not implement exactly the same API as UVM. Rather, it takes some key concepts that are useful for testbench design and omits features that are rarely used, present to work around language limitations in SystemVerilog, encourage outdated or overly opinionated design patterns, or otherwise don't add significant value. The ROHD Verification Framework eliminates the macros and boilerplate that are associated with UVM.

The ROHD Verification Framework offers a simple, clean, and scalable methodology for developing testbenches with moderate to high complexity. Verifying a very small and simple design may be easier with a "peek & poke" type methodology using Logic.value and Logic.inject directly on the DUT interfaces. A "peek & poke" methodology for a larger, more complex design is usually not a scalable approach. The "startup cost" associated with building a full testbench with the ROHD Verification Framework is drastically lower than one might traditionally expect if they had experience with UVM in the past.

Testbenches

A testbench is software used to interact with and test a device under test (DUT). ROHD Verification Framework testbenches are organized in a modular and extenable way using simple base classes which have specific roles. The diagram below shows what a typical testbench might look like. More details about each of the objects in the testbench are described below. This should look very familiar if you've used UVM.

Testbench Diagram

Example

Dive right in with a full example testbench for a counter. The example includes Monitors, a Driver, a Sequencer, an Agent, an Env, a Test, the same DUT as the ROHD counter example, a Sequence with SequenceItems, a scoreboard, and a main function to kick it all off, all in a single commented file.

Constructing Objects

The ROHD Verification Framework does not come with a built-in "factory" (like UVM) for constructing Components in the testbench. Instead, objects can just be constructed like any other object. It is a good idea to build a testbench with modularity and configurability in mind so that behavior can be easily changed depending on the desired test. There is no restriction against using a factory design pattern to build a testbench if that's the right approach for a specific situation. You also might be interested in using other approaches, such as dependency injection. ROHD-VF doesn't push a strong opinion here.

Phases

A lot of setup for the testbench can occur in the constructor of the object. ROHD-VF comes with some phasing (similar to UVM) to help configure, connect, and run a testbench in coordinated steps. Every Component goes through phases.

  • The constructor
    • Clearly enumerate what is required to build the component as part of the constructor parameters.
    • Construct any sub-components.
  • void build()
    • A function which gets called when the test is started, but before the Simulator is running.
  • Future<void> run(Phase phase)
    • A time-consuming function which starts executing when the test and Simulator are running.
    • Use phase to create Objections.
  • void check()
    • A function that gets called at the end of the simulation, for checking the end state for correctness.

Components

A Component is an object which holds a fixed hierarchical position in the testbench. The hierarchy is determined at construction time by passing information about each Component's parent (null if no parent / top level). All of the below classes extend Component. You can build your testbench extending these subclasses of Component or directly extend Component.

Monitor

A Monitor is responsible for watching an interface and reporting out interesting events onto an output stream. This bridges the hardware world into an object that can be manipulated in the testbench. Many things can listen to a Monitor, often logging or checking logic.

Driver

A Driver is responsible for converting a SequenceItem into signal transitions on a hardware interface. The driver accepts incoming items from a Sequencer.

Sequencer

A Sequencer accepts SequenceItems from stimulus sources (e.g. Sequences) and determines how to pass them to the appropriate Driver(s). The default behavior of a Sequencer is to directly pass them to the Driver immediately, but they can be more complex than that.

Agent

The Agent is a wrapper for related components, often which all look at a single interface or set of interfaces. Typically, an Agent constructs some Monitors, Drivers, and Sequencers, and then connects them up appropriately to each other and interfaces.

Env

The Env is a wrapper for a collection of related components, often each with their own hierarchy. Envs are usually composed of Agents, scoreboards, configuration & coordination logic, other smaller Envs, etc.

Test

A Test is like a top-level testing entity that contains the top testbench Env and kicks off Sequences. Only one Test should be running at a time. The Test also contains a central Random object to be used for randomization in a reproducible way.

Stimulus

Sending stimulus through the testbench to the device under test is done by passing SequenceItems through a Sequencer to a Driver.

SequenceItem

A SequenceItem represents a collection of information to transmit across an interface. A typical use case would be an object representing a transaction to be driven over a standardized hardware interface.

Sequence

A Sequence is a modular object which has instructions for how to send SequenceItems to a Sequencer. A typical use case would be sending a collection of SequenceItems in a specific order.

Virtual Sequencers & Sequences

It is possible to create a "virtual" Sequencer whose role is to distribute Sequences or SequenceItems to other sub-sequencers. Sequences that run on a "virtual" Sequencer are called "virtual" Sequences. There's no special support in ROHD-VF for these, but the standard Sequencer and Sequence objects can be easily used for this purpose.

Logging

ROHD-VF uses the Dart logging package for all logging. It comes with a variety of verbosity levels and excellent customizability.

The Test object contains settings for killLevel and failLevel which will, respectively, immediately end the test or cause a test failure when the simulation finishes running. These levels are associated with the levels from the logging package.

To log a message from any ROHD-VF object or component, just use the inherited logger object.


2021 November 9
Author: Max Korbel <[email protected]>

Copyright (C) 2021 Intel Corporation
SPDX-License-Identifier: BSD-3-Clause

Comments
  • The `StreamController` in `Sequencer` should be synchronous

    The `StreamController` in `Sequencer` should be synchronous

    Describe the bug

    A listener of an asynchronous Stream will receive events interleaved with other events in the Dart event loop, which includes ticks from the ROHD Simulator. This can have strange consequences like things appearing to come out of a Stream every clock edge (or other event) rather than all at once. Making the Stream synchronous will ensure listeners receive events immediately.

    bug 
    opened by mkorbel1 1
  • Remove requirement to call super on check for tests

    Remove requirement to call super on check for tests

    Description & Motivation

    It's inconvenient for users to have to call super.check(), especially since it's important that it's called at the end of the check() instead of at the top. Moving the mechanism that Test uses to call check() on all sub-components removes the need to call super's and avoids bugs.

    Related Issue(s)

    N/A

    Testing

    Existing tests

    Backwards-compatibility

    Is this a breaking change that will not be backwards-compatible? If yes, how so?

    No

    Documentation

    Does the change require any updates to documentation? If so, where? Are they included?

    No

    opened by mkorbel1 0
  • Refactor `Test` print level checks

    Refactor `Test` print level checks

    Description & Motivation

    Too much copy-paste in previous implementation where Test checks the printLevel to determine if it should print out. Refactored into a single function called multiple times.

    Related Issue(s)

    N/A

    Testing

    Existing tests

    Backwards-compatibility

    Is this a breaking change that will not be backwards-compatible? If yes, how so?

    No

    Documentation

    Does the change require any updates to documentation? If so, where? Are they included?

    No

    opened by mkorbel1 0
  • Improvements for `Test` failures via `Logger`

    Improvements for `Test` failures via `Logger`

    Description & Motivation

    • Sub-components now accessible in Components
    • The check phase now runs synchronously at the end of the Test rather than triggered through the ROHD Simulator, giving greater control when handling error conditions.
    • Added printLevel to control printing independently of Logger level, which can also disable failures/kills.
    • Made handling of test failures/kills more robust and easier to handle.
    • Fixed a bug where Logger subscriptions could persist across tests.
    • Added more Test tests to cover these areas.
    • Fixed a bug where failures reported via the Logger and found during check phase would sometimes not cause a test to fail.

    Related Issue(s)

    None

    Testing

    Added new testing, and existing tests pass

    Backwards-compatibility

    Is this a breaking change that will not be backwards-compatible? If yes, how so?

    No, shouldn't be

    Documentation

    Does the change require any updates to documentation? If so, where? Are they included?

    opened by mkorbel1 0
  • bugfix Tracker.record to not discard default values

    bugfix Tracker.record to not discard default values

    Description & Motivation

    Tracker.record had a bug where it was discarding its defaults optional argument as if defaults never existed even when given.

    Related Issue(s)

    None

    Testing

    tracker_test.dart has been updated to check to see that the defaults argument is used.

    Backwards-compatibility

    Is this a breaking change that will not be backwards-compatible? If yes, how so?

    Nope

    Documentation

    Does the change require any updates to documentation? If so, where? Are they included?

    Nope; this was the originally intended behavior of Tracker.record

    opened by bbracker-int 0
  • Make `columnWidth` optional for `TrackerField`, fix #10

    Make `columnWidth` optional for `TrackerField`, fix #10

    Description & Motivation

    See #10, not always interested in it.

    Related Issue(s)

    Fix #10

    Testing

    Existing tests updated

    Backwards-compatibility

    Is this a breaking change that will not be backwards-compatible? If yes, how so?

    Yes! This is an API change for TrackerField.

    Documentation

    Does the change require any updates to documentation? If so, where? Are they included?

    API documentation and examples updated.

    opened by mkorbel1 0
  • Make lints more strict and upgrade Dart SDK

    Make lints more strict and upgrade Dart SDK

    Description & Motivation

    • Improving the documentation and code quality with stricter linting.
    • Upgrading the Dart SDK to match an upcoming ROHD version and use new language features.

    Related Issue(s)

    Fix #16 Fix #19

    Testing

    Existing tests pass

    Backwards-compatibility

    Is this a breaking change that will not be backwards-compatible? If yes, how so?

    Yes, the Dart SDK is upgraded.

    Documentation

    Does the change require any updates to documentation? If so, where? Are they included?

    No

    opened by mkorbel1 0
  • Update to contributor covenant v2.1, fix #20

    Update to contributor covenant v2.1, fix #20

    Description & Motivation

    Updating contributor covenant code of conduct to newer version

    Related Issue(s)

    Fix #20

    Testing

    Doc update only

    Backwards-compatibility

    Is this a breaking change that will not be backwards-compatible? If yes, how so?

    No

    Documentation

    Does the change require any updates to documentation? If so, where? Are they included?

    No

    opened by mkorbel1 0
  • Outdated and overly lenient lints

    Outdated and overly lenient lints

    Motivation

    See similar issue in ROHD: https://github.com/intel/rohd/issues/165

    The "lints" package is outdated here as well, and ROHD-VF should have similarly strict rules as ROHD.

    Desired solution

    Upgrade "lints" package, make lints more strict, and fix new lint issues that show up.

    enhancement 
    opened by mkorbel1 0
  • Fix bugs related to test completion and objections

    Fix bugs related to test completion and objections

    Description & Motivation

    Fix bugs so that:

    • Objections don't matter anymore once the simulation has completed
    • The Test.start doesn't finish until the simulation completes

    Related Issue(s)

    Fix #9 Fix #11

    Testing

    Added new tests to cover the bug fixes.

    Backwards-compatibility

    Is this a breaking change that will not be backwards-compatible? If yes, how so?

    No, but if user code relied on buggy functionality there may be some change noticed.

    Documentation

    Does the change require any updates to documentation? If so, where? Are they included?

    No

    opened by mkorbel1 0
  • Make the tracker file writing asynchronous for performance

    Make the tracker file writing asynchronous for performance

    Description & Motivation

    Writing to files synchronously can cause performance issues on the main program thread. Writing asynchronously with the tracker can improve test run time.

    Related Issue(s)

    Fix #12

    Testing

    Existing test covers it.

    Backwards-compatibility

    Is this a breaking change that will not be backwards-compatible? If yes, how so?

    No, but Tracker.terminate is now asynchronous so if it was not safely awaited before it could reveal a new issue in usage.

    Documentation

    Does the change require any updates to documentation? If so, where? Are they included?

    No

    opened by mkorbel1 0
  • Expose the randomSeed

    Expose the randomSeed

    Motivation

    The purpose of the central Test.random is to make it so you can reproduce randomized content. It would be nice to be able to see what the randomSeed was.

    Desired solution

    Expose the randomSeed variable in Test, and perhaps print it out so it shows up in logs by default.

    enhancement 
    opened by mkorbel1 0
  • Provide `Test.random` even if there is no `Test` running.

    Provide `Test.random` even if there is no `Test` running.

    Motivation

    It's recommended to use Test.random for randomization in verification components so that you can have reproducible behavior if you keep the seed the same. However, this can be annoying for unit testing that doesn't need the rest of the Test infrastructure, but still wants to use Test.random.

    Desired solution

    Allow Test.random to return a valid Random object even if there is no instance yet.

    Alternatives considered

    At least throw a more useful Exception instead of accessing a null late Test instance if Test.random is accessed at an unsafe time.

    Additional details

    It's probably a good idea to warn somehow if a non-Test Random object is used and then later a Test is started, since that could produce some unpredictability.

    enhancement 
    opened by mkorbel1 0
  • Add an easy mechanism to add arbitrary phases

    Add an easy mechanism to add arbitrary phases

    Motivation

    It would be nice to offer the ability for users to add any number of arbitrary phases as needed for their project.

    Desired solution

    Perhaps some Futures that could be listened to might be helpful here? It would be nice of Components could detect if a desired phase never got registered? Should all new phases be part of (subphases of) the run phase? It requires some thought for a nice API to implement this.

    enhancement 
    opened by mkorbel1 0
  • Improve GitHub Actions workflow and local testing

    Improve GitHub Actions workflow and local testing

    Motivation

    @chykon proposed some great changes in ROHD in https://github.com/intel/rohd/pull/177, and we should include similar changes in ROHD-VF.

    Desired solution

    Implement similar stuff in ROHD-VF as ROHD for GitHub Actions and local testing.

    enhancement 
    opened by mkorbel1 0
  • Use issue forms instead of old template

    Use issue forms instead of old template

    Motivation

    @chykon made some nice changes in ROHD for the issue templates, we should do something similar in ROHD-VF: https://github.com/intel/rohd/pull/176

    Desired solution

    Update issue templates

    enhancement 
    opened by mkorbel1 0
  • Add some example(s) on how code looks in UVM/SystemVerilog vs. ROHD-VF to help new-comers

    Add some example(s) on how code looks in UVM/SystemVerilog vs. ROHD-VF to help new-comers

    Motivation

    Many people coming to ROHD-VF will have some experience in SystemVerilog UVM. An example of how they differ would be valuable.

    Desired solution

    Add documentation and/or an example with the same thing implemented in SystemVerilog/UVM and ROHD-VF.

    Alternatives considered

    Leave documentation solely focused on ROHD-VF.

    enhancement 
    opened by mkorbel1 0
Releases(v0.4.1)
  • v0.4.1(Nov 29, 2022)

    What's Changed

    • bugfix Tracker.record to not discard default values by @bbracker-int in https://github.com/intel/rohd-vf/pull/27
    • Improvements for Test failures via Logger by @mkorbel1 in https://github.com/intel/rohd-vf/pull/28
    • Refactor Test print level checks by @mkorbel1 in https://github.com/intel/rohd-vf/pull/29
    • Remove requirement to call super on check for tests by @mkorbel1 in https://github.com/intel/rohd-vf/pull/30

    New Contributors

    • @bbracker-int made their first contribution in https://github.com/intel/rohd-vf/pull/27

    Full Changelog: https://github.com/intel/rohd-vf/compare/v0.4.0...v0.4.1

    Source code(tar.gz)
    Source code(zip)
  • v0.4.0(Oct 24, 2022)

    What's Changed

    • Update to contributor covenant v2.1, fix #20 by @mkorbel1 in https://github.com/intel/rohd-vf/pull/21
    • Make lints more strict and upgrade Dart SDK by @mkorbel1 in https://github.com/intel/rohd-vf/pull/25
    • Make columnWidth optional for TrackerField, fix #10 by @mkorbel1 in https://github.com/intel/rohd-vf/pull/26

    Full Changelog: https://github.com/intel/rohd-vf/compare/v0.3.1...v0.4.0

    Source code(tar.gz)
    Source code(zip)
  • v0.3.1(Aug 19, 2022)

    What's Changed

    • Make sequencer and monitor streams synchronous by @mkorbel1 in https://github.com/intel/rohd-vf/pull/13
    • Make the tracker file writing asynchronous for performance by @mkorbel1 in https://github.com/intel/rohd-vf/pull/14
    • Fix bugs related to test completion and objections by @mkorbel1 in https://github.com/intel/rohd-vf/pull/15

    Full Changelog: https://github.com/intel/rohd-vf/compare/v0.3.0...v0.3.1

    Source code(tar.gz)
    Source code(zip)
  • v0.3.0(May 9, 2022)

    What's Changed

    • Upgrade ROHD to v0.3.0 by @mkorbel1 in https://github.com/intel/rohd-vf/pull/7

    Full Changelog: https://github.com/intel/rohd-vf/compare/v0.2.0...v0.3.0

    Source code(tar.gz)
    Source code(zip)
  • v0.2.0(Feb 9, 2022)

    What's Changed

    • Tracker by @mkorbel1 in https://github.com/intel/rohd-vf/pull/4
    • Tracker flexibility updates by @mkorbel1 in https://github.com/intel/rohd-vf/pull/5

    Full Changelog: https://github.com/intel/rohd-vf/compare/v0.1.1...v0.2.0

    Source code(tar.gz)
    Source code(zip)
  • v0.1.1(Feb 9, 2022)

An extension of the Flutter ListView widget for incrementally loading items upon scrolling

incrementally_loading_listview An extension of the Flutter ListView widget for incrementally loading items upon scrolling. This could be used to load

Michael Bui 174 Sep 27, 2022
Flutter plugin for detecting all hardware buttons

DEPRECATED This plugin is deprecated :( We're no longer able to maintain it. hardware_buttons A Flutter plugin for iOS and Android for detecting vario

Flutter Moum 36 Aug 21, 2021
Flutter + Firebase Auth Demo App that uses Google, Facebook, Email/Password Signup/Login, Email Verification and more!

Flutter Firebase Auth Demo Flutter + Firebase Auth Demo App that uses Google, Facebook, Email/Password Signup/Login, Email Verification and more! Feat

Rivaan Ranawat 55 Jan 7, 2023
A Flutter package that help you create a verification input.

flutter_verification_code A Flutter package that help you create a verification input. Based on https://github.com/tiny-express/flutter_verification_c

Alex Awaik 21 Dec 15, 2022
Flying Fish is full-stack Dart framework - a semi-opinionated framework for building applications exclusively using Dart and Flutter

Flying Fish is full-stack Dart framework - a semi-opinionated framework for building applications exclusively using Dart and Flutter.

Flutter Fish 3 Dec 27, 2022
Project demonstrates building a simple chat application using Flutter framework and Firebase cloud

Flutter Chat on Firebase Project demonstrates building a simple chat application using Flutter framework and Firebase cloud. App does not poll for new

Sukitha Udugamasooriya 8 Feb 2, 2022
Arna Framework - A unique set of widgets for building applications with Flutter.

Arna Arna Framework - A unique set of widgets for building applications with Flutter. This Framework is in active development. Any contribution, idea,

Mahan 86 Dec 11, 2022
Dartness is a progressive dart framework for building efficient and scalable server-side applications

Dartness is a framework for building efficient, scalable dart server-side applications. It provides an easy and quick way to develop modern standalone server.

Ricardo Romero 33 Dec 12, 2022
🦜 Parrot - A progressive Dart framework for building efficient, reliable and scalable server-side applications.

?? Parrot A progressive Dart framework for building efficient, reliable and scalable server-side applications. What is Parrot? Parrot is a Dart framew

Odroe 8 Nov 11, 2022
Another breakpoint framework. Aims to simplify as much as possible building adaptive layouts.

Another breakpoint framework. Aims to simplify as much as possible building adaptive layouts. Features Really simple implementation Works with and wit

null 3 Sep 26, 2022
Easy form building in Flutter with a fluent schema API.

former - Easy form building in Flutter Motivation Formik is one of my favorite React libraries. It is a form library that drastically reduces boilerpl

Kenneth 21 Oct 11, 2022
Learn to Code While Building Apps - The Complete Flutter Development Bootcamp

BMI Calculator ?? Our Goal The objective of this tutorial is to look at how we can customise Flutter Widgets to achieve our own beautiful user interfa

London App Brewery 146 Jan 1, 2023
A beautiful design and useful project for Building a flutter knowledge architecture

Flutter Dojo Change log Flutter Dojo Change Log 我的网站 https://xuyisheng.top/ Wiki Flutter Dojo Wiki 体验APK Github Actions APK download 认识Flutter是在18年,移动

xuyisheng 1.4k Dec 21, 2022
A library for building REST APIs easily with Dart

A library for building REST APIs easily with Dart modeled after Express JS for Node Js. The library is still a work in progress and open to contributi

Albert Oboh 43 Oct 4, 2022
An experiment in building a better XMPP client. using Flutter

moxxy An experimental XMPP client that tries to be as easy, modern and beautiful

null 28 Dec 15, 2022
Flutter plugin for building pull to refresh effects with PullToRefreshNotification and PullToRefreshContainer quickly.

pull_to_refresh_notification Language: English | 中文简体 widget to build pull to refresh effects. Web demo for PullToRefreshNotification Chinese blog pul

FlutterCandies 165 Dec 28, 2022
A convenience wrapper for building Flutter apps with PDFTron mobile SDK.

About PDFTron Flutter PDFTron's Flutter PDF library brings smooth, flexible, and stand-alone document viewing and editing solutions using Flutter code

PDFTron Systems Inc. 157 Dec 26, 2022
Responsive UI building in Flutter

Responsive UI building in Flutter - Flutter project based on explaining and learning the responsive design concept in flutter as flutter makes hybrid applications so responsive deign is necessary for that purpose.

Habib ullah 3 May 29, 2022
⚒️ A monorepo containing a collection of packages that provide useful functionality for building CLI applications in Dart.

⚒️ Dart CLI Utilities A monorepo containing a collection of packages that provide useful functionality for building CLI applications in Dart. Document

Invertase 14 Oct 17, 2022