A crisis is looming in functional-verification coverage metrics. The industry has no choice but to fix it now. Otherwise—in light of more logic and processor cores being squeezed into a single product—complex system-on-a-chip (SoC) design could very well come to a screeching halt. Industry observers and pundits have recently jousted over the coverage-metrics crisis. One side says that constrained random testing doesn’t work and we have to dump it. Why? They cite the inefficiency of testing complex functionality (by some estimates 80% of constrained random simulation cycles test 20% of the design) as well as the lack of a link between what we need to test (functional intent) and randomly generated inputs.

The other side argues that constrained random isn’t so bad. After all, it does allow verification to be automated so that we can burn compute cycles instead of man-hours to verify designs. We just wish there was a more efficient way to test all of the complex functional intent that we put into our designs.
Both sides are right and wrong. Verification engineers and design project managers need to have a practical, comprehensive way to verify SoC designs. Both approaches want to get to 100% coverage (i.e., to be sure that everything has been tested). What we lack is a decent way to define what 100% coverage means.

What’s the problem? When a clock is tested, code coverage reveals whether every cog in the clock has moved. But it cannot convey whether combinations of cogs have turned together. Functional coverage informs the designer if the cogs in the implementation have been turned to every possible position—and if combinations of cogs have moved. But neither measure connotes whether the designer has tested that the clock keeps time. Achieving 100% coverage using these two metrics still means that the clock could be missing a cog or two needed for proper operation. But the engineer forgot to test the cases in which those cogs were needed.

A metric is needed for understanding how much of a design’s functional intent has been tested. It should convey what outcomes (and combinations of outcomes) have been seen from the design (e.g., has a packet been routed or dropped) instead of only measuring how much of the implementation has been exercised. There should be an automated method of achieving 100% coverage on this metric.

To get the best of both worlds, one option is to build intelligent testbenches that comprehend the functional intent of a design and generate directed functional test cases to ensure that all of this intent is covered. In my opinion, constructing coverage models using a combination of temporal graphs and path constraints provides a simple and logical approach to decomposing complex design specifications. It also distills the implied functional intent into a concise representation.

Once the verification engineer constructs the coverage model, functional-verification software can traverse the model to generate a suite of directed test cases. As a result, coverage of the captured verification intent will be guaranteed. Each generated test case can then drive input stimulus that will exercise a desired outcome and check for the desired behavior. To be practical, the generated test cases must work with established, constrained random testbench methodologies, such as VMM/OVM, as well as established testbench environments like System Verilog, Specman, System C, and VHDL.

By defining what exactly is meant by functional intent coverage, designers can make tests efficient. After all, each test exercises a specific combination of outcomes. At the same time, designers can retain the benefits of an automated approach, which allows computers to work on the verification problem instead of human beings. The best way to realize the promise of the intelligent testbench is to build coverage models using a combination of temporal graphs and path constraints. This approach has proven to be effective for defining coverage intent that is simple and well structured, concise, and suitable for the automated generation of stimulus, result checking, and coverage analysis.