In the business of providing intellectual-property (IP) cores and design services, Trilinear Technologies has the good fortune of working with a wide and diverse group of customers. The size of our customers ranges from the very small to the very large. Nowhere are the differences in customers more evident than in their approaches to design verification. Before a sale is completed, we typically spend twice as much time discussing verification as we do discussing the design itself. With design complexity growing every year, We—like all companies—have increasingly focused on proper verification.

While there’s some correlation between the size of the company and the importance placed on design verification, we’ve found many exceptions to the general rule. Interestingly, we found no direct link between the amount of resources spent on verification and design quality. More expensive verification tools and larger verification groups don’t necessarily lead to better designs. Observing this curious fact, we decided to investigate further. Our findings from a survey of our customer base may be surprising to some.

Most notably, we found absolutely no correlation between the verification language and the quality of design verification. Teams using advanced verification languages, such as SystemVerilog and ‘e,’ were no more likely to have bug-free code than those using the base languages like Verilog and VHDL. In addition, those using the advanced verification languages primarily make use of constrained random testing while those using the base languages implement directed testing. Constrained random testing was found to generate significantly more simulation cycles than directed testing. But it didn’t result in the expected increase in quality.

After investigating these findings, we discovered that what really separated the good verification teams from the great verification teams was methodology. A good verification methodology and a commitment to execute it turned out to be far more important than the tools chosen to implement it. Specifically, those customers with lower bug rates had a well-developed verification plan and a strict method for tracking their progress with respect to that plan. A typical plan was subject to peer review and published on a regular basis to all members of the verification team.

Many teams make the mistake of investing in expensive verification tools only to find that without better methodologies, their results don’t improve. With these new tools, the focus begins to shift from functionality to coverage. The automated reports generated by the simulator become the goal rather than the tool. Status turns to coverage percentage and number of runs rather than functionality. While these numbers may make everyone feel like progress is being made in the short term, they also can hide lapses in testing.

A good verification plan simplifies the implementation and makes testing more effective. In standard environments, directed tests are well planned with specific goals and objectives. The engineer implementing the test has a clearly defined task with a known outcome. Advanced verification environments also benefit from proper planning with fewer but more effective coverage points and more efficient use of simulation cycles.

We’ve learned from our customers that methodology matters far more than tools in delivering working hardware designs. While certain EDA languages can make engineers more productive, it’s far more important to get the verification team and engineering management to commit to a well-defined plan. From small to large teams, those that start with a solid plan are better positioned for design success.