Published on October 20th, 2005
Project management is all about planning and execution -- "beginning with the end in mind." A good plan contains goals using metrics, with optimal resource usage and realistic schedule estimates.
When managers and their teams engage in planning, the results of that process typically do not address the problems that cause slips or productivity and quality issues. Most plans focus solely on task performance rather than defining the verification problem, independent of its solution. Project management is all about planning and execution -- "beginning with the end in mind." Most teams don't do complete verification planning; in fact, the verification plan isn't really a plan, it's merely a set of incomplete discussion notes that atrophy as the project moves forward.
What is good planning?
Verification planning needs to change from a focus on "how" to a focus on "what." By starting with the important goals of what needs to be verified, the team can ensure the plan is complete and balanced. The plan in this case is more than an engineering specification of how the verification is to be accomplished.
Basics of verification planningThe six basic steps of verification planning are:
Analyze the device specifications
All projects begin with several forms of specifications. There are customer requirements, resource and schedule limitations, build versus buy constraints, systems architecture limitations, and hardware and software design objectives that must be considered comprehensively in order to develop a good plan.
The biggest risk to the project is not anticipating the moving target. Some teams attempt to capture all their requirements and then move through the project sequentially, sometimes called the �waterfall� method. In reality, the process is quite iterative and does not allow for managing the frequency and burden of those iterations.
Scope the verification objectives
The most common failure at the scoping stage is not requiring a comprehensive verification discussion with everyone involved. All stakeholders must participate in scoping objectives: management, marketing, systems designers, hardware designers, software designers, and the verification team.
The verification team needs to drive the discussions, continuously challenging the team to fill the gaps and identify the unverifiable choices � a key design-for-verification objective. If any of these points of view are missed, the gaps and risks will remain.
Identify the feature set of the design
Historically, engineers have designed tests, implemented checkers, and used code coverage. Test lists have become obsolete as a metric because they have become too numerous to specify or implement.
Total coverage is the only means to have a complete picture of the verification status of a project. Functional coverage correlates directly to the features. Assertion coverage relates to functional coverage and also to the implementation integrity. Hardware and software code coverage tell us how well we have exercised the design.
Design detailed coverage models
The scoping process results in a list of the features to be verified. The specification must describe those features so that they can be measured. Coverage is used to define the metrics of verification, derived from design features.
The typical failure at this stage of planning is lack of review of the coverage models. Judgment is crucial because it's a lot better to have verified 100% of the most critical functionality than to have some critical functionality lost in a coverage model that is too detailed.
Select aggregate metrics to track progress
Management is all about planning and executing. Tracking progress is the only way to know your team is executing. Increasingly teams define milestones that expose possible system integration issues. This requires a fine-grained definition of features that must be working at each milestone.
By using coverage to define those features, the project milestone becomes important for much more than simply marking a box in the schedule as complete. It truly reduces quality and schedule risk in the project.
With this picture, "good" verification planning can be defined as using the input of all stakeholders to capture and review the verification plan, using metric-based schedule estimates, all captured in an executable verification plan.
Steve Brown is Director of Marketing for Enterprise Verification Process Automation at Cadence Design Systems. He is a 20 year veteran of the EDA verification industry and has held various engineering and marketing positions at Cadence, Verisity, Synopsys, and Mentor Graphics. He specializes in solving engineering, management, and marketing challenges that arise when new technology and products enter the market. He earned BSEE and MSEE degrees from Oregon State University and has studied marketing strategy at Harvard, Stanford, Kellogg, and Wharton.