System Level Design sat down to discuss issues in system-level verification with Frank Schirrmeister, director of product development in Synopsys’ solutions group; Donald Cramb, director of professional services at Eve; Patrick Sheridan, director of marketing at CoWare, and Scott Sandler, president of SpringSoft USA. What follows are excerpts of that conversation.

By Ed Sperling
SLD: Will the drivers in design change as we move to more levels of abstraction?
Schirrmeister: The driver still will be re-use. If I’m the manager of a chip project and I want to focus my efforts on the different levels of abstraction, system-level design always head the issue that the ROI was badly or not at all quantifiable. Where software is part of the game, it is very clearly quantifiable. It may be my hardware abstraction layer, my drivers, my middleware. If I start very early on the software development, I have a model there. Now I re-use that verification when I run my RTL. I bring my transaction model. I even re-use it when I go into hardware-assisted verification. And in post-silicon verification, I still can use that software. So verification re-use across the abstraction levels is an important driver.
Sandler: You have to have a way to go from validation on one level to verification.

SLD: What effect do restrictive design rules have on verification?
Sheridan: At the system level, it’s hard to make that connection. But what it boils down to is avoiding getting too excited about achieving performance and overdesigning and not taking into account effects on area and power. When you do have the right design in the end, you have the right performance without the overdesign that would cause power problems. This is an area where system-level or virtualization tools can help, but it does require the user to understand the impact of clock rates, for example, against the target fabrication.
Schirrmeister: It’s a very long feedback loop, but what it boils down to is models and the ability to hold the right data in the models. What people do today with virtualization is to simulate at the system level, use virtual platforms to identify multiple voltage regions and develop the drivers that will control a power management IC to induce a core voltage. So now it comes down to which of those back-end model parameters you can capture and use upwards and automate. You need to be able to characterize a model at the gate level and below to know in this state and this voltage scenario it will consume the following energy. Your system-level model can help you make the right decisions. It’s definitely a challenge from the model perspective to keep all those levels in sync and to characterize upwards.
Sheridan: But it’s going up in abstraction that does enable the user to tell how the system is performing when you make these kinds of tradeoffs. And given that, it’s software in the context of the hardware.
Cramb: You also have the ability to turn all of that stuff off.
Schirrmeister: Yes, the software guy wants one model and the verification guy wants another model, and the architect wants a model in between. That’s one of the issues around modeling. We haven’t found the silver bullet there to enable one model that does it all. That’s why this mix of things in an FPGA prototype with full RTL carried out, and it does what it’s supposed to do, together with virtual models for things you don’t have in RTL becomes more and more important.
Sheridan: You do have to get specific about the design area to make all the different levels of abstraction work.
Schirrmeister: The big issue is to make this universal and be able to bridge between these abstractions.
Sheridan: To deliver something complete you have to decide what you’re going to support, even though the building blocks are general-purpose.

SLD: Chips are no longer just digital. What does that do to the verification process?
Schirrmeister: In all practicality, there are a lot of people putting in mixed signal as a daughtercard to a hardware prototype. You have a PHY interface, which talks to the prototype, which talks back to the model. You bring these in these interfaces. From a simulation perspective, that’s a tough cookie. You have sub-cycle accuracy if you want to go into the analog world, and then you have something that tries to run for 500 cycles before it comes back. That doesn’t typically work all that well unless you put behavioral models in for the analog.
Cramb: People are very prepared to make the tradeoff where they don’t do much with the analog parts and just use the interface. If you include the analog stuff and you have a high-speed serial interface, for example, that becomes the fastest clock in the system. So you’re slowing down the whole emulation or simulation just for that one piece.
Schirrmeister: You may keep the analog portion out, just as the analog may leave out the digital part. But we have users trying to do mechatronic stuff with an analog simulator to connect this to the software. There are use cases where you want to develop your software. But is that the market that will bring us a couple hundred million new users? No.
Sandler: We haven’t really automated system-level verification or validation yet. We’re doing a lot more, but there are so many ways to model. People stitch these together, and there’s another art to manage a project from intent down to chip verification. There may be as many different methods for doing that as projects. But to make money, you can’t supply a commercial solution for every method.
Sheridan: It is possible to invest in all the pieces of one solution and make it profitable. Sandler: But we don’t know today which will be the one that takes off. In the RTL synthesis days, there were multiple vendors and it eventually settled out. The same will happen with system-level design. There will be translation tools and transactions that take off. But there’s a chicken and egg in automation between methodology and representation and tool algorithms. One drives the other in upredictable way.
Schirrmeister: We all have cell phones that seem to work without at least one reboot per day. But they also are heavily constrained. You cannot do certain things. That’s why we can’t make cell phone calls in

System-Level Design sat down to discuss issues in system-level verification with Frank Schirrmeister, director of product development in Synopsys’ solutions group; Donald Cramb, director of professional services at Eve; Patrick Sheridan, director of marketing at CoWare, and Scott Sandler, president of SpringSoft USA. What follows are excerpts of that conversation.

By Ed Sperling
SLD: There are now six or seven levels of abstraction in a leading-edge design. How many levels will we have in the future?
Schirrmeister: There is no definitive answer. When you went from gate level to RTL, which is the last major shift we did, at least you stayed within the constraint of hardware. You were either designing a block or the whole chip. IP re-use came to the rescue with making sure the automatic synthesis can stay focused on the block level, and then you do the rest with synthesis or re-use of the components. Now that we’re trying to go up another level, it’s completely impractical for a user to keep eight different models in sync. You need models for software that are fast but which don’t need all the architectural requirements. For the verification team, you may not need all the functionality but you need all the performance and architectural aspects. Bringing software into the mix has made things much more complex.
Sandler: There have always been a lot of different levels of abstraction, and if you look at what one person calls TLM (transaction-level modeling) it’s probably different than what another person calls TLM. Modeling is an art. There are certainly some standards, but there are also differences. As we automate more, these levels will get standardized. The notion of RTL used to be pretty loose 20 years ago. Synopsys absolutely codified what RTL meant. It was a very specific subset of the Verilog and VHDL languages with very specific semantics about how that subset was translated into hardware. That became the definition of RTL. But until we have automation and translation tools that work above the level of abstraction above the RTL, that will continue to float and be more artistic than scientific.
Cramb: For us, this is the key—how you move through the levels of abstraction for the entire methodology, whether it’s architectural, RTL, integral or functional. It’s how you mix and match these levels of abstraction, and how you move through them to get to the end point, which is a piece of silicon with software running on that silicon.

SLD: Can it be automated?
Schirrmeister: The automation may only work for subsets of this technology. We have reached a level of complexity where it’s probably not practical for high-level synthesis to cover a whole chip, which may be 50 blocks. Block development, which is IP synthesis, is either re-using IP or starting with a higher-level model to build your hardware accelerator. Then you have the integration between the blocks, where you have companies like Sonics and Arteris trying to automate that level. It never will be a high-level model of the complete thing where you push a button and out comes the whole thing.
Sandler: In 10 years or 20 years, we may be having a different discussion. But we can’t see that at the moment.

SLD: How are the chip makers dealing with this?
Schirrmeister: From what we hear from customers, there are a number of different requirements for bringing together hardware and software. They want to know when it is available. Speed is important. So is accuracy. Even within TLM there are loosely timed views, approximately timed views and then cycle-accurate models. Modeling needs ROI. If you have hardware and software involved, there is overhead. Then you need to debug it, control it and then ideally you want this model to point to the system environment. No model will fit all those requirements. This is where transaction-based verification comes in.
Sandler: The real renaissance is that you’re starting to see real system-level commercial design automation. That’s the leading edge. To be able to get IP that implements transaction-level models that interface with the ones you write yourself and to have some standards for the transaction-level modeling rather than writing it yourself is a big step. We’re just on the cusp of seeing these things get standardized and flushed out with version 2.0 and 3.0—that’s a big change.
Schirrmeister: We have some standards. SystemC helps with the assembly of systems. It also has a synthesizable subset. Obviously, the market determines what this synthesizable subset looks like and the acceptance of it. Standardization helps. But we’re at a stage where traditional hardware verification is undergoing very significant changes. When you go up to the next level you have the software in the system. Our customers tell us that more and more software running on the embedded processor in their designs becomes part of the verification regression suite. That’s tough because the guy who used to be the System Verilog expert for verification now needs to be able to bring up a Lauterbach debugger to be able to have the embedded software in the system. That is a fundamental change.

SLD: Are there enough people trained to do this?
Cramb: That’s a key problem—finding someone in a company who can move from wave forms to bringing up a Lauterbach debugger for software. People are beginning to verify software. Verification guys are in a strong comfort zone. There aren’t many people who can write complex test benches in Vera or e. It’s a very valuable skill, and it’s hard for them to move into a more general skill.
Sandler: There’s been a huge distraction in schools. They’ve been using the commercial tools. I’m not sure there are a lot of kids learning proper system-level verification techniques. That’s fairly esoteric stuff. So when you look at that shortage of people who know how to do these things, they have to be getting on-the-job training.
Sheridan: The people are there, but they’re software developers. They have to cultivate some interest to cross over. In our experience, the people who are quickest to pick up the system-level verification come from a software background. They can help their hardware colleagues, but teaming up together is a good combination.
Schirrmeister: When I first started using logic synthesis in the 1990s, there was a guy next to me who had never seen a schematic. I started with full custom layout and moved up through the abstractions. The guy next to me programmed, ‘If a variable is larger than 398 multiplied by another variable,’ he was surprised the logic synthesis tool wouldn’t work because it would run at 100KHz and he had a 20MHz target. If you don’t have the understanding of the underlying hardware, know how the schematic looks in the hardware world, know how the chip works, then you have big issues. It’s an education issue.

System Level Design sat down to discuss issues in system-level verification with Frank Schirrmeister, director of product development in Synopsys’ solutions group; Donald Cramb, director of professional services at Eve; Patrick Sheridan, director of marketing at CoWare, and Scott Sandler, president of SpringSoft USA. What follows are excerpts of that conversation.

By Ed Sperling

SLD: Verification has been the biggest time hog in chip design, and it has become a system-level problem at advanced process nodes. How are we going to resolve this?
Sheridan: You’re right that it isn’t just a chip-level problem anymore. It’s a system verification problem. We have to do things differently. In the context of multicore, it’s potentially multiple subsystems, multiple processors, multiple software stacks, integrating them and debugging both the hardware and software side.
Cramb: I think it’s the word ‘verification’ that gets people confused. There’s this thing called chip verification. But from a system-level, it’s validation. You bring the software and hardware together and validate the whole environment. You’re not just verifying against a spec. You’re validating how this thing is going to work for the end user
Sandler: Is it validation if it’s a whole system and verification if it’s a chip?
Schirrmeister: No, validation is against intent. Is it really what the user wanted? There are three elements to verification. One is the traditional hardware verification. The second is system validation, which is the chip in the context of the overall system. The third is software. It’s the whole aspect of software development and verification in the context of hardware. Whenever we had too much complexity in the past, we went up in levels of abstraction. The same will happen in verification. We are using transaction-based verification, and eventually we will have to figure out how to verify these higher-level models.

SLD: Is validation the same as debugging?
Sandler: There’s a general tendency to say you’re debugging when you’re verifying or validating. Ultimately what happens is when you validate or verify, things happen that you don’t understand. In my view, that’s the point at which you shift to debugging. You let the thing run, it does something you don’t understand, and then a human has to get involved. Debugging can only happen when someone is looking at some output or data.
Schirrmeister: Validation and verification can both be done if you express the design intent for validation and if you have a specification that you can verify against. Both can be done offline. You have a regression suite and it says, ‘It works.’ Debug is the last resort when this regression suite says, ‘It doesn’t work.’ Now you need to go in and debug. The hardware-software interface becomes critical there.
Cramb: The level of abstraction for verification is getting higher, but the same is true for debug.
Sandler: No matter what level you do the validation or verification, you have to be able to do the debug at that same level. The same things that affect simulators and models affect the data capture mechanisms you need to hook the person into the process and show people what is going on.

SLD: So what’s new here?
Cramb: For us the term is smart debug. You bring together not just wave forms and monitors, but you raise it to the right level of abstraction.
Sandler: Debug is the same thing we were doing years ago with the logic analyzer and the oscilloscope years ago on the bench. Then we went to wave forms and source views. When you do that at the next level up with transactions and C-level source code and you have the debugger linked in and synchronized, it’s just a progression.
Schirrmeister: It’s not the word ‘debug’ that’s in question. It’s the scope of the work. There’s IP debug, which is block-level, there’s chip debug and there’s system debug. The way we phrase things at Synopsys is system prototyping. You want to mix the different abstraction levels. You want to have virtual, including the software, connected to prototypes, which run in hardware-assisted environments. Together with system prototyping, you have system debug. You need to be able to correlate what happens in the software on one side with what happens on the hardware side. One of the things users are looking for is the ability to set an assertion in hardware, or set a break point in software, which is kind of the same thing if it’s conditional, and be able to hold the whole system. The scope really has changed since the complexity has increased.
Sheridan: When you’re validating, you’re still in the process of designing and making tradeoffs of features vs. performance vs. cost. You may not be at the point where you’re verifying whether it’s being implemented correctly. Those problems become more important to solve at the system level. Moving up in abstraction is the only way to make this work productive, whether you’re the designer of the system who is looking at architectural issues, or the people implementing hardware or software. They all have to come together.

SLD: In the past, verification occurred several steps down the line after the initial design. With increased complexity does it now need to move forward in the design flow?
Cramb: That’s validation.
Schirrmeister: It’s a thin line. Validation is against intent. On the architecture side, it’s validation.
Sandler: Since the beginning of architectural design, there’s been some intent specified. It has to run this fast, it has to be able to handle these kinds of problems, this is the input and this is the output. There’s always been a level of validation that had to be done. I don’t think there’s much change here. There’s more tools for this now and more commercialization.
Schirrmeister: Yes, it always was a requirement, but commercially it was not supported by tools. People always did it with C and C++ models. We tried to commercialize it, but it was too early. But over the past two years, the pressure has been growing to solve this issue. If you get your design out in time but you haven’t met all the architectural requirements for performance, your design will be very short-lived and your career as a project manager may be very short-lived, also. Pressure has increased over the last two or three years. We always wait when we make these transitions for projects to run into the wall over and over again. But people now realize they will have to switch to system-level design.
Sheridan: If you have a system design that’s multicore with multiple subsystems and you’re changing the camera piece from 3 megapixel to 8 megapixel, that’s more that just verification tests. It’s adjusting the challenges that impact performance of the entire product. So you have a system-level issue that’s on top of the verification of how you build the camera. The complexity is greater, and the impact of any changes is greater.

By Cheryl Ajluni

As with death and taxes, when it comes to design some things are just inevitable. For one, as design geometries shrink, design complexity will continue to increase. For another, verification is the single most time-consuming and intensive part of the entire design cycle.

While new tools and methodologies have enabled designers to work through many of the existing complexity issues to design innovative new electronic products, the ability to verify resulting designs has not followed suit. Despite ongoing efforts to improve verification technology so that it can keep pace with design creation, a verification bottleneck still exists. Intelligent verification or intelligent testbench—a concept that uses the design to automatically direct test generation—now offers design engineers real hope of a breakthrough in addressing this problem. But what exactly does intelligent verification mean and how will it benefit today’s designers? Let’s take a closer look.

Under the hood

Many factors are motivating interest in a new verification solution, including the need to increase verification confidence, track hard to reach design areas and eliminate test redundancy. Achieving confidence that a design is functionally correct typically requires the verification engineer to make an instinctual call that enough verification has been performed—an approach that often leaves the design vulnerable to bugs residing deep within its code structure and which will likely not be found until after fabrication. Increasing design size and complexity also plays a role here as it requires more testbench and assertion code to be written that can lead to redundant tests, and which itself requires verification.

Constrained random simulation methodologies offer a way to improve verification quality and time using hardware verification languages, but constrained random testing does not scale well to large designs. And because it bases test generation on external input rather than design structure, a disconnect exists between coverage metrics and test input, making it impossible to close the loop from coverage back into test creation.

Mark Olen, product manager of Mentor Graphics’ Advanced Functional Verification group, points to another issue with constrained random testing: “Verification toolsets focused on constrained random test generation promote their ability to generate more simulation sequences than ever before, covering more design functionality than even large verification teams can test with directed test sequences. But is quantity alone enough? Real advances in design productivity have not been based on just producing more raw gates, or more raw transistors, merely for quantity sake. They have come from doing things smarter. It’s time for verification to get smarter. Should we generate more test sequences? Sure, but let’s also make sure that each test sequence is useful and tests something new and important.”

This is exactly where intelligent verification comes in. As a form of functional verification, it can be used to verify that a design conforms to specification prior to fabrication. It performs this task using information derived from the design to automatically update the test description to target design functionality not verified, or already covered by existing tests. Algorithms automate the generation of simulation sequences, data and checks from a concise behavioral description of a design’s specifications.

As a supplement to constrained random simulation methodologies, intelligent verification uses existing logic simulation testbenches, along with the design, to automatically determine how to maximize coverage. It also provides the user with direction as to why certain coverage points were not detected. The benefits of this approach are significant. Intelligent verification does the following:

• Increases functional coverage at the module, sub-module and system level, and finds more bugs faster than traditional methods, in turn reducing overall testbench programming;

• Enables project teams to design with a higher level of confidence, realize improved design quality and dramatically minimize design respins;

• Avoids the unnecessary random wandering of constrained random test methods to find a bug. Instead, it creates new tests by automatically tracking efficient traces through the design structure to coverage points, and;

• Ensures that no simulation cycles are wasted verifying items that have already been tested.

Options Ahead

While there has been substantial research into intelligent verification, commercial tools that leverage this concept are just beginning to emerge. One solution comes from Nusym Technology. No specific tool name has yet been cited by the company, but it claims that unlike other approaches, which either treat the design as a “black box” or are impractical for use on large designs, its intelligent approach to verification uses insight into the design to automatically create “directed” tests and achieve rapid verification closure.

Another solution comes from Mentor Graphics. Its inFact intelligent testbench automation tool uses systematic algorithms to rapidly produce unique, non-redundant test cases (Figure 1). Users develop testbenches by specifying rules that are compiled into graphs. As Mark Olen explains, “inFact focuses on more verification per cycle, not just more cycles of verification, giving verification engineers the power to test much more functionality, with fewer test sequences. It can completely eliminate redundant sequence generation early in the verification process, achieving functional coverage over 10 times faster than constrained random testing. Later in the verification process, it can allow varying degrees of redundancy to create sequentially robust regression test suites.”

Figure 1. With inFact, intelligent algorithms are used to generate stimulus sequences, monitor the results and ensure generation of non-redundant sequences.

SpringSoft is another player in the intelligent verification market, although the company prefers to refer to it as “verification enhancement.” Recently, SpringSoft acquired Certess and the Certitude functional qualification system, which uses “mutation analysis” technology to inject artificial bugs (faults) into the design to check whether the verification environment catches them or not (Figure 2). When the verification environment does not catch an artificial bug, it’s an indication that the environment would likely miss a real bug in the same area of the design as the injected fault. According to Scott Sandler, president of SpringSoft USA, “At SpringSoft we don’t make a simulator; we make tools that enhance the simulation-based verification flow—such as our debug tools. Our recent acquisition of Certess fits directly into this category.”

Figure 2. The Certitude Functional Qualification System provides an overall assessment of verification quality, exposing a verification environment’s holes and weaknesses.

The Bottom Line

While intelligent verification offers a myriad benefits that will likely increase its rate of adoption, there are challenges ahead, not the least of which is that verification engineers may be reticent to change from what they’re already doing unless they are forced to. Regardless of when the change to intelligent verification comes, or even if this concept becomes commonplace in the electronics design industry, it is unlikely that verification will cease to be a top concern any time soon. As with death and taxes, it will continue to remain challenging to designers and design tool providers alike.