With complex protocols and frame structures, mobile wireless designs can be extremely difficult to verify. A functional verification environment must generate a large volume of randomized tests that achieve a high level of coverage. At Samsung Electronics, we recently achieved that goal by using SystemVerilog and the Verification Methodology Manual (VMM) to verify a broadband base station design based on the mobile WiMax (IEEE 802.16e) standard.

Since some portions of the design were reused from a previous design, we were able to compare the effectiveness of the VMM-based approach to a legacy Verilog RTL verification environment used two years previously. The VMM-based approach found bugs that were undetected in the legacy environment, providing a more complete verification. We were also able to achieve 94 to 97 percent functional coverage with the new approach.

Moreover, by using SystemVerilog and VMM, test generation was many times faster. For example, in the legacy environment, it was typical for two or three engineers to work two hours to generate one test case. Randomization capabilities in SystemVerilog and VMM allowed one engineer to run 1,000 test cases in a matter of hours. Finally, the new approach reused almost the entire C++ code from the legacy environment, along with some portions of the original Verilog testbench.

In this article, we describe a VMM-based verification environment consisting of C++ code as a golden reference model, a Verilog device under test (DUT), a SystemVerilog testbench, and a register abstraction layer format (RALF) model. This environment was able to verify the complex protocol associated with the DUT, contributing to the first-pass silicon success achieved by the system-on-chip (SoC) that contained the DUT.

A Challenging Design to Verify
The design under test (DUT) implements WiBro (Wireless Broadband), a mobile WiMax technology under development by the South Korean telecom industry. WiBro uses TDD for duplex operation and Orthogonal Frequency Division Multiple Access (OFDMA) for multiple access. With a channel bandwidth of 8.75 MHz, WiBro seeks to overcome the data rate limitations of current mobile phones and to add mobility to broadband Internet access. Base stations for WiBro devices will offer an aggregate data throughput of 30 to 50 MBits/second and cover a radius of 1-5 km.

The DUT is part of the base station of a wireless broadband mobile application. A modem block within a SoC, the DUT primarily consists of two sub-blocks – an Uplink PHY (UL) and a Downlink PHY (DL). The DUT comprises around 3 million logic gates and 1.25 million memory gates.

The UL and DL use Time Division Multiple Access (TDMA) burst transmission. The UL transmits data over a number of time slots. Each burst is designed to carry variable-length MAC Protocol Data Units (PDUs). Time slots can serve multiple uses such as registration, contention, or user traffic. The MAC layer controls these time slots and can vary them over time to optimize performance. The DL includes a transmission convergence sub-layer that inserts a pointer byte at the beginning of the payload to help the receiver identify the beginning of a MAC PDU.

OFDMA PHY user data is exchanged in frames. A frame consists of a DL sub-frame and a UL sub-frame. While a downlink sub-frame consists of only one PHY PDU, the sub-frame can include up to 64 DL bursts, each with its own modulation coding. Bursts can be organized in up to five “zones.” Each burst can include 96 MAC PDUs.

A UL sub-frame is identical in structure. DL verification, however, assumed only one burst per UL PHY PDU. UL verification was similar in that DL bursts were kept fixed. Since the UL and DL were verified separately, this practice helped reduce the complexity of the verification.

Due to the complex interrelationships between the data in the frames, all the constraints on these data needed to be solved together. A user would have to write a complicated mathematical equation to do that manually. Fortunately, the parallel constraint solver in the Synopsys VCS simulator was able to automate constraint solving for the DUT.

The Legacy Verification Environment
We used a legacy environment to verify previous generations of the DUT. This environment had two main components – a C++ golden reference model, and a conventional Verilog testbench. The reference model was created in Microsoft Visual Studio, and it had to be ported to a Unix environment for simulation.

The zero-delay reference model was developed and maintained by the software development team at Samsung. The model had sub-modules for the UL and DL blocks. The reference model read in a configuration file for the OFDMA PHY frame, and based on the configuration parameters, it output a text file containing a set of register and address values. The reference model would also output expected values of various operations in DL or UL frame transmissions.

Creating the configuration file was a multi-step, paper-and-pencil process. First, engineers defined a configuration scenario and established zones. Next, they defined a frame structure that divided bursts into zones. Finally, they defined configuration parameters. As shown in Figure 1, these parameters specified various properties of an individual burst as well as its relative position in the frame.

The RTL Verilog testbench included DUT initialization routines and monitoring blocks that collected the DUT response to a set of configuration values. The monitors output this response to a set of plain text files.

A multi-step process was needed to obtain simulation results. First, register files generated by the reference model were transferred to the Unix environment. Then the DUT was initialized with a given set of register values. The DUT was simulated for the desired time, and the monitored response was collected in text files. Finally, the Unix “diff” utility compared the simulation results with the output of the reference model.

This legacy verification environment had several drawbacks. For example:

• Creating even a single test vector was time-consuming and error prone because of the complex interrelationships and frame components, and the lack of randomization.
• With no functional coverage, it was difficult to judge how much verification still needed to be done.
• The heterogeneous setup was spread over two computing environments – Windows and Unix – causing portability issues and often requiring multiple engineers for a single simulation run.
• Verifying a bug in the reference model or DUT was tough because the testbench was not self-checking in nature.

How SystemVerilog and VMM Improved Verification
The verification environment that we set up using SystemVerilog and VMM resolved most of the challenges we experienced with the legacy environment. One benefit was significantly faster, and easier, test generation. As mentioned, the legacy environment frequently required two or three engineers working for several hours to generate one test case. This is because of the paper-and-pencil approach described above. In the VMM-based environment, we automatically generated randomized test vectors in as little as 10 seconds of CPU time.

Another significant benefit of the VMM-based environment was easier customization. With VMM, we were able to create a “plug and play” testbench environment. For example, we could run data collection and comparison separately. We could generate specific corner scenarios or collect functional coverage for particular test scenarios. Our engineers could generate new test cases without having to modify the whole testbench and disturb what was already working.

Reuse was another important feature. The VMM environment allowed the reuse of the pre-existing golden reference model, along with the initialization routine and the protocol-dependent monitoring logic. Finally, the new environment found bugs that were not detected in the legacy environment. These included bugs related to calculating the average channel estimation value, controlling the frequency offset estimation, and calculating co-variance values.

Finally, the new environment found bugs that were not detected in the legacy environment. These included bugs related to calculating the average channel estimation value, controlling the frequency offset estimation, and calculating co-variance values.

Figure 2 shows the SystemVerilog and VMM setup for the DL block. The UL block was verified using an identical setup.

Testbench Components
The verification environment used interfaces for connecting the DUT to the program block. This allowed easy logical grouping of related signals, and avoided races between the testbench and design.

The VMM environment class was a top-level testbench block that encapsulated sub-blocks such as the random configuration generator, monitor, scoreboard, and driver bus-functional model (BFM).

The configuration class consisted of elements for two main purposes – controlling simulation scenarios that included single or multiple OFDMA frame simulations and normal/RFIC mode simulations, and defining OFDMA frame parameters. Randomized parameter values were accessible by other testbench components, such as the monitor block or golden reference model. Monitors collected data from the bus or buffers of the DL and UL blocks.

Several VMM features were useful for the testbench environment. A set of vmm_notify events was provided at each step of simulation during monitoring. Different severity levels of vmm_log messages, along with notification events, greatly simplified protocol-level debugging and maintenance of monitoring logic.

The vmm_callback class was used to provide a rich set of callback functions. Callbacks made it possible to customize the verification environment without modifying the core monitoring or the driver logic.

Use of SystemVerilog DPI
The verification environment used the SystemVerilog Direct Programming Interface (DPI) for collecting data from the reference model. The C++ reference model was ported to the GNU C compiler in Unix from Microsoft Visual Studio, and dynamically linked into VCS simulation as a shared library. After that, by using the DPI, it was easy to access this code from within the VMM-based environment. The DPI thus helped avoid problems resulting from the use of two computer environments.

Solving Constraints
One of the most important features of this verification environment involved the use of constraints to generate a valid set of configurations for every simulation. Because the OFDMA frame structure is so complex, with critical timing relationships between the subsections of each frame, solving of constraints for random testing is a difficult task. A typical, randomly-generated configuration vector consisted of parameters arranged in the following groups:

• Group 1 – parameters for every DL zone including type of zone, and start and end symbol coordinates of the zone.
• Group 2 – parameters for every UL zone
• Group 3 – burst parameters such as number of slots (sub-channels) per burst, number of sub-channels, and number of symbols
• Group 4 – PDU parameters

These configuration parameters all had a unidirectional dependency on each other. There were also interdependencies between parameters of individual components. For these reasons, randomizing all parameters posed a tough mathematical challenge.

To solve this, we developed the strategy of solving only related parameters together. This scheme took advantage of the parallel constraint solver inside VCS. For example, PDU parameters depended on Group 3 parameters, and Group 3 parameters depended on Group 2 parameters, but Group 1 parameters did not depend on any other variable in the configuration.

In terms of SystemVerilog syntax, this meant that randomizing complex arrays of related objects was unavoidable.

We used a selective, step-by-step randomization approach (see Figure 3). We first randomized only zone coordinates and sub-channel types. Next, we randomized all bursts. Finally, using procedural calculations based on random burst values, we randomized PDUs. This approach reduced mathematical complexity in the constraints, and allowed randomization at each step to be done quickly.

Register Abstraction Layer
The DUT contained a large register set, and there were periodic changes to the register set during the verification. The VMM register abstraction layer (RAL) and RAL Format (RALF) allowed for such modifications with minimal effort. A RALF file (Figure 4) contains a detailed specification of the registers and memories in a design.

RALF let us specify the DUT’s register set using symbolic names instead of fixed addresses. It permitted hierarchy and instantiation, and established access rights for protection. Once the DUT’s register set was described in RALF, the VMM ralgen tool was used to convert the RALF description into SystemVerilog classes and methods.

With this level of abstraction for doing read or write operations, the verification engineers did not have to deal with signal-level access to registers.

Functional Coverage Model
In the VMM-based environment, we relied on functional coverage to provide easily measurable metrics of our verification process, and to identify the untested parts of the design. The functional coverage model evaluated the quality of the random generator and was thus an integral part of the configuration class definition. We achieved 94% functional coverage for the DL and 97% functional coverage for the UL.

In addition to cumulative coverage, SystemVerilog allows the use of “instance-based” coverage. For example, in the DL block, the instance-based covergroup allowed us to collect coverage on properties of individual zone, burst and PDU elements. To gauge coverage on burst properties for every zone, and for coverage of PDU properties on every burst, cross-coverage was used. We eliminated some redundant or uninteresting combinations with the ignore_bins construct.

The initial verification effort focused on bringing the verification environment and the DUT to a stable version. After “sanity test cases” started passing, we set up a complete random simulation for regression. These random simulations showed that several parameters of the burst configuration were not completely covered. The configuration generator was enhanced to cover these missing combinations, and subsequently, several corner-case bugs were caught.

A Self-Checking Testbench
As noted earlier, the legacy verification environment was not self-checking, making it hard to verify bugs in the reference model or DUT. The VMM-based environment was a closed-loop system that simulated the design as follows:

1. Generate the configuration (frame) randomly in VMM
2. Execute the C++ model to read in the configuration (frame) and obtain:

              2a. DUT register data
              2b. Golden response for the given configuration

3. Write register data to DUT using VMM
4. Collect DUT response from VMM environment
5. Compare 2b and 4 in VMM using scoreboard

A bug in the random generation, reference model output, or DUT response would cause step 5 to fail, resulting in a self-checking environment.

Challenges and Workarounds With Our Approach
With the VMM-based approach, we faced challenges in two areas – randomizing large arrays of complex data objects, and constraint solver diagnostics.

The randomly generated configuration object inherently consisted of large arrays of complex class objects. An initial attempt to randomize all such objects simultaneously failed. To overcome this challenge, the constraint model had to be constructed such that the constraint partitioning was carefully implemented. For example, since the zone class variables did not depend on PDU class variables, the “zone” object array was randomized first. This resulted in values that were well distributed over “solution space” and faster simulations.

While declaring the variables, an informed choice of SystemVerilog data types was made. For example, a bit vector of desired length was chosen over a 32-bit integer data type. This further reduced the time to solve the constraint problem. Also, the use of complex arithmetic operators was reduced as much as possible.

The constraint solver log and the failure reports required a close examination and in-depth understanding of the solver algorithm. Better debugging aids for constraint analysis and cleaner reports may have allowed a better understanding of randomization.

Conclusion
The DUT described here for a wireless broadband application was complex, and verifying it was challenging. The complex protocols reflected in the nested OFDMA frame structure demanded a strong randomization environment. Another key challenge was the extensive use of registers.

In a comparison to a legacy verification environment, we have shown how a SystemVerilog, VMM-based environment was able to dramatically speed test generation times, provide sufficient randomization to test the DUT protocols, allow easy customization of the verification environment, permit register verification at a high level of abstraction, and achieve high functional coverage. These improvements occurred even though much of the legacy verification environment was reused.

The SoC containing the DUT described here achieved first-pass silicon success, and the chip is already being sampled to engineers both in and outside Samsung. The success we had with verification has thus turned into a business success. Moreover, due to the work described here, the verification environment for our next design is already in place.