System Verilog Functional Coverage

A Unique Functional Coverage Flow using SystemVerilog
and NTB

Richard Raimi
ARM, Inc
rraimi@arm.com

Dennis Strouphauer
Synopsys, Inc
dstroup@synopsys.com

ABSTRACT

There are two constructs in SystemVerilog that support functional coverage: the covered
property and the covergroup. At ARM, Inc. in Austin, Texas, we wanted to use SystemVerilog
functional coverage on a next generation processor design; however, only the covered property
and not the covergroup was going to be supported by Synopsys in the time frame of our project.
Feeling that both were needed, we chose to write our functional coverage in SystemVerilog,
using both the covered property and the covergroup, and translate the SystemVerilog
covergroups to NTB / Vera ‘coverage_groups’, a similar, but not identical, construct. In this
way, we were able to train our team in SystemVerilog, use the code we wrote in the present, and
allow ourselves the option of reusing it in the future, when the covergroup would be fully
supported in VCS. In this paper, we describe this translation process, as well as the pros and
cons of functional coverage in both NTB / Vera and SystemVerilog.

Table of Contents

1.0 Introduction: Functional Coverage ....................................................................................3
2.0 Functional Coverage Constructs in SystemVerilog and Vera / NTB ................................3
2.1 NTB coverage_groups .......................................................................................................8
2.2 Bind Directives ..................................................................................................................9
3.0 SystemVerilog for Functional Coverage at ARM-Austin ...............................................10
3.1 Relating Design Components to Coverage Components.................................................11
3.2 Code Writing Standards...................................................................................................11
4.0 Translating SystemVerilog to NTB : The 'covermap' Translator ....................................12
5.0 Results and Performance..................................................................................................15

Table of Figures

Figure 1 : SVA Sequence...............................................................................................................4
Figure 2 : SVA Properties..............................................................................................................4
Figure 3 : SVA Assert and Cover ..................................................................................................5
Figure 4 : SVA Covergroup...........................................................................................................6
Figure 5 : SVA Covergroup Advanced..........................................................................................7
Figure 6 : NTB (Vera) coverage_group example ..........................................................................8
Figure 7 : SVA Bind ....................................................................................................................10
Figure 8 : Flow Diagram..............................................................................................................13

SNUG Boston 2005 2 A Unique Functional Coverage Flow

1.0 Introduction: Functional Coverage
Functional Coverage provides an indication of the thoroughness of the implementation of the
verification plan and can help answer questions such as how much of the design specification has
been exercised? and what actually happened during a test? As chip designs grow larger and
more complex with thousands of possible states and transitions, a comprehensive verification
environment must be created that minimizes wasted effort. Functional coverage is used as a
guide to direct the verification resources by identifying the tested and untested parts of the
design. When properly used, functional coverage can serve as a formal specification of a test
plan. In turn, the process of writing functional coverage can bring holes in a test plan to the
attention of verification personnel.

Functional coverage, because it is a measure of what is determined to be of interest and relevant,
must be manually specified by members of the design and verification team. It cannot be
extracted automatically from the design source code. This paper describes the use of the
SystemVerilog language to create functional coverage models for an ARM processor.

2.0 Functional Coverage Constructs in SystemVerilog and Vera / NTB
The constructs in SystemVerilog that are related to functional coverage are:

1. sequences and properties
2. covered properties and assertions
3. covergroups

The sequence construct is a basic building block. A sequence is a description of temporal
behavior using logical and temporal operators of the SystemVerilog Assertion (SVA) language.
SVA is a subset of SystemVerilog, useful for describing behaviors that unfold over multiple time
steps.

In its simplest form a sequence can be just a Boolean expression, which is considered to be a
sequence of zero time steps. It is optional to define what constitutes a time step in a sequence.
A time step may be thought of as a directive indicating when to sample variables appearing in
the sequence. The ability to write a sequence without defining the meaning of a time step allows
reuse of a sequence in many different contexts, each with different definitions of time steps.
Labeling a sequence also facilitates reuse, as we will see below. A sequence has no defined truth
value, but there is a concept of a sequence being matched, meaning that the behavior of a design
unfolds as described in the sequence. Here, we also describe this as the sequence succeeding, or
successfully concluding. Figure 1 shows an example of a sequence describing a signal, req,
going high, and one time step later, a signal, ack, also going high.


sequence s_1;
req ##1 ack
endsequence

Figure 1 : SVA Sequence

A sequence can be reused, via its label, inside of a property. The body of a property is made up
of one or more sequences operated on by temporal operators or logic operators. The sequences
can be written explicitly in the property, meaning, one can form expressions with variables and
logic and temporal operators, or one can reuse sequences that are defined elsewhere via their
label. (At ARM, we always do the latter.) A property will have a time step, or sampling
definition. This is done using Verilog syntax, for example “@(posedge clk)”. If a sequence is
reused by referencing its label inside of a property, the sequence will take on the time step
definition of the property. Figure 2 shows how a sequence can be reused inside a property: we
show sequence s_1 from Figure 1 being reused inside properties p_1 and p_2. Note that
evaluation of a property can be subject to constraints. Both properties in Figure 2 will only be
evaluated if the signal, reset, is low.

property p_1;
@(posedge clk) disable iff (reset) s_1;
endproperty

property p_2;
@(posedge clk) disable iff (reset) s_1 |-> done;
endproperty

Figure 2 : SVA Properties

Unlike a sequence, a SystemVerilog property does have a Boolean truth value. At its simplest, if
the body of a property is just a Boolean expression, the property is true on every time step in
which the expression is true. In more complicated properties, the property is true, at a given
sample point, when the mix of temporal and logical operators within it evaluates to true. For
example, if s is the label of a sequence, and o is a variable representing a design signal, then the
expression
s |-> o


in the body of a property, where |-> is the implication operator, is true whenever:
• s has not successfully concluded, or
• s has successfully concluded, and at that same instant, signal o is high
As an aside, the fact that the implication operator evaluates to true when its left hand argument,
known as the antecedent, evaluates to false, is often confusing to people. This is called “vacuous
truth”, and must be taken into account when setting up a functional coverage and assertion based
verification system. For example, property p_2 in Figure 2 will be vacuously true at every
sample point at which sequence s_1 does not match. It would be desirable to have an
independent means of verifying that the property also was true a finite number of time when
sequence s_1 did match. Counting how often property p_1 is true accomplishes this.

Properties can be labeled, and, like sequences, they can be reused in a higher level context by
their label. In particular, properties can be the target of the SystemVerilog cover and assert
directives. When a cover directive is applied to a property, a count is generated of how often the
property is true at its sample intervals. This is known as a covered property. When an assert
directive is applied to a property, an error is signaled if the property is ever false at a sample
point. This is known as an assertion. Note that this allows reuse of sequences for both
functional coverage and assertions. Figure 3 shows cover and assert directives applied to the
properties of Figure 2, p_1 and p_2. Note that applying both these directives at the same time
solves the problem of vacuous truth for property p_2. If property p_2 is forever vacuously true,
the cover directive, c_1, will have a zero count. If, on the other hand, property p_2, and,
therefore, assertion a_2 is actually stressed, cover c_1 will have a positive, non-zero count, to the
extent that that occurs.

c_1: cover property (p_1);

a_2: assert property (p_2)

Figure 3 : SVA Assert and Cover

SystemVerilog covergroups, in contrast to sequences and properties, are primarily targeted
towards combinational logic. They are not part of the SVA subset of SystemVerilog.
SystemVerilog covergroups are closely related to the Vera ‘coverage_group’ construct that
preceded it. A covergroup will have a sampling, or time step definition, using the same syntax
as we saw previously with properties (e.g., @(posedge clk)).

The two main components of a covergroup are the coverpoint and the cross. A coverpoint is
defined relative to a signal of a particular width. The default behavior of a coverpoint is to
generate a counter for each possible permutation of the underlying signal’s values, up to some
preset limit. Thus, for example, a 3-bit wide signal would generate 8 counters, one for each


possible 0/1 permutation. These counters are referred to as bins. A cross generates counters for
the cross product of two or more coverpoints, meaning, counters for the occurrence of all
possible simultaneous hits to the bins of the underlying coverpoints. For example, if one
coverpoint is related to a 2-bit signal and another to a 3-bit signal, then the default behavior is to
create 4 bins for the first coverpoint and 8 for the second. The cross of both coverpoints would,
by default, create 32 distinct bins, one for each possible, simultaneous update to the bins of the
coverpoints. Figure 4 shows a covergroup like this, where coverpoints cp_1 and cp_2 are
defined from 2 and 3 bit signals, respectively, and a cross of these two coverpoints, cr_1,
generates 32 bins.

reg [1:0] two_bit_sig;
reg [2:0] three_bit_sig;

covergroup cg_1 @(posedge clk);
cp_1 : coverpoint two_bit_sig;
cp_2 : coverpoint three_bit_sig;
cr_1 : cross cp_1,cp_2;
endgroup

Figure 4 : SVA Covergroup

If one does not constrain the generation of bins, a combinatorial explosion could quickly result.
Fortunately, SystemVerilog has a great many ways of specifying constraints. One can put guard
conditions on coverpoints and crosses, so that they are updated only when the guards hold. One
can apply guards more finely, putting them directly on specific, user-defined bins. One can
restrict the range of values of a bin via user-defined bins, and one can label particular ranges as
ignorable or illegal. The illegal designation acts much like an assertion, in that it causes the
simulator to post an error if a value in the specified range is hit. In general, constraining
covergroups involves some method of not allowing SystemVerilog to automatically generate the
number of bins it would otherwise generate by default. This often involves creating bins
explicitly. The bins keyword is used to create explicit bins on either coverpoints or crosses, as is
shown in Figure 5, below. Note that the square bracket operator, ‘[]’, will generate individual
bins for each number in its range. Thus, the definition of bins c of coverpoint cp_1, Figure 5,
will generate 3 bins, one each for the values 200, 201 and 202. In contrast, the definition of bins
a of coverpoint cp_1 will generate only a single bin, for all of the 65 distinct values in its range.
With bins a, it will be impossible to know how many times the individual values in the range
were hit. If this information is needed, the ‘[]’ operator must be used.


reg [10:0] v_a;
reg [1:0] v_b;

covergroup cg_2 @(posedge clk);
cp_1: coverpoint v_a {
bins a = { [0:63],65 };
bins b[] = { [127:150], [148:191] };
bins c[] = { 200, 201, 202 } iff (!reset) ;
bins d = { [1000:2000] };
illegal_bins bad_vals = { [2001: 2047] }; }
cp_2: coverpoint v_b;
cr_1 : cross cp_1, cp_2 {
bins b_cr = binsof(cp_1.b) && binsof(cp_2); }
endgroup

Figure 5 : SVA Covergroup Advanced

Bins definitions for crosses are more complicated than those for coverpoints. This is because, in
a cross, one is crossing all combinations of defined bins for the coverpoints, and one needs a
mechanism to choose these combinations. SystemVerilog has a binsof operator for this purpose.
The argument of the binsof operator can be either a coverpoint, in which case all defined bins for
that coverpoint are meant, or a coverpoint bin, in which case only that bin is selected. One can
use a limited set of logic operators on binsof expressions, to further refine one’s selections. To
explain this, consider the following cross bins definition shown in figure 5:

• The cross is labeled cr_1
• The cross operates on coverpoints cp_1 and cp_2
• The cross has a single, user defined bin, b_cr
• The first binsof operator selects a single bin, b, within the coverpoint cp_1. Other
bins defined for cp_1 are ignored.
• The second binsof operator selects all bins of coverpoint cp_2. This coverpoint will
have 4 automatically generated bins, since it is 2 bits wide and the user had not defined
any explicit bins for it.
• The && operator will cause the cross bin, b_cr, to be updated only when the specific
bin b in coverpoint cp_1 is hit, and a valid bin (any) within cp_2 is also hit.


2.1 NTB coverage_groups

In figure 6, we show how one would write an NTB coverage_group that has the same semantics,
but slightly different syntax, as the SystemVerilog covergroup of Figure 5. Note that the NTB
sample is roughly equivalent to the SystemVerilog coverpoint. In addition, we show an NTB
interface block, which is the means we used to connect to design signals in NTB. The NTB
m_state construct, used in the cp_1 sample point, has a similar role to the SystemVerilog []
operator, in that it creates an individual bin for every value in its range.

interface cg_if {
input clk CLOCK hdl_node "`design_module.clk";
input reset INPUT_EDGE INPUT_SKEW hdl_node "`DESIGN_PATH.reset";
input [10:0] v_a INPUT_EDGE INPUT_SKEW hdl_node "`DESIGN_PATH.va";
input [1:0] v_b INPUT_EDGE INPUT_SKEW hdl_node "`DESIGN_PATH.vb";
}

coverage_group cg_2 {
sample_event = @(posedge cg_if.clk);
sample cp_1 (ex_sysvlog_if.v_a)
{
state a (0:63,65);
m_state b (127:150], [148:191]);
m_state c (200:202) if (!cg_if.reset );
state d (1000:2000);
bad_state bad_vals (2001:2047);
}
sample cp_2 (cg_if.v_b);
cross cr_1 (cp_1,cp_2)
{
state b_cr ( (( (cp_1 >= 127) && (cp_1 <= 191) ) ) && (( (cp_2 >= 0) && (cp_2 <= 3) ) ));
}
}

Figure 6 : NTB (Vera) coverage_group example


2.2 Bind Directives

Sampling of signals in SystemVerilog is made simple by the bind directive. Bind directives
cause one module to be instantiated within another, without having to explicitly alter the code of
either. This facilitates a strict separation between a design’s implementation code and its
verification code, whether the latter is code for functional coverage, for testbench modeling, or
any other purpose.

The first three fields of the bind directive are as follows:

1. The name of the module or instance that is the target of the binding
2. The name of the module that is being bound to the target module, and
3. The instance name of the module being bound

After these three fields there follows a port list, with the same syntax as for a normal, Verilog
module instantiation. Figure 7 illustrates this. There, we show a module with functional
coverage code, cover_mod. It contains a single covergroup, cg_sig, which uses signals local to
cover_mod. But, these signals are actually connected to those of a design module, design_mod,
which is not shown. A bind directive (note that it is defined outside of module scope) makes this
connection. The signal names in the port list are those local to the design module. They are
connected to the inputs of the coverage module, cover_mod, in order of appearance. Similar to
the rules for traditional Verilog instantiations, we also could have used dot operator connections
in the port list. These are safer, being less error prone, but for brevity, we chose to not to show
them.

When the bind directive is executed during compile time, the bound module is instantiated
directly below the target module or instance. Since the target may be named by its module or
instance name, there are really two types of bindings. If module binding is used, an instance of
the bound module is placed below each instance of the target module. If instance binding is
used, an instance of the bound module is placed below only one, particular instance of the target
module. Usually, for functional coverage, there is no need for module binding, as coverage is
generally written for modules that are rather high up in the hierarchy and have only one instance.

Signals in the target instance are bound to inputs in the bound module through the bind port list.
Thus, the bound module has access to any and all signals in the scope of the target instance,
simply by putting them in the port list, which makes sampling of design signals for functional
coverage rather easy. The hierarchical path to the coverage module instance also makes clear
which parts of the design are being covered, because the design instance whose signals are being
sampled is placed immediately above the coverage module instance in the hierarchy. Thus, if
one adopts a naming convention that makes it easy to recognize coverage modules, one can
know which design modules they are sampling simply by reading the hierarchical path name of
the coverage module.


module cover_mod (clk, reset, enable, sig1, sig2, sig3);
input clk;
input reset;
input enable;
input [1:0] sig1;
input [2;0] sig2;
input [3:0] sig3;

covergroup cg_sig @(posedge clk)
cp_1 : coverpoint sig1 iff (!reset && enable);
cr_1 : cross sig1,sig2,sig3;
endgroup
endmodule

bind design_mod cover_mod u_cover_mod (d_clk, d_rst, d_en, d_sig1, d_sig2, d_sig3);

Figure 7 : SVA Bind

3.0 SystemVerilog for Functional Coverage at ARM-Austin
At ARM in Austin, we debated several choices for a functional coverage vehicle for our current
project, the design of a next generation, high performance ARM core. In the end, we decided on
SystemVerilog as implemented in Synopsys’ VCS. We were motivated by the fact that
SystemVerilog was an emerging standard that would eventually have wide vendor support, and
that it was being integrated at no extra cost into VCS. We also liked the fact that, via its
sequence and property constructs, with the associated cover and assert directives, one could
reuse SystemVerilog functional coverage code for assertions, and visa-versa. In addition,
SystemVerilog supported coverage of permutations of values of bit-vectors, through its
covergroup construct. This was extremely important to us, and, it turns out, the overwhelming
majority of our coverage bins ended up coming from covergroups. None of the other, competing
functional coverage vehicles had the same degree of support for all these aspects of functional
coverage. SystemVerilog stood out in this regard.

However, we had to deal with the problem that SystemVerilog support was being put into VCS
incrementally, and not all coverage constructs we wanted to use were available yet. The most
important one that was missing was the covergroup. However, at the suggestion of Synopsys,
we decided this could be overcome via a translator. Synopsys was building up support for NTB,
the natively compiled version of Vera now being bundled with VCS, at the same time as it was
building up support for SystemVerilog,. The NTB / Vera coverage_group construct was very


similar to that of the SystemVerilog covergroup and was immediately available in VCS. We
decided translating between the two was feasible. We created a translator and called it
covermap. With covermap, we were able to train our team in the use of SystemVerilog in the
present, while building up a body of code that could be reused in the future, when SystemVerilog
would be fully supported. In the meantime, covermap would insure (hopefully) that our
functional coverage code executed with the expected functionality of SystemVerilog.

3.1 Relating Design Components to Coverage Components
As part of our functional coverage methodology, we mandated that:
• Functional coverage code be written inside of standard, Verilog modules, as opposed
to newer SystemVerilog constructs such as program blocks or classes. This allowed our
hierarchy to be composed of similar structures, and we did not have a strong need for the
other structures, as they were more targeted towards testbenches and we were using an all
Verilog testbench.
• Bind directives be used to create a one-to-one relationship between a coverage
module and a design module. This facilitated our post-processing tools relating the
sampled modules to the modules doing the sampling.
• Only standard Verilog reg and wire variables be used in coverage modules. While
other constructs were available in SystemVerilog and had some advantages, this kept the
coverage modules as similar, in coding style, to the design modules as possible.
• Local variables be allowed in coverage modules, but bracketed between formatted
comments. This allowed the covermap translator to recognize them easily. The local
variables were used to form Boolean expressions, or, quite often, complicated state
machines from sampled, design variables. This, in turn, simplified the task of functional
coverage. For example, bus signals were used to create state machines that kept track of
transaction histories, and, in turn, functional coverage was written in terms of these
histories.
In our configuration management structure there is a directory for each distinct unit of our
design. We made a subdirectory under each such unit directory, for functional coverage files.
We initiated a naming convention where the file names for functional coverage files reused the
names of the design modules they were covering, with an added “_sysvlog.v” suffix. We created
a small Perl script that found all such files with these suffixes, whenever we compiled a
simulation model with functional coverage, and put them in a file list for the VCS compiler.

3.2 Code Writing Standards

For covered properties, we mandated that all coverage code writers would use the sequence
construct to describe temporal behavior, and that all sequences would be labeled for reuse. We
then created covered properties by labeling the property, and applying a cover directive to that
label. We created assertions from sequences in a similar manner, using the assert directive.
Within assertions, a sequence would typically occur as the antecedent of an implication.
Assertions and covered properties work synergistically, in that the covered property measures
how often the design has been stressed, while the assertion monitors for correct behavior
whenever the stress is applied. As stated above, “vacuous truth” is a problem in a verification
system. Using our methodology, if an assertion never fired (which is desired) one could easily


ascertain, from looking at hit total for the related covered property, whether that was because it
was never stressed, i.e., whether its left hand side, the antecedent, had never been true.

For covergroups, we supported most of the constructs that are in the current SystemVerilog 3.1a
LRM; except for transition coverage (no one felt a strong need for it).

For all coverage constructs, we mandated a naming convention. For example, covered property
names would begin with “c_”, covergroups with “cg_”, coverpoints with “cp_”, etc. This
facilitated easy recognition of an item’s type in our output, when a coverage bin was listed using
its full hierarchical path. In that case, with these conventions it was easy to see that it was, for
example, a bin within a cross within a covergroup, and to know which design module’s signals
were being sampled.

We also utilized a priority labeling system, with each covered property and each coverage bin
allowed to have a separate priority. We allowed three levels:
1. L1. Meaning, important enough that it must be hit in top level simulations
2. L2. Meaning, desirable to hit at top level, but acceptable if hit only in unit level
simulation (where it is easier to generate specific input sequences)
3. L3: Meaning, desirable to hit at any level, but not required
Use of anything but L1 coverage was discouraged, and had to be justified. L3 coverage was
used only rarely, and had to be rigorously justified. Usually, it was used as a temporary holding
place for coverage points that were new and not thoroughly tested.

We implemented the priority labeling system through formatted comments containing a coverage
item’s name and its priority level. The covermap translator then applied that label to that item
and to all items below it in the coverage hierarchy, unless the label was contradicted by a
specific directive found for an item lower down. The highest element in the coverage hierarchy
for any Verilog module was the module itself, the next highest were individual covered
properties and covergroups, and, within a covergroup, coverpoints and crosses were next highest,
with the lowest level being the bins of coverpoints and crosses.

4.0 Translating SystemVerilog to NTB : The 'covermap' Translator
The main task of the covermap translator is to read in SystemVerilog files, find covergroup
constructs, and translate these to NTB coverage_group constructs. But, it performs many other
useful tasks, as well, for example enforcing our priority labeling system described above. The
translator would signal an error if a coverage item did not have an associated priority label.

Each file read in by covermap is expected to contain a single, Verilog module definition, and one
or more SystemVerilog bind directives. The Verilog coverage module may contain one or more
of the following:

• SVA sequence and property definitions
• SVA covered property and assertion declarations
• SystemVerilog covergroup definitions and instantiations


• Local variable definitions, and Verilog code (including tasks and functions) utilizing
these

When covermap translates ‘n’ SystemVerilog input files, it will produce 2n + 1 output files, of
the following type:

• ‘n’ files with all the original SVA covered properties and assertions, and with all Verilog
code utilizing local variables, but with covergroups and their instantiations commented
out
• ‘n’ NTB files holding NTB coverage_groups derived from the original SystemVerilog
covergroups
• A “master” NTB file holding a program block

NTB begins execution in a program block, a construct somewhat similar to the ‘main’ function
in C. There must be at most only one such block in any body of NTB code. All the NTB
coverage_groups must be instantiated in this program block, even if they are defined in separate
files. covermap facilitates this by generating include directives within the master NTB file for
all the separate NTB files.

The following diagram illustrates the functional coverage flow:

Test Stimulus
Original SV Files Verilog Design
Files Files

NTB Files Compile and
Sim Scripts
covermap
translator
SV Files
NTB SVA

DB Init DB Post processing
File Viewer Programs

Figure 8 : Flow Diagram


In this flow, we start with hand coded SystemVerilog files. As covermap reads in these files, it
stores information in a set of C++ classes that give it a picture of the coverage hierarchy. It then
traverses this hierarchy down to individual coverage bins and prints out a list of all these bins in
a database initialization file (“DB Init File”). This file is used to initialize an SQL database used
by an HTML viewer for coverage results. Along with the hit counts for each bin, the database
stores a reference to the test names that hit it, and Perl scripts operate on the database to generate
a minimized suite of tests that can hit all the coverage points. In addition to using this SQL
database and HTML viewer for functional coverage, we track pass / fail numbers on regression
suites and track progress of the debug effort, with it, as well. One of the most important reasons
for creating our own means of post processing functional coverage results was to integrate those
results with this type of information coming from our simulations, so that, within one viewing
tool, we could track all pertinent aspects of our project’s progress.

In the normal translation flow, the original SystemVerilog files are translated into 2 types of
files: (1) NTB files holding coverage_groups, and (2) SystemVerilog files holding all the
original code except that related to covergroups. When the Verilog from the design is compiled
for simulation, these files are compiled in as well.

As covermap translates, it changes names of coverage items in order to transmit information on
design hierarchy into NTB data. Since NTB coverage_groups are instantiated in the NTB
program block, which is in the testbench, no information is readily available in output files as to
the design blocks being sampled by a coverage_group. covermap is able to recreate the
hierarchical path that would have existed had the NTB coverage_group been a SystemVerilog
covergroup. It encodes this information in a new name for the NTB coverage_group. In turn,
post processing scripts are able to parse this name, and show the user where the coverage items
are in relation to the design hierarchy.

As test stimulus is loaded and simulations are run, two different types of coverage databases are
formed: SVA and NTB. The SVA database holds hit counts for covered properties, while the
NTB database holds the same for the bins created by NTB coverage_groups. Special scripts and
programs then read these databases, and reformat the data to make it suitable for the HTM
database viewer (“DB Viewer”). This requires anticipating the names VCS / NTB will apply to
automatically generated coverage_group bins.

When the coverage files are translated, the resulting set of NTB and SVA files are compiled with
the code for the design and testbench. During covermap’s translation, the user may obtain two
types of optional output:
• statistics on the distribution of properties and covergroups through the design hierarchy,
as well as a breakdown within the covergroups on the quantities of bins in each member
coverpoint or cross, and
• an initialization file for loading our SQL coverage database, listing the hierarchical path
to each coverage bin of all the covergroup coverpoints and crosses, as well as the
hierarchical path to each covered property.
Great care is taken that the names of bins coverpoint and cross bins correspond to what will be
generated after simulation. For instance, NTB has certain naming conventions for auto-
generated bins, and certain formatting requirements for fields within names. We take care to


produce names consistent with those in our SQL initialization file, because we will use string
matching to compare names of bins hit in simulation to those in the initialization file.

After simulations have concluded, functional coverage data exists in two separate sets of files,
one for NTB coverage_groups and one for SVA covered properties. We then transform this data
for our SQL / HTML viewer. In our post process scripts, we label it as L1, L2 and L3 priority
level. The covermap translator enables this by prepending the priority level to the translated
name of the coverage item. We run scripts that use the Tcl SVA API (application programming
interface) to extract the covered properties that were hit, and the number of hits for each. We run
a C program that utilizes Synopsys new UCAPI (Unified Coverage API) library, to read out the
same information for the NTB coverage_group members. The UCAPI is a relatively new library
intended to be a single interface through which all the various types of coverage formats in use
by Synopsys may be read.

5.0 Results and Performance
At this point, we are still in the midst of our design project, and while we have written most of
our intended coverage code, we have not written all of it. At the unit level, our functional
coverage is well above 90%, and it is approaching that for top level.

One important issue is the cost of functional coverage, i.e., how much it reduces simulation
performance. Of course, this will be in proportion to the size of the functional coverage code
base relative to the design code. There is no universally accepted way to measure the size of a
body of functional coverage code. We feel the most appropriate way is to count coverage bins,
in other words, to determine how many separate events are being monitored. When we expand
our code, we currently generate around 100,000 coverage bins; but, among these there can be a
wide variance in simulation time. A simple coverpoint bin, where the coverpoint hasn’t any
complex sampling guards, would be very quick to simulate, since it can be implemented with
efficient, integer operations. A covered property (which can be considered to implement a “bin”,
if we understand that to mean a counter of how often the property was hit), on the other hand,
could be very costly to simulate, since it may describe a very complex sequence of events, the
tracking of which requires a large overhead. However, the vast majority of our coverage bins
come from covergroups.

Right now, we are experiencing slowdowns of 300% to 450% in top level simulations with
functional coverage. In other words, if a simulation takes, say, 100 CPU seconds to simulate,
using a simulation model compiled without functional coverage code, it will take between 300 to
450 CPU seconds to simulate when our functional coverage code is compiled in. This is just the
simulation time, and does not include any post processing time for analyzing results. When
considering these numbers, it should be borne in mind that the testbench for our full design (as
opposed to our unit level testbenches) is an all Verilog testbench, using only a very few PLI
calls. For many users of functional coverage, they will be adding functional coverage to the
already existing overhead of a testbench in a specialized language, such as NTB / Vera. In this
case, the effect of functional coverage may not seem as large. Ours is a rather unusual case,
allowing one to see the overhead of functional coverage relative to a rather pure body of
traditional Verilog code (note that in our design and testbench we are not using SystemVerilog


constructs, only in our functional coverage files, and much of our testbench is written in
synthesizable Verilog).

We have attempted to analyze these slowdowns, with help from Synopsys R&D teams. One
major aspect, which cannot be resolved quickly, is that much of the work VCS does to analyze
functional coverage is still being done through function calls into libraries. While this is quite
normal, it obviously would be better if the function call overhead could be eliminated. VCS gets
much of its speedup on regular Verilog code from transforming it into natively compiled
constructs. Synopsys plans to eventually do all functional coverage work in a native compile as
well, and this is where the largest performance gains are expected. Until then, performance
improvements must come from incremental sources.

One workaround we are investigating is compiling only those coverage items that have not been
sufficiently hit. The idea is to create a feedback loop between the coverage results database and
the covermap translator. The translator would inspect a list of coverage items hit to date on a
specific version of the design, and not translate those that have been sufficiently hit.

While the reduced performance we have experienced is a major problem, we do not consider this
a defect specific to SystemVerilog or VCS, we suspect that slowdowns like this are common
throughout the industry. Since use of functional coverage is only now starting to spread, the
problem may not be widely recognized; but, it is certainly important. It is obviously desirable to
do functional coverage; but, at the same time it is very costly. Design teams are facing the
difficult choice of having to split their resources between simulating the design, which is their
primary means of finding bugs, and tracking whether their simulations are effective, through
functional coverage. Given the current performance problems of functional coverage, it is
tempting to skip it in favor doing more simulation. Synopsys must work hard to eliminate the
need for this tradeoff, by boosting the performance of functional coverage.


System Verilog Functional Coverage

Recommended

More Related Content

What's hot (20)

Viewers also liked (16)

Similar to System Verilog Functional Coverage (20)

System Verilog Functional Coverage