Coverage
| Category | Ratio | Coverage | OK Items | Total Items |
| Use Cases | 91.7% | 11 | 12 | |
| System Requirements | 13.8% | 4 | 29 | |
| Software Requirements | 14.3% | 2 | 14 | |
| System Tests | 100.0% | 31 | 31 | |
| Software Tests | 80.0% | 8 | 10 |
| Category | Ratio | Coverage | OK Items | Total Items |
| Use Cases | 91.7% | 11 | 12 | |
| System Requirements | 13.8% | 4 | 29 | |
| Software Requirements | 14.3% | 2 | 14 | |
| System Tests | 100.0% | 31 | 31 | |
| Software Tests | 80.0% | 8 | 10 |
As a traceability manager I want the traceability report to list items that
fulfill the tracing policy, and how they do so,
so that I can decide whether to release my product.
Example:
If the tracing policy is: Tests --(trace to)--> Requirements,
then the report shall list all tests that link to at least one requirement,
and the report shall list all linked requirements per test.
The use case applies to the reverse direction as well.
As a traceability manager I want the traceability report to list items that
are not covered by other items, where coverage rules are specified by the
tracing policy,
so that I can identify gaps in the coverage.
Example:
If the tracing policy is: Tests --(trace to)--> Requirements,
then the report shall list all tests that do not link to any requirement.
The use case applies to the reverse direction as well.
As a traceability manager I want the traceability report to show the fraction
of the number of items in one level (of the tracing policy)
which fulfill the tracing policy,
divided by the total number of items in that level (of the tracing policy).
The fraction shall be shown as percentage.
Example:
If the tracing policy is: Tests --(trace to)--> Requirements,
then the report shall show the fraction of requirements which are covered by at
least one test, divided by the total number of requirements.
As a requirements manager I want the traceability report to show the list of
requirements which are covered by tests.
As a requirements manager I want the traceability report to show the list of
requirements which are not covered by tests.
As a requirements manager I want the traceability report to show the list of
tests covering requirements.
As a requirements manager I want the traceability report to show the list of
tests which are not covering any requirement.
As a requirements manager I want the traceability report to show the tracing
policy used to generate the report.
As a requirements manager I want to write the tracing policy as a YAML file,
so that I can use a widely known, tool-supported file format.
As a requirements manager I want the traceability report to mention the
codebeamer link to each requirement in codebeamer.
As a CI administrator
I want that the tool exits with a non-zero return code whenever it encounters
invalid input
so that the CI pipeline can catch invalid input automatically.
As a user of LOBSTER I want to have a good usability and standard
command-line ergonomics, like built-in help or automatic creation of output
destinations,
so that using the tools is intuitive and seamless in scripts and interactive
use.
IF the tracing policy configuration declares one or more source files for a level,
THEN the tool shall load, for that level, exactly the items contained in those
declared source files
AND shall merge their items into one set of items for that level, each retaining
exactly the data declared for it in its own source file.
Note: The testable behavior of this requirement is that the generated LOBSTER output
file will contain exactly the items from the configured source files for each level.
IF the same tag is defined more than once among the items loaded from all of the
tracing policy's source files, whether the duplicate definitions appear in the
same source file or in different source files,
THEN the tool shall report, for every definition after the first, the location of
that duplicate definition and the location of the first definition of the same
tag, where "first" is determined solely by the order in which the tracing policy
configuration declares its levels and each level's source files (never by
filesystem or operating-system enumeration order),
AND shall exit with code 1.
IF "trace to" is an invalid entry given in the configuration file according to the
schema,
THEN the tool shall exit with a non-zero return code and with an error message
"lobster error: unknown item".
IF the required tracing policy file for the tool does not exist or cannot be accessed,
THEN the tool shall report "file not found <file_path>" and exit with a non-zero
return code.
IF the tracing policy configuration references input files that cannot be found,
THEN the tool shall report an error
AND shall exit with a non-zero return code.
IF an input file given in the configuration is not valid JSON,
THEN the tool shall raise an error of type LOBSTER_Error
AND shall exit with a non-zero return code
AND shall report the name of that input file, together with the line and column
at which the JSON syntax error was found.
IF a JSON input file does not contain a dictionary at the top level,
THEN the tool shall exit with a non-zero return code.
IF a lobster input file declares a schema other than "lobster-req-trace",
"lobster-imp-trace", or "lobster-act-trace",
THEN the tool shall report "unknown schema kind <schema>"
AND shall exit with a non-zero return code.
IF a lobster input file declares "lobster-req-trace" with a version other than 3
or 4, or declares "lobster-imp-trace" or "lobster-act-trace" with a version other
than 3,
THEN the tool shall report "version <X> for schema <Y> is not supported"
AND shall exit with a non-zero return code.
IF the input .lobster file is missing a schema key,
THEN the tool shall exit with a non-zero return code.
IF the input .lobster file is missing a version key,
THEN the tool shall exit with a non-zero return code.
IF an item's location data is not an object, is missing the required "kind"
field, or specifies an unrecognized "kind",
THEN the tool shall print at least "LOBSTER Error"
AND shall exit with a non-zero return code.
IF the lobster configuration file (tracing policy file) and every source file it
declares are valid,
THEN the tool shall write one report file containing
- every level declared in the configuration, with its items and its coverage,
- and the tracing policy
to the path specified by the --out parameter (or its default value),
AND shall exit with return code 0.
IF the tool generates a report,
THEN the "policy" section of the generated report file shall semantically equal the
tracing policy read from the input configuration file.
IF an input item carries a codebeamer source reference,
THEN the generated report shall preserve that reference unchanged for that item.
IF the user provides a tracing policy configuration file with a ".yaml" or ".yml"
file extension,
THEN the tool shall process the source files specified for each level in that policy,
determine the tracing status of each loaded item according to the policy.
IF the tracing policy configuration file given to the tool does not have a ".yaml"
or ".yml" file extension,
THEN the tool shall print a warning to standard output stating that the
LOBSTER tracing-policy format is deprecated and will be removed in a future release,
AND shall recommend using a ".yaml" file instead,
AND shall process the file and generate a LOBSTER report file with content
identical to what would have been generated if the tracing policy were provided in
YAML format with semantically equivalent content.
IF the tracing policy configuration file with a ".yaml" or ".yml" file extension
is not valid YAML or does not conform to the tracing policy schema,
THEN the tool shall print to standard output a message
of the form "LOBSTER Error: <file>:<line>: invalid tracing policy: <details>", where <line> is
the line in the file at which the first problem was found,
AND shall exit with code 1
AND shall not generate a LOBSTER report file.
IF the level of an [[Item]] has a [[Required_Up_Reference]], one or more
[[Required_Down_Reference_Group]]s, or both,
AND the [[Item]] satisfies its [[Required_Up_Reference]] (if any) and every one
of its [[Required_Down_Reference_Group]]s (if any) with an actual reference each,
THEN the tool shall set the tracing status of that [[Item]] to "OK".
IF the level of an [[Item]] has neither a [[Required_Up_Reference]] nor any
[[Required_Down_Reference_Group]]s,
AND the [[Item]] has no justification and no message in its source data,
THEN the tool shall set the tracing status of that [[Item]] to "OK".
IF the level of an [[Item]] has a [[Required_Up_Reference]], one or more
[[Required_Down_Reference_Group]]s, or both,
AND at least one of those is not satisfied by an actual reference,
AND the [[Item]]'s unsatisfied references are covered by a justification each (a
global justification, or a justification specific to the direction it covers),
THEN the tool shall set the tracing status of that [[Item]] to "JUSTIFIED".
IF the level of an [[Item]] has both a [[Required_Up_Reference]] and one or more
[[Required_Down_Reference_Group]]s,
AND, of the up direction and the down direction, exactly one is fully satisfied
(by one or more actual references or a justification) while the other is not
satisfied at all,
THEN the tool shall set the tracing status of that [[Item]] to "PARTIAL".
IF the level of an [[Item]] has a [[Required_Up_Reference]], one or more
[[Required_Down_Reference_Group]]s, or both,
AND neither the up direction (if required) nor the down direction (if required)
is satisfied for that [[Item]] by an actual reference or a justification,
THEN the tool shall set the tracing status of that [[Item]] to "MISSING"
AND shall record, in the message section of that [[Item]] (according to the
schema), which required reference is missing.
IF an [[Item]] carries at least one message that was already present when it was
loaded from its source file (for example, a message recorded by the tool that
extracted the item),
THEN the tool shall set the tracing status of that [[Item]] to "PARTIAL", even
when every reference required for its level is present.
IF a level defined in the tracing policy has at least one item,
THEN the tool shall set the coverage of that level, in the generated report file,
to the number of items in that level with tracing status "OK" or "JUSTIFIED",
divided by the total number of items in that level, expressed as a percentage.
IF a level defined in the tracing policy has zero items in the corresponding input
file,
THEN the tool shall set the coverage for that level to 0.0 in the generated report
file.
IF a reference in a lobster file points to a tag that does not exist in the set of
loaded items,
THEN the tool shall set the message of that [[Item]] to "unknown tracing target <tag>"
AND its tracing status to MISSING.
IF a reference specifies a version but the destination tag is unversioned,
THEN the tool shall set the message of that [[Item]] to
"tracing destination <tag> is unversioned"
AND its tracing status to MISSING.
IF a reference specifies a version and the destination tag has a different version,
THEN the tool shall set the message of that [[Item]] to "tracing destination <tag>
has version <x> (expected <y>)"
AND its tracing status to MISSING.
IF a raw tracing policy contains two or more levels declared with the same name,
THEN the "build_tracing_policy" function SHALL raise a fatal LOBSTER_Error
with the message "duplicate declaration" at the location of the repeated declaration.
IF a level's source file entry names a file that does not exist on disk,
THEN the "build_tracing_policy" function SHALL raise a fatal LOBSTER_Error
with the message "cannot find file <path>", where <path> is the file path
given in the source entry.
IF a level's "trace to" list contains the level's own name,
THEN the "build_tracing_policy" function SHALL raise a fatal LOBSTER_Error
with the message "cannot trace to yourself".
IF a level's "trace to" list references a level name that is not declared
anywhere in the raw tracing policy,
THEN the "build_tracing_policy" function SHALL raise a fatal LOBSTER_Error
with the message "unknown item <target>", where <target> is the undeclared
level name.
IF a level's "requires" entry names a candidate level that is not declared
anywhere in the raw tracing policy,
THEN the "build_tracing_policy" function SHALL raise a fatal LOBSTER_Error
with the message "unknown level <name>", where <name> is the undeclared
level name.
IF a level's "requires" entry names a candidate level that does not have
a "trace to" entry pointing at the requiring level,
THEN the "build_tracing_policy" function SHALL raise a fatal LOBSTER_Error
with the message "<candidate> cannot trace to <level> items".
IF a level does not declare any "requires" entry,
THEN the "build_tracing_policy" function SHALL derive its
"breakdown_requirements" as one single-candidate OR-group per other level
that declares a "trace to" entry pointing at it.
OTHERWISE, the function SHALL translate each declared "requires" entry
into one OR-group containing the candidate levels named in that entry, in
their declared order.
FOR each valid "trace to" entry from a level to another level, the
"build_tracing_policy" function SHALL append the target name to the declaring
level's "traces" list, set "needs_tracing_up" to true on the declaring level,
AND set "needs_tracing_down" to true on the targeted level.
IF a raw tracing policy contains uniquely named levels,
THEN the "build_tracing_policy" function SHALL return one level definition
for each raw level, in the same order as the raw levels, with each definition
retaining the raw level's name and kind.
IF the file name given to the "load_tracing_policy" function has the
extension ".yaml" or ".yml" (compared case-insensitively),
THEN the function SHALL read the file with the YAML policy reader,
AND SHALL NOT use the legacy parser,
AND SHALL NOT print a deprecation warning,
AND SHALL return the level definitions built from the raw policy.
IF the file name given to the "load_tracing_policy" function does not have
the extension ".yaml" or ".yml",
THEN the function SHALL print a warning to standard output that names the file
and states that the tracing-policy in the LOBSTER configuration language is
deprecated,
AND SHALL parse the file with the legacy parser,
AND SHALL return the level definitions built from the raw policy.
IF the file given to the "read_yaml_policy" function is a valid YAML tracing
policy,
THEN the function SHALL return a raw policy with one raw level per entry of
the "levels" list, in the same order,
AND each raw level SHALL retain its name, kind, "source" entries,
"trace-to" entries and "requires" OR-groups, with omitted optional entries
being empty,
AND every location stored in the raw policy SHALL be the policy file.
IF the file given to the "read_yaml_policy" function is not valid YAML
or violates the tracing policy schema,
THEN the function SHALL raise a LOBSTER_Exception
whose message states "invalid tracing policy"
AND begins, for each reported problem, with "<file>:<line>:", where <line> is
the 1-based line of the offending element in the file
(or of the nearest enclosing element if the offending element is missing).
The "validate_yaml_file" function SHALL load the given schema file and read the
given YAML file as UTF-8, independent of the platform's default encoding,
AND SHALL validate the file content against the loaded schema,
AND SHALL report the absolute path of the YAML file in validation errors,
AND SHALL let validation errors propagate to the caller.