L.O.B.S.T.E.R.

Lightweight Open BMW Software Traceability Evidence Report
!

Overview

Coverage

Category Ratio Coverage OK Items Total Items
Use Cases 0.0% 0.00% 0 11
Potential Errors 100.0% 100.00% 3 3
Test Specifications 0.0% 0.00% 0 3
System Requirements 0.0% 0.00% 0 20
Software Requirements 0.0% 0.00% 0 1
Code 0.0% 0.00% 0 4
System Tests 94.4% 94.44% 17 18
Software Tests 0.0% 0.00% 0 2

Tracing policy

LOBSTER Tracing Policy n_1cf7e301510c711b73d2b182b9dcf084 Use Cases n_8a4b9a55ec70e0ca32d3abcc64ac6879 Potential Errors n_1cf7e301510c711b73d2b182b9dcf084->n_8a4b9a55ec70e0ca32d3abcc64ac6879 n_419cfc02b324dc7d8faefa90ed608fb0 System Requirements n_1cf7e301510c711b73d2b182b9dcf084->n_419cfc02b324dc7d8faefa90ed608fb0 n_eeb09e7009c699f439a0daadf03458e7 Test Specifications n_8a4b9a55ec70e0ca32d3abcc64ac6879->n_eeb09e7009c699f439a0daadf03458e7 n_3f7ae1d4bec7e6d318ef39c7a7dcaa65 System Tests n_eeb09e7009c699f439a0daadf03458e7->n_3f7ae1d4bec7e6d318ef39c7a7dcaa65 n_9d273bd32c1fceb91b7d6a4d40e98bdd Software Requirements n_419cfc02b324dc7d8faefa90ed608fb0->n_9d273bd32c1fceb91b7d6a4d40e98bdd n_419cfc02b324dc7d8faefa90ed608fb0->n_3f7ae1d4bec7e6d318ef39c7a7dcaa65 n_ca0dbad92a874b2f69b549293387925e Code n_9d273bd32c1fceb91b7d6a4d40e98bdd->n_ca0dbad92a874b2f69b549293387925e n_c4738e043501e27ee3ef08d6a4883d27 Software Tests n_9d273bd32c1fceb91b7d6a4d40e98bdd->n_c4738e043501e27ee3ef08d6a4883d27

Filtering

Item Filters

Show Issues

Filter

Issues

Detailed report

Requirements and Specification

Use Cases

lobster/use_cases.trlc
TRLC Usecase UseCases.Trace_Requirements_to_Tests
As a requirements manager I want to have a report of traceability between
requirements (in codebeamer or TRLC) and tests (in C++ or JSON).
Issues:
  • missing reference to Potential Errors
  • missing reference to System Requirements
TRLC Usecase UseCases.List_Requirements_to_Tests
As a requirements manager I want the traceability report to show the list of
requirements (from codebeamer or TRLC) which are covered by tests (in C++ or JSON).
Issues:
  • missing reference to System Requirements
TRLC Usecase UseCases.List_Requirements_without_Tests
As a requirements manager I want the traceability report to show the list of
requirements (from codebeamer or TRLC) which are not covered by tests (in C++ or JSON).
Issues:
  • missing reference to System Requirements
TRLC Usecase UseCases.List_Tests_to_Requirements
As a requirements manager I want the traceability report to show the list of
tests (in C++ or JSON) covering requirements in (in codebeamer or TRLC).
Issues:
  • missing reference to System Requirements
TRLC Usecase UseCases.List_Tests_without_Requirements
As a requirements manager I want the traceability report to show the list of
tests (in C++ or JSON) which are not covering any requirement in (in codebeamer or TRLC).
Issues:
  • missing reference to System Requirements
TRLC Usecase UseCases.Requirements_to_Test_Coverage
As a requirements manager I want the traceability report to show the fraction of
the number of requirements (in codebeamer or TRLC) which are covered by tests (in C++ or JSON)
divided by the total number of requirements.
The fraction shall be shown as percentage.
Traces to:
Issues:
  • missing reference to System Requirements
TRLC Usecase UseCases.Show_Tracing_Policy
As a requirements manager I want the traceability report to show the tracing policy used to generate the report.
Issues:
  • missing reference to Potential Errors
  • missing reference to System Requirements
TRLC Usecase UseCases.Show_codebeamer_links
As a requirements manager I want the traceability report to mention the codebeamer link to requirements in codebeamer.
Issues:
  • missing reference to Potential Errors
  • missing reference to System Requirements
TRLC Usecase UseCases.Item_Source
As a requirements manager
I want the traceability report
between requirements in codebeamer and C++ tests
to mention the source location of each item.
Issues:
  • missing reference to Potential Errors
  • missing reference to System Requirements
TRLC Usecase UseCases.git_hash
As a requirements manager I want the traceability report
which includes sources from a local git clone
to show the git hashes of the git commit from which files were taken.
Issues:
  • missing reference to Potential Errors
  • missing reference to System Requirements
TRLC Usecase UseCases.List_Findings
As a requirements manager I want the traceability report
between requirements (in codebeamer or TRLC) and tests (in C++ or JSON)
to show a list with all findings
where the tracing policy is violated.
Issues:
  • missing reference to Potential Errors
  • missing reference to System Requirements

Potential Errors

lobster/tools/core/report/requirements/potential_errors.trlc
TRLC Potentialerror UseCases.Incorrect_Coverage_Calculation
Calculation of total number of requirements and requirements covered is not correct.
example, error while converting to percentage value.
Traces to:
Derived from:
TRLC Potentialerror UseCases.Wrong_Mapping_of_Requirement_to_Software_Test
Some Requirements are not mapped to the tests correctly.
example, requirement mapping is missed or requirements are swapped while mapping.
TRLC Potentialerror UseCases.Wrong_Mapping_of_Software_Test_to_Requirement
Some software tests are not mapped to the requirements correctly.
example, software test mapping is missed or software tests are swapped while mapping.

Test Specifications

lobster/tools/core/report/requirements/test_specifications.trlc
TRLC Testspecification UseCases.Coverage_calculation_in_Output
The test shall verify the output file contains correct coverage calculation
based on total number of requirements and number of covered requirements.
Derived from:
Issues:
  • missing reference to System Tests
TRLC Testspecification UseCases.Requirement_to_software_Test_Mapping_in_Output
The test shall verify output file contains correct mapping of requirement to software tests
based on requirement reference mentioned in software tests.
Issues:
  • missing reference to System Tests
TRLC Testspecification UseCases.Software_Test_to_Requirement_Mapping_in_output
The test shall verify output file contains correct mapping of software tests to requirement
based on requirement reference mentioned in software tests.
Issues:
  • missing reference to System Tests

System Requirements

lobster/tools/core/report/requirements/requirements.trlc
TRLC System_requirement core_report_req.Report_Input_Duplicate_Definition
If any of the source files given in the lobster configuration file contains a tag
more than once, then the tool must exit with code 1.
Issues:
  • missing up reference
  • missing reference to Software Requirements
  • missing reference to System Tests
TRLC System_requirement_aspect core_report_req.Invalid_Trace_To
IF "trace to" is an invalid entry given in the configuration file according to the schema,
THEN the tool shall exit with a non-zero return code and with an error message "lobster error: unknown item".
Issues:
  • missing up reference
  • missing reference to Software Requirements
  • missing reference to System Tests
TRLC System_requirement_aspect core_report_req.Status_Justified_Global
IF the "just_global" list of the [[item]] contains at least one value,
THEN the tool shall set the tracing status to "JUSTIFIED" of that [[Item]].
Issues:
  • missing up reference
  • missing reference to Software Requirements
TRLC System_requirement_aspect core_report_req.Status_Justified_Up
OTHERWISE
IF the "just_up" list of the [[item]] contains at least one value
AND there is at least one level in "trace to" in the tracing policy for the level of the item,
THEN the tool shall set the tracing status to "JUSTIFIED" of that [[Item]].
Issues:
  • missing up reference
  • missing reference to Software Requirements
TRLC System_requirement_aspect core_report_req.Status_Justified_Down
OTHERWISE
IF the "just_down" list of the [[item]] contains at least one value
AND the level of the item is mentioned in at least one "trace to" entry of any other level,
THEN the tool shall set the tracing status to "JUSTIFIED" of that [[Item]].
Issues:
  • missing up reference
  • missing reference to Software Requirements
TRLC System_requirement_aspect core_report_req.Status_Missing
OTHERWISE, IF an [[Item]] has no references to an [[Item]] from a level mentioned in trace to,
THEN the tool shall set the tracing status to "MISSING" of that [[Item]] and write "missing up reference" into the message section according to the schema.
Issues:
  • missing up reference
  • missing reference to Software Requirements
TRLC System_requirement_aspect core_report_req.Status_Ok
OTHERWISE, IF an [[Item]] has references to an [[Item]] from a level mentioned in trace to,
THEN the tool shall set the tracing status to "OK" of that [[Item]]
Issues:
  • missing up reference
  • missing reference to Software Requirements
TRLC System_requirement_aspect core_report_req.Zero_Items_Coverage
IF a level defined in the tracing policy has zero items in the corresponding input file,
THEN the tool shall set the coverage for that level to 0.0 in the generated report file.
Issues:
  • missing up reference
  • missing reference to Software Requirements
TRLC System_requirement_aspect core_report_req.Invalid_JSON_File
IF an input file given in the configuration is not valid JSON,
THEN the tool shall raise an error of type LOBSTER_Error
AND shall exit with a non-zero return code
AND shall report the error location and message.
Issues:
  • missing up reference
  • missing reference to Software Requirements
TRLC System_requirement_aspect core_report_req.Invalid_JSON_Not_Dict
IF a JSON input file does not contain a dictionary at the top level,
THEN the tool shall raise a Exception,
AND the tool shall exit with a non-zero return code.
Issues:
  • missing up reference
  • missing reference to Software Requirements
TRLC System_requirement_aspect core_report_req.Invalid_Schema
IF a lobster input file contains a schema that is not in the list of supported schemas,
THEN the tool shall:
- Report "unknown schema kind <schema>"
- Exit with a non-zero return code
Issues:
  • missing up reference
  • missing reference to Software Requirements
TRLC System_requirement_aspect core_report_req.Invalid_Version
IF a lobster input file contains a schema with an unsupported version number,
THEN the tool shall:
- Report "version <X> for schema <Y> is not supported"
- Exit with a non-zero return code
Issues:
  • missing up reference
  • missing reference to Software Requirements
  • missing reference to System Tests
TRLC System_requirement_aspect core_report_req.Missing_Schema_Key
IF the input .lobster file is missing a schema key,
THEN the tool shall raise a LOBSTER_Exception,
AND the tool shall exit with a non-zero return code.
Issues:
  • missing up reference
  • missing reference to Software Requirements
TRLC System_requirement_aspect core_report_req.Missing_Version_Key
IF the input .lobster file is missing a version key,
THEN the tool shall raise a LOBSTER_Exception,
AND the tool shall exit with a non-zero return code.
Issues:
  • missing up reference
  • missing reference to Software Requirements
TRLC System_requirement_aspect core_report_req.Lobster_Exception_Dump_Invalid_Input
IF the tool detects an invalid input file
AND raises a LOBSTER_Exception,
THEN the tool shall invoke the exception's dump() method
to display the error details in the standard output
AND exit with a non-zero return code.
Issues:
  • missing up reference
  • missing reference to Software Requirements
TRLC System_requirement_aspect core_report_req.Unknown_Tracing_Target
IF a reference in a lobster file points to a tag that does not exist in the set of loaded items,
THEN the tool shall report "unknown tracing target <tag>" and exit with non-zero return code.
Issues:
  • missing up reference
  • missing reference to Software Requirements
TRLC System_requirement_aspect core_report_req.Tracing_Destination_Unversioned
IF a reference specifies a version but the destination tag is unversioned,
THEN the tool shall report "tracing destination <tag> is unversioned" and exit with non-zero return code.
Issues:
  • missing up reference
  • missing reference to Software Requirements
TRLC System_requirement_aspect core_report_req.Tracing_Destination_Version_Mismatch
IF a reference specifies a version and the destination tag has a different version,
THEN the tool shall report "tracing destination <tag> has version <x> (expected <y>)" and exit with non-zero return code.
Issues:
  • missing up reference
  • missing reference to Software Requirements
TRLC System_requirement_aspect core_report_req.File_Not_Found
IF a required input file for the tool does not exist or cannot be accessed,
THEN the tool shall report "file not found <file_path>" and exit with a non-zero return code.
Issues:
  • missing up reference
  • missing reference to Software Requirements
TRLC System_requirement_aspect core_report_req.Message_Trace_Coverage
IF an item contains a "message" and traces to another item,
THEN the traced item shall have 100% coverage,
AND the item with the message shall have 0% coverage.
Issues:
  • missing up reference
  • missing reference to Software Requirements

Software Requirements

TRLC Software_requirement core_report_req.Dummy_Requirement_Unit_Test
This is not really a requirement. It will be used only to generate a minimal tracing report for each tool.
It can be deleted as soon as all the tools get their real requirements.
Issues:
  • missing up reference
  • missing reference to Code
  • missing reference to Software Tests

Implementation

Code

lobster/tools/core/report/report.py
Python Constructor report.ReportTool.__init__
Issues:
  • missing up reference
Python Method report.ReportTool._run_impl
Issues:
  • missing up reference
Python Function report.generate_report_file
Issues:
  • missing up reference
Python Function report.main
Issues:
  • missing up reference

Verification and Validation

System Tests

tests_system/lobster_report/test_items_coverage.py
PyUnit Test test_items_coverage.ReportItemsCoverageTest.test_zero_items_coverage:11
Derived from:
PyUnit Test test_items_coverage.ReportItemsCoverageTest.test_items_message_trace_coverage:34
Derived from:
tests_system/lobster_report/test_json_scanario.py
PyUnit Test test_json_scanario.ReportInvalidJsonTest.test_invalid_json:11
Derived from:
PyUnit Test test_json_scanario.ReportInvalidJsonTest.test_missing_input_file:29
Derived from:
PyUnit Test test_json_scanario.ReportInvalidJsonTest.test_lobster_exception_dump:42
Derived from:
PyUnit Test test_json_scanario.ReportInvalidJsonTest.test_invalid_json_not_dict:62
Derived from:
tests_system/lobster_report/test_justification.py
PyUnit Test test_justification.ReportJustificationTest.test_justification_and_coverage:12
Derived from:
tests_system/lobster_report/test_resolve_references_errors.py
PyUnit Test test_resolve_references_errors.ReportResolveReferencesErrorsTest.test_unknown_tracing_target:11
Derived from:
PyUnit Test test_resolve_references_errors.ReportResolveReferencesErrorsTest.test_tracing_destination_unversioned:30
Derived from:
PyUnit Test test_resolve_references_errors.ReportResolveReferencesErrorsTest.test_tracing_destination_version_mismatch:49
Derived from:
tests_system/lobster_report/test_schema_and_version.py
PyUnit Test test_schema_and_version.ReportSchemaAndVersionTest.test_invalid_schema:11
Derived from:
PyUnit Test test_schema_and_version.ReportSchemaAndVersionTest.test_missing_schema:32
Derived from:
PyUnit Test test_schema_and_version.ReportSchemaAndVersionTest.test_invalid_version:50
Issues:
  • unknown tracing target req core_report_req.Invalid_Schema_Version
  • missing up reference
PyUnit Test test_schema_and_version.ReportSchemaAndVersionTest.test_missing_version:71
Derived from:
tests_system/lobster_report/test_status_justified.py
PyUnit Test test_status_justified.ReportJustifiedTest.test_status_justified:11
Derived from:
tests_system/lobster_report/test_status_missing.py
PyUnit Test test_status_missing.ReportJustifiedTest.test_status_missing:11
Derived from:
PyUnit Test test_status_missing.ReportJustifiedTest.test_status_missing_mixed:33
Derived from:
tests_system/lobster_report/test_status_ok.py
PyUnit Test test_status_ok.ReportOkTest.test_status_ok:11
Derived from:

Software Tests

tests_unit/lobster_report/test_report.py
PyUnit Test test_report.ReportTests.test_compute_coverage:8
Issues:
  • missing up reference
PyUnit Test test_report.ReportTests.test_generate_report_file:52
Issues:
  • missing up reference

LOBSTER Version: 0.14.4-dev