L.O.B.S.T.E.R.

Lightweight Open BMW Software Traceability Evidence Report
!

Overview

Coverage

Category Ratio Coverage OK Items Total Items
Use Cases 0.0% 0.00% 0 7
Potential Errors 71.4% 71.43% 5 7
Test Specifications 80.0% 80.00% 4 5
System Requirements 0.0% 0.00% 0 3
Software Requirements 0.0% 0.00% 0 2
Code 0.0% 0.00% 0 35
System Tests 41.2% 41.18% 7 17
Software Tests 0.0% 0.00% 0 10

Tracing policy

LOBSTER Tracing Policy n_1cf7e301510c711b73d2b182b9dcf084 Use Cases n_8a4b9a55ec70e0ca32d3abcc64ac6879 Potential Errors n_1cf7e301510c711b73d2b182b9dcf084->n_8a4b9a55ec70e0ca32d3abcc64ac6879 n_419cfc02b324dc7d8faefa90ed608fb0 System Requirements n_1cf7e301510c711b73d2b182b9dcf084->n_419cfc02b324dc7d8faefa90ed608fb0 n_eeb09e7009c699f439a0daadf03458e7 Test Specifications n_8a4b9a55ec70e0ca32d3abcc64ac6879->n_eeb09e7009c699f439a0daadf03458e7 n_3f7ae1d4bec7e6d318ef39c7a7dcaa65 System Tests n_eeb09e7009c699f439a0daadf03458e7->n_3f7ae1d4bec7e6d318ef39c7a7dcaa65 n_9d273bd32c1fceb91b7d6a4d40e98bdd Software Requirements n_419cfc02b324dc7d8faefa90ed608fb0->n_9d273bd32c1fceb91b7d6a4d40e98bdd n_419cfc02b324dc7d8faefa90ed608fb0->n_3f7ae1d4bec7e6d318ef39c7a7dcaa65 n_ca0dbad92a874b2f69b549293387925e Code n_9d273bd32c1fceb91b7d6a4d40e98bdd->n_ca0dbad92a874b2f69b549293387925e n_c4738e043501e27ee3ef08d6a4883d27 Software Tests n_9d273bd32c1fceb91b7d6a4d40e98bdd->n_c4738e043501e27ee3ef08d6a4883d27

Filtering

Item Filters

Show Issues

Filter

Issues

Detailed report

Requirements and Specification

Use Cases

lobster/use_cases.trlc
TRLC Usecase UseCases.Trace_Requirements_to_Tests
As a requirements manager I want to have a report of traceability between
requirements (in codebeamer or TRLC) and tests (in C++ or JSON).
TRLC Usecase UseCases.List_Requirements_to_Tests
As a requirements manager I want the traceability report to show the list of
requirements (from codebeamer or TRLC) which are covered by tests (in C++ or JSON).
Issues:
  • missing reference to System Requirements
TRLC Usecase UseCases.List_Requirements_without_Tests
As a requirements manager I want the traceability report to show the list of
requirements (from codebeamer or TRLC) which are not covered by tests (in C++ or JSON).
Issues:
  • missing reference to System Requirements
TRLC Usecase UseCases.List_Tests_to_Requirements
As a requirements manager I want the traceability report to show the list of
tests (in C++ or JSON) covering requirements in (in codebeamer or TRLC).
Issues:
  • missing reference to System Requirements
TRLC Usecase UseCases.List_Tests_without_Requirements
As a requirements manager I want the traceability report to show the list of
tests (in C++ or JSON) which are not covering any requirement in (in codebeamer or TRLC).
TRLC Usecase UseCases.Requirements_to_Test_Coverage
As a requirements manager I want the traceability report to show the fraction of
the number of requirements (in codebeamer or TRLC) which are covered by tests (in C++ or JSON)
divided by the total number of requirements.
The fraction shall be shown as percentage.
Issues:
  • missing reference to System Requirements
TRLC Usecase UseCases.Item_Source
As a requirements manager
I want the traceability report
between requirements in codebeamer and C++ tests
to mention the source location of each item.
Issues:
  • missing reference to Potential Errors
  • missing reference to System Requirements

Potential Errors

lobster/tools/cpptest/requirements/potential_errors.trlc
TRLC Potentialerror UseCases.Too_few_Extraction_from_CPP_Test_Files
Some c++ test file are ignored
example, some c++ test file are ignored while extracting referance data.
TRLC Potentialerror UseCases.Too_few_tests_Extraction_from_test_file
Some tests from c++ test file are ignored
TRLC Potentialerror UseCases.Too_few_requirement_referance_Extraction_from_test
Some requirement referances from test in c++ file are ignored
TRLC Potentialerror UseCases.Too_many_Extraction_from_CPP_Test_Files
Some non relevant test file are extracted.
example, Some test file are extracted from file which is not relevant.
TRLC Potentialerror UseCases.Too_many_requirement_referance_Extraction_from_test
Some non referanced requirement referances from test in c++ file are extracted
example: While extracting requirement referances from a test case, an additional referance considered
with partial data from previous requirement referance.
TRLC Potentialerror UseCases.Output_Despite_Missing_Config_File
The user does not provide a valid path to a configuration file,
but the lobster-cpptest generates valid output nevertheless, potentially based on
irrelevant C++ files.
For example, the lobster-cpptest might consider the current working directory
as source of C++ files, and these do in fact contain links to real requirements,
such that lobster-report would compute a non-zero coverage value.
Traces to:
Issues:
  • unknown tracing target req UseCases.List_Findings
TRLC Potentialerror UseCases.Output_Despite_Config_File_Error
The user provides an invalid configuration file, but the lobster-cpptest
generates valid output nevertheless, potentially based on
irrelevant C++ files.
This includes cases where the config file:
- contains invalid YAML syntax (e.g., incorrect indentation, missing colons, or other formatting errors),
- is missing required keys or attributes,
Traces to:
Issues:
  • unknown tracing target req UseCases.List_Findings

Test Specifications

lobster/tools/cpptest/requirements/test_specifications.trlc
TRLC Testspecification UseCases.Incorrect_Number_of_Cpp_Tests_in_Output
The test shall verify number of tests in output file are same as available at input.
Issues:
  • missing reference to System Tests
TRLC Testspecification UseCases.Incorrect_number_of_requirement_references_in_Output
The test shall verify number of requirement references available for a c++ test in output file
are same as as requirement references available in c++ tests file.

example : If a test case contains 2 requirement references in c++ test file,
then test shall verify output file contains 2 requirement references for the c++ test.
TRLC Testspecification UseCases.Config_File_Missing
The test shall verify that the lobster-cpptest exits with a non-zero return code when the config file does not exist.
Example: Verify that the lobster-cpptest throws an error when a non-existent config file is provided.
TRLC Testspecification UseCases.Config_File_Syntax_Error
The test shall verify that lobster-cpptest tool exits with a non-zero return code when the config file contains a syntax error.
Example: Verify that lobster-cpptest throws an error when a config file with invalid YAML is provided.
TRLC Testspecification UseCases.Config_File_Key_Error
The test shall verify that lobster-cpptest tool exits with a non-zero return code when the config file is missing required keys.

System Requirements

lobster/tools/cpptest/requirements/input_files.trlc
TRLC System_requirement cpptest_req.Input_File
IF a list element given through the yaml config file option "files" is a file
Issues:
  • missing up reference
  • missing reference to Software Requirements
  • missing reference to System Tests
TRLC System_requirement_aspect cpptest_req.Input_File_Valid_Cpp_Test_File
AND IF the file content is valid,
THEN the tool shall write the output in the LOBSTER interchange format to the file given in the out parameter
AND exit with a zero return code.
Issues:
  • missing up reference
  • missing reference to Software Requirements
TRLC System_requirement_aspect cpptest_req.Input_File_Invalid_Cpp_Test_File
OTHERWISE IF the file content is invalid,
THEN the tool shall write an error message to stderr
AND exit with a non-zero return code.

Software Requirements

lobster/tools/cpptest/requirements/requirements.trlc
TRLC Software_requirement cpptest_req.Dummy_Requirement
This is not really a requirement. It will be used only to generate a minimal tracing report for each tool.
It can be deleted as soon as all the tools get their real requirements.
Issues:
  • missing up reference
  • missing reference to Code
  • missing reference to Software Tests
TRLC Software_requirement cpptest_req.Dummy_Requirement_Unit_Test
This is not really a requirement. It will be used only to generate a minimal tracing report for each tool.
It can be deleted as soon as all the tools get their real requirements.
Issues:
  • missing up reference
  • missing reference to Code
  • missing reference to Software Tests

Implementation

Code

lobster/tools/cpptest/constants.py
Python Constructor constants.Constants.__init__
Issues:
  • missing up reference
lobster/tools/cpptest/cpptest.py
Python Function cpptest.parse_config_file
Issues:
  • missing up reference
Python Function cpptest.get_test_file_list
Issues:
  • missing up reference
Python Function cpptest.collect_test_cases_from_test_files
Issues:
  • missing up reference
Python Function cpptest.create_lobster_items_output_dict_from_test_cases
Issues:
  • missing up reference
Python Function cpptest.write_lobster_items_output_dict
Issues:
  • missing up reference
Python Function cpptest.lobster_cpptest
Issues:
  • missing up reference
Python Constructor cpptest.CppTestTool.__init__
Issues:
  • missing up reference
Python Method cpptest.CppTestTool._run_impl
Issues:
  • missing up reference
Python Method cpptest.CppTestTool._print_error
Issues:
  • missing up reference
Python Method cpptest.CppTestTool._execute
Issues:
  • missing up reference
Python Function cpptest.cpptest_items_to_lobster_file
Issues:
  • missing up reference
Python Function cpptest.main
Issues:
  • missing up reference
lobster/tools/cpptest/requirements_parser.py
Python Method requirements_parser.ParserForRequirements.collect_test_cases_for_test_files
Issues:
  • missing up reference
Python Method requirements_parser.ParserForRequirements.collect_test_cases
Issues:
  • missing up reference
lobster/tools/cpptest/testcase.py
Python Constructor testcase.TestCase.__init__
Issues:
  • missing up reference
Python Method testcase.TestCase._set_test_details
Issues:
  • missing up reference
Python Method testcase.TestCase._definition_end
Issues:
  • missing up reference
Python Method testcase.TestCase._set_test_and_suite_name
Issues:
  • missing up reference
Python Method testcase.TestCase._set_base_attributes
Issues:
  • missing up reference
Python Method testcase.TestCase._get_requirements_from_docu_lines
Issues:
  • missing up reference
Python Method testcase.TestCase._get_testmethod_tag
Issues:
  • missing up reference
Python Method testcase.TestCase._get_version_tag
Issues:
  • missing up reference
Python Method testcase.TestCase._add_multiline_attribute
Issues:
  • missing up reference
Python Method testcase.TestCase.is_line_commented
Issues:
  • missing up reference
Python Method testcase.TestCase.has_no_macro_or_commented
Issues:
  • missing up reference
Python Method testcase.TestCase.has_no_macro_or_commented_general
Issues:
  • missing up reference
Python Method testcase.TestCase.is_special_case
Issues:
  • missing up reference
Python Method testcase.TestCase.try_parse
Issues:
  • missing up reference
Python Method testcase.TestCase.try_parse_general
Issues:
  • missing up reference
Python Method testcase.TestCase._get_uri_from_requirement_detection
Issues:
  • missing up reference
Python Method testcase.TestCase._add_new_requirement_to_requirement_list
Issues:
  • missing up reference
Python Method testcase.TestCase._get_require_tags
Issues:
  • missing up reference
Python Method testcase.TestCase.notracing_special_case
Issues:
  • missing up reference
Python Method testcase.TestCase.get_range_for_doxygen_comments
Issues:
  • missing up reference

Verification and Validation

System Tests

tests_system/lobster_cpptest/test_config_exceptions.py
PyUnit Test test_config_exceptions.ConfigParserExceptionsCpptestTest.test_missing_config_file:16
Derived from:
PyUnit Test test_config_exceptions.ConfigParserExceptionsCpptestTest.test_config_file_errors:31
Derived from:
PyUnit Test test_config_exceptions.ConfigParserExceptionsCpptestTest.test_config_file_syntax_error:75
Derived from:
tests_system/lobster_cpptest/test_directories.py
PyUnit Test test_directories.DirectoriesCpptestTest.test_all_files_from_current_directory_consumed:18
Issues:
  • unknown tracing target req Usecases.Incorrect_Number_of_Cpp_Tests_in_Output
  • missing up reference
PyUnit Test test_directories.DirectoriesCpptestTest.test_files_from_specified_directory_consumed:54
Issues:
  • unknown tracing target req Usecases.Incorrect_Number_of_Cpp_Tests_in_Output
  • missing up reference
PyUnit Test test_directories.DirectoriesCpptestTest.test_specified_directory_and_files_consumed:84
Issues:
  • unknown tracing target req Usecases.Incorrect_Number_of_Cpp_Tests_in_Output
  • missing up reference
PyUnit Test test_directories.DirectoriesCpptestTest.test_no_cpptest_file:117
Issues:
  • missing up reference
tests_system/lobster_cpptest/test_extension.py
PyUnit Test test_extension.ExtensionCpptestTest.test_valid_extension_file:18
Issues:
  • unknown tracing target req Usecases.Incorrect_number_of_requirement_references_in_Output
  • unknown tracing target req Usecases.Incorrect_Number_of_Cpp_Tests_in_Output
  • missing up reference
PyUnit Test test_extension.ExtensionCpptestTest.test_invalid_extension_file:45
Issues:
  • missing up reference
PyUnit Test test_extension.ExtensionCpptestTest.test_no_input_file:70
Issues:
  • missing up reference
tests_system/lobster_cpptest/test_multiple_files.py
PyUnit Test test_multiple_files.MultipleFilesCpptestTest.test_multiple_files:18
Issues:
  • unknown tracing target req Usecases.Incorrect_number_of_requirement_references_in_Output
  • unknown tracing target req Usecases.Incorrect_Number_of_Cpp_Tests_in_Output
  • missing up reference
PyUnit Test test_multiple_files.MultipleFilesCpptestTest.test_multiple_valid_invalid_files:53
Issues:
  • unknown tracing target req Usecases.Incorrect_number_of_requirement_references_in_Output
  • unknown tracing target req Usecases.Incorrect_Number_of_Cpp_Tests_in_Output
  • missing up reference
PyUnit Test test_multiple_files.MultipleFilesCpptestTest.test_no_input_file:88
Issues:
  • unknown tracing target req Usecases.Incorrect_Number_of_Cpp_Tests_in_Output
  • missing up reference
tests_system/lobster_cpptest/test_references.py
PyUnit Test test_references.ReferencesCpptestTest.test_no_references_cpptest_file:18
PyUnit Test test_references.ReferencesCpptestTest.test_one_reference_in_cpptest_file:42
PyUnit Test test_references.ReferencesCpptestTest.test_many_references_in_cpptest_file:66
tests_system/lobster_cpptest/test_valid_scenario.py
PyUnit Test test_valid_scenario.InputFileCpptestTest.test_valid_input_cpptest_file:16
Derived from:

Software Tests

tests_unit/lobster_cpptest/test_cpptest.py
PyUnit Test test_cpptest.LobsterCpptestTests.test_collect_test_cases_from_test_files:51
Issues:
  • missing up reference
PyUnit Test test_cpptest.LobsterCpptestTests.test_parse_config_file:65
Issues:
  • missing up reference
PyUnit Test test_cpptest.LobsterCpptestTests.test_get_test_file_list:75
Issues:
  • missing up reference
PyUnit Test test_cpptest.LobsterCpptestTests.test_get_test_file_list_no_file_with_matching_extension:90
Issues:
  • missing up reference
PyUnit Test test_cpptest.LobsterCpptestTests.test_get_test_file_list_not_existing_file_dir:106
Issues:
  • missing up reference
PyUnit Test test_cpptest.LobsterCpptestTests.test_single_file:122
Issues:
  • missing up reference
PyUnit Test test_cpptest.LobsterCpptestTests.test_single_directory:153
Issues:
  • missing up reference
PyUnit Test test_cpptest.LobsterCpptestTests.test_not_existing_file_dir:175
Issues:
  • missing up reference
PyUnit Test test_cpptest.LobsterCpptestTests.test_separate_output_config:201
Issues:
  • missing up reference
PyUnit Test test_cpptest.LobsterCpptestTests.test_test_case_parsing:255
Issues:
  • missing up reference

LOBSTER Version: 0.14.4-dev