L.O.B.S.T.E.R.

Lightweight Open BMW Software Traceability Evidence Report
!

Overview

Coverage

Category Ratio Coverage OK Items Total Items
Use Cases 0.0% 0.00% 0 8
Potential Errors 100.0% 100.00% 4 4
Test Specifications 100.0% 100.00% 3 3
System Requirements 0.0% 0.00% 0 7
Software Requirements 0.0% 0.00% 0 1
Code 3.8% 3.85% 1 26
System Tests 85.7% 85.71% 6 7
Software Tests 0.0% 0.00% 0 20

Tracing policy

LOBSTER Tracing Policy n_1cf7e301510c711b73d2b182b9dcf084 Use Cases n_8a4b9a55ec70e0ca32d3abcc64ac6879 Potential Errors n_1cf7e301510c711b73d2b182b9dcf084->n_8a4b9a55ec70e0ca32d3abcc64ac6879 n_419cfc02b324dc7d8faefa90ed608fb0 System Requirements n_1cf7e301510c711b73d2b182b9dcf084->n_419cfc02b324dc7d8faefa90ed608fb0 n_eeb09e7009c699f439a0daadf03458e7 Test Specifications n_8a4b9a55ec70e0ca32d3abcc64ac6879->n_eeb09e7009c699f439a0daadf03458e7 n_3f7ae1d4bec7e6d318ef39c7a7dcaa65 System Tests n_eeb09e7009c699f439a0daadf03458e7->n_3f7ae1d4bec7e6d318ef39c7a7dcaa65 n_9d273bd32c1fceb91b7d6a4d40e98bdd Software Requirements n_419cfc02b324dc7d8faefa90ed608fb0->n_9d273bd32c1fceb91b7d6a4d40e98bdd n_419cfc02b324dc7d8faefa90ed608fb0->n_3f7ae1d4bec7e6d318ef39c7a7dcaa65 n_ca0dbad92a874b2f69b549293387925e Code n_9d273bd32c1fceb91b7d6a4d40e98bdd->n_ca0dbad92a874b2f69b549293387925e n_c4738e043501e27ee3ef08d6a4883d27 Software Tests n_9d273bd32c1fceb91b7d6a4d40e98bdd->n_c4738e043501e27ee3ef08d6a4883d27

Filtering

Item Filters

Show Issues

Filter

Issues

Detailed report

Requirements and Specification

Use Cases

lobster/use_cases.trlc
TRLC Usecase UseCases.Trace_Requirements_to_Tests
As a requirements manager I want to have a report of traceability between
requirements (in codebeamer or TRLC) and tests (in C++ or JSON).
Issues:
  • missing reference to System Requirements
TRLC Usecase UseCases.List_Requirements_to_Tests
As a requirements manager I want the traceability report to show the list of
requirements (from codebeamer or TRLC) which are covered by tests (in C++ or JSON).
Issues:
  • missing reference to System Requirements
TRLC Usecase UseCases.List_Requirements_without_Tests
As a requirements manager I want the traceability report to show the list of
requirements (from codebeamer or TRLC) which are not covered by tests (in C++ or JSON).
Issues:
  • missing reference to System Requirements
TRLC Usecase UseCases.List_Tests_to_Requirements
As a requirements manager I want the traceability report to show the list of
tests (in C++ or JSON) covering requirements in (in codebeamer or TRLC).
Issues:
  • missing reference to Potential Errors
  • missing reference to System Requirements
TRLC Usecase UseCases.List_Tests_without_Requirements
As a requirements manager I want the traceability report to show the list of
tests (in C++ or JSON) which are not covering any requirement in (in codebeamer or TRLC).
Issues:
  • missing reference to Potential Errors
  • missing reference to System Requirements
TRLC Usecase UseCases.Requirements_to_Test_Coverage
As a requirements manager I want the traceability report to show the fraction of
the number of requirements (in codebeamer or TRLC) which are covered by tests (in C++ or JSON)
divided by the total number of requirements.
The fraction shall be shown as percentage.
Issues:
  • missing reference to Potential Errors
  • missing reference to System Requirements
TRLC Usecase UseCases.Show_codebeamer_links
As a requirements manager I want the traceability report to mention the codebeamer link to requirements in codebeamer.
Issues:
  • missing reference to Potential Errors
  • missing reference to System Requirements
TRLC Usecase UseCases.Item_Source
As a requirements manager
I want the traceability report
between requirements in codebeamer and C++ tests
to mention the source location of each item.
Issues:
  • missing reference to Potential Errors
  • missing reference to System Requirements

Potential Errors

lobster/tools/codebeamer/requirements/potential_errors.trlc
TRLC Potentialerror UseCases.Incomplete_Extraction_from_Codebeamer
Some requirements are not extracted completely.
For example, some information of a requirement is missing.
TRLC Potentialerror UseCases.Wrong_Extraction_from_Codebeamer
Some requirements are extracted with wrong data.
For example, some information of the requirement is incorrect.
TRLC Potentialerror UseCases.Too_few_Extraction_from_Codebeamer
Some requirements are not extracted from codebeamer.
For example, there are 100 applicable requirements in codebeamer tracker but LOBSTER extracts 75 requirements.
TRLC Potentialerror UseCases.Too_many_Extraction_from_Codebeamer
Extra requirements are extracted from codebeamer.
For example, there are 100 applicable requirements in codebeamer tracker but LOBSTER extracts 125 requirements with 25 requirements from not applicable section of tracker.

Test Specifications

lobster/tools/codebeamer/requirements/test_specifications.trlc
TRLC Testspecification UseCases.Wrong_Codebeamer_IDs_in_Output
The test shall check that the IDs (including their version) downloaded from
codebeamer are exactly the same as written in the output.

Example 1: verify that the tool does NOT truncate IDs. For example,
it could download 10 IDs as expected, but those ID strings could be truncated by one digit each.

Example 2: Verify that the tool does NOT duplicate IDs.

Test steps:
- download a set of items from a mock server
- verify that the obtained IDs are the same as provided by the mock server
TRLC Testspecification UseCases.Codebeamer_Summary_in_Output
The test shall check that the LOBSTER items in the output file use the summary
as downloaded from codebeamer as name for the LOBSTER item.
TRLC Testspecification UseCases.Incorrect_Number_of_Codebeamer_Items_in_Output
The test shall verify that the number of items downloaded from codebeamer are same as written in the output.

Example: Verify that tool does NOT duplicate items.

System Requirements

lobster/tools/codebeamer/requirements/import_query.trlc
TRLC System_requirement codebeamer_req.Query_Id_Parameter
IF an element given through the yaml config file option "import_query" is a valid codebeamer query id
(integer greater than zero),
THEN the tool shall request items from the codebeamer server with the given query id
AND write the obtained codebeamer items in the LOBSTER interchange format to the file given in the "out" parameter
Issues:
  • missing up reference
  • missing reference to Software Requirements
TRLC System_requirement codebeamer_req.Query_String_Parameter
IF an element given through the yaml config file option "import_query" is valid codebeamer query string,
THEN the tool shall request items from the codebeamer server with the given query string
AND write the obtained codebeamer items in the LOBSTER interchange format to the file given in the "out" parameter
Issues:
  • missing up reference
  • missing reference to Software Requirements
  • missing reference to System Tests
lobster/tools/codebeamer/requirements/requirements.trlc
TRLC System_requirement codebeamer_req.Empty_Query_String_Parameter
IF the configuration parameter "import_query" is empty,
THEN the tool shall display an error message and exit with a non-zero return code.
Issues:
  • missing up reference
  • missing reference to Software Requirements
  • missing reference to System Tests
TRLC System_requirement codebeamer_req.Empty_Query_Message
IF the codebeamer server returns an empty list as (a result of the query),
THEN the tool shall display a message to the user.
Traces to:
Issues:
  • missing up reference
  • missing reference to System Tests
TRLC System_requirement codebeamer_req.Retry_On_Specific_HTTPS_Status_Codes
IF the actual HTTPS return code is in the list specified by the configuration parameter "retry_error_codes"
AND the configuration parameter "num_request_retry" is greater than 0 or not defined,
THEN the tool shall retry the request up to a maximum of "num_request_retry" times (default is 5 if not set).
Issues:
  • missing up reference
  • missing reference to Software Requirements
TRLC System_requirement codebeamer_req.Missing_Error_Code
IF the configuration parameter "retry_error_codes" is not defined,
THEN the retry logic shall not be executed, regardless of the value of "num_request_retry".
Issues:
  • missing up reference
  • missing reference to Software Requirements
TRLC System_requirement codebeamer_req.References_Field_Support
IF an element given through the yaml config file option "refs" is list of codebeamer field names,
THEN the tool shall extract the corresponding references from the given codebeamer item fields
AND add a Tracing_Tag entry to the LOBSTER item for each referenced item ID
Issues:
  • missing up reference
  • missing reference to Software Requirements

Software Requirements

TRLC Software_requirement codebeamer_req.Get_Query_Zero_Items_Message
The function "get_query" shall print a message to the user IF the query returns zero items.
Traces to:
Derived from:
Issues:
  • missing reference to Software Tests

Implementation

Code

lobster/tools/codebeamer/bearer_auth.py
Python Constructor bearer_auth.BearerAuth.__init__
Issues:
  • missing up reference
Python Method bearer_auth.BearerAuth.__call__
Issues:
  • missing up reference
lobster/tools/codebeamer/codebeamer.py
Python Method codebeamer.SupportedConfigKeys.as_set
Issues:
  • missing up reference
Python Function codebeamer.get_authentication
Issues:
  • missing up reference
Python Function codebeamer.query_cb_single
Issues:
  • missing up reference
Python Function codebeamer.get_single_item
Issues:
  • missing up reference
Python Function codebeamer.get_many_items
Issues:
  • missing up reference
Python Function codebeamer.get_query
Derived from:
Python Function codebeamer.get_schema_config
Issues:
  • missing up reference
Python Function codebeamer.to_lobster
Issues:
  • missing up reference
Python Function codebeamer._create_common_params
Issues:
  • missing up reference
Python Function codebeamer._create_lobster_item
Issues:
  • missing up reference
Python Function codebeamer.import_tagged
Issues:
  • missing up reference
Python Function codebeamer.ensure_list
Issues:
  • missing up reference
Python Function codebeamer.update_authentication_parameters
Issues:
  • missing up reference
Python Function codebeamer.load_config
Issues:
  • missing up reference
Python Function codebeamer.parse_config_data
Issues:
  • missing up reference
Python Constructor codebeamer.CodebeamerTool.__init__
Issues:
  • missing up reference
Python Method codebeamer.CodebeamerTool._run_impl
Issues:
  • missing up reference
Python Method codebeamer.CodebeamerTool._print_error
Issues:
  • missing up reference
Python Method codebeamer.CodebeamerTool._execute
Issues:
  • missing up reference
Python Function codebeamer._get_out_stream
Issues:
  • missing up reference
Python Function codebeamer._cb_items_to_lobster
Issues:
  • missing up reference
Python Function codebeamer.cb_query_to_lobster_file
Issues:
  • missing up reference
Python Function codebeamer.main
Issues:
  • missing up reference
lobster/tools/codebeamer/config.py
Python Method config.Config.base
Issues:
  • missing up reference

Verification and Validation

System Tests

tests_system/lobster_codebeamer/lobster_codebeamer_test_runner.py
PyUnit Test lobster_codebeamer_test_runner.LobsterCodebeamerTestRunner.run_tool_test:86
Issues:
  • missing up reference
tests_system/lobster_codebeamer/test_extract_requirements.py
PyUnit Test test_extract_requirements.LobsterCodebeamerExtractRequirementsTest.test_extract_requirements_scenarios:21
tests_system/lobster_codebeamer/test_lobster_codebeamer.py
PyUnit Test test_lobster_codebeamer.LobsterCodebeamerTest.test_retry_if_configured:23
Derived from:
PyUnit Test test_lobster_codebeamer.LobsterCodebeamerTest.test_retry_then_success:49
Derived from:
PyUnit Test test_lobster_codebeamer.LobsterCodebeamerTest.test_no_retry_if_not_configured:100
Derived from:
tests_system/lobster_codebeamer/test_valid_flow.py
PyUnit Test test_valid_flow.LobsterCodebeamerTest.test_valid_query_id:23
Derived from:
PyUnit Test test_valid_flow.LobsterCodebeamerTest.test_references_tracing_tag_added:70
Derived from:

Software Tests

tests_unit/lobster_codebeamer/test_authentication.py
PyUnit Test test_authentication.AuthenticationTest.test_get_bearer_auth:13
Issues:
  • missing up reference
PyUnit Test test_authentication.AuthenticationTest.test_get_basic_auth:29
Issues:
  • missing up reference
tests_unit/lobster_codebeamer/test_codebeamer.py
PyUnit Test test_codebeamer.QueryCodebeamerTest.test_get_query_with_ID:52
Issues:
  • missing up reference
PyUnit Test test_codebeamer.QueryCodebeamerTest.test_get_query_with_query:88
Issues:
  • missing up reference
PyUnit Test test_codebeamer.QueryCodebeamerTest.test_get_query_with_invalid_data:122
Issues:
  • missing up reference
PyUnit Test test_codebeamer.QueryCodebeamerTest.test_get_single_item:134
Issues:
  • missing up reference
PyUnit Test test_codebeamer.QueryCodebeamerTest.test_get_single_item_invalid_id:149
Issues:
  • missing up reference
PyUnit Test test_codebeamer.QueryCodebeamerTest.test_get_many_items:156
Issues:
  • missing up reference
PyUnit Test test_codebeamer.QueryCodebeamerTest.test_import_tagged:175
Issues:
  • missing up reference
PyUnit Test test_codebeamer.ParseYamlTests.test_codebeamer_base:211
Issues:
  • missing up reference
tests_unit/lobster_codebeamer/test_codebeamer_schema.py
PyUnit Test test_codebeamer_schema.CbConfigTest.test_missing_config_file:20
Issues:
  • missing up reference
PyUnit Test test_codebeamer_schema.CbConfigTest.test_directory_given_as_config:33
Issues:
  • missing up reference
PyUnit Test test_codebeamer_schema.CbConfigTest.test_missing_config_field:46
Issues:
  • missing up reference
PyUnit Test test_codebeamer_schema.CbConfigTest.test_unsupported_config_keys:59
Issues:
  • missing up reference
PyUnit Test test_codebeamer_schema.CbConfigTest.test_cb_config_without_credentials_no_netrc:70
Issues:
  • missing up reference
PyUnit Test test_codebeamer_schema.CbConfigTest.test_cb_config_with_token:91
Issues:
  • missing up reference
PyUnit Test test_codebeamer_schema.CbConfigTest.test_cb_config_with_user_pass:108
Issues:
  • missing up reference
PyUnit Test test_codebeamer_schema.CbConfigTest.test_cb_config_with_netrc:122
Issues:
  • missing up reference
PyUnit Test test_codebeamer_schema.CbConfigTest.test_cb_config_with_netrc_and_sub_root:134
Issues:
  • missing up reference
PyUnit Test test_codebeamer_schema.CbConfigTest.test_cb_config_netrc_missing_machine_entry:146
Issues:
  • missing up reference

LOBSTER Version: 0.14.4-dev