Xray Test Management for Jira
Overview
Planview Hub offers seamless integration with Xray Test Management for Jira through the Atlassian Jira connector. This collaboration enables users to leverage the powerful test management capabilities of Xray within the Planview Hub platform.
For more information, please refer to the sections below.
Connector Setup Details
- Xray Cloud Client ID and Xray Cloud Client Secret must be provided in the Hub repository settings.
- Beginning in Hub version 24.4, Xray Cloud Location can be set from the repository schema in the Hub repository settings.
See additional connector setup details for the Jira connector here.
Other Configuration Settings
Cross-Artifact Linking
On-prem
Planview Hub: 22.2 and later
Xray Test Management for Jira: 6.21
Cross-artifact linking between Xray types are supported, including:
-
Xray Test
-
Pre-Conditions associated with a Test
-
Test Plans associated with a Test
-
Test Sets association with a Test
-
Test Repository Path
- Test Execution Associated
-
-
Test Plan
-
Tests Associated
-
Test Execution Associated
-
-
Test Set
-
Test Associated
-
-
Pre Condition
-
Test Associated
-
-
Test Execution
-
Test Associated
-
Cloud
Planview Hub: 22.4 and later
Xray Test Management for Jira (Cloud)
Cross-artifact linking between Xray artifact types are supported, including:
-
Xray Tests link to:
-
Pre-Conditions associated with a Test
-
Test Plans associated with a Test
-
Test Sets association with a Test
-
-
Test Plans link to:
-
Tests associated with Test Plan
-
-
Test Sets link to:
-
Test associated with Test Set
-
-
Preconditions link to:
-
Test associated with Precondition
-
-
*Test Executions and Sub-Test Executions link to:
-
Test Plans
-
Test Runs
-
Tests
-
-
*Tests link to:
-
Test Runs/Executions
-
-
*Test Plans link to:
-
Test Executions
-
-
*Test Runs link to:
-
Defects
-
Note: Cross-artifact linking for Test Executions, Sub-Test Executions, Tests, Test Plans, and Test Runs are only supported in Planview Hub version 23.4+.
Test Management
Learn more about supported test integration scenarios using Xray here.
Improper Configuration of Xray Test Fields
When "Xray Test" type test fields are associated with non-Xray issue types or are partially configured, it may lead to errors. The following details provide further insights into this issue.
Jira administrators can access the "Where is My Field" option in the issue screen (preferably the create issue screen) to validate the status of requirements for a field to be visible on the screen.
If an "Xray Issue Type" field, such as "Manual Test Steps," is configured to be present in the field configuration, project, and issue type but not in the corresponding issue type screen of the project, the status of the "Xray Issue Type" field requirements may appear as shown in the attached screenshot.
For the above configuration, API responses for Issue and "Issue Type" may be inconsistent with respect to the field, leading to an unexpected state in Hub. This may cause artifact activities to get blocked, even for the entire collection containing such a project. Therefore, it is advised to correctly configure the "Xray Issue Type" fields for Xray Issue Types and avoid associating them with non-Xray Test Issue types like "Story."
Non-Xray Issue Types:
For non-Xray Issue Types, all required configurations for the field related to the Xray Issue type in the "Create Issue" screen should be unchecked (crossed out) in the "Where is My Field" requirements pane, as shown below. Alternatively, ensure that at least the "Project and issue type scope" status is unchecked.
Xray Issue Types:
For Xray Issue Types, all required configurations for the field related to the Xray Issue type in the "Create Issue" screen should be checked in the "Where is My Field" requirements pane, as shown below.
Change Detection
Xray Cloud fields do not support change detection.
Supported Features
See additional details on supported features here.
Special Features Supported
|
Feature |
Custom Type Supported? |
Applicable Hub Version |
Applicable Repository Versions |
Default Maximum Size in Repository |
|---|---|---|---|---|
|
|
N/A |
Attachments
Supported Artifacts
Supported Work Items
Learn about the difference between containers and work items in Planview Hub here.
On-prem
Learn more about supported test integration scenarios using Xray here.
| Supported Artifact Type | Applicable Hub Versions | Applicable Repository Versions | Unique URL? |
|---|---|---|---|
|
Xray Test (as Test Case) |
Planview Hub: 22.3 and later Tasktop Sync: 4.31 and later |
Xray Test Management for Jira (On-prem) |
Yes |
|
Test Set |
Planview Hub: 22.3 and later Tasktop Sync: 4.31 and later |
Xray Test Management for Jira (On-prem) |
Yes |
|
Test Plan |
Planview Hub: 22.3 and later Tasktop Sync: 4.31 and later |
Xray Test Management for Jira (On-prem) |
Yes |
|
Test Execution |
Planview Hub: 22.3 and later Tasktop Sync: 4.31 and later |
Xray Test Management for Jira (On-prem) |
Yes |
|
Sub Test Execution |
Planview Hub: 22.3 and later Tasktop Sync: 4.31 and later |
Xray Test Management for Jira (On-prem) |
Yes |
|
Pre-Condition |
Planview Hub: 22.3 and later Tasktop Sync: 4.31 and later |
Xray Test Management for Jira (On-prem) |
Yes |
|
Xray Test Run |
Planview Hub: 22.3 and later Tasktop Sync: 4.31 and later |
Xray Test Management for Jira (On-prem) |
No |
Cloud
Learn more about supported test integration scenarios using Xray here.
Note: To detect these types, the descriptions of the types should not be changed from default.
|
Supported Artifact Type |
Applicable Hub Versions |
Applicable Repository Versions |
Unique URL? |
|---|---|---|---|
|
Xray Test (as Test Case) |
Planview Hub: 22.4 and later |
Xray Test Management for Jira (Cloud) |
Yes |
|
Test Set |
Planview Hub: 22.4 and later |
Xray Test Management for Jira (Cloud) |
Yes |
|
Test Plan |
Planview Hub: 22.4 and later |
Xray Test Management for Jira (Cloud) |
Yes |
|
Pre-Condition |
Planview Hub: 22.4 and later |
Xray Test Management for Jira (Cloud) |
Yes |
|
Test Run (as Test Result) |
Planview Hub: 23.4 and later |
Xray Test Management for Jira (Cloud) |
Yes |
| Test Executions | Planview Hub: 23.4 and later | Xray Test Management for Jira (Cloud) | Yes |
| Sub Test Executions | Planview Hub: 23.4 and later | Xray Test Management for Jira (Cloud) | Yes |
Supported Containers
Learn more about containment in Planview Hub here.
|
Containers that can synchronize between repositories |
Applicable Hub Versions |
Applicable Repository Versions |
Unique URL? |
|---|---|---|---|
|
Test Repository Folder Note: Beginning in Hub version 22.4, containers can be treated as work items. Learn more here. |
Planview Hub: 22.3 and later Tasktop Sync: 4.31 and later |
Any supported repository version: |
No |
|
Containers used to define the boundary of a collection (When clicking 'Manage Projects' on a Collection) |
|||
|
N/A |
|
|
|
|
Containers used for artifact routing |
|
|
|
|
N/A |
Supported Field Types
Note: See details on partially configured Xray type fields in the section above
On-prem
-
Only the following default fields are currently supported:
-
Test Step Action (Required)
-
Expected Result
-
Data
-
-
All custom fields are not supported.
-
Some custom fields are read-only.
Cloud
-
Only the following default fields are currently supported:
-
Test Step Action (Required)
-
Expected Result
-
Data
-
-
All custom fields are not supported
See additional details on supported field types in Jira here.
Functional Limitations
|
Category |
Limitation |
Applicable Hub Versions |
Applicable Repository Versions |
|---|---|---|---|
| Good to Know |
Custom Fields Xray On-prem includes advanced custom field types. These field types are typically used for the standard Xray fields, but can also be used for custom fields. Creating custom fields with these types may result in impaired functionality and unexpected behavior, even if the fields aren’t included in the integration. |
Planview Hub: 22.2 and later | Xray Test Management for Jira: 6.21 and later |
| Third Party API Limitation |
Change Detection Xray Cloud fields do not support change detection. |
Planview Hub: 22.4 and later | Xray Test Management for Jira (Cloud) |
| Third Party API Limitation |
Test Runs A Test Run is generated when a Test Execution is linked to a Test Case. Due to an API limitation, Hub cannot generate multiple Test Runs for a Test Case/Test Execution pair. |
Planview Hub: 22.3 and later | Xray Test Management for Jira (Cloud) |
| Third Party API Limitation |
Manual Tests Custom types, Generic and Cucumber are only partially supported. |
Planview Hub: 22.2 and later | Xray Test Management for Jira: 6.21 and later |
| Third Party API Limitation |
Manual Tests Scenario Type and Scenario are read-only. |
Planview Hub: 22.2 and later | Xray Test Management for Jira: 6.21 and later |
| Third Party API Limitation |
Manual Tests Definition is read-only. |
Planview Hub: 22.2 and later | Xray Test Management for Jira: 6.21 and later |
| Third Party API Limitation |
Test Steps Reorder of Test Steps is not supported by the API. |
Planview Hub: 22.2 and later | Xray Test Management for Jira: 6.21 and later |
| Third Party API Limitation |
Test Steps Test Call in Test Step is not currently supported. |
Planview Hub: 22.2 and later | Xray Test Management for Jira: 6.21 and later |
| Third Party API Limitation |
Test Steps All custom fields are not supported |
Planview Hub: 22.2 and later | Xray Test Management for Jira: 6.21 and later |
| Third Party API Limitation |
Test Steps Updates to the Test Step performed via the API do not mark an artifact as updated. Note: This does not affect updates done via the UI. |
Planview Hub: 22.2 and later | Xray Test Management for Jira: 6.21 and later |
| Third Party API Limitation |
Test Runs Attachments are not supported. |
Planview Hub: 22.2 and later | Xray Test Management for Jira: 6.21 and later |
| Third Party API Limitation |
Test Runs Test Step Results on Test Runs are not supported. |
Planview Hub: 22.2 and later | Xray Test Management for Jira: 6.21 and later |
| Third Party API Limitation |
Test Runs Defects Associated With a Test Run are not supported. |
Planview Hub: 22.2 and later | Xray Test Management for Jira: 6.21 and later |
| Third Party API Limitation |
Xray Tests Updates to test steps require the test to have a test type where the kind is “Steps” (e.g., Manual). |
Planview Hub: 22.4 and later | Xray Test Management for Jira (Cloud) |
| Third Party API Limitation |
Xray Tests Updates to unstructured definition require the test to have a test type where the kind is “Unstructured” (e.g., Generic). |
Planview Hub: 22.4 and later | Xray Test Management for Jira (Cloud) |
| Third Party API Limitation |
Xray Tests Updates to the gherkin scenario require the test to have a test type where the kind is “Gherkin” (e.g., Cucumber). |
Planview Hub: 22.4 and later | Xray Test Management for Jira (Cloud) |
| Third Party API Limitation |
Test Steps Reorder of Test Steps is not supported by the API. |
Planview Hub: 22.4 and later | Xray Test Management for Jira (Cloud) |
| Third Party API Limitation |
Test Steps Test Call in Test Step is not currently supported. |
Planview Hub: 22.4 and later | Xray Test Management for Jira (Cloud) |
| Third Party API Limitation |
Test Steps Updates to the Test Step do not mark an artifact as updated. |
Planview Hub: 22.4 and later | Xray Test Management for Jira (Cloud) |
| Third Party API Limitation |
Test Runs The 'Started On,' 'Finished On,' 'Assignee,' and 'Executed By' fields cannot be emptied once set. |
Planview Hub: 22.4 and later | Xray Test Management for Jira (Cloud) |
| Third Party API Limitation |
Test Runs The 'Finished On' field cannot be updated until the Test Run’s status is in a final state. |
Planview Hub: 22.4 and later | Xray Test Management for Jira (Cloud) |
| Third Party API Limitation |
Test Runs Test Run Steps are supported as read-only fields. |
Planview Hub: 22.4 and later | Xray Test Management for Jira (Cloud) |