Discrepancy in creating defect between UI and restApi

Question

When I create a defect on UI, I cannot assign it only to test case, user story is getting assigned automatically. If I try to delete the User Story, test case also disappears.

But through restApi, I can relate only to Test Case, without mentioning the User Story.

I think it is a bug, since behavior should be similar regardless how you create a Defect.

I did not mention it, but we have always Test Cases associated with User Stories.

nickm · Answer 1 · 2014-03-26T15:45:38.667

It is possible to associate a defect to both a user story and a testcase via API and UI. But this happens in a special case when you first associate a story to a test case, and then associate a defect to this story, and next, you click on TestCase field on the defect's details page and it will present the only option in the picker - the test case already associated with the story.

What follows is confusing.

There are two ways to associate a Test Case to a Defect in Rally UI:

using "Test Case" field on the Details page

using "Test Cases" link on the left of the Details page

Not only this allows inconsistent practices in the UI, but also causes confusion when users examine results of WS API requests.

Let's say an existing TestCase is associated with a Defect using "Test Case" field in on the Defect's details page, and then a new Test Case is associated with the same Defect using "Test Cases" link on the left side.

At this point we have two separate TestCases associated with the same Defect : TC7 and TC59.

However when we query this Defect and its TestCases collection it shows only one TestCase, (TC59 in my example), which was added via "Test Cases" link on the left.

Here is the first query that fetches TestCases collection:

https://rally1.rallydev.com/slm/webservice/v2.0/defect?workspace=https://rally1.rallydev.com/slm/webservice/v2.0/workspace/1111&query=(FormattedID%20%3D%20DE19)&fetch=TestCases

And result returned by the first query:

{ 
QueryResult: { 
_rallyAPIMajor: "2", 
_rallyAPIMinor: "0", 
Errors: [ ], 
Warnings: [ ], 
TotalResultCount: 1, 
StartIndex: 1, 
PageSize: 20, 
Results: [ 
{ 
_rallyAPIMajor: "2", 
_rallyAPIMinor: "0", 
_ref: "https://rally1.rallydev.com/slm/webservice/v2.0/defect/12655123753", 
_refObjectUUID: "91f66a35-d860-41e0-a10d-db6dad460f28", 
_objectVersion: "7", 
_refObjectName: "Error found in US20: story C2", 
TestCases: { 
_rallyAPIMajor: "2", 
_rallyAPIMinor: "0", 
_ref: "https://rally1.rallydev.com/slm/webservice/v2.0/Defect/12655123753/TestCases", 
_type: "TestCase", 
Count: 1 
}, 
_type: "Defect" 
} 
] 
} 
}

Using the URI of the TestCases collection we got from the result of the first query we ran another query to examine the collection itself, fetching only FormattedID and WorkProduct for brevity:

https://rally1.rallydev.com/slm/webservice/v2.0/Defect/12655123753/TestCases?fetch=FormattedID,WorkProduct

The result includes only TC59:

https://rally1.rallydev.com/slm/webservice/v2.0/Defect/12655123753/TestCases?fetch=FormattedID,WorkProduct 

{ 
QueryResult: { 
_rallyAPIMajor: "2", 
_rallyAPIMinor: "0", 
Errors: [ ], 
Warnings: [ ], 
TotalResultCount: 1, 
StartIndex: 1, 
PageSize: 20, 
Results: [ 
{ 
_rallyAPIMajor: "2", 
_rallyAPIMinor: "0", 
_ref: "https://rally1.rallydev.com/slm/webservice/v2.0/testcase/17593081118", 
_refObjectUUID: "dbdbd621-97f7-4c27-b24a-f0ca92be73ff", 
_objectVersion: "1", 
_refObjectName: "Error found in US20: story C2", 
FormattedID: "TC59", 
WorkProduct: { 
_rallyAPIMajor: "2", 
_rallyAPIMinor: "0", 
_ref: "https://rally1.rallydev.com/slm/webservice/v2.0/defect/12655123753", 
_refObjectUUID: "91f66a35-d860-41e0-a10d-db6dad460f28", 
_objectVersion: "7", 
_refObjectName: "Error found in US20: story C2", 
FormattedID: "DE19", 
_type: "Defect" 
}, 
_type: "TestCase" 
} 
] 
} 
}

Next, we want to create a new TestCase via Web Services API and associate it with the same Defect DE19. Important: The UI method of associating a new TestCase to a Defect via "Test Cases" link on the left is equivalent to the following POST request in the WS API.

In this example, "WorkProduct":"/defect/12655123753" references the ObjectID of the same defect .

Request URL:

https://rally1.rallydev.com/slm/webservice/v2.0/testcase/create?key=7b193d4b....

Payload:

{ 
"TestCase":{ 
"Name": "some tc", 
"WorkProduct":"/defect/12655123753" 
} 
}

We verify in the UI that the WS API create request was successful. Next we query the same TestCases collection and see two results, as expected. This time only FormattedID is fetch for brevity. TC59 and TC60 are returned.

https://rally1.rallydev.com/slm/webservice/v2.0/Defect/12655123753/TestCases?fetch=FormattedID

As we see, TC7 that was created in the UI via "Test Case" field on the details page of Defect is not being returned by this query. In Web Services API Defect object has two attributes relevant to this discussion: TestCase and TestCases.

This query:

https://rally1.rallydev.com/slm/webservice/v2.0/defect?workspace=https://rally1.rallydev.com/slm/webservice/v2.0/workspace/1111&query=(FormattedID%20%3D%20DE19)&fetch=TestCase,FormattedID

returns TC7:

{ 
QueryResult: { 
_rallyAPIMajor: "2", 
_rallyAPIMinor: "0", 
Errors: [ ], 
Warnings: [ ], 
TotalResultCount: 1, 
StartIndex: 1, 
PageSize: 20, 
Results: [ 
{ 
_rallyAPIMajor: "2", 
_rallyAPIMinor: "0", 
_ref: "https://rally1.rallydev.com/slm/webservice/v2.0/defect/12655123753", 
_refObjectUUID: "91f66a35-d860-41e0-a10d-db6dad460f28", 
_objectVersion: "9", 
_refObjectName: "Error found in US20: story C2", 
FormattedID: "DE19", 
TestCase: { 
_rallyAPIMajor: "2", 
_rallyAPIMinor: "0", 
_ref: "https://rally1.rallydev.com/slm/webservice/v2.0/testcase/12476557645", 
_refObjectUUID: "a8d50053-e250-43c1-aebc-58472fec2988", 
_objectVersion: "10", 
_refObjectName: "myTS3", 
FormattedID: "TC7", 
_type: "TestCase" 
}, 
_type: "Defect" 
} 
] 
} 
}

Now that we know how to associate a TestCase to a Defect in a scenario when the TestCase is added to Defect's TestCases collection both via UI and WS API, we want to find an API equivalent of associating a TestCase to a Defect with TestCase field.

We will use a defect that does not have a story linked to it, since the original defect (DE19 in my example) has a US20 already linked. Per note on Defect's TestCase attribute in the WS API documentation, only either one has to be specified.

Note The test case that this defect is regarding. Either Requirement or TestCase can be specified, but not both.

We are going to add a pre-existing TestCase referenced by ObjectID 14604252931 to another Defect, DE81, referenced by Object ID 14710889543:

Request URL:

https://rally1.rallydev.com/slm/webservice/v2.0/defect/14710889543?key=7b193d4b-....

Payload:

{"Defect":{ 
"TestCase":"/testcase/14604252931" 
} 
}

Note that if at this point the second Defect DE81 is associated to a random story in the UI, the association with the TestCase will be removed without any error or warning.

Now this is becoming even more confusing. The first Defect's DE19 details page we see that it has both US20 and TC7 associated with it. It seem impossible to associate DE81 with a TestCase using "Test Case" field on the details page. When we click on the magnifying glass icon the picker dialog is empty. It turns out that the only way to associate DE81 to both a UserStory and a TestCase is to create a new TestCase associated with the UserStory.

The two steps are as follows: a. Link UserStory to a TestCase b. Go back to the Defect and click on "Test Case" field on the details page. This time the picker dialog will be populated with the new TestCase created above. Now both the TestCase and the UserStory are associated with this Defect.

If at this point an attempt is made to replace TC61 with some other TestCase that is not associated with the UserStory it will fail. If this attempt is made via API as shown below:

Request URL:

https://rally1.rallydev.com/slm/webservice/v2.0/defect/14710889543?key=7b193d4b-....

Payload:

{"Defect":{ 
"TestCase":"/testcase/14604252931" 
} 
}

it will result in this error:

{"OperationResult": {"_rallyAPIMajor": "2", "_rallyAPIMinor": "0", "Errors": ["Validation error: Defect.TestCase is an invalid relationship "], "Warnings": []}}

UPDATE re: case described in Stella's comment:

Created a UserStory in UI, called it StoryG /userstory/17824476422

Associated a new TestCase with it via TestCases link on the left of the details page. /testcase/17824476889

Now this TestCAse is associated with the story via TestCases collection on User Story.

Next, created a defect via API:

https://rally1.rallydev.com/slm/webservice/v2.0/defect/create?key=733c0893-0e5b-4c66-8204-9a2afa548d36

Payload:

{"Defect":{ 
"Name":"Bug with StoryG",
"Requirement":"/hierarchicalrequirement/17824476422",
"TestCase":"/testcase/17824476889"
}
}

This works: the new defect is associated with both the UserStory and the TestCase. Indeed this is consistent with the UI. When using API both the Requirement and the TestCase must be referenced in the payload. And when creating a defect in the UI, both have to be specified: please see the first paragraph of the response. When selecting a TestCase, the TestCase picker only displays one option: a TestCase already associated with the UserStory.

Below is a scenario that will not work, which is consistent between API and UI. Let's see what happens if I try to link unrelated UserStory and TestCase to the defect

Created a UserStory, StoryH /userstory/17824480704

Associated a new TestCase with it via TestCases link on the left of the details page. /testcase/17824481042

Now this TestCase is associated with the story via TestCases collection on User Story.

Next, I reference unrelated TestCase when creating a defect: /testcase/17824638756

https://rally1.rallydev.com/slm/webservice/v2.0/defect/create?key=733c0893-0e5b-4c66-8204-9a2afa548d36

Payload:

{"Defect":{ 
"Name":"Bug with StoryH",
"Requirement":"/hierarchicalrequirement/17824480704",
"TestCase":"/testcase/17824638756"
}
}

This returns error, as expected, and consistent with UI:

{"CreateResult": {"_rallyAPIMajor": "2", "_rallyAPIMinor": "0", "Errors": ["Validation error: Defect.TestCase is an invalid relationship "], "Warnings": []}}

There is another way of associating all members of this defect-userstory-testcase triangle. Here is where the discrepancy between UI and API can be observed:

In UI:

a) associate a user story with a test case, using TestCases link on the left of story's details page b) create new defect, associate it with a test case using Test Case field on the Defect's details page.

The story automatically shows after a test case is selected from the test case picker.

in API:

a) associate a user story with a test case, using TestCases link on the left of story's details page b) create a defect using browser's REST client, while setting its TestCase field, and expecting the story to be linked automatically.

https://rally1.rallydev.com/slm/webservice/v2.0/defect/create?key=5e10c58...

Payload:

{"Defect":{ 
"Name":"Bug with TC77",
"TestCase":"/testcase/17866645137"
}
}

Defects(1) link on the left of testcase details page shows the newly created defect. However the Defect's details page does not show the story. I have submitted a bug.

To recap this triangle as it appears in the WS API:

Defect - this artifact can have a reference to a UserStory via Requirement attribute and a reference to a TestCase via TestCase attribute. Per WSAPI documentation says it can have one or the other, but not both. When linking these three artifacts, they must all be consistent. If a defect is defined to point to a Userstory and a Testcase, then those two must also point to each other, which is effectively an exception to the rule of Either/Or. TestCase attribute on the Defect is entirely separate from the TestCases collection on a Defect.
UserStory - this artifact has TestCases collection and Defect collection.
TestCase - this artifact can have a pointer to a Requirement (which is a UserStory) via WorkProduct attribute.

Yes, it is very confusing, but I still believe that there is a bug. Here our scenario. We have UserStory US1, that has associated test case, let's call it TC1. After executing TC1 I want to file a defect through RestApi and specify that this defect belongs to TC1 that actually belongs to US1. Case 1: I do it through UI, then after selecting TC1, US1 automatically gets populated. If I try to delete US1 (un-associate it from this defect), TC1 association is also gone. — stella, Mar 24 '14 at 16:16
Case 2: I want to file a defect using RestApi and associate it to TC1, assuming that it also will trigger by itself association with US1. Because it is how UI works.
This is desired for me behavior. But Rally does not do it. It associates defect ONLY to test case, thus it is inconsistent from UI and does not give me correct defect count when I query it through User Story. — stella, Mar 24 '14 at 16:21
I added UPDATE section to my post above. When I reference both the Requirment and the TestCase fields in the payload both are associated successfully to the Defect that is being created via WS API as long as these two artifacts are already linked to each other. If I use unrelated TestCase I get an expected error: Defect.TestCase is an invalid relationship. — nickm, Mar 25 '14 at 00:20
Nick, I believe I was not clear. Will try to explain again. Desired behavior from WS API: as long as I associate defect with test case, the requirement (that is parent of this test case) also will be populated. This is how it happens on UI. But using WS API, it does not happen. — stella, Mar 25 '14 at 20:41
Stella, thank you. You are correct. There are many ways to create userstory-testcase-defect trianlge. I was associating a story to a testcase and then creating a defect while linking it to a story and in that scenario there is no discrepancy between UI and API. But if I associate a story to a testcase, and then create a defect while linking to a test case, there is a discrepancy between UI and API. The former adds a story association automatically, the latter does not. To achieve the 3 way link both the TestCase and Requirement references have to be included in the payload when using API. — nickm, Mar 26 '14 at 15:53
Thank you Nick.Is it an easy way to retrieve US based on TC number? Looks like I will have to do a lot of heavy lifting in a code to achieve desired result. We create US, then create TC for this US and then file a defect if Test Case fails. That is why I wanted to see US to be populated immediately after I file defect and specify TC - like it happens on UI. — stella, Mar 26 '14 at 15:59
Defect object has TestCase attribute, which is a reference to a test case it is linked to. After you get the reference, you may get the the actual TestCase object, which has WorkProduct attribute, which is a reference to the HierarchicalRequirement (user story). — nickm, Mar 26 '14 at 16:14
I tried WorkProduct to find US associated with the test case, but it does not coming back. — stella, Mar 26 '14 at 16:22
Has to correct myself, WorkProduct returns associated UserStory, please disregard my previous comment. Thank you. — stella, Mar 26 '14 at 16:33

Discrepancy in creating defect between UI and restApi

1 Answers1