3

I am using QuickFIX/N 1.8 and when it has to create the DataDictionary based on an XML, it fails because my FIX50SP1_TRTN.xml (provided by Thomson Reuters) contains one message (AllocationReport) with two components (TrdInstrmtLegGrp, InstrmtLegAllocGrp), and both components have a group with a same name (NoLegs - 555).

QuickFIX/N is trying to create a dictionary per message, containing the groups of all its components, where the key of each group is the id. Therefore, it's trying to insert the key 555 twice, throwing an Exception the second time.

System.ArgumentException: 'An item with the same key has already been added.'

\QuickFIXn\DataDictionary\DataDictionary.cs

else if(childNode.Name == "group")
{
    DDField fld = FieldsByName[childNode.Attributes["name"].Value];
    DDGrp grp = new DDGrp();
    XmlAttribute req = childNode.Attributes["required"];
    if (req != null && req.Value == "Y"
    && (componentRequired == null || componentRequired.Value == true))
    {
        ddmap.ReqFields.Add(fld.Tag);
        grp.Required = true;
    }
    if (!ddmap.IsField(fld.Tag))
    {
        ddmap.Fields.Add(fld.Tag, fld);
    }
    grp.NumFld = fld.Tag;
    parseMsgEl(childNode, grp);
    ddmap.Groups.Add(fld.Tag, grp); //########### It fails when the second group is processed ###########
}

Summarized content of my FIX50SP1_TRTN.xml

<fix major="5" minor="0">
  <header/>
  <trailer/>

  <messages>
    <message name="AllocationReport" msgtype="AS" msgcat="app">
      <component name="TrdInstrmtLegGrp" required="N"/>
      <component name="InstrmtLegAllocGrp" required="N"/>
    </message>
  </messages>

  <components>
    <component name="TrdInstrmtLegGrp">
      <group name="NoLegs" required="N"> <!-- 555 -->
         (content A)
      </group>
    </component>
    <component name="InstrmtLegAllocGrp">
      <group name="NoLegs" required="N">
         (content B)
      </group>
    </component>
  </components>

  <fields>
    <field number="555" name="NoLegs" type="NUMINGROUP"/>
  </fields>
</fix>

My questions:

  1. Is QuickFIX/N supposed to support this situation?
  2. Have you ever faced this problem? How did you solve it?
  3. Do you know some explicit constraint (in QuickFIX/N, or the FIX protocol itself) about this situation? (maybe there is an explicit restriction saying that a message cannot contain more than one component with a group with a same name).
Santiago
  • 31
  • 3

2 Answers2

2

I believe that this is not allowed by the FIX specification. I say this because the FIX 4.4 and FIX 5.0 specification documents both state:

A tag number (field) should only appear in a message once. If it appears more than once in the message it should be considered an error with the specification document.

In addition, if you search through a FIX specification document or a FIX data dictionary file for PartyID, you will see that the concept of a repeating group containing PartyID, PartyIDSource and PartyRole appears in several places, but each with different names for the repeating group and the fields it contains. There are:

  • NoPartyIDs
  • NoNestedPartyIDs
  • NoNested2PartyIDs
  • NoNested3PartyIDs
  • NoNestedParties4
  • NoDerivativeInstrumentParties
  • NoInstrumentParties
  • NoRootParties
  • NoSettlPartyIDs
  • NoTargetParties
  • NoUndlyInstrumentParties

All of those repeating groups contain three fields denoting a party ID, its source and its role, but the names of those fields is never reused from one repeating group to another, and the same repeating group is not used more than one in a message. For example, the ExecutionReport message contains:

  • NoPartyIDs
  • NoNestedPartyIDs
  • NoNested2PartyIDs
  • NoNested3PartyIDs
  • NoNestedParties4

There are a few other cases of structurally-similar-but-differently-named repeating groups in FIX.

My point is that if FIX intended to allow a repeating group to appear multiple times in a message, then there would not have been any need for FIX to define structurally-similar-but-differently-named repeating groups in the FIX specification.

I suggest you raise this concern with Thomson Reuters. I once encountered a similar issue with a different trading venue; it turned out I had been given a buggy draft version of the venue's FIX specification, and they were happy to send me an up-to-date, corrected version of their FIX specification.

Ciaran McHale
  • 2,126
  • 14
  • 21
1

This is an open issue, which I myself discovered last September.

I'm fairly sure that this DD situation is allowed in FIX, meaning this is a QF/n problem. (I'd wager money that some of the other QF language ports have this problem too.) Update: Per Ciaran McHale's excellent answer, this is not allowed in FIX.

I do intend to fix this, but I can't say when it'll happen. Update: Even though it's not allowed in FIX, I still want to support it in QF/n because counterparties do stupid things.

I don't have a good workaround right now. In my case, I was lucky that we didn't actually use that message in the application that I was developing, so I could just comment that problem group out of the DD.

Grant Birchmeier
  • 17,809
  • 11
  • 63
  • 98
  • Interesting answer by Ciaran, pointing out that using the same ID more than once isn't allowed by the FIX standard specification. – TT. Jun 12 '19 at 09:42
  • Thanks for the heads-up, TT. Ciaran pretty much nailed it. However, counterparties violate the spec all the time, so it still probably wouldn't hurt to add this support in anyway. – Grant Birchmeier Jun 13 '19 at 02:01
  • Perhaps using a special configuration flag to allow this. – TT. Jun 13 '19 at 04:24
  • No flag needed. Once it's implemented, there's no great reason to ever not allow it. – Grant Birchmeier Jun 13 '19 at 14:22
  • @Grant Birchmeier: I'm not sure it is a good idea for a FIX product to support this non-compliant behaviour for a tag of type `NUMINGROUP`, because it might open a can of worms. For example, if you provide such support, then are you also going to provide similar support for a tag whose type is not `NUMINGROUP`? And is such support going to be provided only for what is in a FIX data dictionary file? Or also for what appears in a transmitted FIX message? – Ciaran McHale Jun 13 '19 at 14:31
  • My thinking is purely practical. To my knowledge, at least 2 counterparties are doing this, so it's a need. (In my experience, counterparties don't often care if they're doing FIX wrong.) I know that I can fix the DD parser and message parser to support it. This would only be allowed with a DD (because you can't have repeating groups without one, of course). Plus, I'm fairly sure this already works for tags that are not NUMINGROUP, provided the tags are in different groups. But let's not get too excited. I'm not promising to do this anytime soon.... – Grant Birchmeier Jun 13 '19 at 14:48
  • An alternative approach would be as follows. Contact counterparties who are violating the FIX specification in their data dictionaries and ask them to fix the problem in their data dictionaries. If they so do, then the problem is solved. If they decline to do so, then you can make a copy of their data dictionary and modify it to fix the problem. This approach is likely to involve a few orders of magnitude less work than modifying a FIX product so it can cope with broken data dictionaries. – Ciaran McHale Jun 13 '19 at 15:11
  • I have experience trying to tell counterparties that they are wrong, and it doesn't always work :) . Beyond that, I don't think you can fix their DD to sidestep the issue; if they're going to send you a message that re-uses a group-counter-tag, I can't think of a way to hack your local DD to accept that. If I could, I would be the first person to recommend it, and I would have answered this question with it. – Grant Birchmeier Jun 13 '19 at 15:29
  • @CiaranMcHale I must admit that I've complained with counterparties, with different success. One changed ExecutionReport messages (invalid tags included), another declined to change their ExecutionReport messages (invalid transition of order state). It's a pain. Especially if you have one code base that needs to serve different flavors (misinterpretations really). – TT. Jun 13 '19 at 16:03
  • Grant and TT, thanks for describing your experiences of working with counterparties. I have worked with about 25 counterparties, and have encountered this type of problem with only one, and even that turned out to be due to an out-of-date, draft specification document. The other problem that TT mentions (invalid transition of order state) is something I have encountered more frequently, and have dealt with it (painfully at times) in application-level code. – Ciaran McHale Jun 13 '19 at 16:47