SAP JCo parse XML to IDOC with field extensions

Question

I have some problems parsing an XML to IDOC using the SAP JCo library. I know I need a connection to the SAP system which is given.

Here is my gradle test code :

     when:
     JCoIDocServer server = JCoIDoc.getServer(sapProperties.get("jco.server.progid"))

     IDocXMLProcessor xmlProcessor = JCoIDoc.getIDocFactory().getIDocXMLProcessor()
     IDocDocumentList iDocDocumentList = xmlProcessor.parse(server.getIDocRepository(), getClass().getResourceAsStream("/data/DEBMAS-406868135.XML"), IDocXMLProcessor.PARSE_IGNORE_UNKNOWN_FIELDS)

The connection works, but JCo finds inconsistency in my XML (exported from SAP, no manual changes!) which seems caused by SAP extension fields:

com.sap.conn.idoc.IDocParseException: (7) IDOC_ERROR_PARSE_FAILURE: IDoc type extension ZDEBMAS3 within the EDI_DC40 control record segment does not match the IDoc-XML root tag :

state=READING_ENDTAG, charPosition=852, lineNumber=24, columnNumber=19

As you can see I tried to use XML parser options which did not help.

What do I have to do the extension fields are accepted by the IDocParser?

Update 1 Here you can find the anonymized IDOC XML.

<?xml version="1.0"?>
<DEBMAS05>
    <IDOC BEGIN="1">
        <EDI_DC40 SEGMENT="1">
            <TABNAM><![CDATA[EDI_DC40]]></TABNAM>
            <MANDT>100</MANDT>
            <DOCNUM>0000000406868135</DOCNUM>
            <DOCREL>750</DOCREL>
            <STATUS>12</STATUS>
            <DIRECT>1</DIRECT>
            <OUTMOD>2</OUTMOD>
            <IDOCTYP>DEBMAS05</IDOCTYP>
            <CIMTYP>ZDEBMAS3</CIMTYP>
            <MESTYP><![CDATA[Z_DEBMAS2]]></MESTYP>
            <SNDPOR>XXX</SNDPOR>
            <SNDPRT>LS</SNDPRT>
            <SNDPRN><![CDATA[P02_100]]></SNDPRN>
            <RCVPOR>INTERNET</RCVPOR>
            <RCVPRT>LS</RCVPRT>
            <RCVPRN>INTERNET</RCVPRN>
            <CREDAT>20190503</CREDAT>
            <CRETIM>085059</CRETIM>
            <SERIAL>20190503081911</SERIAL>
        </EDI_DC40>
        <E1KNA1M SEGMENT="1">
            <MSGFN>004</MSGFN>
            <KUNNR>XXX</KUNNR>
            <ANRED>XXX</ANRED>
            <BBBNR>0000000</BBBNR>
            <BBSNR>00000</BBSNR>
            <BUBKZ>0</BUBKZ>
            <KTOKD>XXX</KTOKD>
            <LAND1>XX</LAND1>
            <NAME1>XXX</NAME1>
            <ORT01>XXX</ORT01>
            <ORT02>XXX</ORT02>
            <PSTLZ>XXX</PSTLZ>
            <REGIO>XXX</REGIO>
            <SORTL>XXX</SORTL>
            <SPRAS>XXX</SPRAS>
            <STRAS>XXX</STRAS>
            <LZONE>0000000001</LZONE>
            <UMJAH>0000</UMJAH>
            <JMZAH>000000</JMZAH>
            <JMJAH>0000</JMJAH>
            <KATR3>XXX</KATR3>
            <KATR7>XXX</KATR7>
            <KATR8>XXX</KATR8>
            <STKZN>X</STKZN>
            <UMSA1>0</UMSA1>
            <HZUOR>00</HZUOR>
            <CIVVE>X</CIVVE>
            <SPRAS_ISO>DE</SPRAS_ISO>
            <ZE1BPAD SEGMENT="1">
                <PARNR>XXX</PARNR>
                <TITLE_P>XXX</TITLE_P>
                <FIRSTNAME>XXX</FIRSTNAME>
                <LASTNAME>XXX</LASTNAME>
                <SORT1_P>XXX</SORT1_P>
                <NAMCOUNTRY>DE</NAMCOUNTRY>
                <NAMCTRYISO>DE</NAMCTRYISO>
                <LANGU_P>D</LANGU_P>
                <LANGUP_ISO>DE</LANGUP_ISO>
            <C_O_NAME>XXX</C_O_NAME>
            <DISTRICT>XXX</DISTRICT>
            <CITY>XXX</CITY>
            <POSTL_COD1>XXX</POSTL_COD1>
            <REGION>XXX</REGION>
            <TRANSPZONE>0000000001</TRANSPZONE>
            <STREET>XXX.</STREET>
            <HOUSE_NO>XXX</HOUSE_NO>
            <PERS_GROUP>XXX</PERS_GROUP>
            <ADDR_GROUP>XXX</ADDR_GROUP>
            <E_MAIL>XXX</E_MAIL>
        </ZE1BPAD>
        <E1KNA11 SEGMENT="1">
            <KNURL>/</KNURL>
            <J_1KFREPRE>/</J_1KFREPRE>
            <J_1KFTBUS>/</J_1KFTBUS>
            <J_1KFTIND>/</J_1KFTIND>
            <PSOIS>/</PSOIS>
            <PSON1>/</PSON1>
            <PSON2>/</PSON2>
            <PSON3>/</PSON3>
            <PSOVN>/</PSOVN>
            <PSOTL>/</PSOTL>
            <PSOO1>/</PSOO1>
            <PSOO2>/</PSOO2>
            <PSOO3>/</PSOO3>
            <PSOO4>/</PSOO4>
            <PSOO5>/</PSOO5>
            <STCD5>/</STCD5>
            <SUFRAMA>/</SUFRAMA>
            <RG>/</RG>
            <EXP>/</EXP>
            <UF>/</UF>
            <RGDATE>/</RGDATE>
            <RIC>/</RIC>
            <RNE>/</RNE>
            <RNEDATE>/</RNEDATE>
            <CNAE>/</CNAE>
            <LEGALNAT>/</LEGALNAT>
            <CRTN>/</CRTN>
            <ICMSTAXPAY>/</ICMSTAXPAY>
            <INDTYP>/</INDTYP>
            <TDT>/</TDT>
            <COMSIZE>/</COMSIZE>
            <DECREGPC>/</DECREGPC>
            <CVP_XBLCK>/</CVP_XBLCK>
        </E1KNA11>
        <E1KNVVM SEGMENT="1">
            <MSGFN>XXX</MSGFN>
            <VKORG>XXX</VKORG>
            <VTWEG>XXX</VTWEG>
            <SPART>XXX</SPART>
            <VERSG>1</VERSG>
            <KALKS>XXX</KALKS>
            <KONDA>XXX</KONDA>
            <AWAHR>XXX</AWAHR>
            <INCO1>FH</INCO1>
            <INCO2>XXX</INCO2>
            <ANTLF>XXX</ANTLF>
            <KZTLF>XXX</KZTLF>
            <LPRIO>XXX</LPRIO>
            <WAERS>XXX</WAERS>
            <KTGRD>XXX</KTGRD>
            <ZTERM>XXX</ZTERM>
            <VWERK>XXX</VWERK>
            <KVGR4>XXX</KVGR4>
            <KVGR5>XXX</KVGR5>
            <UEBTO>XXX</UEBTO>
            <UNTTO>XXX</UNTTO>
            <PVKSM>XXX</PVKSM>
            <PODTG>XXX</PODTG>
            <BLIND>/</BLIND>
            <CARRIER_NOTIF>/</CARRIER_NOTIF>
            <CVP_XBLCK_V>/</CVP_XBLCK_V>
            <INCOV>/</INCOV>
            <INCO2_L>/</INCO2_L>
            <INCO3_L>/</INCO3_L>
            <ZE1KNVV SEGMENT="1">
                <HASH_CODE>XXX</HASH_CODE>
            </ZE1KNVV>
            <E1KNVPM SEGMENT="1">
                <MSGFN>XXX</MSGFN>
                <PARVW>XXX</PARVW>
                <KUNN2>XXX</KUNN2>
                <PARZA>XXX</PARZA>
            </E1KNVPM>
            <E1KNVPM SEGMENT="1">
                <MSGFN>XXX</MSGFN>
                <PARVW>XXX</PARVW>
                <KUNN2>XXX</KUNN2>
                <PARZA>XXX</PARZA>
            </E1KNVPM>
            <E1KNVPM SEGMENT="1">
                <MSGFN>XXX</MSGFN>
                <PARVW>WE</PARVW>
                <KUNN2>XXX</KUNN2>
                <PARZA>XXX</PARZA>
            </E1KNVPM>
        </E1KNVVM>
        <E1KNB1M SEGMENT="1">
            <MSGFN>XXX</MSGFN>
            <BUKRS>XXX</BUKRS>
            <ZUAWA>XXX</ZUAWA>
            <AKONT>XXX</AKONT>
            <ZWELS>XXX</ZWELS>
            <ZTERM>XXX</ZTERM>
            <VZSKZ>XXX</VZSKZ>
            <ZINDT>XXX</ZINDT>
            <ZINRT>XXX</ZINRT>
            <FDGRV>XXX</FDGRV>
            <VLIBB>XXX</VLIBB>
            <VRSZL>XXX</VRSZL>
            <VRSPR>XXX</VRSPR>
            <VERDT>XXX</VERDT>
            <WEBTR>XXX</WEBTR>
            <DATLZ>XXX</DATLZ>
            <XZVER>XXX</XZVER>
            <KULTG>XXX</KULTG>
            <PERNR>XXX</PERNR>
            <GMVKZD>/</GMVKZD>
            <AVSND>/</AVSND>
            <SMTP_ADDR>/</SMTP_ADDR>
            <CVP_XBLCK_B>/</CVP_XBLCK_B>
            <E1KNB5M SEGMENT="1">
                <MSGFN>XXX</MSGFN>
                <MAHNA>XXX</MAHNA>
                <MADAT>XXX</MADAT>
                <MAHNS>XXX</MAHNS>
                <GMVDT>XXX</GMVDT>
            </E1KNB5M>
            <E1KNB5M SEGMENT="1">
                <MSGFN>XXX</MSGFN>
                <MABER>XXX</MABER>
                <MAHNA>XXX</MAHNA>
                <MADAT>XXX</MADAT>
                <MAHNS>XXX</MAHNS>
                <GMVDT>XXX</GMVDT>
            </E1KNB5M>
        </E1KNB1M>
        <E1KNKKM SEGMENT="1">
            <MSGFN>XXX</MSGFN>
            <KKBER>XXX</KKBER>
            <KLIMK>XXX</KLIMK>
            <KNKLI>XXX</KNKLI>
            <CTLPC>XXX</CTLPC>
            <DTREV>XXX</DTREV>
            <SBGRP>XXX</SBGRP>
            <NXTRV>XXX</NXTRV>
            <PAYDB>XXX</PAYDB>
            <REVDB>XXX</REVDB>
            <SBDAT>XXX</SBDAT>
            <DBEKR>XXX</DBEKR>
            <DBMON>XXX</DBMON>
        </E1KNKKM>
    </E1KNA1M>
    <EDI_DS40 SEGMENT="1">
        <MANDT>XXX</MANDT>
        <DOCNUM>XXX</DOCNUM>
        <LOGDAT>XXX</LOGDAT>
        <LOGTIM>XXX</LOGTIM>
        <STATUS>XXX</STATUS>
        <STAMNO>XXX</STAMNO>
        <UNAME><![CDATA[BTCSD_01]]></UNAME>
        <SEGNUM>XXX</SEGNUM>
    </EDI_DS40>
    <EDI_DS40 SEGMENT="1">
        <MANDT>XXX</MANDT>
        <DOCNUM>XXX</DOCNUM>
        <LOGDAT>XXX</LOGDAT>
        <LOGTIM>XXX</LOGTIM>
        <STATUS>XXX</STATUS>
        <STAMQU>XXX</STAMQU>
        <STAMID>XXX</STAMID>
        <STAMNO>XXX</STAMNO>
        <STATYP>XXX</STATYP>
        <STAPA1>XXX</STAPA1>
        <STAPA2>XXX</STAPA2>
        <STAPA3>XXX</STAPA3>
        <STAPA4>XXX</STAPA4>
        <STATXT><![CDATA[&, &, &, &.]]></STATXT>
        <UNAME><![CDATA[BTCSD_01]]></UNAME>
        <SEGNUM>XXX</SEGNUM>
    </EDI_DS40>
    <EDI_DS40 SEGMENT="1">
        <MANDT>XXX</MANDT>
        <DOCNUM>XXX</DOCNUM>
        <LOGDAT>XXX</LOGDAT>
        <LOGTIM>XXX</LOGTIM>
        <STATUS>XXX</STATUS>
        <STAMQU>XXX</STAMQU>
        <STAMID>XXX</STAMID>
        <STAMNO>XXX</STAMNO>
        <STAPA1><![CDATA[SAP_ALE_Z_DEBMAS2]]></STAPA1>
        <STAPA2>XXX</STAPA2>
        <STATXT>XXX</STATXT>
        <UNAME><![CDATA[BTCSD_01]]></UNAME>
        <REPID>XXX</REPID>
        <SEGNUM>XXX</SEGNUM>
    </EDI_DS40>
    <EDI_DS40 SEGMENT="1">
        <MANDT>XXX</MANDT>
        <DOCNUM>XXX</DOCNUM>
        <LOGDAT>XXX</LOGDAT>
        <LOGTIM>XXX</LOGTIM>
        <STATUS>XXX</STATUS>
        <STAMNO>XXX</STAMNO>
        <UNAME><![CDATA[BTCSD_01]]></UNAME>
        <REPID>RBDMOIND</REPID>
        <SEGNUM>XXX</SEGNUM>
    </EDI_DS40>
</IDOC>

Update 2

Just for testing I tried to create an IDOC manually with the following piece of code

     JCoIDocServer server = JCoIDoc.getServer(sapProperties.get("jco.server.progid"))
     IDocDocument idoc = JCoIDoc.getIDocFactory().createIDocDocument(server.getIDocRepository(), "DEBMAS05")
     idoc.setValue("MANDT", 100)
     idoc.setValue("IDOCTYP", "DEBMAS05")
     idoc.setValue("CIMTYP", "ZDEBMAS3")
     idoc.setIDocNumber("0000000406868135")


     JCoIDocSegment segment = idoc.rootSegment.addChild("E1KNA1M")

     segment.addChild("ZE1BPAD")

This leads to the same error as using the xmlProcessor.

But if I change this loc adding the IDOC extension type

 IDocDocument idoc = JCoIDoc.getIDocFactory().createIDocDocument(server.getIDocRepository(), "DEBMAS05","ZDEBMAS3")

the error disappears.

Update 3

I seem to mix up two problems. The first occurs while JCo parses the XML causing the exception described here.

In JCo 3.0.17 the causing implementation is in DefaultIDocXMLParser in line 1321

if (cimType.length() > 0 && !cimType.equals(this.openTags.getFirst())
    || cimType.length() == 0 && !iDocType.equals(this.openTags.getFirst())) {
  throw new IDocParseException("IDoc type " 
      + (cimType.length() > 0 ? "extension " + cimType : iDocType)
      + " within the EDI_DC40 control record segment does not match the IDoc-XML root tag <"
      + (String)this.openTags.getFirst() 
      + ">", this.state.name(), this.charPosition, this.lineNumber, this.columnNumber, 
      (String)null, 0);
}

To avoid this exception one can remove the element which causes the same exception as the programmatically created IDOC without the extension type.

The question is now: How can the XMLParser be configured to accept extensions?

Would you please show us the XML file content? – Trixx May 15 '19 at 06:46 — Trixx, May 15 '19 at 06:46

score 0 · Answer 1 · answered May 15 '19 at 06:46

0

You need to pass a valid reader or input stream. An XML file is not a resource for a class. So would you please try

new FileInputStream(new File("/data/DEBMAS-406868135.XML"))

instead?

answered May 15 '19 at 06:46

Trixx

1,796
1
15
18

In my Unit-Test the XML file is a class resource and it is found perfectly. – flexguse May 15 '19 at 07:12
I try to anonymize the IDOC XML and you will find it in my original question. – flexguse May 15 '19 at 07:20

score 0 · Accepted Answer · answered May 16 '19 at 06:50

0

Now after you added the IDoc-XML content I would say that the parser is indeed right with its error message. The IDoc-XML file content violates the IDoc-XML specification from SAP. In this case the root tag must be <ZDEBMAS3> instead of <DEBMAS05>.

The root tag must describe the most detailed IDoc type value, which means it has to contain the IDOCTYP field value as long as there is no more defining CIMTYP available. But as soon as there is an additional CIMTYP, then the root tag has to contain the CIMTYP field value instead. It's a strange rule but this is the rule.

So not the IDoc-XML parser is wrong but the one who produced the IDoc-XML file. I am afraid that there is no option for this IDoc-XML parser to accept this erroneous IDoc-XML data. You need to get the software corrected that produces this IDoc-XML file.

answered May 16 '19 at 06:50

Trixx

1,796
1
15
18

Thanks a lot for your reply! I ended up writing a custom SAXParser heavily inspired by the ibissource iaf https://github.com/ibissource/iaf/blob/master/sap/src/main/java/nl/nn/adapterframework/extensions/sap/jco3/IdocXmlHandler.java which is able to generate IDOCs from our invalid XML. Unfortunately I have no possibility to create valid IDoc-XML. – flexguse May 16 '19 at 08:43
Thanks for sharing. Parsing on your own is an option, of course. However, I would prefer the offered classes from SAP if possible. I think that the SAP implementation is faster here and most likely with less errors in more difficult cases. This parser from SAP is especially made for IDoc-XML which defines several rules that a standard XML parser cannot handle out-of-the-box. For example think of special character escaping rules that are needed when IDoc segments or fields contain chars that are in general allowed for IDocs but not for standard XML. But I don't want to be the killjoy here. ;-) – Trixx May 16 '19 at 09:57

SAP JCo parse XML to IDOC with field extensions

2 Answers2