I have created a json extraction query from a string field in BigQuery following this answer. But interestingly this query is always working for a first few rows then it is returning empty result.
I checked whether the empty results are because they are malformed, but if I remove the rows of the first few correct output rows from the input table then again I am left with first few proper output rows and the rest of them are empty. It seems like BigQuery is getting exhausted after json extraction from string in first few rows.
#standardSQL
SELECT ObjectID,
ARRAY(
SELECT AS STRUCT
key,
ARRAY_LENGTH(SPLIT(value)) `count`,
ARRAY(SELECT cast(x as int64) FROM UNNEST(SPLIT(SUBSTR(value, 2 , LENGTH(value) - 2))) x) AS positions
FROM UNNEST(REGEXP_EXTRACT_ALL(JSON_EXTRACT(indexed_abstract, '$.InvertedIndex'), r'"[^"]+":\[[\d,]*?]')) pair,
UNNEST([STRUCT(REPLACE(SPLIT(pair, ':')[OFFSET(0)], '"', '') AS key, SPLIT(pair, ':')[OFFSET(1)] AS value)])
) keyword
FROM `project.dataset.objects_100`
The input table project.dataset.objects_100 is created from this newline delimited objects_100.json file.
When I run the above query on the input table in the output I get the correct result for the first 12 rows then for the following rows I get empty result which looks like below.
I am also putting this strange output of the query as newline delimited json file in this results_100.json file.
Can someone help me figure out why the output for later rows are becoming empty?
Edit:
Here are a few rows of json where the query returned empty:
{"objectID":"2876639571","indexed_abstract":"{\"IndexLength\":105,\"InvertedIndex\":{\"PROBLEM\":[0],\"TO\":[1],\"BE\":[2],\"SOLVED:\":[3],\"To\":[4],\"provide\":[5],\"an\":[6,68],\"optical\":[7,31],\"operation\":[8,32],\"processor\":[9],\"with\":[10,46,84],\"which\":[11],\"inputting/\":[12],\"outputting\":[13],\"from\":[14],\"plural\":[15,34],\"functional\":[16,20,35,98],\"elements\":[17,36,99],\"to\":[18],\"one\":[19],\"element\":[21],\"consisting\":[22],\"of\":[23,49,55,58],\"the\":[24,47,56,77,97],\"device\":[25,81],\"is\":[26,82],\"enabled.\":[27],\"SOLUTION:\":[28],\"Concerning\":[29],\"this\":[30,80],\"processor,\":[33],\"(thin\":[37],\"film\":[38],\"elements)\":[39],\"2,\":[40,100],\"4,\":[41,101],\"6\":[42,102],\"and\":[43,52,90,103],\"8\":[44],\"provided\":[45,83],\"monomolecule\":[48],\"organic\":[50],\"compound\":[51],\"nanoparticles\":[53],\"composed\":[54],\"aggregate/agglomerate\":[57],\"these\":[59],\"monomolecules\":[60],\"are\":[61],\"arranged\":[62],\"for\":[63,92],\"processing\":[64],\"information\":[65,74],\"by\":[66],\"operating\":[67],\"external\":[69],\"signal\":[70],\"upon\":[71],\"incidental\":[72],\"two-dimensional\":[73],\"light.\":[75],\"At\":[76],\"same\":[78],\"time,\":[79],\"light\":[85],\"sources\":[86],\"I,\":[87],\"II,\":[88],\"III\":[89],\"IV\":[91],\"transferring\":[93],\"image\":[94],\"signals\":[95],\"among\":[96],\"8.\":[104]}}"}
{"objectID":"2787463273","indexed_abstract":"{\"IndexLength\":131,\"InvertedIndex\":{\"A\":[0],\"permanent-magnetic\":[1],\"stepping\":[2,16,38],\"motor\":[3,17,22,39],\"is\":[4,18,23,121],\"disclosed.\":[5],\"According\":[6],\"to\":[7,118],\"the\":[8,21,52,68,79,101,107,115,119],\"utility\":[9],\"model,\":[10],\"noise\":[11],\"of\":[12,31,67,82,100],\"a\":[13,36,41,49,74,83,97],\"salient-pole\":[14],\"PM-type\":[15,37],\"reduced\":[19],\"when\":[20],\"driven.\":[24],\"In\":[25],\"an\":[26,45,92,111],\"inner\":[27,53,64,102],\"magnet\":[28,54,65,94,103],\"yoke\":[29,95,104],\"(111)\":[30,56,105],\"each\":[32],\"stator\":[33,47],\"for\":[34,62,90],\"forming\":[35],\"with\":[40,78],\"structure\":[42],\"combined\":[43],\"by\":[44],\"A-phase\":[46],\"and\":[48,59,96,110,126],\"B-phase\":[50],\"stator,\":[51],\"yoke\\u0027s\":[55],\"closing\":[57],\"pin\":[58],\"hole\":[60],\"arranged\":[61],\"engaging\":[63],\"yokes\":[66],\"two\":[69,87],\"stators\":[70],\"are\":[71],\"positioned\":[72],\"at\":[73],\"line\":[75,81,117],\"symmetry\":[76,108],\"position\":[77,85,120],\"center\":[80,116],\"middle\":[84],\"between\":[86],\"bumps\":[88],\"(111e)\":[89],\"fitting\":[91],\"outer\":[93],\"central\":[98],\"point\":[99],\"as\":[106],\"axis;\":[109],\"angle\":[112],\"theta\":[113],\"from\":[114],\"less\":[122],\"than\":[123,128],\"90\":[124],\"degrees\":[125],\"greater\":[127],\"30\":[129],\"degrees.\":[130]}}"}
{"objectID":"2140096340","indexed_abstract":"{\"IndexLength\":198,\"InvertedIndex\":{\"The\":[0],\"perceptual\":[1,25,141,183],\"subband\":[2,19],\"image\":[3,179],\"coder\":[4],\"(PIC)\":[5],\"introduced\":[6],\"by\":[7],\"Safranek\":[8],\"and\":[9,102,189],\"Johnston\":[10],\"(1989),\":[11],\"selects\":[12],\"a\":[13],\"noise\":[14,29],\"target\":[15,30],\"level\":[16,38],\"for\":[17,43,136,177],\"each\":[18,83,137],\"based\":[20,96],\"on\":[21,62,97,165],\"an\":[22,73,87],\"empirically\":[23],\"derived\":[24],\"masking\":[26,91],\"measure.\":[27,92],\"These\":[28],\"levels\":[31],\"are\":[32,155],\"used\":[33],\"to\":[34,57,86,111,130,150,161],\"set\":[35],\"the\":[36,40,63,66,78,90,98,105,112,114,128,132,143,157,166,169,182,194],\"quantization\":[37],\"in\":[39,156,193],\"DPCM\":[41],\"quantizer\":[42,79,133],\"every\":[44],\"particular\":[45],\"subband.\":[46,138],\"It\":[47],\"achieves\":[48,146],\"high\":[49],\"quality\":[50],\"output\":[51],\"at\":[52,82],\"bit\":[53],\"rates\":[54],\"from\":[55],\"0.1\":[56],\"0.9\":[58],\"bits/pixel\":[59],\"(bpp)\":[60],\"depending\":[61,164],\"complexity\":[64],\"of\":[65,89,104,159,168],\"image.\":[67,170],\"In\":[68,124],\"this\":[69],\"paper,\":[70],\"we\":[71],\"present\":[72],\"algorithm\":[74,172],\"that\":[75],\"locally\":[76],\"adapts\":[77],\"step\":[80,134],\"size\":[81,135],\"pixel\":[84],\"according\":[85],\"estimate\":[88,94],\"This\":[93],\"is\":[95,185,190],\"already\":[99],\"coded\":[100,108],\"pixels\":[101],\"predictions\":[103],\"not\":[106,118,191],\"yet\":[107],\"pixels.\":[109],\"Compared\":[110],\"PIC,\":[113],\"proposed\":[115,144],\"method\":[116,145],\"does\":[117],\"require\":[119],\"any\":[120],\"additional\":[121],\"side\":[122],\"information.\":[123],\"fact,\":[125],\"it\":[126],\"eliminates\":[127],\"need\":[129],\"transmit\":[131],\"For\":[139],\"comparable\":[140],\"quality,\":[142],\"compression\":[147,180],\"gains\":[148],\"up\":[149],\"40\":[151],\"percent.\":[152],\"Typical\":[153],\"values\":[154],\"order\":[158],\"20\":[160],\"30\":[162],\"percent,\":[163],\"nature\":[167],\"Our\":[171],\"has\":[173],\"also\":[174],\"better\":[175],\"performance\":[176],\"supra-threshold\":[178],\"since\":[181],\"error\":[184],\"distributed\":[186],\"more\":[187],\"evenly\":[188],\"concentrated\":[192],\"most\":[195],\"sensitive\":[196],\"regions.\":[197]}}"}
{"objectID":"44353701","indexed_abstract":"{\"IndexLength\":232,\"InvertedIndex\":{\"Severe\":[0],\"coronary\":[1,13,83],\"artery\":[2,84],\"stenosis\":[3],\"with\":[4,43,205],\"endothelial\":[5],\"injury\":[6],\"in\":[7,55,124,187,202,230],\"the\":[8,28,32,79,101,104,112,139,180,200,203],\"canine\":[9],\"model\":[10],\"induces\":[11],\"cyclic\":[12],\"flow\":[14,88,96,129],\"variations\":[15],\"(CFVs),\":[16],\"which\":[17],\"are\":[18],\"partially\":[19],\"due\":[20],\"to\":[21,45,100,195,215],\"spontaneous\":[22],\"platelet\":[23],\"aggregation\":[24],\"and\":[25,39,59,68,103,116,135,172],\"dislodgement\":[26],\"at\":[27],\"stenotic\":[29],\"site.\":[30],\"In\":[31],\"present\":[33],\"study,\":[34],\"we\":[35],\"used\":[36],\"anesthetized\":[37],\"open-chest\":[38],\"unsedated\":[40],\"closed-chest\":[41],\"dogs\":[42,189,204],\"CFVs\":[44,58,61,70,107,115,191,206,211],\"investigate\":[46],\"whether\":[47,60],\"oxidative\":[48],\"metabolic\":[49],\"burst\":[50],\"(hydrogen\":[51],\"peroxide\":[52,122,227],\"generation)\":[53],\"occurred\":[54],\"neutrophils\":[56,125],\"during\":[57,210],\"were\":[62,71],\"attenuated\":[63],\"by\":[64,73,91,110,128],\"superoxide\":[65],\"dismutase\":[66],\"(SOD)\":[67],\"catalase.\":[69],\"produced\":[72],\"placing\":[74],\"a\":[75,94,168],\"cylindrical\":[76],\"constrictor\":[77],\"on\":[78],\"left\":[80],\"anterior\":[81],\"descending\":[82],\"(LAD).\":[85],\"LAD\":[86,118],\"blood\":[87,119],\"was\":[89,108,126,136,207],\"monitored\":[90],\"means\":[92],\"of\":[93,106,114,143,151,170],\"Doppler\":[95],\"probe\":[97],\"placed\":[98],\"proximally\":[99],\"constrictor,\":[102],\"severity\":[105],\"expressed\":[109,137],\"both\":[111],\"frequency\":[113],\"mean\":[117,140,181],\"flow.\":[120],\"Hydrogen\":[121],\"generation\":[123],\"measured\":[127],\"cytometry,\":[130],\"using\":[131],\"single\":[132],\"cell\":[133],\"analysis,\":[134],\"as\":[138],\"fluorescence\":[141,182],\"intensity\":[142,183,201],\"2\u2019,\":[144],\"7\u2019-dichlorofluorescein.\":[145],\"Dogs\":[146],\"received\":[147],\"an\":[148],\"intravenous\":[149],\"infusion\":[150],\"saline\":[152],\"(n\":[153],\"=\":[154,160,165,177],\"8),\":[155],\"SOD\":[156,171],\"(5\":[157],\"mg/kg,\":[158],\"n\":[159,164,176],\"7),\":[161,166],\"catalase\":[162,173],\"(5mg/kg,\":[163],\"or\":[167],\"combination\":[169],\"(same\":[174],\"doses,\":[175],\"7).\":[178],\"Although\":[179],\"did\":[184],\"not\":[185],\"change\":[186],\"sham-operated\":[188],\"without\":[190],\"(61.7\":[192],\"\u00b1\":[193,197,213,217],\"15.8\":[194],\"60.4\":[196],\"11.8;\":[198],\"NS),\":[199],\"significantly\":[208],\"increased\":[209],\"(62.2\":[212],\"13.7\":[214],\"79.8\":[216],\"9.8;\":[218],\"P\":[219],\"\\u003c\":[220],\"0.005).\":[221],\"These\":[222],\"results\":[223],\"indicate\":[224],\"that\":[225],\"hydrogen\":[226],\"is\":[228],\"generated\":[229],\"neutrophils.\":[231]}}"}
{"objectID":"2317258756","indexed_abstract":"{\"IndexLength\":227,\"InvertedIndex\":{\"Administration\":[0],\"of\":[1,16,32,57,155,212],\"single\":[2,30,70],\"dose\":[3,31,71],\"nevirapine\":[4],\"(sdNVP)\":[5],\"to\":[6,11,18,68,106,123,187],\"mothers\":[7,127],\"at\":[8,88],\"delivery\":[9],\"and\":[10,78,95,177,194,214,217],\"their\":[12],\"newborns\":[13],\"for\":[14,45],\"prevention\":[15],\"mother\":[17],\"child\":[19],\"HIV\":[20],\"transmission\":[21,25],\"(PMTCT)\":[22],\"significantly\":[23,203],\"decreases\":[24],\"risk.1,2\":[26],\"However\":[27],\"even\":[28],\"a\":[29],\"NVP\":[33,72,162],\"can\":[34,43],\"induce\":[35],\"non-nucleoside\":[36],\"reverse\":[37],\"transcriptase\":[38],\"inhibitor\":[39],\"(NNRTI)\":[40],\"resistance\":[41],\"which\":[42],\"persist\":[44],\"more\":[46,170],\"than\":[47,182],\"six\":[48],\"months.3\":[49],\"Recent\":[50],\"trials\":[51,81],\"therefore\":[52],\"evaluated\":[53],\"whether\":[54,189,219],\"the\":[55,131,172,190],\"effectiveness\":[56],\"NVP-based\":[58,119,147],\"versus\":[59],\"protease\":[60],\"inhibitor-based\":[61],\"antiretroviral\":[62],\"therapy\":[63],\"(ART)\":[64],\"might\":[65],\"differ\":[66],\"according\":[67],\"prior\":[69,99],\"exposure\":[73,100,163],\"in\":[74,126,137,151,159,198],\"both\":[75,103],\"mothers4,\":[76],\"5\":[77],\"children.6,7\":[79],\"Both\":[80],\"included\":[82],\"cohorts\":[83],\"who\":[84,96],\"had\":[85,97,134],\"received\":[86],\"sdNVP\":[87],\"least\":[89],\"6\":[90],\"months\":[91],\"before\":[92],\"enrollment\":[93],\"(PrNVP)\":[94],\"no\":[98,207],\"(no\":[101],\"PrNVP);\":[102],\"randomized\":[104],\"participants\":[105],\"either\":[107],\"NVP-\":[108,193],\"or\":[109],\"lopinavir/ritonavir\":[110],\"(LPV/r)\":[111],\"\u2013\":[112],\"based\":[113],\"ART.\":[114],\"ACTG\":[115],\"5208\":[116],\"demonstrated\":[117],\"that\":[118,146],\"ART\":[120,125,148,196],\"was\":[121,149],\"inferior\":[122,150],\"LPV/r-based\":[124,195],\"with\":[128,200,206],\"PrNVP4,\":[129],\"but\":[130],\"two\":[132],\"regimens\":[133],\"similar\":[135],\"efficacy\":[136],\"previously\":[138],\"unexposed\":[139],\"mothers5.\":[140],\"In\":[141],\"contrast,\":[142],\"IMPAACT\":[143],\"P1060\":[144],\"found\":[145],\"young\":[152],\"children\":[153,160,199],\"regardless\":[154],\"PrNVP6,7.\":[156],\"This\":[157],\"finding\":[158],\"without\":[161],\"has\":[164],\"major\":[165],\"implications\":[166],\"since\":[167],\"LPV/r\":[168],\"is\":[169,179],\"expensive,\":[171],\"liquid\":[173],\"formulation\":[174],\"requires\":[175],\"refrigeration,\":[176],\"it\":[178],\"less\":[180],\"palatable\":[181],\"NVP.8\":[183],\"Our\":[184],\"objectives\":[185],\"were\":[186],\"investigate\":[188],\"differences\":[191,221],\"between\":[192],\"observed\":[197],\"PrNVP\":[201],\"differed\":[202],\"from\":[204],\"those\":[205],\"PrNVP,\":[208],\"identify\":[209],\"other\":[210,225],\"predictors\":[211],\"virologic\":[213],\"clinical\":[215],\"response,\":[216],\"determine\":[218],\"treatment\":[220],\"depended\":[222],\"on\":[223],\"these\":[224],\"predictors.\":[226]}}"}
{"objectID":"2831392535","indexed_abstract":"{\"IndexLength\":202,\"InvertedIndex\":{\"PURPOSE:\":[0],\"To\":[1],\"simply\":[2],\"assemble\":[3],\"a\":[4,19,34,38,44,58,62,91,96,123,147,171,175,184,190,194],\"self-inking\":[5],\"stamp\":[6],\"by\":[7],\"coupling\":[8],\"both\":[9],\"control\":[10,51],\"straps\":[11,52,77,115,138],\"to\":[12,56,90,161],\"become\":[13,57],\"an\":[14,25,30,107,162],\"integral\":[15],\"U-shaped\":[16,59,86],\"member\":[17,60],\"via\":[18,61,183],\"connecting\":[20,63],\"piece\":[21,64,102,142],\"extended\":[22,105],\"upward\":[23],\"above\":[24,106],\"ink\":[26,108],\"pad,\":[27],\"and\":[28,140],\"providing\":[29],\"oscillating\":[31,81],\"protrusion\":[32,39],\"at\":[33,72,75],\"resilient\":[35,69],\"tongue\":[36],\"in\":[37,46,174],\"axis\":[40],\"direction\":[41],\"formed\":[42,71],\"of\":[43,95,113,122,131,136,179,197],\"slit\":[45],\"the\":[47,76,101,111,114,120,129,133,137,141,156,165,180,198],\"strap.\":[48],\"CONSTITUTION:\":[49],\"Both\":[50],\"12\":[53,116,139],\"are\":[54,70,159,168],\"coupled\":[55],\"14.\":[65],\"Tongues\":[66],\"16\":[67,158],\"being\":[68],\"slits\":[73],\"15\":[74],\"12,\":[78,88],\"which\":[79],\"hold\":[80],\"protrusions\":[82,166],\"17\":[83,167],\"for\":[84,193],\"supporting\":[85,172],\"members\":[87,134],\"14\":[89,103,143],\"short\":[92,176,199],\"side\":[93,177,200],\"wall\":[94,178],\"lower\":[97,148,195],\"portion\":[98,149,181],\"1.\":[99],\"Since\":[100],\"is\":[104,144],\"pad\":[109],\"11,\":[110],\"movements\":[112],\"do\":[117],\"not\":[118],\"disturb\":[119],\"movement\":[121],\"printing\":[124],\"plate\":[125],\"holder\":[126],\"8.\":[127],\"In\":[128,153],\"case\":[130],\"assembling,\":[132],\"made\":[135],\"introduced\":[145],\"into\":[146],\"1\":[150,182],\"from\":[151],\"below.\":[152],\"this\":[154],\"case,\":[155],\"tongues\":[157],\"urged\":[160],\"inside\":[163],\"until\":[164],\"engaged\":[169],\"with\":[170],\"opening\":[173],\"chamfered\":[185],\"part\":[186],\"18\":[187],\"serving\":[188],\"as\":[189],\"contact\":[191],\"surface\":[192],\"edge\":[196],\"wall.\":[201]}}"}
{"objectID":"1552493613","indexed_abstract":"{\"IndexLength\":187,\"InvertedIndex\":{\"Prion\":[0,177],\"diseases\":[1,44,107],\"are\":[2,12,53,57,108],\"neurodegenerative\":[3],\"disorders\":[4],\"associated\":[5],\"with\":[6],\"misfolding\":[7],\"of\":[8,31,36,40,85,101,134,153,160,170],\"prion\":[9,32,43,86,106,116,135,163],\"protein.\":[10],\"They\":[11],\"transmissible,\":[13],\"often\":[14],\"rapidly\":[15],\"progressive,\":[16],\"fatal\":[17],\"conditions\":[18],\"that\":[19,56,131],\"occur\":[20,93],\"in\":[21,77,94,114,180],\"both\":[22,95],\"humans\":[23],\"and\":[24,28,48,62,68,98,120,158],\"animals.\":[25],\"Sporadic,\":[26],\"acquired,\":[27],\"inherited\":[29,99],\"forms\":[30,61,100],\"disease\":[33,80,103,126,136],\"exist.\":[34],\"One\":[35],\"the\":[37,96,102,115,154,161,174],\"distinct\":[38],\"characteristics\":[39],\"all\":[41,60],\"human\":[42,162],\"is\":[45,129,137,143],\"their\":[46],\"clinical\":[47,54,155],\"pathological\":[49],\"heterogeneity;\":[50],\"however,\":[51],\"there\":[52],\"features\":[55],\"common\":[58],\"to\":[59,92,148],\"these\":[63],\"include\":[64],\"progressive\":[65],\"cognitive\":[66],\"impairment\":[67],\"movement\":[69],\"disorders,\":[70],\"including\":[71],\"chorea.\":[72],\"Chorea\":[73],\"occurs\":[74],\"most\":[75],\"frequently\":[76],\"variant\":[78],\"Creutzfeldt-Jakob\":[79],\"(vCJD),\":[81],\"an\":[82],\"acquired\":[83],\"form\":[84],\"disease,\":[87],\"but\":[88],\"has\":[89],\"been\":[90],\"reported\":[91],\"sporadic\":[97],\"too.\":[104],\"Inherited\":[105],\"caused\":[109],\"by\":[110,166],\"autosomal\":[111],\"dominant\":[112],\"mutations\":[113],\"protein\":[117],\"gene\":[118,141],\"(PRNP)\":[119],\"can\":[121],\"be\":[122],\"mistaken\":[123],\"for\":[124],\"Huntington\u2019s\":[125],\"(HD).\":[127],\"It\":[128],\"important\":[130],\"a\":[132,150,184],\"diagnosis\":[133],\"considered\":[138],\"if\":[139],\"HD\":[140],\"testing\":[142],\"negative.\":[144],\"This\":[145],\"chapter\":[146],\"aims\":[147],\"give\":[149],\"general\":[151],\"overview\":[152],\"features,\":[156],\"investigations,\":[157],\"pathophysiology\":[159],\"diseases,\":[164],\"accompanied\":[165],\"representative\":[167],\"case\":[168],\"reports\":[169],\"patients\":[171],\"seen\":[172],\"at\":[173],\"NHS\":[175],\"National\":[176],\"Clinic,\":[178],\"UK,\":[179],\"whom\":[181],\"chorea\":[182],\"was\":[183],\"prominent\":[185],\"feature.\":[186]}}"}
{"objectID":"1497663348","indexed_abstract":"{\"IndexLength\":1,\"InvertedIndex\":{\"\u201c\u4ea4\u8fd8\u201d\u4e00\u8bcd,\u5728\u5927\u591a\u6570\u8f9e\u4e66\u4e2d\u89e3\u91ca\u4e3a\u5f52\u8fd8,\u8fd9\u662f\u65e0\u53ef\u7f6e\u7591\u7684;\u7136\u800c\u9664\u6b64\u4e4b\u5916,\u5728\u4e70\u5356\u3001\u79df\u8d41\u3001\u5178\u5f53\u7b49\u5185\u5bb9\u7684\u5951\u7ea6\u6587\u4e66\u4e2d,\u5b83\u8fd8\u53ef\u4ee5\u8868\u793a\u4ea4\u7ed9\u3001\u4ed8\u7ed9,\u5176\u4e2d\u7684\u201c\u8fd8\u201d\u6ca1\u6709\u8868\u4e49,\u8bcd\u4e49\u7531\u201c\u4ea4\u201d\u5b57\u627f\u62c5,\u5373\u201c\u4ea4\u8fd8\u201d\u53d1\u5c55\u4e3a\u4e00\u4e2a\u504f\u4e49\u590d\u8bcd.\":[0]}}"}
