how to use expath and Xquery to fetch a CSV file?

Question

How can this example of expath be adapted to grab and store a CSV file directly from a website?

xquery version "3.1";

import module namespace hc = "http://expath.org/ns/http-client";

let $url := "https://www.kingjamesbibleonline.org/Genesis-Chapter-1_Original-1611-KJV/"
let $request := 
    <hc:request href="{$url}" method="GET">
        <hc:header name="Connection" value="close"/>    
    </hc:request>
return
    hc:send-request($request)

Browsing to:

http://www.bccdc.ca/health-info/diseases-conditions/covid-19/data

the specific CSV file is:

http://www.bccdc.ca/Health-Info-Site/Documents/BCCDC_COVID19_Dashboard_Case_Details.csv

Of course, there are a multitude of ways to grab that file. But, how would it be done with xquery as above?

Obviously, changing the URL is the starting point, which, indeed, results in a large CSV file printed to the console.

BaseX allows you to do `csv:doc('http://www.bccdc.ca/Health-Info-Site/Documents/BCCDC_COVID19_Dashboard_Case_Details.csv')`, for instance, without the complications of using EXPath and a HTTP request/response mode. The CSV module is documented at https://docs.basex.org/wiki/CSV_Module. If you wanted the plain text then there is `unparsed-text('http://www.bccdc.ca/Health-Info-Site/Documents/BCCDC_COVID19_Dashboard_Case_Details.csv')` — Martin Honnen, Dec 13 '20 at 13:21
syntax or other error: https://stackoverflow.com/q/65325993/4531180 with `csv:doc` . does the namespace need to be imported, @MartinHonnen ? — Nicholas Saunders, Dec 16 '20 at 15:43
I can run the query `csv:doc('http://www.bccdc.ca/Health-Info-Site/Documents/BCCDC_COVID19_Dashboard_Case_Details.csv')` in BaseX 9.4 GUI without importing the module or namespace. — Martin Honnen, Dec 16 '20 at 15:57

score 2 · Accepted Answer · answered Dec 13 '20 at 13:54

As @MartinHonnen already pointed out, BaseX has a CSV module.

Here is how to use it.

csv file addressbook.csv

FName, LName, Address, City
John, Doe, 3851 SW 52nd St., Miami
Mary, Dowson, 770 SE 21nd St., Orlando

XQuery

let $text := file:read-text("c:\Users\Yitzhak\Downloads\addressbook.csv")
return csv:parse($text, map { 'header': true() })

Output

<csv>
  <record>
    <FName>John</FName>
    <LName> Doe</LName>
    <Address> 3851 SW 52nd St.</Address>
    <City> Miami</City>
  </record>
  <record>
    <FName>Mary</FName>
    <LName> Dowson</LName>
    <Address> 770 SE 21nd St.</Address>
    <City> Orlando</City>
  </record>
</csv>

how to use expath and Xquery to fetch a CSV file?

1 Answers1