How to export IBM Watson conversation history?

Question

Before running the code, install ibm-watson & ibm-cloud-sdk-core package and also pip instll PyJWT==1.7.1.

I found in IBM document that "For a Python script you can run to export logs and convert them to CSV format, download the export_logs_py.py file from the Watson Assistant GitHub) repository."

But I don't really know where & how should I modify in order to connect my ibm skill. There is no demo or instruction about where I can find those argument. I only find these information in skill api details but it seems it needs more. Do anyone have an example version about how to use the .py they provided？ (I'm a coding beginner, not really understand every lines in the .py)

The .py shows an error after I run the file without modification：

runfile('C:/export_logs.py', wdir='C:/Users/admin/Downloads')
usage: export_logs.py [-h] [--logtype {ASSISTANT,WORKSPACE,DEPLOYMENT}]
                      [--language LANGUAGE] [--filetype {CSV,TSV,XLSX,JSON}]
                      [--url URL] [--version VERSION]
                      [--totalpages TOTALPAGES] [--pagelimit PAGELIMIT]
                      [--filter FILTER] [--strip STRIP]
                      apikey id filename
export_logs.py: error: the following arguments are required: apikey, id, filename
An exception has occurred, use %tb to see the full traceback.

SystemExit: 2

The conversation I want to download：

The script does not need any modification. You invoke it with parameters. If you don't provide any parameters, there should be some help — data_henrik, Dec 14 '20 at 10:12
@data_henrik Thanks for your reply. I try to run the script without modification, but it shows an error. (I add the error output of the script in the post. ) Can you explain what does " invoke it with parameters." means？ — S.F. Yeh, Dec 15 '20 at 03:12

score 1 · Answer 1 · answered Dec 15 '20 at 06:19

As you can see, the script reported an error and said that you have to provide the apikey, the id and the (presumably output) filename as parameters. It also showed that additional parameters can be specified.

usage: export_logs.py [-h] [--logtype {ASSISTANT,WORKSPACE,DEPLOYMENT}]
                      [--language LANGUAGE] [--filetype {CSV,TSV,XLSX,JSON}]
                      [--url URL] [--version VERSION]
                      [--totalpages TOTALPAGES] [--pagelimit PAGELIMIT]
                      [--filter FILTER] [--strip STRIP]
                      apikey id filename

Your next step could be now to invoke the script again, but provide an API key for Watson Assistant, the skill ID and a filename as additional paramaters. Next, I would try to something like, e.g., trying out to specify the output type:

export_logs.py --filetype CSV myapikey skillID output.csv

I am not the author of that script, but that is how I would approach it if I wanted to use it

Thanks for your generous, you help me with a lot of watson problems. — S.F. Yeh, Dec 16 '20 at 08:43

Vidyasagar Machupalli · Accepted Answer · 2020-12-16T09:52:14.350

1

First of all, Workspaces in IBM Watson Assistant are now called Skills.

To understand what arguments(positional and optional) you need to pass to the Python script, run the below command

python export_logs_py.py -h

Wherever you see workspace, you can replace it with skill.

To export the logs in the .csv file format, run the below command

python export_logs_py.py --filetype CSV --url <URL> <API_KEY> <SKILL_ID> output.csv

Replace placeholders <URL, <API_KEY> and <SKILL_ID> with appropriate values mentioned below.

<URL> & <API_KEY> - You can find them under the Manage page of your Watson Assistant service page
<SKILL_ID> - The same as the one in the image you uploaded. Check this StackOverflow answer for more info.

For Assistant logs, add --logtype ASSISTANT. The default is WORKSPACE.

You can also find the logs in the UI under Analytics section of your Skill

edited Dec 16 '20 at 09:52

answered Dec 15 '20 at 06:45

Vidyasagar Machupalli

2,737
1
19
29

Thank you for providing so much details, I think I'm almost there. But after run the command, it shows another error：`Reading page 1. ERROR:root:Resource not found.` Do you know why this happen? (URL is assistant url) – S.F. Yeh Dec 16 '20 at 08:36
1

It's not the assistant URL. As mentioned in my answer above, it's the URL that you find under the manage page of Watson assistant service or under `Service credentials` > Auto-generated credentials (Expand). The URL should look like `https://api.us-south.assistant.watson.cloud.ibm.com/instances/ff199a77-72c4-4705-xxxx-xxxxxx` – Vidyasagar Machupalli Dec 16 '20 at 08:42
I think the input is all fine now, but the result is weird `Reading page 1. No Logs found. :(Writing 0 records to: output.csv as file type: CSV` I did see many conversation in the Analytics>conversation history page. – S.F. Yeh Dec 16 '20 at 08:54
No conversation data is available for analysis. Typically, data is not available until your skill is deployed and has participated in conversations with users. See [Adding integrations to learn about deployment options.](https://cloud.ibm.com/docs/assistant?topic=assistant-deploy-integration-add) You can find the logs under Analytics > User conversations of your skill. Check the updated answer – Vidyasagar Machupalli Dec 16 '20 at 09:10
Do you mean User conversations can only be checked in the IBM skill, but cannot be download? Then what does this .py file can do？ (I have integrate the skill with preview link and I did see logs in the page you mentioned as I update in my question post) – S.F. Yeh Dec 16 '20 at 09:47
Along with the `.py` file, you can also see the logs in the UI. If you are seeing the logs in the UI, you should be able to download the SAME logs with the script. Did you run the pythong script after seeing the logs in the UI? – Vidyasagar Machupalli Dec 16 '20 at 09:49
Yes, I run the script after seeing the logs in the UI – S.F. Yeh Dec 16 '20 at 09:54
Actually, I don't see output.csv either (I think it should be in the same directory as the ,py ?) – S.F. Yeh Dec 16 '20 at 10:30
Yes. It should be in the same directory as the `.py` file – Vidyasagar Machupalli Dec 16 '20 at 10:41
Hi, I found the problem caused by "--logtype ASSISTANT". After I delete this everything works fine. – S.F. Yeh Feb 24 '21 at 08:14

How to export IBM Watson conversation history?

2 Answers2