1

I am writing an R script to program a Qualtrics survey from an Excel template I created. I grab the survey items from the Excel template via the xlsx package and then do a bit of munging to create a .txt file formatted the way I want it. The problem is that I can't upload it to Qualtrics unless I select all in the resulting .txt file, copy, and paste into a blank .txt file.

This is not a question about Qualtrics, per se, and I don't think one needs to know anything about this platform to consider the question. I believe that my problem is in the R output file format. I hope if you continue reading, you will see what I mean.

Everything in the following code block should work.

require(xlsx)
# import survey file; get from Dropbox link above
  survey <- read.xlsx("template.xlsx", "survey")
  survey <- subset(survey, !is.na(survey$questiontext)) # drop if blank
  choices <- read.xlsx("template.xlsx", "choices")
# create vector to hold results
  output <- "[[AdvancedFormat]]"
# loop through every question
  for (i in 1:nrow(survey)) {
    # insert start block if one exists
      bs <- ifelse(is.na(survey$blockname[i]), "", 
                   paste0("[[Block:", 
                          as.character(survey$blockname[i]), "]]"))
    # insert end block if one exists
      be <- ifelse(is.na(survey$endblock[i]), "", 
               as.character(survey$endblock[i]))
    # insert item ID if one exists
      qid <- ifelse(is.na(survey$questionid[i]), "", 
                    paste0("[[ID:", 
                           as.character(survey$questionid[i]), "]]"))
    # subset choice sheet to selected choice
      choices.sub <- subset(choices, choices$listname==survey$choicelist[i])
      responses <- ""
    # create vector of choices
      for (ch in 1:nrow(choices.sub)) {
        responses <- paste(responses, choices.sub$label[ch], sep="\r") 
      }
    # insert page break if one exists
      pb <- ifelse(is.na(survey$pagebreak[i]) | i==nrow(survey), "", 
                   as.character(survey$pagebreak[i]))
    # add to output vector
      output <- paste(output, 
                      bs,
                      survey$questiontype[i],
                      qid,
                      survey$questiontext[i],
                      survey$choicetype[i],
                      responses,
                      pb,
                      be,
                      sep="\r")
  }
# write to txt file
  cat(output, file = (con <- file("foo1.txt", "w", encoding = "UTF-8"))); close(con)

Here's the resulting foo1.txt file from R. Qualtrics rejects this file. If I select all from this file and copy into a new blank .txt file (using ST3), foo2.txt, Qualtrics is happy.

What is different about these two files? I'm not changing anything in foo2.txt. Just copy/paste into a new file.

Eric Green
  • 7,385
  • 11
  • 56
  • 102
  • 1
    I haven't checked your solutions, but from the top of my head, are there any special characters not visible? What about encoding? If you're using UTF, do you use BOM? – Roman Luštrik Jun 23 '14 at 11:33
  • I don't think there are any hidden characters, unless my carriage returns `\r` count. I specified "UTF-8" in `file()` because the default format of Sublime Text 3 is "default_encoding": "UTF-8". I figured since `foo2.txt` works and since `foo2.txt` is in UTF-8, it would work. – Eric Green Jun 23 '14 at 11:39
  • Here's what I found about BOM in `?file`: "As from R 3.0.0 the encoding "UTF-8-BOM" is accepted and will remove a Byte Order Mark if present." I tried using `encoding = "UTF-8-BOM"`, but I got an error: `unsupported conversion from 'UTF-8-BOM' to ''`. This [SO question](http://stackoverflow.com/questions/2223882/whats-different-between-utf-8-and-utf-8-without-bom) makes me think it should not be required. But I'm not sure. – Eric Green Jun 23 '14 at 11:42
  • It makes me wonder what happens when you save the new `.txt` file. What program do you use to save it? – Roman Luštrik Jun 23 '14 at 12:20
  • Sublime Text 3 (Mac). Just command-A, command-C, command-N, command-V, command-S. – Eric Green Jun 23 '14 at 12:23
  • 1
    This is almost always a conflict between `` and `-` as end-of-line terminators. – Carl Witthoft Jun 23 '14 at 13:48
  • Thanks, @CarlWitthoft. You prompted me to search for `R` and end-of-line terminators. I stumbled upon this [SO question](http://stackoverflow.com/questions/1279779/what-is-the-difference-between-r-and-n) and the discussion of `\r` vs `\n`. Changing my `\r` to `\n` did the trick. – Eric Green Jun 23 '14 at 14:09
  • Glad you found the answer. Go ahead and either answer your own question or mark it as a dupe if you prefer. – Carl Witthoft Jun 23 '14 at 16:55

1 Answers1

0

@CarlWitthoft pointed me in the right direction with his comment:

This is almost always a conflict between <CR> and <CR>-<LF> as end-of-line terminators.

I searched on the topic and found this SO thread that led me to change my \r to \n. That did the trick.

Community
  • 1
  • 1
Eric Green
  • 7,385
  • 11
  • 56
  • 102