1

I am trying to parse LTSpice data produced with a .step function. This splits the resulting dataset into parts which I do not know how to effectively handle using R.

library(tidyverse)

data <- read_tsv("FileName")
## 
## ── Column specification ────────────────────────────────────────────────────────
## cols(
##   time = col_character(),
##   `V(out)` = col_double(),
##   `V(in)` = col_double()
## )
## Warning: 24 parsing failures.
## row col  expected    actual                           file
##   1  -- 3 columns 1 columns 'FileName'
##   6  -- 3 columns 1 columns 'FileName'
##  11  -- 3 columns 1 columns 'FileName'
##  16  -- 3 columns 1 columns 'FileName'
##  21  -- 3 columns 1 columns 'FileName'
## ... ... ......... ......... ..............................
## See problems(...) for more details.

head(data)
## # A tibble: 6 x 3
##   time                                                          `V(out)` `V(in)`
##   <chr>                                                            <dbl>   <dbl>
## 1 Step Information: Resistor1=10 Resistor2=1M Offset=-3  (Run…  NA       NA     
## 2 0.000000000000000e+00                                         -1.50e-5  0     
## 3 4.913626562387649e-09                                          3.08e-2  0.0309
## 4 4.996634374887648e-09                                          3.13e-2  0.0314
## 5 5.000000000000000e-09                                          3.14e-2  0.0314
## 6 Step Information: Resistor1=10.01K Resistor2=1M Offset=-3  …  NA       NA

As you can see, the data is split into parts, which are separated by information about the step. I would like to change the data so that the rows containing “Step Information” are deleted, and the corresponding parameters are written to a matching column:

## # A tibble: 6 x 6
##   time                    `V(out)` `V(in)` Resistor1 Resistor2 Offset
##   <chr>                      <dbl>   <dbl> <lgl>     <lgl>     <lgl> 
## 1 0.000000000000000e+00 -0.0000150  0      NA        NA        NA    
## 2 4.913626562387649e-09  0.0308     0.0309 NA        NA        NA    
## 3 4.996634374887648e-09  0.0313     0.0314 NA        NA        NA    
## 4 5.000000000000000e-09  0.0314     0.0314 NA        NA        NA    
## 5 0.000000000000000e+00 -0.00749    0      NA        NA        NA    
## 6 4.908743749887649e-09  0.00789    0.0308 NA        NA        NA

EDIT:

Part of the raw data:

time    V(out)  V(in)
Step Information: Resistor1=10 Resistor2=1M Offset=-3  (Run: 1/24)
0.000000000000000e+00   -1.498494e-05   0.000000e+00
4.913626562387649e-09   3.082234e-02    3.086832e-02
4.996634374887648e-09   3.134312e-02    3.138962e-02
5.000000000000000e-09   3.136424e-02    3.141076e-02
Step Information: Resistor1=10.01K Resistor2=1M Offset=-3  (Run: 2/24)
0.000000000000000e+00   -7.485015e-03   0.000000e+00
4.908743749887649e-09   7.887614e-03    3.083766e-02
4.996634374887648e-09   8.162769e-03    3.138962e-02
5.000000000000000e-09   8.173305e-03    3.141076e-02
Step Information: Resistor1=20K Resistor2=1M Offset=-3  (Run: 3/24)
...
kaos
  • 53
  • 5
  • 1
    Are you able to provide a sample of the raw `tsv` text data - maybe just 10 lines? You might be able to handle it during the import procedure. Is it something like this? `x <- "Step Information: Resistor1=10 Resistor2=1M Offset=-3 (Run: 1/24)\n0.000+00\t-0.000\t0.000\n4.913-09\t0.0308\t0.0308\n4.996e-09\t0.0313\t0.0313\n5.000e-09\t0.0313\t0.0314\nStep Information: Resistor1=10.01K Resistor2=1M Offset=-3 (Run: 2/24)\tNA\tNA\n0.000e+00\t-0.007\t0.000\n4.908e-09\t0.007\t0.0308\n4.996-09\t0.008\t0.0313\n5.00e-09\t0.008\t0.0314"` - which you can see using `cat(x)` – thelatemail May 18 '21 at 22:28
  • 1
    I added some raw data to my question. Handling everything during import would be great. – kaos May 18 '21 at 22:35

1 Answers1

1

It's going to always be a bit ugly, but I reckon you're best served working with the raw data. Pull the lines in, flag the rows which have the Step information, separate the data out, then combine:

## read everything in as raw lines
## for this example use this code
##    tmp <- readLines(textConnection(txt))
## for your real data use this code
tmp <- readLines("FileName")

## flag the Step Information rows
stl <- grepl("^Step", tmp)

## get all name=value pairs out of the Step Information rows and join together
nms <- regmatches(tmp[stl], gregexpr("\\S+(?=\\=)",    tmp[stl], perl=TRUE))
val <- regmatches(tmp[stl], gregexpr("(?<=\\=)(\\S+)", tmp[stl], perl=TRUE))
nvp <- do.call(rbind, Map(function(v,n) as.data.frame(as.list(v), col.names=n), val, nms))

## finally put it all together
cbind(
  read.table(text=tmp[!stl], header=TRUE),
  nvp[ cumsum(stl)[!stl][-1], ]
)

#            time        V.out.      V.in. Resistor1 Resistor2 Offset
#1   0.000000e+00 -1.498494e-05 0.00000000        10        1M     -3
#1.1 4.913627e-09  3.082234e-02 0.03086832        10        1M     -3
#1.2 4.996634e-09  3.134312e-02 0.03138962        10        1M     -3
#1.3 5.000000e-09  3.136424e-02 0.03141076        10        1M     -3
#2   0.000000e+00 -7.485015e-03 0.00000000    10.01K        1M     -3
#2.1 4.908744e-09  7.887614e-03 0.03083766    10.01K        1M     -3
#2.2 4.996634e-09  8.162769e-03 0.03138962    10.01K        1M     -3
#2.3 5.000000e-09  8.173305e-03 0.03141076    10.01K        1M     -3

The example data I used was:

txt <- "time\tV(out)\tV(in)
Step Information: Resistor1=10 Resistor2=1M Offset=-3  (Run: 1/24)
0.000000000000000e+00\t-1.498494e-05\t0.000000e+00
4.913626562387649e-09\t3.082234e-02\t3.086832e-02
4.996634374887648e-09\t3.134312e-02\t3.138962e-02
5.000000000000000e-09\t3.136424e-02\t3.141076e-02
Step Information: Resistor1=10.01K Resistor2=1M Offset=-3  (Run: 2/24)
0.000000000000000e+00\t-7.485015e-03\t0.000000e+00
4.908743749887649e-09\t7.887614e-03\t3.083766e-02
4.996634374887648e-09\t8.162769e-03\t3.138962e-02
5.000000000000000e-09\t8.173305e-03\t3.141076e-02"
thelatemail
  • 91,185
  • 12
  • 128
  • 188
  • Thank you! This works great. Thanks for the comments. I am pretty unexperienced with `grepl` etc.. Would a similar solution be possible with `stringr`? – kaos May 19 '21 at 06:56
  • 1
    Absolutely. grepl is analogous to str_detect I think, and the regmatches bits are just str_extract. If I find time I'll have a crack at converting the code. – thelatemail May 19 '21 at 09:23
  • Great, I'll try the same and update my question in case I get it to work. As of now, thank you very much. Your solution works great :) – kaos May 19 '21 at 14:58