How to read and process a pipe delimited file in Rust?

Question

I want to read a pipe delimited file, process the data, and generate a result in CSV format.

Input file data

A|1|Pass
B|2|Fail
A|3|Fail
C|6|Pass
A|8|Pass
B|10|Fail
C|25|Pass
A|12|Fail
C|26|Pass
C|26|Fail

I'm want to apply a group by function on column 1 and column 3 and generate column 2's sum according to a particular group.

I'm stuck at the point of how to maintain the records to apply the group by function:

use std::fs::File;
use std::io::{BufReader};
use std::io::{BufRead};
use std::collections::HashMap;

fn say_hello(id: &str, value: i32, no_change : i32) {

    if no_change == 101 {
        let mut data = HashMap::new();
    }
    if value == 0 {
        if data.contains_key(id) {
            for (key, value) in &data {
                if value.is_empty() {

                }
            }
        } else {
            data.insert(id,"");
        }
    } else if value == 2 {
        if data.contains_key(id) {
            for (key, value) in &data {
                if value.is_empty() {

                } else {

                }
            }
        } else {
            data.insert(id,"");
        }
    }
}

fn main() {

    let f = File::open("sample2.txt").expect("Unable to open file");
    let br = BufReader::new(f);
    let mut no_change = 101;
    for line in br.lines() {
        let mut index = 0;
        for value in line.unwrap().split('|') {
            say_hello(&value,index,no_change);
            index = index + 1;
        }
    }
}

I'm expecting a result like:

name,result,num
A,Fail,15
A,Pass,9
B,Fail,12
C,Fail,26
C,Pass,57

Is there any specific technique to read a pipe-delimited file and process the data like above? Python's pandas accomplished this requirement but I want to do it in Rust.

Use the csv crate? https://docs.rs/csv – BurntSushi5 May 10 '17 at 10:52 — BurntSushi5, May 10 '17 at 10:52

score 2 · Accepted Answer · edited May 23 '17 at 12:34

As was mentioned, use the csv crate to do the heavy lifting of parsing the file. Then it's just a matter of grouping each row by using a BTreeMap which also helpfully performs sorting. The entry API helps efficiently insert into the BTreeMap.

extern crate csv;
extern crate rustc_serialize;

use std::fs::File;
use std::collections::BTreeMap;

#[derive(Debug, RustcDecodable)]
struct Record {
    name: String,
    value: i32,
    passed: String,
}

fn main() {
    let file = File::open("input").expect("Couldn't open input");
    let mut csv_file = csv::Reader::from_reader(file).delimiter(b'|').has_headers(false);

    let mut sums = BTreeMap::new();
    for record in csv_file.decode() {
        let record: Record = record.expect("Could not parse input file");
        let key = (record.name, record.passed);
        *sums.entry(key).or_insert(0) += record.value;
    }

    println!("name,result,num");
    for ((name, passed), sum) in sums {
        println!("{},{},{}", name, passed, sum);
    }
}

You'll note that the output is correct:

name,result,num
A,Fail,15
A,Pass,9
B,Fail,12
C,Fail,26
C,Pass,57

@Shepmaster- How I can pushed this result in one csv file like output.csv? — ketan, May 11 '17 at 04:06

score 1 · Answer 2 · edited May 10 '17 at 13:14

I'd suggest something like this:

use std::str;
use std::collections::HashMap;
use std::io::{BufReader, BufRead, Cursor};

fn main() {
    let data = "
A|1|Pass
B|2|Fail
A|3|Fail
C|6|Pass
A|8|Pass
B|10|Fail
C|25|Pass
A|12|Fail
C|26|Pass
C|26|Fail";
    let lines = BufReader::new(Cursor::new(data))
        .lines()
        .flat_map(Result::ok)
        .flat_map(parse_line);
    for ((fa, fb), s) in group(lines) {
        println!("{}|{}|{}", fa, fb, s);
    }
}

type ParsedLine = ((String, String), usize);

fn parse_line(line: String) -> Option<ParsedLine> {
    let mut fields = line
        .split('|')
        .map(str::trim);
    if let (Some(fa), Some(fb), Some(fc)) = (fields.next(), fields.next(), fields.next()) {
        fb.parse()
            .ok()
            .map(|v| ((fa.to_string(), fc.to_string()), v))
    } else {
        None
    }
}

fn group<I>(input: I) -> Vec<ParsedLine> where I: Iterator<Item = ParsedLine> {
    let mut table = HashMap::new();
    for (k, v) in input {
        let mut sum = table.entry(k).or_insert(0);
        *sum += v;
    }
    let mut output: Vec<_> = table
        .into_iter()
        .collect();
    output.sort_by(|a, b| a.0.cmp(&b.0));
    output
}

playground link

Here a HashMap is used for grouping entries and then results are moved to a Vec for sorting.

@swizard- Can you please explain little bit coding stuff I'm new to rust. — ketan, May 10 '17 at 12:01

How to read and process a pipe delimited file in Rust?

2 Answers2