C++ reading in from file with words seperated by whitespace and new lines

Question

I'm having an issue reading in from a file that has words seperated by spaces, and with new lines randomly. Here is my code:

vector<string> _vecIgnoreWords;
vector<string> _vecHungerGames;

void readTextFile(char *fileNameHungerGames, vector<string>& _vecHungerGames){
    ifstream fileInHungerGames;
    string newline;

    fileInHungerGames.open(fileNameHungerGames);
    if(fileInHungerGames.is_open()){
        while(getline(fileInHungerGames, newline)){
            stringstream iss(newline);
            while(iss){
                iss >> newline;
                if(!(isCommonWord(newline, _vecIgnoreWords))){
                    _vecHungerGames.push_back(newline);
                    cout << newline << endl;
                }
            }
        }

        fileInHungerGames.close();  
    }

The call in main:

string fileName = argv[2];
string fileNameIgnore = argv[3];
char* p = new char[fileNameIgnore.length() + 1];
memcpy(p, fileNameIgnore.c_str(), fileNameIgnore.length()+1);
getStopWords(p, _vecIgnoreWords);
char* hungergamesfile_ = new char[fileName.length() + 1];
memcpy(hungergamesfile_, fileName.c_str(), fileName.length()+1);
readTextFile(hungergamesfile_, _vecHungerGames);

The stop words void:

void getStopWords(char *ignoreWordFileName, vector<string>& _vecIgnoreWords){
    ifstream fileIgnore;
    string line;
    fileIgnore.open(ignoreWordFileName);
    if(fileIgnore.is_open()){
        while(getline(fileIgnore, line)){
            _vecIgnoreWords.push_back(line);
        }
    }
    fileIgnore.close();
    return;
}

My problem currently is that My output for this code ends up something like:

bread
is
is 
slipping
away 

take

I'm not sure why i'm getting repeats (is is) and the empty lines when I am using a string stream?

my output should look like:

bread 
is 
slipping
away 
from 
me

Also slightly less important but my while loop is looping once too many which is why I have the if(_vecHungerGames.size() == 7682) is there a way to fix this loop from looping once too many?

File example:

bread is 
slipping away from me 
i take his hand holding on tightly preparing for the

the file is extremely long ( the whole hunger games book one) but here is an example of part of it: — andrew fay, Sep 01 '16 at 19:07
bread is slipping away from me i take his hand holding on tightly preparing for the — andrew fay, Sep 01 '16 at 19:08
no punc, symbols or capitals, just separated by whitespace and new lines — andrew fay, Sep 01 '16 at 19:08
@andrewfay Post a [MCVE] that reproduces your problem please. — πάντα ῥεῖ, Sep 01 '16 at 19:09
I've removed all the commented out code and left only the code being used. — andrew fay, Sep 01 '16 at 19:12
Welcome to StackOverflow, but you're too early. First try [debugging](https://ericlippert.com/2014/03/05/how-to-debug-small-programs/). If and when that fails, seek help here. — Mike Kinghan, Sep 01 '16 at 19:34
**while (iss)** checks for " whether an error flag is set (either failbit or badbit)". You better turn the check into **while (iss >> newline)**. That will also solve "off by one" case. — blackpen, Sep 01 '16 at 20:16
You overcomplicated your code and get problems. Why do you need to read into temporary string? just read directly from file — Slava, Sep 01 '16 at 20:41

Remy Lebeau · Accepted Answer · 2016-09-01T21:11:14.390

Try something more like this:

#include <iostream>
#include <vector>
#include <string>
#include <fstream>
#include <sstream>

std::vector<std::string> _vecIgnoreWords;
std::vector<std::string> _vecHungerGames;

void getStopWords(const char *filename, std::vector<std::string>& output)
{
    std::ifstream file(fileName);
    std::string s;

    while (std::getline(file, s))
        output.push_back(s);
}

void readTextFile(const char *filename, std::vector<std::string>& output)
{
    std::ifstream file(fileName);
    std::string s;

    while (file >> s)
    {
        if (!isCommonWord(s, _vecIgnoreWords))
        {
            output.push_back(s);
            std::cout << s << std::endl;
        }
    }
}

int main()
{
    getStopWords(argv[3], _vecIgnoreWords);
    readTextFile(argv[2], _vecHungerGames);

    // use _vecHungerGames as needed...

    return 0;
}

Reading to temporary `string` is useless, while( file >> s ) will do what author needs — Slava, Sep 01 '16 at 20:38

C++ reading in from file with words seperated by whitespace and new lines

1 Answers1