C# Best way to store strings from an input file for manipulation and use?

Question

I've got a file of blocks of strings, each which end with a certain keyword. I've currently got a stream reader setup which adds each line of the file to a list up until the end of the current block(line contains keyword indicating end of block).

listName.Add(lineFromFile);

Each block contains information e.g. Book bookName, Author AuthorName, Journal JournalName etc. So each block is hypothetically a single item (book, journal, conference etc)..

Now with around 50 or so blocks of information(items) i need some way to store the information so i can manipulate it and store each author(s), Title, pages etc. and know what information goes with what item etc.

While typing this I've come up with the idea of possibly storing each Item as an object of a class called 'Item', however with potentially several authors, I'm not sure how to achieve this, as i was thinking maybe using a counter to name a variable e.g.

int i = 0;
String Author[i] = "blahblah";
i++;

But as far as i know it's not allowed? So my question is basically what would be the simplest/easiest way to store each item so that i can manipulate the strings to store each item for use later.

@yamen here's an example of the file:

Author Bond, james
Author Smith John A
Year 1994
Title For beginners
Book Accounting
Editor Smith Joe
Editor Doe John
Publisher The University of Chicago Press
City Florida, USA
Pages 15-23
End

Author Faux, M
Author Sedge, M
Author McDreamy, L
Author Simbha, D
Year 2000
Title Medical advances in the modern world
Journal Canadian Journal of medicine
Volume 25
Pages 1-26
Issue 2
End


Author McFadden, B
Author Goodrem, G
Title Shape shifting dinosaurs
Conference Ted Vancouver
City Vancouver, Canada
Year 2012
Pages 2-6
End

Answer 1

Update in lieu of your sample

How to parse the string is beyond the scope of this answer - you might want to have a go at that yourself, and then ask another SO (I suggest reading the golden rules of SO: https://meta.stackexchange.com/questions/128548/what-stack-overflow-is-not).

So I'll present the solution assuming that you have a single string representing the full block of book/journal information (this data looks like citations). The main change from my original answer is that you have multiple authors. Also you might want to consider whether you want to transform the authors' names back to [first name/initial] [middle names] [surname].

I present two solutions - one using Dictionary and one using Linq. The Linq solution is a one-liner.

Define an Info class to store the item:

public class Info
{
   public string Title { get; private set; }
   public string BookOrJournal { get; private set; }
   public IEnumerable<string> Authors { get; private set; }
   //more members of pages, year etc.
   public Info(string stringFromFile)
   {
     Title = /*read book name from stringFromFile */;
     BookOrJournalName = /*read journal name from stringFromFile */;
     Authors = /*read authors from stringFromFile */;
   }
}

Note that the stringFromFile should be one block, including newlines, of citation information.

Now a dictionary to store each info by author:

Dictionary<string, List<Info>> infoByAuthor = 
  new Dictionary<string, List<Info>>(StringComparer.OrdinalIrgnoreCase);

Note the OrdinalIgnoreCase comparer - to handle situations where an author's name is printed in a different case.

Given a List<string> that you're adding to as per your listName.Add, this simple loop will do the trick:

List<Info> tempList;
Info tempInfo;
foreach(var line in listName)
{
  if(string.IsNullOrWhiteSpace(line))
    continue;
  tempInfo = new Info(line);
  foreach(var author in info.Authors)
  {
    if(!infoByAuthor.TryGetValue(author, out tempList))
      tempInfo[author] = tempList = new List<Info>();
    tempList.Add(tempInfo);
  }
}

Now you can iterate through the dictionary, and each KeyValuePair<string, List<Info>> will have a Key equal to the author name and the Value will be the list of Info objects that have that author. Note that the casing of the AuthorName will be preserved from the file even though you're grouping case-insensitively such that two items with "jon skeet" and "Jon Skeet" will be grouped into the same list, but their original cases will be preserved on the Info.

Also the code is written to ensure that only one Info instance is created per citation, this is preferable for many reasons (memory, centralised updates etc).

Alternatively, with Linq, you can simply do this:

var grouped = listName.Where(s => !string.IsNullOrWhiteSpace(s))
  .Select(s => new Info(s))
  .SelectMany(i => 
    s.Authors.Select(ia => new KeyValuePair<string, Info>(ia, i))
  .GroupBy(kvp => kvp.Key, kvp => kvp.Value, StringComparer.OrdinalIgnoreCase);

Now you have enumerable of groups, where the Key is the Author Name and the inner enumerable is all the Info objects with that author name. The same case-preserving behaviour regarding 'the two Skeets' will be observed here, too.

Answer 2

You can use a class with simple attributes like these:

class Book {
    string Title;
    int PageCount;
}

You can either initialize Book[] lines = Book[myFile.LineCount]; or maintain a List<Book>, but string[] is easier to access individual line numbers (lines[34] means 34'th book, and 34th line).

But basically a System.Data.DataTable may be better suited, because you have rows that contain multiple columns. With DataTable, you can access individual rows and access their columns by name.

Example:

DataTable dt = new DataTable();
DataTable.Columns.Add("bookName");

DataRow dr = dt.NewRow();
dr["bookName"] = "The Lost Island";
dt.Rows.Add(dr);

//You can access last row this way: 
dt.Rows[dt.Rows.Count-1]["bookName"].

One more good thing about a DataTable is that you can use grouping and summing on its rows like on an ordinary SQL table.

Edit: Initially my answer used structs but as @AndrasZoltan pointed out, it may be better to use classes when you're not sure what the application will evolve in.

Answer 3

You should create a class Book

public class Book
 {
    public string Name { get; set; }
    public string Author { get; set; }
    public string Journal { get; set; }

 }

and maintain a List<Book>

var books = new List<Book>();
books.Add(new Book { Name = "BookName", Author = "Some Auther", Journal = "Journal" });

Answer 4

I would use a multi value dictionary for this:

public struct BookInfo
    {
        public string Title;
        public string Journal;
    }

Then create a dictionary object:

var dict = new Dictionary<Author, BookInfo>();

This way, if you do run into multiple authors, the data will be sorted by author, which makes writing future code to work with this data easy. Printing out a list of all books under some author will be dead easy and not require a cumbersome search process.

Answer 5

You are well on your way to inventing the relational database. Conveniently, these are already available. In addition to solving the problem of storing relationships between entities, they also handle concurrency issues and are supported by modelling techniques founded in provable mathematics.

---

Parsers are a subject unto themselves. Since SQL is out of the question, this being a contrived university assignment, I do have some observations.

• The easy way is with a regex. However this is extremely inefficient and a poor solution for large input files.
• In the absence of regexes, String.IndexOf() and String.Split() are your friends.
• If your assessor can't cope with SQL then LINQ is going to be quite a shock, but I really really like Zoltan's LINQ solution, it's just plain elegant.

Answer 6

Here is the complete code for this problem. It is written with a simple, straight forward approach. It can be optimized, there's no error checking and the AddData Method can be written in a much more efficient way by using reflection. But it does the job in an elegant way.

using System;
using System.Collections.Generic;
using System.IO;

namespace MutiItemDict
{
    class MultiDict<TKey, TValue>  // no (collection) base class
    {
        private Dictionary<TKey, List<TValue>> _data = new Dictionary<TKey, List<TValue>>();

        public void Add(TKey k, TValue v)
        {
            // can be a optimized a little with TryGetValue, this is for clarity
            if (_data.ContainsKey(k))
                _data[k].Add(v);
            else
                _data.Add(k, new List<TValue>() { v });
        }

        public List<TValue> GetValues(TKey key)
        {
            if (_data.ContainsKey(key))
                return _data[key];
            else
                return new List<TValue>();
        }
    }

    class BookItem
    {
        public BookItem()
        {
            Authors = new List<string>();
            Editors = new List<string>();
        }

        public int? Year { get; set; }
        public string Title { get; set; }
        public string Book { get; set; }
        public List<string> Authors { get; private set; }
        public List<string> Editors { get; private set; }
        public string Publisher { get; set; }
        public string City { get; set; }
        public int? StartPage { get; set; }
        public int? EndPage { get; set; }
        public int? Issue { get; set; }
        public string Conference { get; set; }
        public string Journal { get; set; }
        public int? Volume { get; set; }

        internal void AddPropertyByText(string line)
        {
            string keyword = GetKeyWord(line);
            string data = GetData(line);
            AddData(keyword, data);
        }

        private void AddData(string keyword, string data)
        {
            if (keyword == null)
                return;

            // Map the Keywords to the properties (can be done in a more generic way by reflection)
            switch (keyword)
            {
                case "Year":
                    this.Year = int.Parse(data);
                    break;
                case "Title":
                    this.Title = data;
                    break;
                case "Book":
                    this.Book = data;
                    break;
                case "Author":
                    this.Authors.Add(data);
                    break;
                case "Editor":
                    this.Editors.Add(data);
                    break;
                case "Publisher":
                    this.Publisher = data;
                    break;
                case "City":
                    this.City = data;
                    break;
                case "Journal":
                    this.Journal = data;
                    break;
                case "Volume":
                    this.Volume = int.Parse(data);
                    break;
                case "Pages":
                    this.StartPage = GetStartPage(data);
                    this.EndPage = GetEndPage(data);
                    break;
                case "Issue":
                    this.Issue = int.Parse(data);
                    break;
                case "Conference":
                    this.Conference = data;
                    break;
            }
        }

        private int GetStartPage(string data)
        {
            string[] pages = data.Split('-');
            return int.Parse(pages[0]);
        }

        private int GetEndPage(string data)
        {
            string[] pages = data.Split('-');
            return int.Parse(pages[1]);
        }

        private string GetKeyWord(string line)
        {
            string[] words = line.Split(' ');
            if (words.Length == 0)
                return null;
            else
                return words[0];
        }

        private string GetData(string line)
        {
            string[] words = line.Split(' ');
            if (words.Length < 2)
                return null;
            else
                return line.Substring(words[0].Length+1);
        }
    }

    class Program
    {
        public static BookItem ReadBookItem(StreamReader streamReader)
        {
            string line = streamReader.ReadLine();
            if (line == null)
                return null;

            BookItem book = new BookItem();
            while (line != "End")
            {
                book.AddPropertyByText(line);
                line = streamReader.ReadLine();
            }
            return book;
        }

        public static List<BookItem> ReadBooks(string fileName)
        {
            List<BookItem> books = new List<BookItem>();
            using (StreamReader streamReader = new StreamReader(fileName))
            {
                BookItem book;
                while ((book = ReadBookItem(streamReader)) != null)
                {
                    books.Add(book);
                }
            }
            return books;
        }

        static void Main(string[] args)
        {
            string fileName = "../../Data.txt";
            List<BookItem> bookList = ReadBooks(fileName);

            MultiDict<string, BookItem> booksByAutor = new MultiDict<string, BookItem>();
            bookList.ForEach(bk =>
                    bk.Authors.ForEach(autor => booksByAutor.Add(autor, bk))
                );

            string author = "Bond, james";
            Console.WriteLine("Books by: " + author);
            foreach (BookItem book in booksByAutor.GetValues(author))
            {
                Console.WriteLine("    Title : " + book.Title);
            }

            Console.WriteLine("");
            Console.WriteLine("Click to continue");
            Console.ReadKey();
        }
    }
}

And I also want to mention that all the parsing stuff can be avoided if you represent the Data in XML. The Data then looks like:

<?xml version="1.0" encoding="utf-8"?>
<ArrayOfBookItem >
  <BookItem>
    <Year>1994</Year>
    <Title>For beginners</Title>
    <Book>Accounting</Book>
    <Authors>
      <string>Bond, james</string>
      <string>Smith John A</string>
    </Authors>
    <Editors>
      <string>Smith Joe</string>
      <string>Doe John</string>
    </Editors>
    <Publisher>The University of Chicago Press</Publisher>
    <City>Florida, USA</City>
    <StartPage>15</StartPage>
    <EndPage>23</EndPage>
  </BookItem>
  <BookItem>
    <Year>2000</Year>
    <Title>Medical advances in the modern world</Title>
    <Authors>
      <string>Faux, M</string>
      <string>Sedge, M</string>
      <string>McDreamy, L</string>
      <string>Simbha, D</string>
    </Authors>
    <StartPage>1</StartPage>
    <EndPage>26</EndPage>
    <Issue>2</Issue>
    <Journal>Canadian Journal of medicine</Journal>
    <Volume>25</Volume>
  </BookItem>
  <BookItem>
    <Year>2012</Year>
    <Title>Shape shifting dinosaurs</Title>
    <Authors>
      <string>McFadden, B</string>
      <string>Goodrem, G</string>
    </Authors>
    <City>Vancouver, Canada</City>
    <StartPage>2</StartPage>
    <EndPage>6</EndPage>
    <Conference>Ted Vancouver</Conference>
  </BookItem>
</ArrayOfBookItem>

And the code for reading it:

using (FileStream stream =
    new FileStream(@"../../Data.xml", FileMode.Open,
        FileAccess.Read, FileShare.Read))
        {
            List<BookItem> books1 = (List<BookItem>)serializer.Deserialize(stream);
        }

Answer 7

Its not quite clear what you need without a better example of the file or how you want to use the data but it sounds like you need to parse the string and put it into an entity. The following is an example using the fields you mentioned above.

public IList<Entry> ParseEntryFile(string fileName)
{
    ...
    var entries = new List<Entry>();

    foreach(var line in file)
    {
        var entry = new Entry();
        ...
        entries.Add(entry);
    }
    return entries;
}


public class Entry
{
    public Book BookEntry { get; set; }
    public Author AuthorEntry { get; set; }
    public Journal JournalEntry { get; set; }
}

public class Book
{
    public string Name{ get; set; }
    ...
}

public class Author
{
    public string FirstName { get; set; }
    public string LastName { get; set; }
}

...

Answer 8

You can create a class for each item:

class BookItem
        {
            public string Name { get; set; }
            public string Author { get; set; }
        }

Read the data from each line into an instance of this class and store them in a temporary list:

var books = new List<BookItem>();
while (NotEndOfFile())
{
    BookItem book= ReadBookItem(...)
    books.Add(book);
}

After you have this list you can create Multi Value Dictionaries and have quick access to any item by any key. For example to find a book by its author:

var booksByAuthor = new MultiDict<string, BookItem>();

add the items to the Dictionary:

books.ForEach(bk => booksByAuthor.Add(bk.Author, bk));

and then you can iterate on it:

string autorName = "autor1";
Console.WriteLine("Books by: " + autorName);
            foreach (BookItem bk1 in booksByAutor)
            {
                Console.WriteLine("Book: " + bk1.Name);
            }

I got the basic Multi Item Dictionary from here:

Multi Value Dictionary?

This is my implementation:

class MultiDict<TKey, TValue>  // no (collection) base class
        {
            private Dictionary<TKey, List<TValue>> _data = new Dictionary<TKey, List<TValue>>();

            public void Add(TKey k, TValue v)
            {
                // can be a optimized a little with TryGetValue, this is for clarity
                if (_data.ContainsKey(k))
                    _data[k].Add(v);
                else
                    _data.Add(k, new List<TValue>() { v });
            }

            // more members

            public List<TValue> GetValues(TKey key)
            {
                if (_data.ContainsKey(key))
                    return _data[key];
                else
                    return new List<TValue>();
            }

        }