3

I managed to parse a pgn file into several games mainly thanks to this forum.

However, as the files I have to deal with have so many games, the process can take two minutes on my recent computer. That's why I would like to animate a progress bar on the GUI application using this parser.

I think the easiest way would be to "ask" spirit how many characters he has already processed, and how many characters remain. (Or how many lines remain and have been processed).

Is it possible ? If so, how do I need to modify the parser file in order to get this ratio ?

Community
  • 1
  • 1
loloof64
  • 5,252
  • 12
  • 41
  • 78
  • 1
    As an alternative, I suppose you could wrap your iterators around a custom type that would report progress as the iterator position. This will be a little inaccurate as the parser will need to backtrack, so either you let the progress move back and forth as the parser does or live with the inaccuracy as is. – SirGuy Dec 10 '15 at 20:53
  • Interesting idea indeed. I am still wondering how I will deal with backtracks : if this happens too often, I will need a way to ignore them and let the progress bar stay at his most advanced position. – loloof64 Dec 10 '15 at 21:00
  • 2
    Just keep track of the maximum distance from the start of the range as the iterator moves about and only update the progress if that max changes. – SirGuy Dec 10 '15 at 21:02
  • That sounds very reasonable to me. Thank you. – loloof64 Dec 10 '15 at 21:05
  • I've sought into the istream documentation a kind of listener in order to increment my personnal counter : but I did not find anyone which could help me. Furthermore, as it is a stream, I have no way to know the total caracters count before having read all the file. So I would need at least one more pass in order to count the caracters/lines. What workaround could I apply ? – loloof64 Dec 10 '15 at 21:22

1 Answers1

2

You can use line_pos_iterator and potentially the iter_pos primitive from the repository.

  • (@GuyGreer:) There is no way to know the amount of backtracking involved (otherwise, there would not need to be backtracking in the first place). So, the best thing to do is accept that you get some kind of "average throughput" that can be a little bursty or laggy at times. If your grammar is that unbalanced that these variations are more than noise, you should consider fixing the the grammar/parser definitions in the first place.

  • To counter the "problem" of not knowing the stream length, you cannot fix it other than not having it as a stream.

    I'd suggest memory mapping. You can use the facilities from boost::iostreams, boost::interprocess or just mmap.

I estimate I have at least 3 answers demonstrating each of the techniques mentioned in this answer, so I'd just search this site for them.

sehe
  • 374,641
  • 47
  • 450
  • 633
  • I don't think I indicated that you could know how much backtracking is required... As for not knowing the stream length, isn't it just the size of the file? – SirGuy Dec 10 '15 at 22:04
  • @GuyGreer who said anything about a file? What if the stream is a socket? – sehe Dec 10 '15 at 22:05
  • Oh yeah, that's right. I was getting that from the question he linked to in which he is parsing a file. In general, yes you cannot know the length of the stream. – SirGuy Dec 10 '15 at 22:06
  • Yeah, obviously my mmap idea hinges on the same assumption :) – sehe Dec 10 '15 at 22:07
  • I found on the following link how to use iter_pos basically, but only when all the parsing is done : http://marko-editor.com/articles/position_tracking/ . Is there a way to turn the tracking as an event, during the parsing, so that I can update my ProgressBar ? You are both right, I want to parse a file : I will hardly use the class another way. I did not notice I could get the size of the file. – loloof64 Dec 11 '15 at 13:15
  • 2
    Use semantic actions. Alternatively use `on_error` (there's a compiler example that uses it to annotate AST nodes with source locations IIRC) – sehe Dec 11 '15 at 13:17
  • I am trying to follow http://marko-editor.com/articles/position_tracking/ in order to get progression (will add semantic actions later). But I can't manage to avoid compilation errors : http://pastebin.com/eLG9TvjZ and the header file : http://pastebin.com/NuJrATY3 – loloof64 Dec 12 '15 at 12:14
  • @loloof64 consider posting that as a question. I'll look at it now: https://www.livecoding.tv/sehe/ – sehe Dec 12 '15 at 23:16
  • 1
    For now, you can see the end results here: https://www.livecoding.tv/video/progress-reporting-on-qi-parser/. I'll explain a bit if you post the question. – sehe Dec 13 '15 at 00:39
  • I could not see the video from my Chromium navigator. I got a 403 error (Forbidden) (Finally solved by creating an account) – loloof64 Dec 13 '15 at 10:06