Best way to read and write time-critical data?

Question

I have .txt files that are overwritten with data from software every 5-10 seconds, I then have a wpf application that reads and displays this data every second. Here are my issues:

Currently the text files are stored on a server and there are a bunch of users running this application to view this "live" data. HOWEVER, due to:

An I/O bug in windows

The files "lock" up periodically and cause all of the applications to lock up (can't even close in task manager).

Therefore I decided to have the data copied from the text files to SQL, however from my understanding there's no way to overwrite the data in the SQL table. One must Drop the Table and Create a new one. This cause a 10+ second delay updating the data, which cannot happen.

My question is, there HAS to be a way to rapidly read and write data from somewhere, be it a database, etc. I am not sure where else to turn.

My constraints:

I'm stuck with Server 2008, have to use these text file, and I have to display it on my wpf application. Does anyone have any suggestions for a method that can handle this type of I/O?

All help is greatly appreciated, I'm at a complete loss..

*from my understanding there's no way to overwrite the data in the SQL table.* - Your understanding is incorrect. — MarcinJuraszek, Aug 25 '15 at 23:17
How much data are you writing? How much of it will be different to the previous write? — Blorgbeard, Aug 25 '15 at 23:18
"from my understanding there's no way to overwrite the data in the SQL table" Really? Does Update ring any bell? — Mihai Caracostea, Aug 25 '15 at 23:20
I can understand if the problem occurs with writing but it is very strange to see it happening with reading unless the files are very large! — NoChance, Aug 25 '15 at 23:24
SQL isn't my strong suit so go easy.. from researching it looks like the UPDATE command only works when pulling data from another table? if i'm wrong can you provide some syntax that will UPDATE the table from the TEXT file? — Logan Lower, Aug 25 '15 at 23:34
@Blorgbeard Most of the text file changes, but it contains only around 30 lines with less than 100 characters each line — Logan Lower, Aug 25 '15 at 23:35
And it takes 10+ seconds to drop a table of that size and create another? I mean, that's about the slowest way you could do it, but it shouldn't be that slow for ~3kb of data.. — Blorgbeard, Aug 25 '15 at 23:55
FYI, time sensitivity in the range of 1 second to 1 minute is traditionally called "time-critical". — RBarryYoung, Aug 26 '15 at 00:03
you can use [this](http://stackoverflow.com/questions/30799514/does-fileshare-none-make-threads-wait-until-the-filestream-is-closed/30799992#30799992) simple logic to manage concurrent text file access. — Kurubaran, Aug 26 '15 at 00:07
The link you reference includes two ways non-programming ways to fix this problem 1) there is a published hotfix for it, or 2) disabling the Power service. Why are these not sufficient? — RBarryYoung, Aug 26 '15 at 00:09
@blorgbeard I am dropping and creating from powershell, maybe that is factoring in on the delay? — Logan Lower, Aug 27 '15 at 12:38
@RBarryYoung We have tried the hotfix and disabling the power service but the issues still occurs unfortunately. — Logan Lower, Aug 27 '15 at 12:39

score 0 · Answer 1 · answered Aug 25 '15 at 23:42

0

It seems like you may not have extensive experience with database technology, so let me propose something different:

string text = System.IO.File.ReadAllText(path);

Then perhaps you can take the text and do what you want with it, dump it in a queue for action in another part of the application.

ReadAllText has some exceptions that are thrown:

https://msdn.microsoft.com/en-us/library/ms143368(v=vs.110).aspx

I'd be on the look out for UnauthorizedAccessException as you said, the file seems to lock up when multiple users are accessing it.

answered Aug 25 '15 at 23:42

Matt

619
3
8
23

I just took a look at the I/O bug that you are having as well... have you tried applying the hotfix? – Matt Aug 25 '15 at 23:43
You are correct, I don't have a lot of experience with DB. and yes the hotfix did not work unfortunately. The issue actually still occurs when even 1 user, as it seems to be caused by the rapid, repeat Read/Write. – Logan Lower Aug 27 '15 at 12:44

Best way to read and write time-critical data?

1 Answers1