I have a windows service which must write about 20kB of data in about 20 file over 2 network shares.
Time to write the files with Total Commander: less than 0.1s.
Time to write the files with my application: ca 10s.
What is wrong? Yes, the files are constantly read from both shares, but it should not be an issue since:
public void WriteData(string text, string fileName, bool forceBackup = false) {
foreach (var dir in Locations) {
var path = string.Format(@"{0}\{1}", dir, fileName);
FileStream stream = null;
try {
stream = new FileStream(
path,
FileMode.Create,
FileAccess.Write,
FileShare.Read
);
using (StreamWriter writer = new StreamWriter(stream)) {
stream = null;
writer.Write(text);
}
}
catch (Exception) { } // irrelevant now, tested it doesn't throw exceptions anyway
finally {
if (stream != null) stream.Dispose();
}
File.SetLastWriteTime(path, DateTime.Now);
}
}
The code above works fine with local files. It takes single milliseconds to write all data to RAM drive. My network shares are located on RAM drives too.
What's important - copying those files to exactly the same locations with Total Commander takes single milliseconds too. I mean - copying to network shares under full load.
There is no sharing violation, and the application writes the files in a single thread. There is no problem when writing to those files from Total Commander, there is no problem with writing to those files with my application without using the network share.
There is no sharing violation, because while writing - those files are only READ by web server, with FileAccess.Read and FileShare.ReadWrite explicitly set.
No, I cannot write those files without using network share since what my service does is synchronization between two servers. No, I can't use DFSR, because the files are updated way too often (2 times per second). No, I can't use 2 separate services to update files on both machines because it would cancel the failsafe feature, when each instance of my service can be stopped without the data update on both servers being stopped.
In details, in my production environment there are 2 instances of this service, when one is updating the files, and the other one constantly monitors if the active one does its job. When a failure is detected, they switch their roles. It all happens in real time and works as charm. With one huge glitch: ultra long delay on writing the files.
If you wonder what File.SetLastWriteTime() stands for - it's a workaround for Windows (.NET) bug when file last write time is not correctly updated with create / write alone. And of course the correct modification time is crucial for the other instance to detect, if the first one is updating the files on time.
Also: I get reported that sometimes some garbage is read from those files. It happens very rarely though. I haven't confirmed this bug.
But the main question is - what is taking so long? There is fast, 1GBit link between my test server and target servers, 10GBit link between production servers. Ping below 1ms. It is NOT A NETWORK ISSUE.
After some more testing I've found buffer size and file options do nothing with the write time. I've found network ping is very important. I cannot test the code on my development machine, because the ping is too large.
Anyway - the code could be optimized to run ca 80% faster if all files was created once, then updated without recreating the streams. It's also very fast when used against local share. Anyway - test code is fast, production code on the very same server is 50x slower. There is however a slight difference - the production files are constantly read by a web server, while the test files are not.
Still - goal not reached. I need 20 x 1kb files be updated twice every second on 2 servers linked with 10GBit/s ethernet. Achieved 200ms delay is acceptable, but it works only with test files, with the shared real files I still get over 6000ms per update.
BTW, leaving files open is not an option here, when reliability is critical. The service should be able to switch all updating to an another instance seamlessly, in case any of file would be deleted, or any network, DB or disk error would occur. Leaving files open could lead to sharing violations, memory leaks and other disasters. The correct handling of constantly open files would also be very complicated which would make the code even harder to debug.
Maybe there's another way to share data between servers? Maybe a zip file could be used to upload files, and then another service would unzip the files? I'm positive that zipping and unzipping 1kB of data would not ever take as long as 1 second!
