2

I'm building a web application that will need to import data from other database servers when it starts.

I would like to have this import done automatically at regular intervals. I would also like to be able to start and stop the import process from my web application.

What would be the best implementation for the import agent - a Windows Service? Something else?

Kev
  • 118,037
  • 53
  • 300
  • 385
maephisto
  • 4,952
  • 11
  • 53
  • 73
  • your question is about databases and database replication but you don't say which database your're using. SQL Server? Have you got SSIS? – Simen S Apr 01 '11 at 12:50
  • @Simen S - Where did you get that this is about replication? – Oded Apr 01 '11 at 12:51
  • 1
    @Oded: I understood the phrase *"other databases"* as *"other databases than my application's database"*. An *import* - in my interpretation - therefore sounds like replication of external database content into the application specific database. – Simen S Apr 01 '11 at 12:54
  • @Simen S - I read that, and the few words before (web application) as meaning that the web application collects data from several databases. – Oded Apr 01 '11 at 12:56
  • @Oded: I guess OP should clarify (which was really what I tried to ask for) – Simen S Apr 01 '11 at 12:57
  • the web application collects data from several databases (mysql, mssql, oracle) and adds it to the application database (oracle). – maephisto Apr 01 '11 at 13:11

4 Answers4

1

If your web application needs to have this data in memory, you can use the Cache class.

Set it to expire every X hours, as you need and when it expires, re-fetch the data..

Oded
  • 489,969
  • 99
  • 883
  • 1,009
  • no, the web application creates live reports on the database. the import agent should refresh the data importing new data from ther databases – maephisto Apr 01 '11 at 12:55
  • @maephisto - You can still use the `Cache` expiration as a "timer". Though "live" and timed imports seem to be at odds with each other. – Oded Apr 01 '11 at 12:57
  • 1
    If it is building *live* reports, you should *not* be doing it on a schedule, but on demand. – Andrew Barber Apr 01 '11 at 12:58
  • yes, true, but the import will take quite some time, so it's better to be made scheduled than haviong the user wait 20 minutes until the data is refreshed. – maephisto Apr 01 '11 at 13:09
1

You could create a Windows Service that uses Quartz.Net to run the scheduled tasks.

You should not run scheduled task from your web app, since you don't have any guarantee that your web app is running. You're at IIS app pool management's mercy.

You might want to look at Best way to run scheduled tasks.

Community
  • 1
  • 1
Marijn
  • 10,367
  • 5
  • 59
  • 80
0

This might be an oversimplification, but can you create a class that does all of this work using a Timer, and then in the application_start of the global.asax, create a BackgroundWorker that kicks off this process?

Your web application could then control the BackgroundWorker object, starting/stopping as necessary.

BrandonZeider
  • 8,014
  • 2
  • 23
  • 20
0

Of what I heard this looks like a description for Microsoft Sync Framework. I have just few information about it for myself but will be pleased to see you pointed into that direction.

I'm not sure about your question because you are talking about hourly syncing. When talking web applications, there can't be a nice way to do such a task. You have to create a console app or best task would be a Windows Service Process (which are easier then it sounds)?

Sync Framework Intro
http://msdn.microsoft.com/en-us/sync/bb821992
Sync Framework Tutorial
http://alexduggleby.com/2007/12/16/sync-framework-tutorial-part-1-introduction/
Sync Framework Samples
http://archive.msdn.microsoft.com/sync

And, when I'm editing the answer with links

Nice guide to create a Windows Service (and setup) http://www.codeproject.com/KB/dotnet/simplewindowsservice.aspx
(if first time, try it on a test project before the production project)

Independent
  • 2,924
  • 7
  • 29
  • 45