Is it better to pull files from remote locations or grant users FTP access to my system?

Question

I need to setup a process to update a database table with user supplied CSV-data (running Coldfusion 8/MySQL 5.0.88).

I'm not sure about the best way to do this.

Should I give users FTP-access to my system, generate a directory for every user and upload files from there, or should I pick files up from external locations, so the user has to setup an FTP folder my system can access. I'm sort of leaning towards the 2nd way and wanted to set this up using cfschedule and cfftp, but I'm not sure this is the best way to go forward. Security wise, I'm mor inclined to have users specify an FTP location, from where I pull, rather than handing out and maintaing FTP folders for every user.

Question:
Which approach is better both in terms of security and automation?

Thanks for input!

score 5 · Answer 1 · answered Dec 01 '12 at 19:56

5

I wouldn't use either approach. I would give the users a web page to upload their csv files. The cf page that accepts the files would place them into a specific folder and make sure they have unique filenames. The cffile tag will help you with that.

The scheduled job would start with a cfdirectory tag on the target folder. This creates a query object. Loop through it and do what you have to do with each file.

Remember to check for the correct file extension. Then look at the first line of the file to ensure it matches the expected format.

Once you have finished processing the file, do something with it so that you don't process it again on the next scheduled job.

answered Dec 01 '12 at 19:56

Dan Bracuk

20,699
4
26
43

something like that. Only I will have to update every 15mins, so I need some sort of automatic file drop and pickup. I have manual CSV uploads, but sure needs manpower to manually update every 15min :-) – frequent Dec 01 '12 at 20:00
1

You don't manually update every 15 minutes. Your scheduled job runs every 15 minutes and does your updating based on whatever csv files have been uploaded. – Dan Bracuk Dec 01 '12 at 20:32
1

+1 to Dan's answer. I'll just add that for automated processing of user-uploaded files you have numerous options: continuously-running scheduled task like Dan suggested; or set up a DirectoryWatcher gateway. If I was doing this, I'd probably go with a database-base file queue + scheduled task - this way you can keep a record of who uploaded which file and when, and notify users back if/when csv was processed (or had errors and needs editing and re-uploading). – azawaza Dec 03 '12 at 12:54

score 2 · Answer 2 · answered Dec 01 '12 at 19:53

2

Setting up a custom FTP server is certainly a possibility, since you are able to create users, and give them privileges (automated). It is also secure. But I don't know the best place to start if you don't have any experience with setting up a FTP server.

answered Dec 01 '12 at 19:53

Tim

5,521
8
36
69

I currently have both running ( some users upload to my system via my sys-admin, some users I'm handling through a remote FTP server). I'm looking for a clean solution, which is both secure, easy for users and which I can automate. So setting up is not the problem, I'm just curious what is the better approach: External FTP or Internal Access for Users? – frequent Dec 01 '12 at 19:58
I suppose both of the approaches can be as secure and automated as possible. The question would rather be, which is the easiest to implement? I would go for FTP, since you have a very large range of existing systems which makes it very simple to create a environment for your needs. Edit: Suposing you have your own server. – Tim Dec 01 '12 at 20:04

score 0 · Answer 3 · 2012-12-01T19:59:26.283

0

Try https://www.dropbox.com/

a.)Create a dropbox account,send invites to your users/clients.

b.)You can upload files/folders into dropbox,your clients/users can access it from their dropbox account/dropbox desktop app..

c.)Your users/clients can upload files/folders and you can access it from your dropbox
website account/desktop app. Dropbox is rank 1 software, better in security and automation.

Is it better to pull files from remote locations or grant users FTP access to my system?

3 Answers3