8

We have a requirement to create large (1G-16G) row data reports compress and encrypt them. Our customers will consume those reports over sFTP. We are replacing an existing implementation so our customer should get this change transparently.

Azure Blob service does not expose sFTP service so we will need some way to facade it with sFTP service. Something similar to FTP to Azure Blob Storage Bridge based on worker role. The worker role will expose sFTP endpoint to the outside world. We will setup a container per customer and limit the access from worker roles only so containers will be protected from direct access.

My questions are:

  1. What do you think of such approach?
  2. Will the sFTP using worker role can be dynamically scaled/shrink while keeping the same DNS name from customer point of view?
  3. Does Azure Blob service support compression or encryption?
  4. Are you familiar with sFTP worker role similar to FTP to Azure Blob Storage Bridge (preferable open source)?

Related Questions:
In Windows Azure: What are web role, worker role and VM role?
Azure Architecture Design

Community
  • 1
  • 1
Haim Raman
  • 11,508
  • 6
  • 44
  • 70
  • How about creating VM and setting up IIS FTP on the folder on VM's VHD. VHD will persist on Blob Storage anyway, you will not have to make too many changes to your normal design. Debugging and maintaining worker roles are painful, that's why Azure websites came in picture and we are more comfortable with that. – Akash Kava Feb 12 '14 at 07:11
  • I need sFTP and IIS provides FTP only; anyway if I will need to intercepts the sftp calls myself and manipulate them I will prefer a java based solution like Apache SSHD. I hoped that using worker roles will help in automatic scale for the sftp access. I am missing a component that will route DNS query to the right worker role. Why did you found worker roles so painful? – Haim Raman Feb 12 '14 at 08:22
  • If you need Java based solution then VM is the best, you can install Java, Tomcat or any software you need. VM can have multiple hard disks and you can scale up when you need more performance. Worker roles do not have ability to install third party software. Worker role has complete different management hassle, one wrong config and you spend hours to figure out what is wrong. – Akash Kava Feb 12 '14 at 08:38
  • You got me confused here I was reading: http://blogs.msdn.com/b/mariok/archive/2011/01/05/deploying-java-applications-in-azure.aspx and it looks like you can deploy a war/application server as a worker role – Haim Raman Feb 12 '14 at 09:00
  • Ok I take my words back, you can install other things in worker role, but it is still complex, not as easy as it is in VM, ultimately worker roles run on some VM, configuring it is nightmare. We are all familiar with Windows OS and its easy to go through remote desktop and investigate things. Worker role does it automatically, but VM will give you everything that worker role has to offer. – Akash Kava Feb 12 '14 at 10:21
  • Just in case you find it useful, Amazon now offers an "AWS Transfer for SFTP" that acts as a SFTP gateway to S3 (the Azure Blob Storage equivalent in AWS) https://aws.amazon.com/sftp/features/. So it does not sound as a crazy idea. You might find some ideas you can translate to Azure there. – Guido Mar 12 '19 at 11:58

1 Answers1

2

You could possibly achieve this by simply exposing the blob storage endpoints directly over HTTPS and look to use Shared Access Signatures (http://www.windowsazure.com/en-us/documentation/articles/storage-dotnet-shared-access-signature-part-1/) to restrict access to those blobs.

Based on your feedback then - perhaps look to leverage Linux to run an sFTP server and use either the Java, Node or PHP Azure SDKs to achieve the same objective of pulling your content from blob storage on a scheduled basis without the need of a worker role (cron should see you right). To be honest you'll be doubling your storage use (not that it's that expensive) as you'll pull the blob out of storage and onto the VHD of your VM but this approach would work.

I notice you said you were encrypting the files written to blob storage - you could just use a PKI approach and share the key with customers allowing them to pull directly via HTTPS. Anyway, sFTP it appears to be :).

Simon W
  • 5,481
  • 3
  • 24
  • 35
  • Thanks you for your response, I am not sure this will work for us. The Customers are decupled from the Azure Blob storage accessing their resource only via sFTP. The worker role should bridge all requests from sftp to Azure Blob. As I see it, the permissions are only an extra security layer to make sure that in case something is buggy with the sFTP bridge the storage will limit the access – Haim Raman Feb 12 '14 at 04:11
  • We are not intending to keep two copies of the files. Our thoughts are: When a sFTP request will hit the worker role we will intercept the request, resolve the customer name from authentication and requested file from the sFTP request. This will give us the Azure Blob Storage url. Then we will open a connection to Azure and serve the file acting as a proxy. Anyway if there is some existing implantation for this we will prefer it the witting our own implementation – Haim Raman Feb 12 '14 at 04:59
  • The problem with your approach will be having to deal with transient failures in retrieving content from Azure blob storage. You may also want to do some performance testing to ensure throughput will mean you can easily stream a 16GB file at an adequate rate to satisfy your sFTP service. – Simon W Feb 12 '14 at 05:45