Connecting to Azure storage account thru proxy server Microsoft Azure Storage SDK for Java

Question

In our project we need to access the Blob Storage through a Proxy Server (squid).

We are planning to use the Microsoft Azure Storage SDK for Java version 2.2.0. But it looks like setting the proxy is not provided by the API. The only way I could make it go through the proxy is by setting the System properties

System.setProperty("http.proxyHost", "127.0.0.1");
System.setProperty("http.proxyPort", "3128");

But this affect all services that are running on my JVM which harms other services that not supposed to go via the Proxy.

Looking at the java code it looks like com.microsoft.azure.storage.core.BaseRequest.createURLConnection(URI, RequestOptions, UriQueryBuilder, OperationContext). Is calling java.net.URL.openConnection() without proxy. While using java.net.URL.openConnection(Proxy) could provide the required support?

It looks wired to me that this is not supported?
Do I miss something here?

UPDATE: I opened an issue on this in azure-storage-java git, I would be happy to get your input as I want to suggest a pull request for this.

Answer 1

So far there have been no Java SDK API support access directly Azure Storage through proxy server, because BaseRequest Class miss "url.openConnection(proxy)" in the function "public static HttpConnection createURLConnection(...)".

Per my experience, there are two ways to help you implement the access function.

The one is that you can use Azure Storage REST API through the java.net.Proxy Class to access storage service.

Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress(host, port));
URLConnection conn = url.openConnection(proxy);
And if you should be authorize proxy user & password, you can do it as the follows:
//Proxy-Authorization: Basic <Base64.encode(user:password)>
String headerKey = "Proxy-Authorization";
String headerValue = "Basic " + Base64.encode(user+":"+password);
conn.setRequestProperty(headerKey, headerValue);

The last one is that you can modify Azure SDK API and overwrite the method “createURLConnection” in Class “BaseRequest” to implement accessing. The Azure Storage SDK v2.2.0 project on GitHub is https://github.com/Azure/azure-storage-java/tree/v2.2.0/.

Note:

public static HttpURLConnection createURLConnection(final URI uri, final RequestOptions options, UriQueryBuilder builder, final OperationContext opContext, java.net.Proxy proxy)

and

final HttpURLConnection retConnection = (HttpURLConnection) resourceUrl.openConnection(proxy);

public static HttpURLConnection createURLConnection(final URI uri, final RequestOptions options, UriQueryBuilder builder, final OperationContext opContext, java.net.Proxy proxy) throws IOException, URISyntaxException, StorageException {
    if (builder == null) {
        builder = new UriQueryBuilder();
    }

    final URL resourceUrl = builder.addToURI(uri).toURL();

    final HttpURLConnection retConnection = (HttpURLConnection) resourceUrl.openConnection(proxy);

    if (options.getTimeoutIntervalInMs() != null && options.getTimeoutIntervalInMs() != 0) {
        builder.add(TIMEOUT, String.valueOf(options.getTimeoutIntervalInMs() / 1000));
    }

    // Note: ReadTimeout must be explicitly set to avoid a bug in JDK 6.
    // In certain cases, this bug causes an immediate read timeout exception to be thrown even if ReadTimeout is not set.
    retConnection.setReadTimeout(Utility.getRemainingTimeout(options.getOperationExpiryTimeInMs(), options.getTimeoutIntervalInMs()));

    // Note : accept behavior, java by default sends Accept behavior as text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
    retConnection.setRequestProperty(Constants.HeaderConstants.ACCEPT, Constants.HeaderConstants.XML_TYPE);
    retConnection.setRequestProperty(Constants.HeaderConstants.ACCEPT_CHARSET, Constants.UTF8_CHARSET);

    // Note : Content-Type behavior, java by default sends Content-type behavior as application/x-www-form-urlencoded for posts.
    retConnection.setRequestProperty(Constants.HeaderConstants.CONTENT_TYPE, Constants.EMPTY_STRING);

    retConnection.setRequestProperty(Constants.HeaderConstants.STORAGE_VERSION_HEADER,
        Constants.HeaderConstants.TARGET_STORAGE_VERSION);
    retConnection.setRequestProperty(Constants.HeaderConstants.USER_AGENT, getUserAgent());
    retConnection.setRequestProperty(Constants.HeaderConstants.CLIENT_REQUEST_ID_HEADER,
        opContext.getClientRequestID());

    return retConnection;
}

By the way, You need to call above method in every CloudXXXClient(CloudBlobClient, etc) Class.

Answer 2

Following the issue-48 opened by me based on this question and additional one opened by strazh issue-65, The proxy support was improved in version 4.2.0 see here.

Added support for setting a library-wide proxy. The default proxy can be set on OperationContext.

See the JUnits for full example https://github.com/Azure/azure-storage-java/blob/master/microsoft-azure-storage-test/src/com/microsoft/azure/storage/GenericTests.java

Look for testDefaultProxy and testProxy

Answer 3

Azure Storage team has released a new SDK (v10), where the Proxy is now supported through the HttpPipeline. You can share the pipeline across all operations by passing it to StorageURL or just use in a single Blob by passing it to the BlobURL object.

AnonymousCredentials creds = new AnonymousCredentials();

// Use PipelineOptions to define a retry strategy and a proxy - you can also pass your own HttpClient to this if you like
PipelineOptions po = new PipelineOptions();

// Proxy configuration shown here as a sample
HttpClientConfiguration configuration = new HttpClientConfiguration(
      new Proxy(Proxy.Type.HTTP, new InetSocketAddress("localhost", 8888)), false); //pass true if the Proxy endpoint is HTTPS
po.client = HttpClient.createDefault(configuration);


// Create a URI with SAS token, and pass it to the PageBlobURL with an HttpPipeline created by createPipeline method
URL u = new URL("https://myaccount.blob.core.windows.net/mycontainer/myfile?sv=2017-04-17&sr=b&si=14169767-6354-41ed-a99b-c9db8dcc66bc&sig=8NUr%2BSpmRH%2BB2z%2FpQZDPDquTQ7rbgWfE9a6AePLlFT0%3D");
PageBlobURL blobURL = new PageBlobURL(u, PageBlobURL.createPipeline(creds, po));