handling of large file in svnsync

Question

We plan to have a SVN mirror repository in another office of ours in Sydney. We use VisualSVN server v2.5.7 in both locations.

I decided to use svnsync to do it. At first I wanted to sync all of our repositories and when all of them are synced with mirror repository, a scheduler will call svnsync every midnight.

It could sync 167 revisions of one of our repositories. But on the 168th revision we have a big file (a zipped oracle file of about 250 MB) that cannot be synced. Even though I modified the timeouts of both our local and remote servers it doesn't work. It sticks about one hour at one point and gives me the following error:

Transmitting file data .......................svnsync: E175002: PUT of '/{some path}/{bigfile}.zip': Could not send request body: An established connection was aborted by the software in your host machine. <{target url}>

Here are the modifications I made in the httpd-custom.conf file in Apache server of VisualSVNs (local, mirror):

Timeout 300000
KeepAlive On
MaxKeepAliveRequests 0
KeepAliveTimeout 300000

<IfModule dav_svn_module="">  
  # Enable a 1 Gb Subversion data cache for both fulltext and deltas.  
  SVNInMemoryCacheSize 1048576  
  SVNCacheTextDeltas On  
  SVNCacheFullTexts On
  #SVNCompressionLevel 9
</IfModule>

I even increased the timeout to 600000 or more, but the result was the same. I launched both servers in http mode. On our local network it can sync all of that repository in 20 minutes.

Regarding our internet connection's uploading speed that is about 256 Kbs, I don't expect this time in an internet environment. But I want the SVN servers to wait for the timeout I set for them, because we can easily commit files of these sizes into other SVN servers that use CollabNet Server. It just takes 2 hours to be committed successfully. I think the 300000 seconds timeout is far far from 2 hours.

1. What was the behavior before you modified https-custom.conf with SVNInMemoryCacheSize 1048576, SVNCacheTextDeltas On, SVNCacheFullTexts On? 2. Do you mean that you sync your repos over 256kb connection? Is it GPRS / 3G? I'd like to reproduce the behavior so any additional details on your network environment would be greatly appreciated. — bahrep, Nov 14 '12 at 09:19
1. I checked it without the properties you mentioned. I removed the entire tag and the result was it sticks just 20 minutes before it get same error. — Mehdi, Nov 15 '12 at 06:02
2. Answer 1:We have another repository we commit our pre-production versions into it. We can commit quite large file like the db dmp file I mentioned in the main topic. It uses CollabNet server using svn schema instead of http or https. It shows using svnsrve is much better than using apache http server in slow connections. Answer 2: we use ADSL but in our country we have limitation on internet speed. The maximum speed of internet in here is 1024/512 Kbs (Download/Upload), which we already use (512/256) Kbs. — Mehdi, Nov 15 '12 at 06:17
Finally, I could succeed to sync all the versions of that repository. It seems our connection upload speed dropped bellow the 256 Kbs and SVN Server didn't wait for the timeout to be expired. I think there are some other factors to close a network connection rather than timeout expiration. — Mehdi, Nov 17 '12 at 05:08

score 2 · Answer 1 · edited Jan 19 '16 at 11:48

Upgrade your VisualSVN Server instances to the latest version.

Starting with Subversion 1.8, serf HTTP client library with better performance is used instead of older neon. Therefore, you may see less problems when using svnsync over unstable low-bandwidth connections.

Regarding our internet connection's uploading speed that is about 256 Kbs

Beginning with version family 3.0, VisualSVN Server Enterprise Edition has a special feature that should help you to eliminate the low bandwidth bottleneck: Multisite Repository Replication (VDFS).

Subversion repositories based on VisualSVN Distributed File System replicate more than 10 times faster compared to replication systems based on Write-Through Proxy (as far as I can tell you are using the Write-Through Proxy now).

In addition to this, VDFS supports locking, replicates user access permissions and ensures consistent execution of SVN hook scripts on all replicated repositories.

handling of large file in svnsync

1 Answers1