1

I have a scenario in which user uploads a file and a django app encodes the file into N distinct segments, each, to be stored on a different remote server. Say if we have 5 segments as the result of the encoding, from only 3 present segments, the file can be constructed (the reason for segmentation)).

the problem however is the distribution of the segments since it seems to be an unconventional way of storage in django; I consulted with the developer of django-Restro package and he suggested to write a django storage back-end from scratch which sounded a herculean task for me as a newbie in django and a close deadline.

After lots of digging someone suggested to use multiple databases and routing, I read the documentation in that regard and from what I grasped, multiple databases seemed to have other purposes. If this was the case, I think my scenario was easily achievable by defining one model per segment and link each to a corresponding database. Is it practical?

If not what way would you suggest for the storage part? My main struggle is in the part where the server wants to store each segment on a different remote server and later upon the user request, wants to retrieve them.

Amir Afianian
  • 2,679
  • 4
  • 22
  • 46
  • intresting question. i have never tried anything like that before. maybe this will help: http://stackoverflow.com/questions/6830564/how-to-use-django-models-with-foreign-keys-in-different-dbs – sobolevn Oct 13 '15 at 09:12
  • you may try to use `ForeignKey`s to file parts, which are stored in other databases. – sobolevn Oct 13 '15 at 09:13
  • 1
    If all remote databases share same db schema then you can load file segments into different instances by using "MyModel.objects.using('database_settings_name')" – Murali Mopuru Oct 13 '15 at 09:18
  • 1
    A different database would not automagically change the physical storage location of the file segment, which is stored on disk. You _need_ a storage backend to handle this. – knbk Oct 13 '15 at 09:59
  • @knbk > Can I not go with Murali Mopuru way? – Amir Afianian Oct 13 '15 at 19:21
  • @sobolevn > indeed, I am surprised that the documentations and projects regarding remote storage are profoundly poor... – Amir Afianian Oct 13 '15 at 19:23
  • The storage backend saves the files, and the default storage backend only ever saves the files in a single location. Saving the path to the file in a different database is not going to change that. Anyway, the documentation ([the API](https://docs.djangoproject.com/en/1.8/ref/files/storage/#the-storage-class) and a [how-to](https://docs.djangoproject.com/en/1.8/howto/custom-file-storage/)) is pretty good, so it shouldn't be too hard to figure out. – knbk Oct 13 '15 at 19:37
  • @knbk "different servers" mean "different locations". – sobolevn Oct 13 '15 at 20:43
  • @sobolevn I'm not sure what you're trying to say. – knbk Oct 13 '15 at 20:54
  • @knbk thank, for the purpose of the storing files on servers, what storage mechanism would you suggest considering my scenario? some would suggest using apache web server or mongoDB for serving the files, what is your idea? – Amir Afianian Oct 14 '15 at 06:22

0 Answers0