0

I have the name of files in the list with folders. The list contains 2000 file names like this

Countries/US/newyork/file1.pdf
Countries/Australia/Sydney/file1.pdf
Countries/Canada/Toronto/bla/blabla/file2.pdf

and so on.

I want to index those files in database so that i can have hierarchical directory structure.

IN my Django app i want to display first the root level menus like

countries --- US , Australia, canada

Then i someone click on country then it get the second level of folders and so on and in end i want to see files if there are no more folders.

rather than querying my storage evry time , i want to store all that info in my database so that my web pages are displayed from DB and when user click download then i get the file from my Storage

i am not able to find how should i make the Model or database table for that

user1958218
  • 1,571
  • 3
  • 18
  • 28
  • Can you rephrase the first sentence "I have the name of files in the list with folders." to something for understandable? Which list? – Jorge Leitao Jun 21 '13 at 07:17
  • @J.C.Leitão , I am getting that list from amazon S3 and i want o index the list in my database – user1958218 Jun 21 '13 at 08:22

1 Answers1

1

I suggest following way:

Create models to store your tree structure and files for example:

class Node(TreeModel):
    parent # foreign key to Node

class File(Model):
    node # foreign key to Node
    name # name of file
    path # path to the file on disk for example

After that move your files in one or few directories (read this How many files can I put in a directory?) also you can rename them (for example by using hash from files).

Update the model File to put there new paths to your files.

Having done this you are able to easy show files and build path to files etc.

For the model Node use [django-mptt][1] (there are other solutions for django, google it) to get an efficient API to manage a Tree-like model.

You can also create your own Django Storage Backend (or find there are many solutions on the Internet).

Updated

You can add new files by using django admin. You should use amazon s3 django storage backend http://django-storages.readthedocs.org/en/latest/backends/amazon-S3.html.

Change:

class File(Model):
    node # foreign key to Node
    name # name of file
    file # django models.FileField

In this case you have not to update index.

Community
  • 1
  • 1
Andrei Kaigorodov
  • 2,145
  • 17
  • 17
  • thanks for the info. I am using amazon S3 as my storage. Also i am thinking of building the index every week or when i add new files. I am thinking of first removing the old notes and repopulating every time. i run the cron job of updating the file index. Is that ok – user1958218 Jun 21 '13 at 08:43
  • i didn't undertsand , why i don't need to update the index. because suppose i delete 3 files from S3 but my database will still have those files – user1958218 Jun 25 '13 at 04:36
  • I mean, if you use django-admin to manage your files you don't need to update index. For example if you are going to delete or create a file you should delete or create it through django-admin. (for delete you have to create signal in which you delete a file physically http://stackoverflow.com/questions/5372934/how-do-i-get-django-admin-to-delete-files-when-i-remove-an-object-from-the-datab). Of course if you use external tools to manage your files that don't work through django you should maintain your index (tables in this case). – Andrei Kaigorodov Jun 25 '13 at 06:28