Storing Data from a CSV File into a database

Question

I have a Django model of a hostel with one of the field FileUpload from which I will take a CSV file and populate the database, I have been trying for past two days searching how to do that, but could not get that working This how to import csv data into django models question works but only if I have the path for the file, and I tried many ways but were not successful. Also after that I tried searching harder How do I get a files absolute path after being uploaded in Django? I saw this but even then I could not get it working as it shows some errors

I used this URL mapping url(r'^test/$',views.FileUpload.as_view(),name="test")

But it is showing some namespace error. Please tell me how should I do it, if you have some suggestions to me how should I start again, I am feeling really lost. I want to get a file from a user and then populate the database

Can this(How do I transfer data in .csv file into my sqlite database in django?) be made like we could create a form with a filefield that accepts csv and then populating the database by parsing that, Problem in that is we could not hard code the path. So how should we do that.

EDIT views.py

from django.views import View
class FileUpload(View):
    def post(self, request):
        form = DocumentForm(request.POST, request.FILES)
        if form.is_valid():
            initial_obj = form.save(commit=False)
            data=initial_obj['document']
            with open((data),'rb') as csvfile:
                spamreader = csv.reader(csvfile)
                for row in spamreader:
                    _, created=Document.objects.get_or_create(
                    name=row[0],
                    description = row[1],
                    importance=row[2],
                    )

            initial_obj.save()
            form.save()
            return redirect('/')
        else:
            return render(request,'file_upload_form.html',{'form':form})

    def get(self, request):
        return render(request, 'file_upload_form.html', {'form': form,})

forms.py

class DocumentForm(forms.ModelForm):
    class Meta:
        model = Document
        fields = ('description', 'document', )

models.py

class Document(models.Model):
    name=models.CharField(max_length=50,blank=True)
    description=models.CharField(max_length=255, blank=True)
    importance=models.CharField(max_length=10, default="High")
    document = models.FileField(upload_to='documents/')
    uploaded_at = models.DateTimeField(auto_now_add=True)

    def __str__(self):
        return self.description

I am not getting any errors but it is not saving data in the database from the CSV file and redirects me to the same page where uploading has to be done and shows the error "This field is required" for the document field.

Updates: I solved the problem more details here

Moe Far · Answer 1 · 2017-06-03T11:32:50.287

0

You can easily override the clean() method of your form, and save your custom data content into database

 def clean(self):
    super(DocumentForm, self).clean()
    import csv
    with open((self.cleaned_data['file']), 'rb') as csvfile:
    spamreader = csv.reader(csvfile)
    for row in spamreader:
        // DO WHATEVER YOU WANT

edited Jun 03 '17 at 11:32

answered Jun 03 '17 at 11:31

Moe Far

2,742
2
23
41

I am a newbie, elaborate how please. – 4rshdeep Jun 03 '17 at 11:32
add this code into your `DocumentForm` , then iterate on your file and save it into database – Moe Far Jun 03 '17 at 11:33
@unpaired_electron saw your question on the other [post](https://stackoverflow.com/a/44343404/1526703) and followed back to check if you still need help. In Moe Far's response, _file_ is the name of the field that accepts the path of the csv file (so you dont have to hard code it). Once you add the above code as Moe suggested, the user can simply enter the path on the front-end and you can feed the csv into the database (how you populate the db from the csv, goes in 'do whatever you want' part in the code above). Post back if you need more clarification – Anupam Jun 03 '17 at 11:57
Sir, I am getting an error 'form' is not defined In views.py(I Fixed that missed form in get), thing is that I could not find documentation about how to use django.views.View (https://docs.djangoproject.com/en/1.11/ref/class-based-views/base/). I have looked at it but I feel it's not enough for a newbie like me. ***TL, DR: I am getting an error 'form' is not defined.*** – 4rshdeep Jun 03 '17 at 13:00
Now, Everything else is working, but when I click on upload, it does not do anything, it goes to the same page showing file is required. – 4rshdeep Jun 04 '17 at 04:40

Storing Data from a CSV File into a database

1 Answers1

Linked