Bulk uploading and creating pages with images in wagtail (migration)

Question

I'm creating a website with wagtail to replace someone's existing weebly site. It would take hours to re-create each of the hundreds of page instances and upload each image for each of those pages.

I already have the page models I need, and my site looks a lot like the getting started tutorial from the wagtail docs. I'm wondering how I might be able to script migrating this content. When searching for answers I'm finding more information about programmatically creating models, rather than pushing the content itself to my wagtail site.

I have already scraped the old site and saved all the images needed, and I have JSON data in the following format:

[
    {
        "page_name": "first page",
        "images": [
            {
                "url": "http://www.a.com/final-1.jpg",
                "filename": "final-1.jpg",
                "caption": "A caption"
            },
            {
                "url": "http://www.a.com/final-2.jpeg",
                "filename": "final-2.jpeg",
                "caption": ""
            }
        ],
        "body": "Body text goes here. "
    },
    {
        "page_name": "page 2",
        "images":

...

I suspect others have faced this issue in the past. I continue to be grateful for the community and all your contributions. Cheers!

If you're talking a one-time population of a database, then you could create a Django management command: https://docs.djangoproject.com/en/3.0/howto/custom-management-commands/. Alternatively, you could create a data migration if that is more appropriate for your use case: https://docs.djangoproject.com/en/3.0/topics/migrations/#data-migrations — Dan Swain, Jul 30 '20 at 21:26
Thanks! This has got me on the right path (still a beginner). Helped me find this which helps me understand the file management in cunjonction with django migrations. https://stackoverflow.com/questions/50063542/django-wagtail-csv-and-photo-upload-management-command — dev-jeff, Jul 30 '20 at 23:17

score 3 · Accepted Answer · answered Jul 31 '20 at 16:04

Got it to work.

data.json is stored in the site root folder, import_content.py is in blog/management/commands

then run pipenv run py manage.py import_content

from django.core.management.base import BaseCommand, CommandError
from blog.models import InstallationPage, GalleryImage, Gallery, InstallationMedium

from wagtail.images.models import Image
from django.core.files.images import ImageFile
from io import BytesIO

import json, os
from datetime import datetime
from slugify import slugify

class Command(BaseCommand):
    help = 'Importing pages and image content, for initial migration.'


    def handle(self, *args, **options):
        with open('data.json', 'r') as f:
            data = json.load(f)['data']
            for page in reversed(data):
                name=page['name']
                body=page['body']
                images=page['images']

                parent = Gallery.objects.first()

                new_page = InstallationPage(
                    title=name,
                    slug=slugify(name),
                    date=datetime.today(),
                    body=json.dumps([{'type': 'paragraph', 'value':body}]) if len(body) else None,
                    mediums=[InstallationMedium.objects.get(name='Painting')]
                    )
                self.stdout.write(f"Initialized page {name}")
                saved_images = []
                for img_data in images:
                    path = os.path.join(r"C:\path\to\image\files",img_data['filename'])

                    with open(path,"rb") as imagefile:

                        image = Image(file=ImageFile(BytesIO(imagefile.read()), name=img_data['filename']), title=name+'-'+img_data['filename'].rsplit('.',1)[0])
                        image.save()
                        gallery_image = GalleryImage(
                            image=image,
                            caption=img_data['caption']
                            )
                        saved_images.append(gallery_image)
                        self.stdout.write(f"    Saved image {img_data['filename']} to database")


                parent.add_child(instance=new_page)
                new_page.save_revision()

                new_page.gallery_images=saved_images
                new_page.save_revision().publish()

                self.stdout.write(f"        Attached images to {name}.")

                self.stdout.write(f"Published page {name} with {str(len(images))} images.")

Bulk uploading and creating pages with images in wagtail (migration)

1 Answers1

Linked