0

There seems to be very little information regarding loading Entities with parent keys using the bulkloader for App Engine. I tried following the instructions that I found on here on StackOverflow...

Uploading Entity with Parent Using Bulkloader

But there must be something I'm still not getting. I was hoping you all could help.

I am trying to load new Entities and give them the parent key of an Entity already in the Datastore. The parent Entity is basically just an "Admin" entity. I want all the "Car" entities that belong to the administrator to have this parent key. This way I can easily search for just those Cars.

The admin.yml and admin.csv are as follows... (I'm skipping the preamble & "transformers")

ADMIN.YML

    - kind: Admin
      connector: csv

      property_map:

            - property: __key__
              external_name: adminKey
              export_transform: transform.key_id_or_name_as_string

            - property: email
              external_name: email

ADMIN.CSV

    email
    foo@foo.com

This works fine. A single Admin entity is created.

Now I want to use this entity as the parent of the "Car" entities that I load next. The car.yml and car.csv are below... (again skipping the preamble & "transformers")

CAR.YML

    - kind: Car
      connector: csv

      property_map:

            - property: __key__ 
              external_name: carKey
              import_transform: transform.create_deep_key(('adminKey', 'adminKey'),('carKey', transform.CURRENT_PROPERTY))

              export:
              - external_name: adminKey
                export_transform: transform.key_id_or_name_as_string_n(0)
              - external_name: carKey
                export_transform: transform.key_id_or_name_as_string_n(1)

            - property: manufacturer
              external_name: manufacturer

            - property: model
              external_name: model

CAR.CSV

    manufacturer,model
    Chevrolet,Impala
    Ford,Focus

Each time I run car.yml with car.csv, I get the message:

[ERROR ] Error in WorkerThread-0: 'adminKey'

I am simply lost at this point. Posting links mostly likely won't help. I've probably seen all of them and still can't seem to understand.

Thanks in advance.

Community
  • 1
  • 1
edcincy
  • 321
  • 2
  • 3
  • 14

1 Answers1

1

The correct syntax is transform.create_deep_key((kind, value),(kind, value),...)

You need something like this:

transform.create_deep_key(('Admin', 'adminKey'),('Car', transform.CURRENT_PROPERTY))
                              ^          ^         ^               ^
                        parent kind parent val  child kind       child val

and then you would need to add adminKey into your car.csv file - your export looks like it would create the correct csv for exporting existing entities, but for importing new entities, you'll need to add that column manually.

crazystick
  • 590
  • 5
  • 14
  • So when you say "you would need to add adminKey into your car.csv file", you mean I need to copy/paste the actual Key value of the Admin entity from the Datastore into the CSV? I was under the assumption that it would use the Entity (Admin) and find/use the Key as it was doing the import. Apparently this is incorrect? – edcincy Feb 05 '14 at 16:15
  • Ok, I have basically figured this out. Thanks so much for the help. I need to put keys for BOTH "adminKey" and "carKey" in my CSV file. But this forces me to have to come up with my own keys for the Car entities. I would rather have the Datastore create the keys for me. Is there any way to allow the datastore to create the "carKey" automatically when bulk uploading, and link it to the parent? – edcincy Feb 05 '14 at 19:01
  • If you have a blank it should generate keys. I haven't tested this for a parent-child import but you could try `transform.create_deep_key(('Admin', 'adminKey'),('Car', transform.CURRENT_PROPERTY, key_is_id=True))` and leave carKey column blank or try to use `transform.none_if_empty` although that may have already been tried, see [this question](http://stackoverflow.com/questions/11873895/how-can-i-use-none-if-empty-with-create-deep-key) – crazystick Feb 06 '14 at 01:40
  • Using `key_is_id=True` is invalid. Changing it `transform.create_deep_key(('Admin', 'adminKey'),('Car', transform.CURRENT_PROPERTY, True))` with the "carKey" column empty, I get **Error in WorkerThread-0: invalid literal for int() with base 10: ' '** which means it's looking for a value in the "carKey" column. Using `transform none_if_empty(transform.create_deep_key(('Admin','adminKey', True),('Car', transform.CURRENT_PROPERTY, True)))` and leaving the "carKey" column blank, the Entities are imported with auto generated IDs, but without a Parent Key! – edcincy Feb 06 '14 at 22:18