How to create dummy data with different datatypes in python?

Question

I am trying to create a logistics dummy dataset for doing some analysis and possible predictions on the data.

Assumed variables are as follows:

 VARIABLES                     RANGES
 awb                           random number eg:235533
 destination_city              random cities
 product                       different products
 product_category              different categories
 origin_city                   random metro cities
 logistics_provider_id         id's eg:1,20,28,27
 dispatch_date                 datetime between mar01-2015 to mar15-2015
 final_delivery_status         created,delivered,returned
 actual_delivery_date          datetime between mar16-2015 to mar30-2015
 promised_delivery_date        datetime between mar25-2015 to Apr6-2015

So, from the above variables assumed i want to create the dummy data with in the ranges mentioned. How can i create the dummy data using python

Expected output:

example_dummy_data:

  awb        destination_city   product               product_category
1 104842891  Byatarayanapura    Wrangler Denim Jeans  Men's Clothing
2 104842938  Bareilly           Sky Blue Denim        Men's Clothing
3 104842942  Saharanpur         puma shoes            Men's Footwear
4 104842943  Saharanpur         classic puma shoes    Men's Foorwear
5 104843066  Mumbai             Elegant black belt    Fashion Accessories

  origin_city  log_prov_id   dispatch date          final_del_status
1 Gurgaon      18            2014-09-02 00:26:11       DEL
2 Bangalore    19            2014-09-01 23:34:30       RTN
3 New Delhi    18            2014-09-01 18:59:41       RTC
4 New Delhi    15            2014-09-02 00:05:33       DEL
5 Hyderabad    16            2014-09-01 22:09:14       UDL

  Actual_del_date        promised_del_date
1 2014-09-03 00:00:00   2014-09-05 20:00:00
2 2014-09-04 00:00:00   2014-09-06 20:00:00
3 2014-09-04 00:00:00   2014-09-06 20:00:00
4 2014-09-04 00:00:00   2014-09-07 20:00:00
5 2014-09-02 00:00:00   2014-09-06 20:00:00

I want to create the data with 10000 rows like above,Is there any best way to create in the ranges mentioned above

Tried:

import random
a = [int(10000*random.random()) for i in xrange(10000)]

Found how to generate random numbers but not with in the ranges and cities i want. So Please help me how to create the dummy data like i have mentioned with 10000 rows with in the ranges i have mentioned.

for example dispatch_date variable ranges of dates should be between mar-01-2015 to mar-15-2015.Between these two dates i want to generate 10,000 datetimes .So how can i do that in python — Surendra babu Pasumarhti, May 26 '15 at 10:19

chthonicdaemon · Accepted Answer · 2022-02-19T10:57:49.150

The faker package is built for this kind of use-case. It handles the names, integers and dates already, although you will probably want to add your own products and product categories.

import pandas
from faker import Factory
import random

faker = Factory.create()
status = 'created,delivered,returned'.split(',')

def date_between(d1, d2):
    f = '%b%d-%Y'
    return faker.date_time_between_dates(datetime.strptime(d1, f), datetime.strptime(d2, f))

def fakerecord():
    return {'awb': faker.numerify('######'),  # random number eg:235533
            'destination_city': faker.city(),  # random cities
            'product': 'random_product',  # different products
            'product_category': 'random_category',  # different categories
            'origin_city': faker.city(),  # random metro cities
            'logistics_provider_id': faker.numerify('##'),  # id's eg:1,20,28,27
            'dispatch_date': date_between('mar01-2015', 'mar15-2015'),  # datetime between mar01-2015 to mar15-2015
            'final_delivery_status': random.choice(status),  # created,delivered,returned
            'actual_delivery_date': date_between('mar16-2015', 'mar30-2015'),  # datetime between mar16-2015 to mar30-2015
            'promised_delivery_date': date_between('mar25-2015', 'apr06-2015'),  # datetime between mar25-2015 to Apr6-2015
            }

example_dummy_data = pandas.DataFrame([fakerecord() for _ in range(1000)])

Thanks for the help, but how do i get indian metro cities and how do i get 100 2 decimal points random numbers with in the range of 0.50 to 10.50. — Surendra babu Pasumarhti, May 27 '15 at 07:47
And just checked ,i am unable to import the Factory by using from faker import Factory after cloning into the faker.Can i know what is the problem — Surendra babu Pasumarhti, May 27 '15 at 07:57
There is a [Hindi localisation](http://fake-factory.readthedocs.org/en/master/locales/hi_IN.html) for the Faker project, but it appears as though you actually want English spellings of Indian cities, which is not currently supplied. You may be able to use a strategy similar to what I used with the status here and [this list](http://en.wikipedia.org/wiki/List_of_cities_and_towns_in_India_by_population) of Indian cities. — chthonicdaemon, May 27 '15 at 07:58
Please install using the recommended install methods [on their documentation](http://fake-factory.readthedocs.org/en/master/index.html#basic-usage) — chthonicdaemon, May 27 '15 at 07:59

score 5 · Answer 2 · edited May 23 '17 at 10:30

Found how to generate random numbers but not with in the ranges and cities i want. So Please help me how to create the dummy data like i have mentioned with 10000 rows with in the ranges i have mentioned.

Random Range:

from random import randint

xs = randint(0, 1000)  # random int between 0 and 1000

Random Selection:

from random import choice

cities = ["Brisbane", "Sydney", "Melbourne"]

random_ciy = choice(cities)  # A randomly selected city from cities

Random Date: (Thanks to Generate a random date between two other dates)

from random import randrange
from datetime import timedelta

def random_date(start, end):
    """Return a random date between two datetime objects start and end"""

    delta = end - start
    int_delta = (delta.days * 24 * 60 * 60) + delta.seconds
    random_second = randrange(int_delta)

    return start + timedelta(seconds=random_second)

Output:

>>> random_date(datetime(2015, 06, 1), datetime(2015, 9, 1))
datetime.datetime(2015, 7, 19, 11, 59, 46)

See:

The rest if up to you how you construct your dataset

! Thanks for the help. the above code is in very detail but my point is for instance from random import choice cities = ["Brisbane", "Sydney", "Melbourne"] random_ciy = choice(cities) gives us single city but how can i generate those 3 cities 100 times — Surendra babu Pasumarhti, May 26 '15 at 09:40
what i mean is how can i generate cities 10 times with in those three cities you have mentioned — Surendra babu Pasumarhti, May 26 '15 at 10:01
I'm sorry; as much as I'd like to "solve all your problems" you need to do a *little work*. Iteration. — James Mills, May 26 '15 at 11:33

How to create dummy data with different datatypes in python?

2 Answers2