I have a column with N row like this
Column A |
---|
1 |
2 |
.. |
.. |
.. |
999,999 |
1,000,000 |
I want outcome like this: ('1','2','3',...,'1000') ('1001','1002',...,'2000') ...
each list contains 1000 value and separate exactly like that.
I have a column with N row like this
Column A |
---|
1 |
2 |
.. |
.. |
.. |
999,999 |
1,000,000 |
I want outcome like this: ('1','2','3',...,'1000') ('1001','1002',...,'2000') ...
each list contains 1000 value and separate exactly like that.
If you have a Python list named my_list you can use:
chunk = 1000
new_list = [my_list[i:i+chunk] for i in range(0, len(my_list), chunk)]
I am not clear about the format of your items, but assuming they are numbers and you want to convert to a list of strings as you seem to indicate then you could use something like the code below and then export to CSV.
result = []
for item in new_list:
x = [str(y) for y in item]
result.append(x)
If you are parsing a CSV, I would suggest you using the pandas
lib like this:
import pandas as pd
df_groups = pd.read_csv('data.csv', chunksize=1000)
for group in df_groups:
print(group)
This will print each group of 1000 rows to the console. You can then process each group of data as needed.