I am a learner in using pandas in python.
I have a millions of driving data signals received from Vehicle in csv format. Example : Vehicle Speed, Brake pressure etc. at different Time intervals
Sample Data : (For reference, I used only 2 Data_Names & sample Values )
Time Data_Name Values
0.5 Vehicle_Speed 10
1 Brake_Pressure 12
2 Vehicle_Speed 30
3 Vehicle_Speed 40
4 Vehicle_Speed 50
5 Brake_Pressure 10
6 Brake_Pressure 15
7 Vehicle_Speed 30
8 Brake_Pressure 15
9 Brake_Pressure 20
10 Vehicle_Speed 25
Requirements :
1st Requirement is : I want the above data to be in the below form.
Time Vehicle_Speed_Values Brake_pressure_Value
0.5 50 0
1 0 12
2 30 0
3 40 0
4 50 0
5 0 10
6 0 15
7 30 0
8 0 15
9 0 20
10 25 0
2nd Requirement : interpolate it there by filling the zero values corresponds to time. 3rd : plotting Vehicle_Speed (Vs) Brake Pressure graph for comparison
My code :
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
# Time Data
b1 = pd.Dataframe([0.5,2,3,4,7,25])
# Vehicle_Speed_Values
b2 = pd.Dataframe([50,30,40,50,30,25])
# Combine two data frames
d1 = pd.concat([b1,b2], axis=1)
# Name the columns
d1.columns = ["Time","Speed"]
# Create a new data frame with column Names as Vehicle_Speed_Values& Brake_pressure_Value filled with zeros
a = pd.Dataframe([0.5,1,2,3,4,5,6,7,8,9,10])
a.columns = ["Time"]
alength = len(a["Time"])
a["Vehicle_Speed_Values"] = pd.Series(np.zeros(alength), index =a.index)
a["Brake_pressure_Value"] = pd.Series(np.zeros(alength), index =a.index)
# using for loop and index functions copy the value from d1 (data frame)to a (dataframe)
for i in range(len(d1.index)):
if d1.Time[i] in a["Time"]:
j = d1.Time[i]
find_index = a.loc[a["Time"] == j].index[0]
a.Vehicle_Speed_Values[find_index] = d1.Speed[i]
Problem facing : On execution Warning message displayed.(SettingWithCopyWarnings) Even though i get the result as expected but for large data set of 10 million values, Warning messages displayed and I cannot able to proceed further.
Requests :
I am not a good coder, any way i am sure that there will be some easy way to do the above process.. Kindly shed some light to avoid this problem or a new way to approach...
