Working with Pandas Dataframes - manipulate data

Question

i was trying to find out the solution by myself but unfortunately couldn't succeed.

-> i have a scientific dataset with 12 columns representing 4 different monitoring stations and each station measures 3 different kind of pollutants. the data contains around 70000 chronological timestamps (rows). since 2 out of the 3 pollutants are given in a different unconvenient unit i have to multiply those columns by a certain value (i.e. have to manipulate 8 columns)

how can i do that so i get the dataframe arranged in the same way but with the manipulated columns integrated?

thank you in advance for any help provided!

*since 2 out of the 3 pollutants are given in a different not convenient unit i have to multiply those columns by a certain value (i.e. have to manipulate 8 columns)* What about just doing what you describe? Just assign to the columns the columns multiplied by a certain value. What is the problem with it? — Claudio, Oct 07 '22 at 22:34
A small example of what your data looks like and what you wanted it to look like would be great. — Marcelo Ruiz, Oct 07 '22 at 22:37
Your question needs a minimal reproducible example consisting of sample input, expected output, actual output, and only the relevant code necessary to reproduce the problem. See [How to make good reproducible pandas examples](https://stackoverflow.com/questions/20109391/how-to-make-good-reproducible-pandas-examples) for best practices related to Pandas questions. — itprorh66, Oct 07 '22 at 23:04
Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking. — Community, Oct 08 '22 at 09:02

score 0 · Answer 1 · answered Oct 08 '22 at 23:08

Let's start with a simplified version of your problem.

Imagine I have a dataframe of temperatures for various cities over time. However, for some of those cities, I have the temperatures in Celsius, and I need to convert those columns to Fahrenheit. Other cities are already in Fahrenheit.

Here's the example dataset:

And here it is in code form:

import pandas as pd

df = pd.DataFrame({'Denver': {0: 21.5, 1: 17.1, 2: 24.6, 3: 20.7, 4: 17.6, 5: 25.0, 6: 21.2, 7: 23.0, 8: 18.6, 9: 20.1}, 'Colorado Springs': {0: 70.2, 1: 79.7, 2: 74.8, 3: 64.6, 4: 68.9, 5: 65.3, 6: 61.5, 7: 69.5, 8: 63.7, 9: 62.4}, 'Aurora': {0: 18.9, 1: 20.7, 2: 21.8, 3: 21.8, 4: 24.6, 5: 17.3, 6: 17.1, 7: 20.7, 8: 16.3, 9: 22.4}, 'Fort Collins': {0: 63.1, 1: 63.3, 2: 69.0, 3: 75.6, 4: 73.4, 5: 78.9, 6: 71.1, 7: 70.4, 8: 71.0, 9: 75.1}, 'Lakewood': {0: 68.9, 1: 64.4, 2: 72.7, 3: 61.1, 4: 66.8, 5: 77.6, 6: 68.9, 7: 77.5, 8: 71.3, 9: 76.0}, 'Thornton': {0: 80.0, 1: 77.9, 2: 75.4, 3: 62.8, 4: 77.3, 5: 69.8, 6: 76.3, 7: 78.0, 8: 69.1, 9: 63.3}, 'Arvada': {0: 23.9, 1: 21.5, 2: 22.6, 3: 15.8, 4: 22.3, 5: 16.2, 6: 25.6, 7: 25.2, 8: 25.7, 9: 25.3}, 'Westminster': {0: 69.2, 1: 68.5, 2: 66.4, 3: 68.6, 4: 80.0, 5: 68.4, 6: 64.6, 7: 66.2, 8: 73.0, 9: 76.6}, 'Pueblo': {0: 62.4, 1: 76.3, 2: 62.3, 3: 75.0, 4: 80.0, 5: 62.9, 6: 67.5, 7: 64.3, 8: 78.7, 9: 71.5}, 'Greeley': {0: 79.0, 1: 64.4, 2: 65.5, 3: 68.7, 4: 73.6, 5: 73.6, 6: 60.6, 7: 74.0, 8: 75.6, 9: 77.6}})

In this example, I want to convert the values for Denver, Arvada, and Aurora from Celsius into Fahrenheit, then save that back into the same dataframe. For every temperature value in those columns, I need to multiply by 1.8, and add 32.

Fortunately, Pandas makes it easy to manipulate an entire column. Converting Denver to Fahrenheit looks like this:

df["Denver"] = df["Denver"] * 1.8 + 32

In Pandas, multiplying a column by a scalar (i.e. single value) means that you multiply each element by that scalar.

Then, I can do the same thing for Aurora and Arvada. Normally, I would use a loop, to avoid writing the same piece of code three times.

celsius_cities = ["Denver", "Arvada", "Aurora"]
for col in celsius_cities:
    df[col] = df[col] * 1.8 + 32

However, since you only have eight columns, and since you're multiplying by a different value each time, a loop might not be helpful for you.

thank you very much @Nick ODell for the only kind and helpful answer! this way i could solve my problem (: — desperate_house_coder, Oct 10 '22 at 21:55
@desperate_house_coder Sure thing! Always happy to help new users, especially people doing scientific work. If my answer helped you, please mark it as "accepted." It's the green checkmark. — Nick ODell, Oct 11 '22 at 18:54

Working with Pandas Dataframes - manipulate data

1 Answers1