0

I have produced a Boxplot/Swarmplot graph using Matplotlib/Seaborn in Pandas. Some outliers can been seen in the graph (as dots outside the "whiskers"/"fence" area). I am looking for a way to trim the dataset directly after they have been identified in the graph and without removing them from the original dataset. I do not want to simply hide the outlier dots.

Some methods have been recommended and pandas quantile looks promising but I am not sure how to implement these with the code I have been using.

My graph with the outliers.

enter image description here

The code I used to produce this graph. The data has been organized into the tidy format.

# Import libraries and modules

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns

# Set seaborn style

sns.set(style="whitegrid", palette="colorblind")

# load length tidy data

length_tidy = pd.read_csv('results/tidy/length_tidy.csv')
score_tidy = pd.read_csv('results/tidy/score_tidy.csv')

# Define and save boxplot and swarmplot for length data

fig, ax = plt.subplots(figsize=(10,6))
ax = sns.boxplot(x='Metric', y='Length', data=length_tidy, ax=ax)
ax = sns.swarmplot(x="Metric", y="Length", data=length_tidy, color=".25")
ax.set_xlabel('Condition')
ax.set_ylabel('Length in micrometers')

plt.savefig('statistics/boxplot/length_boxplot.png', dpi=300)

fig, ax = plt.subplots(figsize=(10,6))
ax = sns.boxplot(x='Metric', y='Score', data=score_tidy, ax=ax)
ax = sns.swarmplot(x="Metric", y="Score", data=score_tidy, color=".25")
ax.set_xlabel('Condition')
ax.set_ylabel('Score')

plt.savefig('statistics/boxplot/score_boxplot.png', dpi=300)

An example of some of the data I am working with in the CSV format.

Object,Metric,Length
M11,B2A10,1.807782
MT1,B2A10,3.2207116666666664
MT1,B2A1,3.57675
MT1,B2A2,2.9474600000000004
MT1,B2A3,2.247772857142857
MT1,B2A4,3.754455
MT1,B2A5,2.716282
MT1,B2A6,2.91325
MT1,B2A7,1.24806
MT1,B2A8,2.00371875
MT1,B2A9,1.5435599999999998
MT1,B2B1,2.2051515384615388
MT1,B2B2,1.5278873333333332
MT1,B2B3,1.7283750000000002
MT1,B2B4,1.4547385714285714
MT1,B2B5,3.237578333333333
MT1,B2B6,2.47016
MT1,B2B7,2.1185947777777776
MT1,B2B8,1.8502877777777773
MT10,B2A10,3.07143
MT10,B2A1,3.34361
MT10,B2A2,2.889958333333333
MT10,B2A3,2.22087
MT10,B2A4,2.87669
MT10,B2A5,1.6745005555555557
MT10,B2A7,2.09018
MT10,B2A8,2.4947450000000004
MT10,B2B1,1.849095882352941
MT10,B2B2,1.5291758000000002
MT10,B2B5,1.6423770999999998
MT10,B2B6,1.9680385714285715
MT10,B2B7,1.7207240000000001
MT10,B2B8,2.9618275
MT12,B2A10,1.7243058333333334
MT12,B2A1,3.3938900000000003
MT12,B2A2,2.00601
MT12,B2A3,2.1720200000000003
MT12,B2A4,2.452923333333333
MT12,B2A5,2.986948
MT12,B2A7,2.08466
MT12,B2A8,1.29047
MT12,B2B1,2.528839230769232
MT12,B2B2,1.4011425454545454
MT12,B2B5,1.626078333333333
MT12,B2B6,1.074394454545455
MT12,B2B7,2.0897078571428573
MT12,B2B8,1.4102533333333336
Christer Edvardsson
  • 137
  • 1
  • 2
  • 10
  • 1
    `matplotlib.cbook.boxplot_stats` is how matplotlib (and maybe seaborn) computes the fences. So you could call that directly and filter your data based on the information it gives you. – Paul H May 15 '20 at 15:00
  • matplotlib.cbook has a section on this (https://matplotlib.org/3.1.1/api/cbook_api.html) but I am unsure how to implement this in a solution to my problem. If you could provide further explanations I would be very grateful! – Christer Edvardsson May 18 '20 at 14:50
  • https://stackoverflow.com/help/minimal-reproducible-example – Paul H May 18 '20 at 14:53
  • I have provided an example in the question. Is there anything you think is missing that would help to find a solution? – Christer Edvardsson May 22 '20 at 07:41
  • Files not found on my machine, so it's not reproducible – Paul H May 22 '20 at 14:49
  • from the pandas wiki on SO: https://stackoverflow.com/questions/20109391/how-to-make-good-reproducible-pandas-examples – Paul H May 22 '20 at 14:50
  • That is a good guide! Thx. :-) I have now added an example of my data in the question. – Christer Edvardsson May 26 '20 at 10:56

0 Answers0