Click here to Skip to main content
15,867,453 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
See more:
I have a column with mixed values as below:
Mixed Data - column name
100 socks
3467867327 - remove
94746723892 - remove
400 order
oils
essentials
b01232435 - remove
b0054543543543 - remove
garments
787 sales

I want to filter the column and remove the rows which are marked as remove above. I can achieve removing the string starting from "b0" but I am unable to apply filter to only numeric values. dtype is object of the columns. Below code does not work for me nor do I want to create a separate column to convert string to numeric.

pd.to_numeric(df['Mixed Data'],errors='coerce').min()

Note - the actual data is of huge count hence I need to a generic code and not specific code. Please help me.


What I have tried:

import pandas as pd
df = pd.DataFrame.from_dict(
    {
        "Mixed Data": [
            "100 socks",
            "3467867327",
            "94746723892",
            "400 order",
            "oils",
            "essentials",
            "b01232435",
            "b0054543543543",
            "garments",
            "787 sales",
        ],
        "Some other column": [
            "a",
            "b",
            "c",
            "d",
            "e",
            "f",
            "g",
            "h",
            "i",
            "j",
        ],
    }
)
df = df[df["Mixed Data"].str.startswith("b0") == False][
    df["Mixed Data"].str.isnumeric() == False
]
print(df)


the above ends up deleting all the records from the table. hence it does not work for me
Posted
Comments
Richard MacCutchan 8-Aug-22 8:31am    
I just ran your code and it does not delete all the records. Here is the output:
python cp.py
cp.py:30: UserWarning: Boolean Series key will be reindexed to match DataFrame index.
  df = df[df["Mixed Data"].str.startswith("b0") == False][
   Mixed Data Some other column
0   100 socks                 a
3   400 order                 d
4        oils                 e
5  essentials                 f
8    garments                 i
9   787 sales                 j

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900