Stack Overflow
Stack Overflow

Questions tagged [swifter]

43 questions
4
votes
0 answers

Failed to install swifter via pip - INFO: pip is looking at multiple versions .. compatible with other requirements

Within my virtual python 3.9x - environment on my Lubuntu 20.04 - System, I tried to install swifter via command line using pip install swifter. This doesn't work out however, as the compatibility checks with several modules, such as jupyter-core,…
Andreas L.
  • 3,239
  • 5
  • 26
  • 65
4
votes
1 answer

Why is swifter slower than vanilla df.apply?

I have a dataframe with 1 million rows. I have a single function (which I can't vectorize) to apply to each row. I looked into swifter which promises to leverage multiple process to speed up computations. On an 8-cores machine it's however not the…
ted
  • 13,596
  • 9
  • 65
  • 107
3
votes
0 answers

AttributeError: 'DataFrame' object has no attribute 'swifter'

I am trying to optimize apply() function that I using for calling a function which uses pandas dataframe as an argument. I am trying to call this using df_final = df.swifter.apply(function, axis=1).
Manali Haresh Damania
  • 31
  • 1
  • 3
3
votes
1 answer

Swifter: what is the difference vectorized and non-vectorized function?

I need to learn about pandas speed optimization. Library that very effective about my problem is swifter. But i don't understand about the documentation, especially vectorized function. My assumption is swifter input is only accept vector input, not…
OctavianWR
  • 217
  • 1
  • 16
2
votes
0 answers

Reasons why swifter/dask/ray only use one core for an apply task?

I have this function that I would like to apply to a large dataframe in parallel: from rdkit import Chem from rdkit.Chem.MolStandardize import rdMolStandardize from rdkit import RDLogger RDLogger.DisableLog('rdApp.*') def…
Soerendip
  • 7,684
  • 15
  • 61
  • 128
2
votes
1 answer

How can I fix the error AttributeError: 'Series' object has no attribute 'swifter'?

This is pretty direct, I was trying to replicate the example used in Swifter page https://github.com/jmcarpenter2/swifter. However, I keep getting the error AttributeError: 'Series' object has no attribute 'swifter'. What's up with that? Update -…
Dumb ML
  • 357
  • 2
  • 12
2
votes
0 answers

swifter apply gets stuck in timing phase

I am trying to use swifter.apply to speed up a standard pandas lambda function, which is otherwise working fine, but addition of swifter causes the function to stall. Delving into the code a bit, it seems to get stuck at this line: timed =…
Peter
  • 107
  • 5
2
votes
2 answers

Flatten pandas dataframe column containing list of dictionaries

I am flattening a data frame in which the column contains a list of dictionaries. I have written the code for it. However, it takes around 25 seconds to process only 5000 rows which is a lot. Here is the sample dataset: event_date timestamp …
user1584253
  • 975
  • 2
  • 18
  • 55
2
votes
0 answers

Swifter + Pandas: script does not move on after using swifter + apply()

I have a script that loads data from MySQL. After that a Pandas DataFrame is built. On it I do apply several apply methods. When I run the script, without swifter, this is the ouput: Duration Job0_sql: 0.011724 Duration Job0_df: 146.875376
Lucas Aimaretto
  • 1,399
  • 1
  • 22
  • 34
1
vote
0 answers

Swifter error "AppRegistryNotReady: Apps aren't loaded yet." with Django models

I'm trying to populate a database with django models and I've used the Swifter structure to parallelize a pandas apply, like this: df.swifter.force_parallel(enable=True).apply(lambda row: migrate(row), axis = 1) When apply function loaded 1000…
João Felipe Holanda
  • 103
  • 1
  • 1
  • 8
1
vote
1 answer

Using Swifter to pull a twitter list will only pull tweets with media links and not ones with full text based on my case let statement in SwiftUI

I am currently building an app to display tweets from a twitter list. For some reason I am only appending tweets with media links, and skipping over tweets with just full text and no media. The JSON data prints all tweets that I request. Does anyone…
eazey12
  • 11
  • 1
1
vote
0 answers

Python Pandas: Partition size is less than overlapping window size

I am trying to compute the slope of a time series using Pandas with Switfer by doing this. My code: import os.path from os import listdir from os.path import isfile, join import numpy as np import pandas_ta from scipy.stats import…
Mircea
  • 1,671
  • 7
  • 25
  • 41
1
vote
1 answer

Python Script Stops Silently

I'm running Python scripts as child processes, spawned using Nodejs. When running locally, or locally using Docker / Kubernetes installation, it works as expected and completes all functions in the script. When running the container in Kubernetes…
chS
  • 80
  • 1
  • 9
1
vote
3 answers

Improving pandas apply function performance

I have a pandas dataframe whose column contains dictionaries. I also have a query dictionary and I want to compute minimum sum of the values of the common keys. For example dicta = {'a': 5, 'b': 21, 'c': 34, 'd': 56, 'r': 67} dictb = {'a': 1, 'b':…
satinder singh
  • 190
  • 12
1
vote
1 answer

With swifter json response

I am create simple rest api with swift using swifter library How i can response json data? import Swifter let server = HttpServer() server["/hello"] = { request in var userList:[Any] = [] for user in users { var b : [String: Any] =…
Wang Liang
  • 4,244
  • 6
  • 22
  • 45
1
2 3