0

i'm struggling with this warning. i understand the difference between a copy and a view, but still can't found the solution. I have this function

 def __setFinishedJobInDB(self, job: Job, guid_list):
    try:
        if sys.argv[1] in ['qa', 'prod']:

            guid_list['status'],  guid_list['job_end_date'] = 3, datetime.datetime.now()
            data_to_firehose = self.transform_dataframe_to_firehose_data_type(guid_list)
            Kinesis().put_record_batch(Kinesis().get_ade_job_runs_firehose(), data_to_firehose)
    except Exception as ex:
        log.error(f'Error updating job in Kinesis firehose on __setFinishedJobInDB: error{ex}')

guid_list is a dataframe i'm getting two warning for each guid_list['status']=3 and guid_list['job_end_date']=datetime.datetime.now().

I do need to change the values of those columns, how can i do it without getting this warning.

David Belhamou
  • 1
  • 1
  • At some point _before_ this provided code you have unsafely subset your DataFrame. Either `new_df = df[cols]` or `new_df = df[mask]` when it should have been `new_df = df[cols].copy()` or `new_df = df[mask].copy()` The warning is letting you know that `df[mask]['col'] = value` may not work because `df[mask]` may produce a copy and recommends that you use `df.loc[mask, 'col'] = value` but that message is not clear here since you're doing something like `new_df = df[mask]` then later doing `new_df[col] = value` which looks to pandas like a (deferred) `df[mask][col] = value` call. – Henry Ecker Mar 10 '22 at 16:12
  • With a function this could also look something like: `__setFinishedJobInDB(job, df[mask])` or `__setFinishedJobInDB(job, df[cols])`. But somewhere you've made a new dataframe by selecting some values from a different DataFrame. – Henry Ecker Mar 10 '22 at 16:13
  • Hi henry, here what i send to the function status_3_df = running_guid[running_guid['run_guid'].isin(finished_permutations_guid)] self.status.jobFinished(request.job, status_3_df) i try to send status_3_df = running_guid_df_status.loc[running_guid_df_status['run_guid'].isin(finished_permutations_guid), :] but i'm getting the same warning – David Belhamou Mar 13 '22 at 12:22
  • Again as all of the accompanying links and my comments mention. That should be `status_3_df = running_guid[running_guid['run_guid'].isin(finished_permutations_guid)] .copy()` – Henry Ecker Mar 13 '22 at 17:08
  • You could also put together a [mre] and [edit] your question to include code that reproduces the issue so we can see exactly where the issue is originating. – Henry Ecker Mar 13 '22 at 17:09

0 Answers0