4

I am utilizing an Azure Synapse Analytics Workspace where I am running a PYSpark notebook from the Notebook tab of the workspace.

The workspace is also connected to an Azure Data Lake Storage Gen 2, which has a container called filesystem.

Usually to save files I write them in Spark using the ABFSS path after exporting my DataFrame.

UC_export=spark.createDataFrame(Error_Report.astype(str), verifySchema=False)
UC_export.coalesce(1).write.format('csv').mode('overwrite').save('abfss://[Container Name].dfs.core.windows.net/Assets_DQ/Error_Report.csv', header=True)

However for the following code, I have no idea how to save figures. How do I run a loop which can export the *.png files to the ABFSS container?

for error in Unique_Conso_df_check['Rule Broken'].unique():
    for column in suitable_columns:
        fig = plt.figure()
        ax = plt.subplot(111)
        pd.crosstab(Unique_Conso_df_Error[column],Unique_Conso_df_Error[error]).plot(kind='bar', rot = 90, ax=ax, figsize=[20,10])
        ax.get_figure().savefig(savedirectory+'/densityplot_'+str(column)+'.png')
        plt.show()
Darren Teo
  • 41
  • 1

0 Answers0