-1

I am trying to load Data frame into file but not able to get exact match. Can you please help me on this?

example:

"From...............\"dawood\"...........\"oral use\"........"

but i am getting:

"From................\"dawood\"...........\\"oral use\\\"......"

i am using below code to write the dataframe:
df.repartition(1).write.format('com.databricks.spark.csv').mode('overwrite').save(output_path,quote='"', sep='|',header='True',nullValue=None)

Can you please help me how to get exact match for all the reords.

shivam patel
  • 67
  • 1
  • 2
  • 9
  • Your question is quite confusing, could you please clarify what you mean, and don't mark all of that as code, because it isn't – incarnadine Apr 01 '20 at 11:52
  • @mvr950, that question doesn't ask (and its answer doesn't show) how to pass arguments through, which is the part the OP doesn't know how to do; the existing linked duplicate is better. – Charles Duffy Apr 01 '20 at 12:00
  • BTW, it's better practice not to use file extensions for executable scripts. Just leave the extension off, and use a shebang to select the correct interpreter. See the essay [Commandname Extensions Considered Harmful](https://www.talisman.org/~erlkonig/documents/commandname-extensions-considered-harmful/), or if you want a more authoritative source, see the factoid database history entry for the irc.freenode.org #bash support channel at http://wooledge.org/~greybot/meta/.sh. – Charles Duffy Apr 01 '20 at 12:06
  • First Python file test.py import sys data = sys.argv[1] print data then bash file pss.sh #!/bin/bash python test.py "$@" –  Apr 01 '20 at 12:36

1 Answers1

1

either just copy this inside your shell script:

python imed_consump.py 'Smart Source'

but then your parameter is always fixed. should this not be desired, then do following inside shell

python imed_consump.py "$1"

and execute your shell like:

bash imed_consump.sh 'Smart Source'
Charles Duffy
  • 280,126
  • 43
  • 390
  • 441
Muhammad Moiz Ahmed
  • 1,799
  • 1
  • 11
  • 9
  • Needs to be `"$1"`, not bare `$1`. Otherwise `$1` (or, in Python, `sys.argv[1]`) will be `Smart` and `$2` / `sys.argv[2]` will be `Source` inside the inner script, instead of keeping that argument together as a single word. – Charles Duffy Apr 01 '20 at 11:56
  • That said, do note the "Answer Well-Asked Questions" section of [How to Answer](https://stackoverflow.com/help/how-to-answer); questions that have been "asked and answered many times before" should be closed, not answered. – Charles Duffy Apr 01 '20 at 11:58
  • thanks @CharlesDuffy, yes I am myslef new here in answering questions. – Muhammad Moiz Ahmed Apr 01 '20 at 12:32