0

I have a list of numbers in ipython through pyspark like this:

df = pd.DataFrame(index=range(N))
df['total'] = data.map(lambda x:(x.features[0]+x.features[1]+x.features[2])).collect()

Now some of the numbers in this list come out to have two digits after decimal like 10.17 but there are numbers which have less than two digits after decimal like 9.1 or some do not have any decimal like 5 or 23. What I want is to have all the numbers two digits after decimal. If they do not have two digits then an ending 0 should be added like for above 9.1 should become 9.10 and 5 should become 5.00. How can I do that?

NOTE: I have seen posts that do that reformatting but my reformatting is in respect to the pyspark and the map() function specifically. How do I do the reformatting there?

Jason Donnald
  • 2,256
  • 9
  • 36
  • 49
  • And they have to be stored as floats? – Tim Aug 18 '15 at 17:54
  • possible duplicate of [Display a float with two decimal places in Python](http://stackoverflow.com/questions/6149006/display-a-float-with-two-decimal-places-in-python) – Tim Aug 18 '15 at 17:55
  • @Tim I saw that post but how do I do that here with data.map() statement. I don't care about them storing in this format. All I want is to display them. – Jason Donnald Aug 18 '15 at 17:57
  • What does it mean _display them_? If you need fixed precision numbers you should use [`decimal`](https://docs.python.org/2/library/decimal.html). If you simply want specific output you can map to strings with specific format. – zero323 Aug 18 '15 at 18:14

1 Answers1

0

The display doesn't happen in the map lambda function. The values are materialized when collect() returns your array of results. You'd then apply formatting to those results.

Jeff L
  • 141
  • 1
  • 1
  • 7