Stack Overflow
Stack Overflow

Questions tagged [numerical]

This tag is for questions concerning problems using numbers which either cannot be exactly solved, or where the exact solution may be much more difficult to acquire than by using numerical methods.

This tag is for questions concerning problems using numbers which either cannot be exactly solved, or where the exact solution may be much more difficult to acquire than by using numerical methods.

It includes questions about algorithms, precision and accuracy of underlying numbers, advantages and disadvantages of several methods, error propagation, problems with the underlying numbers and data structures.

508 questions
85
votes
7 answers

What range of numbers can be represented in a 16-, 32- and 64-bit IEEE-754 systems?

I know a little bit about how floating-point numbers are represented, but not enough, I'm afraid. The general question is: For a given precision (for my purposes, the number of accurate decimal places in base 10), what range of numbers can be…
Nate Parsons
  • 14,431
  • 13
  • 51
  • 67
79
votes
4 answers

How do I type a floating point infinity literal in python

How do I type a floating point infinity literal in python? I have heard inf = float('inf') is non portable. Thus, I have had the following recommended: inf = 1e400 Is either of these standard, or portable? What is best practice?
fmark
  • 57,259
  • 27
  • 100
  • 107
54
votes
4 answers

Does Python have a function to reduce fractions?

For example, when I calculate 98/42 I want to get 7/3, not 2.3333333, is there a function for that using Python or Numpy?
LWZ
  • 11,670
  • 22
  • 61
  • 79
49
votes
12 answers

How do I determine the standard deviation (stddev) of a set of values?

I need to know if a number compared to a set of numbers is outside of 1 stddev from the mean, etc..
dead and bloated
  • 493
  • 1
  • 5
  • 5
47
votes
5 answers

SQL ORDER chars numerically

I have a column of numbers stored as chars. When I do a ORDER BY for this column I get the following: 100 131 200 21 30 31000 etc. How can I order these chars numerically? Do I need to convert something or is there already an SQL command or…
T.T.T.
  • 33,367
  • 47
  • 130
  • 168
38
votes
13 answers

Any way faster than pow() to compute an integer power of 10 in C++?

I know power of 2 can be implemented using << operator. What about power of 10? Like 10^5? Is there any way faster than pow(10,5) in C++? It is a pretty straight-forward computation by hand. But seems not easy for computers due to binary…
szli
  • 36,893
  • 11
  • 32
  • 40
35
votes
3 answers

Does JavaScript have double floating point number precision?

I know it's an odd question, but does JavaScript have the capacity to work with double's as opposed to single floats? (64 bit floats vs. 32 bits.)
Travis
  • 7,391
  • 12
  • 43
  • 52
33
votes
7 answers

Convert numbers into corresponding letter using Python

I was wondering if it is possible to convert numbers into their corresponding alphabetical value. So 1 -> a 2 -> b I was planning to make a program which lists all the alphabetical combinations possible for a length specified by a user. See I know…
user3556962
  • 353
  • 2
  • 4
  • 4
31
votes
8 answers

How to write fast (low level) code?

I would like to learn more about low level code optimization, and how to take advantage of the underlying machine architecture. I am looking for good pointers on where to read about this topic. More details: I am interested in optimization in the…
Szabolcs
  • 24,728
  • 9
  • 85
  • 174
30
votes
7 answers

Efficient way to compute geometric mean of many numbers

I need to compute the geometric mean of a large set of numbers, whose values are not a priori limited. The naive way would be double geometric_mean(std::vector const&data) // failure { auto product = 1.0; for(auto x:data) product *= x; …
Walter
  • 44,150
  • 20
  • 113
  • 196
30
votes
3 answers

How to detect significant change / trend in a time series data?

So I have an array of say 25 samples and I would want to be able to note the trends of whether it's decreasing n or increasing from those 25 sample time interval(basically 25 samples array is my buffer that is being filled by every say 1 ms). Note…
as3rdaccount
  • 3,711
  • 12
  • 42
  • 62
28
votes
4 answers

Plotting a line over several graphs

I don't know how this thing is called, or even how to describe it, so the title may be a little bit misleading. The first attached graph was created with pyplot. I would like to draw a straight line that goes through all graphs instead of the…
Artium
  • 5,147
  • 8
  • 39
  • 60
27
votes
4 answers

What is the correct way to raise an integer to a positive integer power in C++?

We know that for various reasons, there is no standard integer power function in C++. I'm performing exact arithmetic with rather small integers, what is the correct way to compute powers?
static_rtti
  • 53,760
  • 47
  • 136
  • 192
24
votes
5 answers

Passing around fixed-size arrays in C++?

Basically I'd like to do something like that: int[3] array_func() { return {1,1,1}; } int main(int argc,char * argv[]) { int[3] point=array_func(); } But that doesn't seem legal in C++. I know I can use vectors, but since I know the size…
static_rtti
  • 53,760
  • 47
  • 136
  • 192
24
votes
7 answers

When to use Fixed Point these days

For intense number-crunching i'm considering using fixed point instead of floating point. Of course it'll matter how many bytes the fixed point type is in size, on what CPU it'll be running on, if i can use (for Intel) the MMX or SSE or whatever…
DarenW
  • 16,549
  • 7
  • 63
  • 102
1
2 3
…
33 34