I wanted to make a function that returns the derivative of a function at a point for part of my app. Obviously, this is the formal definition of a limit.
But what kind of a function would be able return the derivative of a function at a point in swift? Any ideas for automatic differentiation in swift?
Here is a simple numerical approach based upon your formula above. You could improve upon this:
derivativeOf takes a function fn and an x-coordinate x and returns a numerical approximation of derivative of fn at x:
func derivativeOf(fn: (Double)->Double, atX x: Double) -> Double {
let h = 0.0000001
return (fn(x + h) - fn(x))/h
}
func x_squared(x: Double) -> Double {
return x * x
}
// Ideal answer: derivative of x^2 is 2x, so at point 3 the answer is 6
let d1 = derivativeOf(fn: x_squared, atX: 3) // d1 = 6.000000087880153
// Ideal answer: derivative of sin is cos, so at point pi/2 the answer is 0
let d2 = derivativeOf(fn: sin, atX: .pi/2) // d2 = -4.9960036108132044e-08
If you are planning on getting the function from the user, that is the tougher part. You could give them some templates to choose from:
y = Ax^3 + Bx^2 + Cx + Dy = A * sin(B*x + C)y = A * cos(B*x + C)y = x ^ (1/N)etc. And then you could have them give you A, B, C, D, or N
Let's look at how that would work for a 3rd order polynomial:
// Take coefficients A, B, C, and D and return a function which
// computes f(x) = Ax^3 + Bx^2 + Cx + D
func makeThirdOrderPolynomial(A a: Double, B b: Double, C c: Double, D d: Double) -> ((Double) -> Double) {
return { x in ((a * x + b) * x + c) * x + d }
}
// Get the coefficients from the user
let a = 5.0
let b = 3.0
let c = 1.0
let d = 23.0
// Use the cofficents to make the function
let f4 = makeThirdOrderPolynomial(A: a, B: b, C: c, D: d)
// Compute the derivative of f(x) = 5x^3 + 3x^2 + x + 23 at x = 5
// Ideal answer: derivative is f'(x) = 15x^2 + 6x + 1, f'(5) = 406
let d4 = derivativeOf(fn: f4, atX: 5) // d4 = 406.0000094341376
The numerical approach is probably the best for you but if you are interested in the analytical approach, it's very simple for derivatives:
Let's declare what a function is (we suppose we have functions with one parameter):
protocol Function {
func evaluate(value: Double) -> Double
func derivative() -> Function
}
Now let's declare basic functions:
struct Constant : Function {
let constant: Double
func evaluate(value: Double) -> Double {
return constant
}
func derivative() -> Function {
return Constant(constant: 0)
}
}
struct Parameter : Function {
func evaluate(value: Double) -> Double {
return value
}
func derivative() -> Function {
return Constant(constant: 1)
}
}
struct Negate : Function {
let operand: Function
func evaluate(value: Double) -> Double {
return -operand.evaluate(value)
}
func derivative() -> Function {
return Negate(operand: operand.derivative())
}
}
struct Add : Function {
let operand1: Function
let operand2: Function
func evaluate(value: Double) -> Double {
return operand1.evaluate(value) + operand2.evaluate(value)
}
func derivative() -> Function {
return Add(operand1: operand1.derivative(), operand2: operand2.derivative())
}
}
struct Multiply : Function {
let operand1: Function
let operand2: Function
func evaluate(value: Double) -> Double {
return operand1.evaluate(value) * operand2.evaluate(value)
}
func derivative() -> Function {
// f'(x) * g(x) + f(x) * g'(x)
return Add(
operand1: Multiply(operand1: operand1.derivative(), operand2: operand2),
operand2: Multiply(operand1: operand1, operand2: operand2.derivative())
)
}
}
struct Divide : Function {
let operand1: Function
let operand2: Function
func evaluate(value: Double) -> Double {
return operand1.evaluate(value) / operand2.evaluate(value)
}
func derivative() -> Function {
// (f'(x) * g(x) - f(x) * g'(x)) / (g(x)) ^ 2
return Divide(
operand1: Add(
operand1: Multiply(operand1: operand1.derivative(), operand2: operand2),
operand2: Negate(operand: Multiply(operand1: operand1, operand2: operand2.derivative()))
),
operand2: Power(operand1: operand2, operand2: Constant(constant: 2))
)
}
}
struct Exponential : Function {
let operand: Function
func evaluate(value: Double) -> Double {
return exp(operand.evaluate(value))
}
func derivative() -> Function {
return Multiply(
operand1: Exponential(operand: operand),
operand2: operand.derivative()
)
}
}
struct NaturalLogarithm : Function {
let operand: Function
func evaluate(value: Double) -> Double {
return log(operand.evaluate(value))
}
func derivative() -> Function {
return Multiply(
operand1: Divide(operand1: Constant(constant: 1), operand2: operand),
operand2: operand.derivative()
)
}
}
struct Power : Function {
let operand1: Function
let operand2: Function
func evaluate(value: Double) -> Double {
return pow(operand1.evaluate(value), operand2.evaluate(value))
}
func derivative() -> Function {
// x ^ y = e ^ ln (x ^ y) = e ^ (y * ln x)
let powerFn = Exponential(
operand: Multiply (
operand1: operand2,
operand2: NaturalLogarithm(operand: operand1)
)
)
return powerFn.derivative()
}
}
struct Sin: Function {
let operand: Function
func evaluate(value: Double) -> Double {
return sin(operand.evaluate(value))
}
func derivative() -> Function {
// cos(f(x)) * f'(x)
return Multiply(operand1: Cos(operand: operand), operand2: operand.derivative())
}
}
struct Cos: Function {
let operand: Function
func evaluate(value: Double) -> Double {
return cos(operand.evaluate(value))
}
func derivative() -> Function {
// - sin(f(x)) * f'(x)
return Multiply(operand1: Negate(operand: Sin(operand: operand)), operand2: operand.derivative())
}
}
The declaration of a function is not very nice:
let xSquared = Power(operand1: Parameter(), operand2: Constant(constant: 2))
but we can evaluate with recursion:
print(xSquared.evaluate(15)) // f(15) = 225
print(xSquared.derivative().evaluate(15)) // f'(15) = 2 * 15 = 30
print(xSquared.derivative().derivative().evaluate(15)) // f''(15) = 2
print(xSquared.derivative().derivative().derivative().evaluate(15)) // f'''(15) = 0
I agree with Collin in that this is a huge topic, and that there is probably no perfect solution. However, for those that are ok with an effective but imperfect solution, vacawama's answer is pleasantly simple. If you want to use your derivative function with some more math-y syntax, you could define an operator, fortunately Swift makes this exceptionally easy. Here is what I did:
I first defined an operator for just returning the function version of the derivative. I personally like the ➚ character for derivatives, but a very large portion of the existing unicode characters are valid Swift identifiers.
postfix operator ➚ {}
postfix func ➚(f: Double -> Double) -> (Double -> Double) {
let h = 0.00000000001
func de(input: Double) -> Double {
return (f(input + h) - f(input)) / h
}
return de
}
Next, lets define a function that we want to differentiate:
func f(x: Double) -> Double {
return x*x + 2*x + 3
}
This can be used as such: f➚, which will return a an anonymous function which will be the derivative of f. If you wish to get f at a specific point (let x = 2, for example), you can call it like this: (f➚)(2)
I decided that I liked the operators, so I made another one to make that syntax look a little better:
infix operator ➚ { associativity left precedence 140 }
func ➚(left: Double -> Double, right: Double) -> Double {
return (f➚)(right)
}
The expression f➚2 will now return the same thing as (f➚)(2) it just is more pleasant to use when you are doing math.
Good question, good answers everyone, I just thought I would tack something extra on. Let me know if you have any questions!
Don't try and reinvent the wheel. Numerical differentiation (numerical analysis, in general) is a huge topic (with many possible solutions *there is no perfect one) and people much smarter than both you and I have come up with solutions already. Unless you're really interested in all the different numerical differential algorithms (their trade offs, implementations and optimizations) I would suggest going another route. You say you're using Swift? Why not switch to Objective-C (I assume you're writing an iOS or OSX app). If you did, you could call out to the GNU Scientific Library (which is a C,C++ library). Maybe you can call c/c++ code directly from Swift? IDK for sure.
If you really wanted, you could look at their code and see how they've implemented their solutions for numerical differentiation (though, I wouldn't do it unless you're ready to tackle some heavy math).
implementing the derivative in C/C++ * you could try and work with this (I doubt this is very robust though). If you need precision and speed, I doubt you'd want to do this is in Swift as well.
This function takes a function as argument and return its derivative function. h is the small shift, order is with respect to the differentiation. It uses recursion for high order differentiation so it may not be very stable.
func differentiate(f:(Double)->(Double),_ h:Double=1e-4,_ order:Int=1)->(Double)->(Double){
var k=order
func diff(x: Double)-> Double{
return (-f(x+2*h)+8*f(x+h)-8*f(x-h)+f(x-2*h))/12/h
}
if(k==0){
return f
}
if(k<0){
fatalError("Order must be non-negative")
}
k=k-1
if(k>=1){
return differentiate(diff,h*10,k)
}
else{
return diff
}
}
print(differentiate({(x:Double)->(Double) in sin(x)},0.0001,4)(0))
