Since you understand everything is a one-argument function, let's start from that point. Keep in mind that (\x y z -> x) is really (\x -> (\y z -> x)), which in turn really is (\x -> (\y -> (\z -> x))), but let's stop at the first step to keep the noise from brackets down.
(f . g) x = f (g x)
hence
((\x -> (\y z -> x)) . (\a b -> a*b)) 2 =
(\x -> (\y z -> x)) ((\a -> (\b -> a*b)) 2) =
(\x -> (\y z -> x)) (\b -> 2*b) =
(\y z -> (\b -> 2*b))
Now remember the second step and expand (\y z -> ...):
(\y z -> (\b -> 2*b)) 3 4 =
(\y -> (\z -> (\b -> 2*b))) 3 4 =
-- \y -> ... given anything, returns a function \z -> ...
(\z -> (\b -> 2*b)) 4 =
-- \z -> ... given anything, returns a function \b -> ...
(\b -> 2*b)
which finally is:
(\b -> 2*b) 5 = 2*5 = 10
The story unfolds differently, if the first function returns y instead of x:
((\x -> (\y z -> y)) . (\a -> (\b -> a*b))) 2 =
(\x -> (\y z -> y)) ((\a -> (\b -> a*b)) 2) =
(\x -> (\y z -> y)) (\b -> 2*b) =
-- \x -> ... given anything, returns a function \y z -> ...
(\y z -> y)
so you get:
(\y -> (\z -> y)) 3 4 5 =
-- \y -> ... given anything, returns a function \z -> ...
(\z -> 3) 4 5 =
-- \z -> ... given anything, returns a constant 3
3 5 -- now still trying to apply 5 to 3
It is attempting to treat 3
as a function that can take 5
.