As Tikhon Jelvis explained in his response, functions map sets to sets, and functions themselves form sets. This is the essence of the untyped lambda calculus. Unfortunately, untyped lambda calculus suffers from the Kleene–Rosser paradox (later simplified to Curry’s paradox).
This paradox can be removed by introducing types, as in the typed lambda calculus. Simple types are equivalent to sets, but in order to pass a function as an argument to another function (or return one), we have to give this function a type. To really understand what a function type is, you need to look into category theory.
The categorical model for the typed lambda calculus is a category in which objects are types and morphism are functions. So if you want to have higher order functions, you have to be able to represent morphisms as objects — in other words, create a type for functions. This is possible only if the category is cartesian closed. In such a category you can define product types and exponential types. The latter correspond to function types.
So that’s a mathematical explanation for higher order.
Math Online Tom Circle 