Basis Function
This is an attempt to structure my own learning and ideas on various topics, and are not intended as comprehensive educational material.
A simple regression is given by the following equation:
This function is a linear with parameters w and x. This adds significant limitations
to the abilities of a regression models. To overcome this limitation we can wrap the input variable x
to some non-linear fuction i.e linear combination of non-linear functions of the input variable.
This allows to model complex realtionships between the input and output.
Some examples of basis functions are as follow:
Here, w = (w0,w1,⋯,wM − 1)T and ϕ = (ϕ0,ϕ1,⋯,ϕM − 1)T
y(x, w) = w0 + w1 * x1 + w2 * x2 + ... + wD * xD
Some examples of basis functions are as follow:
Polynomial Basis Function
$$y(x,w) = \sum_{i=1}^{M-1} w_j\phi_j(x)$$
ϕj(x) is the basis functions.Here, w = (w0,w1,⋯,wM − 1)T and ϕ = (ϕ0,ϕ1,⋯,ϕM − 1)T
Gaussian Basis Function
$$\phi_j(x) =
exp\{\frac{-(x-\mu_j)^2}{2s^2}\}$$
Here, μj represents the location of
the basis function. s is their spatial scale.
Sigmoidal Basis Function
$$\phi_j =
\sigma(\frac{x-\mu_j}{s})$$
σ is the sigmoid function given by:
$$\sigma(a) =
\frac{1}{1+exp(-a)}$$