Finds the mean difference between a hypothesis and the provided output values. Cost will be zero if the hypothesis perfectly aligns with the data, and will increase with difference.
The generalized equation for cost function is:
$$ J(\theta) = \frac{1}{m}\sum_{i=1}^m \text{Cost}(h_\theta(x),y) $$
There are various cost functions that can be used depending on the problem.