What Is a Loss Function?

A loss function or cost function is a function that maps the value of a random event or its related random variable to a non-negative real number to represent the "risk" or "loss" of the random event. In the application, the loss function is usually associated with the optimization problem as a learning criterion, that is, the model is solved and evaluated by minimizing the loss function. For example, it is used for parameter estimation of models in statistics and machine learning ^[1] , and it is used for risk mangement and decision making in macroeconomics ^[2] , and it is applied in control theory Optimal control theory ^[3] .

in

Loss function case one

The output of a factory staff is calculated at how many yuan per hour, and the loss function shows that the output changes according to the indoor ventilation conditions. Everyone working in the factory has its own loss function. In order to simplify the explanation, it is assumed that the loss function of each person is a parabola, and the bottom point represents the ventilation conditions when the output value is the largest. When the loss functions of all personnel are superimposed, the overall loss function of the company must be a parabola. If ventilation conditions deviate from this optimum level, additional losses will occur. When the parabola is tangent to the horizontal axis, there are small sections on the left and right of the tangent point that almost coincide with the horizontal axis. In other words, there is an optimum point that deviates a short distance, and the loss is so small that it can be ignored. Therefore, when the indoor ventilation conditions slightly deviate from the equilibrium point, the losses that occur are negligible. But away from the equilibrium point, someone always has to pay for the loss. If we can derive a loss function with a specific number, we can calculate the optimal equilibrium point, what is the most suitable ventilation condition in the equilibrium point, and what is the required expenditure.

Loss function case two

Take the catch train as an example of meeting the specifications. Suppose our time value is n yuan per minute, and the sloping line on the left side of the figure below is the slope of the loss line; reaching the platform one minute earlier will cause us to lose n yuan and reach 2n yuan early. On the other hand, if we fail to catch the train, our loss is M dollars. Just like half a minute late or 5 minutes late, the loss function jumps from zero to M directly.

Of course, the problem can also be complicated. For example, the time at which the train leaves the station changes every day, so you can also draw an allocation map. The limit of the three standard deviations of train arrival time may be 8 seconds. To complicate the problem in this way is not particularly helpful for us to understand and apply the loss function, so we will stop here. Another example is the parking problem encountered during Sunday worship at 11.15am. The church's parking lot has a maximum capacity of 50 cars, but these parking spaces are still full at about 10:50 because the owners who had finished the last week are still drinking coffee. As soon as they leave, these vacancies will soon be filled by the long queues waiting for them. If you want to occupy a parking space, you have to go in line early. Those who arrive late can't find a parking space here and can only find it on the street, but they often return in vain. Therefore, the best strategy is to wait a little earlier, to bear the loss of waiting to occupy the position.

This theory can also be applied to any planned deadline. Someone requires that work must be completed before the deadline, and failure to catch up with this time will delay or make a mistake. In order to complete on time, an outline of the work content and steps can be drawn up. Setting the deadline for a step for a period of time is easier than setting it for a fixed period of time, and having time to make final revisions may make the plan better.