Published on

Loss Functions

Loss Function

Quantify the error between a model's predictions and the ground truth, guiding the optimization process during training to minimize this error.

Measuring Model Performance

The choice of performance metric depends on the task:

  • Classification:

    • Accuracy: The proportion of correct predictions.
    • Precision: The ratio of true positive predictions to the total predicted positives.
    • Recall: The ratio of true positive predictions to the actual positives.
    • F1 Score: The harmonic mean of precision and recall, balancing both metrics.

  • Regression:

    • Mean Absolute Error (MAE): The average of absolute differences between predicted and actual values.
    • Mean Squared Error (MSE): The average of squared differences between predicted and actual values.
    • R-squared: The proportion of variance in the dependent variable explained by the independent variables.

Evaluation Methods

  • Training-Validation-Test Split: Divide the dataset into three parts to train the model, validate its performance during training, and test its generalization on unseen data.
  • Cross-Validation: Split the dataset into multiple folds, training and validating the model on different subsets to ensure robustness and reduce overfitting.

Chanllenges

  • Imbalanced Data: When one class is significantly more frequent than others, leading to biased predictions.
  • Domain-separation: The model may perform well on the training data but poorly on unseen data due to differences in distribution.

Cases

  • Fraud Detection: Using classification metrics to identify fraudulent transactions in financial datasets.
  • Recommandation Systems: Employing regression metrics to evaluate the accuracy of predicted ratings or preferences.

Codes Examples

import torch

def mae_loss(y_true, y_pred):
    return torch.mean(torch.abs(y_pred - y_true))

y_true = torch.tensor([3.0, 5.0, 2.5])
y_pred = torch.tensor([2.5, 5.0, 3.0])
print(mae_loss(y_true, y_pred))

# loss_fn = torch.nn.L1Loss()

loss_fn = torch.nn.MSELoss()

def r_squared(y_true, y_pred):
    y_mean = torch.mean(y_true)
    ss_total = torch.sum((y_true - y_mean) ** 2)
    ss_res = torch.sum((y_true - y_pred) ** 2)
    return 1 - ss_res / ss_total

y_true = torch.tensor([3.0, 5.0, 2.5])
y_pred = torch.tensor([2.5, 5.0, 3.0])
print(r_squared(y_true, y_pred))

Examples

import torch
import torch.nn as nn

# 假设一个简单的线性模型
model = nn.Linear(1, 1)
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)

# 模拟数据
X = torch.randn(100, 1)
y_true = 2 * X + 1 + 0.1 * torch.randn(100, 1)

# 训练循环
for epoch in range(100):
    y_pred = model(X)
    loss = mse_loss(y_true, y_pred)
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

# 评估
with torch.no_grad():
    y_pred = model(X)

    print(f"MAE: {mae_loss(y_true, y_pred):.4f}")
    print(f"MSE: {mse_loss(y_true, y_pred):.4f}")
    print(f"R²: {r_squared(y_true, y_pred):.4f}")

THE END