0. ML Overview/8. Evaluation Metrics in ML.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "3d95d7a6-0a3c-4d23-a3b5-1fdbabe929e6",
   "metadata": {},
   "source": [
    "# Simple Evaluation Metrics in Machine Learning\r\n",
    "\r\n",
    "Evaluation metrics are crucial for assessing the performance of machine learning models. Different metrics are used based on the type of model and the specific problem being solved. Here's an overview of some simple evaluation metrics and their typical applications\n",
    "## 1. Mean Absolute Error (MAE)\n",
    "- **Description**: The average of the absolute differences between the predicted values and actual values.\n",
    "- **Formula**: `MAE = sum(|Actual - Predicted|) / n`\n",
    "- **Used in**: Commonly used in regression models to measure the average error in predictions.\n",
    "\n",
    "## 2. Mean Squared Error (MSE)\n",
    "- **Description**: The average of the squared differences between the predicted values and actual values.\n",
    "- **Formula**: `MSE = sum((Actual - Predicted)^2) / n`\n",
    "- **Used in**: Also used in regression models, particularly when large errors are more significant.\n",
    "\n",
    "## 3. Root Mean Squared Error (RMSE)\n",
    "- **Description**: The square root of the mean squared error.\n",
    "- **Formula**: `RMSE = sqrt(MSE)`\n",
    "- **Used in**: Regression models, similar to MSE but more interpretable as it's in the units of the response variable.\n",
    "\n",
    "## 4. R-squared (Coefficient of Determination)\n",
    "- **Description**: Represents the proportion of the variance in the dependent variable that's predictable from the independent variables.\n",
    "- **Formula**: `R^2 = 1 - (Sum of Squared Residuals / Total Sum of Squares)`\n",
    "- **Used in**: Regression models to measure how well the model explains the variance in the data.\n",
    "  \n",
    "## 5. P-Value\n",
    "- **Description**: The P-value is a statistical measurement used to assess the significance of findings or to test hypotheses. In machine learning, it's often used in feature selection.\n",
    "- **Application**: A low P-value (typically ≤ 0.05) indicates that the feature is likely to be meaningful in predicting the target variable.\n",
    "\n",
    "fic objectives and constraints of the problem at hand.\r\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a87b3483-d9d5-430a-8018-1d00538d2b71",
   "metadata": {},
   "source": [
    "\n",
    "## 1. Accuracy\n",
    "- **Description**: The ratio of correctly predicted observations to the total observations.\n",
    "- **Formula**: `Accuracy = (True Positives + True Negatives) / Total Observations`\n",
    "- **Used in**: Commonly used in classification models, especially when the classes are balanced.\n",
    "\n",
    "## 2. Precision\n",
    "- **Description**: The ratio of correctly predicted positive observations to the total predicted positives.\n",
    "- **Formula**: `Precision = True Positives / (True Positives + False Positives)`\n",
    "- **Used in**: Important in scenarios where false positives are a bigger concern than false negatives (e.g., spam detection).\n",
    "\n",
    "## 3. Recall (Sensitivity)\n",
    "- **Description**: The ratio of correctly predicted positive observations to all observations in the actual class.\n",
    "- **Formula**: `Recall = True Positives / (True Positives + False Negatives)`\n",
    "- **Used in**: Crucial for models where missing a positive (false negatives) is costly (e.g., disease diagnosis).\n",
    "\n",
    "## 4. F1 Score\n",
    "- **Description**: The weighted average of Precision and Recall, used when seeking a balance between Precision and Recall.\n",
    "- **Formula**: `F1 Score = 2 * (Precision * Recall) / (Precision + Recall)`\n",
    "- **Used in**: Useful in classification tasks where both false positives and false negatives are important.\n",
    "\n",
    "## 5. Area Under the ROC Curve (AUC-ROC)\n",
    "- **Description**: Measures the ability of a classifier to distinguish between classes.\n",
    "- **Formula**: Area under the Receiver Operating Characteristic (ROC) curve.\n",
    "- **Used in**: Binary classification models, especially useful for imbalanced datasets.\n",
    "\n",
    "## 6. CAP (Cumulative Accuracy Profile) Curve\n",
    "- **Description**: The CAP Curve is used to evaluate the performance of classification models. It shows the cumulative percentage of true positives out of the total positive instances as a function of the percentage of the classifier's population.\n",
    "- **Application**: Used to understand the discriminatory power of a classification model, often compared against a random classifier and an ideal classifier.\n",
    "\n",
    "## 7. Confusion Matrix\n",
    "- **Description**: A table used to describe the performance of a classification model on a set of test data for which the true values are known.\n",
    "- **Formula**: Tabulates Actual vs. Predicted values.\n",
    "- **Used in**: All classification models to visualize the performance, especially for multiclass classification.\n",
    "\n",
    "Each of these metrics provides different insights into the model's performance and should be selected according to the specific objectives and constraints of the problem at hand."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d4fad635-f5cf-40ae-af3c-c36d139b4da6",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}