Deep Learning Track WiSe 24/25
Deep Learning Track WiSe 24/25
Deep Learning Track WiSe 24/25
  • Welcome to the Deep Learning Track
  • Setup
  • Learning Material
  • Section 1 - The Math
    • Derivatives and Gradients
    • Vectors, Matrices and Tensors
    • The power of matrix computation
    • Exercise - Matrix Computation
  • Section 2 - The Data
    • PyTorch Datasets and Data Loaders
    • Working with Data Tables
    • Exercise - Loading Data from a CSV file
    • Working with Images
    • Exercise - Image Datasets
    • Working with Text
  • Section 3 - Neural Networks
    • Activation Functions
    • Exercise - Activation Functions
    • Exercise - The Softmax Function
    • The Neuron
    • Two type of applications: Regression and Classification
    • Loss Functions
    • Exercise - Regression Loss Functions
    • Exercise - Classification Loss Functions
    • The Gradient Descent Algorithm
    • Exercise - Implementing Gradient Descent
    • Exercise - PyTorch Autograd
    • Exercise - Regression with Neural Networks
    • Exercise - Classification with Neural Networks
    • Playground - Neural Networks
  • Section 4 - Convolutional Neural Networks
    • Convolution
    • Convolutional Neural Networks
    • Classifying handwritten digits
    • Playground - Convolutional Neural Networks
    • Transfer Learning
  • Final Project - Text Classification
  • Further Resources
    • Computer Vision Libraries
    • Image Classification with PyTorch
    • Object Detection with PyTorch
    • Deep AI Explainability
Powered by GitBook
On this page
  • Regression
  • Classification / Logistic Regression
  1. Section 3 - Neural Networks

Two type of applications: Regression and Classification

We will have a short look on two types of problems that can be solved by neuran networks: regression and classification.

Regression

In mathematics, regression refers to a statistical modeling technique used to analyze the relationship between a dependent variable and one or more independent variables. It is commonly employed to make predictions or estimate the value of the dependent variable based on the values of the independent variables.

The goal of regression analysis is to find the best-fitting mathematical function or model that describes the relationship between the variables. This allows us to understand how changes in the independent variables influence the dependent variable.

The most basic form of regression is simple linear regression, which assumes a linear relationship between the dependent variable and a single independent variable. The relationship is described by a straight line equation of the form:

y = mx + b

where "y" represents the dependent variable, "x" represents the independent variable, "m" represents the slope of the line, and "b" represents the y-intercept.

However, regression analysis can be extended to handle more complex relationships using techniques such as multiple linear regression, polynomial regression, logistic regression, and many others. These methods allow for the inclusion of multiple independent variables and the modeling of non-linear relationships.

Regression models are typically fitted to data through a process called "training," where the model's parameters (such as the slope and intercept in linear regression) are adjusted to minimize the difference between the predicted values and the actual observed values. This is often done using optimization algorithms that seek to find the best-fitting parameters.

Once the regression model is trained, it can be used to make predictions or estimate the values of the dependent variable for new or unseen data points based on the values of the independent variables.

Classification / Logistic Regression

In mathematics, classification is a technique used to categorize or label data into different classes or categories based on their characteristics or attributes. It is a fundamental concept in statistics and machine learning and is used to solve problems where the goal is to assign a class or label to a given input based on its features.

In classification, we have a set of predefined classes or categories, and the task is to develop a model or algorithm that can learn from labeled training data and accurately predict the class of new, unseen data points.

Logistic regression is a statistical modeling technique used for binary classification problems, where the goal is to predict the probability of an event belonging to one of two possible outcomes or classes.

Despite its name, logistic regression is a classification algorithm rather than a regression algorithm. The term "logistic" refers to the logistic function that is used to model the relationship between the input variables and the binary outcome.

In logistic regression, the dependent variable is binary (taking values 0 or 1), representing the two possible classes. The independent variables can be either continuous or categorical.

The logistic regression model uses the logistic function (also known as the sigmoid function) to map the linear combination of the independent variables to a probability value between 0 and 1. The logistic or sigmoid function is defined as:

p = 1 / (1 + e^(-z))

Where:

  • p represents the probability of the event occurring (i.e., belonging to class 1).

  • z represents the linear combination of the independent variables and their corresponding coefficients.

The logistic regression model estimates the values of the coefficients that maximize the likelihood of the observed data. This is typically done using optimization algorithms such as maximum likelihood estimation or gradient descent.

During training, the logistic regression model learns the relationship between the independent variables and the binary outcome by adjusting the coefficients. Once the model is trained, it can be used to predict the probability of the event occurring for new, unseen data points. A threshold value is then applied to convert the probabilities into class labels.

Logistic regression has several advantages, including simplicity, interpretability, and efficiency. It is widely used in various fields, including medicine, social sciences, finance, and machine learning, for tasks such as predicting disease outcomes, credit risk assessment, customer churn prediction, and spam email classification, among others.

PreviousThe NeuronNextLoss Functions