Artificial Neurons

Structure of An Artificial Neuron

An artificial neuron consists of several components that mimic the biological neuron:

• Inputs: These are the signals received from other neurons or external sources.
• Weights: Each input is associated with a weight that determines its importance.
• Bias: An additional parameter that helps adjust the output independently of the inputs.
• Activation Function: This function processes the weighted sum of inputs to produce an output.

Neural's Calculation

The output of an artificial neuron can be mathematically expressed as:

• Weights sum Mutiply each input by its corresponding weight, then sum these products, and add the bias.

• Activation function Apply an activation function to the weighted sum to produce the final output.

Example of an Artificial Neuron

import torch
class SimpleNeuron(torch.nn.Module):
    def __init__(self, input_size):
        super(SimpleNeuron, self).__init__()
        self.weights = torch.nn.Parameter(torch.randn(input_size))
        self.bias = torch.nn.Parameter(torch.randn(1))

    def forward(self, x):
        return torch.sigmoid(torch.dot(x, self.weights) + self.bias)

Layers of Neurons

Artificial neurons are the building blocks of neural networks, designed to mimic the behavior of biological neurons. Neurons are organized into layers, with each layer performing specific computations.

• Input Layer: Receives input data.
• Hidden Layers: Perform intermediate computations and feature extraction.
• Output Layer: Produces the final output of the network.

Chanllenge

• Vanishing gradient Problem: In deep networks, gradients can become very small, making it difficult to update weights effectively.
• Exploding gradient Problem: Conversely, gradients can become too large, leading to unstable training.
• Overfitting: The model learns noise in the training data, resulting in poor generalization
• Underfitting: The model is too simple to capture the underlying patterns in the data.
• Computational Complexity: Training deep networks can be computationally expensive and time-consuming.