Neural Network Basics

What is a Neural Network?

Neural networks are computing systems inspired by biological neural networks. They consist of interconnected nodes (neurons) organized in layers that process information.

At its core, a neural network learns by adjusting the weights of connections between neurons. The network receives input, processes it through hidden layers, and produces an output. This allows it to learn complex patterns in data.

Basic Neural Network Structure

A neural network has three main types of layers:

neural_network_structure.py
# Neural Network Structure

print("Neural Network Architecture:")
print("=" * 50")

print("\n1. Input Layer:")
print("   - Receives the input features")
print("   - Number of neurons = number of input features")
print("   - Example: 3 features → 3 input neurons")

print("\n2. Hidden Layers:")
print("   - Process information between input and output")
print("   - Can have multiple hidden layers (deep networks)")
print("   - Each neuron applies: output = activation(weighted_sum)")
print("   - Example: 2 hidden layers with 5 neurons each")

print("\n3. Output Layer:")
print("   - Produces the final prediction")
print("   - Number of neurons = number of output classes")
print("   - Example: Binary classification → 1 output neuron")

# Example: Simple neural network
print("\nExample Network Structure:")
print("  Input: 3 features")
print("  Hidden 1: 5 neurons")
print("  Hidden 2: 3 neurons")
print("  Output: 1 neuron")
print("  Total connections: 3×5 + 5×3 + 3×1 = 33 weights")

How Neurons Work

Each neuron in a neural network performs a simple computation:

neuron_computation.py
# How a Single Neuron Works

# Example: Calculating neuron output
inputs = [0.5, 0.8, 0.3]  # Input values
weights = [0.2, -0.5, 0.7]  # Connection weights
bias = 0.1  # Bias term

print("Neuron Computation Steps:")
print(f"  Inputs: {inputs}")
print(f"  Weights: {weights}")
print(f"  Bias: {bias}")

# Step 1: Calculate weighted sum
weighted_sum = sum(x * w for x, w in zip(inputs, weights)) + bias
print(f"\nStep 1 - Weighted Sum: {weighted_sum:.3f}")
print("  Formula: sum(inputs × weights) + bias")

# Step 2: Apply activation function (sigmoid example)
import math
def sigmoid(x):
    return 1 / (1 + math.exp(-x))

output = sigmoid(weighted_sum)
print(f"\nStep 2 - Activation (Sigmoid): {output:.3f}")
print("  Formula: 1 / (1 + e^(-weighted_sum))")
print("  Range: 0 to 1")

print(f"\nFinal Neuron Output: {output:.3f}")

Forward Propagation

Forward propagation is how data flows through the network from input to output:

forward_propagation.py
# Forward Propagation Through Network

print("Forward Propagation Process:")
print("=" * 50")

# Simple network: 2 inputs → 2 hidden → 1 output
input_layer = [1.0, 0.5]  # Input features

print(f"\nStep 1: Input Layer")
print(f"  Values: {input_layer}")

# Hidden layer weights (2 inputs × 2 neurons)
hidden_weights = [
    [0.5, -0.3],  # Weights for neuron 1
    [0.2, 0.8]   # Weights for neuron 2
]
hidden_bias = [0.1, -0.2]

print(f"\nStep 2: Hidden Layer Computation")
print("  For each hidden neuron:")
print("    1. Calculate weighted sum")
print("    2. Add bias")
print("    3. Apply activation function")

# Simulate hidden layer output
hidden_output = [0.7, 0.6]  # After activation
print(f"  Hidden layer output: {hidden_output}")

# Output layer
output_weights = [0.4, 0.6]
output_bias = -0.1

print(f"\nStep 3: Output Layer Computation")
output = sum(h * w for h, w in zip(hidden_output, output_weights)) + output_bias
print(f"  Final output: {output:.3f}")

print("\nForward propagation complete!")

Exercise: Build a Simple Neural Network

Complete the exercise on the right side:

Task 1: Define input values and weights for a neuron
Task 2: Calculate weighted sum (inputs × weights + bias)
Task 3: Apply activation function (sigmoid) to get output
Task 4: Simulate forward propagation through multiple layers

Write your code to implement basic neural network computations!

💡 Learning Tip

Practice is essential. Try modifying the code examples, experiment with different parameters, and see how changes affect the results. Hands-on experience is the best teacher!

🎉

Lesson Complete!

Great work! Continue to the next lesson.