Reinforcement Learning

Learning through trial, error, and rewards

Imagine training a dog! When the dog does something good (sits when asked), you give it a treat (reward). When it does something bad, no treat. Over time, the dog learns which actions lead to treats and does them more often. Reinforcement Learning works the same way - an AI agent takes actions in an environment, gets rewards for good actions and penalties for bad ones, and learns the best strategy over time. This is how AI learns to play games, control robots, and make decisions!

What is Reinforcement Learning?

Reinforcement Learning (RL) is a type of machine learning where an agent learns to make decisions by interacting with an environment. The agent takes actions, receives feedback in the form of rewards or penalties, and learns to maximize cumulative rewards over time. Unlike supervised learning (which needs labeled data) or unsupervised learning (which finds patterns), RL learns through experience and trial-and-error.

python
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
# Simple Reinforcement Learning Example: Robot Learning to Navigate
import random
# The Environment: 5x5 grid world
# Goal: Reach position (4,4) from (0,0)
# Actions: up, down, left, right
class GridWorld:
    def __init__(self):
        self.position = [0, 0]  # Start position
        self.goal = [4, 4]      # Goal position
    def reset(self):
        self.position = [0, 0]
        return tuple(self.position)
    def step(self, action):
        # Agent takes an action
        if action == 'up' and self.position[0] > 0:
            self.position[0] -= 1
        elif action == 'down' and self.position[0] < 4:
            self.position[0] += 1
        elif action == 'left' and self.position[1] > 0:
            self.position[1] -= 1
        elif action == 'right' and self.position[1] < 4:
            self.position[1] += 1
        # Environment gives reward
        if self.position == self.goal:
            reward = 100  # BIG reward for reaching goal!
            done = True
        else:
            reward = -1   # Small penalty for each step (encourages efficiency)
            done = False
        return tuple(self.position), reward, done
# Agent exploring the environment
env = GridWorld()
actions = ['up', 'down', 'left', 'right']
# Agent tries random actions and learns from rewards
state = env.reset()
total_reward = 0
steps = 0
print("Agent learning by trial and error:")
while steps < 10:
    action = random.choice(actions)
    next_state, reward, done = env.step(action)
    total_reward += reward
    steps += 1
    print(f"Step {steps}: Action={action}, Position={next_state}, Reward={reward}")
    if done:
        print(f"\n🎉 Goal reached! Total reward: {total_reward}")
        break
# Over many episodes, the agent learns the optimal path!

Key Components of RL

Every RL system consists of these essential elements:

🤖 Agent

The learner/decision-maker (e.g., game player, robot, trading algorithm)

Mario in Super Mario Bros

🌍 Environment

The world the agent interacts with and receives feedback from

The game level, obstacles

📍 State (S)

Current situation/configuration of the environment

Mario's position, enemies nearby

⚡ Action (A)

Possible moves/decisions the agent can make

Jump, move right, move left

🎁 Reward (R)

Immediate feedback (+ or -) after taking an action

+100 for coin, -50 for hitting enemy

🎯 Policy (π)

Strategy: which action to take in each state

If enemy nearby, jump; else, move right

The RL Cycle

State
→
Action
→
Reward
→
New State

This cycle repeats until the agent learns the optimal policy

How RL Works: The Learning Loop

The reinforcement learning process is a continuous cycle:

  1. 1.

    Observe State

    Agent perceives the current state of the environment

  2. 2.

    Choose Action

    Agent selects an action based on its current policy (strategy)

  3. 3.

    Execute Action

    Agent performs the action in the environment

  4. 4.

    Receive Reward & New State

    Environment gives feedback (reward/penalty) and transitions to new state

  5. 5.

    Update Policy

    Agent learns from experience and updates its strategy to maximize future rewards

  6. 6.

    Repeat

    Continue until the agent converges to an optimal policy

python
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
# Q-Learning: A Popular RL Algorithm
import numpy as np
import random
# Environment: Simple 1D world [0, 1, 2, 3, 4]
# Goal: Reach position 4 from position 0
# Actions: move left (-1) or right (+1)
# Initialize Q-table: Q[state, action]
# Rows = states (0-4), Columns = actions (left=0, right=1)
Q = np.zeros((5, 2))
# Hyperparameters
alpha = 0.1    # Learning rate
gamma = 0.9    # Discount factor (importance of future rewards)
epsilon = 0.2  # Exploration rate
episodes = 1000
for episode in range(episodes):
    state = 0  # Start at position 0
    while state != 4:  # Until reaching goal
        # Choose action: explore (random) or exploit (best known)
        if random.uniform(0, 1) < epsilon:
            action = random.choice([0, 1])  # Explore
        else:
            action = np.argmax(Q[state, :])  # Exploit
        # Execute action
        if action == 0:  # Move left
            next_state = max(0, state - 1)
        else:  # Move right
            next_state = min(4, state + 1)
        # Get reward
        reward = 100 if next_state == 4 else -1
        # Q-Learning update rule (THE CORE OF LEARNING!)
        Q[state, action] = Q[state, action] + alpha * (
            reward + gamma * np.max(Q[next_state, :]) - Q[state, action]
        )
        state = next_state
print("Learned Q-Table:")
print(Q)
print("\nOptimal Policy: Always move right to reach goal!")
print("Best action at each state:", np.argmax(Q, axis=1))
# After training, the agent learns:
# State 0: move right (action 1)
# State 1: move right (action 1)
# State 2: move right (action 1)
# State 3: move right (action 1)
# State 4: goal reached!

Popular RL Algorithms

Common approaches to solving RL problems:

Q-Learning

Model-Free, Value-Based

Learns Q-values (quality of actions) for each state-action pair

DQN (Deep Q-Network)

Deep RL

Uses neural networks to approximate Q-values; used by DeepMind for Atari games

SARSA

On-Policy

Similar to Q-Learning but updates based on action actually taken

Policy Gradient

Policy-Based

Directly learns the policy (what to do) instead of value function

Actor-Critic

Hybrid

Combines policy-based (actor) and value-based (critic) approaches

PPO (Proximal Policy Optimization)

Advanced

State-of-the-art algorithm, stable and efficient; used by OpenAI

A3C (Asynchronous Actor-Critic)

Parallel Training

Multiple agents learn in parallel, speeds up training

AlphaGo (Monte Carlo Tree Search)

Game Playing

Combines RL with search; beat world champion in Go

Key Concepts

Agent

The learner or decision-maker that interacts with the environment (e.g., game player, robot, trading bot).

Environment

The world in which the agent operates and receives feedback (e.g., game board, physical world, stock market).

State (S)

The current situation or configuration of the environment at a given time.

Action (A)

Possible moves or decisions the agent can make in a given state.

Reward (R)

Immediate feedback (positive or negative) the agent receives after taking an action.

Policy (π)

The strategy that defines which action to take in each state. The goal is to find the optimal policy.

Value Function

Estimates the long-term reward expected from a state (how good it is to be in that state).

Q-Function (Q-Value)

Estimates the expected reward for taking a specific action in a specific state.

Interview Tips

  • 💡Explain RL as 'learning by trial and error with rewards and penalties' - different from supervised (labeled data) and unsupervised (pattern finding)
  • 💡Know the core components: Agent, Environment, State, Action, Reward, Policy
  • 💡Understand the exploration vs exploitation tradeoff: try new things vs. use known good strategies
  • 💡Common algorithms: Q-Learning (model-free, value-based), DQN (deep Q-learning with neural networks), Policy Gradients, Actor-Critic, PPO
  • 💡Key challenges: credit assignment (which action caused the reward?), sparse rewards, convergence time
  • 💡Real-world applications: game playing (AlphaGo, Atari), robotics, autonomous vehicles, recommendation systems, resource allocation
  • 💡Explain Q-learning update rule: Q(s,a) = Q(s,a) + α[r + γ·max Q(s',a') - Q(s,a)]