PyTorch Tensors

Introduction

PyTorch tensors are similar to NumPy arrays but with GPU acceleration and automatic differentiation support.

Creating Tensors

import torch

# From Python list
tensor = torch.tensor([1, 2, 3])
matrix = torch.tensor([[1, 2], [3, 4]])

# From NumPy
import numpy as np
arr = np.array([1, 2, 3])
tensor = torch.from_numpy(arr)

# Special tensors
zeros = torch.zeros(3, 4)
ones = torch.ones(2, 3)
rand = torch.rand(3, 3)
randn = torch.randn(4, 4)  # Normal distribution

Tensor Operations

a = torch.tensor([[1, 2], [3, 4]])
b = torch.tensor([[5, 6], [7, 8]])

# Element-wise
c = a + b
d = a * b

# Matrix operations
e = torch.matmul(a, b)
f = a @ b  # Same as matmul

# Slicing
print(a[:, 1])  # Second column
print(a[0, :])   # First row

GPU Tensors

# Check GPU
print(torch.cuda.is_available())

# Move to GPU
if torch.cuda.is_available():
    device = torch.device('cuda')
    tensor_gpu = tensor.to(device)

# Create directly on GPU
tensor = torch.zeros(3, 3, device='cuda')

# Move back to CPU
tensor_cpu = tensor.cpu()

Gradients

x = torch.tensor([1.0, 2.0, 3.0], requires_grad=True)
y = x ** 2
z = y.sum()  # z = 1 + 4 + 9 = 14

z.backward()
print(x.grad)  # [2, 4, 6] = 2*x

In-place Operations

a = torch.tensor([1, 2, 3])
a.add_(5)    # In-place: modifies a
a.mul_(2)    # In-place: modifies a

Practice Problems

1. Create tensors from lists and arrays
2. Perform matrix operations
3. Move tensors between CPU and GPU
4. Compute gradients
5. Use in-place operations