torch random uniform

Mac Grediser

2 min read

·

27-11-2024

1

0

Share

Understanding PyTorch's torch.rand() and torch.randn() for Random Number Generation

PyTorch, a powerful deep learning framework, provides efficient tools for generating random numbers, crucial for tasks like initializing model weights, data augmentation, and stochastic gradient descent. Two fundamental functions for this are torch.rand() and torch.randn(), both creating tensors filled with random numbers, but with key differences in their distributions.

torch.rand(size): Uniform Distribution

torch.rand() generates random numbers following a uniform distribution within the range [0, 1). This means each number in this range has an equal probability of being selected. The size argument specifies the shape of the output tensor.

import torch

# Create a tensor of size (3, 4) filled with random numbers from a uniform distribution
uniform_tensor = torch.rand(3, 4)
print(uniform_tensor)

This will produce a 3x4 tensor where each element is a random float between 0 and 1 (exclusive of 1).

Controlling the Range: While torch.rand() inherently produces numbers between 0 and 1, you can easily scale and shift the output to generate random numbers within a different range [a, b):

a = 2.0
b = 10.0
scaled_tensor = a + (b - a) * torch.rand(3, 4)
print(scaled_tensor)

This code generates a tensor with random numbers between a (2.0) and b (10.0).

torch.randn(size): Normal (Gaussian) Distribution

torch.randn() generates random numbers from a standard normal (Gaussian) distribution, which has a mean of 0 and a standard deviation of 1. The bell-shaped curve of the normal distribution is commonly used in many statistical and machine learning applications.

# Create a tensor of size (2, 2) filled with random numbers from a standard normal distribution
normal_tensor = torch.randn(2, 2)
print(normal_tensor)

This produces a 2x2 tensor with numbers centered around 0, with most values concentrated near 0 and fewer values further away.

Adjusting Mean and Standard Deviation: You can modify the mean and standard deviation to create a normal distribution with different characteristics:

mean = 5.0
std = 2.0
adjusted_tensor = mean + std * torch.randn(2, 2)
print(adjusted_tensor)

This generates a tensor with a normal distribution centered around mean (5.0) with a standard deviation of std (2.0).

Choosing Between rand() and randn():

The choice between torch.rand() and torch.randn() depends on your specific needs:

• Uniform distribution (rand()): Use when you need equal probability for all numbers within a specified range. This is often suitable for tasks like randomly initializing weights where a wide range of initial values might be beneficial for exploration.

• Normal distribution (randn()): Use when you need numbers clustered around a mean, following the bell curve. This is common for weight initialization in neural networks (e.g., Xavier or He initialization) as it helps with gradient flow during training and prevents exploding or vanishing gradients.

Understanding the properties of these functions is essential for effectively leveraging PyTorch's capabilities in deep learning and other numerical computations. Remember to always consider the statistical implications of your choice of distribution when generating random numbers.