beginner/examples_tensor/polynomial_numpy

Run in Google Colab

Colab

Download Notebook

Notebook

View on GitHub

GitHub

Warm-up: numpy

Created On: Dec 03, 2020 | Last Updated: Dec 03, 2020 | Last Verified: Nov 05, 2024

A third order polynomial, trained to predict \(y=\sin(x)\) from \(-\pi\) to \(pi\) by minimizing squared Euclidean distance.

This implementation uses numpy to manually compute the forward pass, loss, and backward pass.

A numpy array is a generic n-dimensional array; it does not know anything about deep learning or gradients or computational graphs, and is just a way to perform generic numeric computations.

99 2898.826973791798
199 1941.414365493124
299 1302.0169966387516
399 874.7513531986671
499 589.0628952616794
599 397.9158940474069
699 269.93797742785534
799 184.19310110533883
899 126.7020996862142
999 88.12557127575029
1099 62.22012913448166
1199 44.80942063509315
1299 33.097926770817
1399 25.21311706345879
1499 19.899801653505513
1599 16.31597095760777
1699 13.896343488013478
1799 12.261109936519393
1899 11.154863504002519
1999 10.405703332892887
Result: y = 0.02702116413951984 + 0.8269871180446964 x + -0.004661600448179268 x^2 + -0.08909818075292819 x^3

import numpy as np
import math

# Create random input and output data
x = np.linspace(-math.pi, math.pi, 2000)
y = np.sin(x)

# Randomly initialize weights
a = np.random.randn()
b = np.random.randn()
c = np.random.randn()
d = np.random.randn()

learning_rate = 1e-6
for t in range(2000):
    # Forward pass: compute predicted y
    # y = a + b x + c x^2 + d x^3
    y_pred = a + b * x + c * x ** 2 + d * x ** 3

    # Compute and print loss
    loss = np.square(y_pred - y).sum()
    if t % 100 == 99:
        print(t, loss)

    # Backprop to compute gradients of a, b, c, d with respect to loss
    grad_y_pred = 2.0 * (y_pred - y)
    grad_a = grad_y_pred.sum()
    grad_b = (grad_y_pred * x).sum()
    grad_c = (grad_y_pred * x ** 2).sum()
    grad_d = (grad_y_pred * x ** 3).sum()

    # Update weights
    a -= learning_rate * grad_a
    b -= learning_rate * grad_b
    c -= learning_rate * grad_c
    d -= learning_rate * grad_d

print(f'Result: y = {a} + {b} x + {c} x^2 + {d} x^3')