The Perceptron Learning Algorithm
In this problem, you will create your own target function f and data set D to see how the Perceptron Learning Algorithm works. Take d = 2 so you can visualize the problem, and assume x = [-1,1] x [-1,1] with a uniform probability of picking each x ∈ X.
In each run, choose a random line in the plane as your target function (do this by taking two random uniformly distributed points in [-1,1] x [-1,1] and taking the line passing through them), where one side of the line maps to + and the other side maps to -.
Choose the inputs Xn of the data set as random points (uniformly in X), and evaluate the target function on each Xn to get the corresponding output Yn.
Now, in each run, use the Perceptron Learning Algorithm to find g. Start the PLA with the weight vector being all zeros (consider sign(0) = 0, so all points are initially misclassified), and at each iteration, have the algorithm choose a point randomly from the set of misclassified points.
We are interested in two quantities: the number of iterations that PLA takes to converge to g, and the disagreement between f and g (the probability that f(x) ≠g(x)) for a random point. You can either calculate this probability exactly or approximate it by generating a sufficiently large separate set of points to estimate it.
In order to get a reliable estimate for these two quantities, you should repeat the experiment for 1000 runs (each run as specified above) and take the average over these runs.
Now, try N = 100. How many iterations does it take on average for the PLA to converge for 100 training points? Pick the value closest to your results.
- 100
- 500
- 1000
- 5000
Which of the following is the closest to P[f(x) ≠g(x)] for N = 1002?
- 0.001
- 0.01