b) Plot the data points on the x-y plane and prepare the element-to-element
distance matrix $d(x_i, x_j) = \sqrt{(x_i - x_j)^2 + (y_i - y_j)^2}$ ($i, j = 1, 2, \dots, 200$)
that will be the input for the clustering algorithm. Also assume each data
point carries the same weight, i.e., $p(x_i) = 1/200$ for all $i$. (2p)
c) Write a code (no restriction on the program language) to implement the
Blahut-Arimoto algorithm to evaluate the clustering membership probabi-
lity, $p(\vec{x}|\vec{x})$, with fixed number of clusters, $N_c$, and compression-distortion
tradeoff parameter, $\beta$. Your code should implement a multiple run each
starting with random initial conditions. Note: Your source code should
include clear comments/documentations to describe what are evaluating.
I may later randomly ask a few students, especially those without clear
documentations, to demonstrate how their code works. (15p)
d) Run your code to construct the information curve for $N_c = 2, 3, 4$. Hint:
Choose different values of $\beta$ in between 1 to 40. (6p)
e) As we have already known that the correct number of clusters is 2, pro-
pose a reasonable way using the quantities evaluated from your code (e.g.
$I(X, \tilde{X})$, $(d(x, \tilde{x}))p(x, \tilde{x})$, the Lagrange function, etc.) and the information
curves to correctly identify the number of clusters. You should clearly ex-
plain your rationale and state explicitly which quantities, graphs and/or
curves are used in the identification. (5p)
