4. Consider the training set given below for determining...

4. Consider the training set given below for determining whether a loan application should be approved or rejected. Draw the full decision tree obtained using entropy as the impurity measure. Show your steps clearly (i.e., the computation of information gain for every candidate attribute must be shown). Compute the training error of the decision tree. | Long-Term Debt | Unemployed | Credit Rating | Down Payment < 20% | Class | | :---: | :---: | :---: | :---: | :---: | | No | No | Good | Yes | Approve | | No | No | Bad | No | Approve | | No | No | Bad | Yes | Approve | | No | No | Bad | No | Approve | | Yes | No | Good | No | Approve | | No | Yes | Good | Yes | Reject | | Yes | No | Bad | No | Reject | | Yes | No | Bad | Yes | Reject | | Yes | No | Bad | Yes | Reject | | Yes | Yes | Bad | No | Reject |

Transcript

00:01 Okay, so we'll start from the long -term debt and calculate the entropy.

00:06 Entropy is given by negative of the sum of the count of a specific class over total times log base 2 of the count i over the total.

00:29 Okay, so we're going to start from the long -term debt.

00:33 So the so there are two categories here no and yes and they have each of the count as so there are five yes and five nodes five over ten log two five over ten plus five over ten and log two five over ten okay so that's equal to one and let's see here and then next you calculate the information gain so this is the total entropy and then you need to calculate now the subset entropy so again there are five here we're again we're doing the long -term debt so five yes and five no and when there is yes we have a how many approve approve and then all the rest would be reject.

02:19 So one approve and four reject.

02:24 And when it's no, we have four approve and one reject.

02:37 And then you calculate, you must calculate entropy based on these.

02:43 So let's calculate entropy for five years.

02:50 A negative 1 over 5 total log 2 1 over 5 plus 4 over 5 log 2 4 over 5 okay so let me calculate this one here 0 .7 to 19 and then entropy for the 5 nose so there are 4 8 4 8 4 approves and then one reject.

03:49 Okay, so it's going to be the same.

03:51 0 .7 -1 -19.

03:54 And then you calculate the subset entry, entropy.

03:59 So this would be the 5.

04:03 So the length of the yes or a number of yes divided by the total which is 10 times the subset entry and entropy plus 5 over 10 times again the subset of the entropy of the subset so that gives 0 .729 and so long -term debt has the information gain of entropy minus the subset entropy which gives 0 .2781 and you're going to do the same for all of it.

05:03 So this is the long -term debt.

05:08 And then similarly, information gain for unemployed is 0 .2365...

Question

Please give Ace some feedback