#+tblfm: $1=@#-1

a.

I went ahead and computed the values anyway with python:

right = 0
for i in data:
    if i[1] == i[2]:
        right += 1
accuracy = right/len(data)
error_rate = 1 - accuracy
out = [["accuracy:", accuracy],["error rate:", error_rate]]
return out

b

• True positive: 2
• False positive: 3
• True negative: 1
• false negative: 2

c, d

• Sensitivity = true positive rate = 1/2
• Specificity = true negative rate = 1/4

The classifier does not have very good performance. It is more likely to return a false positive or false negative than a true positive or negative.

e.

• Accuracy = precision = (2/(2+3)) = 2/5
• recall = 2/(2+2) = 1/2

The accuracy and recall are very far from 1, this is a bad classifier.

f.

\(F1 = \frac{2 \cdot pred \cdot recall}{pred + recall} = \frac{8/20}{18/20}\)

2.

a-b

#+tblfm: $1=@#-1

#+tblfm: $4='(if (>= $3 0.8) 1 -1 );N

#+tblfm: $5='(if (>= $3 0.5) 1 -1 );N

#+tblfm: $1=@#-1

#+tblfm: $4='(if (>= $3 0.8) 1 -1 );N

#+tblfm: $5='(if (>= $3 0.5) 1 -1 );N

def true_false(name, data):
    true_positives_8 = 0
    false_positives_8 = 0
    true_negatives_8 = 0
    false_negatives_8 = 0
    true_positives_5 = 0
    false_positives_5 = 0
    true_negatives_5 = 0
    false_negatives_5 = 0
    for i in data:
            if i[3] == 1:

                if i[1] ==1:
                    true_positives_8 += 1
                else:
                    false_positives_8 += 1
            if i[3] == -1:

                if i[1] ==1:
                    false_negatives_8 += 1
                else:
                    true_negatives_8 += 1
            if i[4] == 1:
                if i[1] == 1:
                    true_positives_5 += 1
                else:
                    false_positives_5 += 1
            if i[4] == -1:

                if i[1] ==1:
                    false_negatives_5 += 1
                else:
                    true_negatives_5 += 1

    def recall(r1 ,r2):
        return r1/(r1+r2)

    out = [[name, "p >= 0.8", "p >= 0.5"],
        ["true positive rate", recall(true_positives_8, false_negatives_8), recall(true_positives_5, false_negatives_5)],
        ["false positive rate", recall(false_positives_8, true_negatives_8), recall(false_positives_5, true_negatives_5)]
        ]
    return out
return true_false("Table 1", table1) + true_false("Table2", table2)

c-d

import matplotlib.pyplot as plt
p1 = data[1][1], data[2][1]
p2 = data[1][2], data[2][2]
p3 = data[4][1], data[5][1]
p4 = data[4][2], data[5][2]
fig,(axes1, axes2) = plt.subplots(1, 2, sharex=True, sharey=True)
axes1.plot((0, p1[1], p2[1]),[0, p1[0], p2[0]])
axes1.scatter((0, p1[1], p2[1]),[0, p1[0], p2[0]])
axes1.set_title("NB1")
axes2.plot((0, p3[1], p4[1]),[0, p3[0], p4[0]])
axes2.scatter((0, p3[1], p4[1]),[0, p3[0], p4[0]])
axes2.set_title("NB2")
fig.supxlabel("False Positive rate")
fig.supylabel("True Positive rate")

plt.show()

NB2 is the better classifier because it continues to improve the true positive rate across the graph, while NB1 is initially good, but after 0.5 true positive rate, it no longer improves.

3.

5 fold validation implies 5 separate "folds" or groups of test data as follows:

#+tblfm: $1=@# -1

4.

• Boostrapping selects sets of data of a fixed size an arbitrary number of times, this forms a new "fake" training set for each member of the ensemble (a set of weak learners)
• In boosting, data in random is assigned a weight based on the performance of the previous classifier, (initialized equal). This method iterates upon previous classifiers by increasing the weight of data that is misclassified. This technique aims to minimize bias, while bagging aims to reduce variance.
• Bagging reduces the variance by averaging output of many classifiers that are trained on random samples of the data. Using the independent and representative nature of random samples, prevents weak learners from being fully independent.
• Boosting reduces the bias by assigning weights to data that is incorrectly classified. This successive optimization minimizes the amount of prejudiced results (bias).