does KNN have a hypothesis space? if yes, Is there a way to clearly define the hypothesis space with the instances?

I am learning this post, "A few useful things to know about machine learning"

The author says

A classifier must be represented in some formal language that the computer can handle.Conversely, choosing a representation for a learner is tantamount to choosing the set of classifiers that it can possibly learn. This set is called the hypothesis space of the learner.

and put KNN and SVM in the instance-based category.

I assume the instance here mean each individual example, for instance, KNN needs to compute the distance to each example.

Assume there are 300 examples to train a KNN classifier, how to understand choosing this specific representation is tantamount to choose hypothesis space?

to clarify the question.

does KNN have a hypothesis space? if yes, Is there a way to clearly define the hypothesis space with the instances?

Topic machine-learning

Category Data Science

1
answered at

The hypothesis in this case is a way to formally describe the model. The model could be a hyperplane, a set of conditions on input with a constant for each branch (decision trees) or whatever you like. The set of all possible hypothesis in a given representation is named hypothesis space. Knn prediction is realized using the first k instances from a training data set. So knn cannot represent a hyperplane because of the finite amount of datapoints. This is why choosing a representation restricts the hypothesis space. You are restricted to things which can be represented by instances and distance. SVM is the same because the support vectors are selected instances from the training dataset and the fitted surface is obtained using support vectors and kernelized distances from those.

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.