2、K 近邻算法的三要素
K 近邻算法使用的模型实际上对应于对特征空间的划分。K 值的选择，距离度量和分类决策规则是该算法的三个基本要素：
K 值的选择会对算法的结果产生重大影响。K值较小意味着只有与输入实例较近的训练实例才会对预测结果起作用，但容易发生过拟合；如果 K 值较大，优点是可以减少学习的估计误差，但缺点是学习的近似误差增大，这时与输入实例较远的训练实例也会对预测起作用，使预测发生错误。在实际应用中，K 值一般选择一个较小的数值，通常采用交叉验证的方法来选择最优的 K 值。随着训练实例数目趋向于无穷和 K=1 时，误差率不会超过贝叶斯误差率的2倍，如果K也趋向于无穷，则误差率趋向于贝叶斯误差率。
该算法中的分类决策规则往往是多数表决，即由输入实例的 K 个最临近的训练实例中的多数类决定输入实例的类别
距离度量一般采用 Lp 距离，当p=2时，即为欧氏距离，在度量之前，应该将每个属性的值规范化，这样有助于防止具有较大初始值域的属性比具有较小初始值域的属性的权重过大。
k最近邻kNN算法的应用
1、kNN代码解读
"""Regression based on k-nearest neighbors.
The target is predicted by local interpolation of the targets associated of the nearest neighbors in the training set.
Read more in the :ref:`User Guide <regression>`.
Parameters
----------
n_neighbors : int, optional (default = 5)
Number of neighbors to use by default for :meth:`kneighbors queries.
weights : str or callable
weight function used in prediction. Possible values:
- 'uniform' : uniform weights. All points in each neighborhood are weighted equally.
- 'distance' : weight points by the inverse of their distance.
in this case, closer neighbors of a query point will have a greater influence than neighbors which are further away.
- [callable] : a user-defined function which accepts an array of distances, and returns an array of the same shape containing the weights.
Uniform weights are used by default.
algorithm : {'auto', 'ball_tree', 'kd_tree', 'brute'}, optional
Algorithm used to compute the nearest neighbors:
- 'ball_tree' will use :class:`BallTree`
- 'kd_tree' will use :class:`KDTree`
- 'brute' will use a brute-force search.
- 'auto' will attempt to decide the most appropriate algorithm
based on the values passed to :meth:`fit` method.
基于k近邻的回归。
通过对训练集中最近邻相关的目标进行局部插值来预测目标。
请参阅:ref: ' User Guide <regression> '。</regression>
参数
---------
n_neighbors: int，可选(默认= 5)
kneighbors:meth: ' kneighbors查询默认使用的邻居数。
权值:str或callable
用于预测的权函数。可能的值:
-“均匀”:重量均匀。每个邻域中的所有点的权值都是相等的。
-“距离”:权重点的距离的倒数。
在这种情况下，查询点附近的邻居比远处的邻居有更大的影响。
- [callable]:一个用户定义的函数，它接受一个距离数组，并返回一个包含权值的形状相同的数组。
默认情况下使用统一的权重。
算法:{'auto'， 'ball_tree'， 'kd_tree'， 'brute'}，可选
计算最近邻的算法:
- 'ball_tree'将使用:class: ' BallTree '
- 'kd_tree'将使用:class: ' KDTree '
-“蛮力”将使用蛮力搜索。
- 'auto'将尝试决定最合适的算法
基于传递给:meth: ' fit '方法的值。
Note: fitting on sparse input will override the setting of this parameter, using brute force.
leaf_size : int, optional (default = 30)
Leaf size passed to BallTree or KDTree. This can affect the speed of the construction and query, as well as the memory required to store the tree. The optimal value depends on the nature of the problem.
p : integer, optional (default = 2)
Power parameter for the Minkowski metric. When p = 1, this is equivalent to using manhattan_distance (l1), and euclidean_distance (l2) for p = 2. For arbitrary p, minkowski_distance (l_p) is used.
metric : string or callable, default 'minkowski'
the distance metric to use for the tree. The default metric is minkowski, and with p=2 is equivalent to the standard Euclidean metric. See the documentation of the DistanceMetric class for a list of available metrics.
metric_params : dict, optional (default = None)
Additional keyword arguments for the metric function.
n_jobs : int, optional (default = 1)
The number of parallel jobs to run for neighbors search.
If ``-1``, then the number of jobs is set to the number of CPU cores.
Doesn't affect :meth:`fit` method.
注意:拟合稀疏输入将覆盖该参数的设置，使用蛮力。
leaf_size: int，可选(默认值为30)
叶大小传递给BallTree或KDTree。这可能会影响构造和查询的速度，以及存储树所需的内存。最优值取决于问题的性质。
p:整数，可选(默认= 2)
Minkowski度规的功率参数。当p = 1时，这相当于在p = 2时使用manhattan_distance (l1)和euclidean_distance (l2)。对于任意p，使用minkowski_distance (l_p)。
度量:字符串或可调用，默认'minkowski'
用于树的距离度量。默认的度量是minkowski, p=2等于标准的欧几里德度量。有关可用指标的列表，请参阅distancem类的文档。
metric_params: dict，可选(默认= None)
度量函数的附加关键字参数。
n_jobs: int，可选(默认值为1)
要为邻居搜索运行的并行作业的数量。
如果' ' -1 ' '，则作业的数量被设置为CPU核心的数量。
不影响:冰毒:'适合'方法。
Examples
--------
>>> X = [[0], [1], [2], [3]]
>>> y = [0, 0, 1, 1]
>>> from sklearn.neighbors import KNeighborsRegressor
>>> neigh = KNeighborsRegressor(n_neighbors=2)
>>> neigh.fit(X, y) # doctest: +ELLIPSIS
KNeighborsRegressor(...)
>>> print(neigh.predict([[1.5]]))
[ 0.5]
See also
--------
NearestNeighbors
RadiusNeighborsRegressor
KNeighborsClassifier
RadiusNeighborsClassifier
例子
--------
>>> X = [[0]， [1]， [2]， [3]]
>>> y = [0, 0, 1, 1]
从sklearn > > >。邻居进口KNeighborsRegressor
>>> neigh = KNeighborsRegressor(n_neighbors=2)
> > >马嘶声。fit(X, y) # doctest: +省略号
KNeighborsRegressor (…)
> > >打印(neigh.predict ([[1.5]]))
[0.5]
另请参阅
--------
NearestNeighbors
RadiusNeighborsRegressor
KNeighborsClassifier
RadiusNeighborsClassifier
Notes
-----
See :ref:`Nearest Neighbors <neighbors>` in the online documentation for a discussion of the choice of ``algorithm`` and ``leaf_size``.
.. warning::
Regarding the Nearest Neighbors algorithms, if it is found that two neighbors, neighbor `k+1` and `k`, have identical distances but different labels, the results will depend on the ordering of the training data.
https://en.wikipedia.org/wiki/K-nearest_neighbor_algorithm
""" 笔记
-----
参见:ref: ' Nearest Neighbors < Neighbors > ' in the online documentation，其中讨论了' '算法' '和' ' leaf_size ' '的选择。
. .警告::
对于最近邻算法，如果发现相邻的‘k+1’和‘k’这两个相邻的距离相同，但是标签不同，那么结果将取决于训练数据的排序。
https://en.wikipedia.org/wiki/K-nearest_neighbor_algorithm
”“”
class KNeighborsRegressor Found at: sklearn.neighbors.regression
class KNeighborsRegressor(NeighborsBase, KNeighborsMixin,
SupervisedFloatMixin,
RegressorMixin):
def __init__(self, n_neighbors=5, weights='uniform',
algorithm='auto', leaf_size=30,
p=2, metric='minkowski', metric_params=None, n_jobs=1, **
kwargs):
self._init_params(n_neighbors=n_neighbors,
algorithm=algorithm, leaf_size=leaf_size, metric=metric, p=p,
metric_params=metric_params, n_jobs=n_jobs, **kwargs)
self.weights = _check_weights(weights)
def predict(self, X):
"""Predict the target for the provided data
Parameters
----------
X : array-like, shape (n_query, n_features), \
or (n_query, n_indexed) if metric == 'precomputed'
Test samples.
Returns
-------
y : array of int, shape = [n_samples] or [n_samples, n_outputs]
Target values
"""
X = check_array(X, accept_sparse='csr')
neigh_dist, neigh_ind = self.kneighbors(X)
weights = _get_weights(neigh_dist, self.weights)
_y = self._y
if _y.ndim == 1:
_y = _y.reshape((-1, 1))
if weights is None:
y_pred = np.mean(_y[neigh_ind], axis=1)
else:
y_pred = np.empty((X.shape[0], _y.shape[1]), dtype=np.
float64)
denom = np.sum(weights, axis=1)
for j in range(_y.shape[1]):
num = np.sum(neigh_indj]_y[ * weights, axis=1)
y_pred[:j] = num / denom
if self._y.ndim == 1:
y_pred = y_pred.ravel()
return y_pred
k最近邻kNN算法的经典案例
1、基础案例
ML之kNN：利用kNN算法对莺尾(Iris)数据集进行多分类预测https://yunyaniu.blog.csdn.net/article/details/87892011
ML之kNN(两种)：基于两种kNN(平均回归、加权回归)对Boston(波士顿房价)数据集(506,13+1)进行价格回归预测并对比各自性能https://yunyaniu.blog.csdn.net/article/details/87913163
CV之kNN：基于ORB提取+kNN检测器、基于SIFT提取+flann检测器的图片相似度可视化https://yunyaniu.blog.csdn.net/article/details/103294339
