sklearn库中的ShuffleSplit()函数和StratifiedShuffleSplit()函数的讲解
from sklearn.model_selection import ShuffleSplit,StratifiedShuffleSplit
这两个函数均是实现了对数据集进行打乱划分，即在数据集在进行划分之前，先进行打乱操作，否则容易产生过拟合，模型泛化能力下降。其中，StratifiedShuffleSplit函数是StratifiedKFold和ShuffleSplit的合并，它将返回StratifiedKFold。折叠是通过保存每个类的样本百分比来实现的。
首先将样本随机打乱，然后根据设置参数划分出train/test对。通过n_splits产生指定数量的独立的【train/test】数据集，划分数据集划分成n组(n组索引值)，其创建的每一组划分将保证每组类比的比例相同。比如第一组训练数据类别比例为2:1，则后面每组类别都满足这个比例。
ShuffleSplit()函数
cv_split = ShuffleSplit(n_splits=6, train_size=0.7, test_size=0.2)
class ShuffleSplit(BaseShuffleSplit):
"""Random permutation cross-validator
Yields indices to split data into training and test sets.
Note: contrary to other cross-validation strategies, random splits do not guarantee that all folds will be different, although this is still very likely for sizeable datasets.
Read more in the :ref:`User Guide <cross_validation>`.
Parameters
----------
n_splits : int, default=10. Number of re-shuffling & splitting iterations.
test_size : float or int, default=None. If float, should be between 0.0 and 1.0 and represent the proportion of the dataset to include in the test split. If int, represents the absolute number of test samples. If None, the value is set to the complement of the train size. If ``train_size`` is also None, it will be set to 0.1.
train_size : float or int, default=None. If float, should be between 0.0 and 1.0 and represent the proportion of the dataset to include in the train split. If int, represents the absolute number of train samples. If None, the value is automatically set to the complement of the test size.
random_state : int or RandomState instance, default=None. Controls the randomness of the training and testing indices produced. Pass an int for reproducible output across multiple function calls.
See :term:`Glossary <random_state>`.
类ShuffleSplit (BaseShuffleSplit):
随机排列交叉验证
生成将数据分割为训练集和测试集的索引。
注：与其他交叉验证策略相反，随机分割并不能保证所有的折叠都是不同的，尽管对于较大的数据集，这种情况仍然很可能发生。
更多信息请参见:ref: ' User Guide <cross_validation> '。</cross_validation>
参数
----------
n_splits : int，默认=10。重新洗牌和分裂迭代的数量。将训练数据分成【train/test】对的组数。
test_size: float或int，默认=None。如果是浮动的，则应该在0.0和1.0之间，并表示要包含在test分割中的数据集的比例。如果int，表示测试样本的绝对数量。如果没有，则将该值设置为train_size的补集。如果train_size也是None，它将被设置为0.1。
test_size用来设置【train/test】对中test所占的比例。
train_size: float或int，默认=None。如果是浮点数，则应该在0.0和1.0之间，并表示要包含在train分割序列中的数据集的比例。如果int，表示train样本的绝对数量。如果没有，该值将自动设置为train size的补集。train_size用来设置【train/test】对中train所占的比例。
random_state: int或RandomState实例，默认为None。控制产生的训练和测试指标的随机性。在多个函数调用之间传递可重复输出的int。
控制将样本随机打乱，用于随机抽样的伪随机数发生器状态。
看:术语:“术语表< random_state >”。
Examples
--------
>>> import numpy as np
>>> from sklearn.model_selection import ShuffleSplit
>>> X = np.array([[1, 2], [3, 4], [5, 6], [7, 8], [3, 4], [5, 6]])
>>> y = np.array([1, 2, 1, 2, 1, 2])
>>> rs = ShuffleSplit(n_splits=5, test_size=.25, random_state=0)
>>> rs.get_n_splits(X)
5
>>> print(rs)
ShuffleSplit(n_splits=5, random_state=0, test_size=0.25, train_size=None)
>>> for train_index, test_index in rs.split(X):
... print("TRAIN:", train_index, "TEST:", test_index)
TRAIN: [1 3 0 4] TEST: [5 2]
TRAIN: [4 0 2 5] TEST: [1 3]
TRAIN: [1 2 4 0] TEST: [3 5]
TRAIN: [3 4 1 0] TEST: [5 2]
TRAIN: [3 5 1 0] TEST: [2 4]
>>> rs = ShuffleSplit(n_splits=5, train_size=0.5, test_size=.25, random_state=0)
>>> for train_index, test_index in rs.split(X):
... print("TRAIN:", train_index, "TEST:", test_index)
TRAIN: [1 3 0] TEST: [5 2]
TRAIN: [4 0 2] TEST: [1 3]
TRAIN: [1 2 4] TEST: [3 5]
TRAIN: [3 4 1] TEST: [5 2]
TRAIN: [3 5 1] TEST: [2 4]
"""
@_deprecate_positional_args
def __init__(self, n_splits=10, *, test_size=None, train_size=None,
random_state=None):
super().__init__(n_splits=n_splits, test_size=test_size, train_size=train_size, random_state=random_state)
self._default_test_size = 0.1
def _iter_indices(self, X, y=None, groups=None):
n_samples = _num_samples(X)
n_train, n_test = _validate_shuffle_split(
n_samples, self.test_size, self.train_size,
default_test_size=self._default_test_size)
rng = check_random_state(self.random_state)
for i in range(self.n_splits):
# random partition
permutation = rng.permutation(n_samples)
ind_test = permutation[:n_test]
ind_train = permutation[n_test:n_test + n_train]
yield ind_train, ind_test
Examples
--------
>>> import numpy as np
>>> from sklearn.model_selection import ShuffleSplit
>>> X = np.array([[1, 2], [3, 4], [5, 6], [7, 8], [3, 4], [5, 6]])
>>> y = np.array([1, 2, 1, 2, 1, 2])
>>> rs = ShuffleSplit(n_splits=5, test_size=.25, random_state=0)
>>> rs.get_n_splits(X)
5
>>> print(rs)
ShuffleSplit(n_splits=5, random_state=0, test_size=0.25, train_size=None)
>>> for train_index, test_index in rs.split(X):
... print("TRAIN:", train_index, "TEST:", test_index)
TRAIN: [1 3 0 4] TEST: [5 2]
TRAIN: [4 0 2 5] TEST: [1 3]
TRAIN: [1 2 4 0] TEST: [3 5]
TRAIN: [3 4 1 0] TEST: [5 2]
TRAIN: [3 5 1 0] TEST: [2 4]
>>> rs = ShuffleSplit(n_splits=5, train_size=0.5, test_size=.25, random_state=0)
>>> for train_index, test_index in rs.split(X):
... print("TRAIN:", train_index, "TEST:", test_index)
TRAIN: [1 3 0] TEST: [5 2]
TRAIN: [4 0 2] TEST: [1 3]
TRAIN: [1 2 4] TEST: [3 5]
TRAIN: [3 4 1] TEST: [5 2]
TRAIN: [3 5 1] TEST: [2 4]
"""
@_deprecate_positional_args
def __init__(self, n_splits=10, *, test_size=None, train_size=None,
random_state=None):
super().__init__(n_splits=n_splits, test_size=test_size, train_size=train_size, random_state=random_state)
self._default_test_size = 0.1
def _iter_indices(self, X, y=None, groups=None):
n_samples = _num_samples(X)
n_train, n_test = _validate_shuffle_split(
n_samples, self.test_size, self.train_size,
default_test_size=self._default_test_size)
rng = check_random_state(self.random_state)
for i in range(self.n_splits):
# random partition
permutation = rng.permutation(n_samples)
ind_test = permutation[:n_test]
ind_train = permutation[n_test:n_test + n_train]
yield ind_train, ind_test
