keras的fashion-mnist数据集的源码为：
def load_data(): """Loads the Fashion-MNIST dataset. # Returns Tuple of Numpy arrays: `(x_train, y_train), (x_test, y_test)`. """ dirname = os.path.join('datasets', 'fashion-mnist') base = 'http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/' files = ['train-labels-idx1-ubyte.gz', 'train-images-idx3-ubyte.gz', 't10k-labels-idx1-ubyte.gz', 't10k-images-idx3-ubyte.gz'] paths = [] for fname in files: paths.append(get_file(fname, origin=base + fname, cache_subdir=dirname)) with gzip.open(paths[0], 'rb') as lbpath: y_train = np.frombuffer(lbpath.read(), np.uint8, offset=8) with gzip.open(paths[1], 'rb') as imgpath: x_train = np.frombuffer(imgpath.read(), np.uint8, offset=16).reshape(len(y_train), 28, 28) with gzip.open(paths[2], 'rb') as lbpath: y_test = np.frombuffer(lbpath.read(), np.uint8, offset=8) with gzip.open(paths[3], 'rb') as imgpath: x_test = np.frombuffer(imgpath.read(), np.uint8, offset=16).reshape(len(y_test), 28, 28) return (x_train, y_train), (x_test, y_test) |
fashion-mnist数据集以四个gzip格式的方式存储在远程服务器上，利用keras的get_file()下载到本地的keras缓存目录。
然后利用gzip的open()打开文件，利用numpy的frombuffer方法直接加载numpy的数组。如果是图像数据的话，需要进行reshape操作。
此处，为什么加载图片数据的时候需要offset=16，标签数据的时候需要offset=8？
fashion-mnist图像数据集的预处理方式和mnist有很大的不同，四个gz文件分别存放了x_train, y_train, x_test, y_test四个部分，然后分别读取四个文件利用np.frombuffer()方式加载。这种处理方式相对mnist来说复杂一些。为什么会这样处理？
