- 对比:使用
pandas
存储数据 VS 使用写文本 方式存储数据
import pandas as pd import time def pandasWrite(): t0 = time.time() colname = [str(i) for i in range(550)] df = pd.DataFrame(columns=colname) for i in range(100): df.loc[len(df)] = dict(zip(colname, range(550))) t1 = time.time() df.to_csv("temp.csv") print("pandas 存储数据用时:", t1-t0) # print(df) def fileWrite(): t0 = time.time() colname = [str(i) for i in range(550)] with open("temp1.txt", 'w', encoding='utf-8') as f: f.write('\t'.join(x for x in colname)) for i in range(100): f.write('\t'.join(str(x) for x in range(550))+'\n') t1 = time.time() print("写文本 存储数据用时:", t1-t0) pandasWrite() fileWrite()
输出:
pandas 存储数据用时: 4.545027494430542 写文本 存储数据用时: 0.03499293327331543
写文本方式,快了 接近 130 倍
工作当中踩过的坑,浪费了大半天时间,大家注意!