背景
某一列是字符串，想要统计该列字符串分词结果后各词出现的词频。
更多、更及时内容欢迎留意微信公众号： 小窗幽记机器学习
示例代码
# -*- coding: utf-8 -*-
# @Time : 2022/2/13 4:18 下午
# @Author : JasonLiu
# @FileName: test.py
import pdb
import pandas as pd
import numpy as np
df = pd.DataFrame(
[[104472, "R.X. Yah & Co"],
[104873, "Big Building Society"],
[109986, "St James's Society"],
[114058, "The Kensington Society Ltd"],
[113438, "MMV Oil Associates Ltd"]], columns=["URN", "Firm_Name"])
# 方法1:
result1 = df.Firm_Name.str.split(expand=True).stack().value_counts()
print("方法1:")
print(result1)
# PS: str.split(expand=True).stack() is a really clever option on small data, but it quickly runs out of memory
# on data of any size. Since it expands out a matrix for every unique word in Firm_Name,
# data sparsity explodes matrix columns without many observations
print("方法2:")
result2 = pd.Series(np.concatenate([x.split() for x in df.Firm_Name])).value_counts()
print(result2)
print("方法3:")
result3 = pd.Series(' '.join(df.Firm_Name).split()).value_counts()
print(result3)
print("方法4:")
temp = df['Firm_Name'].str.cat(sep=' ')
# pdb.set_trace()
from collections import Counter
word_count = Counter(temp.split(' '))
print(word_count)
print("方法5:")
results = Counter()
df['Firm_Name'].str.split().apply(results.update)
print(results)
运行结果如下:
方法4:
Counter({'Society': 3, 'Ltd': 2, 'R.X.': 1, 'Yah': 1, '&': 1, 'Co': 1, 'Big': 1, 'Building': 1, 'St': 1, "James's": 1, 'The': 1, 'Kensington': 1, 'MMV': 1, 'Oil': 1, 'Associates': 1})
方法5:
Counter({'Society': 3, 'Ltd': 2, 'R.X.': 1, 'Yah': 1, '&': 1, 'Co': 1, 'Big': 1, 'Building': 1, 'St': 1, "James's": 1, 'The': 1, 'Kensington': 1, 'MMV': 1, 'Oil': 1, 'Associates': 1})
【更多、更及时内容欢迎留意微信公众号： 小窗幽记机器学习 】
