# word2vec深入浅出，保证你一看就会（5）【完结】

$$F(w_i,w_j,w_k)=P(i|k)/P(j|k)$$

$$F((w_i-w_j)^Tw_k)=P(i|k)/P(j|k)$$

$$F((w_i-w_j)^Tw_k)=F(w_i^Tw_k)/F(w_j^Tw_k)$$

$$F(w_i^Tw_k)=P(i|k)$$

$$w_i^Tw_k+b_i+b_k=log(X_{ik})$$

## Glove更新计算量和调优

$$LOSS=\sum^V_{i,k=1} f(*)(w_i^Tw_k+b_i+b_k-log(X_{ik}))^2$$

$$w_i^Tw_k+b_i+b_k=log(X_{ik})$$

$$LOSS= f(*)(w_i^Tw_k+b_i+b_k-log(X_{ik}))^2$$

## 代码解析

$$w_i^{new}=w_i^{old}-η*输出层错误*w_k$$

$$w_k^{new}=w_k^{old}-η*输出层错误*w_i$$

$$b_k^{new}=b_k^{old}-η*输出层错误$$

$$2 * f() * (w_i^Tw_k+b_i+b_k-log(X_{ik}))$$

fdiff *= eta; // for ease in calculating gradient
for(b = 0; b < vector_size; b++) {
// learning rate times gradient for word vectors
temp1 = fdiff * W[b + l2];
temp2 = fdiff * W[b + l1];
W[b + l1] -= temp1 / sqrt(gradsq[b + l1]);
W[b + l2] -= temp2 / sqrt(gradsq[b + l2]);
gradsq[b + l1] += temp1 * temp1;
gradsq[b + l2] += temp2 * temp2;
}
W[vector_size + l1] -= fdiff / sqrt(gradsq[vector_size + l1]);
W[vector_size + l2] -= fdiff / sqrt(gradsq[vector_size + l2]);
fdiff *= fdiff;

temp1 = fdiff * W[b + l2];
temp2 = fdiff * W[b + l1];
W[b + l1] -= temp1 / sqrt(gradsq[b + l1]);
W[b + l2] -= temp2 / sqrt(gradsq[b + l2]);

3223 0

4559 0

8052 0

9477 0
word2vec深入浅出，保证你一看就会（1）

7137 0
word2vec深入浅出，保证你一看就会（5）【完结】

3015 0

657 0

4063 0
+关注
6

0

《SaaS模式云原生数据仓库应用场景实践》

《看见新力量：二》电子书