fast.ai 深度学习笔记（二）（2）

add_datepart(weather, "Date", drop=False)
add_datepart(googletrend, "Date", drop=False)
add_datepart(train, "Date", drop=False)
add_datepart(test, "Date", drop=False)

m = md.get_learner(
    emb_szs, 
    len(df.columns)-len(cat_vars), 
    0.04, 1,
    [1000,500], 
    [0.001,0.01], 
    y_range=y_range
)
lr = 1e-3

m.fit(lr, 3, metrics=[exp_rmspe])
'''
A Jupyter Widget
[ 0\.       0.02479  0.02205* *0.19309**]                          
[ 1\.       0.02044  0.01751* *0.18301**]                          
[ 2\.       0.01598  0.01571* *0.17248**]*

m.fit(lr, 1, metrics=[exp_rmspe], cycle_len=1)
'''
[ 0\.       0.00676  0.01041  0.09711]* 

learn = ConvLearner.pretrained(arch, data, ps=0., precompute=True)

m = md.get_learner(
    emb_szs, 
    len(df.columns)-len(cat_vars), 
    0.04, 1,
    [1000,500], 
    [0.001,0.01], 
    y_range=y_range
)

md = ColumnarModelData.from_data_frame(
    PATH, val_idx, df, 
    yl.astype(np.float32), 
    cat_flds=cat_vars, 
    bs=128, 
    test_df=df_test
)

m = md.get_learner(
    emb_szs, 
    len(df.columns)-len(cat_vars), 
    0.04, 1,
    [1000,500], 
    [0.001,0.01], 
    y_range=y_range
)

' '.join(md.trn_ds[0].text[:150])
'''
'<cat> csni <summ> the exploitation of mm - wave bands is one of the key - enabler for 5 g mobile \n radio networks . however , the introduction of mm - wave technologies in cellular \n networks is not straightforward due to harsh propagation conditions that limit \n the mm - wave access availability . mm - wave technologies require high - gain antenna \n systems to compensate for high path loss and limited power . as a consequence , \n directional transmissions must be used for cell discovery and synchronization \n processes : this can lead to a non - negligible access delay caused by the \n exploration of the cell area with multiple transmissions along different \n directions . \n    the integration of mm - wave technologies and conventional wireless access \n networks with the objective of speeding up the cell search process requires new \n'
'''

sample_model(m, "<CAT> csni <SUMM> algorithms that")
'''
...use the same network as a single node are not able to achieve the same performance as the traditional network - based routing algorithms . in this paper , we propose a novel routing scheme for routing protocols in wireless networks . the proposed scheme is based ...
'''

sample_model(m, "<CAT> cscv <SUMM> algorithms that")
'''
...use the same data to perform image classification are increasingly being used to improve the performance of image classification algorithms . in this paper , we propose a novel method for image classification using a deep convolutional neural network ( cnn ) . the proposed method is ...
'''

sample_model(m,"<CAT> cscv <SUMM> algorithms. <TITLE> on ")
'''
...the performance of deep learning for image classification <eos>*sample_model(m,"<CAT> csni <SUMM> algorithms. <TITLE> on ")*...the performance of wireless networks <eos>*sample_model(m,"<CAT> cscv <SUMM> algorithms. <TITLE> towards ")*...a new approach to image classification <eos>*sample_model(m,"<CAT> csni <SUMM> algorithms. <TITLE> towards ")*...a new approach to the analysis of wireless networks <eos>
'''

from fastai.learner import *
import torchtext
from torchtext import vocab, data
from torchtext.datasets import language_modeling
from fastai.rnn_reg import *
from fastai.rnn_train import *
from fastai.nlp import *
from fastai.lm_rnn import *
import dill as pickle

PATH = 'data/aclImdb/'
TRN_PATH = 'train/all/'
VAL_PATH = 'test/all/'
TRN = f'{PATH}{TRN_PATH}'
VAL = f'{PATH}{VAL_PATH}'
%ls {PATH}
'''
imdbEr.txt  imdb.vocab  models/  README  test/  tmp/  train/
'''

trn_files = !ls {TRN}
trn_files[:10]
'''
['0_0.txt',
 '0_3.txt',
 '0_9.txt',
 '10000_0.txt',
 '10000_4.txt',
 '10000_8.txt',
 '1000_0.txt',
 '10001_0.txt',
 '10001_10.txt',
 '10001_4.txt']
'''
review = !cat {TRN}{trn_files[6]}
review[0]
'''
"I have to say when a name like Zombiegeddon and an atom bomb on the front cover I was expecting a flat out chop-socky fung-ku, but what I got instead was a comedy. So, it wasn't quite was I was expecting, but I really liked it anyway! The best scene ever was the main cop dude pulling those kids over and pulling a Bad Lieutenant on them!! I was laughing my ass off. I mean, the cops were just so bad! And when I say bad, I mean The Shield Vic Macky bad. But unlike that show I was laughing when they shot people and smoked dope.<br /><br />Felissa Rose...man, oh man. What can you say about that hottie. She was great and put those other actresses to shame. She should work more often!!!!! I also really liked the fight scene outside of the building. That was done really well. Lots of fighting and people getting their heads banged up. FUN! Last, but not least Joe Estevez and William Smith were great as the...well, I wasn't sure what they were, but they seemed to be having fun and throwing out lines. I mean, some of it didn't make sense with the rest of the flick, but who cares when you're laughing so hard! All in all the film wasn't the greatest thing since sliced bread, but I wasn't expecting that. It was a Troma flick so I figured it would totally suck. It's nice when something surprises you but not totally sucking.<br /><br />Rent it if you want to get stoned on a Friday night and laugh with your buddies. Don't rent it if you are an uptight weenie or want a zombie movie with lots of flesh eating.<br /><br />P.S. Uwe Boil was a nice touch."
'''

!find {TRN} -name '*.txt' | xargs cat | wc -w
'''
17486581
'''
!find {VAL} -name '*.txt' | xargs cat | wc -w
'''
5686719
'''

TEXT = data.Field(lower=True, tokenize=spacy_tok)

bs=64; bptt=70
FILES = dict(train=TRN_PATH, validation=VAL_PATH, test=VAL_PATH)
md = LanguageModelData.from_text_files(
    PATH, TEXT, **FILES, bs=bs, 
    bptt=bptt, min_freq=10
)

# 'itos': 'int-to-string' 
TEXT.vocab.itos[:12]
'''
['<unk>', '<pad>', 'the', ',', '.', 'and', 'a', 'of', 'to', 'is', 'it', 'in']
'''
# 'stoi': 'string to int'
TEXT.vocab.stoi['the']
'''
2
'''

md.trn_ds[0].text[:12]
'''
['i',
 'have',
 'always',
 'loved',
 'this',
 'story',
 '-',
 'the',
 'hopeful',
 'theme',
 ',',
 'the']
'''
TEXT.numericalize([md.trn_ds[0].text[:12]])
'''
Variable containing:
   12
   35
  227
  480
   13
   76
   17
    2
 7319
  769
    3
    2
[torch.cuda.LongTensor of size 12x1 (GPU 0)]
'''