COVID-19 Cases Prediction (Regression)(二)

简介: COVID-19 Cases Prediction (Regression)

Feature Selection

Choose features you deem useful by modifying the function below.

def select_feat(train_data, valid_data, test_data, select_all=True):
    '''Selects useful features to perform regression'''
    y_train, y_valid = train_data[:,-1], valid_data[:,-1]
    raw_x_train, raw_x_valid, raw_x_test = train_data[:,:-1], valid_data[:,:-1], test_data
    if select_all:
        feat_idx = list(range(raw_x_train.shape[1]))
    else:
        feat_idx = [0,1,2,3,4] # TODO: Select suitable feature columns.
    return raw_x_train[:,feat_idx], raw_x_valid[:,feat_idx], raw_x_test[:,feat_idx], y_train, y_valid

Training Loop

def trainer(train_loader, valid_loader, model, config, device):
    criterion = nn.MSELoss(reduction='mean') # Define your loss function, do not modify this.
    # Define your optimization algorithm. 
    # TODO: Please check https://pytorch.org/docs/stable/optim.html to get more available algorithms.
    # TODO: L2 regularization (optimizer(weight decay...) or implement by your self).
    optimizer = torch.optim.SGD(model.parameters(), lr=config['learning_rate'], momentum=0.9) 
    writer = SummaryWriter() # Writer of tensoboard.
    if not os.path.isdir('./models'):
        os.mkdir('./models') # Create directory of saving models.
    n_epochs, best_loss, step, early_stop_count = config['n_epochs'], math.inf, 0, 0
    for epoch in range(n_epochs):
        model.train() # Set your model to train mode.
        loss_record = []
        # tqdm is a package to visualize your training progress.
        train_pbar = tqdm(train_loader, position=0, leave=True)
        for x, y in train_pbar:
            optimizer.zero_grad()               # Set gradient to zero.
            x, y = x.to(device), y.to(device)   # Move your data to device. 
            pred = model(x)             
            loss = criterion(pred, y)
            loss.backward()                     # Compute gradient(backpropagation).
            optimizer.step()                    # Update parameters.
            step += 1
            loss_record.append(loss.detach().item())
            # Display current epoch number and loss on tqdm progress bar.
            train_pbar.set_description(f'Epoch [{epoch+1}/{n_epochs}]')
            train_pbar.set_postfix({'loss': loss.detach().item()})
        mean_train_loss = sum(loss_record)/len(loss_record)
        writer.add_scalar('Loss/train', mean_train_loss, step)
        model.eval() # Set your model to evaluation mode.
        loss_record = []
        for x, y in valid_loader:
            x, y = x.to(device), y.to(device)
            with torch.no_grad():
                pred = model(x)
                loss = criterion(pred, y)
            loss_record.append(loss.item())
        mean_valid_loss = sum(loss_record)/len(loss_record)
        print(f'Epoch [{epoch+1}/{n_epochs}]: Train loss: {mean_train_loss:.4f}, Valid loss: {mean_valid_loss:.4f}')
        writer.add_scalar('Loss/valid', mean_valid_loss, step)
        if mean_valid_loss < best_loss:
            best_loss = mean_valid_loss
            torch.save(model.state_dict(), config['save_path']) # Save your best model
            print('Saving model with loss {:.3f}...'.format(best_loss))
            early_stop_count = 0
        else: 
            early_stop_count += 1
        if early_stop_count >= config['early_stop']:
            print('\nModel is not improving, so we halt the training session.')
            return

Configurations

config contains hyper-parameters for training and the path to save your model.

device = 'cuda' if torch.cuda.is_available() else 'cpu'
config = {
    'seed': 5201314,      # Your seed number, you can pick your lucky number. :)
    'select_all': True,   # Whether to use all features.
    'valid_ratio': 0.2,   # validation_size = train_size * valid_ratio
    'n_epochs': 3000,     # Number of epochs.            
    'batch_size': 256, 
    'learning_rate': 1e-5,              
    'early_stop': 400,    # If model has not improved for this many consecutive epochs, stop training.     
    'save_path': './models/model.ckpt'  # Your model will be saved here.
}

Dataloader

Read data from files and set up training, validation, and testing sets. You do not need to modify this part.

# Set seed for reproducibility
same_seed(config['seed'])
# train_data size: 2699 x 118 (id + 37 states + 16 features x 5 days) 
# test_data size: 1078 x 117 (without last day's positive rate)
train_data, test_data = pd.read_csv('./covid.train.csv').values, pd.read_csv('./covid.test.csv').values
train_data, valid_data = train_valid_split(train_data, config['valid_ratio'], config['seed'])
# Print out the data size.
print(f"""train_data size: {train_data.shape} 
valid_data size: {valid_data.shape} 
test_data size: {test_data.shape}""")
# Select features
x_train, x_valid, x_test, y_train, y_valid = select_feat(train_data, valid_data, test_data, config['select_all'])
# Print out the number of features.
print(f'number of features: {x_train.shape[1]}')
train_dataset, valid_dataset, test_dataset = COVID19Dataset(x_train, y_train), \
                                            COVID19Dataset(x_valid, y_valid), \
                                            COVID19Dataset(x_test)
# Pytorch data loader loads pytorch dataset into batches.
train_loader = DataLoader(train_dataset, batch_size=config['batch_size'], shuffle=True, pin_memory=True)
valid_loader = DataLoader(valid_dataset, batch_size=config['batch_size'], shuffle=True, pin_memory=True)
test_loader = DataLoader(test_dataset, batch_size=config['batch_size'], shuffle=False, pin_memory=True)

Start training!

it may take lots of time(depends on GPU you drew),be sure stay front your computer or get some scripts or devices to make your screen stay light.

image.png

model = My_Model(input_dim=x_train.shape[1]).to(device) # put your model and data on the same computation device.
trainer(train_loader, valid_loader, model, config, device)

and if you never modify any above code,it may train 1883 times

11.png

Plot learning curves with tensorboard (optional)

tensorboard is a tool that allows you to visualize your training progress.If this block does not display your learning curve, please wait for few minutes, and re-run this block.

It might take some time to load your logging information.

%reload_ext tensorboard
%tensorboard --logdir=./runs/

you will get a picture like this.

image.png

Testing

The predictions of your model on testing set will be stored at pred.csv.

def save_pred(preds, file):
    ''' Save predictions to specified file '''
    with open(file, 'w') as fp:
        writer = csv.writer(fp)
        writer.writerow(['id', 'tested_positive'])
        for i, p in enumerate(preds):
            writer.writerow([i, p])
model = My_Model(input_dim=x_train.shape[1]).to(device)
model.load_state_dict(torch.load(config['save_path']))
preds = predict(test_loader, model, device) 
save_pred(preds, 'pred.csv')         
100%|██████████| 5/5 [00:00<00:00, 554.88it/s]

after these cells,you will get a pred.csv in the files

image.png

you can download this doc and submit it in kaggle

image.png

and the kaggle will give you a score,if you never modify,you may have a low score like this:13.png

Some optimization

if you work above code,you will pass the Simple Baseline. About how can pass Medium Baseline &Strong Baseline, i will try to give the answer in the future(maybe in 09/2022), teaching assistant gave some hints:

  • 特征选择(Feature selection-what other features are useful?)
  • DNN结构:层数,维度,激活函数(DNN construction-layers, dimension, activation function)
  • 训练(training-mini batch, optimizer, leaning rate)
  • L2 regularization

Reference

all the code was from HUNG-Yi LEE(李宏毅),you can study the 《MACHINE LEARNING 2022SPRING》 in https://speech.ee.ntu.edu.tw/~hylee/ml/2022-spring.php


above all was my study note, if you have any suggestions, welcome to comment.

目录
相关文章
|
存储 Prometheus 监控
Prometheus vs. ELK Stack:容器监控与日志管理工具的较量
随着容器化技术的广泛应用,容器监控与日志管理成为了关键任务。本文将对两种常用工具进行比较与选择,分别是Prometheus和ELK Stack。Prometheus是一款开源的监控系统,专注于时序数据的收集和告警。而ELK Stack则是一套完整的日志管理解决方案,由Elasticsearch、Logstash和Kibana三个组件组成。通过比较它们的特点、优势和适用场景,读者可以更好地了解如何选择适合自己需求的工具。
|
机器学习/深度学习 自然语言处理 数据处理
论文《Attention is All You Need》
论文《Attention is All You Need》
1087 1
|
3月前
|
Ubuntu 安全
修复Ubuntu系统文件损坏问题:手动fsck指令
总结一下,使用 `fsck`修复文件系统是一个复杂的过程,涉及到诊断问题、安全卸载文件系统、执行检查与修复、并且通常在维护模式下完成。这个过程对于维护系统的健康和稳定至关重要。虽然命令和步骤可以按照标准手册执行,但每个系统的具体情况可能都有所不同,因此修复操作需要对系统和维护命令有所了解。在执行这一系列操作时请确保谨慎,并对系统有一定的认识,以避免不必要的数据丢失。
622 8
|
搜索推荐 算法 前端开发
美食物管理与推荐系统Python+Django网站开发+协同过滤推荐算法应用【计算机课设项目推荐】
美食物管理与推荐系统Python+Django网站开发+协同过滤推荐算法应用【计算机课设项目推荐】
424 4
美食物管理与推荐系统Python+Django网站开发+协同过滤推荐算法应用【计算机课设项目推荐】
|
11月前
|
机器学习/深度学习 存储 人工智能
白话文讲解大模型| Attention is all you need
本文档旨在详细阐述当前主流的大模型技术架构如Transformer架构。我们将从技术概述、架构介绍到具体模型实现等多个角度进行讲解。通过本文档,我们期望为读者提供一个全面的理解,帮助大家掌握大模型的工作原理,增强与客户沟通的技术基础。本文档适合对大模型感兴趣的人员阅读。
1659 121
白话文讲解大模型| Attention is all you need
|
10月前
|
PyTorch Shell API
Ascend Extension for PyTorch的源码解析
本文介绍了Ascend对PyTorch代码的适配过程,包括源码下载、编译步骤及常见问题,详细解析了torch-npu编译后的文件结构和三种实现昇腾NPU算子调用的方式:通过torch的register方式、定义算子方式和API重定向映射方式。这对于开发者理解和使用Ascend平台上的PyTorch具有重要指导意义。
|
编译器 开发工具 C++
【Python】已解决error: Microsoft Visual C++ 14.0 or greater is required. Get it with “Microsoft C++ Build
【Python】已解决error: Microsoft Visual C++ 14.0 or greater is required. Get it with “Microsoft C++ Build
6694 0
|
消息中间件 JSON Java
Springboot支付宝沙箱支付---完整详细步骤
Springboot支付宝沙箱支付---完整详细步骤
2659 1
|
弹性计算 前端开发 Java
通义千问API:让大模型写代码和跑代码
基于前面三章的铺垫,本章我们将展示大模型Agent的强大能力。我们不仅要实现让大模型同时使用多种查询工具,还要实现让大模型能查询天气情况,最后让大模型自己写代码来查询天气情况。
通义千问API:让大模型写代码和跑代码
|
Ubuntu Go 开发工具
Unbuntu 一键安装制定版本的Go环境
该文章提供了在Ubuntu系统上一键安装指定版本的Go语言环境的详细步骤,包括删除旧版本Go、下载并解压新版本Go到指定目录、配置Go环境变量,以及安装其他必要的开发工具。
563 2