ML之回归预测：利用两种机器学习算法(LiR，XGBoost(调优+重要性可视化+特征选择模型))对无人驾驶汽车系统参数(2017年的data,18+2)进行回归预测值VS真实值

输出结果

1、LiR模型

LiR：The value of default measurement of LiR is 0.8729775261968014

LiR：R-squared value of DecisionTreeRegressor: 0.8729775261968014

2、XGBoost模型

ML之XGBoost：XGBoost参数调优之经验总结——DIY十多个案例

T1、调用XGBR_GSCV_Shuffle()函数，调优+重要性可视化+特征选择模型

   XGBR_model = XGBRegressor( learning_rate=0.06, max_depth= 4, n_estimators=100 )    #XGBR_GSCV_Shuffle()函数，第一次得到最佳参数组合，输出准确度： 0.9312586298921468

   XGBR_model = XGBRegressor( learning_rate=0.15, max_depth= 4, n_estimators=100 )    #XGBR_GSCV_Shuffle()函数，第二次得到最佳参数组合，输出准确度： 0.9361222829659452

   XGBR_model = XGBRegressor( learning_rate=0.03, max_depth= 5, n_estimators=200 )    #XGBR_GSCV_Shuffle()函数，第三次得到最佳参数组合，输出准确度： 0.9335316602435876

Best: -7.044124 using {'learning_rate': 0.03, 'max_depth': 5, 'n_estimators': 200}

XGBR_GSCV_Shuffle score: 2.65407690407428

-13.263615 (3.437888) with: {'learning_rate': 0.03, 'max_depth': 4, 'n_estimators': 100}

-7.085101 (5.544846) with: {'learning_rate': 0.03, 'max_depth': 4, 'n_estimators': 200}

-13.266334 (3.458229) with: {'learning_rate': 0.03, 'max_depth': 5, 'n_estimators': 100}

-7.044124 (5.800332) with: {'learning_rate': 0.03, 'max_depth': 5, 'n_estimators': 200}

-13.379665 (3.515279) with: {'learning_rate': 0.03, 'max_depth': 6, 'n_estimators': 100}

-7.185696 (5.878527) with: {'learning_rate': 0.03, 'max_depth': 6, 'n_estimators': 200}

-13.479146 (3.629065) with: {'learning_rate': 0.03, 'max_depth': 7, 'n_estimators': 100}

-7.324944 (5.854973) with: {'learning_rate': 0.03, 'max_depth': 7, 'n_estimators': 200}

-7.143094 (5.637506) with: {'learning_rate': 0.06, 'max_depth': 4, 'n_estimators': 100}

-7.593377 (6.216784) with: {'learning_rate': 0.06, 'max_depth': 4, 'n_estimators': 200}

-7.098928 (5.750214) with: {'learning_rate': 0.06, 'max_depth': 5, 'n_estimators': 100}

-7.597613 (6.403983) with: {'learning_rate': 0.06, 'max_depth': 5, 'n_estimators': 200}

-7.210929 (5.854905) with: {'learning_rate': 0.06, 'max_depth': 6, 'n_estimators': 100}

-7.759291 (6.498452) with: {'learning_rate': 0.06, 'max_depth': 6, 'n_estimators': 200}

-7.348396 (5.867050) with: {'learning_rate': 0.06, 'max_depth': 7, 'n_estimators': 100}

-7.914092 (6.526464) with: {'learning_rate': 0.06, 'max_depth': 7, 'n_estimators': 200}

-7.514619 (6.090178) with: {'learning_rate': 0.09, 'max_depth': 4, 'n_estimators': 100}

-7.792390 (6.343156) with: {'learning_rate': 0.09, 'max_depth': 4, 'n_estimators': 200}

-7.506378 (6.281410) with: {'learning_rate': 0.09, 'max_depth': 5, 'n_estimators': 100}

-7.757921 (6.477667) with: {'learning_rate': 0.09, 'max_depth': 5, 'n_estimators': 200}

-7.626987 (6.321250) with: {'learning_rate': 0.09, 'max_depth': 6, 'n_estimators': 100}

-7.830667 (6.496410) with: {'learning_rate': 0.09, 'max_depth': 6, 'n_estimators': 200}

-7.873006 (6.432751) with: {'learning_rate': 0.09, 'max_depth': 7, 'n_estimators': 100}

-8.036536 (6.584526) with: {'learning_rate': 0.09, 'max_depth': 7, 'n_estimators': 200}

-7.672704 (6.222572) with: {'learning_rate': 0.12000000000000001, 'max_depth': 4, 'n_estimators': 100}

-7.916448 (6.418164) with: {'learning_rate': 0.12000000000000001, 'max_depth': 4, 'n_estimators': 200}

-7.724868 (6.419296) with: {'learning_rate': 0.12000000000000001, 'max_depth': 5, 'n_estimators': 100}

-7.893062 (6.541605) with: {'learning_rate': 0.12000000000000001, 'max_depth': 5, 'n_estimators': 200}

-7.849538 (6.506693) with: {'learning_rate': 0.12000000000000001, 'max_depth': 6, 'n_estimators': 100}

-7.949133 (6.580036) with: {'learning_rate': 0.12000000000000001, 'max_depth': 6, 'n_estimators': 200}

-8.021275 (6.522834) with: {'learning_rate': 0.12000000000000001, 'max_depth': 7, 'n_estimators': 100}

-8.115124 (6.590436) with: {'learning_rate': 0.12000000000000001, 'max_depth': 7, 'n_estimators': 200}

-7.851446 (6.380559) with: {'learning_rate': 0.15000000000000002, 'max_depth': 4, 'n_estimators': 100}

-7.962357 (6.452926) with: {'learning_rate': 0.15000000000000002, 'max_depth': 4, 'n_estimators': 200}

-7.752629 (6.526754) with: {'learning_rate': 0.15000000000000002, 'max_depth': 5, 'n_estimators': 100}

-7.870802 (6.591447) with: {'learning_rate': 0.15000000000000002, 'max_depth': 5, 'n_estimators': 200}

-7.828501 (6.482895) with: {'learning_rate': 0.15000000000000002, 'max_depth': 6, 'n_estimators': 100}

-7.892413 (6.509397) with: {'learning_rate': 0.15000000000000002, 'max_depth': 6, 'n_estimators': 200}

-8.141324 (6.636931) with: {'learning_rate': 0.15000000000000002, 'max_depth': 7, 'n_estimators': 100}

-8.182099 (6.635204) with: {'learning_rate': 0.15000000000000002, 'max_depth': 7, 'n_estimators': 200}

-7.938719 (6.490107) with: {'learning_rate': 0.18000000000000002, 'max_depth': 4, 'n_estimators': 100}

-8.017980 (6.506082) with: {'learning_rate': 0.18000000000000002, 'max_depth': 4, 'n_estimators': 200}

-7.938695 (6.610782) with: {'learning_rate': 0.18000000000000002, 'max_depth': 5, 'n_estimators': 100}

-8.012643 (6.660132) with: {'learning_rate': 0.18000000000000002, 'max_depth': 5, 'n_estimators': 200}

-8.011816 (6.616109) with: {'learning_rate': 0.18000000000000002, 'max_depth': 6, 'n_estimators': 100}

-8.052129 (6.641090) with: {'learning_rate': 0.18000000000000002, 'max_depth': 6, 'n_estimators': 200}

-8.118405 (6.560621) with: {'learning_rate': 0.18000000000000002, 'max_depth': 7, 'n_estimators': 100}

-8.131590 (6.550569) with: {'learning_rate': 0.18000000000000002, 'max_depth': 7, 'n_estimators': 200}

-7.915589 (6.338897) with: {'learning_rate': 0.21000000000000002, 'max_depth': 4, 'n_estimators': 100}

-8.019436 (6.383854) with: {'learning_rate': 0.21000000000000002, 'max_depth': 4, 'n_estimators': 200}

-7.956674 (6.487618) with: {'learning_rate': 0.21000000000000002, 'max_depth': 5, 'n_estimators': 100}

-8.028267 (6.514906) with: {'learning_rate': 0.21000000000000002, 'max_depth': 5, 'n_estimators': 200}

-8.036983 (6.583115) with: {'learning_rate': 0.21000000000000002, 'max_depth': 6, 'n_estimators': 100}

-8.085323 (6.596389) with: {'learning_rate': 0.21000000000000002, 'max_depth': 6, 'n_estimators': 200}

-8.254193 (6.565100) with: {'learning_rate': 0.21000000000000002, 'max_depth': 7, 'n_estimators': 100}

-8.269231 (6.561241) with: {'learning_rate': 0.21000000000000002, 'max_depth': 7, 'n_estimators': 200}

-8.143765 (6.593441) with: {'learning_rate': 0.24000000000000002, 'max_depth': 4, 'n_estimators': 100}

-8.218321 (6.600359) with: {'learning_rate': 0.24000000000000002, 'max_depth': 4, 'n_estimators': 200}

-8.191637 (6.690425) with: {'learning_rate': 0.24000000000000002, 'max_depth': 5, 'n_estimators': 100}

-8.222861 (6.676068) with: {'learning_rate': 0.24000000000000002, 'max_depth': 5, 'n_estimators': 200}

-8.230726 (6.661499) with: {'learning_rate': 0.24000000000000002, 'max_depth': 6, 'n_estimators': 100}

-8.260381 (6.658228) with: {'learning_rate': 0.24000000000000002, 'max_depth': 6, 'n_estimators': 200}

-8.470876 (6.728413) with: {'learning_rate': 0.24000000000000002, 'max_depth': 7, 'n_estimators': 100}

-8.480391 (6.731554) with: {'learning_rate': 0.24000000000000002, 'max_depth': 7, 'n_estimators': 200}

-7.967612 (6.650860) with: {'learning_rate': 0.27, 'max_depth': 4, 'n_estimators': 100}

-8.051922 (6.656565) with: {'learning_rate': 0.27, 'max_depth': 4, 'n_estimators': 200}

-8.121717 (6.575363) with: {'learning_rate': 0.27, 'max_depth': 5, 'n_estimators': 100}

-8.160381 (6.577406) with: {'learning_rate': 0.27, 'max_depth': 5, 'n_estimators': 200}

-8.070251 (6.545575) with: {'learning_rate': 0.27, 'max_depth': 6, 'n_estimators': 100}

-8.099537 (6.546567) with: {'learning_rate': 0.27, 'max_depth': 6, 'n_estimators': 200}

-8.128970 (6.796862) with: {'learning_rate': 0.27, 'max_depth': 7, 'n_estimators': 100}

-8.138011 (6.797582) with: {'learning_rate': 0.27, 'max_depth': 7, 'n_estimators': 200}

-8.291199 (6.426882) with: {'learning_rate': 0.3, 'max_depth': 4, 'n_estimators': 100}

-8.336647 (6.448367) with: {'learning_rate': 0.3, 'max_depth': 4, 'n_estimators': 200}

-8.148781 (6.531570) with: {'learning_rate': 0.3, 'max_depth': 5, 'n_estimators': 100}

-8.181453 (6.528953) with: {'learning_rate': 0.3, 'max_depth': 5, 'n_estimators': 200}

-8.488194 (6.555250) with: {'learning_rate': 0.3, 'max_depth': 6, 'n_estimators': 100}

-8.499785 (6.552571) with: {'learning_rate': 0.3, 'max_depth': 6, 'n_estimators': 200}

-8.432480 (6.699986) with: {'learning_rate': 0.3, 'max_depth': 7, 'n_estimators': 100}

-8.437643 (6.698732) with: {'learning_rate': 0.3, 'max_depth': 7, 'n_estimators': 200}

XGBR_GSCV_Shuffle_time： 137.92240889430929

T2、调用XGBR_GSCV_Time()函数，调优+重要性可视化+特征选择模型

   XGBR_model = XGBRegressor( learning_rate=0.2, max_depth= 2, n_estimators=100 )     #XGBR_GSCV_Time()函数，第一次得到最佳参数组合，输出准确度： 0.929254087319193

Best: 0.8637 using {'learning_rate': 0.2, 'max_depth': 2, 'n_estimators': 100}

XGBR_GSCV_Time score： 0.92951600308978

-53.652829 (8.524886) with: {'learning_rate': 0.0001, 'max_depth': 2, 'n_estimators': 50}

-53.111776 (8.432226) with: {'learning_rate': 0.0001, 'max_depth': 2, 'n_estimators': 100}

-52.045728 (8.249767) with: {'learning_rate': 0.0001, 'max_depth': 2, 'n_estimators': 200}

-49.976399 (7.896034) with: {'learning_rate': 0.0001, 'max_depth': 2, 'n_estimators': 400}

-53.652829 (8.524886) with: {'learning_rate': 0.0001, 'max_depth': 4, 'n_estimators': 50}

-53.111776 (8.432226) with: {'learning_rate': 0.0001, 'max_depth': 4, 'n_estimators': 100}

-52.045728 (8.249767) with: {'learning_rate': 0.0001, 'max_depth': 4, 'n_estimators': 200}

-49.976399 (7.896034) with: {'learning_rate': 0.0001, 'max_depth': 4, 'n_estimators': 400}

-53.652829 (8.524886) with: {'learning_rate': 0.0001, 'max_depth': 6, 'n_estimators': 50}

-53.111776 (8.432226) with: {'learning_rate': 0.0001, 'max_depth': 6, 'n_estimators': 100}

-52.045728 (8.249767) with: {'learning_rate': 0.0001, 'max_depth': 6, 'n_estimators': 200}

-49.976399 (7.896034) with: {'learning_rate': 0.0001, 'max_depth': 6, 'n_estimators': 400}

-53.652829 (8.524886) with: {'learning_rate': 0.0001, 'max_depth': 8, 'n_estimators': 50}

-53.111776 (8.432226) with: {'learning_rate': 0.0001, 'max_depth': 8, 'n_estimators': 100}

-52.045728 (8.249767) with: {'learning_rate': 0.0001, 'max_depth': 8, 'n_estimators': 200}

-49.976399 (7.896034) with: {'learning_rate': 0.0001, 'max_depth': 8, 'n_estimators': 400}

-48.970063 (7.724229) with: {'learning_rate': 0.001, 'max_depth': 2, 'n_estimators': 50}

-44.237571 (6.918364) with: {'learning_rate': 0.001, 'max_depth': 2, 'n_estimators': 100}

-36.078697 (5.538471) with: {'learning_rate': 0.001, 'max_depth': 2, 'n_estimators': 200}

-23.929587 (3.519751) with: {'learning_rate': 0.001, 'max_depth': 2, 'n_estimators': 400}

-48.970063 (7.724229) with: {'learning_rate': 0.001, 'max_depth': 4, 'n_estimators': 50}

-44.237571 (6.918364) with: {'learning_rate': 0.001, 'max_depth': 4, 'n_estimators': 100}

-36.078697 (5.538471) with: {'learning_rate': 0.001, 'max_depth': 4, 'n_estimators': 200}

-23.929587 (3.519751) with: {'learning_rate': 0.001, 'max_depth': 4, 'n_estimators': 400}

-48.970063 (7.724229) with: {'learning_rate': 0.001, 'max_depth': 6, 'n_estimators': 50}

-44.237571 (6.918364) with: {'learning_rate': 0.001, 'max_depth': 6, 'n_estimators': 100}

-36.078697 (5.538471) with: {'learning_rate': 0.001, 'max_depth': 6, 'n_estimators': 200}

-23.929587 (3.519751) with: {'learning_rate': 0.001, 'max_depth': 6, 'n_estimators': 400}

-48.970063 (7.724229) with: {'learning_rate': 0.001, 'max_depth': 8, 'n_estimators': 50}

-44.237571 (6.918364) with: {'learning_rate': 0.001, 'max_depth': 8, 'n_estimators': 100}

-36.078697 (5.538471) with: {'learning_rate': 0.001, 'max_depth': 8, 'n_estimators': 200}

-23.929587 (3.519751) with: {'learning_rate': 0.001, 'max_depth': 8, 'n_estimators': 400}

-19.414644 (2.830758) with: {'learning_rate': 0.01, 'max_depth': 2, 'n_estimators': 50}

-6.744672 (0.933997) with: {'learning_rate': 0.01, 'max_depth': 2, 'n_estimators': 100}

-0.216053 (0.050337) with: {'learning_rate': 0.01, 'max_depth': 2, 'n_estimators': 200}

0.848897 (0.024814) with: {'learning_rate': 0.01, 'max_depth': 2, 'n_estimators': 400}

-19.414644 (2.830758) with: {'learning_rate': 0.01, 'max_depth': 4, 'n_estimators': 50}

-6.743499 (0.932824) with: {'learning_rate': 0.01, 'max_depth': 4, 'n_estimators': 100}

-0.254126 (0.091086) with: {'learning_rate': 0.01, 'max_depth': 4, 'n_estimators': 200}

0.831512 (0.008093) with: {'learning_rate': 0.01, 'max_depth': 4, 'n_estimators': 400}

-19.414644 (2.830758) with: {'learning_rate': 0.01, 'max_depth': 6, 'n_estimators': 50}

-6.743499 (0.932824) with: {'learning_rate': 0.01, 'max_depth': 6, 'n_estimators': 100}

-0.260028 (0.093910) with: {'learning_rate': 0.01, 'max_depth': 6, 'n_estimators': 200}

0.829355 (0.015182) with: {'learning_rate': 0.01, 'max_depth': 6, 'n_estimators': 400}

-19.414644 (2.830758) with: {'learning_rate': 0.01, 'max_depth': 8, 'n_estimators': 50}

-6.743499 (0.932824) with: {'learning_rate': 0.01, 'max_depth': 8, 'n_estimators': 100}

-0.258236 (0.092933) with: {'learning_rate': 0.01, 'max_depth': 8, 'n_estimators': 200}

0.831777 (0.028036) with: {'learning_rate': 0.01, 'max_depth': 8, 'n_estimators': 400}

0.852283 (0.003829) with: {'learning_rate': 0.1, 'max_depth': 2, 'n_estimators': 50}

0.813154 (0.046960) with: {'learning_rate': 0.1, 'max_depth': 2, 'n_estimators': 100}

0.829779 (0.037321) with: {'learning_rate': 0.1, 'max_depth': 2, 'n_estimators': 200}

0.832717 (0.031505) with: {'learning_rate': 0.1, 'max_depth': 2, 'n_estimators': 400}

0.785207 (0.061920) with: {'learning_rate': 0.1, 'max_depth': 4, 'n_estimators': 50}

0.757671 (0.097880) with: {'learning_rate': 0.1, 'max_depth': 4, 'n_estimators': 100}

0.772923 (0.083151) with: {'learning_rate': 0.1, 'max_depth': 4, 'n_estimators': 200}

0.777985 (0.077499) with: {'learning_rate': 0.1, 'max_depth': 4, 'n_estimators': 400}

0.800020 (0.031554) with: {'learning_rate': 0.1, 'max_depth': 6, 'n_estimators': 50}

0.722744 (0.115322) with: {'learning_rate': 0.1, 'max_depth': 6, 'n_estimators': 100}

0.718966 (0.120953) with: {'learning_rate': 0.1, 'max_depth': 6, 'n_estimators': 200}

0.716761 (0.123083) with: {'learning_rate': 0.1, 'max_depth': 6, 'n_estimators': 400}

0.816402 (0.004015) with: {'learning_rate': 0.1, 'max_depth': 8, 'n_estimators': 50}

0.766141 (0.059941) with: {'learning_rate': 0.1, 'max_depth': 8, 'n_estimators': 100}

0.756297 (0.069550) with: {'learning_rate': 0.1, 'max_depth': 8, 'n_estimators': 200}

0.755626 (0.070178) with: {'learning_rate': 0.1, 'max_depth': 8, 'n_estimators': 400}

0.855146 (0.003964) with: {'learning_rate': 0.2, 'max_depth': 2, 'n_estimators': 50}

0.863665 (0.002430) with: {'learning_rate': 0.2, 'max_depth': 2, 'n_estimators': 100}

0.862916 (0.000224) with: {'learning_rate': 0.2, 'max_depth': 2, 'n_estimators': 200}

0.849430 (0.007344) with: {'learning_rate': 0.2, 'max_depth': 2, 'n_estimators': 400}

0.758113 (0.097414) with: {'learning_rate': 0.2, 'max_depth': 4, 'n_estimators': 50}

0.759158 (0.098429) with: {'learning_rate': 0.2, 'max_depth': 4, 'n_estimators': 100}

0.754193 (0.102434) with: {'learning_rate': 0.2, 'max_depth': 4, 'n_estimators': 200}

0.748421 (0.107894) with: {'learning_rate': 0.2, 'max_depth': 4, 'n_estimators': 400}

0.780980 (0.061204) with: {'learning_rate': 0.2, 'max_depth': 6, 'n_estimators': 50}

0.773959 (0.067553) with: {'learning_rate': 0.2, 'max_depth': 6, 'n_estimators': 100}

0.773742 (0.067638) with: {'learning_rate': 0.2, 'max_depth': 6, 'n_estimators': 200}

0.773425 (0.067856) with: {'learning_rate': 0.2, 'max_depth': 6, 'n_estimators': 400}

0.804540 (0.032247) with: {'learning_rate': 0.2, 'max_depth': 8, 'n_estimators': 50}

0.800325 (0.036309) with: {'learning_rate': 0.2, 'max_depth': 8, 'n_estimators': 100}

0.800133 (0.036625) with: {'learning_rate': 0.2, 'max_depth': 8, 'n_estimators': 200}

0.800134 (0.036625) with: {'learning_rate': 0.2, 'max_depth': 8, 'n_estimators': 400}

0.804575 (0.055743) with: {'learning_rate': 0.3, 'max_depth': 2, 'n_estimators': 50}

0.823723 (0.042951) with: {'learning_rate': 0.3, 'max_depth': 2, 'n_estimators': 100}

0.832058 (0.027793) with: {'learning_rate': 0.3, 'max_depth': 2, 'n_estimators': 200}

0.824320 (0.028952) with: {'learning_rate': 0.3, 'max_depth': 2, 'n_estimators': 400}

0.684716 (0.174854) with: {'learning_rate': 0.3, 'max_depth': 4, 'n_estimators': 50}

0.683423 (0.176741) with: {'learning_rate': 0.3, 'max_depth': 4, 'n_estimators': 100}

0.676494 (0.183628) with: {'learning_rate': 0.3, 'max_depth': 4, 'n_estimators': 200}

0.676418 (0.183173) with: {'learning_rate': 0.3, 'max_depth': 4, 'n_estimators': 400}

0.533161 (0.294224) with: {'learning_rate': 0.3, 'max_depth': 6, 'n_estimators': 50}

0.520398 (0.307576) with: {'learning_rate': 0.3, 'max_depth': 6, 'n_estimators': 100}

0.520455 (0.307122) with: {'learning_rate': 0.3, 'max_depth': 6, 'n_estimators': 200}

0.520411 (0.307169) with: {'learning_rate': 0.3, 'max_depth': 6, 'n_estimators': 400}

0.666960 (0.156246) with: {'learning_rate': 0.3, 'max_depth': 8, 'n_estimators': 50}

0.668800 (0.154254) with: {'learning_rate': 0.3, 'max_depth': 8, 'n_estimators': 100}

0.668832 (0.154209) with: {'learning_rate': 0.3, 'max_depth': 8, 'n_estimators': 200}

0.668832 (0.154209) with: {'learning_rate': 0.3, 'max_depth': 8, 'n_estimators': 400}

XGBR_GSCV_Time_time： 61.41017997421118