How it works...

The following is the performance of the grid-searched model on both the training and cross-validation datasets. We can observe that the AUC has increased by one unit in both training and cross-validation scenarios, after performing a grid search. The training and cross validation AUC after the grid search is 0.996 and 0.997 respectively.

# Performance on Training data after grid search
> train_performance.grid <- h2o.performance(best_dl_model,train = T)
> train_performance.grid@metrics$AUC
[1] 0.9965881

# Performance on Cross validation data after grid search
> xval_performance.grid <- h2o.performance(best_dl_model,xval = T)
> xval_performance.grid@metrics$AUC
[1] 0.9979131

Now, let's assess the performance of the best grid-searched model on the test dataset. We can observe that the AUC has increased by 0.25 units after performing the grid search. The AUC on the test data is 0.993.

# Predict the outcome on test dataset
yhat <- h2o.predict(best_dl_model, occupancy_test.hex) 

# Performance of the best grid-searched model on the Test dataset
> yhat$pmax <- pmax(yhat$p0, yhat$p1, na.rm = TRUE) 
> roc_obj <- pROC::roc(c(as.matrix(occupancy_test.hex$Occupancy)), c(as.matrix(yhat$pmax)))
> pROC::auc(roc_obj)
Area under the curve: 0.9932