Evaluating Your Models

Perfect! You should at least have three random forest models now (one for the regressions we gave as an example, one for your own regression model and one extension). Now it is time to diagnose which "does the best"!

Just like with our previous regression models, there is no definite metric on what constitutes a model doing "well". The black box issue of random forest models certainly does not make it easier. Lets again start with evaluating our model by calculating the RMSE. Refer back to the formula in the previous section, calculate the RMSE of all your models and display them next to your model in an appropriate format.

Depending on what package and programming language you use here, things can become a little bit more tricky now: we want to see which variables of the models are the most relevant. While RMSE evaluated our predictions, figuring out which variables are more relevant can be compared to evaluating or regression coefficients for statistical significance - at least to some degree. The objective is to find out which variable in our random forest model "explains" log GDP per capita best. Both R and Python can check this extensive document for more information. The below plot is an example evaluating variable relevance using the percent influence of a variable on MSE. However, depending on your language and package you use, you can also pick other criteria for variable importance. Explain us your choice of criteria and provide a plot. Explain whether your findings, in your educated opinion, make sense or not, but don't worry too much, we are not asking your for a detailed economic analysis.

At the end, select the model you deem to be best and save it for the last exercise.

You can used the permutation_importances() function from the sklearn package to get an absolute value for the importances or retrieve the .feature_importances_ attribute from your random forest regressor model for the relative importances. Use a package of your choice, such as matplotlib or seaborn to plot these.