Algorithm selection rationale (Random Forest vs Logistic Regression vs SVM)

Question

I want to understand the criteria of selection of ML algorithms, i.e., what are the guidelines on which algorithm to be selected in which case?

The reasons I know are:

1. Logistic regression to be picked in case we want to advise the impact on y variable on what changes on any x variable.
2. Random forest works good on mixed data and very effective for categorical data. Also, it does feature selection first (so dimension reduction is not needed).
3. Random forest not to be picked with high featured and multiple category data due to its high processing time.
4. SVM works well with the closely placed data points like in image processing identification of dog vs cat.

But these are not sufficient enough to pick anyone, as I don't have any reason for why which algorithm not to be picked, like when to choose SVM over Logistic regression or RF over Logistic regression.

The only rationale I have is the performance, so I run all algorithms and who ever performs best that I select (but this is not right way).

Answer 1

I suppose I will suggest as a starting point and expand on what you suggested by just adding the following

5. Knowing the type of data you are working with and it's characteristics, (categorical, supervised/unsupervised, data size etc.).

6. Knowing what accuracy requirements you need, timeframe and computational power you have at your disposal vs accuracy and really answering "why, am I trying to solve this problem?"

After answering these questions you can at least narrow down slightly what you may use (and eliminate those you clearly don't believe fit). After that, I suppose it's trial and error, experience and comparing to others who dealt with similar datasets and problems.

I have this crude flow chart I found in my favourites from the scikitlearn website. Not sure where I found it to be honest. Take it for what you will, hopefully it helps somewhat:

https://scikit-learn.org/stable/tutorial/machine_learning_map/index.html