Decision Tree Machine Learning Model

Most likely, the decision tree is a machine learning model that’s closest to everyday life and conditions. The decision trees are one of the simplest and yet most versatile machine learning techniques. A lot of people who don’t even deal with machine learning or artificial intelligence use it on many occasions. Of course, in its simplified version, but it’s still a fully-fledged decision tree machine learning model! Today, we will take a closer look at this machine learning model and examine some of its advantages and possible uses. Read on!

The entire model, in its basic version, is based on asking a series of questions and obtaining answers. The rule of thumb is–the more questions and more answers you have, you ought to come up with more accurate decisions or insight. Let’s demonstrate how it works on a straightforward decision tree machine learning example.

We want to achieve an answer to the question: “Should I go to the restaurant?” If we’re going to receive a precise answer, we need more questions (more data) and more answers. With just this one question, both answer YES and NO can be correct. This isn’t sufficient for us, so we’re digging further.

Let’s ask a couple of additional questions:

• Am I hungry?
• Do I have money to pay for the food?
• Do I have time to go to the restaurant?

And again, we can answer all these questions with YES or NO.

And, depending on our specific situation and the initial question asked, we could have more questions and more options. For instance, your problem could be more complicated, if it indicates two possible solutions: “Should I go to the restaurant, or should I cook dinner myself?” The more complicated is your initial question, the more sub-questions and options there are.

Interested in machine learning? Read our article: Machine Learning. What it is and why it is essential to business?

How is the machine learning decision tree constructed?

The machine learning decision trees are generally built in the form of ‘if-then-else’ statements. In machine learning, the decision tree is built on two major entities, which are called nodes (or branches) and leaves. The initial question is also called the root (hence the decision tree model name). The leaves are the decisions or final outcomes. And the decision nodes/branches are where data splits. In other words, leaves represent class labels and branches represent conjunctions of features that lead to these class labels[1].

Again, let’s use our example:

• Root: Should I go to the restaurant?
• Leaf: Am I hungry?
• Node/branch: YES/NO

So, like a real tree, it all starts with the root and goes further, to the treetops, where there are more branches and more leaves. But our tree, unlike real ones, grows upside down. If you’d use a piece of paper to draw this decision tree, you should start at the top and go downwards. The results (treetops) are at the bottom of your piece of paper.

Naturally, it’s a simplification of this machine learning model, but more or less, this is what it looks like. The decision tree machine learning model is all about obtaining outcome/decision/insight based on our previous knowledge and variables.

What are the machine learning decision trees used for?

This model is versatile, hence the decision tree algorithm in machine learning finds many applications in:

• Data mining
• Data science
• Statistics

Ultimately, the goal of this technique is to create a model that predicts the value of a target variable based on several input variables. This technique is used to answer such questions as i.a.: Should X be done? Should we choose A or B? Hence, this model is commonly used for:

Decision analysis

It’s eagerly used in the decision-making process. The decision tree machine learning model can be used in data visualization, to explicitly present the decision process and its outcome. This is one of the major advantages of the decision trees–they are easy to understand and legible, even for someone who doesn’t deal with machine learning in their work.

Classification

As you know from our other blog posts, classification is a machine learning technique used to predict or explain a class value. With the classification algorithms, you can assess the probability of an occurrence of an event based on one or more inputs. This is exactly what’s the decision tree machine learning model is all about! For instance, it could be used to divide e-mails into spam and non-spam. Your algorithm asks a series of questions and, based on the answers, decides if a given e-mail is a spam or not.

Regression

It’s one of the supervised machine learning techniques. Regression helps you to predict (or to explain) a particular value based on a set of prior data. You can use regression to predict such matters as an employee’s salary, disease spread, or property value. In this case, we call it the regression tree, and its outcome is always a specific number.

As we mentioned earlier, the decision trees are also broadly used in data mining software, i.a.: IBM SPSS Modeler, RapidMiner, SAS Enterprise Miner, Matlab, R (an open-source software environment for statistical computing), Weka, Orange, KNIME, and Microsoft SQL Server[2].

In real-life conditions, this model is widely used by people. Every single time you think if you should meet your friends or watch television–you use the decision tree technique, even if it’s only in your mind. That’s why it’s so common in various aspects of machine learning. It’s a technology that tries to simulate the way people learn, and people eagerly use this technique, even if they are not aware they do!

The characteristics of the decision tree machine learning model

The way the decision tree is built is quite straightforward. We start with our initial question/problem (the root). The rule of thumb is–you ought to keep your tree as small as possible. That’s why you should only ask valid questions, and as few of them as needed to obtain the accurate answer. This stage of building the tree is called ‘induction’.

The question is valid if it provides you with as much possible knowledge/insight as possible. It’s called information gain. It’s used to decide which feature to split on at each step in building the tree. You should always choose only these features/questions that will help you make a better decision.

For instance, let’s go back to our “Should I go to the restaurant?” question. You should omit irrelevant questions, like “Is it going to rain?” Remember, keep it simple and consistent. There’s no need to stretch the tree to the limits of common sense just to make it look bigger and more decent.

With regard to this issue, you should also be familiar with the term ‘pruning’. It’s a technique used to reduce the size of a machine learning tree without reducing its predictive accuracy. In other words, it means that the branches of low importance are to be removed from the tree. If we reduce the complexity of the tree, it’s accuracy can be even higher. In order to do so, you should go through each node and each leaf in the tree and evaluate the effect of removing it. If it doesn’t change much to the outcome, then it should be removed without hesitation.

Now, like every other machine learning model, the machine learning decision tree has its pros and cons. Let’s examine them.

Advantages of the decision tree machine learning model

First of all, it’s easy to use and understand, even for a person who has no knowledge of machine learning. All you need to build your own decision tree is to devise a root and a relevant dataset. And, of course, you need a list of all questions that need to be asked. Another significant advantage is the versatility of this model. It can handle categorical and numerical data alike.

Furthermore, this technique requires little to none data preparation. You can build your decision tree machine learning model with almost every form and kind of data. And last but not least, this technique can operate on large datasets and do it effectively!

Disadvantages of the decision tree machine learning model

Obviously, there are also downsides. At least three of them, actually. First of all, this technique requires some kind of measurement of its efficiency. You have to have a way to verify if your outcome is reasonable and correct.

The next thing is related to this model’s ‘delicacy’. You have to be careful with parameter tuning because even a small, unnoticeable change can result in a completely different outcome.

And finally, when it comes to data including categorical variables with varying numbers of levels, information gain in decision trees is biased in favor of attributes with more levels of variables.

All in all, if this method is properly used, it’s very effective. We also use machine learning decision trees in our work. And we do so on many occasions! Certainly, your company also could benefit from machine learning and artificial intelligence.

The vast majority of organizations are still prior to this step. Many are afraid of the costs and complexity of this shift. But with our help, it’s easy and quick. The decision trees are just the doorstep of the machine learning world. With machine learning services your company will work in a faster, smarter, more efficient way. Drop us a line, and let’s talk about the benefits that await you!

Also, see our machine learning services to find out more.

References

[1] Wikipedia. Decision tree learning. URL: https://en.wikipedia.org/wiki/Decision_tree_learning. Accessed Apr 28, 2020.

[2] Microsoft. Microsoft Decision Trees Algorithm Technical Reference. URL: https://docs.microsoft.com/en-us/analysis-services/data-mining/microsoft-decision-trees-algorithm-technical-reference?redirectedfrom=MSDN&view=asallproducts-allversions&viewFallbackFrom=sql-server-ver15. Accessed Apr 28, 2020.