Possible “solution”
Step 1: design
The iris
dataset is composed by 5 variables, hence the
question may be answered by basically “trying to predict” each one of
them and measuring how hard it is. If we intend to follow this path, we
need:
- to be able to “try to predict”
- a way for measuring how hard it is
Concerning 1, and noted that we can tackle the prediction problem as
a supervised learning problem, we know a learning technique (officially
just one) that fits the scenario: learning of decision/regression trees.
Interestingly, 4 of the variables are numeric and 1 is categorical:
fortunately, we have a variant of the learning technique for each of the
two cases.
Concerning 2, we note that the notion of hardness, as captured by the
corresponding word in natural language, is broad. “Hard” could refer to
computational effort, time needed to design a ML system for doing
prediction, required knowledge, … We decide to interpret “hard to be
predicted” as “degree to which, on some data, a model learned with some
suitable technique correctly predicts the dependent variable”. In brief,
we plan to interpret hardness as (in)effectiveness of prediction; we
know several ways to measure (in)effectiveness. One is error.
(I am being purposely vague here…). Note that, while in principle this
“definition” is not bound to a specific dataset or to a specific
learning technique, we have only one dataset an only one learning
technique.
In practice, hence, we are going to build 5 models (4 regression and
1 classification models), measure their errors, and say that the
hardest variable is the one for which the error is the largest.
Note that the last part (the one in italic), is a key step, even if it
appears trivial and it does not require any big implementation effort.
Recall that we want to “design a method for answering the question”:
when executed, the method is required to result in an answer. Without
that step, the method execution would result in a collection of 5
numerical values, not in an answer!
Step 2: implementation
The method designed above and described in natural language can be,
in principle, applied to any dataset. Again in principle, we could
implement as a function that takes a dataset (a dataframe, in terms of
R), and outputs the name of the hardest variable.
However, for simplicity we’ll start by doing an implementation mixed
with the execution. This is done also beacuse there are still some
ambiguities in the description of the method (e.g., how to measure the
error?) and we will discover and address them as we meet them. Ideally,
one should first fully define the method (i.e., the corresponding
algorithm), possibly in the form of a piece of pseudocode.
Simplicity: implementation + execution
Load iris
.
d = iris
How can we learn a tree? Using a package! There are many packages
that implement (slightly different versions of) the learning technique
that we saw in the lectures. My choice is for the package
tree
(that you probably have to install, first, using
install.packages("tree")
). (Another option is
rpart
).
require(tree)
Supervised learning conventions in R
Most of the packages for supervised learning share the following
structure. They have:
- a function for doing the learning, usually with the name of the
technique itself
- a function for doing the prediction, usually named
predict()
tree
adheres this “convention” (see
the documentation).
The learning function usually has two ways for being invoked: one in
which \(\{(x^{(i)},y^{(i)})\}_i\) is
specified by giving \(\{x^{(i)}\}_i\)
and \(\{y^{(i)}\}_i\) separately and
explicitly, the other in which a dataframe (without indication of the
dependent variable) is passed together with an indication of the
dependency. (Actually tree()
only supports the second
way). The latter is done using a data type, peculiar of R, known as
formula. The syntax for specifying a formula literal
(i.e., a constant value for a data type) is a ~ b+c
(this
is an example, rather than a formal specification of the syntax): this
value means a
depends on b
and c
.
A common case is that one wants to say that a variable depends on all
the other variables: this can be done with a ~ .
.
Summarizing, for building a tree for predicting the Species based on
the other variable on the full dataset d
, we do:
tree(Species ~ ., d)
node), split, n, deviance, yval, (yprob)
* denotes terminal node
1) root 150 329.600 setosa ( 0.33333 0.33333 0.33333 )
2) Petal.Length < 2.45 50 0.000 setosa ( 1.00000 0.00000 0.00000 ) *
3) Petal.Length > 2.45 100 138.600 versicolor ( 0.00000 0.50000 0.50000 )
6) Petal.Width < 1.75 54 33.320 versicolor ( 0.00000 0.90741 0.09259 )
12) Petal.Length < 4.95 48 9.721 versicolor ( 0.00000 0.97917 0.02083 )
24) Sepal.Length < 5.15 5 5.004 versicolor ( 0.00000 0.80000 0.20000 ) *
25) Sepal.Length > 5.15 43 0.000 versicolor ( 0.00000 1.00000 0.00000 ) *
13) Petal.Length > 4.95 6 7.638 virginica ( 0.00000 0.33333 0.66667 ) *
7) Petal.Width > 1.75 46 9.635 virginica ( 0.00000 0.02174 0.97826 )
14) Petal.Length < 4.95 6 5.407 virginica ( 0.00000 0.16667 0.83333 ) *
15) Petal.Length > 4.95 40 0.000 virginica ( 0.00000 0.00000 1.00000 ) *
Note that:
- the formula is not quoted: it is not a string representing a
dependency, it is a formula!
- the outcome of the execution of the above command is the textual
representation (on the standard output) of the learned tree [there might
be an error here…]
You should also note that there are more decorations, on that tree,
than the ones we expected to see. Look at the documentation for more
info.
As an aside, a tree can also be plot:
t = tree(Species~., d)
plot(t)
text(t)
![](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAArwAAAGwCAMAAAB8TkaXAAAC6FBMVEUAAAABAQECAgIDAwMEBAQFBQUGBgYHBwcICAgJCQkKCgoLCwsMDAwNDQ0ODg4PDw8QEBARERESEhITExMUFBQWFhYXFxcYGBgZGRkaGhobGxscHBwdHR0eHh4fHx8gICAhISEiIiIjIyMkJCQlJSUmJiYnJycoKCgpKSkqKiorKyssLCwtLS0uLi4vLy8wMDAxMTEyMjIzMzM0NDQ1NTU2NjY3Nzc4ODg5OTk6Ojo7Ozs8PDw9PT0+Pj4/Pz9AQEBBQUFCQkJDQ0NERERFRUVGRkZHR0dISEhJSUlKSkpLS0tMTExNTU1OTk5PT09QUFBRUVFSUlJTU1NUVFRVVVVWVlZXV1dYWFhZWVlaWlpbW1tcXFxdXV1eXl5fX19gYGBhYWFiYmJjY2NkZGRlZWVmZmZnZ2dqampra2tsbGxtbW1ubm5vb29wcHBxcXFycnJzc3N0dHR1dXV2dnZ3d3d4eHh5eXl6enp7e3t8fHx9fX1+fn5/f3+CgoKDg4OEhISFhYWGhoaHh4eIiIiJiYmKioqLi4uMjIyNjY2Ojo6Pj4+QkJCRkZGSkpKTk5OUlJSVlZWWlpaXl5eYmJiZmZmampqcnJydnZ2enp6fn5+goKChoaGioqKjo6OkpKSlpaWmpqanp6eoqKipqamqqqqrq6usrKytra2urq6vr6+wsLCxsbGysrKzs7O0tLS1tbW2tra3t7e4uLi6urq7u7u8vLy9vb2+vr6/v7/AwMDBwcHCwsLDw8PExMTFxcXGxsbHx8fIyMjJycnKysrLy8vMzMzNzc3Ozs7Pz8/Q0NDR0dHS0tLT09PU1NTV1dXW1tbX19fY2NjZ2dna2trb29vc3Nzd3d3e3t7g4ODh4eHi4uLj4+Pk5OTl5eXm5ubn5+fo6Ojp6enq6urr6+vs7Ozt7e3u7u7v7+/w8PDx8fHy8vLz8/P09PT19fX29vb39/f4+Pj5+fn6+vr7+/v8/Pz9/f3+/v7////wKT9iAAAACXBIWXMAAA7DAAAOwwHHb6hkAAAaD0lEQVR4nO3df2AU5Z3H8WeT3RDyAxLzA0JEoVwgYFEIkhx6ei1oiChFkerVyNVrULhiaFdRqlTo2daUAgUlB1RFfig5moRfAlU4PQ8tFPVaKzVgsVWgAYRgQiDJ8+89z8zszC4bQn488919ls/rj83M7Gxmh3mbncnMRMYBNMUi/QYAugvxgrYQL2gL8YK2EC9oC/GCthAvaAvxgrYQL2gL8YK2EC9oC/GCthAvaAvxgrYQL2gL8YK2EC9oC/GCthAvaAvxgrYQL2gL8YK2EC9oC/GCthAvaAvxgrYQL2gL8YK2EC9oC/GCthAvaAvxgrYQL2grVuP1MyFp1LqgSQd+FzSSucz4MuHmy38r+brcxZd8+leFKUOfu+CMb90kHjbJxbOyLr1l6KqYjTepqqqqsojVOpNKpwQ934V45esuHe8C9oMtT3iftsc/TioVjxVZK4TdXX3X0CUxG2+afPyq3yRnUqfjvRA62mG8zX1mi8cf9m6xxs8XMBnvzPHdeM/QRbEdL79tOOcvFiSNWM15kfgc//LMo0MSBy9oaydeay6eu3wC6zvtBOet864Z+GT5FPN1uYueGtz33hPhyznEdojHKnbYGp87tkDGO/FhV9cODLEdb2P/Yr7UN39bued5fmLqxKNt96RX1D7K1oXHG5iL56Y/sPvn3lmcz06v3Diq1xTzdbnZpXsWJ84OWkDbjo3yy7m6c+JxTu8mc+rulINFMt684tHJ11cSresVK2bjTa6url59E6s6m7FAjM7IMj/+p64QI0MfD4vXmSv3BvFz+a6b+OcJIvG/eadYuw03iKen32p/+9NL89LW2GOveB8zB04NfIHLeFsTMpZUl7EKklW9csVsvPJoPz5vFX+XvVNfX7+GHbH2eRsOrPT5w+J15sp9Ur66iNewBjEwNhDvPDFSHvgx/dGslBtXNQYWdfxB9l1rN/m+Em7E27zhkBib3qeVYlWvXDEbb5o18CozfWhE+PZIT05JTni8zly5z3Ej3mW95fQ7A/H+ktvxHp6QXLbPWdLWrEHV1uD6jC/MeE2bWJ2LawixH+8b7Lg1JCI86Zt5TOQVHq8zV678rPcXiWOws2KgaErQbxuseN9LLljfbC9oa/zMpsBwufWfQPWxfWLfg9eyoy6tHRhiPt7jvVaJx6fHGxHukD8Lm/qHx+vMFYj3cNydnH+R0E68/NSiIdlPHTGHL+Q86Cz04C4hf/yu4zvZWjH6yEB31/GKF/Px8rkJC7c95lnC+UMj9h3yTXurpjD5jnoZb+X95/iE/Grpj85cgXj5P/VeXVM44B7jdS0h8XLetq3Ed/d+OfRb9viLUpPx7SS529BSmLVg6+y4jaSrfOWJ/XjbKq5LGi5/yfDmoNTTG4Ymj938Uvp8GW+Z2DGYYH7OP+HMZce7vCB72Nri6ebrLopX+GTOQvllhbWncNT4dpKxz9tYPix13Haidb1ixWq8PdbwwMMi/Gt/Eun3AZeGeC+h7arBdSefSfg80u8DLg3xXsqPspkn7/VIvwvoAOK9lBX+0w2Rfg/QIYp4T8+do6HxBZF+B93xMsH2jBoU8S6Ky9VQemqk30E3ZPYi2J5RgyLe564iWIhyK/yRfgfdUIt4FUO8ZBCvaoiXDOJVDfGSQbyqIV4yiFc1xEsG8aqmZ7zNTZefJ+ogXtX0jFdLiFc1xEsG8aqGeMkgXtUQLxnEqxriJYN4VUO8ZBCvaoiXDOJVDfGSQbyqIV4yiFc1xEsG8aqGeMkgXtUQLxnEqxriJYN4VUO8ZBCvaoiXDOJVDfGSQbyqIV4yiFc1xEsG8aqGeMkgXtUQLxnEqxriJYN4VUO8ZBCvaoiXDOJVDfGSQbyqIV4yiFc1xEsG8aqGeMkgXtUQLxnEqxriJYN4VUO8ZBCvaoiXDOJVDfGSQbyqIV4yiFc1xEsG8aqGeMkgXtUQLxnEqxriJYN4VUO8ZBCvaoiXDOJVDfGSQbyqIV4yiFc1xEsG8aqGeMkgXtUQLxnEqxriJYN4VUO8ZBCvaoiXDOJVDfGSQbyqIV4yiFc1xEsG8aqGeMkgXtUQLxnEqxriJYN4VUO8ZBCvaoiXDOJVDfGSQbyqIV4yiFc1xEsG8aqGeMkgXtUQLxnEqxriJYN4VUO8ZBCvaoiXDOJVDfGSQbyqIV4yiFc1xEsG8aqGeMkgXtUQLxnEqxriJYN4VUO8ZBCvaoiXDOJVDfGSQbyqIV4yiFc1xEsG8aqGeMkgXtUQLxnEqxriJYN4VUO8ZBCvaoiXDOJVDfGSQbyqIV4yiFc1xEsG8aqGeMkgXtUQLxnEqxriJYN4VUO8ZBCvaoiXDOJVDfGSQbyqIV4yiFc1xEsG8aqGeMkgXtUQLxnEqxriJYN4VUO8ZBCvaoiXDOJVDfGSQbyqIV4yiFc1xEsG8aqGeMkgXtUQLxnEqxriJYN4VUO8ZBCvaoiXDOJVDfGSQbyqIV4yiFc1xEsG8aqGeMkgXtUQLxnEqxriJYN4VUO8ZBCvaoiXDOJVDfGSQbyqIV4yiFc1xEsG8aqGeMkgXtUQLxnEqxriJYN4VUO8ZBCvaoiXDOJVDfGSQbyqIV4yiFc1xEsG8aqGeMkgXtUQLxnEqxriJYN4VUO8ZBCvaoiXDOJVDfGSQbyqIV4yiFc1xEsG8aqGeMkgXtUQLxnEqxriJYN4VUO8ZBCvaoiXDOJVDfGSQbyqIV4yiFc1xEsG8aqGeMkgXtUQLxnEqxriJYN4VUO8ZBCvaoiXDOJVDfGSQbyqIV4yiFc1xEsG8aqGeLvAz4SkUeuCJh34XdBI5jL5OJd9Kh5fZfPkSPYEnrLSfLbWm2nOn7u4g2XsrLEGdjFTNd9kfC1TtRY0EG+U8SdVVVVVFrFaZ1LplKDnzXi3M1n3DDZGPB5mC/j9O81njXjl/B3F2zrWbw39dYX0cOIhXpElh3arXBP3Id4o40+Tj1/1m+RMaifeBu/3xeOg0XF/53wd+2/72U7E+9nyW5g/eELL6Gc5nzleybunhXijjBkvv2045y8WJI1YzXmR+Dz/8syjQxIHL2gLxMsLCzg/xLbHb+D8+4nneJrYbTj7SG5uiYjXmD930VOD+957op0FbLv55sSQeJeMvMD5xIddXzP1EG+UMeNt7F/Ml/rmbyv3PM9PTJ14tO2e9IraR+W+ghXvXO9XvLJvS9H3OB/zz9yI9xt9Fm28jmWa8+dml+5ZnDg76Bu37dgYGBwSHG99n9fFY17x6OTrKylWUCHEG2X8ydXV1atvYlVnMxaI0RlZ5m7A1BViZOjjdrzb2W4+bQqffzVv8v3YiPcNuZv8X57AbsMNYqbpt9rf9vTSvLQ1gZGQeJ8oFA+tCRlLqstYBckqKoN4o4zx24b4vFX8XfZOfX39GnbE2udtOLDS57fjbfA+25qxjL/N/vQWe8OI96fpYnJtfCDeeWKs/Gbrm340K+XGVY32MoLj/XuyPDRs3nBIPE7v00qwhuog3ihj7fPK34OZPjRifHukJ6ckx4mXF961n/2JX+iz5LmERiPe8ut48AHbL7kd7+EJyWX7gpcRHO+irAv28CZW5+KaqYd4o4wd7xvsuDUkYjzpm3lMHLoFxTs386cDxJdv3XmPLFTEa/wrOz955W8brHjfSy5Y3xy8jOB4r5e/tuDH9rXJV7Ojbq2WKxBvlLHjPd5rlXh8erwR4w75M7Gpf1C829nX/lV8WZ6c/SNuxLuHbeG82tNOvPzUoiHZTx1xlhEU70H2W/llJ1srHh8Z6O66qYZ4o4wdL5+bsHDbY54lnD80Yt8h37S3agqT76iX8Vbef07u9DJ5APYJYzu49duGvr+qKZDxivlbQuLlvG1bie/u/YExM17j2zwf3yCHWwqzFmydHWf/PkIPiDfKOPG2VVyXNFz+kuHNQamnNwxNHrv5pfT5Mt4ydlZMLWR/k3MN8soRGW/Dw1f3L/ZlmvNfFK/wyZyFgUEzXuPb3HuDOamxfFjquO2ur51aiDem4MIc1RAvGcSr2pUXb3PFzyLju94ILfhnr0Xgn5ki3j2TCRYSVTZ4roqMNF+EFtzHF4F/Zop4rzxrkyL9Dqjt8UZgoYjXDYiXBOJ1A+Il4Xa8IbewXDEQLwm34w25C+CKgXhJIF43IF4SiuP9oDgt4255CUjwLSytC4enFm4Oeta+pyVWIV4SauNtzB63vjLnNh56C8usxGe3PMRqnGfte1piFeIloTbefWwX59V+HnILy2fepWJk0ij7WeeelliFeEmojbc+9eu//lx8DbmFxbzE+aW45sCz3L6nJVYhXhKK93n335XIRm0KvYVlRZzcu93J/hJ41rmnJVYhXhLKf9vQuKM4/uOQW1hq2TEx8IqnKfCsc09LrEK8JNTG+1reWfnnh7aH3MJyxCvvXJk80n7WuaclViFeEmrj/XPCpC3rb888EXILS8vM3hWvz2DV9rOf2ve0xCrES0LxbsPmguTMO96/6BaWlmfyU8bWBj1r39MSqxAvCVyY4wbESwLxugHxkkC8bkC8JBCvGxAvCcTrBsRLAvG6AfGSQLxuQLwkEK8bEC8JxOsGxEsC8boB8ZJAvG5AvCQQrxsQLwnE6wbESwLxugHxkkC8bkC8JBCvGxAvCcTrBsRLAvG6AfGSQLxuQLwkEK8bEC8JxOsGxEsC8boB8ZJAvG5AvCQQrxsQLwnE6wbESwLxugHxkkC8bkC8JBCvGxAvCcTrBsRLAvG6AfGSQLxuQLwkEK8bEC8JxOsGxEsC8boB8ZJAvG5AvCQQrxsQLwnE6wbESwLxugHxkkC8bkC8JBCvGxAvCcTrBsRLAvG6AfGSQLxuQLwkEK8bEC8JxOsGxEsC8boB8ZJAvG5AvCQQrxsQLwnE6wbESwLxugHxkkC8bkC8JBBvOD8TkkatC5p04HdBI5nLjC8Tbr7kd7Djla/LXdzRwrZusgbOPzOs7/j3xMAmuXhW1uW33QM9X2M7Xso1Rrzh/ElVVVWVRazWmVQ6Jej5LsQrX9fhpvw4qdQaejBt8ZYHkj/kvCJrhbC7O++8u3q+xna8lGuMeMP50+TjV/0mOZM6vSkvGI+djvd8AbM25Udsg3i87UHOZ47v3tvugZ6vcafjVbnGiDecuSn5bcM5f7EgacRqzovEp9qXZx4dkjh4QVs7m9Kai+cun8D6TjvB+Su+awY+WT7FfF3uoqcG9733RLuLmju2wNqU69hR8fiLPpxPfNjVtWtPz9f4zbgIrDHiDWduysb+xXypb/62cs/z/MTUiUfb7kmvqH2UrQvflIG5eG76A7t/7p3FebGncuOoXlPM1+Vml+5ZnDg7aAFtOzaaA7tTDhZZm3Ine1c8fp+d5XnFo5Ovr6RZVUvP13gqi8AaI95w/uTq6urVN7GqsxkLxOiMLPPDcOoKMTL08bBN6cyVe4P4KXXXTfxzby/O/+adYn2I3iCenn6r/e1PL81LW2MMnRr4Ag9syq+uLfzozJoEdqQ1IWNJdRmrIFtdrmKNffERWGPEG8449o7PW8XfZe/U19evYUesPcCGAyt9/rBN6cyV+6R8dRGvYXKfd2xgU84TI+WBH1ofzUq5cVWjOXxfCbc3JT+Qz9g1P2BfNW84JMam92klWVeTgjWW+7zUa4x4w1l7gJy/ykwfGpvk7ZGenJKc8E3pzJX7HDc25bIEGe+dgU35S25vysMTksv2BZazPuOLoE3J2z75Y+uiVGtkE6tzdyVD9HyNe8l4qdcY8YazN+Ub7Lg1JDbJSd/MY+JfPnxTOnPlyk8+fxGvMn7yFk0JOva2NuV7yQXrmwPLKbcaqJYj5//QIB6/M44f2yc+iXmtcTRDRcEay3ip1xjxhrM35fFeq8Tj0+ONTbJD/mRo6h++KZ25ApvycJzY5/0ioZ1NyU8tGpL91BFz+OAuIX/8LqOE8ylzOD+W+oI4jlkrRh8ZSLGmAQrWOD4Ca4x4w9mbks9NWLjtMc8Szh8ase+Qb9pbNYXJd9TLTVl5/zk+Ib9a+qMzV2BT8vGe1TWFA+4xXtcSsinFZ+W2Et/d++2FGR+i8tvNTnphw+gRX/KWwqwFW2fHbaRbXxVrPJlFYI0RbzhnU7ZVXJc0XB5yvzko9fSGocljN7+UPl9uyjJ2lk8wPwKfcOayN+XL3uxha4unm6+7aFMKn8xZaA8bm1J+u6byfgMe/LsYaSwfljpuO8WK2nq+xr+Ni8AaI14XNPxbb5HBtT+J9Pug0/BDbwTWGPG6oC0jvu7kMwmfR/p90GnL9kRgjfWP9507ByR+7YenO57p8leW2Dq+LOriC6B21rQ39SdxzJP3+uWXFTtWeiKwxtrHu8c7cf2Wn/Yb2dThXF2It+MrSy66AKp1rL+dqXxt0umGyy8pluzxRmCNtY+3ZIw8MfM+Wxkyta0ldK7LXxZl6zjekAugPlt+C/OHTeW4GJ2I9vHmfdP48uQ2bl/rVMe2D/QM+5mY0NnLor6Zn5+f6fNlXJWa31t8/g/1ZWf64vrk5YdLSQsauTopyZMRNlUYENfOS3tkOd2/6De78/6u8XRrtcp79E61j3e654mPrcHAtU51rPeMar/nx5x39rIoT0npsITCW66Kv7r02wNz7i1NShx0+5j4YaVBxt9ifOmTc5U3vdCZnDq8van3jSlVK+sbdP+icZO78Qb/ZXR3Viv/6h69U+3jPXmflw186NXmoGud6thUMfBY6plOXxbl2ft5wrrOXBYVdgHUEH97U5W7hTLeg2SLKr/C4+X81G/+/R/YoIPOtU51rEpM/j/jctHOXRbl2VvD5PHGZS+LCrsAyojX/QvBEG97tI/XPDLb2afEudapjr0tJp0SCXf2sijP3mW95fTLXhZlCroAyog3bKpylPHGHyFb1BPX9ujlusf7B7bH+Prdq51rncyfvB+x/+n0ZVGevVXsLO/EZVFhF0AZ8bp/IRhlvJ/SLepCz85q6B7vuYxvyw/r5uGTnWud6ti3xcC8pFOdvizKs/dw/IbOXBYVdgGUEa/7F4K5FW/Kyg6nhD8dVYvSPV7+WuI//mLDsqKkD5wroupY4sytT8bN45929rIoz17+SEZnLotyLoCS345b8bp/IZhb8d6/s8Mp4U9H1aK0j5fvvzcvcfB3PuTOFVF17DeT0vL+Q3yUd/ayKBHv+TmduizKvgCqzNjPsPZ5Xb8QzOXdhubzF0+50N5s0bYo/eMNV8f2XX6mEPF7G1b8JZovBFMc792jxEPbgJk8TXxY93vVn/5p6zzz1nVjin1DuxxpmT80/fY/BJ3wiZ5FIV4pfm/bwEnRfCGY4njXMrEP/7/sHbOoMd/a2DQ73bx13SwqcEO7HPle3+Vrb0454pzwiZ5FIV4pfi///ehovhBMcbyney0XO0uDzGT6jWzjzjkaoyjrhnY58ue418QzCUucEz7Rs6hYjPfCp82XnymEiFf8M0fxhWCq93nvLOZ80NNWUXM5d87RGEVZN7TLkZe9cp/0jLxozzrhEz2LisV4u86IN5qpjvfXCWd+zz62ilrEuXOOxijKuqFdjizsZ73EPuETPYuKtXgrSrrDM647r3qEbrVUx3vSt/FHY7hV1GLOnXM0RlGBO9PEyMoE+Xv0AwedEz7Rs6hYizcnp6gbsm7sxotGxtGt1strFX/D20uHyVPggaKcczQXF/W+/LunTRm/cE74RM+iYi7ex8gW9QFhvMr9Z2L8F9wpyjlHc3FRfFpG5Y7Jff/qnPCJnkUh3m5zK16SU7bH42+XX+yi7HM0YUU1+7+Weut7QSd8omdRiLfb3IqX8JStjfAcjcJFxVq8D9Refh5FThW5+u0JT9mKmOjO0ShcVKzFqy/CU7btIDxHo25RiDdaEJ6ybRfhORpVi0K80YLwlG2siKl42z0Qt65Dj/5lEZ6yjRExFW+7B+Iuxat+WYSnbGNEDMQbflge4uKgLjN75JZFeMo2RugVr31Ebv/FG+Ow/IPitIy7j4QeiLcuHJ5auJmbQdkjxuzRtywT4Snb2KBXvPYRuf0Xb+Rh+cnscesrc24LPRCflfjslodYjRmUPWIcxUffskyEp2xjg17xBo7Inb94Iw/L97FdnFf7Qw7EP/MuFXNMGmUE5YzI2aNwWSbCU7axQa94A0fkzl+8kYfl9alf/7VxwiboQNz8IwovxTXLoJwROXs0Lqs9UX5bXRTQLF7riNz5izfysJzvvyuRjdoUciC+Ik7+2NvJ/iKDckaM2aNwWe2J8tvqooBm8VpH5M5fvOln/iXdxh3F8R8HH4jXMnEczl/xNJk/DQMj/Tr4/5FHclntiu7b6qKAZvFaR+TOX7yRhbyWd1Ycm7PtwQfiR7zyt1aTRxr7oc5Il4KiXFb7ovm2uiigW7zWEbn9F29kIX9OmLRl/e2ZJ4IPxPnM3hWvz5D/q0X5GwB7pEtBUS4LukG3eK0jcvsv3hiFbC5Izrzj/dAD8ZZn8lPGyusjZVD2SJeColwWdINu8QLYEC9oC/GCthAvaAvxgrYQL2gL8YK2EC9oC/GCthAvaAvxgrYQL2gL8YK2EC9oC/GCthAvaAvxgrYQL2gL8YK2EC9oC/GCthAvaAvxgrYQL2gL8YK2EC9oC/GCthAvaAvxgrYQL2gL8YK2EC9oC/GCthAvaAvxgrYQL2gL8YK2EC9oC/GCthAvaAvxgrYQL2gL8YK2EC9oC/GCthAvaAvxgrYQL2gL8YK2EC9oC/GCthAvaAvxgrYQL2gL8YK2EC9oC/GCthAvaAvxgrYQL2gL8YK2EC9oC/GCthAvaAvxgrYQL2gL8YK2EC9oC/GCthAvaAvxgrYQL2gL8YK2EC9oC/GCthAvaAvxgrYQL2gL8YK2EC9oC/GCthAvaAvxgrYQL2gL8YK2EC9oC/GCthAvaAvxgrYQL2gL8YK2EC9oC/GCtv4f+nDlUbnSMK0AAAAASUVORK5CYII=)
text()
is needed to add the labels to the tree.
With the rpart
package
require(rpart)
rpart(Species~., d)
n= 150
node), split, n, loss, yval, (yprob)
* denotes terminal node
1) root 150 100 setosa (0.33333333 0.33333333 0.33333333)
2) Petal.Length< 2.45 50 0 setosa (1.00000000 0.00000000 0.00000000) *
3) Petal.Length>=2.45 100 50 versicolor (0.00000000 0.50000000 0.50000000)
6) Petal.Width< 1.75 54 5 versicolor (0.00000000 0.90740741 0.09259259) *
7) Petal.Width>=1.75 46 1 virginica (0.00000000 0.02173913 0.97826087) *
t = rpart(Species~., d)
plot(t)
text(t)
![](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAArwAAAGwCAMAAAB8TkaXAAACuFBMVEUAAAABAQECAgIDAwMEBAQFBQUGBgYHBwcICAgJCQkKCgoLCwsMDAwNDQ0ODg4PDw8QEBARERESEhITExMUFBQWFhYXFxcYGBgZGRkaGhobGxscHBwdHR0eHh4fHx8gICAhISEiIiIjIyMkJCQmJiYnJycoKCgpKSkqKiorKyssLCwtLS0uLi4vLy8wMDAxMTEyMjIzMzM0NDQ1NTU2NjY3Nzc4ODg6Ojo7Ozs8PDw+Pj4/Pz9AQEBBQUFDQ0NERERFRUVGRkZHR0dISEhJSUlKSkpLS0tMTExNTU1OTk5PT09QUFBRUVFSUlJTU1NUVFRVVVVWVlZXV1dYWFhbW1tcXFxdXV1eXl5fX19gYGBhYWFjY2NkZGRlZWVmZmZnZ2dpaWlqampra2ttbW1ubm5vb29wcHBxcXF0dHR1dXV2dnZ3d3d4eHh5eXl6enp7e3t8fHx9fX1+fn5/f3+CgoKDg4OEhISFhYWHh4eIiIiKioqLi4uMjIyNjY2Ojo6Pj4+QkJCRkZGSkpKUlJSVlZWWlpaYmJiZmZmbm5ucnJydnZ2enp6fn5+goKChoaGioqKjo6OkpKSlpaWmpqanp6eoqKipqamqqqqrq6usrKyurq6wsLCxsbGysrKzs7O0tLS1tbW2tra3t7e4uLi6urq7u7u8vLy9vb2+vr6/v7/AwMDBwcHCwsLDw8PExMTGxsbHx8fJycnKysrLy8vMzMzNzc3Ozs7Pz8/Q0NDR0dHS0tLT09PU1NTV1dXW1tbX19fY2NjZ2dna2trb29vc3Nzd3d3e3t7f39/g4ODh4eHi4uLj4+Pk5OTl5eXm5ubn5+fo6Ojp6enq6urr6+vs7Ozt7e3u7u7v7+/w8PDx8fHy8vLz8/P09PT19fX29vb39/f4+Pj5+fn6+vr7+/v8/Pz9/f3+/v7///+ZdAGnAAAACXBIWXMAAA7DAAAOwwHHb6hkAAAQs0lEQVR4nO3c+38V5Z3A8QGSGAm3mBAF6paugGhTEUuw9CJKEWlFpetSWwu4UI2npd5oYe2aRW4mC17WRUAgCQgCoq1a21CL7ba1iqtxq0AjLYYSKEmef6Mzc27fgJUnEk6+35zP+4fJzJznTGbg48PkvDIGDjAq6O0TAD4p4oVZxAuziBdmES/MIl6YRbwwi3hhFvHCLOKFWcQLs4gXZhEvzCJemEW8MIt4YRbxwizihVnEC7OIF2YRL8wiXphFvDCLeGEW8cIs4oVZxAuziBdmES/MIl6YRbwwi3hhFvHCLOKFWcQLs4gXZhEvzCJemEW8MCu/4k0EoQFj1rpXgpdbWlqeDJrdnBujF1pfXVOYcK58VTxs6pTk8OyoUfdE757sGoLWcGVS+J7ofaPuDTeqo8G/XTjo82uPpb/NoW8G3z6ZXP2X6c5NnuPciQ37w63bhnTk7mL7vjyLd1hqZWOQ9Foc4UuV/UZMH3F6vNlRox5ycbyrzo/235CO92EXx/vW1JK5Tdnv8szw0fWp1fVl7yfjTdoSvHmOrzCv5Gm8e4JDqbUwwsOFCw6GiZ0eb3bUqBoXx7s5OBquTE7Hu9zF8f68ZOL6E5lv8syABW3p9epU/vUHmzrDzcbgwDm8uryTp/EeOm9tuHzgmjjCndF82HbR6fFmR6XirZ03YINz7xedEq/787JLKu5vTr7p5IhvZr/h67tDl16z+9CuYF24ecfFObnMfJGn8bpFRUu3f6/fCuduv6xpf+HsFxuqSq5vieKtu/W4m3ppfeR/s6PS8SbuKHu0oWrkTfH72jPxOte5fXrhrL3R2nPB9x+PtMWHikS3De1Vw5c8c2f/Tb1w0X1XvsbbWXP5wPG14crzowcf2TC2ZNLWJ0oXR/HODW8Mpib/rf9BdlQm3r/dXTFu3bTbku8T8YbeuHtp9KU2dadwID5UJL7nPVY9bvDVO3J6tX1efsV7tmrvqn0nDP/T/97bJ4II8XZHbeLiGW8e/lHRe719IogQb3fUJn55ZdBvzLO9fR6IEW931CacO9La22eBlFzEe3zL033EvJm9fQY9paE9B3/x51ou4t0zfHYfMXFsb59BTxn66xz8xZ9ruYh39zU5+CY5Ed029A2V+3r7DHoA8XYH8apCvN1BvKoQb3cQryrE2x3EqwrxdgfxqkK83XGi7cxjbCBeT30n3r6DeD0Rrz7E64l49SFeT8SrD/F6Il59iNcT8epDvJ6IVx/i9US8+hCvJ+LVh3g9Ea8+xOuJePUhXk/Eqw/xeiJefYjXE/HqQ7yeiFcf4vVEvPoQryfi1Yd4PRGvPsTriXj1IV5PxKsP8XoiXn2I1xPx6kO8nohXH+L1RLz6EK8n4tWHeD0Rrz7E64l49SFeT8SrD/F6Il59iNcT8epDvJ6IVx/i9US8+hCvJ+LVh3g9Ea8+xOuJePUhXk/Eqw/xeiJefYjXE/HqQ7yeiFcf4vVEvPoQryfi1Yd4PRGvPsTriXj1IV5PxKsP8XoiXn2I1xPx6kO8nohXH+L1RLz6EK8n4tWHeD0Rrz7E64l49SFeT8SrD/F6Il59iNcT8epDvJ6IVx/i9US8+hCvJ+LVh3g9Ea8+xOuJePUhXk/Eqw/xeiJefYjXE/HqQ7yeiFcf4vVEvPoQryfi1Yd4PRGvPsTriXj1IV5PxKsP8XoiXn2I1xPx6kO8nohXH+L1RLz6EK8n4tWHeD0Rrz7E64l49SFeT8SrD/F6Il59iNcT8epDvJ6IVx/i9US8+hCvJ+LVh3g9Ea8+xOuJePUhXk/Eqw/xeiJefYjXE/HqQ7yeiFcf4vVEvPoQryfi1Yd4PRGvPsTriXj1IV5PxKsP8XoiXn2I1xPx6kO8nohXH+L1RLz6EK8n4tWHeD0Rrz7E64l49SFeT8SrD/F6Il59iNcT8epDvJ6IVx/i9US8+hCvJ+LVh3g9Ea8+xOuJePUhXk/Eqw/xeiJefYjXE/HqQ7yeiFcf4vVEvPoQryfi1Yd4PRGvPsTriXj1IV5PxKsP8XoiXn2I1xPx6kO8nohXH+L1RLz6EK8n4tWHeD0R77mRCEIDJzwldr36C7FRvipaLgreDpcbg3ujjYqpbtCaaCWMt7Y8OX7U8n9w/F0NqZXdQVK92xJ/ndvDF/JJEa9diYGbN2+umxw0ZnfNuVG8nox3RxDVPS+4Kly+FSxxt+6K9qbijcb/o3g7JiVSa/9fG5lfvN/VDI/WXuj5a/lEiNeuxLBo+dcLZ2R3fUS8rQXfDZejr+z/J+eeCn6aeu2M8b67+otBQu5ov/JB5xao+qskXruS8bprxzv3+MSBlz3q3OTw3/S/fHjXJcWfWdKZjtdVTXRuf7BjwAbnvlt83A0LbxuO3lFY8W/Ly5PjRy27/zNDb/nglINvnzKluEu8KypPOvfV+Tm5Mk/Ea1cy3mMXTXMrCxdvr+73iPvg5q8e6LyptKbxruheIRXvooK/urqh7ZO/49xVX3ZxvF8ZMqLmS4PKk+NHVcz5yfLiO1MH7dy5KX38S2S8LUOeDZdjpl1Z8rm6HF3fGRGvXYmS+vr6R78QbD5atiTcnDc8eRtwc224Mfb7mXh3BC+42Te6xZ9ybYU/jOPdEzRW7js+Mn3bcEU46LYvxWOPrBwz7Mn08bvE+4OqcNFRVLaifm5Qk7NL/HjEa1f8acOAMWvdK8HLLS0tTwbNqXve1lfXFCYy8bYWPNhRtsq9FPzuxWBPHO+PS6N73up0vNHnENVTwsVvFw76/NpjmePLeP9UEv1YeGLD/nB525CO3F3kxyFeu1L3vNHnYEmvxTG+VNlvxPQR2Xhd1cy9we/cySErHio6FsdbfXkU77J0vA+7ON63ppbMbZLHl/EuG34ys74lePMcX5kn4rUrE++e4FBqLYzxcOGCg+GPbiLeReU/Hhl++foNN0XTaxjvQxdE8d4nP20I4/15ycT1J+TxZbyfiz6ycAebwp8DXWNw4Fxelj/itSsT76Hz1obLB66JY9wZzYttF4l4dwT//K3wy+qSivtcHO9Pgm2V+9rHnRKv+/OySyrub84eX8T7evBc9GVXsC5c3nHxub82L8RrVyZet6ho6fbv9Vvh3O2XNe0vnP1iQ1XJ9S1RvHW3Ho9ueoPoh7A3gmCnS33aMHTkiumfKo/Ht2fida5z+/TCWXvTR03GGx/ikQGt0Xp71fAlz9zZf5PTgXjtysbbWXP5wPHRhwzPjx58ZMPYkklbnyhdHMU7Nzga7q0K/hiNGl0QbUTxts4vLJ+/rTw5XsQbeuPupemjJuOND3HLFcldx6rHDb56R06uzgPx5id+MccT8epDvJ6IVx/i9fQc8Wb933wVLrilt88gtv6s/ixzEe/RpjOPyRuNlXUaLFrd22cQmf+Ns/qzzEW8EBpn9vYZKLKReE0hXkFpvF2eR0EW8QpK4+3yK/3IIl6BeG0hXkFZvPJ5lI6l4wdXbQ137ps2rGxWs3OZB1TyGPEKuuLt8jzKwuIHt90eNLhjFVevrxtxrXOZB1TyGPEKuuKVz6O8W7Ay3JgxwTUFu52rT4gHVPIY8Qq64pXPoyR/Z/mJ/idaBn/2sfeSr6ceUMljxCsoi1c8j1LbP7q73RW84/bOLA4mbBEPqOQx4hWUxSueR2kMDoYr/9OvLVwe2zltwO+zD6jkMeIVdMUrn0dpLogeQ/lapXt6zNHo/zS0I/uASh4jXkFXvPJ5lPYF59c8Oy+od38omrFt/XXlH7ydeUAljxGvoCveLs+jtP/o0kGTosf9t04sKb/+V85lHlDJY8Qr6IoXZ0K8AvHaQrwC8dpCvALx2kK8AvHaQrwC8dpCvALx2kK8AvHaQrwC8dpCvALx2kK8AvHaQrwC8dpCvALx2kK8AvHaQrwC8dpCvALx2kK8AvHaQrwC8dpCvALx2kK8AvHaQrwC8dpCvALx2kK8AvHaQrwC8dpCvALx2kK8AvHaQrwC8dpCvALx2kK8AvHaQrwC8dpCvALx2kK8AvHaQrwC8dpCvALx2kK8AvHaQrwC8dpCvALx2kK8AvHaQrwC8dpCvALx2kK8AvHaQrwC8dpCvALx2kK8AvHaQrwC8dpCvALx2kK8AvHaQrwC8dpCvALx2kK8AvHaQrwC8dpCvALx2kK8AvHaQrwC8dpCvALx2kK8AvHaQrwC8dpCvALx2kK8AvHaQrwC8dpCvALx2kK8AvHaQrwC8dpCvALx2kK8AvHaQrwC8dpCvALx2kK8AvHaQrwC8dpCvALx2kK8AvHaQrwC8dpCvALx2kK8AvHaQrwC8dpCvALx2kK8AvHaQrwC8dpCvALx2kK8AvHaQrwC8dpCvALx2kK8AvHaQrwC8dpCvALx2kK8AvHaQrwC8dpCvALx2kK8AvHaQrwC8dpCvALx2kK8AvHaQrwC8dpCvALx2kK8AvHaQrwC8dpCvALx2kK8AvHaQrwC8dpCvALx2kK8AvHaQrwC8dpCvALx2kK8AvHaQrwC8dpCvALx2kK8AvHaQrwC8dpCvALx2kK8AvHaQrwC8dpCvALx2kK8AvHa8vrq3j4DRX79X2f1duKFWcSLXBi05mP3nP6yD+JFLty662P3nP6yD+LNqY+cYcpX5fw8esuJv5265+RZHI14c+ojZ5g+HO+sCeGic+QCNyz8r/bCjYnStzvu/aeL76m+0cV7Rq2eGgyd/UFyo33x2NLrfuPch3f5Hp14c+D0+aaLU+M9w3BL1gXNzv0seDkZ71Vf39R2Z2ndpgnnpeMt/dcX/qNgYXLjO0NXr5syqNndVOp7dOLtWZmpxj0+ceBlj7rUfLNv2rCyWc1dZ5iOpeMHV211yXgzG/Hw3r2GHnTkvNXOJUYnL/zCyk73XtFTzv2xIB3vFZ3OzfxCvPGH/k+HrxStcDfX+h6deHtWZqpZWbh4e3W/R5LzzeGKq9fXjbi26wyzsPjBbbcHDcl4Mxvx9NTbV9Fzbpjm3OgHUvEucq4haA33TkrHe0+4kZgcb/x3QXT7+2F3rp14e1Z6qjlatiTcmjc8Od80Bbudq090mWHeLVgZjpgxIY43uxEN70MeK/rwl8HvU/Euc27V+dHeG9LxPuQy8S69MPWWlyp9D068PSw11bwSvNzS0vJkOA9H803L4M8+9l70qphhGoMD4dcn+p+I4s1uRMP7kMOFm+67yqXiXe7c5uBouDU5HW+Ny8S7pqgj3Hj19cOFC3wPTrw9LDXVbAySXovnG7d3ZnEwYUuXGaa2fzTF7greieLNbsTD+5Dr5ox72GXjfWvABufeL/qIeH8VNDrXVvafO4M3fY9NvDCLeGEW8cIs4oVZxAuziBdmES/MIl6YRbwwi3hhFvHCLOKFWcQLs4gXZhEvzCJemEW8MIt4YRbxwizihVnEC7OIF2YRL8wiXphFvDCLeGEW8cIs4oVZxAuziBdmES/MIl6YRbwwi3hhFvHCLOKFWcQLs4gXZhEvzCJemEW8MIt4YRbxwizihVnEC7OIF2YRL8wiXphFvDCLeGEW8cIs4oVZxAuziBdmES/MIl6YRbwwi3hhFvHCLOKFWcQLs4gXZhEvzCJemEW8MIt4YRbxwizihVnEC7OIF2YRL8wiXphFvDCLeGEW8cIs4oVZxAuziBdmES/MIl6YRbwwi3hhFvHCLOKFWcQLs4gXZhEvzCJemEW8MIt4YRbxwizihVnEC7OIF2YRL8wiXphFvDDr7yVw6CBaumIcAAAAAElFTkSuQmCC)
Back to design+implementation
We know how to build a tree. How should we measure the
error? We have some options:
- on the learning data
- with a static training/test division (e.g., 20% of the overall data
used as test)
- with a k-fold CV
- with a leave-one-out CV (LOOCV), i.e., a k-fold CV
with \(k=n\), \(n\) being the number of observations in the
available data
The easiest (in terms of effort to be realized) option is the 1st:
but we know that, when \(n_\text{min}=1\), the error on the learning
data is 0 by definition. It would hence pointless to compare a set of
0s… We need to ascertain which is the default value of \(n_\text{min}\) in tree()
, that
requires to consume the documentation: I leave this for your
enjoyment.
We go for the 2nd option. We use sample()
for shuffling
the set of row-indexes of d
and take a subset of this set
that will act as the indexes of the learning data.
indexes.learning = sample(c(1:nrow(d)))[1:(nrow(d)*0.8)]
Now we can learn the tree (let’s start with Species):
t.species = tree(Species~., d[indexes.learning,])
plot(t.species)
text(t.species)
![](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAArwAAAGwCAMAAAB8TkaXAAAC8VBMVEUAAAABAQECAgIDAwMEBAQFBQUGBgYHBwcICAgJCQkKCgoLCwsMDAwNDQ0ODg4PDw8QEBARERESEhITExMUFBQVFRUWFhYXFxcYGBgZGRkaGhobGxscHBwdHR0eHh4fHx8gICAhISEiIiIjIyMkJCQlJSUmJiYnJycoKCgpKSkqKiorKyssLCwtLS0uLi4vLy8wMDAxMTEyMjIzMzM0NDQ1NTU2NjY3Nzc4ODg5OTk6Ojo7Ozs8PDw9PT0+Pj4/Pz9AQEBBQUFCQkJDQ0NERERFRUVGRkZHR0dISEhJSUlKSkpLS0tMTExNTU1OTk5PT09QUFBRUVFSUlJTU1NUVFRVVVVWVlZXV1dYWFhZWVlaWlpbW1tcXFxdXV1eXl5fX19gYGBhYWFjY2NkZGRlZWVmZmZnZ2doaGhqampra2tsbGxtbW1ubm5vb29wcHBxcXFycnJzc3N0dHR1dXV2dnZ3d3d4eHh5eXl6enp7e3t8fHx9fX1+fn5/f3+BgYGCgoKDg4OEhISFhYWGhoaHh4eIiIiJiYmKioqLi4uMjIyNjY2Ojo6Pj4+QkJCRkZGSkpKTk5OUlJSVlZWWlpaXl5eYmJiZmZmampqbm5ucnJydnZ2enp6fn5+goKChoaGioqKjo6OkpKSlpaWmpqanp6eoqKipqamqqqqrq6usrKytra2urq6vr6+wsLCxsbGysrKzs7O0tLS1tbW2tra3t7e4uLi5ubm6urq7u7u8vLy9vb2+vr6/v7/AwMDBwcHCwsLDw8PExMTFxcXGxsbHx8fJycnKysrLy8vMzMzNzc3Ozs7Pz8/Q0NDR0dHS0tLT09PU1NTV1dXW1tbX19fY2NjZ2dna2trb29vc3Nzd3d3e3t7g4ODh4eHi4uLj4+Pk5OTl5eXm5ubn5+fo6Ojp6enq6urr6+vs7Ozt7e3u7u7v7+/w8PDx8fHy8vLz8/P09PT19fX29vb39/f4+Pj5+fn6+vr7+/v8/Pz9/f3+/v7///99cRRiAAAACXBIWXMAAA7DAAAOwwHHb6hkAAAY8ElEQVR4nO3df3wU9Z3H8W9CEkI2gcRwCYSIBWn4oUUgSlKl1TvQ8EsbRapXU640aOGK8bqIINIzWL2aUqBgkxZQqwhKjobfHlDF0/O0oVivtaABxapAAxSDCZEk37/u+53dndll95t8gc3u57v7fj4ebmZmZzMz+DKZ3e8MMg5gKBbtHQC4WIgXjIV4wViIF4yFeMFYiBeMhXjBWIgXjIV4wViIF4yFeMFYiBeMhXjBWIgXjIV4wViIF4yFeMFYiBeMhXjBWIgXjIV4wViIF4yFeMFYiBeMhXjBWIgXjIV4wViIF4yFeMFYiBeMhXjBWIgXjIV4wVjxGK+bCWmjX/BbtP/3fjN9V1pfJozr+lvJ1+UvUz79i6L0oU+ec+a3bxIPm+TmWfkF7TKEEpfxptXW1tYUsy3OorJSv+cvIF75OnW8lezftj2U9Ig9fyCtTDxW5VQLey90ryFIXMabKR+/6DfFWaQd77nA2U7jbe09Vzz+qFebd/7LQibjnT3+IvYZQojfePnNIzh/pjDtqrWcF4vf43///P4hqYMrO0LE612L56+awPpMP8F5+8IrBi6oKPW8Ln/posF97jwRvJ1DbJd4rGWHvfPzxxbKeCfe261HF0fiN97m/iV8RfLiHRUJT/ET0yYe7bgjq2rL/eyF4Hh9a/H8rHv2/jRpDudzs2o2ju5Z6nldfm7Za8tS5/ptoGPXRvnlbMNZ8fhArxbP0r3pB4tlvAUlY1zX1EToWGNaXMbrqqurW3sDqz2TXSlmZ+V4fv1PqxYzQx8MitdZK3+U+Ll86w380xSR+CdJpd7ThlHi6Rk32t/+9IqCzOfsueeT5nkmTg38JZfxtqdkL68rZ1UROdTYFpfxynf7PQrW8LfYm42Njc+xI95z3qb9q5PdQfE6a+UvkK8u5ptZk5gY64t3oZip8P2Yfm9O+nVrmn2bOv5d9j3vafJdk7gVb+uGQ2JuRu/2yBxsLIvLeDO9Ey8yj3etCN8YmZA3KS84Xmet/Ce5Fe/KXnL5VF+8P+d2vIcnuMrrnS1tzxlU551cn/2ZJ16PTayhOw8xPsR1vK+w494pEeHJ5NnHRF7B8Tpr5cvf9SLeWnZGTBSX+n3a4I33bVfh+lZ7Q9t7zG7xTVd4/xOoO1Yvzj34Fna0uw4vfsR1vMd7rhGPj4y3Itwlfxa29A+O11nLF+/hxKmcf5YSIl5+aumQ3EVHPNPn8r7rbPTgHmH4+D3Hd7N1Yva+gd18kPEgruPl81OW7JiXsJzzmVfVH0qe/vrmItfkRhlvzd1n+YThddKfnbV88fJv9Fq7uWjAHdbr2gLi5bxjx6Tk2/fJqd+xB5+RWqxvJ8nThrainMrtcxM3RvigY1F8x9tRdXXaCPkhw6uDMk5vGOoau/XZrMUy3nJxYjDB83v+IWctO95VhbnD1pXM8LzuvHiFDx5YIr9Ue88UjlrfTrLOeZsrhmVcvzNiRxvD4jHeS9Z0z70i/K88Fu39iHeI9yJ0XDa44eSjKZ9Gez/iHeK9GA/nsoSCl6O9F3EP8V6MavfppmjvA0Qk3tPzH4gx4wujvQdhtzQCIYRbJOJdmpgfY7Iyor0HYWfir+BI7POTl0VgIxFV7Y72HoTbOcQbGuKlD/EqIF76EK8C4qUP8SogXvoQrwLipQ/xKsRevK0tXa9jFsSrEHvxxh7Eq4B46UO8CoiXPsSrgHjpQ7wKiJc+xKuAeOlDvAqIlz7Eq4B46UO8CoiXPsSrgHjpQ7wKiJc+xKuAeOlDvAqIlz7Eq4B46UO8CoiXPsSrgHjpQ7wKiJc+xKuAeOlDvAqIlz7Eq4B46UO8CoiXPsSrgHjpQ7wKiJc+xKuAeOlDvAqIlz7Eq4B46UO8CoiXPsSrgHjpQ7wKiJc+xKuAeOlDvAqIlz7Eq4B46UO8CoiXPsSrgHjpQ7wKiJc+xKuAeOlDvAqIlz7Eq4B46UO8CoiXPsSrgHjpQ7wKiJc+xKuAeOlDvAqIlz7Eq4B46UO8CoiXPsSrgHjpQ7wKiJc+xKuAeOlDvAqIlz7Eq4B46UO8CoiXPsSrgHjpQ7wKiJc+xKuAeOlDvAqIlz7Eq4B46UO8CoiXPsSrgHjpQ7wKiJc+xKuAeOlDvAqIlz7Eq4B46UO8CoiXPsSrgHjpQ7wKiJc+xKuAeOlDvAqIlz7Eq4B46UO8CoiXPsSrgHjpQ7wKiJc+xKuAeOlDvAqIlz7Eq4B46UO8CoiXPsSrgHjpQ7wKiJc+xKuAeOlDvAqIlz7Eq4B46UO8CoiXPsSrgHjpQ7wKiJc+xKuAeOlDvAqIlz7Eq4B46UO8CoiXPsSrgHjpQ7wKiJc+xKuAeOlDvAqIlz7Eq4B46UO8CoiXPsSrgHjpQ7wKiJc+xKuAeOlDvAqIlz7Eq4B46UO8CoiXPsSrgHjpQ7wKiJc+xKuAeOlDvAqIlz7Eq4B46UO8CoiXPsSrgHjpQ7wKiJc+xKuAeOlDvAqIlz7Eq4B46UO8CoiXPsSrgHjpQ7wKiJc+xKuAeOlDvAqIlz7Eq4B46UO8CoiXPsSrgHjpQ7wKiJc+xKuAeOlDvAqIlz7Eq4B46UO8CoiXPsSrgHjpQ7wKiJc+xKuAeOlDvAqIlz7Eq4B46UO8CoiXPsSrgHjpQ7wKiJc+xKuAeOlDvAqIlz7Eq4B46UO8CoiXPsSrgHjpQ7wKiJc+xKuAeOlDvAqIlz7Eq4B46UO8CoiXPsSrgHjpQ7wKiJc+xKuAeOlDvAqIlz7Eq4B46UO8Cog3EtxMSBv9gt+i/b/3m+m7Uj7OZx+KxxfZQjmTO4Gnr/Y8e4719ayfv6yTbeze7J3Ywzzq+Cbra3nYDuNCIN5Y4U6rra2tKWZbnEVlpX7Pe+LdyWTds9i14vEwq+R37/Y8a8Ur1+8s3vaxbu/UX6ule1MP8aocObU3nEeiDfHGCnemfPyi3xRnUYh4m5J+KB4HjUn8G+cvsP+2n9WI9+NV32Ru/wVtYx7nfPb4sOz9RUG8scITL795BOfPFKZdtZbzYvH7/O+f3z8kdXBlhy9eXlTI+SG2s8cGzn+YepZnitOGM/fl598n4rXWz1+6aHCfO0+E2MCOceNSA+JdPvIc5xPv7fYjU0K8scITb3P/Er4iefGOioSn+IlpE4923JFVteV+ea7gjXd+0he8pk9b8fc5v/YmbsX7j72XbvymiNdaPz+37LVlqXP9vnHHro2+ySH+8Tb2flk8FpSMcV1TE4kDDIZ4Y4XbVVdXt/YGVnsmu1LMzsrxnAZMqxYzQx+0493J9vLppXzx5bwl+cdWvK/I0+Qz9mnDKLHSjBvtb3t6RUHmc76ZgHgfKhIP7SnZy+vKWVVEDvF8iDdWWJ829ChYw99ibzY2Nj7HjnjPeZv2r0522/E2JT3enr2Sv8H+8jp7xYr3iSzuf84rP4eoGOf9pu/NSb9uTbO9Df94/+aSbw1bNxwSjzN6t0fkGM+DeGOF95xXfg7m8a4V4xsjE/Im5Tnx8qJb97G/8HO9lz+Z0mzFW3E194/359yO9/AEV3m9/zb8412ac86e3sQauvHIlBBvrLDjfYUd906JGE8mzz4m3rr5xTu/7xMDxJdvTb1DFiritf71BH7a4I33bVfh+lb/bfjHe4382IIfqxfvBfkWdrS7DqsziDdW2PEe77lGPD4y3opxl/yZ2NLfL96d7Mp/EV9WuXIf5la8r7FtnJ8NFS8/tXRI7qIjzjb84j3Ifie/7GbrxON9A7v10FQQb6yw4+XzU5bsmJewnPOZV9UfSp7++uYi1+RGGW/N3WflSS+Tb8A+YGwX937a0OcXmyfKeMX6bQHxct6xY1Ly7ft8c554rW/zVI8mOd1WlFO5fW6i/XlERCHeWOHE21F1ddoI+SHDq4MyTm8Y6hq79dmsxTLecnZGLC1in8i1BiXJGRlv072X9y+X8cr1z4tX+OCBJb5JT7zWt7lzlGdRc8WwjOt3dvvRhYR4QcKFOQqIlz7Eq4B4L0jT4Sh4n0Vjq4ebLulPKhLxbrgyAhuJHYMTooFFZaNfvaQ/KRN/W8S4vHnR3oOIqbj8kl6OeMlBvLoQLzmIV1d3xxtwJwroQLy6ujvegIv5QQfi1YV4yUG8usIc7x9LMrNvl1dy+N+J0r5kREbRVr9n7VtTIATEqyu88TbnXr++Ju9mHngnypzUx7fNZJudZ+1bUyAExKsrvPHWsz2c17l5wJ0oHyetEDNTRtvPOremQAiIV1d4423M+NrTn4qvAXeieK5Ufjax1fcst29NgRAQr64wn/PuuzWVjd4UeCdKdaI8u93NPvI969yaAiEgXl1h/7SheVdJjwMBd6JsYcfExPMJLb5nnVtTIATEqyu88b5UcEb+LUI7A+5EOZIkb0C5baT9rHNrCoSAeHWFN973U6ZsW39L3xMBd6K0ze5V9fIsVmc/+6F9awqEgHh1hfm0YWuhq+/kd867E6Xt0eHpY7f4PWvfmgIhIF5duDCHHMSrC/GSg3h1IV5yEK8uxEsO4tWFeMlBvLoQLzmIVxfiJQfx6kK85CBeXYiXHMSrC/GSg3h1IV5yEK8uxEsO4tWFeMlBvLoQLzmIVxfiJQfx6kK85CBeXYiXHMSrC/GSg3h1IV5yEK8uxEsO4tWFeMlBvLoQLzmIVxfiJQfx6kK85CBeXYiXHMSrC/GSg3h1IV5yEK8uxEsO4tWFeMlBvLoQLzmIVxfiJQfx6kK85CBeXYiXHMSrC/GSg3h1IV5yEK8uxEsO4tWFeMlBvLoQLzmIVxfiJQfx6kK85CBeXYiXHMSrC/GSg3h1IV5yEK8uxEsO4tWFeMlBvLoQLzmIVxfiJQfx6kK85CBeXYiXHMSrC/GSg3h1IV5yEK8uxEsO4tWFeMlBvLoQLzmIVxfiJQfx6kK83cDNhLTRL/gt2v97v5m+K60vE8aFfHVAvPJ1+cs629j2Td6Jc0981TVm44XvbRQhXnrcabW1tTXFbIuzqKzU7/kLiFe+rtN4D6SVeacW9Xx82w/Y9ovZ4WhBvPS4M+XjF/2mOIu04z13YfF+Wch88eZViIfr7ryYHY4WxEuPJ15+8wjOnylMu2ot58XiPOLvn98/JHVwZUeIeL1r8fxVE1ifXj/kvH3hFQMXVJR6Xpe/dNHgPneeCLmp+WMLffH+w7+Lh8lTu/PAwg3x0uOJt7l/CV+RvHhHRcJT/MS0iUc77siq2nI/eyE4Xt9aPD/rnr0/ZaM5n5tVs3F0z1LP6/Jzy15bljrXbwMdu7zntnvTDxb74v3BgLcaa5KfjswhhgfipcftqqurW3sDqz2TXSlmZ+V4fv1PqxYzQx8MitdZK3+U+LmcOoB/miIS/ySp1HvaMEo8PeNG+9ufXlGQ+Zw1dWrgL7kd75fXih/TsyN1jGGBeOmxPm3oUbCGv8XebGxsfI4d8Z7zNu1fnewOitdZK3+BmE3P45tZk5gY64t3oZip8P2Yfm9O+nVrmj3Td03iTryz8p5+4ycZv4rUQYYD4qXHe87L+YvM410rwjdGJuRNyguO11kr/0luxbuyl1w+1Rfvz7kd7+EJrvJ633bWZ3/mxHuA7RKPldntkTjCMEG89NjxvsKOe6dEhCeTZx8TrQXH66yVX8WteGvZGTFRXOr3aYM33rddhetbfdup8FZfJ2d+wxrF4072QbceWnghXnrseI/3XCMeHxlvRbiLNXDe0j84XmctX7yHe2zg/LOUEPHyU0uH5C464pk+uEcYPn6P1f5bbLN4XJT6ZUQOMTwQr6Y3pw5IvfJHpztfqfPhgwCdjH2JeDdZPxLL56cs2TEvYTnnJQPrDyVPf8yzVGyl5u6zfMLwOunP3F7LFy+/L3vt5qIBd3A+86r6toB4Oe/YMSn59n32xqzTBvntJmevePnh5Mqu95wOxKvntaSJ67c90W9kS6drXUC8nQwfiHircqqFvR1VV6eNqOa8fVifjNMbhqYkz/le2pS9Yivl4sRggueX/kPct5YT75cP5A5bVzKD81cHZZw+L17hgweW2NNWvPLbfbGgwDXq1x1d7zkdiFfPpGvlO5l32OqApR1tgWt1PvYVoPOxr9nj/WY+XvVN5g5aqpI3jzdVfyT27SuPaaxtNMSrp+CfrC8LdnB7QKuB7RyYMOw/xALNsa/pJ7THvibe6zezY9y4VHfQUhURb8fAKQ0nH0359OIO1RyIV8+MhIcOeCd9A1oNrNesOnfCjznXHPtKmqM99lVQMsZ1TY2zfIg71NKQ5LUNfxjDEgpevvSjJg7x6jl5VxIbOPPFVr8BrQY2TUzMy/hcd+zr1ht0x77aU7KX15WzKvsZK96gpSF5Lsw53RSmA6cM8eo69dt//SobdNAZ0GpgtWLx/7G3uObYl3tY2U3s7rKysuzLy8oGiX/SrhYzw3LKPG4dmpT99X+2Jr/zjVLxODj5Hu9TZRkjQi0NqdeITp8Oj5kk/ttAvHo878x2957kDGg1sDfEolMiYc2xL3d6er+E4UJ6xvDhfcQ/ybli5rI0uWj4EFdi5qDhAfLZlb7JlOxQS0PKGtjp0+GR8J/R+dcQaE/FJb08XuL9E3vN+vq9y50BLc9P3vfY/+iOfbnTJ2uOfR2rlx9ZbWFHfQus04agpdGTatYtF6HFS7xns78tPyprHXGbM6DVwL4tJhamndId+xLxao597WbrxON9A+3tW/EGLY2eyMebvrrTJcFPdy1e4uUvpX79ZxtWFqf9kdsDWg0sdfb2BYkL+YfJ01/fXOSa3Mg7H/sS8WqOfbUV5VRun5u40TP2xb3xOkujLvLx3r270yXBT3ctbuLl++4sSB38nXfFlG9Aq4H9dkpmwU/Er/INQ11jtz6btZh3PvYl49Uc+2quGJZx/U7uGfvivo/K7KVRF7XThtagay/OhVpNS/zEG6yB1Xe9UoCiybEy9hWxeG8fLR46BszmmeK8oN+L7qwPfcM81hJ78EfOtC0emnXLn/zGjLqAeC9E0eRYGfuKWLzrmHgb8L/sTU+8135rY4tvmMcTr2/wR858v8+qdePSjzhjRl1AvBeiaHKsjH1FLN7TPVeJ861Bnjr7jezwG+ax4vUO/siZ9xNfEs+kLHfGjLoQz/Ge+7C165UCiHhjZOwrcue8U0s4H/SIN9753O8WJytea/Cn2Jr5TZI8/f1cXvfnHTPqQhzF+8XhSzbqpkv/Hmej/edgiVy8T6d8/gd2wBvvUu53i5MVrzX444l3ST/vS+wxoy7EUbyXJZBwRbT/HCw5r0dqSyeTNz58LffGu4z73eJkxWt9iuOJd3WK/Ch+/0FnzKgLcRRv+q+jvQfSotxo70Gk3VI2TN5B6ovXGeY5P9535N+P1ZL9M2fMqAuIN8LiL95fpfb4jDvxOsM858fLp2fX7Lqtz1+dMaMuIN4Ii794j/e4RX6x47WHeYLibXVfmXHj235jRl1AvBFGId6Q1xF4L+vodmEc5omjeCftj/YeSP91V7T3QHEdQaTiDeMwTxzFG++CryoIcH68Xax+8cI3zIN4Y5F9QYF9A6l1VcEfSzKzbz8SeB1B+5IRGUVbuSdee8Zavdt2L1zDPIg3FtkXFNg3kMqrCk7mXr++Ju/mwOsI5qQ+vm2m/Mt2ZLz2jHURQrSPokvxEm9U36NEfDd8FxQ4N5DKqwrq2R7O69wB1xF8nLRCrDFltLUXzoxcnb54iTeq71EivxveCwqcG0jlVQWNGV972nqf5HcdgeeepGcTW+VeODNydfpiO14i71EivxveCwqcG0jlVQV8362pbPSmgOsIqhPlj9jd7CO5F86MtTp5MRMvkfcoRHbDe0GBcwNpP89fTNW8q6THAf/rCLawY+Lr8wktnp+8vpl+nf7fs6iImXiJvEchshveCwqcG0hljS8VnOH8MNvpfx3BkST5Y/+2kdZeODOIN6KIvEchshu+CwrsG0hlje+nTNm2/pa+J/yvI+Cze1W9PEv+9dTyPyF7BvFGFpH3KER2w3tBgX0DqVXj1kJX38nvBF5H0Pbo8PSx8v92KOO1ZxBvZBF5j0JkN+JC7MRL5D0Kkd2IC7ETL5X3KER2Ix7EULxE3qMQ2Y14EEPxEnmPQmQ34kEMxQvxBvGCsRAvGAvxgrEQLxgL8YKxEC8YC/GCsRAvGAvxgrEQLxgL8YKxEC8YC/GCsRAvGAvxgrEQLxgL8YKxEC8YC/GCsRAvGAvxgrEQLxgL8YKxEC8YC/GCsRAvGAvxgrEQLxgL8YKxEC8YC/GCsRAvGAvxgrEQLxgL8YKxEC8YC/GCsRAvGAvxgrEQLxgL8YKxEC8YC/GCsRAvGAvxgrEQLxgL8YKxEC8YC/GCsRAvGAvxgrEQLxgL8YKxEC8YC/GCsRAvGAvxgrEQLxgL8YKxEC8YC/GCsRAvGAvxgrEQLxgL8YKxEC8YC/GCsRAvGAvxgrEQLxgL8YKxEC8YC/GCsRAvGAvxgrEQLxgL8YKxEC8YC/GCsRAvGAvxgrEQLxgL8YKxEC8YC/GCsRAvGAvxgrEQLxgL8YKxEC8YC/GCsRAvGOv/AaBmgq9u4FoLAAAAAElFTkSuQmCC)
How to measure the error?
The predict()
function takes a dataframe with possibly
new observations and predict the corresponding labels: the results is
hence a vector.
predicted.y = predict(t.species, d[-indexes.learning,], type="class")
Note that:
- the
-
preceding indexes.learning
means
“select all but those”
type="class"
is needed to obtain a vector of factors,
rather than a more complex thing: see the documentation of
predict.tree()
predict()
doesn’t cheat: even if
d[-indexes.learning,]
actually contains also the correct
\(y\) values, it is not using them
Now we can compute the classification error rate by
comparing predicted.y
against the expected \(y\):
length(which(predicted.y!=d$Species[-indexes.learning]))/length(predicted.y)
[1] 0.03333333
Note that which()
returns the indexes of a Boolean
vector elements that are true.
Another way is to “compute” the confusion matrix and
then obtaining the error from that. The confusion matrix shows the
number misclassifications, class by class:
table(predicted.y, d$Species[-indexes.learning])
predicted.y setosa versicolor virginica
setosa 8 0 0
versicolor 0 7 0
virginica 0 1 14
Given that matrix, the accuracy of classification is:
conf.matrix = table(predicted.y, d$Species[-indexes.learning])
sum(diag(conf.matrix))/sum(conf.matrix)
[1] 0.9666667
and the error rate can be computed as:
error.species = 1-sum(diag(conf.matrix))/sum(conf.matrix)
error.species
[1] 0.03333333
As a function
Out of simplicity, we might build a function that does all those
operations together, with some parameters:
computeErrorRate = function(categorical.y.name, data, learner, p.learn = 0.8, ...) {
indexes.learning = sample(c(1:nrow(data)))[1:(nrow(data)*p.learn)]
model = learner(formula(paste0(categorical.y.name,"~.")), data[indexes.learning,], ...)
predicted.y = predict(model, d[-indexes.learning, ], type="class")
length(which(predicted.y!=d[-indexes.learning, categorical.y.name]))/length(predicted.y)
}
Note that:
p.learn = 0.8
in the signature means that there is a
default value for this parameter
...
in the signature denotes “any other parameter”,
that can be then passed to other functions with ...
again
- the return statement is implicit: the returned value is the last
computed outcome just before the end
learner
here is intended to assume functions as values
(try with tree
or rpart
)
print(computeErrorRate("Species", d, tree))
[1] 0.03333333
print(computeErrorRate("Species", d, rpart))
[1] 0.03333333
A similar function could be made for doing k-fold CV or LOOCV. Do it
by yourself.
We can do nice things with such a function. For example, we can
compute the error for different values of the learning-to-test data
ratio (that 0.8 above):
ratio = seq(0.1, 0.99, 0.05)
error = ratio %>% map_dbl(function(r){computeErrorRate("Species", d, tree, r)})
tibble(ratio=ratio, error=error) %>% ggplot(aes(x=ratio,y=error))+geom_line()+ylim(0,1)
![](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAArwAAAGwCAMAAAB8TkaXAAAC+lBMVEUAAAABAQECAgIDAwMEBAQFBQUGBgYHBwcICAgJCQkKCgoLCwsMDAwNDQ0PDw8QEBARERESEhITExMUFBQVFRUWFhYXFxcYGBgZGRkaGhobGxscHBwdHR0eHh4fHx8gICAhISEiIiIjIyMkJCQlJSUmJiYnJycoKCgpKSkqKiorKyssLCwtLS0uLi4vLy8wMDAxMTEyMjIzMzM0NDQ1NTU2NjY3Nzc4ODg5OTk6Ojo7Ozs8PDw9PT0+Pj4/Pz9AQEBBQUFCQkJDQ0NERERFRUVGRkZHR0dISEhJSUlKSkpLS0tMTExNTU1OTk5PT09QUFBRUVFSUlJTU1NUVFRVVVVWVlZXV1dYWFhZWVlaWlpbW1tcXFxdXV1eXl5fX19gYGBhYWFiYmJjY2NkZGRlZWVmZmZnZ2doaGhpaWlqampra2tsbGxtbW1ubm5vb29wcHBxcXFycnJzc3N0dHR1dXV2dnZ3d3d4eHh5eXl6enp7e3t8fHx9fX1+fn5/f3+AgICBgYGCgoKDg4OEhISFhYWGhoaHh4eIiIiJiYmKioqLi4uMjIyNjY2Ojo6Pj4+QkJCRkZGSkpKTk5OUlJSVlZWWlpaXl5eYmJiZmZmampqbm5ucnJydnZ2enp6fn5+goKChoaGioqKkpKSlpaWmpqanp6eoqKipqamqqqqrq6usrKytra2urq6vr6+wsLCxsbGysrKzs7O0tLS1tbW2tra3t7e4uLi5ubm6urq7u7u8vLy9vb2+vr6/v7/AwMDBwcHCwsLDw8PExMTFxcXGxsbHx8fIyMjJycnKysrLy8vMzMzNzc3Ozs7Pz8/Q0NDR0dHS0tLT09PU1NTV1dXW1tbX19fY2NjZ2dna2trb29vc3Nzd3d3e3t7f39/g4ODh4eHi4uLj4+Pk5OTl5eXm5ubn5+fo6Ojp6enq6urr6+vs7Ozt7e3u7u7v7+/w8PDx8fHy8vLz8/P09PT19fX29vb39/f4+Pj5+fn6+vr7+/v8/Pz9/f3+/v7///9RJYlgAAAACXBIWXMAAA7DAAAOwwHHb6hkAAAYj0lEQVR4nO3deXxU5bnA8QOEiLS9gAjY6tUqLdLlogaw2AVpveitBKmlQCGQUqCghdgisXoxxJoCrTVFAlhajEuxC4VaFYJVMcpWbQmFikDksghBDJEteyYT3s/nzmSbyXknyTnJ82bm4O/3BzM5njy8xC/Dm9liKSKPZkV7AURtDbzk2cBLng285NnAS54NvOTZ2o632E2Vrs52Vml1qYGpJlZaUl1mYKqJlRZXlxsYWlEiO6+83XiL3FTl6mxnnVFnDEw1sdJT6pyBqSZWWqRKDAytOCU77xx4Iwde+cCrBV4DgVcPvPKBF7y2wCsfeLXAayDw6oFXPvCC1xZ45QOvFngNBF498MoHXvDaAq984NUCr4EuILzbfHWXe2ZPWVXTeAFel4FXdp4jvAXjy2ov/VN3V9yX23ABXreBV3aeE7w/H5tYhzcvVakt6Q0X4HUbeGXnOds2fLsO78YspY7c3XARuCUuKCg4d9pNPldnO+ucKjYw1cRKz6oSA1NNrPS0KjMwtPKM7LxiN3jXrFKqMKnhQqkTCQkJWS1+IpG5qusvHeHNWRa4yZ3VcKFU1VtvvXX0rJuqXZ3trBJVYmCqiZUWqzIDU02s9KwqNzC0qlh2XqkbvHn3K7U9veGiPle7FPa88rHndYA3v9yffLBmwSZVfwFet4FXdp4LvGP2qn0pM7PPq4YL8LoMvLLzeIStmcArH3i1wGsg8OqBVz7wgtcWeOUDrxZ4DQRePfDKB17w2gKvfODVAq+BwKsHXvnAC15b4JUPvFrgNRB49cArH3jBawu88oFXC7wGAq8eeOUDL3htgVc+8GqB10Dg1QOvfOAFry3wygdeLfAaCLx64JUPvOC1BV75wKsFXgOBVw+88oEXvLbAKx94tcBrIPDqgVc+8ILXFnjlA68WeA0EXj3wygde8NoCr3zg1QKvgcCrB175wAteW+CVD7xa4DUQePXAKx94wWsLvPKBVwu8BvqI4K1wU42rs51VpaoMTDWx0krlMzDVxEorjKzUXyk8sN14Xf2oY35wtnwe+sHZZ2XnOfvB2S3l6oaebYN8bBvAawu88oFXC7wGAq8eeOUDL3htgVc+8GqB10Dg1QOvfOAFry3wygdeLfAaCLx64JUPvOC1BV75wKsFXgOBVw+88oEXvLbAKx94tcBrIPDqgVc+8ILXFnjlA68WeA0EXj3wygde8NoCr3zg1QKvgcCrB175wAteW+CVD7xa4DUQePXAKx94wWsLvPKBVwu8BgKvHnjlAy94bYFXPvBqgddA4NUDr3zgBa8t8MoHXi3wGgi8euCVD7zgtQVe+cCrBV4DgVcPvPKBtyW8e2ZPWVUTvLI2MdhZdW/g15XgdR14Zec5wOufurvivtzgtRqfz5f3sFJJxT6fH7yuA6/sPAd481KV2pLeIHluoapIYtvQpsArO88B3o1ZSh25u/6D9U8pdThpzsSM0+B1HXhl5znAu2aVUoX1N7bVM88otX/RKX/m4sBHp+66667nfG6qcXW2s6pVtYGpZlbqNzDVxEp9nlhpZet4c5YFbnln1V1/PaP+YP74wC9nfvKTn+RUuqnG1dnOqlI+A1M/2iutVNUGhvqrZOc5+NnDefcrtb1+z5u+IQg3X6mDjfteVzf0bBvkY9vQ0r0NyQdrFmwKmC1XlWNOBA5snVpYs3Q5eF0HXtl5Tu7n3ZcyM/u8UmP2ql0/qD2wbnrykjLwug68svN4hK2ZwCsfeLXAayDw6oFXPvCC1xZ45QOvFngNBF498MoHXvDaAq984NUCr4HAqwde+cALXlvglQ+8WuA1EHj1wCsfeMFrC7zygVcLvAYCrx545QMveG2BVz7waoHXQODVA6984AWvLfDKB14t8BoIvHrglQ+84LUFXvnAqwVeA4FXD7zygRe8tsArH3i1wGsg8OqBVz7wgtcWeOUDrxZ4DQRePfDKB17w2gKvfODVAq+BwKsHXvnAC15b4JUPvFrgNRB49cArH3jBawu88oFXC7wG+ojgPeMmn6uznVWsSgxMNbHSc6rUwFQTKz2jyg0MrTonO6+k3XjL3eR3dbazKlWlgakmVlqhqgxMNbHScuUzMLS6Qnhgu/G6uqFn2yAf2wbw2gKvfODVAq+BwKsHXvnAC15b4JUPvFrgNRB49cArH3jBawu88oFXC7wGAq8eeOUDL3htgVc+8GqB10Dg1QOvfOAFry3wygdeLfAaCLx64JUPvOC1BV75wKsFXgOBVw+88oEXvLbAKx94tcBrIPDqgVc+8ILXFnjlA68WeA0EXj3wygde8NoCr3zg1QKvgTyKt2x0AXjB60286pZnwQtej+J986bFL20NBF7pwCs7LwJeqz7wSgde2Xl8w9ZM4JWvI/Cef33l8tzz4BUPvLLzIuAtGtzl6v5xQz4Er3TglZ0XAe/EG99T6uiXJoNXOvDKzouAt+/rwV+39AOvdOCVndc83r7glQ68svMibhuOBrcNk8ArHXhl50X6hm1Il/794wYXNR7YM3vKqpraa/cmJiauDD8AXjeBV3ZexLvKcleseC10V5l/6u6K+3JrryYV+3z+8APgdRN4Zec5eGJOXmpgB5wevFaRZDsAXleBV3aegyfmbMxS6sjdwWuHk+ZMzDgdOlDxyiuvHCh2U7Wrs51VpsoMTDWx0hJVbmCqiZUWqwoDQ30lsvNKdbz2J+asWaVUYe1t7v5Fp/yZi0MHTiQkJGQpouhUXX/ZwhNzcpYFbmhnNXyUPz50oObcuXOVH7qpytXZzjqrzhqYamKlp1WxgakmVvqhKjUwtPK07LxiHa+9vPuV2l67xc3PV+pgUtiBYK52Kex55WPP28I3bP7kgzULNgXolm+dWlizdHnjAfC6DLyy85y8kmJfyszs80qN2avWTU9eUtZ4ALwuA6/sPF5J0UzglY9XUmiB10BexesyV78deOUDbzjeM09kqN1+8IoHXtl5EfAe/VxfS934+SPglQ68svMi4L19kj9OFQz/FnilA6/svAh4P/EPFafUqz3AKx14ZedFwHv55iDenCvAKx14ZedFwPuDEWfi1N7P/BC80oFXdl4EvGdHdLX6dbqzDLzSgVd2XsT7ed986i/vOqQLXheBV3YeD1I0E3jlA68WeA0EXj3wygde8NoCr3zg1QKvgcCrB175wAteW+CVD7xa4DUQePXAKx94wWsLvPKBVwu8BgKvHnjlAy94bYFXPvBqgddA4NUDr3zgBa8t8MoHXi3wGgi8euCVD7zgtQVe+cCrBV4DgVcPvPKBF7y2wCsfeLXAayDw6oFXPvCC1xZ45QOvFngNBF498MoH3rbj9bnpvKuznVWtqg1MNbNSv4GpJlbqM7LSGuH/UZXc8kaOW175Yu+W19VvB175wAteW+CVD7xa4DUQePXAKx94wWsLvPKBVwu8BgKvHnjlAy94bYFXPvBqgddA4NUDr3zgBa8t8MoHXi3wGgi8euCVD7zgtQVe+cCrBV4DgVcPvPKBF7y2wCsfeLXAayDw6oFXPvCC1xZ45QOvFngNBF498MoHXvDaAq984NUCr4HAqwde+cALXlvglQ+8WuA1EHj1wCsfeMFrC7zygVcLvAYCrx545QMveG2BVz7waoHXQODVA6984AWvLfDKB14t8BoIvHrglQ+8LeHdM3vKqpraa3+fPTHtpFL3JiYmrgSv68ArO88BXv/U3RX35QavFY7Pr1j+U6WSin0+P3hdB17ZeQ7w5qUqtSU9eC13oVLHJqqKJLYNbQq8svMc4N2YpdSRu4PXKkqUej1NHU6aMzHjNHhdB17ZeQ7wrlkV2DA03NhuTt6n9i865c9cHNxGjBgx4jfn3aRcnc3QC2uodL7W8eYsC9zyzqq9Wpxxz5G6g/njA7+UPv300ztK3eR3dbazylW5gakmVlqmKgxMNbHSUlVlYGh1mey88tbx5t2v1PbaPa/vR9nng3DzlTrYuO91dUPPtkE+tg0t3duQfLBmwaaA2fI3Hqg9sHVqYc3S5eB1HXhl5zm5n3dfyszgLe6YvdmJgSYptW568pIy8LoOvLLzeIStmcArH3i1wGsg8OqBVz7wgtcWeOUDrxZ4DQRePfDKB17w2gKvfODVAq+BwKsHXvnAC15b4JUPvFrgNRB49cArH3jBawu88oFXC7wGAq8eeOUDL3htgVc+8GqB10Dg1QOvfOAFry3wygdeLfAaCLx64JUPvOC1BV75wKsFXgOBVw+88oEXvLbAKx94tcBrIPDqgVc+8ILXFnjlA68WeA0EXj3wygde8NoCr3zg1QKvgcCrB175wAteW+CVD7xa4DUQeIt27rcdAK984DWD98uXv9r0AHjlA68ZvLuui1/a5AB45QNv2/GWtFTRJGvq6bCP/S2e3bbKVLmBqSZWWqoqDEytNjCzRFUaGOorlZ3X8OPU2o63uOWWdh12IPRRdStnt6UyVWZgqomVlqhyA1NNrLRYVRgY6iuRnVfabryt3ba/0Puylxo/YNsgH9sGg3eV7bo+tPEFr3zgNXk/b8EEa8rxuqvglQ+8Zh+kyOx64zu1V8ArH3gNP8L2wqV1G1/wygde0w8P774+/rEi8IJXsg57bkPdxhe88oG3A56Yk9l16DvglQ+8HfGsshcvvWyz7OprA6+BwGtv9/UXLZFdfjDwGgi8WgXJjff4ygVeA4FXr+pncfX3+Mp1weDdsXL6DfHDf+v2bzd4O+yVFH+9JOypDiJdCHgPPzf/1t6W9anR866zes5w950BeDvuZUC7b4j/legfwet4dyyfMbSr1XXQjOV5wQ+3pfSyBi181/lU8Hbga9iOT5Td+HoY76H1aSN7WVbf0QvXh31FjmeP7HLR6LUnHU4Fb4e+AHNR3FDBjW8reE9uzRx3/V25bqeaxnsi95EJ/TtZnxg+99l8/cwdcz9p9U/f62gqeDv21cPP9+63UeyP0ALe4znpt/WyOg9M7GFdO/9frqaaxPvOMyk3dbc6Xzvpsc2FzZ37wR9u79r19t9/0PpU8HbwS9//nSC38W0G75G1qcO7WXGDZmQHbtmOrx53cWAnGeE2rrkM4T2xJXPcgOANburqVre1BzI/Z/VN+Wdrp4G3o9+3oWCClSy08Y2Ad9fKaZ/rbPUY+WDYVvLAr4Z1ih/1jNPf1ACJE3+dOyQ+8Ndp2oq3nH7K+gndO311ZUGL54C34990ZHHXIXtE/ghN8RZufmTs5ZZ1xXd+qf+TvHP+AKtH8ouOvhOSJnHkyXE9rT6JD7141N3nHcq8weoxvaUtO3ij8I456/v0y5H4I4Twnnh14ejAFnfAlLo7nSK1JfUK65MzHHz7Jkoif/no7tZ/zvhLdZseYduWcok1IK3ZLQ94o/F2T7uuj8+0n1B4YO+OLa8+v/bJVZk/T0tNmTZlzOjx01LSfrHi6bWvvvn24Ugz6/C++2zK0Hir200//uOhlpfwwZ/Hf9walNHa3R1yJLanDencOeHBre14ePj4qps7Xzzuhcj/ZIA3Ku9VFtj4jg0I/f6UO0YNHzZowJW9e3Sywure44qrBiUMH3RN3483HuvxqQEJw0d/d8aP0x5dufq5V/+572gA7+7Hk6/tZPW67aEcZzvao7+5Ja7LiBVHnK60cOeaxdNvnfX4Vgff+zetMGdOfyv+G4/WbZDa89yGnfM+ZV2TFmmjBd4ovdHe4os+1uOKTw8aPHzk6O9OmX1P2sOZy7P/sPblTTt2HTjW5MQD/9r60trsrEX/m5L87Vu/POiq3t0aOHfp0ceyrhq/ZJvTO/Vr27c4weo+9o8nWllpfk7WPaMGxgd+l6tuuizw12no9GWbm/8cWwW/n3yp1WNsduO/F+17Yk7hnxLj4/5ntfbbg9d77xL5wYG8N3LWZP8q44H5T7XpO7+35l1lXTrj5cj/8diOJ+dPGNwz8Jej95cmLXhmW/Amfe8fHrg98L1gt4TvP/ba+61Nf3fFqI9Zl09fG35iu59Vtv/hAVafkdMeyn457B0Mwes9vI214+HhnGm9rP737wg/9EHemkXTbr4isH/p/sUxc3/98v/ZPiV/zYOJV1pW/HVTMl9pdpOyc+FX4qzPz9tkOyzxlMic6TdfHfiXwLp4wH9/Pz37b/vA+1HFW1T0/uox3ToNeST4rfz+DUtTRl0bgBH36Vtm/fJvu1v4rAPr0sdc3cnq+sVJv3jpmP0/bpr3BSvuKwt36p8m9Xzek29v+PX8ySOuCSLuNuC2qQueeKn1R5IP7ty07snMB+dMHvXVL1w19R+tnA1evZjDG+jQsq91jr85oUdAQp+bJqf/7u+1N6itr/TQ8w+P/WxnK27ghMUb3qs/9v666Zdb3UetiPzomfST0U++nbNy/pRb+l8URPzZb3wvbdXGsDtRjr29+cXfLcuYO+3Orydc3atz3XcIF39y4LBvfndyz863b2hxNHj1YhFvoH8/9NVvzVv5SvidbA5XemT9ogkD46wuA76T8fzb2WN7WJdOfrbZB8SMvZJiT85vHkwe8Zkg4os+8/VJE2678dp+F9Vpjevdf8gtY6fPW7h89Yatexo2Ou8tvtIa/GQLd56AVy9G8UbIzUqPbvz5xC90DUjpPyen2WfaFHXAy4D2bPxt2ve+/tn/Gn5H8j3pS556LvdfEe8ZD/bBk4OtKxe/19x/Bq/ehYm3tuMvP769lVNi7DVs67/ZueePmnmwBrx6FzBeB8UY3qKiHTMujh8X8UVH4NUDr3ztWunBhZdZQ1frj++AVw+88rVzpceXX2t9PtP+TSZ49cArX7tXenLtSKtP6oEmx8CrB175JFaaO67rx2aEP6wCXj3wyiez0j2p/9F5ZOiFheDVA698Uis9vPBya2h2/QMXFwzePbOnrKoJvxY6AF43xTbeoqL3s2+wPr2w9skaFwpe/9TdFfflhl0LHQCvq2Idb6D1Iztdkpp/4eDNS1VqS3rYtdAB8LrKA3iLit6a0S1+3LYLBe/GLKWO3B12rfFAcVZW1rZyN/ldne2sSlVpYKqJlVaoKgNTxVd6KLVn5ztS75WvDTMfa3GlreNds0qpwqSwa40HTo4ePfoZv5vOuzrbWTWqxsBUEyv1e2Wl5x677urY6M6WllnVOt6cZYEb2llh10IH2Da4yhPbhroulG1D3v1KbU8PuxY6AF5XgVd2npN7G5IP1izYpFR+ef21hgPgdRl4Zec5uZ93X8rM7PNKjdnbcK3+ArwuA6/sPB5haybwygdeLfAaCLx64JUPvOC1BV75wKsFXgOBVw+88oEXvLbAKx94tcBrIPDqgVc+8ILXFnjlA68WeA0EXj3wygfetuONeoVZJ6K9BIeVZh2M9hIcVpP172gvwUUexvtOgle+0IUJr0V7CQ6rSfhztJfgIvB2QOA1E3g7IPCaycN4qwqqWj8pJvIXlLd+UmxUUBrtFbjIw3jpox54ybN5D2/oDdP+Pnti2kml7k1MTFwZ5UVFLLTS+iU2ea+3WKpxYWsTg52N3a9pkzyHN/SGaYXj8yuW/1SppGKfzx/lVUUq7K3d6pbY9L3eYqjQwmp8Pl/ew7H7NW2a5/CG3jAtd6FSxyaqiqRoL6mZQiutX2LT93qLoZoszD+3MHa/pk3zHN7QO6hVlCj1epo6nDRnYsbpaC8rQqGV1i8xdCDGarKw9U+p2P2aNs1zeEPvoBZoc/I+tX/RKX/m4qiuKXKhldYvscnSY6nwhVXPPKNi92vaNM/hDXvDtOKMe47UHcwfH8UVNVfTt3YLLLHpgRgqfGGvZ9QfjMmvadM8hzf0hmm+H9W+fU9+vlIHY/H2LLTS+iU2fa+3GCp8YekbVAx/TZvmObyhd1B744HaA1unFtYsXR7lVUUqtNL6JTZ9r7cYKrRSVTkm+FS9mP2aNs1zeEPvoJYdvEtyklLrpicvKYv2qiIVeq+3+iU2ea+3WCq00l0/qD0Qs1/TJnkPL1F94CXPBl7ybOAlzwZe8mzgJc8GXvJs4I1OJ0tUpbUr2qvweOCNTl95VFXPOx7tVXg88HZkoQetAnipvYG3w7LeHDlaHbijzyeG71KDLeu24Lbhw6R+l00qivbKvBp4Oyxr2OpjauCITa99bUjtLW8A7/khQ3Nzhw6J9sq8Gng7LGuRUucfPaTU73o34M3t8p5SR7q8Ee2leTTwdljWS4FfKv40/9s9G/E+3j/4H66O/dfpxmbg7bCsrUqVXDf4kRefbsS7ohZv/9h/5mxsBt4OK4j3hfjTSq1uxPta3DGljsblRntpHg28HVYQ7zbriaPrruxepL72wzPBb9gGD9u6ZdjgmHyCugcCb4cVxKsW9us1dv/AIeqJS+4M3lVWNLFvP+4qa2vgJc8GXvJs4CXPBl7ybOAlzwZe8mzgJc8GXvJs4CXPBl7ybOAlz/b/07aOhB4piCYAAAAASUVORK5CYII=)
For regression
Now, let’s do “the same” on the first numerical variable:
names(d)
[1] "Sepal.Length" "Sepal.Width" "Petal.Length" "Petal.Width" "Species"
t.sl = tree(Sepal.Length~., d[indexes.learning,])
predicted.y = predict(t.sl, d[-indexes.learning,], type="vector")
predicted.y
19 21 26 27 36 37 44 47 71 77 81 82 84 86
5.383333 4.796552 4.796552 5.318750 4.796552 4.796552 5.318750 5.383333 6.591667 6.204348 5.318750 5.318750 6.204348 6.591667
95 100 109 110 112 114 119 121 123 129 134 137 140 144
5.828571 5.828571 6.814286 7.600000 6.204348 6.204348 7.600000 6.814286 7.600000 6.204348 6.204348 6.591667 6.591667 6.814286
147 148
6.204348 6.204348
Wait! Now the \(y\) is numerical, we
cannot just check for equality, we need to compute another metric
instead of the classification error. We could go for the RSS, or the
MAE, or the MSE… Let’s go with the MAE (mean absolute
error):
error.sl = mean(abs(predicted.y-d$Sepal.Length[-indexes.learning]))
error.sl
[1] 0.2850388
Again, we can write a function for doing everything:
computeMAE = function(numerical.y.name, data, learner, p.learn = 0.8, ...) {
indexes.learning = sample(c(1:nrow(data)))[1:(nrow(data)*p.learn)]
model = learner(formula(paste0(numerical.y.name,"~.")), data[indexes.learning,], ...)
predicted.y = predict(model, d[-indexes.learning, ], type="vector")
mean(abs(predicted.y-d[-indexes.learning, numerical.y.name]))
}
Step 3: execution and answer
Since we indeed built the functions, we can now compute the errors
for all the variables:
for (variable in names(d)) {
if (is.numeric(d[,variable])) {
error = computeMAE(variable, d, tree)
} else {
error = computeErrorRate(variable, d, tree)
}
cat(paste(variable, error, "\n"))
}
Sepal.Length 0.273899831649832
Sepal.Width 0.27055765993266
Petal.Length 0.336315207780725
Petal.Width 0.133231968810916
Species 0.133333333333333
Apparently, the largest error is obtained for Petal.Length, so it is
the hardest variable to be predicted.
But, wait! Comparing an error rate (something that is, by
definition, in \([0,1]\)) against a MAE
(that is, in general, unbounded), is pointless! (As an example,
try multiplying one of the numerical variable by 10 and see that the
error will increase by 10 too!). This is intrinsically related to the
nature of the \(y\) in the two cases:
categorical and numerical. It turns out, hence, that the idea of using
“the effectiveness of a model learned with default parameter” as a proxy
for the hardness (actually, the opposite of the effectiveness), is
wrong.
How can we overcome this limitation?
Alternative step 1: re-design
Since the problem is that we cannot compare the absolute values of
the error, one possibility is to consider the complexity a model
exhibits for obtaining an error of 0. The rationale is that 0 is
comparable with 0, since they both represent the absence of any error,
regardless of the scale. We need hence a way for measuring the
complexity: for trees, this is easy, since we can simply count the
number of nodes.
So summarizing, the new method for answering would be:
- for each variable
- learn a tree that obtains a 0 error
- count the number of nodes of the tree
- say that the hardest variable is the one for which the count is the
greatest
Note that with the tree learning procedure that we know, one can
obtain a tree with error 0 by simply setting \(n_\text{min}\) (provided that the dataset
\(\{(x^{(i)},y^{(i)})\}_i\) does not
contain different observations with the same \(y\)).
Alternative step 2: implementation
We need to find out:
- how to set the \(n_\text{min}\) in
tree()
- how to count the number of nodes in a tree
By digging the documentation of tree()
we find that the
parameters for the learning procedure are described in the documentation
of the function tree.control()
. It is said that:
To produce a tree that fits the data perfectly, set
mindev = 0
and minsize = 2
, if the limit on
tree depth allows such a tree.
By reading further, we infer that minsize
is related
with \(n_\text{min}\).
Let’s try. Normal tree (default flexibility, i.e., default value for
minsize
):
t=tree(Species~., d)
plot(t)
text(t)
![](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAArwAAAGwCAMAAAB8TkaXAAAC6FBMVEUAAAABAQECAgIDAwMEBAQFBQUGBgYHBwcICAgJCQkKCgoLCwsMDAwNDQ0ODg4PDw8QEBARERESEhITExMUFBQWFhYXFxcYGBgZGRkaGhobGxscHBwdHR0eHh4fHx8gICAhISEiIiIjIyMkJCQlJSUmJiYnJycoKCgpKSkqKiorKyssLCwtLS0uLi4vLy8wMDAxMTEyMjIzMzM0NDQ1NTU2NjY3Nzc4ODg5OTk6Ojo7Ozs8PDw9PT0+Pj4/Pz9AQEBBQUFCQkJDQ0NERERFRUVGRkZHR0dISEhJSUlKSkpLS0tMTExNTU1OTk5PT09QUFBRUVFSUlJTU1NUVFRVVVVWVlZXV1dYWFhZWVlaWlpbW1tcXFxdXV1eXl5fX19gYGBhYWFiYmJjY2NkZGRlZWVmZmZnZ2dqampra2tsbGxtbW1ubm5vb29wcHBxcXFycnJzc3N0dHR1dXV2dnZ3d3d4eHh5eXl6enp7e3t8fHx9fX1+fn5/f3+CgoKDg4OEhISFhYWGhoaHh4eIiIiJiYmKioqLi4uMjIyNjY2Ojo6Pj4+QkJCRkZGSkpKTk5OUlJSVlZWWlpaXl5eYmJiZmZmampqcnJydnZ2enp6fn5+goKChoaGioqKjo6OkpKSlpaWmpqanp6eoqKipqamqqqqrq6usrKytra2urq6vr6+wsLCxsbGysrKzs7O0tLS1tbW2tra3t7e4uLi6urq7u7u8vLy9vb2+vr6/v7/AwMDBwcHCwsLDw8PExMTFxcXGxsbHx8fIyMjJycnKysrLy8vMzMzNzc3Ozs7Pz8/Q0NDR0dHS0tLT09PU1NTV1dXW1tbX19fY2NjZ2dna2trb29vc3Nzd3d3e3t7g4ODh4eHi4uLj4+Pk5OTl5eXm5ubn5+fo6Ojp6enq6urr6+vs7Ozt7e3u7u7v7+/w8PDx8fHy8vLz8/P09PT19fX29vb39/f4+Pj5+fn6+vr7+/v8/Pz9/f3+/v7////wKT9iAAAACXBIWXMAAA7DAAAOwwHHb6hkAAAaD0lEQVR4nO3df2AU5Z3H8WeT3RDyAxLzA0JEoVwgYFEIkhx6ei1oiChFkerVyNVrULhiaFdRqlTo2daUAgUlB1RFfig5moRfAlU4PQ8tFPVaKzVgsVWgAYRgQiDJ8+89z8zszC4bQn488919ls/rj83M7Gxmh3mbncnMRMYBNMUi/QYAugvxgrYQL2gL8YK2EC9oC/GCthAvaAvxgrYQL2gL8YK2EC9oC/GCthAvaAvxgrYQL2gL8YK2EC9oC/GCthAvaAvxgrYQL2gL8YK2EC9oC/GCthAvaAvxgrYQL2gL8YK2EC9oC/GCthAvaAvxgrYQL2grVuP1MyFp1LqgSQd+FzSSucz4MuHmy38r+brcxZd8+leFKUOfu+CMb90kHjbJxbOyLr1l6KqYjTepqqqqsojVOpNKpwQ934V45esuHe8C9oMtT3iftsc/TioVjxVZK4TdXX3X0CUxG2+afPyq3yRnUqfjvRA62mG8zX1mi8cf9m6xxs8XMBnvzPHdeM/QRbEdL79tOOcvFiSNWM15kfgc//LMo0MSBy9oaydeay6eu3wC6zvtBOet864Z+GT5FPN1uYueGtz33hPhyznEdojHKnbYGp87tkDGO/FhV9cODLEdb2P/Yr7UN39bued5fmLqxKNt96RX1D7K1oXHG5iL56Y/sPvn3lmcz06v3Diq1xTzdbnZpXsWJ84OWkDbjo3yy7m6c+JxTu8mc+rulINFMt684tHJ11cSresVK2bjTa6url59E6s6m7FAjM7IMj/+p64QI0MfD4vXmSv3BvFz+a6b+OcJIvG/eadYuw03iKen32p/+9NL89LW2GOveB8zB04NfIHLeFsTMpZUl7EKklW9csVsvPJoPz5vFX+XvVNfX7+GHbH2eRsOrPT5w+J15sp9Ur66iNewBjEwNhDvPDFSHvgx/dGslBtXNQYWdfxB9l1rN/m+Em7E27zhkBib3qeVYlWvXDEbb5o18CozfWhE+PZIT05JTni8zly5z3Ej3mW95fQ7A/H+ktvxHp6QXLbPWdLWrEHV1uD6jC/MeE2bWJ2LawixH+8b7Lg1JCI86Zt5TOQVHq8zV678rPcXiWOws2KgaErQbxuseN9LLljfbC9oa/zMpsBwufWfQPWxfWLfg9eyoy6tHRhiPt7jvVaJx6fHGxHukD8Lm/qHx+vMFYj3cNydnH+R0E68/NSiIdlPHTGHL+Q86Cz04C4hf/yu4zvZWjH6yEB31/GKF/Px8rkJC7c95lnC+UMj9h3yTXurpjD5jnoZb+X95/iE/Grpj85cgXj5P/VeXVM44B7jdS0h8XLetq3Ed/d+OfRb9viLUpPx7SS529BSmLVg6+y4jaSrfOWJ/XjbKq5LGi5/yfDmoNTTG4Ymj938Uvp8GW+Z2DGYYH7OP+HMZce7vCB72Nri6ebrLopX+GTOQvllhbWncNT4dpKxz9tYPix13Haidb1ixWq8PdbwwMMi/Gt/Eun3AZeGeC+h7arBdSefSfg80u8DLg3xXsqPspkn7/VIvwvoAOK9lBX+0w2Rfg/QIYp4T8+do6HxBZF+B93xMsH2jBoU8S6Ky9VQemqk30E3ZPYi2J5RgyLe564iWIhyK/yRfgfdUIt4FUO8ZBCvaoiXDOJVDfGSQbyqIV4yiFc1xEsG8aqmZ7zNTZefJ+ogXtX0jFdLiFc1xEsG8aqGeMkgXtUQLxnEqxriJYN4VUO8ZBCvaoiXDOJVDfGSQbyqIV4yiFc1xEsG8aqGeMkgXtUQLxnEqxriJYN4VUO8ZBCvaoiXDOJVDfGSQbyqIV4yiFc1xEsG8aqGeMkgXtUQLxnEqxriJYN4VUO8ZBCvaoiXDOJVDfGSQbyqIV4yiFc1xEsG8aqGeMkgXtUQLxnEqxriJYN4VUO8ZBCvaoiXDOJVDfGSQbyqIV4yiFc1xEsG8aqGeMkgXtUQLxnEqxriJYN4VUO8ZBCvaoiXDOJVDfGSQbyqIV4yiFc1xEsG8aqGeMkgXtUQLxnEqxriJYN4VUO8ZBCvaoiXDOJVDfGSQbyqIV4yiFc1xEsG8aqGeMkgXtUQLxnEqxriJYN4VUO8ZBCvaoiXDOJVDfGSQbyqIV4yiFc1xEsG8aqGeMkgXtUQLxnEqxriJYN4VUO8ZBCvaoiXDOJVDfGSQbyqIV4yiFc1xEsG8aqGeMkgXtUQLxnEqxriJYN4VUO8ZBCvaoiXDOJVDfGSQbyqIV4yiFc1xEsG8aqGeMkgXtUQLxnEqxriJYN4VUO8ZBCvaoiXDOJVDfGSQbyqIV4yiFc1xEsG8aqGeMkgXtUQLxnEqxriJYN4VUO8ZBCvaoiXDOJVDfGSQbyqIV4yiFc1xEsG8aqGeMkgXtUQLxnEqxriJYN4VUO8ZBCvaoiXDOJVDfGSQbyqIV4yiFc1xEsG8aqGeMkgXtUQLxnEqxriJYN4VUO8ZBCvaoiXDOJVDfGSQbyqIV4yiFc1xEsG8aqGeMkgXtUQLxnEqxriJYN4VUO8ZBCvaoiXDOJVDfGSQbyqIV4yiFc1xEsG8aqGeMkgXtUQLxnEqxriJYN4VUO8ZBCvaoiXDOJVDfGSQbyqIV4yiFc1xEsG8aqGeMkgXtUQLxnEqxriJYN4VUO8ZBCvaoiXDOJVDfGSQbyqIV4yiFc1xEsG8aqGeMkgXtUQLxnEqxriJYN4VUO8ZBCvaoiXDOJVDfGSQbyqIV4yiFc1xEsG8aqGeMkgXtUQLxnEqxriJYN4VUO8ZBCvaoiXDOJVDfGSQbyqIV4yiFc1xEsG8aqGeLvAz4SkUeuCJh34XdBI5jL5OJd9Kh5fZfPkSPYEnrLSfLbWm2nOn7u4g2XsrLEGdjFTNd9kfC1TtRY0EG+U8SdVVVVVFrFaZ1LplKDnzXi3M1n3DDZGPB5mC/j9O81njXjl/B3F2zrWbw39dYX0cOIhXpElh3arXBP3Id4o40+Tj1/1m+RMaifeBu/3xeOg0XF/53wd+2/72U7E+9nyW5g/eELL6Gc5nzleybunhXijjBkvv2045y8WJI1YzXmR+Dz/8syjQxIHL2gLxMsLCzg/xLbHb+D8+4nneJrYbTj7SG5uiYjXmD930VOD+957op0FbLv55sSQeJeMvMD5xIddXzP1EG+UMeNt7F/Ml/rmbyv3PM9PTJ14tO2e9IraR+W+ghXvXO9XvLJvS9H3OB/zz9yI9xt9Fm28jmWa8+dml+5ZnDg76Bu37dgYGBwSHG99n9fFY17x6OTrKylWUCHEG2X8ydXV1atvYlVnMxaI0RlZ5m7A1BViZOjjdrzb2W4+bQqffzVv8v3YiPcNuZv8X57AbsMNYqbpt9rf9vTSvLQ1gZGQeJ8oFA+tCRlLqstYBckqKoN4o4zx24b4vFX8XfZOfX39GnbE2udtOLDS57fjbfA+25qxjL/N/vQWe8OI96fpYnJtfCDeeWKs/Gbrm340K+XGVY32MoLj/XuyPDRs3nBIPE7v00qwhuog3ihj7fPK34OZPjRifHukJ6ckx4mXF961n/2JX+iz5LmERiPe8ut48AHbL7kd7+EJyWX7gpcRHO+irAv28CZW5+KaqYd4o4wd7xvsuDUkYjzpm3lMHLoFxTs386cDxJdv3XmPLFTEa/wrOz955W8brHjfSy5Y3xy8jOB4r5e/tuDH9rXJV7Ojbq2WKxBvlLHjPd5rlXh8erwR4w75M7Gpf1C829nX/lV8WZ6c/SNuxLuHbeG82tNOvPzUoiHZTx1xlhEU70H2W/llJ1srHh8Z6O66qYZ4o4wdL5+bsHDbY54lnD80Yt8h37S3agqT76iX8Vbef07u9DJ5APYJYzu49duGvr+qKZDxivlbQuLlvG1bie/u/YExM17j2zwf3yCHWwqzFmydHWf/PkIPiDfKOPG2VVyXNFz+kuHNQamnNwxNHrv5pfT5Mt4ydlZMLWR/k3MN8soRGW/Dw1f3L/ZlmvNfFK/wyZyFgUEzXuPb3HuDOamxfFjquO2ur51aiDem4MIc1RAvGcSr2pUXb3PFzyLju94ILfhnr0Xgn5ki3j2TCRYSVTZ4roqMNF+EFtzHF4F/Zop4rzxrkyL9Dqjt8UZgoYjXDYiXBOJ1A+Il4Xa8IbewXDEQLwm34w25C+CKgXhJIF43IF4SiuP9oDgt4255CUjwLSytC4enFm4Oeta+pyVWIV4SauNtzB63vjLnNh56C8usxGe3PMRqnGfte1piFeIloTbefWwX59V+HnILy2fepWJk0ij7WeeelliFeEmojbc+9eu//lx8DbmFxbzE+aW45sCz3L6nJVYhXhKK93n335XIRm0KvYVlRZzcu93J/hJ41rmnJVYhXhLKf9vQuKM4/uOQW1hq2TEx8IqnKfCsc09LrEK8JNTG+1reWfnnh7aH3MJyxCvvXJk80n7WuaclViFeEmrj/XPCpC3rb888EXILS8vM3hWvz2DV9rOf2ve0xCrES0LxbsPmguTMO96/6BaWlmfyU8bWBj1r39MSqxAvCVyY4wbESwLxugHxkkC8bkC8JBCvGxAvCcTrBsRLAvG6AfGSQLxuQLwkEK8bEC8JxOsGxEsC8boB8ZJAvG5AvCQQrxsQLwnE6wbESwLxugHxkkC8bkC8JBCvGxAvCcTrBsRLAvG6AfGSQLxuQLwkEK8bEC8JxOsGxEsC8boB8ZJAvG5AvCQQrxsQLwnE6wbESwLxugHxkkC8bkC8JBCvGxAvCcTrBsRLAvG6AfGSQLxuQLwkEK8bEC8JxOsGxEsC8boB8ZJAvG5AvCQQrxsQLwnE6wbESwLxugHxkkC8bkC8JBCvGxAvCcTrBsRLAvG6AfGSQLxuQLwkEK8bEC8JxOsGxEsC8boB8ZJAvG5AvCQQrxsQLwnE6wbESwLxugHxkkC8bkC8JBBvOD8TkkatC5p04HdBI5nLjC8Tbr7kd7Djla/LXdzRwrZusgbOPzOs7/j3xMAmuXhW1uW33QM9X2M7Xso1Rrzh/ElVVVWVRazWmVQ6Jej5LsQrX9fhpvw4qdQaejBt8ZYHkj/kvCJrhbC7O++8u3q+xna8lGuMeMP50+TjV/0mOZM6vSkvGI+djvd8AbM25Udsg3i87UHOZ47v3tvugZ6vcafjVbnGiDecuSn5bcM5f7EgacRqzovEp9qXZx4dkjh4QVs7m9Kai+cun8D6TjvB+Su+awY+WT7FfF3uoqcG9733RLuLmju2wNqU69hR8fiLPpxPfNjVtWtPz9f4zbgIrDHiDWduysb+xXypb/62cs/z/MTUiUfb7kmvqH2UrQvflIG5eG76A7t/7p3FebGncuOoXlPM1+Vml+5ZnDg7aAFtOzaaA7tTDhZZm3Ine1c8fp+d5XnFo5Ovr6RZVUvP13gqi8AaI95w/uTq6urVN7GqsxkLxOiMLPPDcOoKMTL08bBN6cyVe4P4KXXXTfxzby/O/+adYn2I3iCenn6r/e1PL81LW2MMnRr4Ag9syq+uLfzozJoEdqQ1IWNJdRmrIFtdrmKNffERWGPEG8449o7PW8XfZe/U19evYUesPcCGAyt9/rBN6cyV+6R8dRGvYXKfd2xgU84TI+WBH1ofzUq5cVWjOXxfCbc3JT+Qz9g1P2BfNW84JMam92klWVeTgjWW+7zUa4x4w1l7gJy/ykwfGpvk7ZGenJKc8E3pzJX7HDc25bIEGe+dgU35S25vysMTksv2BZazPuOLoE3J2z75Y+uiVGtkE6tzdyVD9HyNe8l4qdcY8YazN+Ub7Lg1JDbJSd/MY+JfPnxTOnPlyk8+fxGvMn7yFk0JOva2NuV7yQXrmwPLKbcaqJYj5//QIB6/M44f2yc+iXmtcTRDRcEay3ip1xjxhrM35fFeq8Tj0+ONTbJD/mRo6h++KZ25ApvycJzY5/0ioZ1NyU8tGpL91BFz+OAuIX/8LqOE8ylzOD+W+oI4jlkrRh8ZSLGmAQrWOD4Ca4x4w9mbks9NWLjtMc8Szh8ase+Qb9pbNYXJd9TLTVl5/zk+Ib9a+qMzV2BT8vGe1TWFA+4xXtcSsinFZ+W2Et/d++2FGR+i8tvNTnphw+gRX/KWwqwFW2fHbaRbXxVrPJlFYI0RbzhnU7ZVXJc0XB5yvzko9fSGocljN7+UPl9uyjJ2lk8wPwKfcOayN+XL3uxha4unm6+7aFMKn8xZaA8bm1J+u6byfgMe/LsYaSwfljpuO8WK2nq+xr+Ni8AaI14XNPxbb5HBtT+J9Pug0/BDbwTWGPG6oC0jvu7kMwmfR/p90GnL9kRgjfWP9507ByR+7YenO57p8leW2Dq+LOriC6B21rQ39SdxzJP3+uWXFTtWeiKwxtrHu8c7cf2Wn/Yb2dThXF2It+MrSy66AKp1rL+dqXxt0umGyy8pluzxRmCNtY+3ZIw8MfM+Wxkyta0ldK7LXxZl6zjekAugPlt+C/OHTeW4GJ2I9vHmfdP48uQ2bl/rVMe2D/QM+5mY0NnLor6Zn5+f6fNlXJWa31t8/g/1ZWf64vrk5YdLSQsauTopyZMRNlUYENfOS3tkOd2/6De78/6u8XRrtcp79E61j3e654mPrcHAtU51rPeMar/nx5x39rIoT0npsITCW66Kv7r02wNz7i1NShx0+5j4YaVBxt9ifOmTc5U3vdCZnDq8van3jSlVK+sbdP+icZO78Qb/ZXR3Viv/6h69U+3jPXmflw186NXmoGud6thUMfBY6plOXxbl2ft5wrrOXBYVdgHUEH97U5W7hTLeg2SLKr/C4+X81G/+/R/YoIPOtU51rEpM/j/jctHOXRbl2VvD5PHGZS+LCrsAyojX/QvBEG97tI/XPDLb2afEudapjr0tJp0SCXf2sijP3mW95fTLXhZlCroAyog3bKpylPHGHyFb1BPX9ujlusf7B7bH+Prdq51rncyfvB+x/+n0ZVGevVXsLO/EZVFhF0AZ8bp/IRhlvJ/SLepCz85q6B7vuYxvyw/r5uGTnWud6ti3xcC8pFOdvizKs/dw/IbOXBYVdgGUEa/7F4K5FW/Kyg6nhD8dVYvSPV7+WuI//mLDsqKkD5wroupY4sytT8bN45929rIoz17+SEZnLotyLoCS345b8bp/IZhb8d6/s8Mp4U9H1aK0j5fvvzcvcfB3PuTOFVF17DeT0vL+Q3yUd/ayKBHv+TmduizKvgCqzNjPsPZ5Xb8QzOXdhubzF0+50N5s0bYo/eMNV8f2XX6mEPF7G1b8JZovBFMc792jxEPbgJk8TXxY93vVn/5p6zzz1nVjin1DuxxpmT80/fY/BJ3wiZ5FIV4pfm/bwEnRfCGY4njXMrEP/7/sHbOoMd/a2DQ73bx13SwqcEO7HPle3+Vrb0454pzwiZ5FIV4pfi///ehovhBMcbyney0XO0uDzGT6jWzjzjkaoyjrhnY58ue418QzCUucEz7Rs6hYjPfCp82XnymEiFf8M0fxhWCq93nvLOZ80NNWUXM5d87RGEVZN7TLkZe9cp/0jLxozzrhEz2LisV4u86IN5qpjvfXCWd+zz62ilrEuXOOxijKuqFdjizsZ73EPuETPYuKtXgrSrrDM647r3qEbrVUx3vSt/FHY7hV1GLOnXM0RlGBO9PEyMoE+Xv0AwedEz7Rs6hYizcnp6gbsm7sxotGxtGt1strFX/D20uHyVPggaKcczQXF/W+/LunTRm/cE74RM+iYi7ex8gW9QFhvMr9Z2L8F9wpyjlHc3FRfFpG5Y7Jff/qnPCJnkUh3m5zK16SU7bH42+XX+yi7HM0YUU1+7+Weut7QSd8omdRiLfb3IqX8JStjfAcjcJFxVq8D9Refh5FThW5+u0JT9mKmOjO0ShcVKzFqy/CU7btIDxHo25RiDdaEJ6ybRfhORpVi0K80YLwlG2siKl42z0Qt65Dj/5lEZ6yjRExFW+7B+Iuxat+WYSnbGNEDMQbflge4uKgLjN75JZFeMo2RugVr31Ebv/FG+Ow/IPitIy7j4QeiLcuHJ5auJmbQdkjxuzRtywT4Snb2KBXvPYRuf0Xb+Rh+cnscesrc24LPRCflfjslodYjRmUPWIcxUffskyEp2xjg17xBo7Inb94Iw/L97FdnFf7Qw7EP/MuFXNMGmUE5YzI2aNwWSbCU7axQa94A0fkzl+8kYfl9alf/7VxwiboQNz8IwovxTXLoJwROXs0Lqs9UX5bXRTQLF7riNz5izfysJzvvyuRjdoUciC+Ik7+2NvJ/iKDckaM2aNwWe2J8tvqooBm8VpH5M5fvOln/iXdxh3F8R8HH4jXMnEczl/xNJk/DQMj/Tr4/5FHclntiu7b6qKAZvFaR+TOX7yRhbyWd1Ycm7PtwQfiR7zyt1aTRxr7oc5Il4KiXFb7ovm2uiigW7zWEbn9F29kIX9OmLRl/e2ZJ4IPxPnM3hWvz5D/q0X5GwB7pEtBUS4LukG3eK0jcvsv3hiFbC5Izrzj/dAD8ZZn8lPGyusjZVD2SJeColwWdINu8QLYEC9oC/GCthAvaAvxgrYQL2gL8YK2EC9oC/GCthAvaAvxgrYQL2gL8YK2EC9oC/GCthAvaAvxgrYQL2gL8YK2EC9oC/GCthAvaAvxgrYQL2gL8YK2EC9oC/GCthAvaAvxgrYQL2gL8YK2EC9oC/GCthAvaAvxgrYQL2gL8YK2EC9oC/GCthAvaAvxgrYQL2gL8YK2EC9oC/GCthAvaAvxgrYQL2gL8YK2EC9oC/GCthAvaAvxgrYQL2gL8YK2EC9oC/GCthAvaAvxgrYQL2gL8YK2EC9oC/GCthAvaAvxgrYQL2gL8YK2EC9oC/GCthAvaAvxgrYQL2gL8YK2EC9oC/GCthAvaAvxgrYQL2gL8YK2EC9oC/GCthAvaAvxgrYQL2gL8YK2EC9oC/GCthAvaAvxgrYQL2gL8YK2EC9oC/GCtv4f+nDlUbnSMK0AAAAASUVORK5CYII=)
Complex tree:
t=tree(Species~., d, mindev=0, minsize=2)
plot(t)
text(t)
![](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAArwAAAGwCAMAAAB8TkaXAAAC91BMVEUAAAABAQECAgIDAwMEBAQFBQUGBgYHBwcICAgJCQkKCgoLCwsMDAwNDQ0ODg4PDw8QEBARERESEhITExMUFBQVFRUWFhYXFxcYGBgZGRkaGhobGxscHBwdHR0eHh4fHx8gICAhISEiIiIjIyMkJCQlJSUmJiYnJycoKCgpKSkqKiorKyssLCwtLS0uLi4vLy8wMDAxMTEyMjIzMzM0NDQ1NTU2NjY3Nzc4ODg5OTk6Ojo7Ozs8PDw9PT0+Pj4/Pz9AQEBBQUFCQkJDQ0NERERFRUVGRkZHR0dISEhJSUlKSkpLS0tMTExNTU1OTk5PT09QUFBRUVFSUlJTU1NUVFRVVVVWVlZXV1dYWFhZWVlaWlpbW1tcXFxdXV1eXl5fX19gYGBhYWFiYmJjY2NkZGRlZWVmZmZnZ2doaGhpaWlqampra2tsbGxtbW1ubm5vb29wcHBxcXFycnJzc3N0dHR1dXV2dnZ3d3d4eHh5eXl6enp7e3t8fHx9fX1+fn5/f3+BgYGCgoKDg4OEhISFhYWGhoaHh4eIiIiJiYmKioqLi4uMjIyNjY2Ojo6Pj4+QkJCRkZGSkpKTk5OUlJSVlZWWlpaXl5eYmJiZmZmampqbm5ucnJydnZ2enp6fn5+goKChoaGioqKjo6OkpKSlpaWmpqanp6eoqKipqamqqqqrq6usrKytra2urq6vr6+wsLCxsbGysrKzs7O0tLS1tbW2tra3t7e4uLi6urq7u7u8vLy9vb2+vr6/v7/AwMDBwcHCwsLDw8PExMTFxcXGxsbHx8fIyMjJycnKysrLy8vMzMzNzc3Ozs7Pz8/Q0NDR0dHS0tLT09PU1NTV1dXW1tbX19fY2NjZ2dna2trb29vc3Nzd3d3e3t7g4ODh4eHi4uLj4+Pk5OTl5eXm5ubn5+fo6Ojp6enq6urr6+vs7Ozt7e3u7u7v7+/w8PDx8fHy8vLz8/P09PT19fX29vb39/f4+Pj5+fn6+vr7+/v8/Pz9/f3+/v7////pFW7sAAAACXBIWXMAAA7DAAAOwwHHb6hkAAAgAElEQVR4nO2cbWAU1b3GTzabEHbzyqYJhAUNYngtb0GSItUqIARQo4jaEmltQSxUTNogikgrVG2hFKgoaQGxVEDJpQGBCNZC5XKvFi+tbX0FLG0VMEgxCASSnA/3nJndmRM22c1Ospv5b5/fh9mZ2bMzZ5797cycmTPLOABEYR1dAQCsAnkBWSAvIAvkBWSBvIAskBeQBfICskBeQBbIC8gCeQFZIC8gC+QFZIG8gCyQF5AF8gKyQF5AFsgLyAJ5AVkgLyAL5AVkgbyALJAXkAXyArJAXkAWyAvIAnkBWSAvIAvkBWSBvIAskBeQBfICskBeQBbIC8gSK/KWM4Fr6EZl1qE/KhOZT2svY0aFXpT8nHd5i2//oiC5z5JL5vTOrWKwVa6eTQ+nxqDNxIy8rsrKyopCtt2cVVKsvB+GvPJzLcu7iH1/x8POx4zp91wlYrg0a7Vgb3h1Bm0kZuRNl8Mvuk40Z7Va3ktNJ4PKW5c6Rwx/0LneN30xn5WIl1mjw64xaDOxJS8f25/z9fmuAes4LxTH8X9//mDvpF6LGpuR11eKe1eNYWlTTnHeMP+Kno+UFuuf8y5b0CvtjlOB6znCdothJTvqm543Ir9EvIy/L4LbBlogtuQ9120cX5mwcFdp3DP81OTxxxtvz1i6/UG2MVBefynuzZi696fO2ZzPyajYMrRTsf45b3bJvuVJc5QVNO7eIl8uHL4ghmWdz+tz9ya/X1giXvPGDXMProjKlgKDmJHXXVVVte5aVnnWs0hMzsjSD/+TV4uJPg8FyGuW8g4R++Wbr+UfJwrF/+Us9p02DBFvT7veWPyZlXnpG4yp3zjn6iOnez7LpbwNiZ4VVdPZ0ihsKDCJGXllaz8+by1/gx2oqanZwI75znlrD61JKA+Q1yzlfUR+upBvY7ViZIRf3vliotRXmL8zO/matef8qzp5D/uW7zT5riKuyVu3+YiYmpbaEIUtBQYxI2+6b+RFpvO2JuH+QXE5RTmB8pqlvEu4Ju/TneX8SX55f84NeY+OcU8/aK5pZ1ZulW90k+cTXV6drexwxLYPNEPMyfsaO+kbExJ+ljDrhNArUF6zlFce64W8leysGCksVq42+OR9052/qc5Y0c74Wef946W+n0DViYPi3INvZ8cjs3GgeWJO3pOd1orhY6M1CXfLfeH5boHymqX88h51TOL8k8Rm5OWnl/XOXnBMH7+Uc4+50vdfFfQb/erJPewFMTmzZ0Q3EVxOzMnL5yUu3jU3bgXn9w44eCRhyuvbCtwTaqS8FXdf4GP6VUn+Zpbyy8u/2nndtoLut2ufq28iL+eNu4oSbntLjv2OPbRecl5bnESeNtQXZC3aOcexJaqbDGJP3salA1395UWG3+emnNncxz3i5eczFkp5p4sTgzH6cf5hs5Qh76r87L4vjJumf+4yeQUfli2WL6t9ZwrHtcVJtHPec6V9U0ZWR2lbgY9YkbfN1E69T4h/5Y87uh6g9UBeH41deh3+7PHEjzu6HqD1QF4/j2azuLxXOroWIAwgr5/V5WdqO7oOICyiIe+ZeWUEGJ3f0TVoFVWh8/6PIRryLnN4CZCR0tE1aA3pXaLwhVEhGvIuIRH46vKOrkFroJFllIC8fiAvOSCvH8hLDsjrB/KSA/L6gbzkgLx+IC85IK+fuvOhy3Q8NLKMEpCXFshSAfLSAlkqQF5aIEsFyEsLZKkAeWmBLBUgLy2QpQLkpQWyVIC8tECWCpCXFshSAfLSAlkqQF5aIEsFyEsLZKkAeWmBLBUgLy2QpQLkpQWyVIC8tECWCpCXFshSAfLSAlkqQF5aIEsFyEsLZKkAeWmBLBUgLy2QpQLkpQWyVIC8tECWCpCXFshSAfLSAlkqQF5aIEsFyEsLZKkAeWmBLBUgLy2QpQLkpQWyVIC8tECWCpCXFshSAfLSAlkqQF5aIEsFyEsLZKkAeWmBLBUgLy2QpQLkpQWyVIC8tECWCpCXFshSAfLSAlkqQF5aIEsFyEsLZKkAeWmBLBUgLy2QpQLkpQWyVIC8tECWCpCXFshSAfLSAlkqQF5aIEsFyEsLZKkAeWmBLBUgLy2QpQLkpQWyVIC8tECWCpCXFshSAfLSAlkqQF5aIEsFyEsLZKkAeWmBLBUgLy2QpQLkpQWyVIC8tECWCpCXFshSAfLSAlkqQF5aIEsFyEsLZKkAeWmBLBUgLy2QpQLkpQWyVIC8tECWCpCXFshSAfLSAlkqQF5aIEsFyEsLZKkAeWmBLBUgLy2QpQLkpQWyVIC8tECWCpCXFshSAfLSAlkqQF5aIEsFyEsLZKkAeWmBLBUgLy2QpQLkpQWyVIC8tECWCpCXFshSAfLSAlkqQF5aIEsFyEsLZKkAeWmBLBUgLy2QpQLkpQWyVIC8tECWCpCXFshSAfLSAlkqQF5aIEsFyEsLZKkAeWmBLBUgLy2QpQLkpQWyVIC8tECWCpCXFshSAfLSAlkqQF5aIEsFyEsLZKkAeWmBLBUgLy2QpQLkpQWyVIC8tECWCpCXFshSAfLSAlkqQF5aIEsFyEsLZKkAeWmBLBUgLy2QpQLkpQWyVIC8tECWCpCXFshSAfLSAlkqQF5aIEsFyEsLZKkAeWmBLBUgLy2QpQLkpQWyVIC8tECWCpCXFshSAfLSAlkqQF5aIEsFyEsLZKkAeWmBLBUgLy2QpQLkpQWyVIC8tECWCpCXFshSAfLSAlkqQF5aIEsFyEsLZKkAeWmBLBUgLy2QpQLkpQWyVIC80aacCVxDNyqzDv1Rmch8Wg7nsY/E8EU2X05kj+HJa/R3l7gz9fLe5UHWsWebb+RVplPFt2qv09ttM+wA5I025a7KysqKQrbdnFVSrLyvy1vNpN0z2HAxPMoW8bv36O9q8sryweRtGFHuG/vnasl9SUf40iw5trc9t6TDgbzRpjxdDr/oOtGc1Yy8tc4HxDB3mONTzjeyPxjvtkLef6y6jpWrM+qHPcn5rNHtUntbAXmjjS4vH9uf8/X5rgHrOC8Ux/N/f/5g76Reixr98vKCfM6PsOr4zZw/kHSBp4vThrMzvd6vuDL18t5lC3ql3XGqmRXsGjUqqYm8KwZd4nz8fRHfsqgDeaONLu+5buP4yoSFu0rjnuGnJo8/3nh7xtLtD8pzBZ+885xf8Iq0+sLvcD78a1yT94bUZVt6xWXq5b3ZJfuWJ81RFty4e4t/tLcqb03qK2KYN26Ye3BFNDYwekDeaFPurqqqWnctqzzrWSQmZ2TppwGTV4uJPg8Z8lazvXxKMV/Yg59P+KEm72vyNPlJh/+0YYgoNO16Y7FnVualb/BPNJH34QIxaEj0rKiazpZGZROjBeSNNtrVhvi8tfwNdqCmpmYDO+Y75609tCah3JC31vlkg+dpvp+9+zp7TZP3qQwxe0mSX155HaJ0lG+h78xOvmbtOWMdqryfumXTsG7zETGcltoQhS2MGpA32vjOeeV1MJ23NRn3D4rLKcox5eUFN7/F3uWXUlcsSTynyVs6UMxd4vLL+3NuyHt0jHv6QXUdqrzLsi4Z41vZ4chtWPSBvNHGkPc1dtI3JmT8LGHWCdF0U+Sdl/lUd/Fy66TbR3FNXi3GJZ3Vqw0+ed9052+qU9ehyjtYXrbgJw6KtiDfzo5HZqM6BsgbbQx5T3ZaK4aPjdZk3C33iee7KfJWs6u+KV5WubMf5Zq8+9gOzn8S34y8/PSy3tkLjpnrUOR9n/1OvuxhL4jhzJ4R3bRoA3mjjSEvn5e4eNfcuBWc3zvg4JGEKa9vK3BPqJHyVtx9QZ70MtkA+5Cx3dx3tSHtF9v6ygabKF/fRF7OG3cVJdz2ln9Kl1dbzDPxtXK8viBr0c45DuN6REwAeaONKW/j0oGu/vIiw+9zU85s7uMe8fLzGQulvNPZWTG3gP1Llsp1ygkpb+19PboVpmTq5S+TV/Bh2WL/qC6vtpg7huizzpX2TRlZHemNiy6QlxbIUgHy0gJZKkBeWiBLhWjIu757FFbSnnRlwAq/jfL3FA15yZG2sqNrQJLUZ6O8QsjbDJDXEpDXDkBeS8ScvE2ecKEC5LVEzMnb5CEBKkBeS0BeOwB5LUFd3j+PS/fcJnuIqE+4NCzun1LwsvKu8ciLTYG8liAu77nskZsqcsbypk+4zE56cse9bJv5rvHIi02BvJYgLu9B9irnVeW8yRMu/3BKFyYONd41H3mxKZDXEsTlrUn58nMfi9cmT7joPaCfd9T53+XGIy82BfJagri8/K2bk9jQrU2fcFntkGe3e9jf/e+aj7zYFMhrCeryitPe3ePi32vyhMt2dkKM/CbuvP9d85EXmwJ5LUFc3pfyzsp/J6pu8oTLMad8sOWWQca75iMvNgXyWoK4vB8kTtyx6abMU02ecKmf1XnpKzNYlfHuR8YjLzYF8lqCuLz85Xx35oQ/XfaES/3j/ZJHbFfeNR55sSmQ1xLU5Y0NIK8lIK8dgLyWgLx2APJaAvLaAchrCchrByCvJSCvHYC8loC8dgDyWgLy2gHIawnIawcgryUgrx2AvJaAvHYA8loC8toByGsJyGsHIK8lIK8dgLyWgLx2APJaAvLaAchrCchrByCvJSCvHYC8loC8dgDyWgLy2gHIawnIawcgryUgrx2AvJaAvHYA8loC8toByGsJyGsHIK8lIK8dgLyWgLx2APJaAvLaAchrCchrByCvJSCvHYC8loC8dgDyWgLy2gHIawnIawcgryUgrx2AvJaAvHYA8loC8toByGsJyGsHIK8lIK8dgLyWgLx2APJaAvLaAchrCchrByCvJSCvHYC8loC8dgDyWgLy2gHIawnIawcgryUgrx2AvJaAvHYA8loC8toByGsJyGsHIK8lIK8dgLyWgLxtp5wJXEM3KrMO/VGZyHxaexkzqsUlGPLKz3mXB1vZzq2+kYuP900b/aYY2SpXz6aHW+sOp+2xGfKGEdulp652D9vCrcUWi/K6KisrKwrZdnNWSbHyfhjyys8F/Rbec5X4xu5JX75jqvttzpdmrRbsDb/eHUzbYzPkDSO2BZ2e3HE/22kttliUN10Ov+g60ZzV6m/hkjZstbwX81mJPvYO2yyGY+/hfNZoK5XueNoeW6vlNWPjOaVicM0d1mKLWXn52P6cr893DVjHeaE4IP378wd7J/Va1NjMt+Arxb2rxrC0KaeEvDdd0fOR0mL9c95lC3ql3XGq2VXNG5Ffoo9tZMfF8GepnI+/L4LbFkHaHlvq+LBj41/6kRhMmGQttpiV91y3cXxlwsJdpXHP8FOTxx9vvD1j6fYH2cbAb8Ffinszpu79qXM254muii1DOxXrn/Nml+xbnjRHWUHj7i36yN7k9wtL9NE97A0xfICd5XnjhrkHV0RjQ9uXDomN39/9jZqKhOe4pdhiUV53VVXVumtZ5VnPIjE5I0s/jk1eLSb6PBTwLZilvEPEDubma/nH7Juc/8tZ7Dv+DRFvT7veWPyZlXnpG7Sx0z2f5f5v4YsrC975fEMiO9aQ6FlRNZ0tjdLGNsuBSd2TrvrBmeCFLpcxSGyHrniIe90txebRmlrZIrZvc/53R0ra6KJi/9xuxuqaj41fHC5KzeLcUmyxKK9MLT5vLX+DHaipqdnAjvlO3moPrUkoD/jWzFLeR+SnC/k2tkSMjPDLO19MlPoK83dmJ1+z9pw+flcRN7+FQ/0Yu+L77Iu6zUfE1LTUhmhsavPsc47ftOOproPOBy0VIG/LsRU6ygPkNUuluURLS5xtbGPiNPee+C/vmOq8UZx6ucXc0V/2LbzF2GbkPLf/iZRfckuxxaK86b6RF5nO29q3sH9QXE5RTqC8ZimvdFbI+3ScbLBN8sv7c27Ie3SMe/pB/3o2eT5RvgXe+OHfGpal+Ca2ssOR3MTgFA2XCvyJrWkyt7G+aakAeVuOLScpUF6zVPLVXI+NPSvarcPEZ7r14Dw5j7citvfYbjFc5PE5G2ZssSzva+ykb0x8C58lzDohQguU1yzllQct8S1UakevwmKl2ez7Ft5052+q86+n1Pf1VcmJi3+tFcNvjOQnDopzD75da791EHk3ai+P7OJGq+owq+4Z1/cnYkaLDbByl68B9iV2nd4Aaxjo6lnmyD2hN8Dc47UGWGBsSYXcF9ty0W7NF5HlO8XckbwVsf2a1YhhNfvQUmyxLO/JTmvF8LHRmoS75Y/6fLdAec1SfnmPsm9x/kliM/Ly08t6Zy84po+//6qg3+hXtS/xYnIZ5ydSnhUttxfE5MyeUdjQlpgW9/B7vlF/q+ow6zyjqjzuh5y32AC7kfkaYDmsUDTAho/mcxKHbOnNxvJTxZ1mNXod12gNMDO2NaIBJmNz9hFNrZtkbN8RW58gIusj2q3OvmLuaF/hFmN7g20TwwVJFy3FFsvy8nmJi3fNjVvB+b0DDh5JmPL6tgL3hBr5rVXcfYGP6Vcl+ZtZyi8vT3Sv21bQ/Xbtc/VN5BVH311FCbe9ZaxMO/7Jxc1xPbt52IB/8/qCrEU75zi2RG97A/jsLifree+LdUqr6jCbLEbmpnzecru1cxL3tVuzRCBXsRUfJ14/4OAB1v31bZnxE2q8ThHblV81YjszJi6pXMbWEOeSTa0rtNhe6pQ49vMNDrbzYpxbzs3116ml2CZ4Vr7yaMIia7HFtLyNSwe6+ssv6/e5KWc293GPePn5jIXyW5sudg1j9KPXw2YpQ960G7L7vjBumv65y+QVfFi22BjXvgW5uPOlXbvf86mYOFfaN2VkdTQ2tGVO//Z7V7Pc981W1WFWKWb/Rbue11K7Ndlot4pAPLmiAbZLbH7vZPeIG/IzFnpdMrav+GMblnzNnf212Oq6yHbugPgGnjo6u+8TbtFuncKSP+3ymJjbL15pgDUb2xeP5LmH/KrRWmwxKG+bqe38IyH+lT/u6HpYRW+Z7UktMltVh9l+Meu0UDiMdmtnOT9ku1VHNLVqk34sY/t+9NqtkDeQRseAw589nvhxR9fDIn9l+7TXb/UwW1X6nvcd9t9htVvP8la0W42mVqNj4OET35OxRavdCnmbIbkHi8t7paNrYZULnjvlwbqu/y1mY/Qwu1OMzHedDqfdGr+5Ne1Ws6nl7sniHLdHsd0KeZshbeWZ2o6uQxt4KekrP9v8dKHrz2aT9TBLmrXzEcd8/lEY7daZnta0W82mVtLwk7VRbbfaSN7WdSidxz7i8gxN3vfi2WN4sv9S/OrMVnQk3bPNGH33Nk8veTOimY6kTTqjR7daRm3a0i34rTvyknp9421uNlkPs99OTM97QhzKw2i3XixrVbvVaGolRLvdaid5W9WhtFpeo+Qz2HAxPMoW8bv36O/+OsPh9bo6e73x6d6WSUzxj3V1JHlSWBevN82RIfiSWsqRoUykxHk8noxElmnOkqsxC2vry5SL8rqZIz7O6WapXpd/gbmZAV0EA/pnNYwo948a3Vr1kTXBtff/ACVBe8MfZgcDPhycd3t2z+gm6u9MCxJnU5rE1jq+HWatmmIneVvVobTW+YAY5g5zfCr7If7BeHequ3NZWb+rysqSv1bWAjNu7M7y/RODPXPKyrrniJGeAQUnzlYm8jvJ4QPuXHOWXI1B5xvl8HuOIWLoZmxM8ahObLLx7tcdIeX9x6rrmCGv0a1VHwneM/aDrKLKh9lv9ImgveHDl3dDQllK7r3fHRl/X0txBjBhdugyTRnWI8xaNcV28obsUFqQz/kRVi1bEw8kXeDp4vh8dqbXe3XXzFAdSXeNGpXkt+Ril5+K4fG3Q3ckDataV8tqfZ2tNqo12RFYrcvkVavFx/lro1cruLzfHSja/F8dqY0H7w1vQV4X/79hkW63lsaYvCE7lM5zfsEr0uoLv8P58K9xzZIbUpdtyRKWhOpIynlvvyVH2e8v/fWEHAvVkTScajlu0KrVc5eo1vr8eMfdW4YxVu1l7sdEte66I5Olqb0KBvqrtb5TgtGt25nhGlzRMP8KZ6+s+MG9gv4azR8gD9Ub/tJHdYEzgyLk5TzS7dYYkrd1/XCr2V4+pZgv7MHPJ/xQk/c1eZr8dWdmqP63XJH3f9iPUxgbdzx0R9JwqtUj7tpuvmp9PWEauyXumf2MdZ5xfe+4H3qHCN3FeYtf9zMrXU5ftVYmdJns69b9jQSXY8J0NipjtdPpLJjOBgX7NZo/QN7+veE1eSNNDMnbun64tc4nGzxP8/3s3dfZa5q8T2WI2VO7ZIbqf8sVeXeybrtr93WfFLojaTjV+uld8azbBNnNLy15kajWjKw349hkUX5uSvf5QvfSUT7dZbX668d72QFBVEvvDX9h85Gbr+VT2At1mw84i/m0hFuD/BqNH6AkzN7wd/QLQXdHqBL9Nof4QkNTFjvytq4fLi+4+S32Lr+UumJJ4jlN3tKBYu7U7MxW3Mc05D3A5HH6WXZanwxyHzO8an3q+momu/J9nsIO3N9PmP6yg1WK8n9hWaJatVOv1nT/kVYtpVt37uwmveEfYfJUYESxqNbYIL9G4weoEV5v+KTrSoJz5/AQBUoypgRdQ2v44Ndt+rgd5Q3eD5fPy3yqu3i5ddLto7gm75Iu4nVqZmaI+5gS9ZyXy2P9ByHvY4ZTrXqtWgNTi7jbZ/rGOLZflD/NPMuF7u4MTfe5WrWa61UgaiPkncmOi5FJxaJaRUF+jZf9AMPqDZ/U9tsHg9oubxuxo7zB++EK4676pnhZ5c5+lGvy7mM7OP9Gp2bkbXIfU2LIy/vLvdxDKQ0h72OGUa1b2T6tWt/qwZPZSa1a/8u0Pe87LOsJoXvpKF13rVqjfHUUPwpftbxLRW2EvGPZWTFSWMxnuoP9Go0foJwIszd89OVNXhN0TuDbobGjvMH74YqzSyZP+T5k2iMk2tWGtF9sy0nIbEX/W90SuZjNCfOr5zt/Hvo+ZhjVes5zZ4Oo1q7+t/A051pRrfGDRzJ2pyj/bVfOd4XupSP9uotqxXn0aokfhaiW+FFUuJ4StRnVd47Dsbm+IDP+K3Mc1wX9Nfp/gHI8zN7wI98N8W2E5v7w/hTLfzephTmBb4fGlvIG7Ycrzi7Zv2SpXKeckPLW3tejW++ema3of6vLqy1m04jkIet56PuY4VRL9irIcoxw/ZmnFyUu/q/BLPU2B0ua9VQ6+4F3odD9lhRF95Fdpvp7FXgmyx/FdPaEqE2X+JHVMz3rtnR1OkdWB/81+n+AdusNH4y6i5fPuWR5WTaSt21M7d/RNZAE9Cp4Mz6CvQqMH2DEe8NbP+bfNlQMGrvP0vYyXV8sz/ioYb7+3yTaHOMfS+RE/cI+GTf9VXnOLgSQN7IIecO9tyWOJKv/brPe8NaP+S+wY/Ki3gFd3uG3bjk/J0P/bxJdXv8/lsiJ76StemFU8jHzObsQQN7IYknexp4TI9obvtkdpb/hGQwLx/wznVaJo0mubmfXQY3848SNvr900eT1/dGLnPjA8ZJ4J3GFeQMoBETkvdS1Swg6xYcq4fnfNlfjgVDrCCCNsdRwP/NSpHsVNLujbE7e9jjmTxonmieP+eSdx/k2Ji+LjPAvwndpW0782il/Cp/L/0rx3QAKARF5a9lPQvBYeagSnX7V5moU9A21kgDuf/iJMD+ROpdHpFdB4G6zCZfLK4u3xzH/ucTP/4+955N3GVcejdMW4XtgTk4s7ur7iHEDKARk5G37MpLbQd4Jba9GSHLmtnkRxh7T+CtHbbf553HpntuONd1RNizun1LwMtflNSa04u1zzP8sYcujsvO1tgjR9jQfjdMW4W+jiok1ifKi36H3zRtAIYC84dAO8n4wsSgEnXJDlbg/1EqMPabxV45yt/lZ9shNFTljm+4oZyc9ueNe+dcfUl5jQtvL8vY55t9U0lfeI/TLaz4ad7m8f5IdrM57fmbeAAoB5A2HVaFbwKFYkFAYgpwBIQoMjQu1Ev8e0+wIJ3ebB9mrnFeVN9lR/sMpbzVMHKrJa07I4pL2OOb/Min+E27Kaz4ad7m8fIqnYvctaf80n7MLAeSNMguy27yIYyHl9e8xzY5wcrdZk/Ll57QrGMqOUr+N/LyjTsprTsjikvY45p+Mv4krizAvYgfIW1d+Vcr1bypXxEMAeVss3v734iXRkde3xzT7/MjdJn/r5iQ2dGuTHeVqh9zF7mF/l/KaE1pxSfsf89vxIjbkbYkI3IuXrB9u7XMKZzwhi/j2mGZHuK76s0Tndo+Lf0/dUW5nsjf7b+LO63te/0RX/6NH7X/Mb8eL2JA3BO15Lz6K6HtMsyOcNO+lvLOyK1q1uqM85pRXyG4ZpJ3zmhOGvBE45rffRWwi8jYMbfsyig61plQk78VbwOrJi2+PaXSEk+Z9kDhxx6abMk+pO0o+q/PSV2bIP8uVVxuMia4tPPTZPsf89rqITUTe6BHJe/EWsHry4ttjGn1+NBtfzndnTvhT0x1l/eP9kkfIv8qQ8hoTLckb6RvX4RGj8lpvbUXyXrz1XgW2OXmJ/OPwYRCj8rahtRXBe/Gt71Vgs5MXFRv9jRsJeaPZCyoS9+LD71Vgs5MXm0JC3qj2gmrbdfl26lUQ2ZOXWMHG8nZQL6i2XZdvr14FET15iRXsIa+dekG17bp8e/UqiGBHwtjBHvLaqhdU267Lt1Ovggh2JIwd7CGvrXpBXUaY1+Xbq1dB5DoSxg72kNdWvaAuI8zr8u3VqyByHQljB5vIa+NeUOFel2+nXgWR60gYO9hEXhv3guJhXpePTK8C+z0NbwNsIq+de0GFSWR6FdirU4FNsIu8tu4FZQds1anAJthF3v+IXlBtw0adCmyCXeSNFNhhxTCxLi92WDFM7MsLYhbIC8gCeQFZIC8gC+QFZIG8gCyQF5AF8gKyQF5AFsgLyAJ5AVkgLyAL5AVkgbyALJAXkAXyArJAXkAWyAvIAnkBWSAvIAvkBWSBvIAskBeQBfICskBeQBbIC8gCeXVmuUMAAADwSURBVAFZIC8gC+QFZIG8gCyQF5AF8gKyQF5AFsgLyAJ5AVkgLyAL5AVkgbyALJAXkAXyArJAXkAWyAvIAnkBWSAvIAvkBWSBvIAskBeQBfICskBeQBbIC8gCeQFZIC8gC+QFZIG8gCyQF5AF8gKyQF5AFsgLyAJ5AVkgLyAL5AVkgbyALJAXkAXyArJAXkAWyAvIAnkBWSAvIAvkBWSBvIAskBeQBfICskBeQBbIC8gCeQFZIC8gC+QFZIG8gCyQF5AF8gKyQF5AFsgLyAJ5AVkgLyAL5AVkgbyALJAXkAXyArJAXkAWyAvIAnkBWf4fkEKYeqnvg2UAAAAASUVORK5CYII=)
Yes! It is larger, and also uglier… (As a consequence, it is less
interpretable).
Concerning the count of nodes, by reading the documentation of
tree()
in the Value section, we found that a “component” of
the returned value, named frame
is:
A data frame with a row for each node, and row.names
giving the node numbers.
So we just need to count the rows of that dataframe:
nrow(tree(Species~., d)$frame)
[1] 11
It works: check with the corresponding image above.
So let’s write a function for doing everything:
computeComplexityOfPerfectTree = function (y.name, data) {
t = tree(formula(paste0(y.name,"~.")), data, mindev=0, minsize=2)
nrow(t$frame)
}
computeComplexityOfPerfectTree("Species", d)
[1] 17
Alternative step 3: execution and answer
As before: Since we indeed built the functions, we can now compute
the errors for all the variables:
for (variable in names(d)) {
complexity = computeComplexityOfPerfectTree(variable, d)
cat(paste(variable, complexity, "\n"))
}
Sepal.Length 253
Sepal.Width 259
Petal.Length 251
Petal.Width 205
Species 17
So, according to this method, it turns out that the hardest variable
is Sepal.Width, differently than with the previous method.
Note that the important thing here is not the actual answer, but the
full path to the answer. Note also that this method still has the
drawback of comparing apples and oranges: indeed the complexity of the
obtained trees clearly shows that the values are very different for the
categorical and numerical variables. This is because the former just
takes 3 values, while the latter ones take many of them: when we impose
the RSS=0 as the only stopping criterion, all of them basically act as
class values. It is hence somehow expected that much more nodes are
needed to perfecly accomodate all those class values.
LS0tCnRpdGxlOiAiTGFiIDI6IHRyZWUgb24gaXJpcyIKb3V0cHV0OiBodG1sX25vdGVib29rCi0tLQoKIyBHb2FsCgpHaXZlbiB0aGUgYGlyaXNgIGRhdGFzZXQsIGZpbmQgYSBNTC1iYXNlZCB3YXkgdG8gdGVsbCB3aGljaCBpcyB0aGUgaGFyZGVzdCB2YXJpYWJsZSB0byBiZSBwcmVkaWN0ZWQsIGdpdmVuIHRoZSBvdGhlciB2YXJpYWJsZXMsIGFuZCBtb3RpdmF0ZSB0aGUgYW5zd2VyLgoKIyMjIEhpbnRzCgpUaGUgZ29hbCBpcyBub3QgdG8gImJ1aWxkIGEgTUwgc3lzdGVtIG9uIHRoaXMgZGF0YSIsIGJ1dCByYXRoZXIgYSBxdWVzdGlvbiB0byBiZSBhbnN3ZXJlZC4KU28gd2Ugc2hvdWxkOgoKMC4gdW5kZXJzdGFuZCB0aGUgcXVlc3Rpb24gKHRoYXQgcHJvYmFibHkgaW5jbHVkZXMgZ2l2aW5nIGFuIGludGVycHJldGF0aW9uIHRvICJoYXJkZXN0IikKMS4gZGVzaWduIGEgbWV0aG9kIGZvciBhbnN3ZXJpbmcgdGhlIHF1ZXN0aW9uCjIuIGltcGxlbWVudCB0aGF0IG1ldGhvZAozLiBwcm92aWRlIHRoZSBhbnN3ZXIgYnkgYXBwbHlpbmcgdGhlIG1ldGhvZAoKIyBQb3NzaWJsZSAic29sdXRpb24iCgojIyBTdGVwIDE6IGRlc2lnbgoKVGhlIGBpcmlzYCBkYXRhc2V0IGlzIGNvbXBvc2VkIGJ5IDUgdmFyaWFibGVzLCBoZW5jZSB0aGUgcXVlc3Rpb24gbWF5IGJlIGFuc3dlcmVkIGJ5IGJhc2ljYWxseSAidHJ5aW5nIHRvIHByZWRpY3QiIGVhY2ggb25lIG9mIHRoZW0gYW5kIG1lYXN1cmluZyBob3cgaGFyZCBpdCBpcy4KSWYgd2UgaW50ZW5kIHRvIGZvbGxvdyB0aGlzIHBhdGgsIHdlIG5lZWQ6CgoxLiB0byBiZSBhYmxlIHRvICJ0cnkgdG8gcHJlZGljdCIKMi4gYSB3YXkgZm9yIG1lYXN1cmluZyBob3cgaGFyZCBpdCBpcwoKQ29uY2VybmluZyAxLCBhbmQgbm90ZWQgdGhhdCB3ZSBjYW4gdGFja2xlIHRoZSBwcmVkaWN0aW9uIHByb2JsZW0gYXMgYSBzdXBlcnZpc2VkIGxlYXJuaW5nIHByb2JsZW0sIHdlIGtub3cgYSBsZWFybmluZyB0ZWNobmlxdWUgKG9mZmljaWFsbHkganVzdCBvbmUpIHRoYXQgZml0cyB0aGUgc2NlbmFyaW86IGxlYXJuaW5nIG9mIGRlY2lzaW9uL3JlZ3Jlc3Npb24gdHJlZXMuCkludGVyZXN0aW5nbHksIDQgb2YgdGhlIHZhcmlhYmxlcyBhcmUgbnVtZXJpYyBhbmQgMSBpcyBjYXRlZ29yaWNhbDogZm9ydHVuYXRlbHksIHdlIGhhdmUgYSB2YXJpYW50IG9mIHRoZSBsZWFybmluZyB0ZWNobmlxdWUgZm9yIGVhY2ggb2YgdGhlIHR3byBjYXNlcy4KCkNvbmNlcm5pbmcgMiwgd2Ugbm90ZSB0aGF0IHRoZSBub3Rpb24gb2YgaGFyZG5lc3MsIGFzIGNhcHR1cmVkIGJ5IHRoZSBjb3JyZXNwb25kaW5nIHdvcmQgaW4gbmF0dXJhbCBsYW5ndWFnZSwgaXMgYnJvYWQuICJIYXJkIiBjb3VsZCByZWZlciB0byBjb21wdXRhdGlvbmFsIGVmZm9ydCwgdGltZSBuZWVkZWQgdG8gZGVzaWduIGEgTUwgc3lzdGVtIGZvciBkb2luZyBwcmVkaWN0aW9uLCByZXF1aXJlZCBrbm93bGVkZ2UsIC4uLgpXZSBkZWNpZGUgdG8gaW50ZXJwcmV0ICJoYXJkIHRvIGJlIHByZWRpY3RlZCIgYXMgImRlZ3JlZSB0byB3aGljaCwgb24gc29tZSBkYXRhLCBhIG1vZGVsIGxlYXJuZWQgd2l0aCBzb21lIHN1aXRhYmxlIHRlY2huaXF1ZSBjb3JyZWN0bHkgcHJlZGljdHMgdGhlIGRlcGVuZGVudCB2YXJpYWJsZSIuCkluIGJyaWVmLCB3ZSBwbGFuIHRvIGludGVycHJldCBoYXJkbmVzcyBhcyAoaW4pZWZmZWN0aXZlbmVzcyBvZiBwcmVkaWN0aW9uOyB3ZSBrbm93IHNldmVyYWwgd2F5cyB0byBtZWFzdXJlIChpbillZmZlY3RpdmVuZXNzLiBPbmUgaXMgKmVycm9yKi4gKEkgYW0gYmVpbmcgcHVycG9zZWx5IHZhZ3VlIGhlcmUuLi4pLgpOb3RlIHRoYXQsIHdoaWxlIGluIHByaW5jaXBsZSB0aGlzICJkZWZpbml0aW9uIiBpcyBub3QgYm91bmQgdG8gYSBzcGVjaWZpYyBkYXRhc2V0IG9yIHRvIGEgc3BlY2lmaWMgbGVhcm5pbmcgdGVjaG5pcXVlLCB3ZSBoYXZlIG9ubHkgb25lIGRhdGFzZXQgYW4gb25seSBvbmUgbGVhcm5pbmcgdGVjaG5pcXVlLgoKSW4gcHJhY3RpY2UsIGhlbmNlLCB3ZSBhcmUgZ29pbmcgdG8gYnVpbGQgNSBtb2RlbHMgKDQgcmVncmVzc2lvbiBhbmQgMSBjbGFzc2lmaWNhdGlvbiBtb2RlbHMpLCBtZWFzdXJlIHRoZWlyIGVycm9ycywgYW5kICpzYXkgdGhhdCB0aGUgaGFyZGVzdCB2YXJpYWJsZSBpcyB0aGUgb25lIGZvciB3aGljaCB0aGUgZXJyb3IgaXMgdGhlIGxhcmdlc3QqLgpOb3RlIHRoYXQgdGhlIGxhc3QgcGFydCAodGhlIG9uZSBpbiBpdGFsaWMpLCBpcyBhIGtleSBzdGVwLCBldmVuIGlmIGl0IGFwcGVhcnMgdHJpdmlhbCBhbmQgaXQgZG9lcyBub3QgcmVxdWlyZSBhbnkgYmlnIGltcGxlbWVudGF0aW9uIGVmZm9ydC4KUmVjYWxsIHRoYXQgd2Ugd2FudCB0byAiZGVzaWduIGEgbWV0aG9kIGZvciBhbnN3ZXJpbmcgdGhlIHF1ZXN0aW9uIjogd2hlbiBleGVjdXRlZCwgdGhlIG1ldGhvZCBpcyByZXF1aXJlZCB0byByZXN1bHQgaW4gYW4gYW5zd2VyLgpXaXRob3V0IHRoYXQgc3RlcCwgdGhlIG1ldGhvZCBleGVjdXRpb24gd291bGQgcmVzdWx0IGluIGEgY29sbGVjdGlvbiBvZiA1IG51bWVyaWNhbCB2YWx1ZXMsIG5vdCBpbiBhbiBhbnN3ZXIhCgojIyBTdGVwIDI6IGltcGxlbWVudGF0aW9uCgpUaGUgbWV0aG9kIGRlc2lnbmVkIGFib3ZlIGFuZCBkZXNjcmliZWQgaW4gbmF0dXJhbCBsYW5ndWFnZSBjYW4gYmUsIGluIHByaW5jaXBsZSwgYXBwbGllZCB0byBhbnkgZGF0YXNldC4KQWdhaW4gaW4gcHJpbmNpcGxlLCB3ZSBjb3VsZCBpbXBsZW1lbnQgYXMgYSBmdW5jdGlvbiB0aGF0IHRha2VzIGEgZGF0YXNldCAoYSBkYXRhZnJhbWUsIGluIHRlcm1zIG9mIFIpLCBhbmQgb3V0cHV0cyB0aGUgbmFtZSBvZiB0aGUgaGFyZGVzdCB2YXJpYWJsZS4KCkhvd2V2ZXIsIGZvciBzaW1wbGljaXR5IHdlJ2xsIHN0YXJ0IGJ5IGRvaW5nIGFuIGltcGxlbWVudGF0aW9uIG1peGVkIHdpdGggdGhlIGV4ZWN1dGlvbi4KVGhpcyBpcyBkb25lIGFsc28gYmVhY3VzZSB0aGVyZSBhcmUgc3RpbGwgc29tZSBhbWJpZ3VpdGllcyBpbiB0aGUgZGVzY3JpcHRpb24gb2YgdGhlIG1ldGhvZCAoZS5nLiwgaG93IHRvIG1lYXN1cmUgdGhlIGVycm9yPykgYW5kIHdlIHdpbGwgZGlzY292ZXIgYW5kIGFkZHJlc3MgdGhlbSBhcyB3ZSBtZWV0IHRoZW0uCklkZWFsbHksIG9uZSBzaG91bGQgZmlyc3QgZnVsbHkgZGVmaW5lIHRoZSBtZXRob2QgKGkuZS4sIHRoZSBjb3JyZXNwb25kaW5nIGFsZ29yaXRobSksIHBvc3NpYmx5IGluIHRoZSBmb3JtIG9mIGEgcGllY2Ugb2YgcHNldWRvY29kZS4KCiMjIyBTaW1wbGljaXR5OiBpbXBsZW1lbnRhdGlvbiArIGV4ZWN1dGlvbgoKTG9hZCBgaXJpc2AuCmBgYHtyfQpkID0gaXJpcwpgYGAKCkhvdyBjYW4gd2UgbGVhcm4gYSB0cmVlPyBVc2luZyBhIHBhY2thZ2UhClRoZXJlIGFyZSBtYW55IHBhY2thZ2VzIHRoYXQgaW1wbGVtZW50IChzbGlnaHRseSBkaWZmZXJlbnQgdmVyc2lvbnMgb2YpIHRoZSBsZWFybmluZyB0ZWNobmlxdWUgdGhhdCB3ZSBzYXcgaW4gdGhlIGxlY3R1cmVzLgpNeSBjaG9pY2UgaXMgZm9yIHRoZSBwYWNrYWdlIGB0cmVlYCAodGhhdCB5b3UgcHJvYmFibHkgaGF2ZSB0byBpbnN0YWxsLCBmaXJzdCwgdXNpbmcgYGluc3RhbGwucGFja2FnZXMoInRyZWUiKWApLgooQW5vdGhlciBvcHRpb24gaXMgYHJwYXJ0YCkuCmBgYHtyfQpyZXF1aXJlKHRyZWUpCmBgYAoKIyMjIyBTdXBlcnZpc2VkIGxlYXJuaW5nIGNvbnZlbnRpb25zIGluIFIKCk1vc3Qgb2YgdGhlIHBhY2thZ2VzIGZvciBzdXBlcnZpc2VkIGxlYXJuaW5nIHNoYXJlIHRoZSBmb2xsb3dpbmcgc3RydWN0dXJlLgpUaGV5IGhhdmU6CgotIGEgZnVuY3Rpb24gZm9yIGRvaW5nIHRoZSBsZWFybmluZywgdXN1YWxseSB3aXRoIHRoZSBuYW1lIG9mIHRoZSB0ZWNobmlxdWUgaXRzZWxmCi0gYSBmdW5jdGlvbiBmb3IgZG9pbmcgdGhlIHByZWRpY3Rpb24sIHVzdWFsbHkgbmFtZWQgYHByZWRpY3QoKWAKYHRyZWVgIGFkaGVyZXMgdGhpcyAiY29udmVudGlvbiIgKHNlZSB0aGUgZG9jdW1lbnRhdGlvbikuCgpUaGUgbGVhcm5pbmcgZnVuY3Rpb24gdXN1YWxseSBoYXMgdHdvIHdheXMgZm9yIGJlaW5nIGludm9rZWQ6IG9uZSBpbiB3aGljaCAkXHsoeF57KGkpfSx5XnsoaSl9KVx9X2kkIGlzIHNwZWNpZmllZCBieSBnaXZpbmcgJFx7eF57KGkpfVx9X2kkIGFuZCAkXHt5XnsoaSl9XH1faSQgc2VwYXJhdGVseSBhbmQgZXhwbGljaXRseSwgdGhlIG90aGVyIGluIHdoaWNoIGEgZGF0YWZyYW1lICh3aXRob3V0IGluZGljYXRpb24gb2YgdGhlIGRlcGVuZGVudCB2YXJpYWJsZSkgaXMgcGFzc2VkIHRvZ2V0aGVyIHdpdGggKmFuIGluZGljYXRpb24gb2YgdGhlIGRlcGVuZGVuY3kqLgooQWN0dWFsbHkgYHRyZWUoKWAgb25seSBzdXBwb3J0cyB0aGUgc2Vjb25kIHdheSkuClRoZSBsYXR0ZXIgaXMgZG9uZSB1c2luZyBhIGRhdGEgdHlwZSwgcGVjdWxpYXIgb2YgUiwga25vd24gYXMgKipmb3JtdWxhKiouClRoZSBzeW50YXggZm9yIHNwZWNpZnlpbmcgYSBmb3JtdWxhIGxpdGVyYWwgKGkuZS4sIGEgY29uc3RhbnQgdmFsdWUgZm9yIGEgZGF0YSB0eXBlKSBpcyBgYSB+IGIrY2AgKHRoaXMgaXMgYW4gZXhhbXBsZSwgcmF0aGVyIHRoYW4gYSBmb3JtYWwgc3BlY2lmaWNhdGlvbiBvZiB0aGUgc3ludGF4KTogdGhpcyB2YWx1ZSBtZWFucyBgYWAgZGVwZW5kcyBvbiBgYmAgYW5kIGBjYC4KQSBjb21tb24gY2FzZSBpcyB0aGF0IG9uZSB3YW50cyB0byBzYXkgdGhhdCBhIHZhcmlhYmxlIGRlcGVuZHMgb24gYWxsIHRoZSBvdGhlciB2YXJpYWJsZXM6IHRoaXMgY2FuIGJlIGRvbmUgd2l0aCBgYSB+IC5gLgoKU3VtbWFyaXppbmcsIGZvciBidWlsZGluZyBhIHRyZWUgZm9yIHByZWRpY3RpbmcgdGhlIFNwZWNpZXMgYmFzZWQgb24gdGhlIG90aGVyIHZhcmlhYmxlIG9uIHRoZSBmdWxsIGRhdGFzZXQgYGRgLCB3ZSBkbzoKYGBge3J9CnRyZWUoU3BlY2llcyB+IC4sIGQpCmBgYApOb3RlIHRoYXQ6CgotIHRoZSBmb3JtdWxhIGlzIG5vdCBxdW90ZWQ6IGl0IGlzIG5vdCBhIHN0cmluZyByZXByZXNlbnRpbmcgYSBkZXBlbmRlbmN5LCBpdCBpcyBhIGZvcm11bGEhCi0gdGhlIG91dGNvbWUgb2YgdGhlIGV4ZWN1dGlvbiBvZiB0aGUgYWJvdmUgY29tbWFuZCBpcyB0aGUgdGV4dHVhbCByZXByZXNlbnRhdGlvbiAob24gdGhlIHN0YW5kYXJkIG91dHB1dCkgb2YgdGhlIGxlYXJuZWQgdHJlZSBbdGhlcmUgbWlnaHQgYmUgYW4gZXJyb3IgaGVyZS4uLl0KCllvdSBzaG91bGQgYWxzbyBub3RlIHRoYXQgdGhlcmUgYXJlIG1vcmUgZGVjb3JhdGlvbnMsIG9uIHRoYXQgdHJlZSwgdGhhbiB0aGUgb25lcyB3ZSBleHBlY3RlZCB0byBzZWUuCkxvb2sgYXQgdGhlIGRvY3VtZW50YXRpb24gZm9yIG1vcmUgaW5mby4KCkFzIGFuIGFzaWRlLCBhIHRyZWUgY2FuIGFsc28gYmUgcGxvdDoKYGBge3J9CnQgPSB0cmVlKFNwZWNpZXN+LiwgZCkKcGxvdCh0KQp0ZXh0KHQpCmBgYApgdGV4dCgpYCBpcyBuZWVkZWQgdG8gYWRkIHRoZSBsYWJlbHMgdG8gdGhlIHRyZWUuCgojIyMjIFdpdGggdGhlIGBycGFydGAgcGFja2FnZQoKYGBge3J9CnJlcXVpcmUocnBhcnQpCmBgYApgYGB7cn0KcnBhcnQoU3BlY2llc34uLCBkKQpgYGAKCmBgYHtyfQp0ID0gcnBhcnQoU3BlY2llc34uLCBkKQpwbG90KHQpCnRleHQodCkKYGBgCgoKIyMjIEJhY2sgdG8gZGVzaWduK2ltcGxlbWVudGF0aW9uCgpXZSBrbm93IGhvdyB0byBidWlsZCBhIHRyZWUuCkhvdyBzaG91bGQgd2UgbWVhc3VyZSB0aGUgKmVycm9yKj8KV2UgaGF2ZSBzb21lIG9wdGlvbnM6CgotIG9uIHRoZSBsZWFybmluZyBkYXRhCi0gd2l0aCBhIHN0YXRpYyB0cmFpbmluZy90ZXN0IGRpdmlzaW9uIChlLmcuLCAyMCUgb2YgdGhlIG92ZXJhbGwgZGF0YSB1c2VkIGFzIHRlc3QpCi0gd2l0aCBhIGstZm9sZCBDVgotIHdpdGggYSAqKmxlYXZlLW9uZS1vdXQqKiBDViAoTE9PQ1YpLCBpLmUuLCBhIGstZm9sZCBDViB3aXRoICRrPW4kLCAkbiQgYmVpbmcgdGhlIG51bWJlciBvZiBvYnNlcnZhdGlvbnMgaW4gdGhlIGF2YWlsYWJsZSBkYXRhCgpUaGUgZWFzaWVzdCAoaW4gdGVybXMgb2YgZWZmb3J0IHRvIGJlIHJlYWxpemVkKSBvcHRpb24gaXMgdGhlIDFzdDogYnV0IHdlIGtub3cgdGhhdCwgd2hlbiAkbl9cdGV4dHttaW59PTEkLCB0aGUgZXJyb3Igb24gdGhlIGxlYXJuaW5nIGRhdGEgaXMgMCBieSBkZWZpbml0aW9uLgpJdCB3b3VsZCBoZW5jZSBwb2ludGxlc3MgdG8gY29tcGFyZSBhIHNldCBvZiAwcy4uLgpXZSBuZWVkIHRvIGFzY2VydGFpbiB3aGljaCBpcyB0aGUgZGVmYXVsdCB2YWx1ZSBvZiAkbl9cdGV4dHttaW59JCBpbiBgdHJlZSgpYCwgdGhhdCByZXF1aXJlcyB0byBjb25zdW1lIHRoZSBkb2N1bWVudGF0aW9uOiBJIGxlYXZlIHRoaXMgZm9yIHlvdXIgZW5qb3ltZW50LgoKV2UgZ28gZm9yIHRoZSAybmQgb3B0aW9uLgpXZSB1c2UgYHNhbXBsZSgpYCBmb3Igc2h1ZmZsaW5nIHRoZSBzZXQgb2Ygcm93LWluZGV4ZXMgb2YgYGRgIGFuZCB0YWtlIGEgc3Vic2V0IG9mIHRoaXMgc2V0IHRoYXQgd2lsbCBhY3QgYXMgdGhlIGluZGV4ZXMgb2YgdGhlIGxlYXJuaW5nIGRhdGEuCmBgYHtyfQppbmRleGVzLmxlYXJuaW5nID0gc2FtcGxlKGMoMTpucm93KGQpKSlbMToobnJvdyhkKSowLjgpXQpgYGAKCk5vdyB3ZSBjYW4gbGVhcm4gdGhlIHRyZWUgKGxldCdzIHN0YXJ0IHdpdGggU3BlY2llcyk6CmBgYHtyfQp0LnNwZWNpZXMgPSB0cmVlKFNwZWNpZXN+LiwgZFtpbmRleGVzLmxlYXJuaW5nLF0pCnBsb3QodC5zcGVjaWVzKQp0ZXh0KHQuc3BlY2llcykKYGBgCgojIyMjIEhvdyB0byBtZWFzdXJlIHRoZSBlcnJvcj8KClRoZSBgcHJlZGljdCgpYCBmdW5jdGlvbiB0YWtlcyBhIGRhdGFmcmFtZSB3aXRoIHBvc3NpYmx5IG5ldyBvYnNlcnZhdGlvbnMgYW5kIHByZWRpY3QgdGhlIGNvcnJlc3BvbmRpbmcgbGFiZWxzOiB0aGUgcmVzdWx0cyBpcyBoZW5jZSBhIHZlY3Rvci4KYGBge3J9CnByZWRpY3RlZC55ID0gcHJlZGljdCh0LnNwZWNpZXMsIGRbLWluZGV4ZXMubGVhcm5pbmcsXSwgdHlwZT0iY2xhc3MiKQpgYGAKTm90ZSB0aGF0OgoKLSB0aGUgYC1gIHByZWNlZGluZyBgaW5kZXhlcy5sZWFybmluZ2AgbWVhbnMgInNlbGVjdCBhbGwgYnV0IHRob3NlIgotIGB0eXBlPSJjbGFzcyJgIGlzIG5lZWRlZCB0byBvYnRhaW4gYSB2ZWN0b3Igb2YgZmFjdG9ycywgcmF0aGVyIHRoYW4gYSBtb3JlIGNvbXBsZXggdGhpbmc6IHNlZSB0aGUgZG9jdW1lbnRhdGlvbiBvZiBgcHJlZGljdC50cmVlKClgCi0gYHByZWRpY3QoKWAgZG9lc24ndCBjaGVhdDogZXZlbiBpZiBgZFstaW5kZXhlcy5sZWFybmluZyxdYCBhY3R1YWxseSBjb250YWlucyBhbHNvIHRoZSBjb3JyZWN0ICR5JCB2YWx1ZXMsIGl0IGlzIG5vdCB1c2luZyB0aGVtCgpOb3cgd2UgY2FuIGNvbXB1dGUgdGhlICpjbGFzc2lmaWNhdGlvbiBlcnJvciogcmF0ZSBieSBjb21wYXJpbmcgYHByZWRpY3RlZC55YCBhZ2FpbnN0IHRoZSBleHBlY3RlZCAkeSQ6CmBgYHtyfQpsZW5ndGgod2hpY2gocHJlZGljdGVkLnkhPWQkU3BlY2llc1staW5kZXhlcy5sZWFybmluZ10pKS9sZW5ndGgocHJlZGljdGVkLnkpCmBgYApOb3RlIHRoYXQgYHdoaWNoKClgIHJldHVybnMgdGhlIGluZGV4ZXMgb2YgYSBCb29sZWFuIHZlY3RvciBlbGVtZW50cyB0aGF0IGFyZSB0cnVlLgoKQW5vdGhlciB3YXkgaXMgdG8gImNvbXB1dGUiIHRoZSAqKmNvbmZ1c2lvbiBtYXRyaXgqKiBhbmQgdGhlbiBvYnRhaW5pbmcgdGhlIGVycm9yIGZyb20gdGhhdC4KVGhlIGNvbmZ1c2lvbiBtYXRyaXggc2hvd3MgdGhlIG51bWJlciBtaXNjbGFzc2lmaWNhdGlvbnMsIGNsYXNzIGJ5IGNsYXNzOgpgYGB7cn0KdGFibGUocHJlZGljdGVkLnksIGQkU3BlY2llc1staW5kZXhlcy5sZWFybmluZ10pCmBgYApHaXZlbiB0aGF0IG1hdHJpeCwgdGhlIGFjY3VyYWN5IG9mIGNsYXNzaWZpY2F0aW9uIGlzOgpgYGB7cn0KY29uZi5tYXRyaXggPSB0YWJsZShwcmVkaWN0ZWQueSwgZCRTcGVjaWVzWy1pbmRleGVzLmxlYXJuaW5nXSkKc3VtKGRpYWcoY29uZi5tYXRyaXgpKS9zdW0oY29uZi5tYXRyaXgpCmBgYAphbmQgdGhlIGVycm9yIHJhdGUgY2FuIGJlIGNvbXB1dGVkIGFzOgpgYGB7cn0KZXJyb3Iuc3BlY2llcyA9IDEtc3VtKGRpYWcoY29uZi5tYXRyaXgpKS9zdW0oY29uZi5tYXRyaXgpCmVycm9yLnNwZWNpZXMKYGBgCgojIyMjIEFzIGEgZnVuY3Rpb24KCk91dCBvZiBzaW1wbGljaXR5LCB3ZSBtaWdodCBidWlsZCBhIGZ1bmN0aW9uIHRoYXQgZG9lcyBhbGwgdGhvc2Ugb3BlcmF0aW9ucyB0b2dldGhlciwgd2l0aCBzb21lIHBhcmFtZXRlcnM6CmBgYHtyfQpjb21wdXRlRXJyb3JSYXRlID0gZnVuY3Rpb24oY2F0ZWdvcmljYWwueS5uYW1lLCBkYXRhLCBsZWFybmVyLCBwLmxlYXJuID0gMC44LCAuLi4pIHsKICBpbmRleGVzLmxlYXJuaW5nID0gc2FtcGxlKGMoMTpucm93KGRhdGEpKSlbMToobnJvdyhkYXRhKSpwLmxlYXJuKV0KICBtb2RlbCA9IGxlYXJuZXIoZm9ybXVsYShwYXN0ZTAoY2F0ZWdvcmljYWwueS5uYW1lLCJ+LiIpKSwgZGF0YVtpbmRleGVzLmxlYXJuaW5nLF0sIC4uLikKICBwcmVkaWN0ZWQueSA9IHByZWRpY3QobW9kZWwsIGRbLWluZGV4ZXMubGVhcm5pbmcsIF0sIHR5cGU9ImNsYXNzIikKICBsZW5ndGgod2hpY2gocHJlZGljdGVkLnkhPWRbLWluZGV4ZXMubGVhcm5pbmcsIGNhdGVnb3JpY2FsLnkubmFtZV0pKS9sZW5ndGgocHJlZGljdGVkLnkpIAp9CmBgYApOb3RlIHRoYXQ6CgotIGBwLmxlYXJuID0gMC44YCBpbiB0aGUgc2lnbmF0dXJlIG1lYW5zIHRoYXQgdGhlcmUgaXMgYSBkZWZhdWx0IHZhbHVlIGZvciB0aGlzIHBhcmFtZXRlcgotIGAuLi5gIGluIHRoZSBzaWduYXR1cmUgZGVub3RlcyAiYW55IG90aGVyIHBhcmFtZXRlciIsIHRoYXQgY2FuIGJlIHRoZW4gcGFzc2VkIHRvIG90aGVyIGZ1bmN0aW9ucyB3aXRoIGAuLi5gIGFnYWluCi0gdGhlIHJldHVybiBzdGF0ZW1lbnQgaXMgaW1wbGljaXQ6IHRoZSByZXR1cm5lZCB2YWx1ZSBpcyB0aGUgbGFzdCBjb21wdXRlZCBvdXRjb21lIGp1c3QgYmVmb3JlIHRoZSBlbmQKLSBgbGVhcm5lcmAgaGVyZSBpcyBpbnRlbmRlZCB0byBhc3N1bWUgZnVuY3Rpb25zIGFzIHZhbHVlcyAodHJ5IHdpdGggYHRyZWVgIG9yIGBycGFydGApCgpgYGB7cn0KcHJpbnQoY29tcHV0ZUVycm9yUmF0ZSgiU3BlY2llcyIsIGQsIHRyZWUpKQpwcmludChjb21wdXRlRXJyb3JSYXRlKCJTcGVjaWVzIiwgZCwgcnBhcnQpKQpgYGAKCkEgc2ltaWxhciBmdW5jdGlvbiBjb3VsZCBiZSBtYWRlIGZvciBkb2luZyBrLWZvbGQgQ1Ygb3IgTE9PQ1YuCkRvIGl0IGJ5IHlvdXJzZWxmLgoKV2UgY2FuIGRvIG5pY2UgdGhpbmdzIHdpdGggc3VjaCBhIGZ1bmN0aW9uLgpGb3IgZXhhbXBsZSwgd2UgY2FuIGNvbXB1dGUgdGhlIGVycm9yIGZvciBkaWZmZXJlbnQgdmFsdWVzIG9mIHRoZSBsZWFybmluZy10by10ZXN0IGRhdGEgcmF0aW8gKHRoYXQgMC44IGFib3ZlKToKYGBge3IsZWNobz1GfQpyZXF1aXJlKHRpZHl2ZXJzZSkKYGBgCmBgYHtyfQpyYXRpbyA9IHNlcSgwLjEsIDAuOTksIDAuMDUpCmVycm9yID0gcmF0aW8gJT4lIG1hcF9kYmwoZnVuY3Rpb24ocil7Y29tcHV0ZUVycm9yUmF0ZSgiU3BlY2llcyIsIGQsIHRyZWUsIHIpfSkKdGliYmxlKHJhdGlvPXJhdGlvLCBlcnJvcj1lcnJvcikgJT4lIGdncGxvdChhZXMoeD1yYXRpbyx5PWVycm9yKSkrZ2VvbV9saW5lKCkreWxpbSgwLDEpCmBgYAoKIyMjIyBGb3IgcmVncmVzc2lvbgoKTm93LCBsZXQncyBkbyAidGhlIHNhbWUiIG9uIHRoZSBmaXJzdCBudW1lcmljYWwgdmFyaWFibGU6CmBgYHtyfQpuYW1lcyhkKQpgYGAKCmBgYHtyfQp0LnNsID0gdHJlZShTZXBhbC5MZW5ndGh+LiwgZFtpbmRleGVzLmxlYXJuaW5nLF0pCnByZWRpY3RlZC55ID0gcHJlZGljdCh0LnNsLCBkWy1pbmRleGVzLmxlYXJuaW5nLF0sIHR5cGU9InZlY3RvciIpCnByZWRpY3RlZC55CmBgYAoKV2FpdCEgTm93IHRoZSAkeSQgaXMgbnVtZXJpY2FsLCB3ZSBjYW5ub3QganVzdCBjaGVjayBmb3IgZXF1YWxpdHksIHdlIG5lZWQgdG8gY29tcHV0ZSBhbm90aGVyIG1ldHJpYyBpbnN0ZWFkIG9mIHRoZSBjbGFzc2lmaWNhdGlvbiBlcnJvci4KV2UgY291bGQgZ28gZm9yIHRoZSBSU1MsIG9yIHRoZSBNQUUsIG9yIHRoZSBNU0UuLi4KTGV0J3MgZ28gd2l0aCB0aGUgTUFFICgqbWVhbiBhYnNvbHV0ZSBlcnJvciopOgpgYGB7cn0KZXJyb3Iuc2wgPSBtZWFuKGFicyhwcmVkaWN0ZWQueS1kJFNlcGFsLkxlbmd0aFstaW5kZXhlcy5sZWFybmluZ10pKQplcnJvci5zbApgYGAKCkFnYWluLCB3ZSBjYW4gd3JpdGUgYSBmdW5jdGlvbiBmb3IgZG9pbmcgZXZlcnl0aGluZzoKYGBge3J9CmNvbXB1dGVNQUUgPSBmdW5jdGlvbihudW1lcmljYWwueS5uYW1lLCBkYXRhLCBsZWFybmVyLCBwLmxlYXJuID0gMC44LCAuLi4pIHsKICBpbmRleGVzLmxlYXJuaW5nID0gc2FtcGxlKGMoMTpucm93KGRhdGEpKSlbMToobnJvdyhkYXRhKSpwLmxlYXJuKV0KICBtb2RlbCA9IGxlYXJuZXIoZm9ybXVsYShwYXN0ZTAobnVtZXJpY2FsLnkubmFtZSwifi4iKSksIGRhdGFbaW5kZXhlcy5sZWFybmluZyxdLCAuLi4pCiAgcHJlZGljdGVkLnkgPSBwcmVkaWN0KG1vZGVsLCBkWy1pbmRleGVzLmxlYXJuaW5nLCBdLCB0eXBlPSJ2ZWN0b3IiKQogIG1lYW4oYWJzKHByZWRpY3RlZC55LWRbLWluZGV4ZXMubGVhcm5pbmcsIG51bWVyaWNhbC55Lm5hbWVdKSkKfQpgYGAKCiMjIFN0ZXAgMzogZXhlY3V0aW9uIGFuZCBhbnN3ZXIKClNpbmNlIHdlIGluZGVlZCBidWlsdCB0aGUgZnVuY3Rpb25zLCB3ZSBjYW4gbm93IGNvbXB1dGUgdGhlIGVycm9ycyBmb3IgYWxsIHRoZSB2YXJpYWJsZXM6CmBgYHtyfQpmb3IgKHZhcmlhYmxlIGluIG5hbWVzKGQpKSB7CiAgaWYgKGlzLm51bWVyaWMoZFssdmFyaWFibGVdKSkgewogICAgZXJyb3IgPSBjb21wdXRlTUFFKHZhcmlhYmxlLCBkLCB0cmVlKQogIH0gZWxzZSB7CiAgICBlcnJvciA9IGNvbXB1dGVFcnJvclJhdGUodmFyaWFibGUsIGQsIHRyZWUpCiAgfQogIGNhdChwYXN0ZSh2YXJpYWJsZSwgZXJyb3IsICJcbiIpKQp9CmBgYAoKQXBwYXJlbnRseSwgdGhlIGxhcmdlc3QgZXJyb3IgaXMgb2J0YWluZWQgZm9yIFBldGFsLkxlbmd0aCwgc28gaXQgaXMgdGhlIGhhcmRlc3QgdmFyaWFibGUgdG8gYmUgcHJlZGljdGVkLgoKQnV0LCB3YWl0IQoqKkNvbXBhcmluZyBhbiBlcnJvciByYXRlIChzb21ldGhpbmcgdGhhdCBpcywgYnkgZGVmaW5pdGlvbiwgaW4gJFswLDFdJCkgYWdhaW5zdCBhIE1BRSAodGhhdCBpcywgaW4gZ2VuZXJhbCwgdW5ib3VuZGVkKSwgaXMgcG9pbnRsZXNzISoqCihBcyBhbiBleGFtcGxlLCB0cnkgbXVsdGlwbHlpbmcgb25lIG9mIHRoZSBudW1lcmljYWwgdmFyaWFibGUgYnkgMTAgYW5kIHNlZSB0aGF0IHRoZSBlcnJvciB3aWxsIGluY3JlYXNlIGJ5IDEwIHRvbyEpLgpUaGlzIGlzIGludHJpbnNpY2FsbHkgcmVsYXRlZCB0byB0aGUgbmF0dXJlIG9mIHRoZSAkeSQgaW4gdGhlIHR3byBjYXNlczogY2F0ZWdvcmljYWwgYW5kIG51bWVyaWNhbC4KSXQgdHVybnMgb3V0LCBoZW5jZSwgdGhhdCB0aGUgaWRlYSBvZiB1c2luZyAidGhlIGVmZmVjdGl2ZW5lc3Mgb2YgYSBtb2RlbCBsZWFybmVkIHdpdGggZGVmYXVsdCBwYXJhbWV0ZXIiIGFzIGEgcHJveHkgZm9yIHRoZSBoYXJkbmVzcyAoYWN0dWFsbHksIHRoZSBvcHBvc2l0ZSBvZiB0aGUgZWZmZWN0aXZlbmVzcyksIGlzIHdyb25nLgoKSG93IGNhbiB3ZSBvdmVyY29tZSB0aGlzIGxpbWl0YXRpb24/CgojIyBBbHRlcm5hdGl2ZSBzdGVwIDE6IHJlLWRlc2lnbgoKU2luY2UgdGhlIHByb2JsZW0gaXMgdGhhdCB3ZSBjYW5ub3QgY29tcGFyZSB0aGUgYWJzb2x1dGUgdmFsdWVzIG9mIHRoZSBlcnJvciwgb25lIHBvc3NpYmlsaXR5IGlzIHRvIGNvbnNpZGVyIHRoZSBjb21wbGV4aXR5IGEgbW9kZWwgZXhoaWJpdHMgZm9yIG9idGFpbmluZyBhbiBlcnJvciBvZiAwLgpUaGUgcmF0aW9uYWxlIGlzIHRoYXQgMCBpcyBjb21wYXJhYmxlIHdpdGggMCwgc2luY2UgdGhleSBib3RoIHJlcHJlc2VudCB0aGUgYWJzZW5jZSBvZiBhbnkgZXJyb3IsIHJlZ2FyZGxlc3Mgb2YgdGhlIHNjYWxlLgpXZSBuZWVkIGhlbmNlIGEgd2F5IGZvciBtZWFzdXJpbmcgdGhlIGNvbXBsZXhpdHk6IGZvciB0cmVlcywgdGhpcyBpcyBlYXN5LCBzaW5jZSB3ZSBjYW4gc2ltcGx5IGNvdW50IHRoZSBudW1iZXIgb2Ygbm9kZXMuCgpTbyBzdW1tYXJpemluZywgdGhlIG5ldyBtZXRob2QgZm9yIGFuc3dlcmluZyB3b3VsZCBiZToKCjEuIGZvciBlYWNoIHZhcmlhYmxlCiAgICBhLiBsZWFybiBhIHRyZWUgdGhhdCBvYnRhaW5zIGEgMCBlcnJvciAKICAgIGIuIGNvdW50IHRoZSBudW1iZXIgb2Ygbm9kZXMgb2YgdGhlIHRyZWUKMi4gc2F5IHRoYXQgdGhlIGhhcmRlc3QgdmFyaWFibGUgaXMgdGhlIG9uZSBmb3Igd2hpY2ggdGhlIGNvdW50IGlzIHRoZSBncmVhdGVzdAoKTm90ZSB0aGF0IHdpdGggdGhlIHRyZWUgbGVhcm5pbmcgcHJvY2VkdXJlIHRoYXQgd2Uga25vdywgb25lIGNhbiBvYnRhaW4gYSB0cmVlIHdpdGggZXJyb3IgMCBieSBzaW1wbHkgc2V0dGluZyAkbl9cdGV4dHttaW59JCAocHJvdmlkZWQgdGhhdCB0aGUgZGF0YXNldCAkXHsoeF57KGkpfSx5XnsoaSl9KVx9X2kkIGRvZXMgbm90IGNvbnRhaW4gZGlmZmVyZW50IG9ic2VydmF0aW9ucyB3aXRoIHRoZSBzYW1lICR5JCkuCgojIyBBbHRlcm5hdGl2ZSBzdGVwIDI6IGltcGxlbWVudGF0aW9uCgpXZSBuZWVkIHRvIGZpbmQgb3V0OgoKLSBob3cgdG8gc2V0IHRoZSAkbl9cdGV4dHttaW59JCBpbiBgdHJlZSgpYAotIGhvdyB0byBjb3VudCB0aGUgbnVtYmVyIG9mIG5vZGVzIGluIGEgdHJlZQoKQnkgZGlnZ2luZyB0aGUgZG9jdW1lbnRhdGlvbiBvZiBgdHJlZSgpYCB3ZSBmaW5kIHRoYXQgdGhlIHBhcmFtZXRlcnMgZm9yIHRoZSBsZWFybmluZyBwcm9jZWR1cmUgYXJlIGRlc2NyaWJlZCBpbiB0aGUgZG9jdW1lbnRhdGlvbiBvZiB0aGUgZnVuY3Rpb24gYHRyZWUuY29udHJvbCgpYC4KSXQgaXMgc2FpZCB0aGF0OgoKPiBUbyBwcm9kdWNlIGEgdHJlZSB0aGF0IGZpdHMgdGhlIGRhdGEgcGVyZmVjdGx5LCBzZXQgYG1pbmRldiA9IDBgIGFuZCBgbWluc2l6ZSA9IDJgLCBpZiB0aGUgbGltaXQgb24gdHJlZSBkZXB0aCBhbGxvd3Mgc3VjaCBhIHRyZWUuCgpCeSByZWFkaW5nIGZ1cnRoZXIsIHdlIGluZmVyIHRoYXQgYG1pbnNpemVgIGlzIHJlbGF0ZWQgd2l0aCAkbl9cdGV4dHttaW59JC4KCkxldCdzIHRyeS4KTm9ybWFsIHRyZWUgKGRlZmF1bHQgZmxleGliaWxpdHksIGkuZS4sIGRlZmF1bHQgdmFsdWUgZm9yIGBtaW5zaXplYCk6CmBgYHtyfQp0PXRyZWUoU3BlY2llc34uLCBkKQpwbG90KHQpCnRleHQodCkKYGBgCgpDb21wbGV4IHRyZWU6CmBgYHtyfQp0PXRyZWUoU3BlY2llc34uLCBkLCBtaW5kZXY9MCwgbWluc2l6ZT0yKQpwbG90KHQpCnRleHQodCkKYGBgClllcyEKSXQgaXMgbGFyZ2VyLCBhbmQgYWxzbyB1Z2xpZXIuLi4KKEFzIGEgY29uc2VxdWVuY2UsIGl0IGlzIGxlc3MgaW50ZXJwcmV0YWJsZSkuCgpDb25jZXJuaW5nIHRoZSBjb3VudCBvZiBub2RlcywgYnkgcmVhZGluZyB0aGUgZG9jdW1lbnRhdGlvbiBvZiBgdHJlZSgpYCBpbiB0aGUgVmFsdWUgc2VjdGlvbiwgd2UgZm91bmQgdGhhdCBhICJjb21wb25lbnQiIG9mIHRoZSByZXR1cm5lZCB2YWx1ZSwgbmFtZWQgYGZyYW1lYCBpczoKCj4gQSBkYXRhIGZyYW1lIHdpdGggYSByb3cgZm9yIGVhY2ggbm9kZSwgYW5kIGByb3cubmFtZXNgIGdpdmluZyB0aGUgbm9kZSBudW1iZXJzLgoKU28gd2UganVzdCBuZWVkIHRvIGNvdW50IHRoZSByb3dzIG9mIHRoYXQgZGF0YWZyYW1lOgpgYGB7cn0KbnJvdyh0cmVlKFNwZWNpZXN+LiwgZCkkZnJhbWUpCmBgYAoKSXQgd29ya3M6IGNoZWNrIHdpdGggdGhlIGNvcnJlc3BvbmRpbmcgaW1hZ2UgYWJvdmUuCgpTbyBsZXQncyB3cml0ZSBhIGZ1bmN0aW9uIGZvciBkb2luZyBldmVyeXRoaW5nOgpgYGB7cn0KY29tcHV0ZUNvbXBsZXhpdHlPZlBlcmZlY3RUcmVlID0gZnVuY3Rpb24gKHkubmFtZSwgZGF0YSkgewogIHQgPSB0cmVlKGZvcm11bGEocGFzdGUwKHkubmFtZSwifi4iKSksIGRhdGEsIG1pbmRldj0wLCBtaW5zaXplPTIpCiAgbnJvdyh0JGZyYW1lKQp9CmNvbXB1dGVDb21wbGV4aXR5T2ZQZXJmZWN0VHJlZSgiU3BlY2llcyIsIGQpCmBgYAoKIyMgQWx0ZXJuYXRpdmUgc3RlcCAzOiBleGVjdXRpb24gYW5kIGFuc3dlcgoKQXMgYmVmb3JlOgpTaW5jZSB3ZSBpbmRlZWQgYnVpbHQgdGhlIGZ1bmN0aW9ucywgd2UgY2FuIG5vdyBjb21wdXRlIHRoZSBlcnJvcnMgZm9yIGFsbCB0aGUgdmFyaWFibGVzOgpgYGB7cn0KZm9yICh2YXJpYWJsZSBpbiBuYW1lcyhkKSkgewogIGNvbXBsZXhpdHkgPSBjb21wdXRlQ29tcGxleGl0eU9mUGVyZmVjdFRyZWUodmFyaWFibGUsIGQpCiAgY2F0KHBhc3RlKHZhcmlhYmxlLCBjb21wbGV4aXR5LCAiXG4iKSkKfQpgYGAKClNvLCBhY2NvcmRpbmcgdG8gdGhpcyBtZXRob2QsIGl0IHR1cm5zIG91dCB0aGF0IHRoZSBoYXJkZXN0IHZhcmlhYmxlIGlzIFNlcGFsLldpZHRoLCBkaWZmZXJlbnRseSB0aGFuIHdpdGggdGhlIHByZXZpb3VzIG1ldGhvZC4KCk5vdGUgdGhhdCB0aGUgaW1wb3J0YW50IHRoaW5nIGhlcmUgaXMgbm90IHRoZSBhY3R1YWwgYW5zd2VyLCBidXQgdGhlIGZ1bGwgcGF0aCB0byB0aGUgYW5zd2VyLgpOb3RlIGFsc28gdGhhdCB0aGlzIG1ldGhvZCBzdGlsbCBoYXMgdGhlIGRyYXdiYWNrIG9mIGNvbXBhcmluZyBhcHBsZXMgYW5kIG9yYW5nZXM6IGluZGVlZCB0aGUgY29tcGxleGl0eSBvZiB0aGUgb2J0YWluZWQgdHJlZXMgY2xlYXJseSBzaG93cyB0aGF0IHRoZSB2YWx1ZXMgYXJlIHZlcnkgZGlmZmVyZW50IGZvciB0aGUgY2F0ZWdvcmljYWwgYW5kIG51bWVyaWNhbCB2YXJpYWJsZXMuClRoaXMgaXMgYmVjYXVzZSB0aGUgZm9ybWVyIGp1c3QgdGFrZXMgMyB2YWx1ZXMsIHdoaWxlIHRoZSBsYXR0ZXIgb25lcyB0YWtlIG1hbnkgb2YgdGhlbTogd2hlbiB3ZSBpbXBvc2UgdGhlIFJTUz0wIGFzIHRoZSBvbmx5IHN0b3BwaW5nIGNyaXRlcmlvbiwgYWxsIG9mIHRoZW0gYmFzaWNhbGx5IGFjdCBhcyBjbGFzcyB2YWx1ZXMuIEl0IGlzIGhlbmNlIHNvbWVob3cgZXhwZWN0ZWQgdGhhdCBtdWNoIG1vcmUgbm9kZXMgYXJlIG5lZWRlZCB0byBwZXJmZWNseSBhY2NvbW9kYXRlIGFsbCB0aG9zZSBjbGFzcyB2YWx1ZXMu