Abstract

Remotely sensed images serve as a valuable source of present and archival information since they provide the geographical distribution of natural and cultural features both spatially and temporally, as well as objects on the earth’s surface. Three machine learning classifiers, namely artificial neural networks (ANN), support vector machines (SVM), and decision tree (DT) algorithms, were applied in order to classify the Sentinel-2A data over the city of Soran. The differences in classification accuracies were evaluated by the confusion matrix. The supervised ANNs obtained the most accurate classification accuracy as compared with support SVM and DTs algorithms. Furthermore, the overall accuracy assessment of ANN was 90%, with SVM at 65%, while DTs were 60%. It can be concluded that ANNs can provide the best classification machine learning technique for land cover classification.

1. Introduction

For last five decades Landsat imagery has presented a unique resource for various researchers and scientists through their studies in many and disparate areas such as forestry, education, agriculture, etc. ¹. In addition, the recent Sentinel-2A multispectral satellite sensor was successfully launched by the European Space Agency (ESA) on 23 June 2015. It provides images for various purposes, and whose characteristics can meet the vast majority of analysis requirements ².

The successful remotely sensed classification is an essential source for many application processes ³, because many environmental, social and economic applications are based on the results of classification ⁴. Moreover, to obtain a successful classification, a suitable classification system is required ⁵. Therefore, the submission of research objectives, questions, and problems are necessary from the end user before employing the classification ³. On the other hand, there are many factors must be taken into account when choosing a classification method for use, such as the spatial resolution of the remotely sensed data, different sources of data, a classification system, and availability of appropriate classification software ⁴. In general, the purpose of image classification is to predict any entered image categories using its features ⁶.

Different classification techniques have been studied by many researchers regarding the accuracy of their maps. Image processing and classification approaches, may affect the success of the classification, as the classification of remote-sensing is a complex process and requires consideration of a number of factors ⁴. Various types of algorithms are used to provide a suitable accuracy of classification ³. Classifying remotely sensed data is one of the basic steps to extracting information ⁶, via any of various classification techniques ⁴. Many advanced classification approaches, in the last two decades, have been applied for image classification such as artificial neural networks (ANNs) ^{7, 8}, support vector machines (SVMs) ^{9, 10}, decision trees (DTs) ^{11, 12}, spectral angle classifiers ^{13, 14}, and rule-base evidential reasoning for building expert and decision-making systems ^{15, 16}. These aforementioned methods are non-parametric classifiers, which do not employ statistical parameters to calculate class separation.

The ideal map is one that has been well validated and achieved, and which should consider a range of factors such as methods for collecting reference points, classification scheme, sample method collection, sample size and sample unit, and the accuracy assessment calculation ^{17, 18}. Furthermore, other important accuracy assessment elements can be derived from the confusion matrix such as producer's accuracy, user’s accuracy, and overall kappa statistics. However, obtaining the accuracy assessment is more difficult than classification ¹⁹.

The performance of different machine learning classifiers, as evaluated through many comparative studies have been investigated ^{1, 20, 21, 22, 23, 24}. For instance, Szantoi, Escobedo ²⁰ implemented ANN and DTs on heterogeneous wetland communities, and found that ANN significantly outperformed the DT classifier. Chen ²¹ found that Neural Networks were the most efficient algorithm and outperformed SVM and DTs. Huang, Davis ¹ assessed SVM for land cover classification, and compared ANN and DTs. They found that SVM can produce a more accurate classification. Khatami, Mountrakis ²² found that SVM considerably outperformed neural networks and decision trees in a direct comparison. Otukei and Blaschke ²³ compared DTs, SVM, and MLC for assessing land cover change, and found DTs to be more effective than SVM or MLC. Mohamed ²⁴ revealed that the classification accuracy of the DT algorithm was better than the ANN algorithm in his comparative research. Otukei and Blaschke ²³ showed decision trees to give better results than MLC and SVM techniques for land cover change assessment. Song and Ying ²⁵ successfully showed the decision tree method to be a powerful statistical tool for classification, prediction, and interpretation that has several potential applications in medical research. Friedl and Brodley ²⁶ found improved accuracy in complex LULC classifications using a decision tree algorithm.

The objectives of this work are to (i) compare the classification accuracy of three different algorithms and (ii) to propose the most suitable algorithm for LU/LC over the city of Soran using Sentinel- 2A data for 2019.

2. Materials and Methods

Soran was selected as a study area from the previous work by ²⁷. The free and open source Sentinel-2A Level image was downloaded from the Copernicus website (https://scihub.copernicus.eu/) with a total of thirteen spectral bands, as acquired on August 20, 2019. The Sentinel-2A data is widely used in land cover classification. For this work, four bands - visible (blue, green, red) and near-infrared with a special resolution of 10 m - were used for the classification, as per Figure 1.

The use of artificial neural networks in remote sensing studies dates back to the early 1990s ²⁸. An artificial neural network (ANN) is a mathematical model ²⁹ that takes the human brain as a model for problem solving ³⁰. An ANN is also referred to as a simulated neural network (SNN) or, more commonly, just a neural network (NN). An ANN is an interconnected group of artificial neurons that uses a mathematical model to manipulate information based on a mathematical link approach. Furthermore, an ANN is an adaptive system that changes its structure based on external or internal information that flows across the network ³¹.

A single ANN architecture typically consist of an input, one or more hidden, and an output layer ³². Kanellopoulos and Wilkinson ³³ identified that the classification accuracy with an ANN is often greater than traditional statistical classifiers. Each layer in an ANN is made up of many neurons where each neuron signifies a variable in the input layer ³⁰, therefore the stimulations work in the same way as biological neural networks in the human brain process information ²⁹. There will be one output node for each class in the classification system, through the image classification process. The general shape of an ANN is shown in Figure 2.

A neural network is generated for a specific application, such as pattern recognition or data classification, through a learning process. ANN is considered the best method in a problem solving approach for traditionally problematic techniques for which algorithmic solution is too complex to be found, or for which such a solution does not exist ³¹. Also, neural networks can perform image categorization reasonably quickly, although the training process itself can be extremely time consuming ³⁵.

Boser, Guyon, and Vapnik first presented support vector machines (SVMs) formally in 1992 ³⁶. The SVM method is a set of non-parametric supervised learning method used for classification techniques ³⁷, and tries to generalize and provide reasonable predictions for new datasets by creating a hyperplane that separates the dataset into classes. Polynomial, radial basis functions, and sigmoid kernels are the most commonly used kernels for image processing ³⁸. SVM based on statistical learning determines the border for distinguishing data belonging to two classes from each other in an optimal manner ¹ if the data to be classified are separated linearly or non-linearly ³⁷. Thus, it can be used for linearly and non-linearly separable data.

The key concept of this discriminant machine learning technique ³⁹ is to find an optimal hyperplane or the largest amount of margin among many hyperplanes between two categories as possible ³⁶, as per Figure 3. Thus, it seeks the ideal hyperplane to find the maximum margin in order separate the classes ⁴⁰, which is why SVM is known as a large margin classifier ³⁹.

SVM was developed from powerful implementation theory, whereas ANNs were experimentally moved from application to theory. SVM does not control the complexity of the model, as ANN does; instead, it determines the complexity model automatically by setting the number of support vectors ³⁹. It has been found that SVM generally outperforms neural networks, which is novel pattern recognition method ^{1, 9}.

Finally, support vector machines have been selected as a group of relatively novel statistical learning algorithms in classifying homogeneous and heterogeneous land cover types, with the associated vigour being confirmed by many various researchers ⁴⁰.

Decision Trees (DTs), or Decision Tree Classifiers (DTCs), are a supervised machine learning method and are one of many classifiers from statistical-based algorithms ⁴² which have been successfully applied in a variety of fields for digital image classification ⁴³. A DTC is a non-parametric classifier ⁴⁴, which is an effective and popular technique for classification ⁴⁵. Decision tree classification problems can be solved through identifying to which set an object belongs ⁴⁶. The tree in the DT approach constitutes a root node at the top, a few hidden nodes (internal nodes) which split the objects into different categories, and a large number of terminal nodes (known leaves which contain most homogeneous classes), which is the output ^{45, 47}. Generally speaking, a decision tree asks a question and then classifies ⁴⁸ through defining the best associated tree structure and decision boundaries ²³.

3. Results

In this work, three machine learning classifiers were compared for LULC classification, namely artificial neural networks, support vector machines, and decision trees.

The confusion matrix results of all three algorithms are reported in Table 1 - Table 3 with overall accuracy, producer’s accuracy, user’s accuracy, and Kappa coefficient for the classified maps for 2019 using Sentinel-2A imaging ³⁵. The overall classification accuracies for ANN, SVM, and DTs are 90%, 65% and 60%, and their Kappa coefficients are 0.86, 0.57, and 0.53, respectively.

The classified areas were measured in the percentage, as per Figure 4. Barren land was the predominant class followed by built-up areas, cultivated land and riparian zones, which covered 51.93%, 29.28%, 15.56% and 3.21%, respectively, according to the ANN algorithm. In the instance of the SVM algorithm, the built-up occupied was 50.25% followed by barren land at 42.6%, cultivated land at 22.06% and riparian zone at 3.07%. By comparison, the decision tree classifier produced another set of results. Furthermore, the maximum percentage was occupied by barren land at 39.42%, whereas the minimum class was computerized for cultivated land by 2.87%. However, the built-up area and riparian zones had almost the same percentage coverages at 29.05% and 28.63%, respectively.

The land use/land cover maps are shown in Figure 5 for the three algorithms used, including the natural colour composites. The red colour represents built-up area, orange indicates barren land, and the light green colour indicates cultivated land, while dark green represents riparian zone.

4. Discussion

The results of the current work verified that the highest classification accuracy was obtained by ANN using the Sentinel-2 data. Most pixels which belong to classes were correctly classified. Thus, this indicates promising results through the application of the ANN to the whole dataset. On the other hand, it achieved only poor results for pixels in classes characterized by lower classification rates, as revealed by both SVM and DTs. The demand for the use of ANN in many real-world applications is increasing with time ⁴⁹ as a result of, their utility in solving problems that do not have algorithmic solutions is due to their powerful methodology ³¹.

In spite of the fact that SVM has been successfully used for data classification in many studies, but it did not prove particularly effective in the current work. In general, the main reasons for SVM producing sub-optimal results was due to the weakness of the soft margin optimization problem and the imbalanced support vector ratio ⁵⁰.

For instance, in the current work, the number of training cases for built-up area are significantly outnumbered by barren land and slightly so by cultivated land, as per Figure 4. More generally, the training data sets for barren land and cultivated land fell into the built-up area classes, thus the class boundary for built-up area learned by the SVMs are sharply skewed towards barren land and cultivated land. In other words, the dataset is imbalanced with regard to built-up area with barren land and cultivated land. This imbalanced dataset, as used to develop the separating hyperplane for the SVM model, was identified by ⁵¹. Thus, the correct distances and correct orientation of the hyperplane in learned SVMs do not perform at their best, which could be due to the hyperplane separation may not being exactly between two categories. Besides, this could also indicate a purely linear SVM classifier that led to the extremely poor performance observed, or in other words there is no clear separation data lining as identified by ³⁹. In general, it is difficult to explain SVM and different related learning algorithms ⁵².

In the current work, the decision tree algorithm did not yield promising results though it has been successfully used in various other fields ^{23, 44, 53}. Kumar and Kumar ⁵⁴ focused on two main disadvantages of DTs, which are overfitting and not being fit for continuous variables. Overfitting, which is a common problem with decision trees, was due to noise in the current study, and can be observed as misclassification of riparian zone, cultivated land, and barren land in the decision tree classifier. Besides, overfitting due to lack of samples could also be another reason for overfitting. Furthermore, the decision trees lost information between three aforementioned classes by categorization variables, or in other words the entropy which is a measure of the uncertainty between classes and that produced low accuracy output from the test data. However more generally, decision trees are prone to overfitting because they are very data concentrated stated Choy and Flom ⁵⁵.

Furthermore, the tree in the decision trees is highly dependent on the training data used, therefore any slight change in the data can cause large variation in the estimated tree, which subsequently creates a completely different tree. Another reason could be due to the physical boundaries between zones. Soran has a homogeneous landscape and all classes can be easily distinguished and separated by the naked eye, thus DTs did not perform well with the physical boundaries between zones. Finally, a lack of, or no interactions between variables also prevent DTs from working particularly well ⁵⁶.

5. Conclusions

The analyses showed differences in the classification accuracies obtained with different algorithms. Artificial neural networks produced considerably better results than the support vector machine and decision trees algorithms, in the current work. In general, SVM and DTs were not as effective at representing the classification over the city of Soran. Hence, ANN is a robust method for classification and detection of change, and could potentially overcome the mixed pixel problem in the image. Relatively speaking, an accurate LULC product can be achieved using ANNs on the recently acquired Sentinel-2 sensor imagery. Thus, using Sentinel-2 data may result in better resolution, and certainly more accurate LULC produces.

Acknowledgments

Many thanks to the Scientific Research Centre (SRC) at Soran University and to Dr. K. Kolo (SRC) for valuable comments on the manuscript.

Conflict of Interests

Author has declared that no competing interests exist.

References