Enhanced Vehicle Classification Using Transfer Learning and a Novel Duplication-based Data Augmentation Technique

Due to the traffic crisis seen in most of the large cities, intelligent transportation system (ITS) applications are widely installed to provide traffic management services. Vehicles’ classification, which is the final step in the vehicle detection process, has shown to be significant in many of these applications, especially those whose main interest is to classify vehicles in the context of monitoring roads and maintaining their safety. For instance, in sensitive areas like airports, it is only allowed for specific vehicle types to park or move while other types are not allowed. That is why in such areas there is a need, not only to detect vehicles but also to categorize their types.
In this Ph.D. dissertation, we proposed a robust Deep Neural Network (DNN) based computer vision model to classify vehicles. In the vehicle classification process, objects are initially classified into two classes; vehicle class or non-vehicle class, and then classified according to their types i.e. Crossover, Sedan, Hatchback, pickup, Van, and Minivan, etc. In the proposed DNN based model, we used Transfer Learning and data duplication- based- augmentation to reach the optimum classification performance. Based on the Transfer Learning approach, a pre-trained ResNet-50 network was used to obtain a highly effective learning model. we removed the final layers of the network, forwarded the produced feature maps, which are the relevant parameters representing the characteristics of the vehicles, and allowed their identifications, to the classification layer that is preceded by connected layers. Afterward, we re-trained the model on a small dataset that includes only vehicle’ categories of interest. Therefore, it learns only those categories’ features during this re-training process without being confused by the features of other classes learned earlier. The problem encountered in this context is that an effective Deep Neural Network usually requires a large amount of data to train on but actually, there isn’t as much data as needed. Hence, data augmentation techniques are implemented to increase, artificially, the amount of training data by using the existing data and, accordingly, improve the generalizability of the model. In the proposed method, a new approach for enhancing training data augmentation was proposed. In this approach, data augmentation by duplication was implemented through which the training dataset instance of each vehicle type was used side by side with their duplicates in every epoch. i.e. In the training process, each instance of each type of vehicle has duplicated multiple times in successive training sessions until the optimal learning results, that make the training process to be at its maximum performance, are reached.
We proved, empirically, that this technique of augmenting training data by duplication, enhances the classification performance by a considerable value. In the experiments, a testing dataset, including 640 images of 6 vehicle types was used to evaluate the proposed method. The model training was implemented in a staged manner. In the first stage, the training dataset has only the baseline instances of the vehicle dataset without any duplication. In every subsequent stage, the number of duplicates per image was increased by one after which the training was performed and the model was re-evaluated. The overall accuracy of the model was recorded in each stage. This is continued until the optimum classification accuracy has been reached.
To ensure the method generalization, we re-tested this optimum model using the Stanford-based custom dataset of 8,000 images and indicated that the model not only behaves properly with the testing dataset but also behaves similarly with a real world dataset. We compared the proposed method with the existing ones and it was shown that it outperforms all of them. From the experimental results, we proved that the overall classification accuracy has improved from 92.68 % without any duplications in the training dataset, to 99.70 % with 4 duplicates for every instance in the training dataset. In future work, it is intended to use the proposed method to enhance the fine-grained classification of vehicles.

内容要旨・審査要旨・論文本文の公開