Time Series Classification via Deep Learning

Screenshot of the developed dashboard showing the activity "standing" for the HAR dataset. The empty box is only relevant for the classification of segments and events.

IBK would like to investigate the potential of deep learning for applications in the flight physics domain. A first study in the field of time series classification has been carried out using the recordings from the Human Activity Recognition (HAR) dataset in a KERAS model. The signals, classes and the result of the LSTM / CNN network are visualized in an interactive dashboard. After a first successful demonstration the prototype shall now be tested on a more complex task, the classification of segments and events based on real flight recordings.

IBK is a small sized company in the aeronautical field. One of our fields of expertise are R&T activities in the flight physics domain, especially in cooperation with our LuFo partner AIRBUS. Therefore we are eager to reveal the potential of new methods considering the need of our partners and customers. Deep neural networks are an interesting candidate considering their recent successes. A project with a partner like AIRBUS would provide us the required amount of data for training of deep neural networks (DNN) and even the possibility of testing on a large scale with an increasing number of flight recordings thanks to the Skywise platform.

In an early study the potential of DNNs was investigated for classification of time series signals. In order to get started fast we used the Human Activity Recognition (HAR) dataset, which does not only provide recordings but also features derived from the recordings. This allows to compare the results of LSTM/CNN networks making use of the time signals with the results of classical machine learning methods (like random forest, logistic regression) making use of the features.

There are quite a few publications about the accuracy of several machine learning methods on UCI datasets incl. among other the HAR dataset. In general the authors claim that the DNNs outperform classical methods. However, at least w.r.t. the HAR dataset the achieved accuracy improvement is comparably small. The great advantage of the DNN is the application on almost unprocessed data, so that no time needs to be spend on feature engineering.

One of the disadvantages of neural networks in general are their black-box characteristics, but in the last years so called “Attention LSTMs” were introduced [1, 2, 3], which allow to understand the decision making of neural networks to some extent. Something similar can be achieved with a “global pooling” layer in a CNN [4]. Our prototype of a time series classifier is based on the publications [5, 6] and makes use of both techniques in order to retrieve the importance of the time signals and/or time points w.r.t. the time window used for classification.

The time signals of the dataset, the classes as well as the results of the DNN are visualized in an interactive dashboard. The development of this intuitive tool is motivated by the fast creation of training data sets (labeling of time slices) and the validation of the ML/DNN methods. The web application is coded in Python using the libraries dash and plotly and offers a multi-user experience.

The results of the demonstrator results are very promising and shall now be tested on the classification of segments and events using flight recordings.

[1]
Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio
Neural Machine Translation by Jointly Learning to Align and Translate
https://arxiv.org/pdf/1409.0473.pdf
2014

[2]
Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, Eduard Hovy
Hierarchical Attention Networks for Document Classification
https://www.cs.cmu.edu/~diyiy/docs/naacl16.pdf
2016

[3]
Colin Raffel, Daniel P. W. Ellis
Feed-Forward Networks with Attention Can Solve Some Long-Term Memory Problems
https://arxiv.org/pdf/1512.08756.pdf
2016

[4]
Min Lin, Qiang Chen, Shuicheng Yan
Network In Network
https://arxiv.org/pdf/1312.4400.pdf
2014

[5]
Karim, Fazle and Majumdar, Somshubra and Darabi, Houshang and Chen, Shun
LSTM fully convolutional networks for time series classification
https://arxiv.org/pdf/1709.05206.pdf
2017

[6]
Harford, Fazle Karim and Somshubra Majumdar and Houshang Darabi and Samuel
Multivariate LSTM-FCNs for Time Series Classification
https://arxiv.org/pdf/1801.04503
2018