Overview

Deep Learning (aka Artificial Neural Network or ANN) is undoubtedly the most sophisticated Machine Learning algorithm so far. It is designed to mimic how the human brain is believed to function. Deep Learning is purely Supervised Learning (with exception of Autoencoders) meaning that it needs pre-classified examples to learn from.

In most scenarios, data experts manually set the “response column”, which can be a class label or a numeric value. Supervised Machine Learning is great, but not without a caveat: It is both laborious and time consuming. On the other hand, providing fewer examples can adversely impact reliability.

So, the question here: is it possible to provide supervised training data with no supervision? Fortunately, the answer is yes (though not in every case).

Time Series

Time series is a set of data points, each of which represents the value of the same variable at different times (Oxford dictionary). Examples are shares prices, sales, temperature, air pressure and so on. Events are typically recorded at equal time or date intervals. In this blog, product sales will be used as a practical example.

In most cases, sales data are stored in Relational databases, which aren’t time series. A sample is illustrated in Table 1.

Table 1: raw sales data as seen in relational database

Each data sample, or observation, represents sold items. One thing should be obvious here: items appear at random intervals. While this may be the norm from a sales manager’s perspective, it is quite problematic for the data scientist because prediction (in the sense of inventory management) is impossible unless the time intervals are identical.

This problem, however, can be easily tackled by data aggregation techniques. Table 2 illustrates the dataset after aggregation. In this case, item sales are summed up on week basis. Below is the R code that aggregates item sales (dataset loaded into H2O).

The output of the R code is presented in Table 2.

Table 2: semi processed data sample with basic aggregation

Table 2 represents more “proper” time series dataset and indeed is ready to train our ANN after removing noise and outliers (i.e. errors and abnormal events). Noise can be found in large datasets. For example, when item code is stored in the Quantity filed due to human error. Outliers may reflect sales promotions and out of stock situations.

At this point, the target column (also known as response column) is Week_3. This is pretty good news for a data scientist for two reasons:

• The response column data is readily available.
• There is plenty of training data. In fact, as many as the total number of inventory items.

You may wonder why we would bother to predict Week_3 when we already have it. Makes no sense at all! I will come back to this point in a minute.

Our model managed to score Validation Deviance of 230. Not sufficiently good for commercial use. Increasing the number of epochs or hidden layers do not always improve accuracy (instead risk overfitting). So, the question here is how can we improve the model's performance?

One way to do so is to provide more item attributes such as item subcategory, product family, and so on. In this blog, I assume no useful attributes are available (because many real datasets do not).  Fortunately, time series actually allows us to make something out of nothing (almost)! The same dataset in Table 2 can be used to create new statistical attributes, namely: min, max, mean and median. The new and enhanced dataset is illustrated in Table 3. Note that some items did not generate sales at all during the period in question, hence the “NA” values.

Table 3: data sample with additional features

Feeding the data from Table 3 into our model resulted in much improved accuracy with validation deviance of 53 only (compared to 230). That’s over 4 times improvement. This isn’t perfect and additional tuning might be required (I typically stop when validation deviance is below 10) but should make the point clear.

Now it’s time to go back to the question: why would we predict Week_3 when it’s already known?

It is essential to understand how Deep Learning works in the first place. Broadly speaking, Deep Learning uses the response column during the training phase to adjust the weights between the neurons iteratively until the prediction and response column are closely matched. How “close” is determined by a number of ways, such as Validation Deviance (for prediction) and Confusion Matrix (for classification).

In the prediction phase, the column Week_1 is discarded. Weeks 2 and 3 become input variables while week4 (the future) becomes the new response column.

Table 4: prediction. Note that Week_1 was discarded and Week_4 is the new response column

Processing Performance

It is hard to talk about Machine Learning without mentioning High Performance Computing (HPC). Anyone involved in Machine Learning appreciates this for an obvious reason: how much time do you spend frustratingly gazing at your screen while waiting for the progress bar to hit 100%? The rule of thumb here is that “faster” or “bigger” machines yield better performance. In practice, however, it is desirable to find a balance between cost and performance.

 For the purpose of this blog, the same computation with same dataset was run on two common but extremely different hardware platforms: a fairly old Intel i3 laptop with 6GB of RAM, and a beefy workstation with an X series i7 6800K with 32GB of RAM. R was used for data pre-processing and manipulation. The actual Deep Learning model was trained on the same machines using H2O. The performance is summarised in the figure below:

Figure 1: Training throughput. Higher values indicate better performance

It is important to understand the performance characteristics of the intended Machine Learning platform as well as the dataset at hand. For example, does a faster machine (in terms of GHz) perform better, or a machine with more cores at lower GHz? Is the workload CPU-bound or memory-bound? Very large datasets can overstress the disks (especially mechanical disks) causing IO bottlenecks. Large Hadoop clusters suffer due to synchronization, especially on relatively slow networks.

H2O is an efficient in-memory, Map-Reduce platform that benefits from larger Cache memory. It scales very nicely with more CPU cores.

Figure 2: H2O puts all 6 cores (12 threads) to work

For more insight on performance, the interested reader can refer to my previous blogs Scaling Performance and Impact of Virtualization on Machine Learning for very detailed discussion.