Recurrent Neural Network :

What’s the need of the previous input state?

Table of Contents :

·      What is RNN?

·      Importance of hidden layer

·      Why do we need RNN?

·      Working of RNN.

·      Time Series Forecasting with RNN(LSTM) +         Code

·      Merits & Demerits

What is RNN?

RNN stands for Recurrent Neural Network –a type of Neural Network where the output from the previous step is fed as input to the current step. It is commonly used in speech recognition and natural language processing.

In a traditional neural network, all the inputs and outputs are independent of each other, but in cases like when it is required to predict the next word of a sentence, the previous words are required and hence there is a need to remember the previous words. Thus, RNN came into existence, which solved this issue with the help of a Hidden Layer. The main and most important feature of RNN is Hidden State, which remembers some information about a sequence.

neuron

Where,

Ht– Output (time step);

A – Hidden Layer;

Xt – Input (time step)

The Importance of the Hidden Layer in RNN.

  • A Hidden Layer is located between the input and output of the algorithm, in which the function applies weights to the inputs and directs them through an activation function as output.
  • Each hidden layer function is specialized to produce a defined output.
  • It allows for the donation of a neural network to be broken down into specific transformations of the data.

RNN has a “Memory” which remembers all information about what has been calculated. It uses the same parameter for each input, performing the same task on all the input or hidden layers to produce the output. This results in reducing the complexity of parameters, unlike other neural network.

Why do we need RNNs?

There are a few issues in the feed-forward network that result in creating the RNN.

The issues are as follows:

  1. Feedforward neural networks can’t handle sequential data.
  2. Feedforward neural network can’t find any memory or past input, but RNN is going to consider the previous input state as well as the current input state.

Working of RNN

RNN works on the principle of saving the output of a particular layer and feeding this back to the input in order to predict the output of the layer.

To properly understand how the RNN works, we need prior knowledge of “Normal Feed Forward Neural Network and Sequential data.

So now a question arises – What is Sequential data?

Sequential Data refers to any data that contain elements that are ordered into a sequence. (it includes time series, DNA sequence, sequence of any other user actions, etc).

RNN

Consider the above diagram in which we have, “x”, “y”, and “h” as the input layer, an output layer, and a hidden layer respectively. A, B, and C are the parameters of the network used to improve the output of the model.

  • At any given time t, the current input is a combination of input at x(t) and x(t-1).

 

Time Series Forecasting with RNN (LSTM)

  LSTM – It is a form of RNN, one of the most widely used networks for the purpose of time series forecasting.

Making the setup ready.

  1. Simple type googles collab on the google search bar
Google collab

     2. Open the first link. And then make a new notebook.

RNN code

      3. Rename this as per your wants.

          After that follow the below step.

Step by an illustration of the LSTM

  1. Import important library + use monthly milk production of a particular factory.
RNN(Lstm)

        2. Make sure that the pandas recognize that there is a date in the time series. Dealing with monthly dates.

RNN database

        3. To see how data looks like – monthly data of the production.

RNN Table

         4. Figure the size df.plot(figsize= (12,6))

RNN matplotlib

           5. Seasonal_decompose: used to decompose different parts of the series, select column then simply plot the graph.

RNN Technical

          Note – Check for stationary, when it is not stationary we have to make it stationary first.

          RNN doesn’t need any stationary data, they can also perform non-stationary data because RNN always deals with the complex dataset. But if still, it            is not predicting the output it is better a good way to make non-stationary a stationary dataset for the correct prediction.

           6. Check the length of the dataset and Split the dataset into training and testing parts. In the training section, we are going to use all the values                           except the last 12 months’ data. And leave the last 12 months as a testing state.

RNN Technical
  1.  Then preprocess the data, we used a minmaxscalar to convert the dataset on a scale of 0 to 7.1 And then just print the first 5 values and the last 5 value using head and tail.
RNN Technical

           7.2. Then scale transformation of both the data and then print the first 10 values.

             Formatting the data exactly to give it to the new molecules.

RNN Technical

7.3. Consider the first 3 values to predict the output i.e the 4th value.

RNN Technical

Now call the sequential class, dense class and LSTM class. And then create a model, and we can see the model summary as well.

RNN Technical

             8. How to actually made the prediction?

                  Fit the model.

RNN Technical
  • Call the generator which has an input or output for training a module and then plot the graph of loss.
RNN Technical graph
  • Picking the last 12 months’ value of training set to make a prediction of 1st month of the test state.
RNN Technical
  • Make an empty list of test predictions, the last 12 months’ values in order to make the predictions is stored as a current prediction. Append the prediction into test prediction. (1: means dropping the first input to make the prediction of another set). Print the test predictions.
RNN Technical

             9. Original testing values are in the range of 800 to 900. Consider scale as a object to perform the inverse transformation And print the graph.

RNN Technical

           10. To simple predict the numbers.

Merits of RNN

  1. The principal advantage of RNN is that RNN can model a collection of records (i.e. time collection) so that each pattern can be assumed to be dependent on previous ones.
  2. Recurrent neural networks are even used with convolutional layers to extends the powerful pixel neighbourhood.

Demerits of RNN

  1. Training an RNN is a completely tough task.
  2. Gradient exploding and vanishing problems.