How to predict a mathematical progression with keras

I try the following model for a many-to-many recurrent network:

                              [170]     [682]     [2730]
                                |         |         |
       o  -------  o  --------  o  -----  o  -----  o
     / | \      /  |  \      /  |  \
   [2, 2, 3], [2, 10, 11], [2, 42, 43]
     {t=1}       {t=2}       {t=3}      {t=4}     {t=5}  

The model should be the following: x1* (x2+x3), Where [ x1=constant ]. [ x3=x2+1 ]. [ x2=x1* (x2+x3) of previous time {t-1} ]

This model should return the result of [ x1* (x2+x3) ] at times t=4; t=5; t=6.

Well, since this test model did not give me correct results, I decided to try it in the Keras and the result has also been wrong.

I leave the code in Keras to see if someone can help me get the correct results and explain why the model does not work.

import numpy as np
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
from keras.models import Sequential

# Data train
X = np.array([ [ [3,3,4], [3,21,22], [3,129,130] ],
               [ [2,2,3], [2,10,11], [2, 42, 43] ],
               [ [2,2,3], [2,10,11], [2, 42, 43] ],
               [ [4,4,5], [4,36,37], [4,292,293] ],
               [ [5,5,6], [5,55,56], [5,555,556] ]
              ],    dtype=int)

Y = np.array([ [[777 ], [4665 ], [27993 ] ],
               [[170 ], [682  ], [2730  ] ],
               [[170 ], [682  ], [2730  ] ],
               [[2340], [18724], [149796] ],
               [[5555], [55555], [555555] ]
              ], dtype=int)

# Model
def get_compiled_model():
  timesteps = 3 #times
  data_dim = 3  #features
  N = 3         #salidas
  model = Sequential()

  model.add(tf.keras.layers.LSTM(1, input_shape=(timesteps, data_dim)))
  model.add(tf.keras.layers.RepeatVector(N))
  model.add(tf.keras.layers.LSTM(1, return_sequences=True))  
  model.add(tf.keras.layers.TimeDistributed(tf.keras.layers.Dense(1)))
  model.add(tf.keras.layers.Activation('linear'))
  
  opt = keras.optimizers.Adam(learning_rate=0.9)
  model.compile(optimizer=opt, loss='MAE')
  return model

# main
model = get_compiled_model()
model.fit(X, Y, epochs=4000)

result = model.predict([[ [2,2,3], [2,10,11], [2,42,43] ]])
print(result)

The truth is that I tried this model using several types of layers in Keras (Dense layer, SimpleRNN, LSTM) and none of them have worked.

I leave an image of the loss curve

Topic keras tensorflow rnn neural-network machine-learning

Category Data Science

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.