LSTM model with exogenous factors

Question

LSTM model with exogenous factors

stupidwebsite

2022年4月25日 11:03

I have the following 3 columns in my dataset: 1.month, 2.day_of_week, 3.quantity.

I would like to predict the future values of quantity, having following variables as explanatory:

One-hot encoding of month (12 variables).
One-hot encoding of day_of_week (7 variables).
The last 2 lags of quantity (2 variables).

Could such an analysis be supported by an LSTM model? I believe I have managed to create an LSTM model which takes the 2 lags as explanatory, but I have no idea how to add the 19 exogenous factors (from the 2 one-hot encodings) as explanatory variables to the model.

Note: I am using the Keras python library for my implementation.

Topic lstm keras tensorflow rnn python

Category Data Science

10xAI · Accepted Answer · 2021年8月11日 02:27

RNN inputs are 3D tensors i.e.

inputs: A 3D tensor with shape [batch, timesteps, feature]

The last 2 lags mean you are considering the timestamps=2. But you should try with longer sequences and observe the result

You can create your dataset as [batch, 1, 20 ] i.e. 7+12+1 [Features] for seq_len=1.
For longer seq. just create the seq. accordingly i.e the way we create a general timeseries data e.g. [0,1,2,3] = [[0,1],[1,2],[2,3]] for seq_len=2.

A toy code -

mon = pd.get_dummies(np.random.randint(0,12,(1000)))
day = pd.get_dummies(np.random.randint(0,7,(1000)))
x = pd.DataFrame(np.random.randint(0,1000,(1000)))
data = pd.concat([x,mon,day], axis=1).to_numpy().reshape(-1,1,20)#seq_len=1, features=20

y = pd.get_dummies(np.random.randint(0,2,(1000)))

model = Sequential()
model.add(layers.GRU(100, return_sequences=True, input_shape=(1, 20)))
model.add(layers.GRU(100, return_sequences=False))
model.add(layers.Dense(1000, activation='relu'))
model.add(layers.Dense(2, activation='softmax'))

model.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=['accuracy'])
history = model.fit(data, y, epochs=10)

Side_note - OHE with a Neural network might not give the best result. Try adding Embedding layers before Month/Day. Or use some other encoding that gives continuous output.

LSTM model with exogenous factors

About