Merge two or more ML models with different size of output togetger

I have several models that classify the input (word embedding) into several classes. My problem is that I need to train these models separately and need to merge the output of these models together to get a label.

For simplicity assume only two models:

Model 1: predicts A, B or C
Model 2: predicts D or E

Then, I need to classify the input X so that I get the joint probability over A, B, C, D and E.

I have tried this code in Keras/Python, in which I define neural networks and remove the softmax activation function after training:

inputs = tf.keras.Input(shape=dim_in)
x = layers.Dense(dim_out)(inputs)
outputs = layers.Activation(activation=softmax)(x)

model = tf.keras.Model(inputs=inputs, outputs=outputs, name='keras_model')
model_without_softmax = tf.keras.Model(inputs=model.inputs[0], outputs=model.layers[-2].output, name='keras_model', trainable=False)

model.compile(optimizer=tf.keras.optimizers.Nadam(),
                  loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
                  metrics=['sparse_categorical_accuracy'])

Then I get final prediction with softmax over joint arrays of predictions:

prediction = softmax(np.append(prediction_1, prediction_2))

It wasn't working because the output values from the neural networks were on a different scale when classifying into 2 or 3 classes. Thus, I tried dividing into a number of classes but without success:

prediction = softmax(np.append(prediction_1/len(idx2label_d1), prediction_2/len(idx2label_d2)))

Can you point me to right direction for the general formula of merging the neural network's outputs together?

Topic keras neural-network python machine-learning

Category Data Science


Maybe you can use the Concatenate layer

outputs = tf.keras.Concatenate()([model1, model2])
full_model = tf.keras.Model(inputs=inputs, outputs=outputs, name='full_model')

This will simply concatenate the two softmax output into one.

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.