How Did Keras Determine The Number of Parameters In My Model

I have the following keras model:

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers

model = keras.Sequential()
layer_in = keras.Input(shape=(256))
layer1 = layers.Dense(2, activation=relu, name=layer1)
layer2 = layers.Dense(3, activation=relu, name=layer2)
layer3 = layers.Dense(4, name=layer3)
model.add(layer_in)
model.add(layer1)
model.add(layer2)
model.add(layer3)
model.build()

Which produces the following when keras.summary() is called

Model: sequential_8
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 layer1 (Dense)              (None, 2)                 514       
                                                                 
 layer2 (Dense)              (None, 3)                 9         
                                                                 
 layer3 (Dense)              (None, 4)                 16        
                                                                 
=================================================================
Total params: 539
Trainable params: 539
Non-trainable params: 0
_________________________________________________________________

How did Keras determine the layers should have 514, 9, and 16 parameters respectively?

I would have thought that the first layer would have 256 parameters since the input layer, layer_in, was instantiated with shape=(256)

Topic keras tensorflow

Category Data Science


As you are using dense layers, Keras determine the params in models as following calculations :

# of params = # of outputs * (# of inputs + 1)

Hence, the calculations is as follows :

Number of params layer 1 = 2 * (256+1) = 2*257 = 514 Params

Number of params layer 2 = 3 * (2+1) = 3*3 = 9 Params

Number of params layer 3 = 4 * (3+1) = 4*4 = 16 Params

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.