Optimum weights for weighted average of 3 prediction models

I have 3 sklearn models which I use to predict a probability score for a binary classification problem. I want to create a weighted average score of all the predictions made by these models. I am stuck at how to find the optimum weights.

I have tried to create a weighted average method that would help me:

def weighted_average(prob: dict, weights: dict = base_weights):
    '''Weighted average of all probabilities
    Prob Dict structure: {
    'mfcc': probability of spoof from MFCC Model,
    'lfcc': probability of spoof from LFCC Model,
    'gfcc': probability of spoof from GFCC Model
    }
    
    returns weighted average of probability
    '''
    num = prob['mfcc']*weights['mfcc'] + prob['lfcc']*weights['lfcc'] + prob['gfcc']*weights['gfcc']
    denom = weights['mfcc'] + weights['lfcc'] + weights['gfcc']
    return num / denom

In order to find the optimum weights (I'm optimizing accuracy_score), I have tried the following:

  1. Exhaustive search over a range.
  2. Tried fitting a Logistic regression model with X as the accuracy scores and Y as 0|1.

In the end, the goal is to get an accuracy score and not 0 or 1.

Topic optimization machine-learning

Category Data Science


Sklearn has built-in ensembling classes for example voting classifier for this job.

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.