Stack Overflow
Stack Overflow

Questions tagged [hyperparameters]

Machine Learning engines use hyper-parameters for their learning phase behaviour. Different values thus modify the learner's model ability to generalise and avoid overfitting and/or bias towards a given DataSET training-phase observed values.

856 questions
45
votes
4 answers

Hyperparameter optimization for Pytorch model

What is the best way to perform hyperparameter optimization for a Pytorch model? Implement e.g. Random Search myself? Use Skicit Learn? Or is there anything else I am not aware of?
Alex
  • 778
  • 1
  • 8
  • 17
43
votes
2 answers

How to insert Keras model into scikit-learn pipeline?

I'm using a scikit-learn custom pipeline (sklearn.pipeline.Pipeline) in conjunction with RandomizedSearchCV for hyper-parameter optimization. This works great. Now I would like to insert a keras model as a first step into the pipeline. The…
machinery
  • 5,972
  • 12
  • 67
  • 118
27
votes
2 answers

How to pass elegantly Sklearn's GridseachCV's best parameters to another model?

I have found a set of best hyperparameters for my KNN estimator with Grid Search CV: >>> knn_gridsearch_model.best_params_ {'algorithm': 'auto', 'metric': 'manhattan', 'n_neighbors': 3} So far, so good. I want to train my final estimator with these…
Hendrik
  • 1,158
  • 4
  • 15
  • 30
23
votes
5 answers

In what order should we tune hyperparameters in Neural Networks?

I have a quite simple ANN using Tensorflow and AdamOptimizer for a regression problem and I am now at the point to tune all the hyperparameters. For now, I saw many different hyperparameters that I have to tune : Learning rate : initial learning…
Paul Rolin
  • 403
  • 2
  • 6
  • 15
23
votes
2 answers

What is a good range of values for the svm.SVC() hyperparameters to be explored via GridSearchCV()?

I am running into the problem that the hyperparameters of my svm.SVC() are too wide such that the GridSearchCV() never gets completed! One idea is to use RandomizedSearchCV() instead. But again, my dataset is relative big such that 500 iterations…
user706838
  • 5,132
  • 14
  • 54
  • 78
20
votes
2 answers

Hyperparameter Tuning of Tensorflow Model

I've used Scikit-learn's GridSearchCV before to optimize the hyperparameters of my models, but just wondering if a similar tool exists to optimize hyperparameters for Tensorflow (for instance number of epochs, learning rate, sliding window size…
mamafoku
  • 1,049
  • 2
  • 14
  • 28
14
votes
2 answers

Is there a way to perform grid search hyper-parameter optimization on One-Class SVM

Is there a way to use GridSearchCV or any other built-in sklearn function to find the best hyper-parameters for OneClassSVM classifier? What I currently do, is perform the search myself using train/test split like this: Gamma and nu values are…
Yustx
  • 326
  • 1
  • 4
  • 12
12
votes
5 answers

Hyper-parameter tuning using pure ranger package in R

Love the speed of the ranger package for random forest model creation, but can't see how to tune mtry or number of trees. I realize I can do this via caret's train() syntax, but I prefer the speed increase that comes from using pure ranger. Here's…
Levi Thatcher
  • 315
  • 1
  • 2
  • 12
11
votes
1 answer

Compare ways to tune hyperparameters in scikit-learn

This post is about the differences between LogisticRegressionCV, GridSearchCV and cross_val_score. Consider the following setup: import numpy as np from sklearn.datasets import load_digits from sklearn.linear_model import LogisticRegression,…
farmer
  • 285
  • 1
  • 13
10
votes
2 answers

incompatibility issue between scikit-learn 0.24.1 and scikit-optimize 0.8.1

I have scikit-learn 0.24.1 and scikit-optimize 0.8.1, when I tried to use the BayesSearchCV function it gave me this error: TypeError: __init__() got an unexpected keyword argument 'iid' when I searched it turned out that the 'iid' is deprecated in…
user12995140
  • 101
  • 4
10
votes
4 answers

How to save the best hyperopt optimized keras models and its weights?

I optimized my keras model using hyperopt. Now how do we save the best optimized keras model and its weights to disk. My code: from hyperopt import fmin, tpe, hp, STATUS_OK, Trials from sklearn.metrics import roc_auc_score import sys X = [] y =…
Eka
  • 14,170
  • 38
  • 128
  • 212
9
votes
0 answers

Training multiple neural networks asynchronously in parallel

The problem I am currently working on a project that I sadly can't share with you. The project is about hyper-parameter optimization for neural networks, and it requires that I train multiple neural network models (more than I can store on my GPU)…
Thomas Angeland
  • 441
  • 5
  • 10
9
votes
2 answers

How to do GridSearchCV for F1-score in classification problem with scikit-learn?

I'm working on a multi classification problem with a neural network in scikit-learn and I'm trying to figure out how I can optimize my hyperparameters (amount of layers, perceptrons, other things eventually). I found out that GridSearchCV is the…
Jonas
  • 103
  • 1
  • 1
  • 6
9
votes
1 answer

GridSearchCV - save result each iteration

I'm using GridSearchCV, and after each iteration I want to save the clf.cv_results_ property to a file, (just in case that the process will crash in the middle). I tried looking for a solution but I just couldn't figure it out. Any help will be…
oshribr
  • 658
  • 7
  • 16
9
votes
1 answer

How to choose the window size of CNN in deep learning?

In Convolutional Neural Network (CNN), a filter is select for weights sharing. For example, in the following pictures, a 3x3 window with the stride (distance between adjacent neurons) 1 is chosen. So my question is: How to choose the window size?…
Fitz999
  • 103
  • 1
  • 2
  • 6
1
2 3
…
57 58