Questions tagged [hyperparameter]
Hyperparameters of a model are the kind of parameters that cannot be directly learned during training but are set beforehand. Hyperparameters can define, for example, the complexity of the model or its capacity to learn.
152 questions
1vote
0answers
28views
Choosing the number of features via cross-validation
I have an algorithm that trains a binary predictive model for a specified number of features from the dataset (features are all of the same type, but not all important.) Thus, the number of features ...
- 119
1vote
2answers
125views
xgboost - colsample_bylevel & colsample_bynode
I don't really understand the meaning of these xgboost parameters or how they differ: If I specify exactly one of these parameters (not both at the same time), ...
- 11
1vote
0answers
44views
Hyper parameter tuning LSTM network on time series data
I am trying to train LSTM model (containing four LSTM layers (500 units each) and three droupouts and a fully connected output layer to do regression) on timeseries data. To start with, I tried to ...
- 299
1vote
0answers
19views
Why I am requiring tiny learning rate to overfit the model?
I am trying to train LSTM model on a timeseries data with 1.6 million records. I have taken window size of 200. Initially I tried to overfit the model (train data = test data) on tiny dataset (few ...
- 299
2votes
1answer
128views
The best algorithm(s) for finding the best hyperparameters (special case)
I would like to ask for help with the following. Given the following dataset, which I have split into train and test sets: ...
0votes
1answer
167views
Optimal combination of variables to minimise output
To be honest I'm not 100% sure how much this is purely a coding issue or a data science issue, but I'll take my chances. I've developed a matrix which is a mixture of various hyperparameters, the ...
0votes
0answers
44views
Hyperparameter Widget
Could you please inform me if there exists a widget designed for the purpose of conducting hyperparameter optimization? I attempted to locate such a tool, but regrettably, I was unable to find one.
0votes
1answer
69views
How does deeper or shallower trees (higher or lower max_depth) affect xgboost model?
I am doing an xgboost model for landslides assessment and I am using max_depth as one of my hyperparameters, but I don't understand how does it affect model ...
- 3
1vote
1answer
30views
Hyperparameter tuning
Jane trains three different classifiers: Logistic Regression, Decision Tree, and Support Vector Machines on the training set. Each classifier has one hyper-parameter (regularisation parameter, depth-...
- 11
1vote
1answer
587views
Ordering of Train/Val/Test set use in hyperparameter tuning
The way I read almost lots of ML advice on these datasets sounds like "You train a model that's randomly chosen hyperparameters first on the training set, then you ignore this bit of the work, ...
- 121
0votes
1answer
173views
Optimizing decision tree
I have a question regarding the technique/technology which could be applied for the issue: Suppose I have a rule-based tree or decision tree which predicts a variable Y based on variables A,B,C. This ...
0votes
1answer
134views
Visualizing correlation between hyper-parameters and metrics for Neural Network
I am working with a neural network and I want to investigate how different settings affect the loss and standard deviation of the network. I can change various parameters such as the loss function, ...
0votes
1answer
742views
Is hyperparameter tuning done on training or validation data set?
Is hyperparameter tuning done on training or validation data set? The post here gives mixed opinion as of whether the training set should be used for hyperparameter tuning. And I would like to know ...
- 421
8votes
1answer
3kviews
Do transformers (e.g. BERT) have an unlimited input size?
There are various sources on the internet that claim that BERT has a fixed input size of 512 tokens (e.g. this, this, this, this ...). This magical number also appears in the BERT paper (Devlin et al. ...
- 253
1vote
1answer
412views
Is hyperparameter tuning with different samples of data on each run a bad idea?
I have 2k time series and want to optimize the hyperparameters of my prophet model. It takes 1 hour to train and evaluate on every time series for each hyperparam combination. So, I want to run it on ...
- 308
