Machine learning

Question

What is the different between traing set and test set ? can give a real time example?

Mehreen Tahir · Answer

Usually you need two types of datasets when performing supervised learning: The dataset that has the input data with correct/expeced output which would be used to train your model. This dataset is known as training set. The data you are going to apply your model to. This is the data in which you are interested for the output of your model and want the expected output. So there're going to be two phases while performing machine learning. Training Phase  in which you will present your data from training set and train your model by pairing the input with expected output. For example we are interested in predicting the type of diabetes a patient has. Our training data will contain all the information related to patient i.e. height weight BMI age blood sugar level etc. along with the expected out either type A or B. Now comes the Test Phase  in which you'll apply your freshly-developed model to the real-world data and get the results. In this phase we will feed our model with the test set that will contain the similar patiet information and our model will predict either patient is suffering from type A diabetes or type B. This phase will help you estimate how well your model has been trained. Most commonly the partition ratio of training and test set is 70/30 .

Rohit Gupta · Answer

A dataset can be divided into 3 parts: 1. Test Set It is the part of the dataset which is used for Black box testing i.e. here the motive is to test the model on data that the model has never seen before. Here we use metrics like the Confusion Matrix accuracy score and F1 score. 2. Training Set It is the part of the dataset which is used to train the model and make the hypothesises more accurate 3. Validation set In machine learning when we generate a model we need to validate the model before sending it for the final testing. So to validate the working of the model we use the validation set For Example: Let us take the MNIST cloth dataset Here we would like to generate a model that is able to predict the type and genre of the cloth. So to do so we would divide the whole fo the dataset into a ratio of 7:3 where 70% of the data is the training+Validation Data and 30% is the Test data In Python to segregate the training and test data we use the sklear.model_selection.train_test_split() X_train X_test y_train y_test = train_test_split( ... X y test_size= 0.3  random_state= 0 ) The above code demostrates how to divide the dataset in the 7:3 ratio

Md Sarfaraj · Answer

Go to below link https://stats.stackexchange.com/questions/19048/what-is-the-difference-between-test-set-and-validation-set