Merge pull request #1 from piaic-official/master #16
Conversation
updated forked repo for assignments
|
`LOAD LIBRARIEs import pandas as pd load data car_df = pd.read_csv('CarPrice_Assignment.csv') chek missing data Out[29]: Out[30]: convert Non-numerical values into numerical values from sklearn.preprocessing import LabelEncoder In [32]: car_df['fueltype'] = labelencoder.fit_transform(car_df['fueltype']) In [33]: car_df.info() <class 'pandas.core.frame.DataFrame'> Column Non-Null Count Dtype0 car_ID 205 non-null int64 car_df.head() Out[34]: car_ID Correlation Matrix plt.figure(figsize=(20,20)) Out[35]: Select Feautures with highest +ve and -ve Correlation car_df.corr()['price'].sort_values(ascending=False) Out[36]: car = car_df[['drivewheel','enginelocation','wheelbase','carlength','carwidth','curbweight','cylindernumber','enginesize', In [38]: car.head() Out[38]: drivewheel Split Data into Train, Test and Validation¶ In [40]: x = (car.loc[:, car.columns != 'price']) In [41]: Split to 50% Train and 50% Testx_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.50, random_state=42) Split 50% Test into further 30% Test and 20% Validationx_test, x_val, y_test, y_val = train_test_split(x_test, y_test, test_size=0.40, random_state=42) In [42]: print(x_train.shape) (102, 14) print(y_train.shape) (102, 1) Use Min Max Scaler to Scale all Features In [45]: min_max_scaler = MinMaxScaler() In [46]: x_train_s = min_max_scaler.fit_transform(x_train) In [72]: print(x_train_s.shape) (102, 14) Create Model Build Model and Architecture of Deep Neural Network Add layers of Neural Network = 4 , input_dim = number oof featuresunits = 10,8,6 is number of neuronsmodel.add(Dense(units = 10, activation='relu', input_dim=14)) In [50]: Loss Function measures how well the model did on training and then triesto improve on it using optimizermodel.compile(optimizer = 'adam', In [51]: hist = model.fit( Epoch 1/100 4/4 [==============================] - 0s 4ms/step - loss: 0.0064 - mae: 0.0576 - val_loss: 0.0120 - val_mae: 0.0929 Check Model's Loss and Mean Absolute Error against Validation DataIn [52]: hist.history.keys() Out[52]: Visuialize the Training and Valudation Loss to see if Model is overfittingplt.plot(hist.history["loss"]) Out[53]: In [54]: Visuialize the Training and Valudation Loss to see if Model is overfittingplt.plot(hist.history["mae"]) Out[54]: Retune Model based on number of Epocs, to get minimum LossIn [55]: Build Model and Architecture of Deep Neural Network Add layers of Neural Network = 4 , input_dim = number oof featuresunits = 10,8,6 is number of neuronsmodel.add(Dense(units = 10, activation='relu', input_dim=14)) In [56]: Loss Function measures how well the model did on training and then triesto improve on it using optimizermodel.compile(optimizer = 'adam', In [57]: hist = model.fit( Epoch 1/37 Visuialize the Training and Valudation Loss to see if Model is overfittingplt.plot(hist.history["loss"]) Out[58]: Get the Model's Predicted Values based on XTest dataIn [59]: predictions = model.predict(x_test_s) In [60]: 61 predictions Out[61]: Get the Model's Predicted Values based on XTest dataIn [62]: y_test['price'].values Out[62]: y_train['Type'] = 'Train' C:\Users\tahir\anaconda3\envs\tensor\lib\site-packages\ipykernel_launcher.py:1: SettingWithCopyWarning: See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy y_test['Type'] = 'Test' In [65]: pred = pd.DataFrame(data = predictions,index = y_test.index, columns=['Predcited Price']) In [67]: plot = pd.DataFrame() In [68]: plot Out[68]: price In [70]: |
|
Task:: import pandas as pd In [26]: In [27]: credit_df.isnull().sum() Out[27]: credit_df.info() <class 'pandas.core.frame.DataFrame'> Column Non-Null Count Dtype0 Time 284807 non-null float64 Correlation plt.figure(figsize=(20,20)) Out[29]: In [30]: credit_df.corr()['Class'].sort_values(ascending=False) Out[30]: Select Features with High Corelation In [32]: credit.head() Out[32]: V11 Split Data In [34]: In [35]: Split to 50% Train and 50% Testx_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.50, random_state=42) Split 50% Test into further 30% Test and 20% Validationx_test, x_val, y_test, y_val = train_test_split(x_test, y_test, test_size=0.40, random_state=42) In [36]: print(x_train.shape) (142403, 9) (142403, 1) Scale Data In [39]: Use MinMax Scaller to scale all featuresmin_max_scaler = MinMaxScaler() In [40]: In [41]: print(x_train_s.shape) (142403, 9) Build Model Build Model and Architecture of Deep Neural Network Add layers of Neural Network = 4 , input_dim = number oof featuresunits = 10,8,6 is number of neuronsmodel.add(Dense(units = 10, activation='relu', input_dim=9)) In [43]: Loss Dunction measures how well the model did on training and then triesto improve on it using optimizermodel.compile(optimizer = 'sgd', In [44]: hist = model.fit( Epoch 1/100 4451/4451 [==============================] - 3s 764us/step - loss: 0.0266 - accuracy: 0.9983 - val_loss: 0.0265 - val_accuracy: 0.9983 Evaluate Accuracy of Data based on Test Data Accuracy of 99.8% 2671/2671 [==============================] - 1s 505us/step - loss: 0.0267 - accuracy: 0.9983 Check Model's Loss against Validation Data Out[46]: Visuialize the Training and Valudation Loss to see if Model is overfittingplt.plot(hist.history["loss"]) Out[47]: Do Predictions on Test Data predictions = model.predict(x_test_s) In [49]: print(len(predictions)) 85442 |
|
[1]: import pandas as pd Test to load and Resize fileIn [2]: p = 64 In [22]: img = load_img('E:/PIAIC AI Course March 2020/Q2/deeplearning assigment/flowers/daisy/5547758_eea9edfd54_n.jpg') Out[22]: Load Pictures and Convert to array and make a Single X(Pixels) and Y(Lables) dataset classification = ['daisy', 'dandelion', 'rose', 'sunflower', 'tulip'] In [32]: path = 'E:/PIAIC AI Course March 2020/Q2/deeplearning assigment/flowers/' In [33]: length_daisy = len(os.listdir(path+'daisy')) In [34]: print(length_daisy) 769 length_x = length_daisy + length_dandelion + length_rose + length_sunflower + length_tulip Out[35]: x = np.zeros((length_x,p,p,3)) Out[36]: Out[38]: Load daisy pictures z = 0 #Counter for image file loading d Out[39]: Load dandelion pictures d Out[40]: Load rose pictures z = 0 #Counter for image file loading d Out[42]: Load sunflower pictures z = 0 #Counter for image file loading d Out[44]: Load tulip pictures d Out[45]: print(x.shape) (1098, 64, 64, 3) Split Data into Train, Test and Validation from sklearn.model_selection import train_test_split In [48]: Split to 60% Train and 40% Testx_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.40, random_state=42) In [50]: print(x_train.shape) (658, 64, 64, 3) One Hot encoding of Labels Convert the labels into a set of 5 numbers to input into NNEvery set of rows will have 5 values, all 0 except the one with actual label index will be 1y_train_one_hot = to_categorical(y_train) In [52]: Out[52]: (658, 5) Normalize pixel values between 0 and 1 x_train = x_train /255 Create Model model = Sequential() First Layer (32 5x5 relu convoluted feature maps)model.add(Conv2D(32, (5,5) , activation = 'relu', input_shape=(p,p,3))) 2nd Layer as pooling Layer (creating a pooling layer with 2x2 pixel filter)model.add(MaxPooling2D(pool_size = (2,2))) 3rd Layer (2nd Colvolution Layer)model.add(Conv2D(32, (5,5) , activation = 'relu')) 4th Layer (2nd Pooling Layer)model.add(MaxPooling2D(pool_size = (2,2))) 5th Layer (Flattening Layer)model.add(Flatten()) 6th Layer with 1000 neurons to feed from previous layersmodel.add(Dense(1000, activation = 'relu')) 7th Layer (Dropout layer with 50%)model.add(Dropout(0.5)) 8th Layer with 500 neurons to feed from previous layersmodel.add(Dense(500, activation = 'relu')) 9th Layer (Dropout layer with 50%)model.add(Dropout(0.5)) 10th Layer with 250 neurons to feed from previous layersmodel.add(Dense(250, activation = 'relu')) 11th Layer with 5 neurons (as there are 10 different Classifications)model.add(Dense(5, activation = 'softmax')) In [56]: Compile the Model In [57]: Train the Model with Validation Data of 20% Epoch 1/15 Check Model's Loss and Accuracy against Validation Data hist.history.keys() Out[62]: Visuialize the Training and Valudation Loss to see if Model is overfittingplt.plot(hist.history["loss"]) Out[63]: In [67]: Visuialize the Training and Valudation Loss to see if Model is overfittingplt.plot(hist.history["accuracy"]) Out[67]: Test the Model with random images online Test Image 1 img = load_img('tulip.jpg') Out[69]: In [70]: In [71]: img_array.shape Out[71]: Out[72]: predictions = model.predict(img_array) Out[73]: Sort predictions from least to Greatestlist_index = [0,1,2,3,4] Show the sorted labels in orderprint(list_index) [0, 1, 2, 3, 4] daisy : 100.0 % Test Image 2 Out[76]: In [77]: In [78]: img_array.shape Out[78]: img_array = np.expand_dims(img_array, axis=0) Out[79]: predictions = model.predict(img_array) Out[80]: Sort predictions from least to Greatestlist_index = [0,1,2,3,4] Show the sorted labels in orderprint(list_index) [0, 1, 2, 3, 4] daisy : 100.0 % Test Image 3 img = load_img('tulip2.jpg') Out[83]: In [84]: img_array = img_to_array(img) In [85]: Out[85]: img_array = np.expand_dims(img_array, axis=0) Out[86]: predictions = model.predict(img_array) Out[87]: Sort predictions from least to Greatestlist_index = [0,1,2,3,4] Show the sorted labels in orderprint(list_index) [0, 1, 2, 3, 4] for i in range(5): daisy : 100.0 % |
|
In [1]: import pandas as pd In [2]: ion_df = pd.read_csv('ionosphere_data.csv') In [3]: ion_df.isnull().sum() Out[3]: <class 'pandas.core.frame.DataFrame'> Column Non-Null Count Dtype0 feature1 351 non-null int64 ion_df['label'] = ion_df['label'].map({'g':1,'b':0}) In [6]: ion_df.info() <class 'pandas.core.frame.DataFrame'> Column Non-Null Count Dtype0 feature1 351 non-null int64 plt.figure(figsize=(20,20)) Out[7]: In [8]: Out[8]: Select Features with high Correlation ion = ion_df[['feature3','feature5','feature1','feature7','feature9', In [10]: ion.head() Out[10]: feature3 Split Data from sklearn.model_selection import train_test_split In [12]: x = (ion.loc[:, ion.columns != 'label']) In [13]: Split to 50% Train and 50% Testx_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.40, random_state=42) In [14]: print(x_train.shape) (210, 11) Scale Data Use MinMax Scaller to scale all featuresmin_max_scaler = MinMaxScaler() In [16]: In [17]: (210, 11) Build Model Build Model and Architecture of Deep Neural Network Add layers of Neural Network = 4 , input_dim = number oof featuresunits = 10,8,6 is number of neuronsmodel.add(Dense(units = 16, activation='relu', input_dim=11)) In [19]: Loss Dunction measures how well the model did on training and then triesto improve on it using optimizermodel.compile(optimizer = 'sgd', In [20]: hist = model.fit( Epoch 1/100 Check Accuracy of Model based on Test Data Model Accuracy is 91% model.evaluate(x_test, y_test)[1] 5/5 [==============================] - 0s 0s/step - loss: 0.3353 - accuracy: 0.9149 Predict Outcomes predictions = model.predict(x_test) In [23]: print(len(prediction2)) 141 |
updated forked repo for assignments