Udemy - 딥러닝의 모든 것(ANN 구축하기)

---

인구 통계학적 세분화 모델

해당 모델은 고객중심의 산업에서는 부가 가치가 굉장히 높고 다양한 시나리오에서 사용될 수 있도록 할 수 있는 모델이다.

우선 이번에 구현해볼 ANN은 은행의 고객 이탈률이 증가하여 그 원인을 조사하여 추후의 새로운 고객에 대해서도 이탈할 고객을 예측하는 모델이다.

우선 자료를 보면 다음과 같다.

Customer Id, Surname, CreditScore, Geography, Gender, Age, Tenure, Balance, NumOfProducts, HasCrCard, IsActiveMember, EstimatedSalary와 같은 변수들은 6개월 전에 조사된 자료이고 Exited 여부는 현재 이탈한 고객인지 알려주는 종속변수이다. 위 자료는 만명의 고객 샘플로 구성되어있다. 실제(?)로는 수백만명이 이용하는 가상은행이지만 만명정도의 샘플을 이용하는 것이다.

---

ANN 구현

google colab을 통하여 구현하는 과정을 거칠 것이다.

전체적인 과정을 살펴보면 크게 4가지 파트를 통해 구현할 수 있다.

파트1은 데이터 전처리, 파트2는 ANN 구축, 파트3는 ANN 훈련, 파트4 예측 및 모델 평가이다.

우선 libraries importing 과정이 필요하였다.

import numpy as np
import pandas as pd
import tensorflow as tf

---

Part 1 - Data Preprocessing

Importing the dataset

dataset = pd.read_csv('Churn_Modelling.csv')	#데이터 파일을 dataset으로 받아옴
X = dataset.iloc[:, 3:-1].values		#X축에서 예측하는데 이용되는 열들을 범위를 지정해줌
y = dataset.iloc[:, -1].values			#y축에서 첫줄은 변수명이므로 제외

print(X)

[[619 'France' 'Female' ... 1 1 101348.88]
 [608 'Spain' 'Female' ... 0 1 112542.58]
 [502 'France' 'Female' ... 1 0 113931.57]
 ...
 [709 'France' 'Female' ... 0 1 42085.58]
 [772 'Germany' 'Male' ... 1 0 92888.52]
 [792 'France' 'Female' ... 1 0 38190.78]]
 
 print(y)
 
 [1 0 1 ... 1 1 0]

Encoding categorical data

Label Encoding the "Gender" column

성별을 0또는1로 encoding하는 과정이다.

from sklearn.preprocessing import LabelEncoder
le = LabelEncoder()
X[:,2] = le.fit_transform(X[:,2])

print(X)

[[619 'France' 0 ... 1 1 101348.88]
 [608 'Spain' 0 ... 0 1 112542.58]
 [502 'France' 0 ... 1 0 113931.57]
 ...
 [709 'France' 0 ... 0 1 42085.58]
 [772 'Germany' 1 ... 1 0 92888.52]
 [792 'France' 0 ... 1 0 38190.78]]

One Hot Encoding the "Geography" column

France, Spain, Germany 세가지 지역을 100,010,001로 encoding하는 과정이다.

from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import OneHotEncoder
ct = ColumnTransformer(transformers=[('encoder', OneHotEncoder(), [1])], remainder='passthrough')
X = np.array(ct.fit_transform(X))

print(X)

[[1.0 0.0 0.0 ... 1 1 101348.88]
 [0.0 0.0 1.0 ... 0 1 112542.58]
 [1.0 0.0 0.0 ... 1 0 113931.57]
 ...
 [1.0 0.0 0.0 ... 0 1 42085.58]
 [0.0 1.0 0.0 ... 1 0 92888.52]
 [1.0 0.0 0.0 ... 1 0 38190.78]]

Splitting the dataset into the Training set and Test set

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 0)

Feature Scaling

from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
X_train = sc.fit_transform(X_train)
X_test = sc.transform(X_test)

---

Part 2 - Building the ANN

Initializing the ANN

ann = tf.keras.models.Sequential()

Adding the input layer and the first hidden layer

ann.add(tf.keras.layers.Dense(units=6, activation='relu'))

add()를 이용하고 Dense()의 주요인자 두개를 입력해준다. 뉴런의 수를 6으로 설정하고 활성화함수를 hidden layer에서 주로 이용되는 Rectified Linear Unit 함수를 뜻하는 relu로 설정한다.

Adding the second hidden layer

ann.add(tf.keras.layers.Dense(units=6, activation='relu'))

첫번째 hidden layer를 생성하는 과정과 같다.

Adding the output layer

ann.add(tf.keras.layers.Dense(units=1, activation='sigmoid'))

output layer는 이탈 여부가 OX로 나뉘는 이진 결과값이므로 0,1만 필요해서 units은 1로 지정하고 활성화함수는 output layer이기 때문에 sigmoid function을 이용한다.

---

Part 3 - Training the ANN

Compiling the ANN

ann.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])

optimizer는 Gradient Descent Algorithm중 하나인 Adam을 이용하고 loss는 출력값이 이진수이니 binary_crossentropy를 이용한다. 출력값이 세개이상일 경우 categorical_crossentropy를 이용한다.

 

Training the ANN on the Training set

ann.fit(X_train, y_train, batch_size = 32, epochs = 100)

batch_size는 일반적으로 32로 설정하고 epochs 수를 유동적으로 설정하면되는데 100으로 잡아보도록하였다.

epochs는 반복 훈련 수라고 보면된다.

Epoch 1/100
250/250 [==============================] - 1s 2ms/step - loss: 0.5384 - accuracy: 0.7935
Epoch 2/100
250/250 [==============================] - 0s 2ms/step - loss: 0.4832 - accuracy: 0.7960
Epoch 3/100
250/250 [==============================] - 0s 2ms/step - loss: 0.4603 - accuracy: 0.7960
Epoch 4/100
250/250 [==============================] - 0s 2ms/step - loss: 0.4462 - accuracy: 0.7972
Epoch 5/100
250/250 [==============================] - 0s 1ms/step - loss: 0.4370 - accuracy: 0.8033
Epoch 6/100
250/250 [==============================] - 0s 2ms/step - loss: 0.4298 - accuracy: 0.8104
Epoch 7/100
250/250 [==============================] - 0s 1ms/step - loss: 0.4261 - accuracy: 0.8123
Epoch 8/100
250/250 [==============================] - 0s 2ms/step - loss: 0.4235 - accuracy: 0.8158
Epoch 9/100
250/250 [==============================] - 0s 1ms/step - loss: 0.4212 - accuracy: 0.8156
Epoch 10/100
250/250 [==============================] - 0s 1ms/step - loss: 0.4191 - accuracy: 0.8206
Epoch 11/100
250/250 [==============================] - 0s 1ms/step - loss: 0.4173 - accuracy: 0.8227
Epoch 12/100
250/250 [==============================] - 0s 1ms/step - loss: 0.4155 - accuracy: 0.8231
Epoch 13/100
250/250 [==============================] - 0s 1ms/step - loss: 0.4139 - accuracy: 0.8229
Epoch 14/100
250/250 [==============================] - 0s 1ms/step - loss: 0.4129 - accuracy: 0.8246
Epoch 15/100
250/250 [==============================] - 0s 1ms/step - loss: 0.4112 - accuracy: 0.8256
Epoch 16/100
250/250 [==============================] - 0s 1ms/step - loss: 0.4097 - accuracy: 0.8266
Epoch 17/100
250/250 [==============================] - 0s 2ms/step - loss: 0.4084 - accuracy: 0.8275
Epoch 18/100
250/250 [==============================] - 0s 1ms/step - loss: 0.4067 - accuracy: 0.8289
Epoch 19/100
250/250 [==============================] - 0s 2ms/step - loss: 0.4053 - accuracy: 0.8300
Epoch 20/100
250/250 [==============================] - 0s 1ms/step - loss: 0.4037 - accuracy: 0.8310
Epoch 21/100
250/250 [==============================] - 0s 1ms/step - loss: 0.4023 - accuracy: 0.8322
Epoch 22/100
250/250 [==============================] - 0s 1ms/step - loss: 0.4008 - accuracy: 0.8329
Epoch 23/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3995 - accuracy: 0.8322
Epoch 24/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3983 - accuracy: 0.8331
Epoch 25/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3970 - accuracy: 0.8341
Epoch 26/100
250/250 [==============================] - 0s 1ms/step - loss: 0.3951 - accuracy: 0.8345
Epoch 27/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3938 - accuracy: 0.8354
Epoch 28/100
250/250 [==============================] - 0s 1ms/step - loss: 0.3928 - accuracy: 0.8356
Epoch 29/100
250/250 [==============================] - 0s 1ms/step - loss: 0.3915 - accuracy: 0.8374
Epoch 30/100
250/250 [==============================] - 0s 1ms/step - loss: 0.3904 - accuracy: 0.8380
Epoch 31/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3890 - accuracy: 0.8391
Epoch 32/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3891 - accuracy: 0.8404
Epoch 33/100
250/250 [==============================] - 0s 1ms/step - loss: 0.3883 - accuracy: 0.8381
Epoch 34/100
250/250 [==============================] - 0s 1ms/step - loss: 0.3872 - accuracy: 0.8424
Epoch 35/100
250/250 [==============================] - 0s 1ms/step - loss: 0.3865 - accuracy: 0.8407
Epoch 36/100
250/250 [==============================] - 0s 1ms/step - loss: 0.3860 - accuracy: 0.8410
Epoch 37/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3844 - accuracy: 0.8424
Epoch 38/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3833 - accuracy: 0.8424
Epoch 39/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3823 - accuracy: 0.8435
Epoch 40/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3795 - accuracy: 0.8438
Epoch 41/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3749 - accuracy: 0.8457
Epoch 42/100
250/250 [==============================] - 0s 1ms/step - loss: 0.3707 - accuracy: 0.8465
Epoch 43/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3679 - accuracy: 0.8464
Epoch 44/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3649 - accuracy: 0.8485
Epoch 45/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3632 - accuracy: 0.8491
Epoch 46/100
250/250 [==============================] - 0s 1ms/step - loss: 0.3612 - accuracy: 0.8509
Epoch 47/100
250/250 [==============================] - 0s 1ms/step - loss: 0.3592 - accuracy: 0.8504
Epoch 48/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3579 - accuracy: 0.8503
Epoch 49/100
250/250 [==============================] - 0s 1ms/step - loss: 0.3560 - accuracy: 0.8518
Epoch 50/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3540 - accuracy: 0.8531
Epoch 51/100
250/250 [==============================] - 0s 1ms/step - loss: 0.3527 - accuracy: 0.8545
Epoch 52/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3517 - accuracy: 0.8537
Epoch 53/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3507 - accuracy: 0.8536
Epoch 54/100
250/250 [==============================] - 0s 1ms/step - loss: 0.3497 - accuracy: 0.8547
Epoch 55/100
250/250 [==============================] - 0s 1ms/step - loss: 0.3483 - accuracy: 0.8553
Epoch 56/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3473 - accuracy: 0.8555
Epoch 57/100
250/250 [==============================] - 0s 1ms/step - loss: 0.3473 - accuracy: 0.8558
Epoch 58/100
250/250 [==============================] - 0s 1ms/step - loss: 0.3460 - accuracy: 0.8553
Epoch 59/100
250/250 [==============================] - 0s 1ms/step - loss: 0.3454 - accuracy: 0.8562
Epoch 60/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3445 - accuracy: 0.8562
Epoch 61/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3438 - accuracy: 0.8574
Epoch 62/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3436 - accuracy: 0.8569
Epoch 63/100
250/250 [==============================] - 1s 3ms/step - loss: 0.3431 - accuracy: 0.8570
Epoch 64/100
250/250 [==============================] - 1s 3ms/step - loss: 0.3427 - accuracy: 0.8569
Epoch 65/100
250/250 [==============================] - 1s 3ms/step - loss: 0.3422 - accuracy: 0.8586
Epoch 66/100
250/250 [==============================] - 1s 3ms/step - loss: 0.3418 - accuracy: 0.8584
Epoch 67/100
250/250 [==============================] - 1s 3ms/step - loss: 0.3417 - accuracy: 0.8568
Epoch 68/100
250/250 [==============================] - 1s 2ms/step - loss: 0.3403 - accuracy: 0.8575
Epoch 69/100
250/250 [==============================] - 1s 2ms/step - loss: 0.3407 - accuracy: 0.8593
Epoch 70/100
250/250 [==============================] - 1s 2ms/step - loss: 0.3402 - accuracy: 0.8572
Epoch 71/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3400 - accuracy: 0.8585
Epoch 72/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3396 - accuracy: 0.8590
Epoch 73/100
250/250 [==============================] - 0s 1ms/step - loss: 0.3398 - accuracy: 0.8571
Epoch 74/100
250/250 [==============================] - 0s 1ms/step - loss: 0.3386 - accuracy: 0.8587
Epoch 75/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3389 - accuracy: 0.8574
Epoch 76/100
250/250 [==============================] - 0s 1ms/step - loss: 0.3391 - accuracy: 0.8605
Epoch 77/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3384 - accuracy: 0.8593
Epoch 78/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3385 - accuracy: 0.8593
Epoch 79/100
250/250 [==============================] - 0s 1ms/step - loss: 0.3382 - accuracy: 0.8601
Epoch 80/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3382 - accuracy: 0.8600
Epoch 81/100
250/250 [==============================] - 0s 1ms/step - loss: 0.3377 - accuracy: 0.8591
Epoch 82/100
250/250 [==============================] - 0s 1ms/step - loss: 0.3371 - accuracy: 0.8610
Epoch 83/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3372 - accuracy: 0.8587
Epoch 84/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3368 - accuracy: 0.8609
Epoch 85/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3376 - accuracy: 0.8595
Epoch 86/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3371 - accuracy: 0.8608
Epoch 87/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3371 - accuracy: 0.8606
Epoch 88/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3376 - accuracy: 0.8619
Epoch 89/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3363 - accuracy: 0.8620
Epoch 90/100
250/250 [==============================] - 0s 1ms/step - loss: 0.3361 - accuracy: 0.8612
Epoch 91/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3356 - accuracy: 0.8624
Epoch 92/100
250/250 [==============================] - 0s 1ms/step - loss: 0.3359 - accuracy: 0.8633
Epoch 93/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3358 - accuracy: 0.8614
Epoch 94/100
250/250 [==============================] - 0s 1ms/step - loss: 0.3358 - accuracy: 0.8611
Epoch 95/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3356 - accuracy: 0.8612
Epoch 96/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3352 - accuracy: 0.8618
Epoch 97/100
250/250 [==============================] - 0s 1ms/step - loss: 0.3351 - accuracy: 0.8601
Epoch 98/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3355 - accuracy: 0.8611
Epoch 99/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3363 - accuracy: 0.8615
Epoch 100/100
250/250 [==============================] - 0s 2ms/step - loss: 0.3355 - accuracy: 0.8611
<keras.callbacks.History at 0x7fb2e69af340>

훈련결과가 길더라도 전부 넣은것은 accuracy가 0.86으로 점진적으로 수렴하는것을 보기위해서이다.

---

Part 4 - Making the predictions and evaluating the model

Predicting the result of a single observation

이제 아래에 주어진 한 고객의 데이터를 이용하여 예측을 해보려고한다.

Geography: France

Credit Score: 600

Gender: Male

Age: 40 years old

Tenure: 3 years

Balance: $ 60000

Number of Products: 2

Does this customer have a credit card? Yes

Is this customer an Active Member: Yes

Estimated Salary: $ 50000

print(ann.predict(sc.transform([[1, 0, 0, 600, 1, 40, 3, 60000, 2, 1, 1, 50000]])) > 0.5)

1/1 [==============================] - 0s 18ms/step
[[False]]

고객이 이탈하지 않을 것으로 예측함을 볼 수 있었다.

 

Predicting the Test set results

y_pred = ann.predict(X_test)
y_pred = (y_pred > 0.5)
print(np.concatenate((y_pred.reshape(len(y_pred),1), y_test.reshape(len(y_test),1)),1))

63/63 [==============================] - 0s 1ms/step
[[0 0]
 [0 1]
 [0 0]
 ...
 [0 0]
 [0 0]
 [0 0]]

두번쨰 고객에 대한 예측은 틀렸지만 높은 정확도를 보임이 확인되었다.

 

Making the Confusion Matrix

from sklearn.metrics import confusion_matrix, accuracy_score
cm = confusion_matrix(y_test, y_pred)
print(cm)
accuracy_score(y_test, y_pred)

[[1516   79]
 [ 204  201]]
0.8585

정확도는 0.8585로 굉장히 높음을 확인할 수 있었다.

---