03. TensorFlow Linear Regression 의 cost 최소화 구현

출처 : 모두를위한 머신러닝 (http://hunkim.github.io/ml/)

---

이론

---

간소화 한 Hypothesis¶

---

W, cost(W) 스캔 해보기¶

In [9]:

import tensorflow as tf
import matplotlib.pyplot as plt

X = [1, 2, 3]
Y = [1, 2, 3]

# W 는 계속 바뀌므로 place holder 로
W = tf.placeholder(tf.float32)

hypothesis = X * W    # H(x) = Wx

cost = tf.reduce_mean(tf.square(hypothesis - Y))    # cost function

sess = tf.Session()

# place holder 가 있으므로 initialize
sess.run(tf.global_variables_initializer())

W_val = []
cost_val = []

for i in range(-30, 50):
    feed_W = i * 0.1    # place holder 에 넘길 w 값  (-3~5 정도)
    curr_cost, curr_w = sess.run([cost, W], feed_dict={W: feed_W})
    W_val.append(curr_w)
    cost_val.append(curr_cost)
    
fig = plt.figure(figsize=(8,6))
plt.plot(W_val, cost_val, marker='o')
plt.grid()
plt.show()

---

기울기 구하여 Gradient descent¶

• 한 점에서 기울기를 구하여 (+) 인 경우 W 에서 빼고
• (-) 인 경우에는 W 에서 더함
• 이 과정을 반복하면 점점 중앙 (최소점) 으로 W 가 이동함

Flow chart¶

In [11]:

import tensorflow as tf

x_data = [1, 2, 3]
y_data = [1, 2, 3]

W = tf.Variable(tf.random_normal([1]), name='weight')
X = tf.placeholder(tf.float32)
Y = tf.placeholder(tf.float32)

hypothesis = X * W
cost = tf.reduce_sum(tf.square(hypothesis - Y))


# Minimize
# W = W - (learning_rate * derivative)
learning_rate = 0.1
gradient = tf.reduce_mean((W * X - Y) * X)
descent = W - learning_rate * gradient
update = W.assign(descent)

# Launch
sess = tf.Session()
sess.run(tf.global_variables_initializer())
for step in range(21):
    sess.run(update, feed_dict={X: x_data, Y: y_data})
    print(step, sess.run(cost, feed_dict={X: x_data, Y: y_data}), sess.run(W))

0 15.6363 [-0.05682445]
1 4.44766 [ 0.43636033]
2 1.26511 [ 0.6993922]
3 0.359854 [ 0.83967584]
4 0.102358 [ 0.9144938]
5 0.0291153 [ 0.95439667]
6 0.0082817 [ 0.97567821]
7 0.00235569 [ 0.98702836]
8 0.000670059 [ 0.99308181]
9 0.000190594 [ 0.99631029]
10 5.42132e-05 [ 0.99803215]
11 1.54209e-05 [ 0.99895048]
12 4.38624e-06 [ 0.99944025]
13 1.24804e-06 [ 0.99970144]
14 3.54786e-07 [ 0.9998408]
15 1.0103e-07 [ 0.99991506]
16 2.87287e-08 [ 0.9999547]
17 8.16691e-09 [ 0.99997586]
18 2.32058e-09 [ 0.99998713]
19 6.55337e-10 [ 0.99999315]
20 1.86379e-10 [ 0.99999636]

• 위에서 만든 Minimze 를 해주는 내장함수
• optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.1)
• train = optimizer.minimize(cost)

In [12]:

import tensorflow as tf

X = [1, 2, 3]
Y = [1, 2, 3]

W = tf.Variable(5.0)

hypothesis = X * W

cost = tf.reduce_mean(tf.square(hypothesis - Y))

optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.1)
train = optimizer.minimize(cost)

sess = tf.Session()
sess.run(tf.global_variables_initializer())

for step in range(10):
    print(step, sess.run(W))
    sess.run(train)

0 5.0
1 1.26667
2 1.01778
3 1.00119
4 1.00008
5 1.00001
6 1.0
7 1.0
8 1.0
9 1.0