[머신러닝-Tensorflow] Lec-05 Cost Minimize 실습

1. hypothesis 요약

cost가 최적화 된다는것은 최저가 되는 지점

Cost Function 을 간단히 요약하면 이렇게 표현이 가능하다.

2.Cost Minimizing Structure

W.Assgin 함수를 통해서 Assgin할수 있다. 이것의 update에 할당해서 실행을 시키게 된다면,

일련의 동작들이 수행되게 된다.

3. matplotlib 설치하기

실습에 앞서

matplotlib 를 사용할것인데, 이것은 시각화 라이브러리라고 생각하면 편하다. 실습에서 사용할것이므로

(tensorflow) kgh-2:tensorflow kgh$ pip install matplotlib

4. 실습 Cost Minimizing -1 

이제 이것을 사용하여 그래프를 그려보자

# Lab 3 Minimizing Cost
	import tensorflow as tf
	import matplotlib.pyplot as plt
	tf.set_random_seed(777) # for reproducibility
	X = [1, 2, 3]
	Y = [1, 2, 3]
	#학습시킬 데이터
	W = tf.placeholder(tf.float32) # 임의대로 바꾸어보겠다.
	# Our hypothesis for linear model X * W
	hypothesis = X * W
	# cost/loss function
	cost = tf.reduce_mean(tf.square(hypothesis - Y))
	# Launch the graph in a session.
	sess = tf.Session()
	#세션 초기화 #Variable 초기화
	# Variables for plotting cost function
	W_history = []
	cost_history = []
	# 범위만큼 움직이기 위함. -30~50
	for i in range(-30, 50): #0.1간격으로 움직임.
	curr_W = i * 0.1
	curr_cost = sess.run(cost, feed_dict={W: curr_W})
	W_history.append(curr_W)
	cost_history.append(curr_cost)
	# Show the cost function # X,Y축의 그래프를 그리기 위함.
	plt.plot(W_history, cost_history)
	plt.show()

이 코드의 결과 Gradient descent Algorigm

미분값을 사용하여 그래프의 기울기를 이용할것이다.

4. 실습 Cost Minimizing -2

Code:

import tensorflow as tf
	tf.set_random_seed(777) # for reproducibility
	x_data = [1, 2, 3]
	y_data = [1, 2, 3]
	# 데이터 선언
	# Try to find values for W and b to compute y_data = W * x_data + b
	# We know that W should be 1 and b should be 0
	# But let's use TensorFlow to figure it out
	W = tf.Variable(tf.random_normal([1]), name='weight')
	# 노드 할당
	X = tf.placeholder(tf.float32)
	Y = tf.placeholder(tf.float32)
	# Our hypothesis for linear model X * W
	hypothesis = X * W
	# cost/loss function
	cost = tf.reduce_mean(tf.square(hypothesis - Y))
	# Minimize: Gradient Descent using derivative: W -= learning_rate * derivative
	learning_rate = 0.1 # H(x)
	gradient = tf.reduce_mean((W * X - Y) * X)
	descent = W - learning_rate * gradient # Assgin(descent)
	update = W.assign(descent)
	# Launch the graph in a session.
	sess = tf.Session()
	# Initializes global variables in the graph.
	sess.run(tf.global_variables_initializer())
	for step in range(21):
	sess.run(update, feed_dict={X: x_data, Y: y_data})
	print(step, sess.run(cost, feed_dict={X: x_data, Y: y_data}), sess.run(W))

Result: 

array([-0.08443087], dtype=float32)

(0, 5.487955, array([-0.08443087], dtype=float32))

array([0.42163688], dtype=float32)

(1, 1.5610181, array([0.42163688], dtype=float32))

array([0.69153965], dtype=float32)

(2, 0.4440231, array([0.69153965], dtype=float32))

array([0.83548784], dtype=float32)

(3, 0.12629984, array([0.83548784], dtype=float32))

array([0.9122602], dtype=float32)

(4, 0.035925273, array([0.9122602], dtype=float32))

array([0.9532054], dtype=float32)

(5, 0.0102187535, array([0.9532054], dtype=float32))

array([0.9750429], dtype=float32)

(6, 0.002906667, array([0.9750429], dtype=float32))

array([0.9866895], dtype=float32)

(7, 0.00082679116, array([0.9866895], dtype=float32))

array([0.9929011], dtype=float32)

(8, 0.00023517467, array([0.9929011], dtype=float32))

array([0.9962139], dtype=float32)

(9, 6.689412e-05, array([0.9962139], dtype=float32))

array([0.9979808], dtype=float32)

(10, 1.9027528e-05, array([0.9979808], dtype=float32))

array([0.99892306], dtype=float32)

(11, 5.412366e-06, array([0.99892306], dtype=float32))

array([0.99942565], dtype=float32)

(12, 1.5394322e-06, array([0.99942565], dtype=float32))

array([0.9996937], dtype=float32)

(13, 4.3781233e-07, array([0.9996937], dtype=float32))

array([0.9998366], dtype=float32)

(14, 1.2458133e-07, array([0.9998366], dtype=float32))

array([0.99991286], dtype=float32)

(15, 3.541662e-08, array([0.99991286], dtype=float32))

array([0.9999535], dtype=float32)

(16, 1.0086865e-08, array([0.9999535], dtype=float32))

array([0.9999752], dtype=float32)

(17, 2.8691527e-09, array([0.9999752], dtype=float32))

array([0.99998677], dtype=float32)

(18, 8.1394563e-10, array([0.99998677], dtype=float32))

array([0.9999929], dtype=float32)

(19, 2.3393554e-10, array([0.9999929], dtype=float32))

array([0.9999962], dtype=float32)

(20, 6.7908935e-11, array([0.9999962], dtype=float32))

5. 텐서플로우 사용시 일일이 미분할필요가 없이 Optimizer를 사용하여 텐서쪽에서 다 처리해주게 된다.

Code:

import tensorflow as tf
	tf.set_random_seed(777) # for reproducibility
	# tf Graph Input
	X = [1, 2, 3]
	Y = [1, 2, 3]
	# Set wrong model weights
	W = tf.Variable(5.0)
	# Linear model
	hypothesis = X * W
	# cost/loss function
	cost = tf.reduce_mean(tf.square(hypothesis - Y))
	# Minimize: Gradient Descent Magic #여기서 간편하게 처리하게해준다
	optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.1)
	train = optimizer.minimize(cost)
	# Launch the graph in a session.
	sess = tf.Session()
	# Initializes global variables in the graph.
	sess.run(tf.global_variables_initializer())
	for step in range(100):
	print(step, sess.run(W))
	sess.run(train)

6. Tensorflow가 gradient의 값을 바꾸고 싶을 경우 

Code: 

# This is optional
	import tensorflow as tf
	tf.set_random_seed(777) # for reproducibility
	# tf Graph Input
	X = [1, 2, 3]
	Y = [1, 2, 3]
	# Set wrong model weights
	W = tf.Variable(5.)
	# Linear model
	hypothesis = X * W
	# Manual gradient
	gradient = tf.reduce_mean((W * X - Y) * X) * 2
	# cost/loss function
	cost = tf.reduce_mean(tf.square(hypothesis - Y))
	# Minimize: Gradient Descent Magic
	optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.01)
	train = optimizer.minimize(cost)
	# Gradients 를 계산한 값을 원하는데로 수정 할 수 있음.
	# Get gradients
	gvs = optimizer.compute_gradients(cost, [W])
	# Optional: modify gradient if necessary
	# gvs = [(tf.clip_by_value(grad, -1., 1.), var) for grad, var in gvs]
	# Apply gradients
	apply_gradients = optimizer.apply_gradients(gvs)
	# Launch the graph in a session.
	sess = tf.Session()
	# Initializes global variables in the graph.
	sess.run(tf.global_variables_initializer())
	for step in range(100):
	print(step, sess.run([gradient, W, gvs]))
	sess.run(apply_gradients)
	# Same as sess.run(train)