Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UKRのメモリ消費抑える修正 #150

Draft
wants to merge 137 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
137 commits
Select commit Hold shift + click to select a range
71f502b
First commit
ae14watanabe Apr 15, 2020
b5c6af6
既存のメソッド名をrefactor
ae14watanabe Apr 15, 2020
261abd2
visualizeメソッドとして作成中。変数の初期化など。
ae14watanabe Apr 15, 2020
f3a85b2
お試しのためにtutorial codeを作る。とりあえずSOMの動物コードを引っ張って来る
ae14watanabe Apr 15, 2020
e381b62
チュートリアルを実行できる形まで書く
ae14watanabe Apr 15, 2020
ce27717
潜在空間上における潜在変数の描画を実行できるところまでvisualizeメソッドをコーディング
ae14watanabe Apr 15, 2020
b0239d9
代表点の定義、tsom2_viwerの__on_click_figをコピーしてコメントアウト
ae14watanabe Apr 17, 2020
086f02d
代表点からその写像先を計算し変数に格納andクリックされた座標にもっとも近い代表点を計算
ae14watanabe Apr 19, 2020
31a928b
axに表示するmethodを見よう見まねで作成
ae14watanabe Apr 19, 2020
d5f278c
カチカチするとバーが表示されるところまで作成
ae14watanabe Apr 19, 2020
223ca57
tutorialで動物と特徴のラベルも与えて描画するように変更
ae14watanabe Apr 20, 2020
5f3eb9c
特徴量のラベルをバーの横軸のラベルに表示するように変更
ae14watanabe Apr 20, 2020
7f7a573
クリックしたポイントの描画とラベル表示の乱数の追加
ae14watanabe Apr 20, 2020
f033001
まだクリックしていない初期状態の表示を設定
ae14watanabe Apr 20, 2020
ee8e4f9
barのどれかを指定した時にどのバーを指定したのか特定する機能を追加
ae14watanabe Apr 20, 2020
06b8efd
selectされた次元に合わせて潜在空間上でpcolormeshするように試作 and refactor
ae14watanabe Apr 20, 2020
3b4ba56
bagをfixしてとりあえずpcolormeshが表示できるまで作成
ae14watanabe Apr 20, 2020
8578df5
contourも表示するように変更and変数のrefactor
ae14watanabe Apr 21, 2020
de6a05c
refactorに伴うtutorial code内の変数名の変更 and 初期値をモデル任せではなくcode内で指定
ae14watanabe Apr 21, 2020
2fff4d8
選択されている特徴量のbarの色を変更
ae14watanabe Apr 21, 2020
6b7a473
tutorialのseed値を変更
ae14watanabe Apr 21, 2020
e89a9c2
Refactor, reformat and delete unnecessary row
ae14watanabe Apr 21, 2020
f162f3b
Refactor
ae14watanabe Apr 21, 2020
20f02dd
UKRの変数の初期化を別メソッドとして定義し実行するように変更
ae14watanabe Apr 22, 2020
13393e7
Fix
ae14watanabe Apr 22, 2020
44490c5
Refactor
ae14watanabe Apr 22, 2020
825ae81
Edit error message
ae14watanabe Apr 22, 2020
5190eb5
First commit
ae14watanabe Apr 22, 2020
eab1446
外部から参照される可能性のあるメソッドの先頭を__から_に変更
ae14watanabe Apr 22, 2020
5dc565a
Delete unnecessary comments
ae14watanabe Apr 22, 2020
08be578
描画の細かい条件などの修正、破壊的な変更になっていないことは確認
ae14watanabe Apr 22, 2020
0234a6c
処理をメソッド単位に分割
ae14watanabe Apr 22, 2020
d94a1ae
Refactor
ae14watanabe Apr 22, 2020
07a10e0
変数の初期化メソッドで外部からfigureを代入できるように変更
ae14watanabe Apr 22, 2020
fe3d1eb
外部からfigureやaxを指定できるように修正。tutorial codeで実行できることを確認
ae14watanabe Apr 22, 2020
deceb52
tutorialを元に戻す
ae14watanabe Apr 22, 2020
14bac9a
Refractor
ae14watanabe Apr 23, 2020
8ed6c71
Refactor and 後からコード見やすいようにメソッドの定義の順番を入れ替え
ae14watanabe Apr 23, 2020
c5f998c
pltのimportの位置を修正
ae14watanabe Apr 23, 2020
4f41307
tutorialの余分なimportを削除
ae14watanabe Apr 23, 2020
77e76d0
コード読みやすくなることを願ってメソッドの定義の配置を変更
ae14watanabe Apr 24, 2020
41f6ffc
Tweak position of defining variables and comment
ae14watanabe Apr 24, 2020
ff41f78
オプションで潜在変数のラベルをマウスオーバーした時に表示できる機能を追加(未テスト)
ae14watanabe Apr 24, 2020
549ba1b
Fix bugs, enable to change colormap and reformat
ae14watanabe Apr 24, 2020
f296c30
動物データのチュートリアルでcmapを指定
ae14watanabe Apr 24, 2020
15c04d8
潜在変数のscatterの大きさをs=10に(デフォルトが20)
ae14watanabe Apr 24, 2020
ed58f67
外部から代表点の座標を与えられるように修正
ae14watanabe Apr 26, 2020
60d077b
代表点に関連する変数を定義するメソッドを別途定義
ae14watanabe Apr 26, 2020
5fb9a1f
外部から潜在空間に描画する値をsetできるメソッドを定義
ae14watanabe Apr 26, 2020
b0096eb
外部から潜在空間の値を代入された時に、ヒストグラムの方では何も指定しないように変数を初期化
ae14watanabe Apr 26, 2020
be0d484
外部からcmapを指定するメソッドを作成
ae14watanabe Apr 26, 2020
ee11cc2
外部からaxのタイトルを
ae14watanabe Apr 29, 2020
e8fa332
外部からaxのタイトルを指定する機能を追加
ae14watanabe Apr 29, 2020
0a360f5
Fix
ae14watanabe Apr 29, 2020
6cf16a8
tutorialでaxのタイトルを指定するように修正
ae14watanabe Apr 29, 2020
71e6690
axのタイトルを設定できるメソッドを定義
ae14watanabe Apr 29, 2020
ae868f5
feature barのラベルのfontsizeを小さく修正
ae14watanabe Apr 29, 2020
9026abc
First commit
ae14watanabe May 4, 2020
ba7de74
Reformat and fix typo in comment
ae14watanabe May 4, 2020
a13c007
デフォルトでは挙動が変わらないようにfit部を重みに対応
ae14watanabe May 4, 2020
c73108a
transformとinverse_transformで対応(calculate_history_of_mappingは前のcommitで対応)
ae14watanabe May 4, 2020
b87cce5
UKRのRefactor
ae14watanabe May 4, 2020
7dda043
pytorch版をweightを考慮するように修正
ae14watanabe May 4, 2020
2936327
testにweightの調査を追加.計算時間が倍になるのがだるかったのでn_components=3に固定
ae14watanabe May 4, 2020
c3d89de
Fix in test code
ae14watanabe May 4, 2020
6ec95e1
Fix in test code
ae14watanabe May 4, 2020
d7a48f9
Fix in pytorch model to test
ae14watanabe May 4, 2020
b4871aa
Fix in test code
ae14watanabe May 4, 2020
145cbb6
Fix in test model
ae14watanabe May 4, 2020
f86f5ab
historyのobj_funcが未修正だったので修正
ae14watanabe May 4, 2020
bc27c97
numpy版の目的関数の計算式が間違っていたので修正
ae14watanabe May 4, 2020
eb24c85
etaを減らすことでallcloseでFalseになるタイミングを後ろに持ってきてテストがpassすることを確認
ae14watanabe May 4, 2020
e6ff051
不要なファイルを削除
ae14watanabe May 4, 2020
d07a4c3
First commit
ae14watanabe May 18, 2020
940ed54
yとxの差の内積の計算を省メモリな計算式に変更(テスト未実行)
ae14watanabe May 18, 2020
31a6944
利用していない変数の確保を修正
ae14watanabe May 28, 2020
3eadda3
Merge branch '136_add_view_in_ukr' into 145_add_weight_in_ukr
ae14watanabe May 28, 2020
5343e0f
add description of visualize method
ae14watanabe May 29, 2020
b9d118d
Update libs/models/unsupervised_kernel_regression.py
ae14watanabe May 29, 2020
57cfe53
オーバーラップしたときのデータラベルやcontourも白枠で縁取りするように変更
ae14watanabe May 29, 2020
d1df319
is_show_all_label_data=Falseの時のbboxを削除
ae14watanabe May 29, 2020
6a4c7b4
Reformat
ae14watanabe May 29, 2020
a7c5852
潜在空間の描画をpcolormeshからimshowに変更しオプションにinterpolationを追加
ae14watanabe Jun 2, 2020
2db9df9
スクリプト内で定義されていたcreate_zetaを削除しsomfのcreate_zetaを利用するように修正
ae14watanabe Jun 2, 2020
618c954
tutorialもモデルの変更に合わせて修正
ae14watanabe Jun 2, 2020
369b504
Merge remote-tracking branch 'origin/136_add_view_in_ukr' into 145_ad…
ae14watanabe Jun 2, 2020
7cf7834
Merge branch '145_add_weight_in_ukr' of github.com:furukawa-laborator…
ae14watanabe Jun 2, 2020
e25df3f
Merge remote-tracking branch 'origin/145_add_weight_in_ukr' into 149_…
ae14watanabe Jun 2, 2020
cf19cb7
contourが上下反転されるバグをfix
ae14watanabe Jun 4, 2020
c1a0e77
Merge branch '136_add_view_in_ukr' into 145_add_weight_in_ukr
ae14watanabe Jun 4, 2020
b9404ff
Merge branch '145_add_weight_in_ukr' into 149_reduce_memory_in_ukr
ae14watanabe Jun 4, 2020
3f0aa26
visualizeのdocstringをNumpy styleに変更
ae14watanabe Jun 7, 2020
3f8f269
引数にカラーマップの中間の色に対応する値を0に固定するオプションを追加
ae14watanabe Jun 9, 2020
c86ca1a
オプションをチュートリアルにも追加
ae14watanabe Jun 9, 2020
356b096
Merge branch '136_add_view_in_ukr' into 145_add_weight_in_ukr
ae14watanabe Jun 9, 2020
e158328
Merge branch '145_add_weight_in_ukr' into 149_reduce_memory_in_ukr
ae14watanabe Jun 9, 2020
625e87e
オプションとして潜在変数のscatterのcmapやcolor, markerを追加
ae14watanabe Jun 10, 2020
3ff6558
imshowやscatterに関するオプションを細かく与えるのではなく,辞書として渡すように修正
ae14watanabe Jun 11, 2020
cf25309
Merge branch '136_add_view_in_ukr' into 145_add_weight_in_ukr
ae14watanabe Jun 11, 2020
7068f6b
Merge branch '145_add_weight_in_ukr' into 149_reduce_memory_in_ukr
ae14watanabe Jun 11, 2020
f86f1a1
外部からimshowのcmapのみを指定できるメソッドを廃し,imshowやscatterの任意のパラメータを指定できるメソッドを追加
ae14watanabe Jun 11, 2020
097cc2f
Merge branch '136_add_view_in_ukr' into 145_add_weight_in_ukr
ae14watanabe Jun 11, 2020
736abd9
Merge branch '145_add_weight_in_ukr' into 149_reduce_memory_in_ukr
ae14watanabe Jun 11, 2020
ecb787c
Fix bugs
ae14watanabe Jun 11, 2020
ad97484
Merge branch '136_add_view_in_ukr' into 145_add_weight_in_ukr
ae14watanabe Jun 11, 2020
027291a
Merge branch '145_add_weight_in_ukr' into 149_reduce_memory_in_ukr
ae14watanabe Jun 11, 2020
a654be6
各データに異なるmarkerを指定できるようにオプションを追加
ae14watanabe Jun 11, 2020
60f1bd1
Merge branch '136_add_view_in_ukr' into 145_add_weight_in_ukr
ae14watanabe Jun 11, 2020
076b339
Merge branch '145_add_weight_in_ukr' into 149_reduce_memory_in_ukr
ae14watanabe Jun 11, 2020
b96ea39
markerのデフォルト値に誤りがあったためこれを修正
ae14watanabe Jul 16, 2020
84a342b
Merge branch '136_add_view_in_ukr' into 149_reduce_memory_in_ukr
ae14watanabe Jul 16, 2020
01772e4
Merge branch '136_add_view_in_ukr' into 145_add_weight_in_ukr
ae14watanabe Jul 16, 2020
bfd0c6e
Merge branch '145_add_weight_in_ukr' into 149_reduce_memory_in_ukr
ae14watanabe Jul 16, 2020
bc180b0
データのscatterのalphaを外部から指定できるように変更
ae14watanabe Aug 18, 2020
996005f
Merge branch '136_add_view_in_ukr' into 145_add_weight_in_ukr
ae14watanabe Aug 18, 2020
a5f569d
Merge branch '145_add_weight_in_ukr' into 149_reduce_memory_in_ukr
ae14watanabe Aug 18, 2020
88698b3
外部から表示する潜在変数を指定するmask配列を導入
ae14watanabe Aug 18, 2020
54dbfd8
Merge branch '136_add_view_in_ukr' into 145_add_weight_in_ukr
ae14watanabe Aug 18, 2020
730614e
Merge branch '145_add_weight_in_ukr' into 149_reduce_memory_in_ukr
ae14watanabe Aug 18, 2020
189bd43
fix bug and adjust range of figure
ae14watanabe Aug 18, 2020
1e38036
Merge branch '136_add_view_in_ukr' into 145_add_weight_in_ukr
ae14watanabe Aug 18, 2020
1bcb0fb
Merge branch '145_add_weight_in_ukr' into 149_reduce_memory_in_ukr
ae14watanabe Aug 18, 2020
ea07a4c
潜在空間上のクリック済みの点をもう一度クリックすると解除されるように修正
ae14watanabe Aug 18, 2020
5eb0b85
Merge branch '136_add_view_in_ukr' into 145_add_weight_in_ukr
ae14watanabe Aug 18, 2020
bcc09b6
Merge branch '145_add_weight_in_ukr' into 149_reduce_memory_in_ukr
ae14watanabe Aug 18, 2020
07168e7
Fix bugs
ae14watanabe Aug 18, 2020
fa38368
Merge branch '136_add_view_in_ukr' into 145_add_weight_in_ukr
ae14watanabe Aug 18, 2020
3980ebd
Merge branch '145_add_weight_in_ukr' into 149_reduce_memory_in_ukr
ae14watanabe Aug 18, 2020
802891e
外部からx印でplotする点を指定できるように修正
ae14watanabe Aug 20, 2020
1ccd33e
Merge branch '136_add_view_in_ukr' into 145_add_weight_in_ukr
ae14watanabe Aug 20, 2020
4fd4461
Merge branch '145_add_weight_in_ukr' into 149_reduce_memory_in_ukr
ae14watanabe Aug 20, 2020
d951201
latent spaceの軸のラベルを表示しないオプションを追加
ae14watanabe Sep 7, 2020
ade157a
Merge branch '136_add_view_in_ukr' into 145_add_weight_in_ukr
ae14watanabe Sep 7, 2020
51bab72
Merge branch '145_add_weight_in_ukr' into 149_reduce_memory_in_ukr
ae14watanabe Sep 7, 2020
b4ab7a0
外部から潜在空間に表示するコメントを追加できるように修正
ae14watanabe Sep 7, 2020
2e0cbad
Merge branch '136_add_view_in_ukr' into 145_add_weight_in_ukr
ae14watanabe Sep 7, 2020
1639334
Merge branch '145_add_weight_in_ukr' into 149_reduce_memory_in_ukr
ae14watanabe Sep 7, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
578 changes: 522 additions & 56 deletions libs/models/unsupervised_kernel_regression.py

Large diffs are not rendered by default.

9 changes: 9 additions & 0 deletions testfunc.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
def func_default(arg1=1, arg2=2, arg3=3):
print(arg1)
print(arg2)
print(arg3)

none_dict = {}
print('dict type is {}'.format(type(none_dict)))
print(isinstance(none_dict, dict))
func_default(4, arg2=2, **none_dict)
1 change: 0 additions & 1 deletion tests/ukr/.pytest_cache/v/cache/nodeids

This file was deleted.

47 changes: 28 additions & 19 deletions tests/ukr/allclose_numpy_vs_pytorch.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@

class TestUKR(unittest.TestCase):
def test_numpy_vs_pytorch(self):
# create artiricial dataset
# create artificial dataset
nb_samples = 500
seed = 1
np.random.seed(seed)
Expand All @@ -20,14 +20,15 @@ def test_numpy_vs_pytorch(self):
X += np.random.normal(0, x_sigma, X.shape)

# set parameter
n_components_list = [1,2,3]
n_components_list = [3]
bandwidth_gaussian_kernel_list = [0.2, 1.0]
nb_epoch = 500

is_compact_list = [True, False]
is_save_history = True
lambda_list = [0.0, 0.005]
eta = 8.0
weights_list = [None, np.random.rand(nb_samples)]
eta = 7.0

for n_components in n_components_list:
# initialize Z
Expand All @@ -36,22 +37,30 @@ def test_numpy_vs_pytorch(self):
for bandwidth_gaussian_kernel in bandwidth_gaussian_kernel_list:
for is_compact in is_compact_list:
for lambda_ in lambda_list:
ukr = UKR(X, n_components=n_components, bandwidth_gaussian_kernel=bandwidth_gaussian_kernel,
is_compact=is_compact, is_save_history=is_save_history, lambda_=lambda_, init=Zinit)
ukr.fit(nb_epoch=nb_epoch, eta=eta)
all_z = ukr.history['z']
all_obj_func = ukr.history['obj_func']

ukr_pytorch = UKR_pytorch(torch.from_numpy(X),
nb_components=n_components,
bandwidth_gaussian_kernel=bandwidth_gaussian_kernel,
is_compact=is_compact, is_save_history=is_save_history, lambda_=lambda_,
init=torch.tensor(Zinit, requires_grad=True, dtype=torch.float64))
ukr_pytorch.fit(nb_epoch=nb_epoch, eta=eta)
all_z_pytorch = ukr_pytorch.history['z'].detach().numpy()
all_obj_func_pytorch = ukr_pytorch.history['obj_func'].detach().numpy()
np.testing.assert_allclose(all_z, all_z_pytorch)
np.testing.assert_allclose(all_obj_func,all_obj_func_pytorch)
for weights in weights_list:
ukr = UKR(X, n_components=n_components, bandwidth_gaussian_kernel=bandwidth_gaussian_kernel,
weights=weights, is_compact=is_compact, is_save_history=is_save_history,
lambda_=lambda_, init=Zinit)
ukr.fit(nb_epoch=nb_epoch, eta=eta)
all_z = ukr.history['z']
all_obj_func = ukr.history['obj_func']

if isinstance(weights, np.ndarray):
weights = torch.from_numpy(weights)
else:
pass

ukr_pytorch = UKR_pytorch(torch.from_numpy(X),
nb_components=n_components,
bandwidth_gaussian_kernel=bandwidth_gaussian_kernel,
is_compact=is_compact, is_save_history=is_save_history,
lambda_=lambda_, weights=weights,
init=torch.tensor(Zinit, requires_grad=True, dtype=torch.float64))
ukr_pytorch.fit(nb_epoch=nb_epoch, eta=eta)
all_z_pytorch = ukr_pytorch.history['z'].detach().numpy()
all_obj_func_pytorch = ukr_pytorch.history['obj_func'].detach().numpy()
np.testing.assert_allclose(all_z, all_z_pytorch)
np.testing.assert_allclose(all_obj_func, all_obj_func_pytorch)


if __name__ == "__main__":
Expand Down
16 changes: 8 additions & 8 deletions tests/ukr/model/unsupervised_kernel_regression_pytorch.py
Original file line number Diff line number Diff line change
@@ -1,19 +1,14 @@
# coding: utf-8
import sys
sys.path.append('../')
#from som.datasets.kura import create_data
# from KSE.lib.datasets.artificial.spiral import create_data

import numpy as np
import torch
import matplotlib.pyplot as plt
import matplotlib.animation as anim
from mpl_toolkits.mplot3d import Axes3D
from tqdm import tqdm

class Unsupervised_Kernel_Regression_pytorch(object):
def __init__(self, X, nb_components, bandwidth_gaussian_kernel=1.0,
is_compact=False, lambda_=1.0,
is_compact=False, lambda_=1.0, weights=None,
init='random', is_loocv=False, is_save_history=False):
self.X = X.clone()
self.nb_samples = X.shape[0]
Expand All @@ -35,6 +30,11 @@ def __init__(self, X, nb_components, bandwidth_gaussian_kernel=1.0,

self.lambda_ = lambda_

if weights is None:
self.weights = torch.ones(self.nb_samples)
else:
self.weights = weights

self._done_fit = False

def fit(self, nb_epoch=100, verbose=True, eta=0.5, expand_epoch=None):
Expand All @@ -54,7 +54,7 @@ def fit(self, nb_epoch=100, verbose=True, eta=0.5, expand_epoch=None):

for epoch in bar:
DistZ = torch.sum((self.Z[:, None, :] - self.Z[None, :, :])**2, dim=2)
H = torch.exp(-0.5 * self.precision * DistZ)
H = torch.exp(-0.5 * self.precision * DistZ) * self.weights[None, :]
if self.is_loocv:
# H -= np.identity(H.shape[0])
assert 'Not Implemented Error'
Expand All @@ -64,7 +64,7 @@ def fit(self, nb_epoch=100, verbose=True, eta=0.5, expand_epoch=None):

Y = torch.mm(R,self.X)
Error = Y - self.X
obj_func = torch.sum(Error**2) / self.nb_samples + self.lambda_ * torch.sum(self.Z**2)
obj_func = torch.sum((Error**2) * self.weights[:, None]) / self.weights.sum() + self.lambda_ * torch.sum(self.Z**2)
obj_func.backward()
with torch.no_grad():
self.Z = self.Z - eta * self.Z.grad
Expand Down
39 changes: 39 additions & 0 deletions tutorials/ukr/fitting_animal_data.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
from libs.models.unsupervised_kernel_regression import UnsupervisedKernelRegression as UKR
from libs.datasets.artificial import animal
from sklearn.utils import check_random_state

if __name__ == '__main__':
n_components = 2
bandwidth = 0.4
lambda_ = 0.0
is_compact = True
is_save_history = True

nb_epoch = 1000
eta = 0.02

X, labels_animal, labels_feature = animal.load_data(retlabel_animal=True,
retlabel_feature=True)

seed = 14
random_state = check_random_state(seed)
init = random_state.normal(0.0, bandwidth * 0.1, size=(X.shape[0], n_components))

ukr = UKR(X, n_components=n_components, bandwidth_gaussian_kernel=bandwidth,
is_compact=is_compact, is_save_history=is_save_history, lambda_=lambda_, init=init)
ukr.fit(nb_epoch=nb_epoch, eta=eta)

params_imshow = {'cmap': 'BrBG',
'interpolation': 'spline16'}
params_scatter = {'s': 30, 'marker': 'x'}
ukr.visualize(
n_grid_points=100,
label_data=labels_animal,
label_feature=labels_feature,
is_middle_color_zero=True,
is_show_all_label_data=True,
params_imshow=params_imshow,
params_scatter=params_scatter,
title_latent_space='Animal map (latent space)',
title_feature_bars='Animal feature'
)
2 changes: 1 addition & 1 deletion tutorials/ukr/fitting_saddle_shape.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@
is_compact=is_compact, is_save_history=is_save_history, lambda_=lambda_)
som.fit(nb_epoch=nb_epoch)
ukr.fit(nb_epoch=nb_epoch, eta=eta)
ukr.calculation_history_of_mapping(resolution=30)
ukr.calculate_history_of_mapping(resolution=30)

fig = plt.figure(figsize=[7, 8])
ax_latent_space_som = fig.add_subplot(2, 2, 1, aspect='equal')
Expand Down