[NLP] Day 35 - Topic Clustering

1. LSA를 사용해서 Topic Clustering

이전 내용을 기반으로해서 News data에 적용시켜보기

In [1]:

import os 
corpus=[]
for file in os.listdir('./News'):
        if file.startswith('정치'):
            with open('./News/'+file,encoding='utf-8') as f:
                corpus.append([file,f.read()])

In [2]:

len(corpus)

Out[2]:

40

In [3]:

corpus[0][1]

Out[3]:

"\n\n\n\n\n// flash 오류를 우회하기 위한 함수 추가\nfunction _flash_removeCallback() {}\n\n  김부겸 행정안전부장관이 14일 서울 여의도 국회 행정안전위원회에 출석해 얼굴을 어루만지고 있다. [뉴스1]           김부겸 행정안전부 장관이 정부의 개각 인사 발표 방식에 대해 “늘 하던 방식이 아닌 출신고별로 발표하는 발상은 누가 했는지 모르지만, 상당히 치졸하다고 생각한다”며 비판적 태도를 보였다.        김 장관은 14일 국회에서 열린 행정안전위원회 업무보고 오후 질의에서 윤재옥 자유한국당 의원의 질문에 이같이 답했다. 이날 질의는 사실상 자신의 마지막 국회 업무보고다.      윤 의원은 “장관 일곱 분 개각이 됐는데 TK(대구ㆍ경북) 출신은 한 명도 없다”며 “정략적으로 고립화한다는 지역 여론이 있다”고 했다. 또 “출신 지역을 숨기고 출신고를 발표했는데 그 결과 호남 출신은 한 명도 없는 것으로 나왔으나 실제로는 4명이었다”며 “특정 지역이 소외감을 느끼는 불균형 인사는 빨리 시정돼야 한다. 국회로 돌아오면 목소리를 같이 내 달라”고 질의했다.      이에 김 장관은 “대한민국에서 인사를 하면 늘 그런 식으로 평가가 엇갈리게 마련이지만, 그런 측면이 있더라도 한 국가의 인사에 그런 잣대를 들이대는 것은 지나치다”고 답했다. 이에 김 장관은 ‘출신고 기준’ 발표 방식이 치졸하다면서 “앞으로는 제가 국회로 돌아가서 그런 문제에 앞장서겠다”고 말했다.      앞서 지난 8일 문재인 대통령은 진영 의원을 새 행안부 장관에 내정했다. 당시 청와대는 개각 명단을 발표하면서 이번에 처음으로 출신지를 제외하고 출생연도와 출신 고교ㆍ대학 등 주요 학력과 경력만을 공개했다.      문재인 대통령이 지난 8일 7개 부처에 대한 중폭 개각을 단행했다. 왼쪽 위부터 시계방향으로 중소벤처기업부장관에 내정된 박영선 더불어민주당 의원, 행안부장관에 내정된 진영 더불어민주당 의원, 통일부장관에 내정된 김연철 통일연구원장, 국토부장관에 내정된 최정호 전 국토부 2차관, 과기부장관에 내정된 조동호 카이스트 교수, 해수부장관에 내정된 문성혁 세계해사대교수, 문체부장관에 내정된 박양우 전 문화관광부 차관. [사진 청와대]           장관 후보자 중 서울 지역 고등학교 졸업자는 조동호 과학기술정보통신부 장관 후보자(서울 배문고), 진영 행정안전부 장관 후보자(서울 경기고), 문성혁 해양수산부 장관 후보자(서울 대신고), 박영선 중소벤처기업부 장관 후보자(서울 수도여고) 등 4명이다. 김연철 통일부 장관 후보자는 강원 북평고, 박양우 문화체육관광부 장관 후보자는 인천 제물포고, 최정호 국토교통부 장관 후보자는 경북 금오공고를 나왔다. 고등학교 기준으로 하면 서울 4명, 인천 1명, 경북 1명, 강원 1명의 분포다.        그러나 종전의 출생지 기준으로 재분류를 하면 전북이 3명(진영ㆍ조동호ㆍ최정호)이고 광주 1명(박양우), 부산 1명(문성혁), 경남 1명(박영선), 강원 1명(김연철)의 분포가 된다. 청와대 발표에는 안 보이던 호남 출신이 4명이다.        당시 청와대는 “지연 중심 문화를 탈피해야 한다는 데 사회의 공감대가 있다”면서 “출신지라는 게 객관적이지도 않아서 그곳에서 태어나 오랫동안 성장한 사람이 있는가 하면 출생만 하고 성장은 다른 곳에서 해온 분들도 있다. 불필요한 논란을 끌지 않기 위해 이번에 고등학교 중심으로 발표했다”고 설명했다.        한영혜 기자 han.younghye@joongang.co.kr  ▶ 중앙일보 '홈페이지' / '페이스북' 친구추가▶ 네이버 구독 1위 신문, 중앙일보ⓒ중앙일보(https://joongang.co.kr), 무단 전재 및 재배포 금지\n\t\n"

In [4]:

from string import punctuation
# punc = list(“,”)
punc = ['“','”']
punc
for i in punc:
    punctuation+=i
    
punctuation

Out[4]:

'!"#$%&\'()*+,-./:;<=>?@[\\]^_`{|}~“”'

In [5]:

corpus2 = list()
for i in range(len(corpus)):
    corpus[i][1] = corpus[i][1].translate(str.maketrans('', '', punctuation))
    corpus[i][1] = corpus[i][1].replace('flash 오류를 우회하기 위한 함수 추가\nfunction flashremoveCallback', '')
    corpus2.append((corpus[i][0],corpus[i][1]))

In [7]:

## 진짜 Noun으로만 DTM 만드는거 
from collections import defaultdict
from konlpy.tag import Kkma

DTM = defaultdict(lambda: defaultdict(int))
dictNoun = list()

for i in range(len(corpus2)):
    for t in Kkma().pos(corpus2[i][1]):
        if len(t[0]) >1 and t[1].startswith('N'):
            DTM[i][t[0]] += 1
            

In [256]:

# DTM

In [9]:

from math import log10

def rawTF(freq):
    return freq

def normTF(freq,totalCount):
    return (freq / totalCount)

def logTF(freq):
    if freq > 0:
        return 1 + log10(freq)
    else:
        return 0

def maxTF(a,freq,maxFreq):   # double normalization K -  doc : 0 / query : 0.5
    return a + ((1-a)* (freq/maxFreq))

In [10]:

def convertInvertedDocument(DTM):
    TDM = defaultdict(lambda: defaultdict(int))
    
    for fileName, termList in DTM.items():  
        maxFreq = max(termList.values())
        for term, freq in termList.items():
            TDM[term][fileName] = maxTF(0,freq,maxFreq)
            
    return TDM

In [145]:

termList = list([] for _ in range(len(corpus2)))
freqList = list([] for _ in range(len(corpus2)))
zero = list([0] for _ in range(2979))

In [60]:

totalword = []
for i in range(len(corpus2)):
    for j,d in DTM[i].items():
        totalword.append(j)

In [146]:

for i in range(len(corpus2)):
    for j,d in DTM[i].items():
        termList[i].append(j)

In [257]:

# termList

In [193]:

coldata = list(set(totalword))
# coldata

In [99]:

coldata = np.array(coldata)

In [107]:

len(termList)

Out[107]:

40

In [194]:

case = list(list(0 for _ in range(len(wordList))) for _ in range(len(docName)))
case = np.array(case)

In [195]:

data = pd.DataFrame(case,columns=coldata)
import tqdm

In [196]:

data

Out[196]:

	발언	30	포스	절전	두고	분산	촉구	강물	평가	사이트	...	핵심	이제	배제	감정	비판적	문체부	일방적	메시지	보험	현종
0	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
1	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
2	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
3	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
4	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
5	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
6	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
7	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
8	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
9	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
10	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
11	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
12	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
13	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
14	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
15	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
16	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
17	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
18	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
19	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
20	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
21	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
22	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
23	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
24	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
25	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
26	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
27	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
28	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
29	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
30	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
31	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
32	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
33	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
34	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
35	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
36	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
37	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
38	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
39	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0

40 rows × 2979 columns

In [197]:

for i in tqdm.tqdm_notebook(range(40)):
    for t in tqdm.tqdm_notebook(termList[i]):
        if t in list(data[i:(i+1)].columns):
            data[i:(i+1)][t] += 1

/Users/charming/anaconda3/lib/python3.7/site-packages/ipykernel_launcher.py:4: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
  after removing the cwd from sys.path.

In [198]:

data

Out[198]:

	발언	30	포스	절전	두고	분산	촉구	강물	평가	사이트	...	핵심	이제	배제	감정	비판적	문체부	일방적	메시지	보험	현종
0	0	0	0	0	0	0	0	0	1	0	...	0	0	0	0	1	1	0	0	0	0
1	0	1	0	0	0	0	0	0	0	1	...	0	0	0	0	0	0	0	0	1	0
2	0	0	0	0	0	0	1	0	0	0	...	0	0	0	0	0	0	0	0	0	0
3	0	0	0	0	0	0	0	0	1	0	...	0	0	0	0	0	0	0	0	0	1
4	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	1	0
5	0	0	1	0	0	0	0	1	1	0	...	1	0	0	0	0	0	0	0	0	0
6	1	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	1	0	0	0
7	0	1	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
8	0	1	0	0	0	0	0	0	0	0	...	1	0	0	0	0	0	0	0	0	0
9	0	0	0	0	0	0	1	0	0	0	...	1	0	0	0	0	0	0	0	0	0
10	1	0	0	0	0	0	1	0	0	0	...	0	0	0	0	0	0	0	0	0	0
11	0	0	0	0	0	0	1	0	0	0	...	0	1	0	0	0	0	0	0	0	0
12	1	1	0	0	0	0	0	0	0	0	...	1	0	0	0	0	0	0	0	0	0
13	1	0	0	0	0	0	1	0	0	0	...	0	1	0	1	0	0	0	0	0	0
14	0	1	0	0	0	0	0	0	1	0	...	0	0	0	0	0	0	0	0	0	0
15	0	0	0	1	0	0	0	0	0	0	...	0	0	0	0	0	0	0	1	0	0
16	0	0	0	0	0	0	1	0	0	0	...	0	0	0	0	0	0	0	0	0	0
17	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
18	1	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
19	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
20	0	0	0	0	0	0	0	0	0	1	...	0	0	0	0	0	0	0	0	0	0
21	1	0	0	0	0	0	0	0	1	0	...	0	0	0	0	0	0	0	1	0	0
22	0	0	0	0	0	0	0	0	0	0	...	0	0	1	0	0	0	0	0	0	0
23	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
24	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
25	0	0	0	0	0	0	1	0	0	0	...	0	0	0	0	0	0	0	0	0	0
26	1	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
27	0	0	0	0	0	0	0	0	0	1	...	0	0	0	0	0	0	0	0	0	0
28	1	0	0	0	0	0	0	0	0	0	...	1	0	0	0	0	0	0	0	0	0
29	1	1	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
30	0	0	0	0	1	0	0	0	0	0	...	0	0	1	0	0	0	0	1	0	0
31	0	1	0	0	0	0	0	0	1	0	...	0	0	0	0	0	0	0	0	0	0
32	1	1	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
33	1	0	0	0	0	0	0	0	1	0	...	0	0	0	0	0	0	0	0	0	0
34	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
35	0	1	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
36	0	0	0	0	0	1	0	0	0	0	...	1	0	0	0	0	0	0	0	0	0
37	1	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
38	1	0	0	0	0	0	1	0	0	0	...	0	0	0	0	0	0	0	0	0	0
39	1	0	0	0	1	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0

40 rows × 2979 columns

In [111]:

termList = np.array(termList)
freqList = np.array(freqList)

In [199]:

U,sigma,Vt = np.linalg.svd(data.T,full_matrices=False)

In [202]:

U.shape, sigma.shape, Vt.shape

Out[202]:

((2979, 40), (40,), (40, 40))

In [203]:

_sigma = np.diag(sigma)
_sigma.shape

Out[203]:

(40, 40)

In [211]:

_D = np.round(U.dot(_sigma.dot(Vt)))
pd.DataFrame(_D,index=coldata)

Out[211]:

	0	1	2	3	4	5	6	7	8	9	...	30	31	32	33	34	35	36	37	38	39
발언	0.0	0.0	0.0	-0.0	0.0	-0.0	1.0	-0.0	-0.0	0.0	...	0.0	0.0	1.0	1.0	0.0	-0.0	-0.0	1.0	1.0	1.0
30	0.0	1.0	0.0	-0.0	0.0	-0.0	-0.0	1.0	1.0	-0.0	...	-0.0	1.0	1.0	0.0	0.0	1.0	-0.0	0.0	-0.0	-0.0
포스	-0.0	0.0	-0.0	-0.0	-0.0	1.0	0.0	-0.0	-0.0	-0.0	...	0.0	-0.0	0.0	-0.0	-0.0	-0.0	0.0	0.0	-0.0	-0.0
절전	0.0	0.0	0.0	0.0	0.0	0.0	0.0	-0.0	0.0	0.0	...	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	-0.0	0.0
두고	-0.0	-0.0	0.0	-0.0	0.0	-0.0	0.0	-0.0	-0.0	0.0	...	1.0	-0.0	0.0	0.0	-0.0	-0.0	-0.0	0.0	-0.0	1.0
분산	-0.0	0.0	0.0	0.0	-0.0	-0.0	0.0	0.0	-0.0	0.0	...	-0.0	0.0	-0.0	0.0	-0.0	0.0	1.0	-0.0	0.0	-0.0
촉구	0.0	0.0	1.0	0.0	0.0	-0.0	-0.0	-0.0	-0.0	1.0	...	-0.0	0.0	0.0	-0.0	0.0	-0.0	-0.0	0.0	1.0	0.0
강물	-0.0	0.0	-0.0	-0.0	-0.0	1.0	-0.0	-0.0	0.0	0.0	...	0.0	0.0	0.0	0.0	-0.0	-0.0	-0.0	-0.0	0.0	-0.0
평가	1.0	0.0	0.0	1.0	0.0	1.0	0.0	-0.0	-0.0	-0.0	...	0.0	1.0	0.0	1.0	0.0	-0.0	-0.0	0.0	0.0	0.0
사이트	0.0	1.0	0.0	-0.0	0.0	-0.0	0.0	0.0	-0.0	-0.0	...	0.0	-0.0	0.0	0.0	0.0	-0.0	-0.0	0.0	-0.0	-0.0
나발	-0.0	0.0	-0.0	-0.0	-0.0	1.0	-0.0	0.0	0.0	0.0	...	0.0	0.0	0.0	0.0	0.0	0.0	0.0	-0.0	-0.0	0.0
일훈	0.0	0.0	-0.0	-0.0	0.0	-0.0	-0.0	0.0	-0.0	-0.0	...	0.0	0.0	0.0	0.0	0.0	-0.0	-0.0	0.0	0.0	0.0
참사	0.0	0.0	0.0	-0.0	0.0	-0.0	0.0	0.0	-0.0	0.0	...	0.0	0.0	1.0	-0.0	0.0	-0.0	-0.0	-0.0	0.0	0.0
부업	0.0	-0.0	0.0	0.0	1.0	-0.0	0.0	-0.0	0.0	0.0	...	-0.0	0.0	-0.0	-0.0	-0.0	0.0	-0.0	-0.0	0.0	-0.0
3000	0.0	0.0	0.0	0.0	0.0	-0.0	0.0	0.0	0.0	-0.0	...	-0.0	-0.0	0.0	0.0	0.0	0.0	-0.0	0.0	0.0	0.0
강경	0.0	0.0	-0.0	-0.0	0.0	-0.0	1.0	0.0	-0.0	-0.0	...	0.0	0.0	-0.0	0.0	0.0	-0.0	-0.0	0.0	-0.0	-0.0
경원	0.0	0.0	0.0	-0.0	0.0	-0.0	1.0	1.0	-0.0	-0.0	...	0.0	0.0	0.0	1.0	0.0	-0.0	-0.0	0.0	1.0	1.0
말씀	0.0	-0.0	-0.0	-0.0	1.0	1.0	0.0	-0.0	0.0	0.0	...	0.0	-0.0	0.0	0.0	0.0	-0.0	1.0	-0.0	-0.0	-0.0
보안법	0.0	-0.0	0.0	-0.0	0.0	-0.0	0.0	0.0	-0.0	0.0	...	0.0	0.0	-0.0	0.0	0.0	-0.0	-0.0	0.0	-0.0	0.0
채널	0.0	1.0	1.0	1.0	0.0	-0.0	-0.0	-0.0	1.0	1.0	...	0.0	-0.0	1.0	1.0	1.0	-0.0	-0.0	1.0	1.0	0.0
자동차	-0.0	0.0	-0.0	-0.0	-0.0	1.0	-0.0	0.0	0.0	0.0	...	-0.0	-0.0	0.0	0.0	0.0	-0.0	-0.0	0.0	0.0	0.0
만이	0.0	0.0	-0.0	-0.0	0.0	-0.0	1.0	-0.0	-0.0	-0.0	...	0.0	-0.0	0.0	-0.0	0.0	-0.0	-0.0	0.0	1.0	-0.0
민주화	-0.0	0.0	0.0	0.0	-0.0	-0.0	0.0	0.0	-0.0	0.0	...	0.0	-0.0	0.0	-0.0	-0.0	-0.0	-0.0	1.0	0.0	-0.0
신기	0.0	-0.0	-0.0	-0.0	0.0	-0.0	0.0	-0.0	0.0	0.0	...	0.0	-0.0	0.0	0.0	-0.0	-0.0	-0.0	-0.0	-0.0	-0.0
여건	0.0	0.0	0.0	-0.0	0.0	-0.0	-0.0	-0.0	0.0	0.0	...	0.0	0.0	-0.0	-0.0	0.0	1.0	-0.0	0.0	-0.0	-0.0
시베리아	0.0	0.0	0.0	0.0	0.0	-0.0	0.0	-0.0	-0.0	-0.0	...	-0.0	0.0	0.0	0.0	-0.0	0.0	-0.0	0.0	0.0	-0.0
호통	-0.0	0.0	0.0	0.0	1.0	-0.0	0.0	-0.0	0.0	-0.0	...	0.0	0.0	-0.0	0.0	-0.0	0.0	-0.0	1.0	0.0	-0.0
시스	0.0	0.0	-0.0	-0.0	0.0	-0.0	-0.0	-0.0	-0.0	0.0	...	0.0	0.0	-0.0	1.0	0.0	-0.0	-0.0	0.0	-0.0	-0.0
호치민	0.0	0.0	0.0	0.0	0.0	-0.0	0.0	-0.0	-0.0	-0.0	...	-0.0	0.0	0.0	-0.0	0.0	-0.0	-0.0	0.0	-0.0	-0.0
교통부	1.0	0.0	0.0	-0.0	0.0	-0.0	0.0	0.0	-0.0	-0.0	...	-0.0	0.0	0.0	-0.0	0.0	-0.0	0.0	-0.0	0.0	0.0
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
부끄러움	0.0	0.0	0.0	0.0	1.0	-0.0	0.0	-0.0	-0.0	0.0	...	0.0	-0.0	0.0	0.0	1.0	0.0	-0.0	1.0	0.0	0.0
가능	-0.0	0.0	-0.0	-0.0	0.0	-0.0	-0.0	0.0	1.0	1.0	...	-0.0	0.0	-0.0	0.0	0.0	-0.0	-0.0	0.0	1.0	0.0
벤처	1.0	0.0	0.0	-0.0	0.0	-0.0	0.0	0.0	-0.0	-0.0	...	-0.0	0.0	0.0	-0.0	-0.0	-0.0	-0.0	0.0	-0.0	0.0
어제	0.0	-0.0	0.0	-0.0	0.0	-0.0	-0.0	0.0	-0.0	1.0	...	0.0	0.0	1.0	0.0	0.0	-0.0	-0.0	0.0	0.0	0.0
칭호	-0.0	0.0	-0.0	-0.0	0.0	-0.0	0.0	0.0	-0.0	-0.0	...	0.0	0.0	-0.0	0.0	0.0	-0.0	-0.0	0.0	0.0	-0.0
아이러니	0.0	-0.0	0.0	-0.0	0.0	-0.0	-0.0	-0.0	-0.0	0.0	...	-0.0	-0.0	0.0	-0.0	0.0	-0.0	-0.0	-0.0	-0.0	0.0
기준	1.0	-0.0	0.0	-0.0	0.0	-0.0	-0.0	-0.0	-0.0	0.0	...	-0.0	-0.0	0.0	-0.0	0.0	-0.0	-0.0	0.0	1.0	0.0
개입	0.0	0.0	1.0	-0.0	0.0	-0.0	-0.0	-0.0	-0.0	1.0	...	-0.0	0.0	0.0	-0.0	0.0	0.0	-0.0	0.0	0.0	0.0
있음	-0.0	-0.0	-0.0	1.0	0.0	-0.0	-0.0	-0.0	-0.0	-0.0	...	-0.0	-0.0	-0.0	0.0	0.0	-0.0	-0.0	0.0	0.0	0.0
매진	0.0	0.0	0.0	-0.0	1.0	-0.0	0.0	-0.0	1.0	0.0	...	0.0	0.0	-0.0	0.0	1.0	1.0	-0.0	0.0	-0.0	0.0
소란	0.0	-0.0	-0.0	0.0	1.0	-0.0	0.0	-0.0	0.0	0.0	...	-0.0	-0.0	-0.0	0.0	0.0	0.0	-0.0	-0.0	0.0	0.0
아시아	0.0	-0.0	-0.0	-0.0	0.0	1.0	0.0	-0.0	0.0	0.0	...	0.0	-0.0	0.0	0.0	0.0	-0.0	-0.0	0.0	0.0	0.0
확립	-0.0	0.0	-0.0	0.0	0.0	-0.0	-0.0	0.0	-0.0	-0.0	...	0.0	0.0	-0.0	-0.0	-0.0	-0.0	-0.0	0.0	0.0	0.0
균형	-0.0	0.0	-0.0	-0.0	-0.0	-0.0	0.0	0.0	-0.0	-0.0	...	0.0	0.0	0.0	0.0	-0.0	-0.0	1.0	-0.0	0.0	0.0
이달	-0.0	-0.0	0.0	-0.0	0.0	-0.0	-0.0	-0.0	0.0	-0.0	...	0.0	0.0	0.0	-0.0	0.0	0.0	-0.0	0.0	0.0	-0.0
의뢰	0.0	0.0	1.0	-0.0	0.0	-0.0	0.0	-0.0	-0.0	-0.0	...	0.0	0.0	1.0	1.0	0.0	0.0	-0.0	0.0	0.0	0.0
변화	-0.0	-0.0	0.0	-0.0	0.0	-0.0	-0.0	-0.0	0.0	-0.0	...	0.0	-0.0	0.0	0.0	0.0	0.0	-0.0	0.0	0.0	1.0
혼자	0.0	0.0	-0.0	-0.0	-0.0	-0.0	0.0	-0.0	-0.0	-0.0	...	0.0	0.0	-0.0	1.0	0.0	0.0	-0.0	0.0	-0.0	0.0
박지원	-0.0	0.0	0.0	0.0	-0.0	-0.0	0.0	0.0	-0.0	0.0	...	0.0	0.0	-0.0	0.0	0.0	0.0	-0.0	1.0	0.0	-0.0
부분적	-0.0	-0.0	-0.0	1.0	0.0	-0.0	-0.0	-0.0	-0.0	-0.0	...	-0.0	-0.0	-0.0	0.0	0.0	-0.0	-0.0	0.0	0.0	0.0
핵심	-0.0	0.0	-0.0	-0.0	0.0	1.0	0.0	0.0	1.0	1.0	...	0.0	0.0	-0.0	0.0	0.0	-0.0	1.0	0.0	0.0	0.0
이제	-0.0	-0.0	-0.0	0.0	0.0	-0.0	-0.0	-0.0	0.0	-0.0	...	-0.0	0.0	0.0	-0.0	0.0	0.0	-0.0	0.0	0.0	-0.0
배제	-0.0	-0.0	-0.0	-0.0	0.0	-0.0	0.0	-0.0	-0.0	-0.0	...	1.0	-0.0	0.0	0.0	0.0	-0.0	-0.0	-0.0	-0.0	-0.0
감정	0.0	-0.0	-0.0	0.0	0.0	-0.0	0.0	-0.0	-0.0	0.0	...	-0.0	0.0	0.0	-0.0	0.0	-0.0	-0.0	0.0	-0.0	-0.0
비판적	1.0	0.0	0.0	-0.0	0.0	-0.0	0.0	0.0	-0.0	-0.0	...	-0.0	0.0	0.0	-0.0	-0.0	-0.0	-0.0	0.0	-0.0	0.0
문체부	1.0	0.0	0.0	-0.0	0.0	-0.0	0.0	0.0	-0.0	-0.0	...	-0.0	0.0	0.0	-0.0	-0.0	-0.0	-0.0	0.0	-0.0	0.0
일방적	0.0	0.0	-0.0	-0.0	0.0	-0.0	1.0	-0.0	-0.0	-0.0	...	0.0	0.0	0.0	-0.0	0.0	-0.0	-0.0	0.0	-0.0	0.0
메시지	0.0	-0.0	0.0	-0.0	0.0	-0.0	0.0	-0.0	-0.0	0.0	...	1.0	-0.0	0.0	-0.0	0.0	-0.0	-0.0	0.0	0.0	-0.0
보험	0.0	1.0	0.0	-0.0	1.0	-0.0	0.0	-0.0	0.0	-0.0	...	-0.0	-0.0	0.0	0.0	0.0	0.0	-0.0	-0.0	-0.0	0.0
현종	-0.0	-0.0	-0.0	1.0	0.0	-0.0	-0.0	-0.0	-0.0	-0.0	...	-0.0	-0.0	-0.0	0.0	0.0	-0.0	-0.0	0.0	0.0	0.0

2979 rows × 40 columns

In [212]:

U.shape

Out[212]:

(2979, 40)

In [214]:

pd.DataFrame(U.dot(_sigma),index=coldata)

Out[214]:

	0	1	2	3	4	5	6	7	8	9	...	30	31	32	33	34	35	36	37	38	39
발언	2.369581	-0.836695	-0.144384	0.813850	-1.161821	0.162323	-0.100678	0.091714	-0.049772	0.148860	...	0.076590	0.459355	0.066907	-0.013506	0.001914	0.221040	0.100225	-0.362063	-0.728112	0.331359
30	1.232922	0.046106	0.403684	0.361928	0.610832	-0.143363	0.693665	0.565884	0.170358	0.525898	...	0.367807	-0.424507	-0.014371	-0.421198	0.165903	0.086279	0.013750	0.119869	-0.260407	-0.595102
포스	0.197766	0.577285	0.536741	-0.425677	-0.366042	-0.053780	-0.019975	-0.007174	-0.004273	-0.089186	...	-0.023613	0.012388	-0.020455	0.007404	0.010441	-0.000078	0.011467	-0.008349	-0.006670	0.002912
절전	0.156872	0.066606	0.141700	0.041088	0.251585	0.107481	-0.346418	0.105462	-0.241583	0.056318	...	0.005409	-0.037925	-0.010387	-0.031392	-0.008988	-0.034426	0.017780	0.001282	0.032712	-0.006153
두고	0.274984	-0.061751	0.049259	0.090926	0.132552	-0.102541	0.059663	0.153236	0.008006	-0.169971	...	0.079298	0.128989	-0.016994	0.011060	0.100688	0.007199	0.065299	0.023569	0.023708	-0.027495
분산	0.239403	0.607510	-0.737459	-0.079645	0.104458	-0.004554	0.010982	-0.006272	0.000990	-0.048650	...	-0.015430	0.009186	-0.000426	0.027339	0.000319	-0.000260	-0.008391	0.011644	0.008795	0.000428
촉구	1.218676	-0.186607	-0.005394	0.705432	-0.459797	-0.185152	0.533796	0.166890	-0.249799	0.670231	...	-0.015529	0.407165	0.073284	0.280464	-0.034711	-0.007589	0.186745	0.783091	-0.242910	0.203356
강물	0.197766	0.577285	0.536741	-0.425677	-0.366042	-0.053780	-0.019975	-0.007174	-0.004273	-0.089186	...	-0.023613	0.012388	-0.020455	0.007404	0.010441	-0.000078	0.011467	-0.008349	-0.006670	0.002912
평가	1.062935	0.561659	0.895490	-0.358161	0.318224	-0.063948	-0.631898	0.561459	0.145027	-0.047984	...	0.181124	0.236198	0.242852	0.338859	-0.214197	-0.026540	-0.052922	-0.085309	0.016330	-0.001337
사이트	0.430040	-0.270124	-0.021477	-0.173082	0.159062	-0.600739	-0.118443	-0.414830	0.116138	-0.262634	...	0.100191	0.121265	-0.157790	0.111194	0.428429	0.302030	-0.436008	-0.057473	-0.001548	0.008746
나발	0.197766	0.577285	0.536741	-0.425677	-0.366042	-0.053780	-0.019975	-0.007174	-0.004273	-0.089186	...	-0.023613	0.012388	-0.020455	0.007404	0.010441	-0.000078	0.011467	-0.008349	-0.006670	0.002912
일훈	0.179643	-0.036318	0.028300	0.065591	0.018432	-0.207722	-0.086565	-0.090929	-0.288729	0.066226	...	0.040127	0.021626	0.025197	-0.034516	0.015118	0.034850	0.029041	0.022119	-0.013406	-0.046313
참사	0.316409	-0.104357	0.010679	0.058574	-0.007663	-0.455446	-0.079217	0.208707	-0.210603	0.409731	...	0.103416	0.109178	-0.014763	0.120089	-0.142586	0.099503	0.006561	-0.087782	0.058818	-0.071663
부업	0.177889	-0.211081	-0.075403	-0.313003	0.016532	0.067352	0.014599	0.048049	-0.090268	0.042147	...	-0.412176	0.108497	0.199895	0.074994	-0.062623	0.187080	0.099474	0.028052	-0.038459	0.003520
3000	0.146436	-0.177308	-0.064159	-0.267153	0.032358	0.067886	-0.008218	0.046320	-0.092291	-0.009298	...	0.328133	-0.099486	-0.019106	0.042915	0.227396	0.334372	-0.333997	-0.020548	-0.021204	0.020148
강경	0.362483	-0.063681	-0.015307	0.398809	-0.460040	0.153092	-0.137744	0.016463	-0.142026	0.056521	...	-0.016167	0.464930	-0.088088	0.066577	0.034248	-0.059507	-0.008024	0.229384	0.018940	-0.032632
경원	1.995896	-0.365314	-0.016019	1.401452	-1.140363	-0.360019	-0.397663	-0.643030	-0.283948	0.354143	...	0.390465	-0.111428	0.148737	0.000038	0.563293	-0.101973	0.086519	0.375483	0.328855	-0.240790
말씀	0.742564	0.950088	-0.251962	-0.805080	-0.238978	-0.065928	0.092798	0.037636	-0.166707	-0.137240	...	-0.398463	-0.052499	0.133233	0.099505	0.127384	0.219081	0.119312	-0.170996	-0.043838	-0.033214
보안법	0.205547	-0.068561	0.031243	0.098642	0.089656	-0.594459	-0.148476	-0.552418	0.256667	-0.233459	...	-0.005443	0.024705	0.008247	-0.005403	0.015003	0.014114	-0.009206	-0.010712	-0.007240	0.003638
채널	3.088727	-0.969424	0.316349	-0.089523	0.874423	0.204035	-0.486886	-0.131872	0.917133	-0.280083	...	0.131316	0.351365	-0.867931	0.124607	0.086200	0.063269	-0.148243	0.507811	-0.001623	-0.566629
자동차	0.197766	0.577285	0.536741	-0.425677	-0.366042	-0.053780	-0.019975	-0.007174	-0.004273	-0.089186	...	-0.023613	0.012388	-0.020455	0.007404	0.010441	-0.000078	0.011467	-0.008349	-0.006670	0.002912
만이	0.710152	-0.047355	0.017757	0.596906	-0.545343	0.189023	-0.452087	0.382782	0.768650	0.632477	...	-0.115039	0.507876	-0.081620	-0.045790	0.129570	-0.072574	-0.025557	0.184233	0.034024	-0.019336
민주화	0.222092	-0.294330	-0.135534	-0.482251	-0.051334	0.190610	-0.087395	-0.192593	0.154683	0.038338	...	0.026472	0.004324	0.071936	-0.008209	0.081331	0.157660	0.042345	0.006350	-0.029562	-0.012229
신기	0.263535	-0.033261	0.031035	0.157818	-0.163540	-0.042264	0.022192	-0.027389	-0.124940	0.032316	...	0.103256	0.040767	-0.098607	0.043941	0.195717	0.010641	0.011477	0.563328	-0.000051	-0.085494
여건	0.123399	0.071824	0.110044	0.019551	0.180713	0.094155	0.328738	-0.112077	-0.007991	0.167470	...	0.158870	0.166873	0.550972	-0.554074	-0.123399	-0.055217	-0.012488	0.000057	0.031717	0.017503
시베리아	0.156872	0.066606	0.141700	0.041088	0.251585	0.107481	-0.346418	0.105462	-0.241583	0.056318	...	0.005409	-0.037925	-0.010387	-0.031392	-0.008988	-0.034426	0.017780	0.001282	0.032712	-0.006153
호통	0.545251	-0.670008	-0.284991	-1.022469	-0.056182	0.292303	-0.113777	-0.173441	-0.000505	0.105931	...	-0.352639	0.150712	0.055726	0.012635	-0.253473	-0.258008	-0.369652	0.028466	0.011099	0.033373
시스	0.525090	-0.049418	0.099217	0.310381	-0.003447	0.081755	-0.234558	0.694147	0.477752	-0.343681	...	-0.123197	0.123392	0.091550	-0.194787	0.074263	0.060611	-0.036826	0.042755	-0.027192	0.036038
호치민	0.156872	0.066606	0.141700	0.041088	0.251585	0.107481	-0.346418	0.105462	-0.241583	0.056318	...	0.005409	-0.037925	-0.010387	-0.031392	-0.008988	-0.034426	0.017780	0.001282	0.032712	-0.006153
교통부	0.144714	-0.055486	0.067303	-0.073055	0.026855	-0.097888	0.050656	-0.000477	-0.112808	0.154227	...	-0.031898	-0.019856	-0.144409	0.047383	0.052654	0.044239	-0.018052	0.029880	-0.026980	0.013759
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
부끄러움	0.525449	-0.660899	-0.277153	-1.015897	0.011664	0.309758	-0.039148	-0.180731	0.031667	0.063505	...	-0.302967	0.076489	0.005942	-0.082149	-0.050345	0.007979	0.847159	0.022324	-0.017174	-0.039672
가능	0.672915	0.118303	0.252102	0.357578	0.371721	0.098156	0.287876	0.255191	0.887020	0.809933	...	-0.085434	0.117597	-0.544208	0.105501	-0.346018	0.400869	-0.054870	-0.055393	0.209869	-0.045066
벤처	0.144714	-0.055486	0.067303	-0.073055	0.026855	-0.097888	0.050656	-0.000477	-0.112808	0.154227	...	-0.031898	-0.019856	-0.144409	0.047383	0.052654	0.044239	-0.018052	0.029880	-0.026980	0.013759
어제	0.471845	-0.127601	0.019384	0.112517	0.051449	-0.628583	0.078015	0.325739	-0.342193	0.409083	...	0.153711	0.238796	-0.289976	-0.014860	-0.664807	0.509221	-0.003573	-0.095797	0.276327	-0.126526
칭호	0.130440	0.022425	0.087373	0.003087	0.214820	-0.116077	-0.204506	0.529317	0.477387	0.086242	...	-0.078207	0.018645	0.021820	-0.035588	0.034603	0.000164	0.009421	-0.004965	0.011169	0.007691
아이러니	0.111598	-0.039413	0.027120	0.003489	-0.010765	-0.130006	-0.014811	0.056837	-0.041237	0.067703	...	0.266914	0.124168	0.282405	0.451492	-0.334273	-0.127715	-0.029909	-0.095920	0.066272	-0.091411
기준	0.473540	-0.100998	0.040113	0.125444	-0.284033	-0.075886	-0.073992	-0.106638	0.279245	0.711643	...	0.214351	0.128613	0.122644	0.422096	-0.220900	-0.096707	-0.074916	-0.106226	0.043207	-0.072047
개입	0.593166	-0.045645	0.134718	0.159660	0.306130	-0.331935	-0.051707	0.425573	-0.623508	0.196482	...	-0.000446	0.002466	0.265481	0.296231	-0.142671	0.102223	0.225198	0.056329	0.546281	-0.196162
있음	0.158328	0.084021	0.149270	0.105622	0.387031	0.361035	-0.430081	-0.199925	-0.088406	-0.244106	...	0.028519	0.036895	0.036162	-0.043585	0.033306	0.001253	0.018745	-0.010929	0.001080	0.029669
매진	0.596566	-0.169524	0.178759	-0.408556	0.641623	0.448665	0.821972	-0.328376	-0.023074	0.427263	...	-0.207428	0.184071	0.209515	-0.275197	-0.174195	-0.200679	0.765124	0.013805	0.021381	-0.013440
소란	0.177889	-0.211081	-0.075403	-0.313003	0.016532	0.067352	0.014599	0.048049	-0.090268	0.042147	...	-0.412176	0.108497	0.199895	0.074994	-0.062623	0.187080	0.099474	0.028052	-0.038459	0.003520
아시아	0.461301	0.544024	0.567776	-0.267858	-0.529581	-0.096044	0.002217	-0.034563	-0.129213	-0.056869	...	0.079643	0.053155	-0.119063	0.051345	0.206158	0.010562	0.022944	0.554979	-0.006721	-0.082582
확립	0.162838	-0.091667	0.028184	-0.025076	0.024234	-0.026400	0.133683	0.029569	0.290536	-0.251457	...	-0.013472	0.083443	-0.014298	0.027256	-0.034940	-0.073571	0.011902	-0.012061	0.033229	0.008907
균형	0.419046	0.571192	-0.709159	-0.014054	0.122890	-0.212276	-0.075583	-0.097201	-0.287739	0.017576	...	0.024697	0.030811	0.024771	-0.007177	0.015437	0.034590	0.020650	0.033763	-0.004611	-0.045885
이달	0.113932	-0.003755	0.060966	0.011110	0.086732	0.022500	0.241971	0.056626	0.091929	-0.055611	...	0.072604	0.034421	-0.098965	-0.090965	0.053405	-0.023499	-0.031495	-0.002136	0.039804	0.002156
의뢰	0.699156	-0.336653	-0.045144	-0.045728	0.046333	-0.108698	-0.134518	0.196913	-0.124050	0.014475	...	-0.036544	-0.022197	0.099950	0.150080	-0.214865	-0.358695	-0.455873	0.040130	-0.270180	-0.433357
변화	0.266071	-0.039293	0.069045	0.070967	0.084302	0.041202	0.261547	0.054417	0.074815	-0.061984	...	0.105890	0.150717	-0.075866	-0.027782	0.080440	0.055592	-0.006594	-0.001841	0.065624	-0.022228
혼자	0.197166	-0.015012	0.029416	0.112143	-0.021130	0.024773	-0.182621	0.012911	-0.080117	0.026364	...	0.034826	0.061218	0.054531	-0.077955	0.018379	0.066178	-0.037166	0.003066	-0.010343	0.029035
박지원	0.222092	-0.294330	-0.135534	-0.482251	-0.051334	0.190610	-0.087395	-0.192593	0.154683	0.038338	...	0.026472	0.004324	0.071936	-0.008209	0.081331	0.157660	0.042345	0.006350	-0.029562	-0.012229
부분적	0.158328	0.084021	0.149270	0.105622	0.387031	0.361035	-0.430081	-0.199925	-0.088406	-0.244106	...	0.028519	0.036895	0.036162	-0.043585	0.033306	0.001253	0.018745	-0.010929	0.001080	0.029669
핵심	1.104896	1.158787	0.074804	-0.305325	0.238107	-0.230230	0.640343	-0.185935	-0.025133	-0.089090	...	0.001050	0.201294	-0.560658	0.245351	-0.450402	0.374877	0.006683	0.003112	0.216733	-0.092429
이제	0.317579	-0.037037	0.056990	0.208030	-0.054266	0.211192	0.329048	-0.041612	-0.118889	-0.083444	...	-0.078451	0.111508	-0.154888	-0.015114	0.040884	-0.000271	-0.002623	-0.015915	0.022769	0.002153
배제	0.241414	-0.010720	0.086217	0.053568	0.275454	-0.137406	-0.145999	0.374430	0.125597	-0.020558	...	0.226042	0.069864	-0.091348	-0.032288	0.028683	-0.130606	0.049429	0.039650	-0.034989	-0.008817
감정	0.203647	-0.033283	-0.003976	0.196920	-0.140998	0.188692	0.087077	-0.098238	-0.210818	-0.027833	...	-0.151055	0.077088	-0.055923	0.075851	-0.012521	0.023228	0.028873	-0.013779	-0.017035	-0.000003
비판적	0.144714	-0.055486	0.067303	-0.073055	0.026855	-0.097888	0.050656	-0.000477	-0.112808	0.154227	...	-0.031898	-0.019856	-0.144409	0.047383	0.052654	0.044239	-0.018052	0.029880	-0.026980	0.013759
문체부	0.144714	-0.055486	0.067303	-0.073055	0.026855	-0.097888	0.050656	-0.000477	-0.112808	0.154227	...	-0.031898	-0.019856	-0.144409	0.047383	0.052654	0.044239	-0.018052	0.029880	-0.026980	0.013759
일방적	0.226454	-0.054046	-0.022182	0.254235	-0.290426	0.120410	-0.072744	0.046885	-0.090241	-0.017346	...	-0.066666	0.241592	-0.035261	0.012403	0.017777	-0.037808	-0.002739	-0.536287	0.011487	0.012788
메시지	0.391314	0.000980	0.210000	0.075646	0.375803	-0.143768	-0.321142	0.317744	-0.257700	-0.039576	...	0.318334	0.098935	0.231925	0.367976	-0.269608	-0.234033	0.028268	-0.071363	0.096871	-0.100675
보험	0.255946	-0.235336	-0.063964	-0.317574	0.053579	-0.006814	0.052849	0.139317	-0.138506	0.022270	...	-0.634675	0.304543	0.052963	0.148677	0.123407	0.140623	0.006669	0.001840	-0.011563	-0.011519
현종	0.158328	0.084021	0.149270	0.105622	0.387031	0.361035	-0.430081	-0.199925	-0.088406	-0.244106	...	0.028519	0.036895	0.036162	-0.043585	0.033306	0.001253	0.018745	-0.010929	0.001080	0.029669

2979 rows × 40 columns

In [215]:

_sigma = np.diag(sigma[:2])
_sigma

Out[215]:

array([[35.64359912,  0.        ],
       [ 0.        , 19.45585383]])

In [216]:

U[:,:2].dot(_sigma)

Out[216]:

array([[ 2.36958057e+00, -8.36695147e-01],
       [ 1.23292230e+00,  4.61061318e-02],
       [ 1.97766076e-01,  5.77285049e-01],
       ...,
       [ 3.91314446e-01,  9.79699246e-04],
       [ 2.55945976e-01, -2.35336430e-01],
       [ 1.58327760e-01,  8.40213821e-02]])

In [218]:

# K = 2
pd.DataFrame(U[:,:2].dot(_sigma.dot(Vt[:2,:])), index=coldata)

Out[218]:

	0	1	2	3	4	5	6	7	8	9	...	30	31	32	33	34	35	36	37	38	39
발언	0.389336	0.205256	0.332963	0.304870	0.598133	-0.014389	0.581822	0.308318	0.297605	0.387768	...	0.313022	0.301451	0.284804	0.479761	0.427403	0.232309	0.058985	0.772529	0.519844	0.390240
30	0.175862	0.095120	0.148449	0.199080	0.209591	0.270447	0.276709	0.169237	0.215135	0.190569	...	0.150249	0.150994	0.133113	0.242398	0.147524	0.155453	0.323176	0.260251	0.267545	0.185937
포스	-0.003412	0.001435	-0.005539	0.079816	-0.086674	0.372369	0.013585	0.037581	0.105871	0.017321	...	0.009162	0.017290	0.003509	0.030327	-0.064947	0.065867	0.398052	-0.125990	0.039440	0.009572
절전	0.019006	0.010629	0.015760	0.030434	0.013847	0.069474	0.031925	0.022645	0.034979	0.022835	...	0.017525	0.018473	0.015035	0.029930	0.009326	0.024142	0.078019	0.015236	0.033671	0.021499
두고	0.043220	0.022962	0.036819	0.038349	0.061951	0.018735	0.065609	0.036427	0.038962	0.044178	...	0.035399	0.034553	0.031944	0.055144	0.044103	0.029498	0.028318	0.079247	0.060111	0.044030
분산	0.000937	0.003952	-0.002002	0.088948	-0.085647	0.398052	0.021381	0.043821	0.116726	0.023091	...	0.013485	0.022040	0.007106	0.038082	-0.064423	0.073176	0.426383	-0.125639	0.048300	0.014833
촉구	0.186713	0.099652	0.158692	0.177271	0.256178	0.133287	0.286060	0.163030	0.183575	0.193764	...	0.154599	0.152073	0.138844	0.243083	0.181920	0.136981	0.178390	0.325582	0.265870	0.192040
강물	-0.003412	0.001435	-0.005539	0.079816	-0.086674	0.372369	0.013585	0.037581	0.105871	0.017321	...	0.009162	0.017290	0.003509	0.030327	-0.064947	0.065867	0.398052	-0.125990	0.039440	0.009572
평가	0.122657	0.069346	0.101102	0.215483	0.070529	0.534450	0.210351	0.155459	0.250828	0.152163	...	0.115853	0.123829	0.098419	0.201143	0.046033	0.171506	0.595684	0.070756	0.227474	0.141754
사이트	0.077221	0.040119	0.066519	0.045391	0.133517	-0.070891	0.111983	0.053789	0.039200	0.073123	...	0.059909	0.056147	0.055390	0.088844	0.095957	0.033665	-0.061150	0.175014	0.095064	0.075026
나발	-0.003412	0.001435	-0.005539	0.079816	-0.086674	0.372369	0.013585	0.037581	0.105871	0.017321	...	0.009162	0.017290	0.003509	0.030327	-0.064947	0.065867	0.398052	-0.125990	0.039440	0.009572
일훈	0.028012	0.014903	0.023846	0.025391	0.039623	0.014561	0.042644	0.023871	0.025957	0.028767	...	0.023020	0.022524	0.020743	0.035965	0.028187	0.019559	0.020943	0.050587	0.039245	0.028621
참사	0.051579	0.027229	0.044081	0.041328	0.078313	0.002331	0.077292	0.041304	0.040662	0.051607	...	0.041605	0.040163	0.037799	0.063952	0.055926	0.031549	0.012352	0.100987	0.069370	0.051847
부업	0.037455	0.019005	0.032632	0.010429	0.076200	-0.086674	0.051692	0.020431	0.003775	0.032557	...	0.027386	0.024434	0.026023	0.038242	0.055140	0.006791	-0.085647	0.101635	0.039930	0.034565
3000	0.031029	0.015731	0.027045	0.008287	0.063476	-0.073397	0.042744	0.016754	0.002663	0.026883	...	0.022637	0.020157	0.021533	0.031534	0.045942	0.005335	-0.072659	0.084709	0.032891	0.028580
강경	0.055990	0.029839	0.047622	0.052041	0.077924	0.034925	0.085528	0.048342	0.053579	0.057823	...	0.046198	0.045332	0.041554	0.072425	0.055382	0.040156	0.048093	0.099248	0.079130	0.057411
경원	0.309103	0.164654	0.262973	0.285312	0.432159	0.183830	0.471723	0.265911	0.293177	0.318727	...	0.254761	0.249784	0.229263	0.399007	0.307224	0.220053	0.255892	0.550795	0.435794	0.316636
말씀	0.054743	0.034918	0.041906	0.197396	-0.068452	0.695326	0.116809	0.118815	0.245065	0.093337	...	0.066316	0.079724	0.051293	0.132146	-0.054559	0.159871	0.754961	-0.114722	0.155512	0.079209
보안법	0.033550	0.017707	0.028676	0.026783	0.051037	0.001071	0.050253	0.026818	0.026319	0.033543	...	0.027048	0.026100	0.024579	0.041556	0.036450	0.020440	0.007557	0.065830	0.045069	0.033708
채널	0.500771	0.264610	0.427772	0.407579	0.754078	0.051211	0.751849	0.404108	0.403102	0.502634	...	0.404845	0.391465	0.367445	0.623544	0.538271	0.311518	0.150517	0.971312	0.676873	0.504368
자동차	-0.003412	0.001435	-0.005539	0.079816	-0.086674	0.372369	0.013585	0.037581	0.105871	0.017321	...	0.009162	0.017290	0.003509	0.030327	-0.064947	0.065867	0.398052	-0.125990	0.039440	0.009572
만이	0.105396	0.056581	0.089312	0.108458	0.136324	0.113107	0.163377	0.096126	0.114661	0.111484	...	0.088480	0.087870	0.078986	0.140729	0.096465	0.084231	0.141245	0.171657	0.154554	0.109725
민주화	0.048471	0.024475	0.042327	0.010433	0.101635	-0.125990	0.066201	0.024944	0.000857	0.041363	...	0.034998	0.030880	0.033454	0.048207	0.073630	0.006266	-0.125639	0.135955	0.050040	0.044249
신기	0.039983	0.021377	0.033951	0.038930	0.053901	0.032917	0.061476	0.035385	0.040586	0.041736	...	0.033246	0.032799	0.029803	0.052459	0.038237	0.030131	0.042885	0.068319	0.057450	0.041276
여건	0.013872	0.007890	0.011396	0.025572	0.006791	0.065867	0.024062	0.018169	0.029948	0.017511	...	0.013276	0.014295	0.011219	0.023252	0.004315	0.020386	0.073176	0.006266	0.026368	0.016221
시베리아	0.019006	0.010629	0.015760	0.030434	0.013847	0.069474	0.031925	0.022645	0.034979	0.022835	...	0.017525	0.018473	0.015035	0.029930	0.009326	0.024142	0.078019	0.015236	0.033671	0.021499
호통	0.116081	0.058812	0.101208	0.030033	0.238420	-0.278953	0.159686	0.062203	0.008690	0.100326	...	0.084544	0.075172	0.080485	0.117563	0.172590	0.019161	-0.276501	0.318299	0.122531	0.106765
시스	0.078730	0.042186	0.066779	0.078984	0.103839	0.075317	0.121580	0.070812	0.082977	0.082767	...	0.065800	0.065147	0.058854	0.104272	0.073566	0.061246	0.095686	0.131163	0.114366	0.081643
호치민	0.019006	0.010629	0.015760	0.030434	0.013847	0.069474	0.031925	0.022645	0.034979	0.022835	...	0.017525	0.018473	0.015035	0.029930	0.009326	0.024142	0.078019	0.015236	0.033671	0.021499
교통부	0.024021	0.012642	0.020561	0.018250	0.037455	-0.003412	0.035770	0.018749	0.017626	0.023784	...	0.019232	0.018463	0.017531	0.029366	0.026784	0.013872	0.000937	0.048471	0.031774	0.023988
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
부끄러움	0.112710	0.057045	0.098316	0.027664	0.232975	-0.277611	0.154709	0.059665	0.006468	0.097036	...	0.081873	0.072627	0.078038	0.113522	0.168689	0.017372	-0.275709	0.311221	0.118173	0.103428
가능	0.090816	0.049656	0.076225	0.116481	0.094733	0.201374	0.145991	0.094073	0.129081	0.101846	...	0.079563	0.081278	0.069735	0.130900	0.066035	0.091534	0.232968	0.114629	0.145455	0.098172
벤처	0.024021	0.012642	0.020561	0.018250	0.037455	-0.003412	0.035770	0.018749	0.017626	0.023784	...	0.019232	0.018463	0.017531	0.029366	0.026784	0.013872	0.000937	0.048471	0.031774	0.023988
어제	0.075363	0.039926	0.064293	0.063985	0.110870	0.019653	0.113748	0.062108	0.064145	0.076308	...	0.061308	0.059552	0.055491	0.094947	0.079042	0.049060	0.035442	0.142350	0.103277	0.076321
칭호	0.017632	0.009638	0.014802	0.022537	0.018470	0.038742	0.028327	0.018226	0.024958	0.019754	...	0.015436	0.015761	0.013534	0.025382	0.012879	0.017707	0.044851	0.022369	0.028199	0.019048
아이러니	0.018337	0.009667	0.015682	0.014357	0.028171	-0.000682	0.027402	0.014520	0.014015	0.018262	...	0.014742	0.014197	0.013413	0.022595	0.020130	0.010940	0.002773	0.036385	0.024483	0.018379
기준	0.074132	0.039413	0.063130	0.066489	0.105556	0.035345	0.112694	0.062827	0.067764	0.075953	...	0.060819	0.059437	0.054843	0.094882	0.075118	0.051180	0.052010	0.134896	0.103482	0.075633
개입	0.088372	0.047408	0.074913	0.090079	0.115153	0.090958	0.136792	0.080179	0.095009	0.093261	...	0.074064	0.073469	0.066165	0.117637	0.081521	0.069918	0.114276	0.145172	0.129131	0.091866
있음	0.018250	0.010321	0.015041	0.032127	0.010429	0.079816	0.031313	0.023163	0.037407	0.022657	...	0.017247	0.018440	0.014649	0.029956	0.006801	0.025572	0.088948	0.010433	0.033881	0.021102
매진	0.095737	0.050678	0.081709	0.080209	0.141906	0.020117	0.144257	0.078375	0.080074	0.096669	...	0.077729	0.075393	0.070415	0.120167	0.101209	0.061440	0.039832	0.182388	0.130625	0.096786
소란	0.037455	0.019005	0.032632	0.010429	0.076200	-0.086674	0.051692	0.020431	0.003775	0.032557	...	0.027386	0.024434	0.026023	0.038242	0.055140	0.006791	-0.085647	0.101635	0.039930	0.034565
아시아	0.036571	0.022812	0.028412	0.118746	-0.032773	0.405287	0.075061	0.072965	0.146456	0.059057	...	0.042408	0.050089	0.033311	0.082786	-0.026710	0.095998	0.440937	-0.057672	0.096890	0.050848
확립	0.028651	0.014934	0.024641	0.018080	0.048316	-0.020714	0.041830	0.020562	0.016173	0.027442	...	0.022407	0.021131	0.020642	0.033482	0.034684	0.013510	-0.016705	0.063145	0.035932	0.028032
균형	0.028949	0.018855	0.021844	0.114339	-0.046024	0.412614	0.064024	0.067692	0.142683	0.051858	...	0.036505	0.044564	0.027849	0.074047	-0.036236	0.092735	0.447326	-0.075052	0.087545	0.043454
이달	0.016696	0.008984	0.014131	0.017723	0.021060	0.020364	0.026003	0.015492	0.018877	0.017796	...	0.014094	0.014051	0.012552	0.022520	0.014879	0.013789	0.024995	0.026408	0.024772	0.017467
의뢰	0.119857	0.062739	0.102866	0.082410	0.195433	-0.056075	0.176522	0.089327	0.076567	0.116500	...	0.094712	0.090037	0.086844	0.142903	0.140067	0.062095	-0.037140	0.254364	0.153930	0.118333
변화	0.040684	0.021722	0.034572	0.038825	0.055625	0.029937	0.062377	0.035621	0.040261	0.042271	...	0.033715	0.033184	0.030268	0.053050	0.039493	0.030011	0.039828	0.070657	0.058038	0.041876
혼자	0.029366	0.015754	0.024892	0.029956	0.038242	0.030327	0.045460	0.026654	0.031601	0.030996	...	0.024614	0.024419	0.021988	0.039100	0.027072	0.023252	0.038082	0.048207	0.042922	0.030530
박지원	0.048471	0.024475	0.042327	0.010433	0.101635	-0.125990	0.066201	0.024944	0.000857	0.041363	...	0.034998	0.030880	0.033454	0.048207	0.073630	0.006266	-0.125639	0.135955	0.050040	0.044249
부분적	0.018250	0.010321	0.015041	0.032127	0.010429	0.079816	0.031313	0.023163	0.037407	0.022657	...	0.017247	0.018440	0.014649	0.029956	0.006801	0.025572	0.088948	0.010433	0.033881	0.021102
핵심	0.095597	0.058138	0.075482	0.272299	-0.048049	0.887461	0.187581	0.172124	0.332726	0.144806	...	0.105356	0.121725	0.084303	0.200452	-0.041548	0.219572	0.968491	-0.095678	0.232948	0.126917
이제	0.048013	0.025688	0.040757	0.047170	0.064312	0.041425	0.073919	0.042697	0.049290	0.050224	...	0.039984	0.039488	0.035819	0.063172	0.045605	0.036529	0.053529	0.081433	0.069213	0.049632
배제	0.035531	0.019104	0.030084	0.037322	0.045208	0.041555	0.055249	0.032776	0.039652	0.037774	...	0.029937	0.029806	0.026683	0.047760	0.031957	0.029020	0.051283	0.056771	0.052507	0.037109
감정	0.031317	0.016703	0.026626	0.029447	0.043252	0.021061	0.047916	0.027205	0.030414	0.032428	...	0.025889	0.025438	0.023267	0.040652	0.030726	0.022739	0.028534	0.055024	0.044441	0.032166
비판적	0.024021	0.012642	0.020561	0.018250	0.037455	-0.003412	0.035770	0.018749	0.017626	0.023784	...	0.019232	0.018463	0.017531	0.029366	0.026784	0.013872	0.000937	0.048471	0.031774	0.023988
문체부	0.024021	0.012642	0.020561	0.018250	0.037455	-0.003412	0.035770	0.018749	0.017626	0.023784	...	0.019232	0.018463	0.017531	0.029366	0.026784	0.013872	0.000937	0.048471	0.031774	0.023988
일방적	0.035770	0.018987	0.030486	0.031313	0.051692	0.013585	0.054203	0.029940	0.031686	0.036456	...	0.029235	0.028494	0.026407	0.045460	0.036816	0.024062	0.021381	0.066201	0.049522	0.036373
메시지	0.056574	0.030521	0.047819	0.062038	0.069404	0.077954	0.088562	0.053464	0.066572	0.060802	...	0.048045	0.048090	0.042676	0.077139	0.048945	0.048358	0.094277	0.086619	0.084999	0.059499
보험	0.050097	0.025686	0.043430	0.020750	0.095205	-0.085239	0.070679	0.030648	0.013993	0.045254	...	0.037610	0.034324	0.035301	0.053997	0.068705	0.014681	-0.081695	0.126110	0.057034	0.047303
현종	0.018250	0.010321	0.015041	0.032127	0.010429	0.079816	0.031313	0.023163	0.037407	0.022657	...	0.017247	0.018440	0.014649	0.029956	0.006801	0.025572	0.088948	0.010433	0.033881	0.021102

2979 rows × 40 columns

In [220]:

# 원래 
pd.DataFrame(U.dot(np.diag(sigma)).dot(Vt),index=coldata)

Out[220]:

	0	1	2	3	4	5	6	7	8	9	...	30	31	32	33	34	35	36	37	38	39
발언	4.511668e-16	1.142421e-15	1.071848e-15	-1.765938e-15	2.210952e-15	-1.067545e-14	1.000000e+00	-1.165556e-15	-2.292612e-15	1.235156e-15	...	1.382884e-15	7.612421e-16	1.000000e+00	1.000000e+00	3.307632e-15	-1.217122e-15	-9.911487e-15	1.000000e+00	1.000000e+00	1.000000e+00
30	3.336287e-16	1.000000e+00	1.529144e-16	-4.689882e-15	2.082784e-15	-4.170081e-15	-8.098133e-16	1.000000e+00	1.000000e+00	-2.153584e-15	...	-1.284910e-15	1.000000e+00	1.000000e+00	2.341709e-16	1.272994e-15	1.000000e+00	-5.632642e-15	1.457603e-15	-1.259300e-15	-1.814579e-15
포스	-1.639695e-17	2.515408e-16	-6.861209e-17	-2.553020e-16	-6.310523e-16	1.000000e+00	5.844549e-16	-4.850555e-16	-3.787971e-16	-1.694962e-17	...	3.932637e-16	-3.869439e-16	4.402897e-16	-3.042305e-16	-2.638965e-16	-3.757404e-16	8.143798e-18	2.803780e-16	-2.187497e-15	-7.326933e-16
절전	4.597195e-16	2.605088e-17	1.747082e-16	6.190021e-16	2.299841e-16	3.819965e-16	1.053478e-16	-1.318783e-16	1.812797e-15	6.606538e-16	...	2.092256e-16	4.534301e-16	4.135441e-16	1.460791e-15	5.581437e-16	1.355660e-15	3.994348e-16	1.682267e-15	-9.774098e-17	1.812806e-16
두고	-9.180194e-17	-1.326430e-17	4.000906e-16	-6.704289e-16	2.870614e-16	-5.066714e-16	2.815634e-15	-4.790930e-17	-1.967159e-16	5.823311e-16	...	1.000000e+00	-2.566413e-16	4.755388e-16	1.249544e-15	-8.239925e-16	-8.174645e-17	-1.826759e-15	9.388768e-16	-3.698883e-16	1.000000e+00
분산	-2.440442e-16	1.496535e-16	4.972386e-17	3.986480e-17	-4.885358e-16	-6.848354e-16	5.208930e-16	2.819547e-16	-2.020721e-16	3.177013e-17	...	-1.337750e-16	2.098993e-16	-3.612618e-16	5.166253e-16	-7.044050e-16	7.309626e-16	1.000000e+00	-4.136106e-16	5.195663e-17	-6.396306e-16
촉구	5.457131e-16	1.710637e-16	1.000000e+00	1.961934e-16	2.756229e-15	-4.571046e-15	-2.350097e-15	-4.038126e-16	-1.390469e-15	1.000000e+00	...	-9.196288e-16	3.200655e-16	1.497461e-15	-1.115237e-15	1.462282e-15	-3.325721e-16	-3.729321e-15	4.344065e-15	1.000000e+00	6.762754e-16
강물	-2.468931e-17	2.512921e-16	-9.468087e-17	-1.243414e-16	-3.090262e-16	1.000000e+00	-1.002925e-16	-1.741639e-16	7.681702e-16	1.348089e-15	...	6.354412e-16	4.937574e-16	3.883982e-16	1.479050e-15	-1.054803e-16	-7.584244e-16	-1.750137e-15	-1.566978e-16	8.868654e-16	-4.231038e-17
평가	1.000000e+00	1.368110e-16	3.204847e-16	1.000000e+00	1.174513e-15	1.000000e+00	5.640569e-17	-4.087164e-16	-1.651178e-15	-9.184983e-16	...	2.452692e-16	1.000000e+00	9.705906e-16	1.000000e+00	7.212501e-16	-3.409187e-15	-3.183174e-15	1.916508e-15	3.044472e-16	4.516649e-16
사이트	3.929215e-16	1.000000e+00	5.630138e-16	-2.180729e-16	4.098521e-16	-1.660619e-15	7.063678e-16	6.006623e-16	-3.579097e-16	-7.800961e-16	...	5.940034e-16	-1.116837e-15	2.745919e-16	3.315917e-16	7.164963e-16	-1.390471e-15	-3.728440e-15	8.808758e-17	-1.502040e-15	-6.700611e-17
나발	-4.542461e-17	2.577704e-16	-1.012918e-16	-1.020088e-16	-3.116262e-16	1.000000e+00	-1.393816e-16	1.210821e-16	1.051449e-16	6.287759e-17	...	5.953511e-16	5.553037e-16	1.509967e-15	1.160403e-15	6.589474e-16	3.466416e-16	8.615199e-16	-1.705346e-16	-1.762902e-15	2.870787e-15
일훈	3.006819e-16	2.217434e-16	-2.640711e-16	-5.038979e-16	3.987142e-16	-5.382811e-16	-1.101200e-16	2.215327e-16	-6.029598e-16	-8.623856e-17	...	1.160702e-17	1.830631e-15	7.902401e-18	3.914009e-16	2.786844e-16	-1.391746e-16	-1.132388e-15	9.442215e-16	2.189720e-16	5.009731e-16
참사	3.021628e-16	1.140508e-16	8.276296e-16	-1.106820e-15	7.501103e-16	-1.109252e-15	1.352104e-16	2.183515e-16	-5.940927e-16	7.727204e-16	...	9.802820e-16	8.650158e-16	1.000000e+00	-5.618253e-16	2.474275e-15	-4.107559e-16	-9.597439e-16	-8.225012e-16	3.288632e-16	2.120899e-15
부업	2.666319e-16	-1.499422e-16	5.507429e-17	1.656491e-16	1.000000e+00	-9.128753e-16	4.659564e-16	-2.311182e-16	3.378026e-16	1.109353e-16	...	-3.335360e-17	6.550148e-17	-4.654981e-16	-6.514475e-17	-2.170014e-16	6.502927e-18	-1.873559e-15	-2.198138e-15	1.054924e-15	-1.129764e-16
3000	1.945806e-16	6.705649e-17	2.457529e-16	2.233831e-16	-2.335262e-17	-6.564076e-16	2.515107e-16	4.623586e-16	1.094529e-16	-1.293552e-16	...	-7.948876e-16	-2.203896e-16	1.832962e-16	5.945960e-16	1.625308e-16	3.999249e-16	-7.416788e-16	2.989951e-16	3.129241e-17	6.032096e-16
강경	3.979218e-16	2.075387e-16	-4.968640e-16	-4.595904e-16	1.114898e-15	-1.178534e-15	1.000000e+00	1.418369e-16	-1.731612e-16	-4.658473e-16	...	1.443939e-16	3.973923e-16	1.116800e-17	4.762633e-16	5.272900e-16	-6.557216e-16	-1.124734e-15	1.439950e-15	-9.307373e-16	-3.423256e-16
경원	1.289635e-15	8.717351e-16	5.435182e-16	-2.268144e-15	3.274809e-15	-7.706843e-15	1.000000e+00	1.000000e+00	-3.136795e-15	-1.089577e-15	...	2.001833e-15	7.736920e-16	4.064411e-16	1.000000e+00	2.635201e-15	-1.601846e-15	-9.738861e-15	3.964623e-15	1.000000e+00	1.000000e+00
말씀	1.633328e-16	-1.102160e-16	-3.678172e-16	-1.744641e-16	1.000000e+00	1.000000e+00	6.472868e-16	-6.478147e-16	5.073468e-16	4.828009e-16	...	5.303748e-16	-7.048546e-16	7.724059e-16	9.321540e-17	2.041717e-15	-9.455163e-16	1.000000e+00	-9.450553e-16	-2.034927e-16	-2.318517e-17
보안법	1.180645e-16	-7.398370e-18	3.564967e-17	-2.723914e-16	1.059019e-16	-6.846583e-16	3.885914e-16	1.757864e-16	-5.177088e-16	6.983360e-17	...	8.978497e-16	5.215270e-16	-4.413166e-15	1.178671e-15	5.986048e-16	-5.750490e-16	-4.024094e-16	2.214893e-16	-1.642945e-15	1.836392e-17
채널	1.407318e-15	1.000000e+00	1.000000e+00	1.000000e+00	4.630984e-15	-1.086535e-14	-1.233736e-16	-1.194865e-15	1.000000e+00	1.000000e+00	...	1.082688e-15	-9.661867e-16	1.000000e+00	1.000000e+00	1.000000e+00	-2.539260e-15	-1.533968e-14	1.000000e+00	1.000000e+00	2.884800e-15
자동차	-6.313339e-18	2.669006e-16	-8.218223e-17	-1.343520e-16	-2.465707e-16	1.000000e+00	-9.776055e-17	1.360615e-16	6.109589e-17	7.372563e-17	...	-1.134761e-15	-1.129681e-16	6.586643e-16	1.218094e-16	2.513428e-16	-5.075660e-16	-8.593307e-16	7.527663e-16	4.097772e-17	3.614712e-16
만이	5.062875e-17	4.685358e-16	-6.774259e-16	-6.473326e-16	2.024655e-15	-2.315017e-15	1.000000e+00	-3.180736e-16	-9.357807e-16	-7.241269e-16	...	-1.276827e-17	-3.275366e-17	5.773009e-16	-1.284923e-15	1.972824e-15	-1.960978e-16	-1.211241e-15	2.403594e-15	1.000000e+00	-9.307931e-16
민주화	-7.885390e-17	2.659273e-16	1.499327e-16	3.337361e-16	-1.454992e-16	-1.260415e-15	3.386095e-16	5.406702e-17	-8.827522e-17	2.749863e-16	...	4.430796e-16	-3.101933e-16	1.465007e-16	-5.806546e-16	1.820320e-17	-4.209278e-16	-3.018863e-15	1.000000e+00	1.122912e-15	-3.567432e-17
신기	4.187398e-16	-3.464448e-16	-9.676182e-16	-2.753753e-18	2.821835e-16	-8.138597e-16	2.525641e-16	-4.761422e-16	2.221648e-16	1.930931e-16	...	1.304184e-15	-4.709820e-16	9.833192e-16	6.964750e-16	-5.152953e-16	-4.814632e-16	-2.083809e-15	-1.308965e-16	-1.606136e-15	-5.142800e-16
여건	1.220887e-16	2.506478e-16	2.254202e-16	-7.037113e-16	1.931531e-16	-4.827899e-16	-2.299824e-16	-1.865925e-16	1.898593e-16	2.339320e-16	...	5.948592e-17	4.690015e-16	-2.033156e-16	-8.059062e-17	5.631545e-17	1.000000e+00	-3.458103e-16	2.524948e-16	-8.198460e-16	-2.849071e-16
시베리아	5.862109e-16	2.691324e-17	1.774368e-16	3.656700e-16	9.031377e-17	-3.895657e-16	1.098044e-17	-2.101936e-16	-4.556776e-16	-1.800539e-16	...	-1.484648e-15	3.570037e-16	9.933804e-16	2.594783e-16	-5.111784e-16	1.205011e-16	-8.961277e-16	6.252006e-16	1.187277e-15	-5.102914e-16
호통	-9.331040e-17	1.079745e-16	7.419846e-16	4.794175e-16	1.000000e+00	-2.812850e-15	9.231259e-16	-8.856258e-16	7.925280e-16	-9.764165e-17	...	1.074559e-15	5.816594e-16	-3.052167e-16	1.455149e-15	-6.168711e-16	1.903078e-16	-3.834766e-15	1.000000e+00	1.407430e-15	-4.013017e-17
시스	1.721318e-16	3.824994e-16	-3.103303e-16	-6.615510e-16	5.632508e-16	-1.733232e-15	-2.015049e-16	-4.742026e-16	-9.297764e-16	2.851297e-16	...	2.116321e-15	1.689165e-15	-1.346595e-15	1.000000e+00	1.392831e-15	-2.286779e-16	-2.090036e-15	9.913909e-16	-1.259248e-16	-2.141422e-16
호치민	5.042455e-16	9.002023e-18	2.131416e-16	4.408888e-16	1.174915e-16	-4.206274e-16	6.190753e-17	-2.153976e-16	-4.687856e-16	-1.901210e-16	...	-4.200499e-16	9.477919e-16	7.593250e-16	-8.339204e-16	2.593931e-16	-2.306440e-16	-1.006097e-15	9.275090e-16	-4.191583e-16	-5.165477e-17
교통부	1.000000e+00	1.283010e-16	1.682673e-16	-1.346571e-16	1.922969e-16	-8.951294e-16	7.842355e-17	3.575696e-17	-1.878072e-16	-1.169783e-17	...	-1.325842e-16	2.245894e-16	3.257213e-16	-1.131536e-15	2.569633e-16	-2.566153e-16	2.254166e-16	-1.602070e-16	5.092335e-17	1.710269e-16
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
부끄러움	1.269359e-17	-1.736880e-18	5.042811e-16	5.068361e-16	1.000000e+00	-2.795013e-15	1.188689e-15	-3.514061e-17	-5.229979e-17	7.186929e-16	...	6.535474e-16	-2.897751e-16	7.576655e-17	6.418363e-16	1.000000e+00	5.169170e-17	-4.091547e-15	1.000000e+00	1.030852e-16	3.656845e-16
가능	1.565054e-17	1.388220e-16	-5.947793e-16	-1.533307e-15	1.204327e-15	-2.122293e-15	-6.123341e-16	2.697781e-17	1.000000e+00	1.000000e+00	...	-4.059552e-17	2.159083e-16	-7.996950e-16	6.531443e-17	1.364121e-15	-8.186006e-16	-2.237341e-15	2.408964e-15	1.000000e+00	3.251166e-16
벤처	1.000000e+00	1.017455e-16	1.768183e-16	-1.055311e-16	2.225640e-16	-8.616252e-16	4.797987e-17	6.331525e-17	-1.730787e-16	-2.263154e-17	...	-1.575282e-16	8.585896e-17	2.679593e-16	-2.277127e-16	-7.375566e-17	-2.096159e-17	-7.628661e-16	2.659221e-16	-1.277343e-16	1.124512e-16
어제	6.874632e-16	-2.057264e-16	4.562329e-16	-1.693421e-15	1.123881e-15	-1.586557e-15	-4.060148e-17	4.246575e-16	-9.149167e-16	1.000000e+00	...	2.715356e-16	1.703363e-16	1.000000e+00	2.710436e-16	1.263911e-15	-1.618567e-16	-2.320301e-15	3.333905e-16	6.797370e-16	9.946882e-16
칭호	-3.023440e-16	3.256248e-16	-9.636503e-17	-1.763011e-16	3.343751e-16	-3.238237e-16	3.222644e-16	1.248441e-16	-4.184457e-16	-3.545329e-16	...	2.520550e-16	1.551038e-16	-2.522339e-16	1.202454e-16	1.894667e-16	-6.155780e-16	-6.150971e-16	3.675251e-16	3.822671e-16	-2.937485e-16
아이러니	1.600374e-16	-3.533154e-16	6.601704e-16	-5.908843e-16	5.455166e-16	-4.477240e-16	-1.793479e-16	-1.472798e-16	-2.507661e-16	2.055540e-16	...	-5.275390e-17	-3.007479e-16	2.286264e-16	-3.714682e-16	7.616648e-17	-2.113635e-16	-6.291345e-16	-2.911885e-16	-1.637698e-16	2.084617e-16
기준	1.000000e+00	-4.422601e-16	7.437923e-16	-6.165874e-16	8.869455e-16	-2.283853e-15	-9.780911e-16	-5.864349e-16	-7.094002e-16	2.900493e-16	...	-1.955109e-16	-1.796286e-16	7.301612e-16	-7.106162e-16	4.322825e-16	-1.994184e-16	-1.777998e-15	9.208457e-16	1.000000e+00	9.223277e-17
개입	1.150305e-15	7.173939e-16	1.000000e+00	-2.448920e-16	7.142415e-16	-1.720427e-15	-5.075911e-16	-9.878014e-17	-2.146405e-15	1.000000e+00	...	-2.063949e-16	1.939317e-16	1.168311e-15	-4.594299e-16	3.695250e-16	7.081793e-17	-2.819325e-15	2.105320e-15	3.987370e-16	6.541624e-16
있음	-1.428390e-16	-3.466876e-16	-2.386881e-17	1.000000e+00	4.704339e-16	-5.033291e-16	-4.017279e-16	-3.798596e-16	-4.623307e-16	-1.496321e-16	...	-4.256385e-16	-7.553894e-16	-3.096594e-17	2.794540e-16	1.948332e-16	-2.457131e-16	-7.229393e-16	6.593767e-16	1.654162e-16	1.724140e-16
매진	3.106727e-16	1.435648e-16	6.932490e-16	-1.258877e-15	1.000000e+00	-2.509813e-15	7.212015e-16	-4.724614e-17	1.000000e+00	5.827422e-16	...	4.192791e-16	5.856879e-17	-5.279074e-16	4.442819e-16	1.000000e+00	1.000000e+00	-3.330754e-15	1.012998e-15	-5.786201e-16	7.057395e-16
소란	2.073391e-16	-1.764376e-16	3.247439e-17	1.851661e-16	1.000000e+00	-9.536697e-16	5.239234e-16	-2.032505e-16	4.262527e-16	6.608960e-17	...	-8.706450e-17	-3.120747e-16	-2.308651e-17	4.800220e-16	2.119441e-16	3.454101e-16	-1.247311e-15	-3.913659e-16	9.147912e-17	3.779023e-16
아시아	5.084913e-16	-4.393881e-17	-1.181097e-15	-1.889898e-16	5.749909e-17	1.000000e+00	-1.225573e-17	-3.788992e-16	1.812197e-16	3.118365e-16	...	9.263653e-16	-1.276196e-17	6.350603e-16	3.146377e-16	3.087344e-16	-7.518176e-16	-1.549606e-15	6.060662e-16	6.894905e-16	3.784213e-16
확립	-1.951192e-16	4.002892e-16	-2.477151e-16	1.227879e-16	4.330421e-17	-6.425090e-16	-3.128465e-17	1.098239e-16	-6.019509e-17	-2.225742e-16	...	1.236228e-16	1.378799e-16	-2.429613e-16	-1.899113e-16	-6.777494e-17	-2.984465e-17	-1.001192e-15	1.248363e-16	8.227746e-17	2.760995e-16
균형	-2.273374e-17	3.994881e-16	-2.087908e-16	-3.848397e-16	-1.759686e-16	-1.052719e-15	2.781553e-16	3.203560e-16	-6.002874e-16	-1.595189e-16	...	2.113147e-16	4.253309e-16	2.519920e-17	1.971263e-16	-1.198314e-16	-6.156241e-17	1.000000e+00	-2.605469e-16	2.622405e-16	1.631526e-16
이달	-8.912446e-17	-3.796699e-16	7.725550e-17	-8.359802e-17	1.401538e-16	-3.683840e-16	-5.032360e-16	-4.418148e-16	3.442211e-16	-4.756102e-16	...	3.742773e-16	1.889777e-16	8.301251e-18	-6.376420e-17	1.432189e-17	2.204580e-16	-4.037298e-16	2.933496e-16	2.037987e-16	-6.051803e-17
의뢰	2.207398e-16	5.251605e-16	1.000000e+00	-9.668526e-16	1.460238e-15	-2.604289e-15	1.847022e-17	-9.146816e-16	-7.068540e-16	-3.211261e-16	...	1.251900e-15	2.291694e-16	1.000000e+00	1.000000e+00	1.027399e-16	3.318693e-16	-3.901783e-15	1.860345e-15	2.513289e-16	1.230600e-15
변화	-1.200636e-16	-3.231444e-16	4.442827e-16	-4.310002e-16	4.821337e-17	-1.162889e-15	-5.872965e-16	-5.351615e-16	9.550488e-16	-3.161273e-16	...	2.624641e-16	-1.195166e-16	6.596578e-17	9.351860e-18	4.495845e-17	1.062827e-16	-1.366489e-15	3.824295e-16	2.050309e-16	1.000000e+00
혼자	1.616156e-16	1.190886e-16	-1.237939e-16	-4.727851e-16	-3.316469e-17	-7.678253e-16	1.185351e-16	-2.598935e-16	-3.814766e-16	1.150709e-17	...	4.669069e-16	2.468203e-16	-5.732111e-16	1.000000e+00	1.553072e-16	1.715588e-16	-6.511977e-16	1.125986e-15	-1.732467e-16	6.372093e-16
박지원	-5.665050e-17	2.862771e-16	1.475419e-16	3.608009e-16	-1.971568e-16	-1.263449e-15	3.335290e-16	8.629444e-17	-1.158498e-16	2.509216e-16	...	2.966789e-16	2.205559e-16	-4.983018e-17	1.154984e-16	1.724135e-16	3.546417e-17	-2.047834e-15	1.000000e+00	1.432283e-16	-1.690714e-16
부분적	-1.411649e-16	-3.558471e-16	-3.618491e-17	1.000000e+00	4.794992e-16	-5.051486e-16	-3.876431e-16	-3.734489e-16	-4.673708e-16	-1.485214e-16	...	-4.106196e-16	-7.469626e-16	-2.262881e-17	2.958177e-16	1.958099e-16	-2.238599e-16	-7.087331e-16	6.864393e-16	1.669806e-16	1.668000e-16
핵심	-5.165226e-17	8.930469e-16	-9.937413e-16	-1.692253e-15	4.007618e-16	1.000000e+00	6.476327e-17	8.918484e-16	1.000000e+00	1.000000e+00	...	4.469924e-16	8.449680e-16	-6.680409e-16	9.600721e-18	4.916428e-16	-3.664129e-16	1.000000e+00	9.786720e-16	4.122725e-16	1.281340e-15
이제	-1.978152e-17	-2.846498e-16	6.244395e-17	4.424334e-16	1.078370e-15	-1.214058e-15	-5.694480e-16	-4.221536e-16	1.956979e-16	-2.150542e-16	...	-1.709340e-16	5.492586e-16	3.608405e-16	-6.576984e-17	8.124091e-16	3.364827e-16	-7.728318e-16	8.070056e-16	4.797450e-17	-6.968775e-17
배제	-3.413509e-17	-1.668367e-16	-3.315866e-17	-1.114470e-16	5.481836e-16	-6.394383e-16	5.062442e-16	-1.671202e-16	-5.829587e-16	-4.615162e-17	...	1.000000e+00	-4.079784e-16	9.285057e-17	1.223855e-16	7.365089e-16	-6.228133e-16	-1.087219e-15	-2.312131e-16	-2.009094e-16	-4.499170e-16
감정	2.545475e-17	3.174320e-18	-3.546369e-17	5.714128e-16	8.877861e-16	-8.265459e-16	1.404766e-17	-1.151509e-16	-2.968444e-16	3.122962e-16	...	-6.280357e-16	1.398827e-16	3.031013e-16	-8.445067e-17	7.035527e-16	-1.286183e-16	-3.939719e-16	5.902020e-16	-1.389763e-16	-1.072180e-16
비판적	1.000000e+00	1.304477e-16	1.824123e-16	-1.461290e-16	1.599323e-16	-8.585250e-16	3.159116e-17	6.584922e-17	-1.716097e-16	-1.181601e-17	...	-9.085250e-17	1.301921e-16	2.440096e-16	-1.893784e-16	-1.262320e-16	-5.148840e-17	-7.322998e-16	3.560617e-16	-1.570949e-16	3.235906e-17
문체부	1.000000e+00	1.582033e-16	1.824123e-16	-1.426595e-16	1.460545e-16	-8.602598e-16	3.506061e-17	6.584922e-17	-1.854874e-16	-2.569380e-17	...	-6.222956e-17	1.440699e-16	2.370707e-16	-1.789701e-16	-1.262320e-16	-2.373282e-17	-7.305651e-16	3.560617e-16	-1.432172e-16	1.327710e-17
일방적	1.007401e-16	2.471163e-16	-1.248165e-16	-5.636086e-16	5.524940e-16	-8.829365e-16	1.000000e+00	-1.222899e-16	-1.801963e-16	-2.451677e-16	...	4.453535e-16	1.884088e-16	1.565845e-16	-1.184261e-18	1.700217e-16	-7.309473e-17	-4.045143e-16	5.810719e-16	-1.052485e-15	1.378215e-16
메시지	6.623673e-16	-3.987856e-16	9.564588e-16	-3.271697e-16	9.715764e-16	-1.051376e-15	1.801281e-16	-9.127584e-16	-1.139223e-15	1.181844e-16	...	1.000000e+00	2.132760e-18	6.171452e-16	-2.203443e-16	2.421658e-16	-9.396799e-16	-1.616319e-15	1.741008e-16	-6.131815e-18	-2.251124e-16
보험	2.074556e-16	1.000000e+00	1.592850e-16	-6.650689e-17	1.000000e+00	-1.403406e-15	3.984420e-16	-2.250567e-16	4.049823e-16	-1.181218e-16	...	-3.856852e-16	-8.476240e-16	4.089501e-16	5.585453e-16	1.676714e-16	2.166523e-16	-1.731809e-15	-3.141403e-16	-4.640783e-17	1.758170e-16
현종	-1.481710e-16	-3.571441e-16	-4.303619e-17	1.000000e+00	4.635859e-16	-4.481806e-16	-3.684724e-16	-3.732042e-16	-5.042715e-16	-1.261339e-16	...	-3.985930e-16	-7.667819e-16	-4.758792e-17	3.353757e-16	1.826931e-16	-2.473741e-16	-7.490404e-16	6.756209e-16	1.293977e-16	1.497172e-16

2979 rows × 40 columns

단어 ~ 단어 Cosine Similarity

In [221]:

_U = U.dot(np.diag(sigma))
_U.shape

Out[221]:

(2979, 40)

In [224]:

# 단어간의 
# Cosine similarity
pd.DataFrame(_U.dot(_U.T) / (np.linalg.norm(_U, axis=1).reshape(2979,1) * np.linalg.norm(_U.T, axis=0).reshape(1,2979)),index=coldata,columns=coldata)

Out[224]:

	발언	30	포스	절전	두고	분산	촉구	강물	평가	사이트	...	핵심	이제	배제	감정	비판적	문체부	일방적	메시지	보험	현종
발언	1.000000e+00	2.672612e-01	-4.031939e-15	1.547491e-15	1.889822e-01	-3.079241e-15	2.834734e-01	-2.098961e-15	2.020305e-01	6.602492e-16	...	2.182179e-01	1.889822e-01	4.523622e-16	2.672612e-01	1.569177e-16	1.485725e-16	2.672612e-01	1.543033e-01	8.295622e-16	-1.824265e-16
30	2.672612e-01	1.000000e+00	-1.805800e-15	1.205687e-15	-4.431395e-16	-1.872207e-15	-2.980433e-16	-1.141319e-15	2.519763e-01	1.924501e-01	...	2.721655e-01	2.428841e-16	-8.793233e-16	4.596401e-18	-4.104865e-17	-4.041213e-18	-4.404116e-17	-1.210510e-15	2.357023e-01	-1.997698e-15
포스	-4.031939e-15	-1.805800e-15	1.000000e+00	7.312311e-16	-6.906868e-16	-1.029443e-16	-3.118161e-15	1.000000e+00	3.779645e-01	-9.608891e-16	...	4.082483e-01	-1.731820e-15	-3.422214e-16	-1.448835e-15	-8.184704e-16	-8.202051e-16	-3.368393e-16	-5.469588e-16	-1.120320e-15	-3.992337e-16
절전	1.547491e-15	1.205687e-15	7.312311e-16	1.000000e+00	5.678522e-16	7.002979e-16	1.368988e-15	6.886505e-16	5.943712e-16	-4.055891e-16	...	1.143539e-15	-2.766023e-16	-2.454099e-17	-3.161317e-16	1.906062e-17	5.182829e-18	-1.969417e-16	5.773503e-01	4.087718e-16	8.092316e-16
두고	1.889822e-01	-4.431395e-16	-6.906868e-16	5.678522e-16	1.000000e+00	-1.701985e-15	9.934992e-17	-1.121916e-17	8.016095e-17	-5.637757e-16	...	5.326886e-16	8.212049e-17	5.000000e-01	-2.239780e-16	1.299002e-16	1.403266e-16	2.088913e-15	4.082483e-01	-1.632470e-16	-4.189833e-16
분산	-3.079241e-15	-1.872207e-15	-1.029443e-16	7.002979e-16	-1.701985e-15	1.000000e+00	-8.995014e-16	-1.862092e-15	-1.865657e-15	-1.718013e-15	...	4.082483e-01	-5.156016e-16	-7.403722e-16	-3.839905e-16	-8.175738e-16	-8.175738e-16	4.207800e-17	-1.387303e-15	-1.092039e-15	-8.893801e-16
촉구	2.834734e-01	-2.980433e-16	-3.118161e-15	1.368988e-15	9.934992e-17	-8.995014e-16	1.000000e+00	-1.007217e-15	-7.169167e-16	-1.160861e-15	...	1.443376e-01	5.000000e-01	-2.787964e-16	3.535534e-01	-1.508455e-17	-2.489763e-17	-7.814280e-16	4.547199e-16	4.215397e-16	1.630744e-16
강물	-2.098961e-15	-1.141319e-15	1.000000e+00	6.886505e-16	-1.121916e-17	-1.862092e-15	-1.007217e-15	1.000000e+00	3.779645e-01	-1.273260e-15	...	4.082483e-01	-1.340623e-15	-8.124476e-17	-1.732294e-15	-8.125122e-16	-8.142470e-16	-1.020719e-15	-4.700472e-16	-8.895341e-16	-2.685442e-16
평가	2.020305e-01	2.519763e-01	3.779645e-01	5.943712e-16	8.016095e-17	-1.865657e-15	-7.169167e-16	3.779645e-01	1.000000e+00	-2.194008e-16	...	1.543033e-01	-2.750663e-17	-4.065218e-16	4.421610e-16	3.779645e-01	3.779645e-01	-5.085478e-16	2.182179e-01	-5.045943e-16	3.779645e-01
사이트	6.602492e-16	1.924501e-01	-9.608891e-16	-4.055891e-16	-5.637757e-16	-1.718013e-15	-1.160861e-15	-1.273260e-15	-2.194008e-16	1.000000e+00	...	-8.732159e-16	1.720707e-17	-5.186785e-16	5.191784e-16	1.566534e-16	1.566534e-16	1.011546e-15	-4.871482e-16	4.082483e-01	2.645252e-18
나발	-1.990028e-15	-1.360914e-15	1.000000e+00	1.645808e-15	1.996906e-15	7.510281e-16	-2.923233e-15	1.000000e+00	3.779645e-01	-4.229613e-16	...	4.082483e-01	-8.799124e-16	3.822305e-17	-3.214967e-16	-8.449298e-16	-8.466645e-16	-1.063285e-15	6.829410e-16	-8.942223e-16	-2.443685e-16
일훈	2.672612e-01	3.333333e-01	-9.861456e-17	-2.209970e-16	9.079508e-16	-1.349699e-15	1.410664e-16	-4.086480e-16	7.261561e-16	1.228368e-15	...	4.082483e-01	8.732022e-16	5.959272e-16	8.876378e-16	7.907563e-17	7.213674e-17	-6.186277e-17	-1.198068e-16	4.861792e-16	-6.090468e-16
참사	4.629100e-01	1.924501e-01	-7.112279e-16	1.164378e-15	1.497523e-15	-1.534184e-15	1.033762e-15	-6.393180e-16	2.182179e-01	9.706597e-16	...	-7.390398e-16	4.714437e-16	-9.795538e-17	2.786236e-16	2.361971e-16	2.311894e-16	-5.851668e-17	3.333333e-01	3.519996e-16	-8.712875e-16
부업	1.070807e-15	8.478870e-16	-1.179621e-15	6.466294e-16	-3.233341e-17	-1.781403e-15	9.543648e-16	-8.835599e-16	7.982555e-17	4.334738e-16	...	-2.624818e-16	3.406332e-16	3.216397e-16	1.802722e-16	2.511500e-16	2.303333e-16	6.695421e-16	9.948147e-16	7.071068e-01	3.291406e-16
3000	1.532009e-15	1.038522e-15	-5.776181e-16	-7.818372e-16	-7.495561e-16	-1.428608e-15	9.934465e-16	3.035353e-16	2.482833e-16	5.773503e-01	...	-6.100627e-16	3.412069e-16	-1.212552e-16	3.859861e-16	2.097688e-16	1.958910e-16	4.790479e-16	-1.087276e-15	2.417842e-16	6.849652e-16
강경	1.889822e-01	-1.508250e-16	-6.596480e-16	-9.513827e-16	1.342132e-15	-3.764467e-16	2.500000e-01	-4.808572e-16	-3.514756e-17	1.295287e-16	...	-5.429130e-16	-7.275317e-16	-4.822874e-16	-4.120275e-16	2.377516e-16	2.426582e-16	7.071068e-01	-9.363677e-16	4.320060e-16	-5.896999e-16
경원	5.640761e-01	3.015113e-01	-3.764325e-15	5.454889e-16	2.132007e-01	-1.995141e-15	4.264014e-01	-2.054144e-15	1.139606e-01	1.740777e-01	...	1.230915e-01	2.132007e-01	7.934186e-16	3.015113e-01	1.753291e-16	1.690526e-16	3.015113e-01	7.684498e-17	5.978865e-16	-4.458086e-16
말씀	-2.022059e-15	-1.213589e-15	5.000000e-01	5.325882e-16	-8.647979e-16	5.000000e-01	-1.416709e-15	5.000000e-01	1.889822e-01	-1.170850e-15	...	4.082483e-01	-3.560547e-16	-2.738033e-16	-2.348565e-16	-5.874369e-16	-5.874369e-16	-2.718188e-16	-3.601166e-16	3.535534e-01	-5.811463e-16
보안법	-1.187865e-15	-1.490129e-15	-8.834375e-16	-2.424362e-16	-3.356497e-17	9.541403e-16	-8.473820e-16	-2.294154e-15	2.278574e-16	5.773503e-01	...	7.976262e-17	-1.886646e-16	8.423345e-16	-2.185495e-16	3.524890e-17	1.096277e-17	1.194490e-15	-5.833658e-16	-4.987346e-16	-1.937304e-16
채널	3.585686e-01	2.981424e-01	-2.779920e-15	2.236068e-01	6.742444e-16	-3.752754e-15	3.162278e-01	-9.008705e-16	2.535463e-01	3.872983e-01	...	2.738613e-01	4.867044e-16	-2.263283e-16	9.073905e-16	8.719741e-17	9.340374e-17	2.812267e-16	1.290994e-01	1.581139e-01	2.236068e-01
자동차	-2.423509e-15	-1.796684e-15	1.000000e+00	2.775422e-16	-9.820412e-16	-9.696599e-16	-1.724844e-15	1.000000e+00	3.779645e-01	-4.717490e-16	...	4.082483e-01	-7.413625e-16	-1.432484e-15	-5.726770e-16	-8.010210e-16	-8.010210e-16	-1.023398e-15	-8.822222e-16	-8.499310e-16	-3.059445e-16
만이	2.672612e-01	1.666667e-01	-1.593529e-15	-3.594393e-16	5.543611e-16	-8.018047e-16	3.535534e-01	-3.818932e-16	1.889822e-01	-4.067924e-16	...	-9.940897e-16	-7.994824e-16	2.728278e-16	-6.089410e-16	-7.187808e-17	-6.493919e-17	5.000000e-01	-1.822301e-16	1.985616e-16	-4.797663e-16
민주화	2.672612e-01	4.894895e-16	-9.111893e-16	1.694840e-15	1.236439e-15	-3.042822e-15	2.042087e-15	-1.338968e-15	-3.329743e-16	3.446112e-16	...	-1.004869e-15	6.903158e-16	5.190756e-16	7.337792e-16	1.564993e-16	1.842549e-16	7.287847e-16	4.200757e-16	1.388336e-16	8.034669e-16
신기	1.179237e-16	8.097646e-16	-5.003771e-16	-1.270199e-16	2.197437e-16	-1.672671e-15	2.500000e-01	-3.286274e-17	-2.368109e-16	-4.099551e-16	...	-5.769326e-16	-1.815313e-16	4.518657e-16	9.935570e-17	2.403939e-16	2.502070e-16	6.723099e-16	-5.810992e-17	1.401019e-17	-5.209760e-17
여건	-6.828427e-16	3.333333e-01	-5.588181e-16	1.007427e-15	-2.564426e-16	1.825763e-16	-5.453939e-16	-9.233485e-16	-1.554519e-15	-4.279073e-16	...	-1.177946e-16	-1.744475e-16	-4.984487e-16	-1.389139e-16	-6.544481e-17	-6.544481e-17	-1.756768e-16	-7.773842e-16	4.083820e-16	-7.908555e-16
시베리아	8.840799e-16	-6.344168e-17	-7.147487e-17	1.000000e+00	-1.122578e-15	-6.025291e-16	1.036844e-15	-8.412810e-17	-1.501819e-16	-1.425827e-15	...	-9.402002e-16	-6.929379e-16	-1.529615e-15	-5.618828e-16	1.271070e-16	1.132293e-16	-2.826490e-16	5.773503e-01	3.010031e-16	6.754328e-16
호통	1.543033e-01	1.136791e-15	-1.381280e-15	1.427338e-15	1.044314e-15	-2.482535e-15	2.220736e-15	-1.565057e-15	8.090835e-16	6.429731e-16	...	-4.612072e-16	1.045819e-15	8.378038e-16	1.063494e-15	1.291103e-16	1.210980e-16	1.124866e-15	1.231586e-15	4.082483e-01	7.405610e-16
시스	3.086067e-01	1.924501e-01	-8.626135e-16	4.306964e-16	9.839793e-16	-7.754337e-16	-6.230571e-16	8.858978e-18	4.364358e-01	4.183501e-16	...	-1.246887e-15	-2.708082e-16	1.033099e-15	1.629011e-17	-5.575005e-17	-5.775314e-17	-9.281776e-17	-3.070160e-17	1.882064e-16	-2.767250e-16
호치민	2.758671e-16	-2.086169e-16	-6.223133e-17	1.000000e+00	-4.660727e-17	-7.119020e-16	3.356821e-16	-1.017457e-16	-3.044839e-16	-1.001298e-15	...	-1.199733e-15	-7.764533e-16	-7.385508e-16	-6.071734e-16	6.487416e-17	5.099637e-17	-2.437566e-16	5.773503e-01	2.973974e-16	6.931617e-16
교통부	-2.450926e-16	-3.139535e-16	-8.489829e-16	5.671229e-17	2.119374e-16	1.446691e-16	7.322861e-17	-8.394943e-16	3.779645e-01	1.860673e-16	...	-5.944910e-16	1.632883e-16	-2.938332e-16	9.582427e-17	1.000000e+00	1.000000e+00	1.401776e-16	1.633676e-16	1.375423e-16	-1.108862e-16
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
부끄러움	1.543033e-01	8.204404e-16	-1.694243e-15	1.798978e-15	5.765568e-16	-2.620976e-15	1.925605e-15	-1.664377e-15	4.182624e-16	4.764923e-16	...	-6.492529e-16	9.815038e-16	6.925882e-16	1.073444e-15	1.389439e-16	1.349377e-16	1.136582e-15	9.342037e-16	4.082483e-01	8.293488e-16
가능	1.336306e-01	3.333333e-01	-2.129361e-15	9.936105e-16	1.840266e-16	-1.527726e-15	3.535534e-01	1.890843e-16	1.889822e-01	-6.790769e-16	...	4.082483e-01	-1.795059e-16	-1.500591e-16	5.775927e-17	-4.130845e-16	-4.200234e-16	-3.341468e-16	-3.263729e-16	-2.504497e-17	-9.831505e-16
벤처	1.465616e-16	-6.833597e-17	-8.098341e-16	-1.065432e-17	1.469673e-16	-8.491971e-16	-4.979419e-17	-8.151178e-16	3.779645e-01	1.692606e-16	...	-9.723270e-16	1.897723e-16	-2.974879e-16	8.601694e-17	1.000000e+00	1.000000e+00	1.340201e-16	9.909559e-17	1.181449e-16	-7.568870e-17
어제	4.008919e-01	1.666667e-01	-1.015545e-15	1.498499e-15	8.610362e-16	-1.907849e-15	1.767767e-01	-2.652066e-16	1.889822e-01	3.949892e-16	...	2.041241e-01	5.321215e-16	-3.598428e-16	8.412977e-16	1.984790e-16	1.967442e-16	-2.481363e-16	2.886751e-01	3.289840e-16	-1.099642e-15
칭호	1.655131e-16	3.333333e-01	3.954247e-16	-5.674830e-16	-1.946303e-16	-1.424620e-15	-2.607685e-16	-5.395634e-16	3.779645e-01	-2.314916e-16	...	-1.363442e-15	-4.375153e-16	3.850119e-16	3.403021e-17	-2.899521e-16	-2.886511e-16	8.454096e-16	-4.635769e-16	-3.770104e-16	-5.462657e-16
아이러니	2.672612e-01	-7.527508e-16	-9.246011e-16	6.275262e-16	3.179035e-16	-1.730666e-15	5.120363e-16	-1.016764e-15	3.779645e-01	-1.480828e-16	...	-1.667039e-16	5.677957e-17	-2.670453e-16	1.957916e-16	1.988127e-16	2.126905e-16	-5.343087e-16	5.773503e-01	1.503970e-16	-7.397933e-16
기준	3.086067e-01	-9.366787e-16	-2.620001e-15	4.153618e-16	1.353856e-16	-1.791282e-15	2.041241e-01	-8.977938e-16	4.364358e-01	-4.703431e-16	...	-7.909057e-16	6.885576e-17	-2.080051e-16	-1.129141e-16	5.773503e-01	5.773503e-01	-1.018439e-15	3.333333e-01	3.265807e-17	-1.570521e-16
개입	1.374363e-15	-4.968431e-16	-1.134199e-15	5.000000e-01	1.917039e-16	-6.435044e-16	5.303301e-01	-3.850819e-16	-9.315854e-16	-2.281346e-16	...	2.041241e-01	-2.474854e-16	-1.311648e-16	2.436532e-16	1.694543e-16	1.555765e-16	-4.154837e-16	2.886751e-01	5.874866e-16	-2.733263e-16
있음	-2.130203e-16	-2.038423e-15	-4.180614e-16	7.614281e-16	-4.156377e-16	-8.489675e-16	1.963651e-16	-3.147073e-16	3.779645e-01	6.724061e-18	...	-1.469305e-15	5.713513e-16	-2.784531e-16	8.663778e-16	-1.265129e-16	-1.230434e-16	-1.019196e-15	-2.338447e-16	-9.010830e-17	1.000000e+00
매진	6.490871e-16	3.333333e-01	-1.893976e-15	2.035965e-15	1.886556e-17	-1.648659e-15	4.880647e-16	-1.251826e-15	-1.089814e-15	1.133122e-16	...	2.041241e-01	5.631877e-16	-1.654325e-16	5.823554e-16	-1.258138e-16	-1.327526e-16	6.147019e-16	-6.003738e-17	3.535534e-01	-6.087419e-16
소란	1.001860e-15	7.224031e-16	-1.249689e-15	6.915044e-16	2.976880e-16	-1.168708e-15	1.039647e-15	-9.278779e-16	2.728889e-16	-2.083528e-16	...	6.399128e-17	5.980881e-16	2.979605e-16	5.089443e-16	1.885977e-16	1.677811e-16	7.135770e-16	8.046681e-16	7.071068e-01	3.657744e-16
아시아	-8.652082e-16	-2.228156e-16	5.773503e-01	4.976037e-17	1.577740e-16	-1.127167e-15	2.041241e-01	5.773503e-01	2.182179e-01	-8.752627e-16	...	2.357023e-01	-5.476476e-16	-9.193232e-17	-2.342298e-16	-2.082962e-16	-1.842592e-16	-5.714552e-17	-5.572780e-16	-4.561401e-16	-2.264465e-16
확립	2.672612e-01	-1.314673e-17	-1.723770e-15	-2.214627e-16	1.073361e-15	-2.225639e-15	7.943987e-16	-3.342901e-16	3.241804e-17	-5.001838e-17	...	4.082483e-01	4.187739e-16	-3.553113e-17	4.652016e-16	-1.719474e-16	-1.997030e-16	-2.430389e-16	6.492411e-16	4.630460e-16	-5.053392e-17
균형	1.889822e-01	2.357023e-01	-1.121773e-17	1.234640e-16	-2.685847e-16	7.071068e-01	-8.061260e-16	-1.489033e-15	-1.228053e-15	-7.973260e-16	...	5.773503e-01	-2.873849e-16	9.763444e-17	-3.111645e-16	-5.711362e-16	-5.527366e-16	-1.379065e-16	-7.318125e-16	-4.540800e-16	-9.980189e-16
이달	6.622936e-16	8.526817e-17	-9.845618e-16	-7.320804e-17	3.062687e-16	-3.642769e-16	3.535534e-01	-1.646712e-16	-5.415693e-16	-5.175999e-16	...	-4.307152e-16	7.071068e-01	1.040311e-16	-1.544741e-16	7.915257e-17	7.915257e-17	-5.453564e-17	-1.561105e-16	-2.996262e-16	-9.470969e-18
의뢰	2.390457e-01	1.490712e-01	-6.080700e-16	9.646622e-16	1.008042e-15	-2.042172e-15	1.581139e-01	2.038280e-16	1.690309e-01	1.046467e-15	...	-1.320712e-15	8.006338e-16	2.320170e-16	6.617438e-16	9.946819e-17	9.946819e-17	2.121939e-16	7.006740e-16	9.204320e-16	-2.199244e-16
변화	1.889822e-01	-2.461831e-16	-1.768328e-15	-3.708481e-16	5.000000e-01	-1.371866e-15	2.500000e-01	-6.804547e-16	-4.704942e-16	-5.262290e-16	...	-9.313993e-17	5.000000e-01	2.146736e-16	-1.103007e-16	2.279462e-16	2.083200e-16	-9.624912e-17	-1.280167e-16	-1.557247e-16	-1.149629e-16
혼자	2.672612e-01	-3.280069e-18	-9.317744e-16	1.413438e-15	1.088538e-15	-2.338691e-16	-6.922492e-16	8.602065e-16	3.779645e-01	1.541006e-16	...	-6.177635e-16	3.502385e-17	5.875253e-17	3.417332e-17	-1.907159e-16	-1.941854e-16	4.134129e-17	-1.933867e-16	2.012455e-16	-6.292850e-17
박지원	2.672612e-01	7.008149e-16	-8.828364e-16	1.707508e-15	1.057155e-15	-2.071522e-15	1.649440e-15	-1.288796e-15	1.484051e-16	3.777626e-16	...	-6.784538e-16	7.096127e-16	3.589086e-16	7.730656e-16	1.804374e-16	1.804374e-16	7.185000e-16	3.366222e-16	1.644674e-16	8.297184e-16
부분적	-1.890682e-16	-1.988226e-15	-4.194743e-16	7.491553e-16	-4.209950e-16	-8.490727e-16	2.042810e-16	-2.873347e-16	3.779645e-01	-9.783881e-18	...	-1.475632e-15	5.699671e-16	-3.076840e-16	8.545778e-16	-1.164905e-16	-1.130210e-16	-1.017308e-15	-2.070046e-16	-1.029779e-16	1.000000e+00
핵심	2.182179e-01	2.721655e-01	4.082483e-01	1.143539e-15	5.326886e-16	4.082483e-01	1.443376e-01	4.082483e-01	1.543033e-01	-8.732159e-16	...	1.000000e+00	-2.319058e-16	-4.784199e-16	5.610211e-17	-9.168681e-16	-9.225337e-16	-6.431829e-16	-6.645728e-16	-1.822889e-16	-1.452701e-15
이제	1.889822e-01	2.428841e-16	-1.731820e-15	-2.766023e-16	8.212049e-17	-5.156016e-16	5.000000e-01	-1.340623e-15	-2.750663e-17	1.720707e-17	...	-2.319058e-16	1.000000e+00	-2.966956e-16	7.071068e-01	1.604476e-16	1.604476e-16	-5.739071e-16	-4.460868e-16	7.260418e-17	5.495369e-16
배제	4.523622e-16	-8.793233e-16	-3.422214e-16	-2.454099e-17	5.000000e-01	-7.403722e-16	-2.787964e-16	-8.124476e-17	-4.065218e-16	-5.186785e-16	...	-4.784199e-16	-2.966956e-16	1.000000e+00	-6.072885e-16	-1.998530e-16	-1.986264e-16	6.078213e-16	4.082483e-01	-1.117246e-16	-2.964576e-16
감정	2.672612e-01	4.596401e-18	-1.448835e-15	-3.161317e-16	-2.239780e-16	-3.839905e-16	3.535534e-01	-1.732294e-15	4.421610e-16	5.191784e-16	...	5.610211e-17	7.071068e-01	-6.072885e-16	1.000000e+00	1.069518e-16	1.277685e-16	-7.740290e-16	-5.045745e-16	3.156100e-16	8.508213e-16
비판적	1.569177e-16	-4.104865e-17	-8.184704e-16	1.906062e-17	1.299002e-16	-8.175738e-16	-1.508455e-17	-8.125122e-16	3.779645e-01	1.566534e-16	...	-9.168681e-16	1.604476e-16	-1.998530e-16	1.069518e-16	1.000000e+00	1.000000e+00	1.211008e-16	1.379819e-16	1.257630e-16	-1.162324e-16
문체부	1.485725e-16	-4.041213e-18	-8.202051e-16	5.182829e-18	1.403266e-16	-8.175738e-16	-2.489763e-17	-8.142470e-16	3.779645e-01	1.566534e-16	...	-9.225337e-16	1.604476e-16	-1.986264e-16	1.277685e-16	1.000000e+00	1.000000e+00	1.106925e-16	1.540065e-16	1.257630e-16	-1.127629e-16
일방적	2.672612e-01	-4.404116e-17	-3.368393e-16	-1.969417e-16	2.088913e-15	4.207800e-17	-7.814280e-16	-1.020719e-15	-5.085478e-16	1.011546e-15	...	-6.431829e-16	-5.739071e-16	6.078213e-16	-7.740290e-16	1.211008e-16	1.106925e-16	1.000000e+00	-1.799724e-16	4.458281e-16	-9.938009e-16
메시지	1.543033e-01	-1.210510e-15	-5.469588e-16	5.773503e-01	4.082483e-01	-1.387303e-15	4.547199e-16	-4.700472e-16	2.182179e-01	-4.871482e-16	...	-6.645728e-16	-4.460868e-16	4.082483e-01	-5.045745e-16	1.379819e-16	1.540065e-16	-1.799724e-16	1.000000e+00	1.129956e-16	-2.154203e-16
보험	8.295622e-16	2.357023e-01	-1.120320e-15	4.087718e-16	-1.632470e-16	-1.092039e-15	4.215397e-16	-8.895341e-16	-5.045943e-16	4.082483e-01	...	-1.822889e-16	7.260418e-17	-1.117246e-16	3.156100e-16	1.257630e-16	1.257630e-16	4.458281e-16	1.129956e-16	1.000000e+00	-1.081326e-16
현종	-1.824265e-16	-1.997698e-15	-3.992337e-16	8.092316e-16	-4.189833e-16	-8.893801e-16	1.630744e-16	-2.685442e-16	3.779645e-01	2.645252e-18	...	-1.452701e-15	5.495369e-16	-2.964576e-16	8.508213e-16	-1.162324e-16	-1.127629e-16	-9.938009e-16	-2.154203e-16	-1.081326e-16	1.000000e+00

2979 rows × 2979 columns

1.1 K 바꾸기

In [226]:

# K = 5
# 단어 관계 찾기
_sigma = np.diag(sigma[:5])
_sigma
_U = U[:,:5].dot(_sigma)

In [227]:

_U.shape

Out[227]:

(2979, 5)

In [228]:

pd.DataFrame(_U.dot(_U.T) / (np.linalg.norm(_U, axis=1).reshape(2979,1) * np.linalg.norm(_U.T, axis=0).reshape(1,2979)),index=coldata,columns=coldata)

Out[228]:

	발언	30	포스	절전	두고	분산	촉구	강물	평가	사이트	...	핵심	이제	배제	감정	비판적	문체부	일방적	메시지	보험	현종
발언	1.000000	0.563623	-0.004572	0.037578	0.649830	-0.007216	0.962735	-0.004572	0.276721	0.570105	...	0.233539	0.895418	0.265923	0.908281	0.537595	0.537595	0.866102	0.307400	0.356009	-0.059951
30	0.563623	1.000000	0.074900	0.844776	0.967839	0.041117	0.663716	0.074900	0.756966	0.656030	...	0.606927	0.789523	0.923575	0.495063	0.701134	0.701134	0.273102	0.949499	0.277950	0.777660
포스	-0.004572	0.074900	1.000000	0.107744	-0.129727	-0.002145	-0.001077	1.000000	0.676156	-0.121059	...	0.595803	0.008641	-0.095320	-0.042273	0.293763	0.293763	-0.000527	0.036084	-0.008466	-0.058237
절전	0.037578	0.844776	0.107744	1.000000	0.749531	-0.010340	0.181010	0.107744	0.745633	0.419364	...	0.553784	0.382491	0.941101	0.016602	0.513947	0.513947	-0.221970	0.953013	0.098076	0.977899
두고	0.649830	0.967839	-0.129727	0.749531	1.000000	-0.004306	0.713470	-0.129727	0.602824	0.757554	...	0.443062	0.814813	0.903016	0.546567	0.711752	0.711752	0.331767	0.906797	0.382240	0.700025
분산	-0.007216	0.041117	-0.002145	-0.010340	-0.004306	1.000000	0.052678	-0.002145	-0.001878	-0.026795	...	0.587395	-0.027738	0.032310	0.003335	-0.216768	-0.216768	-0.028645	-0.046929	-0.007648	0.023601
촉구	0.962735	0.663716	-0.001077	0.181010	0.713470	0.052678	1.000000	-0.001077	0.334841	0.453729	...	0.326982	0.972194	0.364423	0.964376	0.440923	0.440923	0.889076	0.406599	0.150577	0.105389
강물	-0.004572	0.074900	1.000000	0.107744	-0.129727	-0.002145	-0.001077	1.000000	0.676156	-0.121059	...	0.595803	0.008641	-0.095320	-0.042273	0.293763	0.293763	-0.000527	0.036084	-0.008466	-0.058237
평가	0.276721	0.756966	0.676156	0.745633	0.602824	-0.001878	0.334841	0.676156	1.000000	0.452452	...	0.800508	0.450311	0.662215	0.157378	0.743364	0.743364	0.009869	0.754965	0.283526	0.594916
사이트	0.570105	0.656030	-0.121059	0.419364	0.757554	-0.026795	0.453729	-0.121059	0.452452	1.000000	...	0.272073	0.461743	0.654681	0.225261	0.889702	0.889702	0.088077	0.639759	0.890524	0.328859
나발	-0.004572	0.074900	1.000000	0.107744	-0.129727	-0.002145	-0.001077	1.000000	0.676156	-0.121059	...	0.595803	0.008641	-0.095320	-0.042273	0.293763	0.293763	-0.000527	0.036084	-0.008466	-0.058237
일훈	0.847789	0.910736	-0.025193	0.555966	0.947671	-0.016428	0.892785	-0.025193	0.573336	0.705296	...	0.438574	0.944168	0.729333	0.763473	0.697053	0.697053	0.598207	0.760029	0.347925	0.476378
참사	0.912042	0.810776	-0.042063	0.390679	0.884700	-0.002942	0.890033	-0.042063	0.495241	0.805279	...	0.375756	0.891580	0.617238	0.747106	0.759177	0.759177	0.616373	0.641998	0.530246	0.293710
부업	0.273249	0.120969	0.000099	-0.038638	0.229524	-0.008025	0.044474	0.000099	0.180217	0.807560	...	0.065464	-0.036277	0.164173	-0.151171	0.705337	0.705337	-0.161913	0.145003	0.986921	-0.139561
3000	0.240128	0.130652	-0.016765	-0.006297	0.237632	-0.001923	0.015926	-0.016765	0.184743	0.812221	...	0.068793	-0.056499	0.192442	-0.185134	0.703734	0.703734	-0.204185	0.168712	0.987466	-0.101730
강경	0.862584	0.286004	0.036051	-0.203431	0.334607	-0.028901	0.890961	0.036051	0.041602	0.078320	...	0.080334	0.810132	-0.068317	0.968929	0.101752	0.101752	0.999122	-0.009936	-0.125058	-0.260944
경원	0.955774	0.559227	-0.001458	0.062306	0.613360	0.013692	0.989436	-0.001458	0.242559	0.350555	...	0.241356	0.944899	0.237478	0.989475	0.346263	0.346263	0.945679	0.285307	0.070554	-0.008258
말씀	0.144729	0.190508	0.672440	0.100995	0.057706	0.661421	0.121730	0.672440	0.558142	0.202931	...	0.850720	0.054698	0.067743	-0.008936	0.325144	0.325144	-0.019137	0.100479	0.316198	-0.025640
보안법	0.696815	0.931701	-0.225015	0.678144	0.988535	-0.054865	0.761701	-0.225015	0.498681	0.720575	...	0.334151	0.853990	0.851981	0.622864	0.647069	0.647069	0.420138	0.851789	0.328148	0.643704
채널	0.718599	0.881148	-0.018316	0.598423	0.933839	0.004687	0.691410	-0.018316	0.628415	0.921042	...	0.458706	0.738216	0.795699	0.484165	0.880770	0.880770	0.307846	0.807397	0.666357	0.501257
자동차	-0.004572	0.074900	1.000000	0.107744	-0.129727	-0.002145	-0.001077	1.000000	0.676156	-0.121059	...	0.595803	0.008641	-0.095320	-0.042273	0.293763	0.293763	-0.000527	0.036084	-0.008466	-0.058237
만이	0.912157	0.478967	0.064061	-0.005247	0.512164	0.022086	0.961202	0.064061	0.210462	0.211744	...	0.235866	0.910564	0.134852	0.993468	0.241195	0.241195	0.973113	0.192697	-0.052430	-0.073829
민주화	0.254681	-0.000388	0.041099	-0.174071	0.107008	0.011864	0.010432	0.041099	0.107689	0.724507	...	0.028377	-0.098025	0.021544	-0.161954	0.630461	0.630461	-0.136646	0.006515	0.955455	-0.278290
신기	0.952899	0.564498	0.121899	0.074767	0.596691	-0.027682	0.980428	0.121899	0.327095	0.347608	...	0.290006	0.939329	0.225261	0.974137	0.400518	0.400518	0.936973	0.291843	0.085866	-0.018807
여건	0.030253	0.837785	0.199560	0.994534	0.721936	0.036726	0.175528	0.199560	0.795653	0.395235	...	0.627439	0.371699	0.917767	0.008247	0.518713	0.518713	-0.224104	0.937898	0.090792	0.959637
시베리아	0.037578	0.844776	0.107744	1.000000	0.749531	-0.010340	0.181010	0.107744	0.745633	0.419364	...	0.553784	0.382491	0.941101	0.016602	0.513947	0.513947	-0.221970	0.953013	0.098076	0.977899
호통	0.284755	0.060167	0.017656	-0.120215	0.170984	0.006784	0.046749	0.017656	0.137493	0.768689	...	0.045843	-0.052340	0.084478	-0.135260	0.667516	0.667516	-0.125574	0.067701	0.972724	-0.223790
시스	0.852964	0.867097	-0.003739	0.501501	0.885200	-0.004159	0.938991	-0.003739	0.519603	0.526391	...	0.425256	0.989782	0.642799	0.860677	0.543115	0.543115	0.712276	0.683085	0.138031	0.435320
호치민	0.037578	0.844776	0.107744	1.000000	0.749531	-0.010340	0.181010	0.107744	0.745633	0.419364	...	0.553784	0.382491	0.941101	0.016602	0.513947	0.513947	-0.221970	0.953013	0.098076	0.977899
교통부	0.537595	0.701134	0.293763	0.513947	0.711752	-0.216768	0.440923	0.293763	0.743364	0.889702	...	0.421599	0.485717	0.633591	0.217530	1.000000	1.000000	0.094145	0.689699	0.791648	0.358778
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
부끄러움	0.255839	0.072567	0.001353	-0.087006	0.182467	0.008059	0.021420	0.001353	0.144381	0.778056	...	0.048052	-0.069269	0.114636	-0.166682	0.671383	0.671383	-0.165486	0.093831	0.977522	-0.185978
가능	0.511271	0.979486	0.054919	0.853776	0.928562	0.064730	0.658941	0.054919	0.710373	0.498722	...	0.597174	0.799540	0.897277	0.528578	0.552000	0.552000	0.306268	0.922780	0.081124	0.810711
벤처	0.537595	0.701134	0.293763	0.513947	0.711752	-0.216768	0.440923	0.293763	0.743364	0.889702	...	0.421599	0.485717	0.633591	0.217530	1.000000	1.000000	0.094145	0.689699	0.791648	0.358778
어제	0.860039	0.878445	-0.073578	0.503584	0.939896	0.035021	0.868726	-0.073578	0.533066	0.798630	...	0.424047	0.897050	0.714850	0.717256	0.743893	0.743893	0.557619	0.731135	0.483119	0.421401
칭호	0.039713	0.833843	0.021581	0.981206	0.770759	0.009840	0.144756	0.021581	0.706102	0.546963	...	0.515181	0.332275	0.966377	-0.049560	0.584892	0.584892	-0.292210	0.959883	0.258792	0.960651
아이러니	0.876561	0.783711	0.135563	0.392283	0.833298	-0.153729	0.827280	0.135563	0.604925	0.816188	...	0.378110	0.837042	0.580659	0.673523	0.864748	0.864748	0.559889	0.636617	0.590186	0.258129
기준	0.980158	0.547716	0.188655	0.035622	0.592887	-0.030076	0.946511	0.188655	0.382645	0.505323	...	0.318943	0.881325	0.215942	0.895970	0.559723	0.559723	0.869212	0.284913	0.316766	-0.090454
개입	0.591840	0.991602	-0.024198	0.813784	0.989755	0.049086	0.671340	-0.024198	0.694176	0.727545	...	0.550892	0.785167	0.931228	0.493452	0.720858	0.720858	0.270247	0.942310	0.356924	0.754857
있음	-0.059951	0.777660	-0.058237	0.977899	0.700025	0.023601	0.105389	-0.058237	0.594916	0.328859	...	0.448714	0.313404	0.920700	-0.033307	0.358778	0.358778	-0.274782	0.906593	-0.007226	1.000000
매진	0.156709	0.707236	0.055935	0.748582	0.706284	0.007633	0.117137	0.055935	0.668472	0.843564	...	0.458112	0.221630	0.833355	-0.140185	0.834368	0.834368	-0.332998	0.821930	0.723724	0.674090
소란	0.273249	0.120969	0.000099	-0.038638	0.229524	-0.008025	0.044474	0.000099	0.180217	0.807560	...	0.065464	-0.036277	0.164173	-0.151171	0.705337	0.705337	-0.161913	0.145003	0.986921	-0.139561
아시아	0.303280	0.250133	0.947343	0.121955	0.074715	-0.010879	0.315335	0.947343	0.719489	0.002224	...	0.634562	0.310900	-0.013876	0.275900	0.395960	0.395960	0.301815	0.126922	0.020016	-0.058947
확립	0.738284	0.751371	-0.019933	0.439030	0.836594	-0.172599	0.650264	-0.019933	0.540723	0.951300	...	0.281591	0.670481	0.657686	0.453733	0.923928	0.923928	0.320440	0.680859	0.769398	0.325383
균형	0.158741	0.218609	-0.007039	0.098582	0.181143	0.980686	0.226472	-0.007039	0.110306	0.111603	...	0.663730	0.157403	0.174458	0.152629	-0.076924	-0.076924	0.088835	0.102501	0.060535	0.116408
이달	0.382732	0.960957	0.107753	0.920694	0.919063	-0.076024	0.459961	0.107753	0.802919	0.693045	...	0.557028	0.615282	0.965380	0.262642	0.768636	0.768636	0.032459	0.991884	0.365276	0.850039
의뢰	0.822582	0.727826	-0.100978	0.343985	0.831347	0.005978	0.739796	-0.100978	0.443780	0.933685	...	0.314100	0.723841	0.602916	0.556910	0.836630	0.836630	0.433038	0.606918	0.745420	0.245244
변화	0.710410	0.975693	0.020092	0.723685	0.985536	-0.026733	0.767924	0.020092	0.689106	0.745228	...	0.511590	0.860097	0.859490	0.602834	0.763103	0.763103	0.403514	0.887965	0.384337	0.643799
혼자	0.889103	0.827825	0.026886	0.429409	0.847082	0.022953	0.965296	0.026886	0.498515	0.507648	...	0.430430	0.996304	0.576781	0.895561	0.527405	0.527405	0.764830	0.620969	0.139612	0.355849
박지원	0.254681	-0.000388	0.041099	-0.174071	0.107008	0.011864	0.010432	0.041099	0.107689	0.724507	...	0.028377	-0.098025	0.021544	-0.161954	0.630461	0.630461	-0.136646	0.006515	0.955455	-0.278290
부분적	-0.059951	0.777660	-0.058237	0.977899	0.700025	0.023601	0.105389	-0.058237	0.594916	0.328859	...	0.448714	0.313404	0.920700	-0.033307	0.358778	0.358778	-0.274782	0.906593	-0.007226	1.000000
핵심	0.233539	0.606927	0.595803	0.553784	0.443062	0.587395	0.326982	0.595803	0.800508	0.272073	...	1.000000	0.367148	0.494424	0.176188	0.421599	0.421599	0.052601	0.532968	0.145862	0.448714
이제	0.895418	0.789523	0.008641	0.382491	0.814813	-0.027738	0.972194	0.008641	0.450311	0.461743	...	0.367148	1.000000	0.527878	0.922231	0.485717	0.485717	0.803882	0.575711	0.093317	0.313404
배제	0.265923	0.923575	-0.095320	0.941101	0.903016	0.032310	0.364423	-0.095320	0.662215	0.654681	...	0.494424	0.527878	1.000000	0.172639	0.633591	0.633591	-0.076013	0.986843	0.310957	0.920700
감정	0.908281	0.495063	-0.042273	0.016602	0.546567	0.003335	0.964376	-0.042273	0.157378	0.225261	...	0.176188	0.922231	0.172639	1.000000	0.217530	0.217530	0.968037	0.218188	-0.064936	-0.033307
비판적	0.537595	0.701134	0.293763	0.513947	0.711752	-0.216768	0.440923	0.293763	0.743364	0.889702	...	0.421599	0.485717	0.633591	0.217530	1.000000	1.000000	0.094145	0.689699	0.791648	0.358778
문체부	0.537595	0.701134	0.293763	0.513947	0.711752	-0.216768	0.440923	0.293763	0.743364	0.889702	...	0.421599	0.485717	0.633591	0.217530	1.000000	1.000000	0.094145	0.689699	0.791648	0.358778
일방적	0.866102	0.273102	-0.000527	-0.221970	0.331767	-0.028645	0.889076	-0.000527	0.009869	0.088077	...	0.052601	0.803882	-0.076013	0.968037	0.094145	0.094145	1.000000	-0.022689	-0.112132	-0.274782
메시지	0.307400	0.949499	0.036084	0.953013	0.906797	-0.046929	0.406599	0.036084	0.754965	0.639759	...	0.532968	0.575711	0.986843	0.218188	0.689699	0.689699	-0.022689	1.000000	0.294424	0.906593
보험	0.356009	0.277950	-0.008466	0.098076	0.382240	-0.007648	0.150577	-0.008466	0.283526	0.890524	...	0.145862	0.093317	0.310957	-0.064936	0.791648	0.791648	-0.112132	0.294424	1.000000	-0.007226
현종	-0.059951	0.777660	-0.058237	0.977899	0.700025	0.023601	0.105389	-0.058237	0.594916	0.328859	...	0.448714	0.313404	0.920700	-0.033307	0.358778	0.358778	-0.274782	0.906593	-0.007226	1.000000

2979 rows × 2979 columns

2. 문서 ~ 문서 Cosine Similarity

In [229]:

_U = np.diag(sigma).dot(Vt)

In [230]:

# 열이 문서차원 
_U.shape

Out[230]:

(40, 40)

In [231]:

# 문서에 대해서 
pd.DataFrame(_U.T.dot(_U) / (np.linalg.norm(_U.T, axis=1).reshape(40,1) * np.linalg.norm(_U, axis=0).reshape(1,40)))

Out[231]:

	0	1	2	3	4	5	6	7	8	9	...	30	31	32	33	34	35	36	37	38	39
0	1.000000	0.108239	0.156720	0.105557	0.200759	0.140802	0.165398	0.162262	0.127839	0.154765	...	0.105324	0.167264	0.163815	0.174660	0.168834	0.196904	0.084909	0.194896	0.147117	0.181832
1	0.108239	1.000000	0.111597	0.066630	0.117507	0.067870	0.106301	0.122210	0.061622	0.170959	...	0.134388	0.152911	0.098703	0.105238	0.118682	0.110732	0.074415	0.103110	0.076824	0.073039
2	0.156720	0.111597	1.000000	0.083717	0.163770	0.086145	0.216441	0.160862	0.103417	0.207129	...	0.125298	0.102923	0.256889	0.207783	0.125533	0.132740	0.119377	0.175306	0.186685	0.144210
3	0.105557	0.066630	0.083717	1.000000	0.091484	0.124947	0.157352	0.102156	0.223209	0.124501	...	0.119617	0.087149	0.087507	0.192432	0.100388	0.121211	0.119871	0.123699	0.131727	0.132283
4	0.200759	0.117507	0.163770	0.091484	1.000000	0.096339	0.221810	0.155432	0.112661	0.160604	...	0.160238	0.160223	0.156919	0.172536	0.370625	0.138318	0.122002	0.396151	0.131529	0.156760
5	0.140802	0.067870	0.086145	0.124947	0.096339	1.000000	0.125935	0.140807	0.161465	0.116972	...	0.087409	0.104623	0.080040	0.139343	0.061440	0.167562	0.158167	0.105388	0.125194	0.101799
6	0.165398	0.106301	0.216441	0.157352	0.221810	0.125935	1.000000	0.280120	0.150660	0.213741	...	0.159721	0.111519	0.185105	0.291472	0.136017	0.151077	0.177674	0.183820	0.311542	0.228800
7	0.162262	0.122210	0.160862	0.102156	0.155432	0.140807	0.280120	1.000000	0.109698	0.167460	...	0.090938	0.104195	0.168144	0.215692	0.103256	0.133392	0.178663	0.134178	0.243622	0.191615
8	0.127839	0.061622	0.103417	0.223209	0.112661	0.161465	0.150660	0.109698	1.000000	0.181725	...	0.137560	0.123136	0.095381	0.167798	0.114141	0.459609	0.121947	0.101484	0.154947	0.150102
9	0.154765	0.170959	0.207129	0.124501	0.160604	0.116972	0.213741	0.167460	0.181725	1.000000	...	0.193488	0.155282	0.168394	0.176336	0.166667	0.155501	0.153783	0.153574	0.174274	0.169706
10	0.147077	0.166812	0.185262	0.087149	0.157461	0.104623	0.231004	0.185598	0.132987	0.484481	...	0.161126	0.125000	0.204407	0.165606	0.172787	0.142246	0.156135	0.144229	0.177131	0.201414
11	0.114494	0.076651	0.181610	0.085431	0.134047	0.115380	0.134699	0.114909	0.200980	0.164399	...	0.118462	0.116409	0.126554	0.121756	0.119563	0.181274	0.094294	0.116138	0.119811	0.141644
12	0.218325	0.092083	0.162331	0.146615	0.167308	0.121008	0.251269	0.166391	0.144493	0.188092	...	0.146829	0.126176	0.149318	0.203980	0.147481	0.131619	0.137200	0.151652	0.156448	0.154691
13	0.166330	0.116384	0.178725	0.171150	0.164468	0.111745	0.311226	0.260496	0.174114	0.190678	...	0.111029	0.124035	0.133440	0.264097	0.157585	0.205840	0.159079	0.153337	0.263645	0.206375
14	0.105304	0.088828	0.144065	0.114914	0.105915	0.109656	0.116346	0.112951	0.134889	0.136083	...	0.141639	0.136931	0.117851	0.153577	0.086598	0.086568	0.095021	0.104494	0.125046	0.101259
15	0.134137	0.109918	0.159576	0.265733	0.128491	0.158606	0.177843	0.102987	0.169532	0.156009	...	0.138786	0.098194	0.153456	0.190737	0.094551	0.135266	0.117580	0.119262	0.127429	0.157497
16	0.132806	0.106693	0.166769	0.142448	0.155486	0.133834	0.584210	0.299896	0.119713	0.174801	...	0.106873	0.092387	0.146795	0.235382	0.112682	0.137482	0.137908	0.128856	0.302116	0.186702
17	0.197581	0.082102	0.235051	0.102945	0.411159	0.091545	0.188189	0.169236	0.081455	0.168753	...	0.105739	0.131250	0.211804	0.204937	0.368189	0.104550	0.127347	0.443502	0.145070	0.193094
18	0.159347	0.091440	0.162486	0.044587	0.145371	0.071369	0.167672	0.119943	0.084238	0.130744	...	0.125615	0.080397	0.613317	0.145246	0.089144	0.091489	0.102705	0.120473	0.142926	0.130569
19	0.107410	0.106594	0.150329	0.153808	0.145255	0.059427	0.174519	0.114140	0.134889	0.197320	...	0.176505	0.121716	0.109994	0.155496	0.207835	0.117732	0.089591	0.170534	0.129358	0.159882
20	0.157164	0.109044	0.152975	0.131928	0.141424	0.105587	0.201708	0.166696	0.146414	0.200024	...	0.162611	0.110096	0.159901	0.199691	0.151067	0.125286	0.128924	0.177694	0.183314	0.168722
21	0.185214	0.128766	0.110169	0.107636	0.143299	0.114860	0.144294	0.136733	0.079083	0.176443	...	0.169521	0.144093	0.248031	0.162331	0.142271	0.109315	0.087237	0.158341	0.186685	0.115368
22	0.118007	0.085323	0.105290	0.143635	0.169559	0.099100	0.162976	0.126577	0.120929	0.114374	...	0.140411	0.056833	0.117393	0.139833	0.083181	0.103478	0.110286	0.098363	0.135299	0.143335
23	0.157388	0.112073	0.136171	0.174153	0.143910	0.129769	0.250298	0.169625	0.177169	0.138675	...	0.148039	0.082690	0.124544	0.208670	0.126068	0.141147	0.124497	0.136300	0.216775	0.195514
24	0.167120	0.112779	0.182909	0.106617	0.179296	0.121258	0.190799	0.143405	0.137006	0.208768	...	0.165996	0.154533	0.116376	0.164518	0.141360	0.168526	0.120640	0.137974	0.131389	0.162392
25	0.181424	0.135699	0.193503	0.094526	0.197757	0.113479	0.230399	0.190714	0.144245	0.431174	...	0.142402	0.138594	0.186704	0.193881	0.154341	0.130287	0.149191	0.158921	0.169071	0.183636
26	0.128119	0.098811	0.188996	0.137660	0.137453	0.103289	0.316136	0.237236	0.153173	0.204199	...	0.137862	0.133278	0.108335	0.191522	0.138474	0.112345	0.142033	0.170867	0.214044	0.212678
27	0.155891	0.105201	0.170619	0.099453	0.418121	0.083785	0.160755	0.126729	0.079875	0.154449	...	0.148390	0.162169	0.108556	0.147780	0.424888	0.123028	0.108515	0.351459	0.117454	0.132546
28	0.168417	0.101477	0.166964	0.127237	0.198971	0.109376	0.170571	0.133109	0.124646	0.169278	...	0.209105	0.108148	0.153580	0.163748	0.158286	0.129222	0.104932	0.209459	0.202291	0.170471
29	0.125503	0.109992	0.176451	0.119717	0.202189	0.103479	0.472716	0.161026	0.141277	0.178159	...	0.165267	0.142866	0.170251	0.223595	0.134038	0.137564	0.125014	0.199260	0.228922	0.167454
30	0.105324	0.134388	0.125298	0.119617	0.160238	0.087409	0.159721	0.090938	0.137560	0.193488	...	1.000000	0.113385	0.130971	0.169419	0.109178	0.122237	0.103824	0.093448	0.086256	0.137964
31	0.167264	0.152911	0.102923	0.087149	0.160223	0.104623	0.111519	0.104195	0.123136	0.155282	...	0.113385	1.000000	0.086066	0.115661	0.162623	0.132763	0.126395	0.141939	0.108641	0.099247
32	0.163815	0.098703	0.256889	0.087507	0.156919	0.080040	0.185105	0.168144	0.095381	0.168394	...	0.130971	0.086066	1.000000	0.190041	0.104973	0.122426	0.129580	0.130043	0.164646	0.135665
33	0.174660	0.105238	0.207783	0.192432	0.172536	0.139343	0.291472	0.215692	0.167798	0.176336	...	0.169419	0.115661	0.190041	1.000000	0.179543	0.113671	0.158307	0.168983	0.219027	0.204414
34	0.168834	0.118682	0.125533	0.100388	0.370625	0.061440	0.136017	0.103256	0.114141	0.166667	...	0.109178	0.162623	0.104973	0.179543	1.000000	0.123373	0.113353	0.379746	0.126745	0.149532
35	0.196904	0.110732	0.132740	0.121211	0.138318	0.167562	0.151077	0.133392	0.459609	0.155501	...	0.122237	0.132763	0.122426	0.113671	0.123373	1.000000	0.131142	0.104208	0.123629	0.146158
36	0.084909	0.074415	0.119377	0.119871	0.122002	0.158167	0.177674	0.178663	0.121947	0.153783	...	0.103824	0.126395	0.129580	0.158307	0.113353	0.131142	1.000000	0.131744	0.180162	0.132823
37	0.194896	0.103110	0.175306	0.123699	0.396151	0.105388	0.183820	0.134178	0.101484	0.153574	...	0.093448	0.141939	0.130043	0.168983	0.379746	0.104208	0.131744	1.000000	0.218004	0.173215
38	0.147117	0.076824	0.186685	0.131727	0.131529	0.125194	0.311542	0.243622	0.154947	0.174274	...	0.086256	0.108641	0.164646	0.219027	0.126745	0.123629	0.180162	0.218004	1.000000	0.188621
39	0.181832	0.073039	0.144210	0.132283	0.156760	0.101799	0.228800	0.191615	0.150102	0.169706	...	0.137964	0.099247	0.135665	0.204414	0.149532	0.146158	0.132823	0.173215	0.188621	1.000000

40 rows × 40 columns

In [232]:

# K = 5 바꾸기
_U = _sigma.dot(Vt[:5,:])
_U.shape

Out[232]:

(5, 40)

In [233]:

# 차원을 줄이면서 정보를 버리게 되면서 관계를 찾아내게 됨.
pd.DataFrame(_U.T.dot(_U) / (np.linalg.norm(_U.T, axis=1).reshape(40,1) * np.linalg.norm(_U, axis=0).reshape(1,40)))

Out[233]:

	0	1	2	3	4	5	6	7	8	9	...	30	31	32	33	34	35	36	37	38	39
0	1.000000	0.958310	0.910656	0.601013	0.807539	0.408136	0.586977	0.686733	0.594785	0.889547	...	0.867824	0.937299	0.896779	0.841393	0.794127	0.750368	0.153827	0.738988	0.584550	0.883858
1	0.958310	1.000000	0.953627	0.716222	0.724401	0.241253	0.647651	0.738493	0.689466	0.975465	...	0.954980	0.946011	0.965065	0.914479	0.728205	0.812651	0.284611	0.635115	0.635457	0.950619
2	0.910656	0.953627	1.000000	0.524699	0.701582	0.196674	0.828351	0.878883	0.495014	0.959872	...	0.850876	0.848479	0.993020	0.959012	0.688865	0.645001	0.291911	0.633335	0.820278	0.986170
3	0.601013	0.716222	0.524699	1.000000	0.188122	0.185708	0.218282	0.316045	0.993289	0.734031	...	0.882100	0.679725	0.600427	0.621611	0.227909	0.960456	0.208869	0.057475	0.177720	0.613423
4	0.807539	0.724401	0.701582	0.188122	1.000000	0.140888	0.297581	0.398194	0.148587	0.582854	...	0.552737	0.814369	0.657489	0.485122	0.995844	0.328145	0.171719	0.990572	0.323751	0.587497
5	0.408136	0.241253	0.196674	0.185708	0.140888	1.000000	0.176139	0.333109	0.281369	0.205208	...	0.157735	0.330308	0.186597	0.274260	0.082846	0.412737	0.158044	0.146330	0.246998	0.225939
6	0.586977	0.647651	0.828351	0.218282	0.297581	0.176139	1.000000	0.964873	0.209543	0.749117	...	0.525211	0.435024	0.810860	0.878808	0.257269	0.338739	0.226489	0.259782	0.989859	0.848187
7	0.686733	0.738493	0.878883	0.316045	0.398194	0.333109	0.964873	1.000000	0.312789	0.822013	...	0.608014	0.592375	0.873461	0.923393	0.359996	0.470151	0.412025	0.357774	0.982909	0.898096
8	0.594785	0.689466	0.495014	0.993289	0.148587	0.281369	0.209543	0.312789	1.000000	0.706928	...	0.854346	0.655774	0.568757	0.608492	0.181375	0.970203	0.181032	0.020686	0.172565	0.590577
9	0.889547	0.975465	0.959872	0.734031	0.582854	0.205208	0.749117	0.822013	0.706928	1.000000	...	0.949293	0.876041	0.983424	0.969370	0.586095	0.813540	0.349845	0.486997	0.733284	0.983190
10	0.878758	0.957668	0.984351	0.629216	0.603601	0.201077	0.823090	0.885690	0.600720	0.989150	...	0.894669	0.847585	0.997310	0.983077	0.598283	0.728247	0.371682	0.521303	0.814624	0.996572
11	0.916508	0.963488	0.868954	0.855171	0.563203	0.338354	0.570313	0.669466	0.847144	0.952764	...	0.980990	0.904034	0.898254	0.890056	0.570315	0.932602	0.234088	0.455668	0.548630	0.903220
12	0.900172	0.963828	0.977855	0.665735	0.582659	0.251655	0.815028	0.871662	0.646225	0.990653	...	0.912305	0.843456	0.988736	0.988782	0.574397	0.767467	0.290010	0.494901	0.798233	0.996995
13	0.691643	0.772248	0.900892	0.407152	0.358629	0.189172	0.978771	0.971223	0.393493	0.864603	...	0.684799	0.574413	0.899156	0.955179	0.330615	0.513658	0.275582	0.297488	0.963463	0.931062
14	0.796677	0.865844	0.691774	0.957370	0.454057	0.275666	0.314821	0.436811	0.947311	0.843510	...	0.958005	0.856086	0.745382	0.726257	0.482942	0.978669	0.240214	0.334610	0.289384	0.742350
15	0.746206	0.818903	0.649100	0.974984	0.328569	0.340988	0.334488	0.450350	0.978158	0.819814	...	0.929719	0.788301	0.708522	0.727169	0.352332	0.996193	0.224360	0.205899	0.307055	0.722231
16	0.605233	0.657308	0.825567	0.257452	0.278093	0.258469	0.995546	0.973922	0.257968	0.759173	...	0.541108	0.451355	0.811411	0.891943	0.235036	0.388981	0.236054	0.237298	0.989158	0.854186
17	0.808494	0.730971	0.736226	0.151768	0.995138	0.139984	0.373288	0.466323	0.111420	0.603027	...	0.542736	0.803884	0.689104	0.524283	0.986036	0.301321	0.200260	0.989582	0.402186	0.622084
18	0.892671	0.953725	0.996797	0.566187	0.649545	0.184468	0.840094	0.888527	0.536404	0.974408	...	0.869054	0.837071	0.997589	0.975280	0.639631	0.674402	0.311083	0.574531	0.828668	0.995300
19	0.905989	0.978473	0.924389	0.746682	0.677261	0.063765	0.612199	0.665533	0.705659	0.960317	...	0.969796	0.892452	0.939774	0.883590	0.693340	0.791577	0.188149	0.576504	0.574882	0.925134
20	0.878052	0.967490	0.953649	0.737631	0.563116	0.135205	0.749758	0.794869	0.707095	0.992507	...	0.954108	0.841380	0.973548	0.962528	0.568476	0.797863	0.249797	0.462234	0.717145	0.977632
21	0.955619	0.965590	0.986032	0.570103	0.703696	0.318615	0.791321	0.852764	0.556003	0.955567	...	0.869518	0.870447	0.976060	0.954793	0.685605	0.708150	0.217480	0.632017	0.781733	0.977315
22	0.838797	0.925685	0.799515	0.922934	0.489714	0.260972	0.474283	0.588642	0.906387	0.926087	...	0.985676	0.887047	0.849432	0.834757	0.512177	0.961760	0.324323	0.373421	0.454321	0.847407
23	0.862530	0.932959	0.927109	0.737830	0.463017	0.340930	0.791828	0.860435	0.732345	0.977691	...	0.913728	0.810802	0.951068	0.984406	0.454117	0.836237	0.309351	0.366982	0.776460	0.972927
24	0.943759	0.976566	0.981451	0.638977	0.660225	0.328094	0.781396	0.858643	0.624565	0.979587	...	0.902487	0.888025	0.985130	0.970721	0.647630	0.766618	0.295842	0.580502	0.775599	0.986951
25	0.894754	0.963459	0.990445	0.614453	0.633409	0.209813	0.818996	0.882100	0.586309	0.986701	...	0.892043	0.857913	0.999177	0.979339	0.626600	0.719818	0.354568	0.553601	0.811137	0.996595
26	0.651799	0.717923	0.877667	0.291973	0.361506	0.164972	0.994949	0.970158	0.277900	0.808862	...	0.603518	0.513165	0.863757	0.917427	0.326893	0.407877	0.238809	0.314520	0.981657	0.895225
27	0.800897	0.720466	0.686825	0.203671	0.999145	0.127015	0.266353	0.369942	0.162634	0.575360	...	0.557085	0.817353	0.645286	0.468378	0.998252	0.336435	0.171400	0.987824	0.291564	0.573031
28	0.972692	0.980155	0.972995	0.581036	0.791578	0.230280	0.696272	0.761106	0.554550	0.941304	...	0.889792	0.911457	0.961384	0.902540	0.783988	0.700523	0.185977	0.718562	0.681783	0.946564
29	0.687800	0.726104	0.891008	0.237345	0.441843	0.170235	0.985454	0.955874	0.222910	0.794587	...	0.587234	0.529346	0.863747	0.901141	0.403078	0.367965	0.176416	0.402702	0.971719	0.890142
30	0.867824	0.954980	0.850876	0.882100	0.552737	0.157735	0.525211	0.608014	0.854346	0.949293	...	1.000000	0.888071	0.888871	0.859748	0.575306	0.912458	0.240344	0.437103	0.491051	0.883715
31	0.937299	0.946011	0.848479	0.679725	0.814369	0.330308	0.435024	0.592375	0.655774	0.876041	...	0.888071	1.000000	0.863968	0.768884	0.824075	0.796010	0.407968	0.741513	0.455660	0.824169
32	0.896779	0.965065	0.993020	0.600427	0.657489	0.186597	0.810860	0.873461	0.568757	0.983424	...	0.888871	0.863968	1.000000	0.971101	0.651919	0.704399	0.355848	0.579518	0.803377	0.993330
33	0.841393	0.914479	0.959012	0.621611	0.485122	0.274260	0.878808	0.923393	0.608492	0.969370	...	0.859748	0.768884	0.971101	1.000000	0.469962	0.728023	0.312030	0.401703	0.864630	0.991164
34	0.794127	0.728205	0.688865	0.227909	0.995844	0.082846	0.257269	0.359996	0.181375	0.586095	...	0.575306	0.824075	0.651919	0.469962	1.000000	0.347335	0.187967	0.980361	0.279727	0.576866
35	0.750368	0.812651	0.645001	0.960456	0.328145	0.412737	0.338739	0.470151	0.970203	0.813540	...	0.912458	0.796010	0.704399	0.728023	0.347335	1.000000	0.267912	0.210440	0.322085	0.718980
36	0.153827	0.284611	0.291911	0.208869	0.171719	0.158044	0.226489	0.412025	0.181032	0.349845	...	0.240344	0.407968	0.355848	0.312030	0.187967	0.267912	1.000000	0.166439	0.328746	0.309756
37	0.738988	0.635115	0.633335	0.057475	0.990572	0.146330	0.259782	0.357774	0.020686	0.486997	...	0.437103	0.741513	0.579518	0.401703	0.980361	0.210440	0.166439	1.000000	0.296486	0.505994
38	0.584550	0.635457	0.820278	0.177720	0.323751	0.246998	0.989859	0.982909	0.172565	0.733284	...	0.491051	0.455660	0.803377	0.864630	0.279727	0.322085	0.328746	0.296486	1.000000	0.834408
39	0.883858	0.950619	0.986170	0.613423	0.587497	0.225939	0.848187	0.898096	0.590577	0.983190	...	0.883715	0.824169	0.993330	0.991164	0.576866	0.718980	0.309756	0.505994	0.834408	1.000000

40 rows × 40 columns

문서 내 단어간의 유사도

In [234]:

# K = 5
_sigma = np.diag(sigma[:5])
_U = U[:,:5].dot(_sigma)

In [236]:

cluster = pd.DataFrame(_U,index=coldata)
cluster

Out[236]:

	0	1	2	3	4
발언	2.369581	-0.836695	-0.144384	0.813850	-1.161821
30	1.232922	0.046106	0.403684	0.361928	0.610832
포스	0.197766	0.577285	0.536741	-0.425677	-0.366042
절전	0.156872	0.066606	0.141700	0.041088	0.251585
두고	0.274984	-0.061751	0.049259	0.090926	0.132552
분산	0.239403	0.607510	-0.737459	-0.079645	0.104458
촉구	1.218676	-0.186607	-0.005394	0.705432	-0.459797
강물	0.197766	0.577285	0.536741	-0.425677	-0.366042
평가	1.062935	0.561659	0.895490	-0.358161	0.318224
사이트	0.430040	-0.270124	-0.021477	-0.173082	0.159062
나발	0.197766	0.577285	0.536741	-0.425677	-0.366042
일훈	0.179643	-0.036318	0.028300	0.065591	0.018432
참사	0.316409	-0.104357	0.010679	0.058574	-0.007663
부업	0.177889	-0.211081	-0.075403	-0.313003	0.016532
3000	0.146436	-0.177308	-0.064159	-0.267153	0.032358
강경	0.362483	-0.063681	-0.015307	0.398809	-0.460040
경원	1.995896	-0.365314	-0.016019	1.401452	-1.140363
말씀	0.742564	0.950088	-0.251962	-0.805080	-0.238978
보안법	0.205547	-0.068561	0.031243	0.098642	0.089656
채널	3.088727	-0.969424	0.316349	-0.089523	0.874423
자동차	0.197766	0.577285	0.536741	-0.425677	-0.366042
만이	0.710152	-0.047355	0.017757	0.596906	-0.545343
민주화	0.222092	-0.294330	-0.135534	-0.482251	-0.051334
신기	0.263535	-0.033261	0.031035	0.157818	-0.163540
여건	0.123399	0.071824	0.110044	0.019551	0.180713
시베리아	0.156872	0.066606	0.141700	0.041088	0.251585
호통	0.545251	-0.670008	-0.284991	-1.022469	-0.056182
시스	0.525090	-0.049418	0.099217	0.310381	-0.003447
호치민	0.156872	0.066606	0.141700	0.041088	0.251585
교통부	0.144714	-0.055486	0.067303	-0.073055	0.026855
...	...	...	...	...	...
부끄러움	0.525449	-0.660899	-0.277153	-1.015897	0.011664
가능	0.672915	0.118303	0.252102	0.357578	0.371721
벤처	0.144714	-0.055486	0.067303	-0.073055	0.026855
어제	0.471845	-0.127601	0.019384	0.112517	0.051449
칭호	0.130440	0.022425	0.087373	0.003087	0.214820
아이러니	0.111598	-0.039413	0.027120	0.003489	-0.010765
기준	0.473540	-0.100998	0.040113	0.125444	-0.284033
개입	0.593166	-0.045645	0.134718	0.159660	0.306130
있음	0.158328	0.084021	0.149270	0.105622	0.387031
매진	0.596566	-0.169524	0.178759	-0.408556	0.641623
소란	0.177889	-0.211081	-0.075403	-0.313003	0.016532
아시아	0.461301	0.544024	0.567776	-0.267858	-0.529581
확립	0.162838	-0.091667	0.028184	-0.025076	0.024234
균형	0.419046	0.571192	-0.709159	-0.014054	0.122890
이달	0.113932	-0.003755	0.060966	0.011110	0.086732
의뢰	0.699156	-0.336653	-0.045144	-0.045728	0.046333
변화	0.266071	-0.039293	0.069045	0.070967	0.084302
혼자	0.197166	-0.015012	0.029416	0.112143	-0.021130
박지원	0.222092	-0.294330	-0.135534	-0.482251	-0.051334
부분적	0.158328	0.084021	0.149270	0.105622	0.387031
핵심	1.104896	1.158787	0.074804	-0.305325	0.238107
이제	0.317579	-0.037037	0.056990	0.208030	-0.054266
배제	0.241414	-0.010720	0.086217	0.053568	0.275454
감정	0.203647	-0.033283	-0.003976	0.196920	-0.140998
비판적	0.144714	-0.055486	0.067303	-0.073055	0.026855
문체부	0.144714	-0.055486	0.067303	-0.073055	0.026855
일방적	0.226454	-0.054046	-0.022182	0.254235	-0.290426
메시지	0.391314	0.000980	0.210000	0.075646	0.375803
보험	0.255946	-0.235336	-0.063964	-0.317574	0.053579
현종	0.158328	0.084021	0.149270	0.105622	0.387031

2979 rows × 5 columns

통상적으로는 첫 번째 컬럼이 변별력이 없기 때문에 버린다

In [251]:

# 2번째
temp = cluster.sort_values(by=[1],ascending=False)
ranking = temp[temp[1] > 0 ][1].to_dict()
print(list(ranking.keys())[:5])
print(list(ranking.values())[:5])

['성과', '제안', '지금', '역사', '보기']
[1.4484458092330283, 1.2961552344051241, 1.2302949924041506, 1.2142382529094597, 1.2031049279258397]

In [253]:

# 3번째
temp = cluster.sort_values(by=[2],ascending=False)
ranking = temp[temp[2] > 0 ][2].to_dict()
print(list(ranking.keys())[:5])
print(list(ranking.values())[:5])

['최근', '이번', '장관', '정상', '정책']
[1.261455103177059, 1.2496831743653438, 1.1790329587285966, 1.129323254832694, 1.0620208403636429]

In [254]:

temp = cluster.sort_values(by=[3],ascending=False)
ranking = temp[temp[3] > 0 ][3].to_dict()
print(list(ranking.keys())[:5])
print(list(ranking.values())[:5])

['대표', '원내', '단체', '교섭', '경원']
[1.7764712272588647, 1.5966030961602944, 1.5616105565313059, 1.4331206562274619, 1.4014516862874566]

In [255]:

temp = cluster.sort_values(by=[4],ascending=False)
ranking = temp[temp[4] > 0 ][4].to_dict()
print(list(ranking.keys())[:5])
print(list(ranking.values())[:5])

['국무', '청와대', '지시', '분석', '대통령']
[1.4519332749861376, 1.393792647999382, 1.3629042239054683, 1.3176724673796523, 1.2828038733880196]

2. LDA 구현하기

In [1]:

# 각 list가 documents를 말함. 
collection = [
    ["Hadoop", "Big Data", "HBase", "Java", "Spark", "Storm", "Cassandra"],
    ["NoSQL", "MongoDB", "Cassandra", "HBase", "Postgres"],
    ["Python", "scikit-learn", "scipy", "numpy", "statsmodels", "pandas"],
    ["R", "Python", "statistics", "regression", "probability"],
    ["machine learning", "regression", "decision trees", "libsvm"],
    ["Python", "R", "Java", "C++", "Haskell", "programming languages"],
    ["statistics", "probability", "mathematics", "theory"],
    ["machine learning", "scikit-learn", "Mahout", "neural networks"],
    ["neural networks", "deep learning", "Big Data", "artificial intelligence"],
    ["Hadoop", "Java", "MapReduce", "Big Data"],
    ["statistics", "R", "statsmodels"],
    ["C++", "deep learning", "artificial intelligence", "probability"],
    ["pandas", "R", "Python"],
    ["databases", "HBase", "Postgres", "MySQL", "MongoDB"],
    ["libsvm", "regression", "support vector machines"]
]

In [3]:

type(collection)

Out[3]:

list

In [401]:

from collections import defaultdict

documents = defaultdict(lambda: defaultdict(int)) # DTM
vocabulary = list()
# i : 문서제목 / d : i번째 문서 내 단어목록 
for i, d in enumerate(collection):  
    for term in d:
        documents[i][term.lower()] += 1
        vocabulary.append(term.lower())
        
vocabulary = list(set(vocabulary))

In [402]:

# D : docu, a,b
# alpha, beta 만들기 
a = 0.1
b = 0.1

K = 3 # 전체 토픽 수
M = len(documents) # 전체 문서의 수 
V = len(vocabulary) # 전체 단어의 수 
# N은 특정 문서마다 항상 다르다. 

# 특정 토픽에 몇 개의 단어가 있는지 -> 분모
# 특정 토픽 : sum(단어)
topicTermCount = defaultdict(int)

# 특정 문서의 단어에 상관없이 토픽 할당 횟수 
docTopicDistribution = defaultdict(lambda: defaultdict(int))
# [document][0번째토픽: 몇 개의 단어, 1번째 토픽:몇 개의 단어]

# 문서에 상관없이 특정 단어의 토픽 할당 횟수 
topicTermDistribution = defaultdict(lambda: defaultdict(int))
# [topic][vocab 0 : 몇 번, ... , n]

# Z_ml = m번째 문서 1번째 단어의 Topic
# M개의 문서만큼 -> N개의 단어 -> Topic
termTopicAssignmentMatrix = defaultdict(lambda:defaultdict(int))
# Z[documents][term] = topic
# n(i,(j,r)) = i번째 토픽의 횟수, j번째 문서의 r번째 단어

In [403]:

from random import randrange,seed

seed(0)

for i,termList in enumerate(collection):
    for j, term in enumerate(termList):
        token = term.lower()
        topic = randrange(K)
        
        topicTermCount[topic] += 1
        docTopicDistribution[i][topic] += 1
        topicTermDistribution[topic][term] += 1
        termTopicAssignmentMatrix[i][j] = topic

In [408]:

from random import random

def collapsedGibbsSampling(i,term):
    sampling = list()
    # k번째 토픽에 대한 확률     
    for k in range(K):
        sampling.append(likelighoodAlpha(i,k) * likelighoodBeta(k,term))
    # 0~1의 실수값을 가짐 
    threshold =  sum(sampling) * random()   
    
    for topicNo, topicProbability in enumerate(sampling):
        threshold -= topicProbability
        
        if threshold <= 0.0:
            return topicNo
    
#     print(sampling)
#     return termTopicAssignmentMatrix[i][term]

In [409]:

def likelighoodAlpha(i,k):
    return docTopicDistribution[i][k] + a

In [410]:

def likelighoodBeta(k,term):
    return  (topicTermDistribution[k][term] +b) / (topicTermCount[k] + b * V)

In [411]:

iterationNumber = 1000

for _ in range(iterationNumber):
    # m을 고정, l을 고정해야함 -> topicTermAssingnmentMatrix
    # m,l => i, j
    for i,termList in enumerate(collection):
        for j,term in enumerate(termList):
            topic = termTopicAssignmentMatrix[i][j]
            
            topicTermCount[topic] -= 1
            docTopicDistribution[i][topic] -= 1
            topicTermDistribution[topic][term] -= 1
            
            topic = collapsedGibbsSampling(i,term)
            
            topicTermCount[topic] += 1
            docTopicDistribution[i][topic] += 1
            topicTermDistribution[topic][term] += 1
            
            termTopicAssignmentMatrix[i][j] = topic

In [412]:

topicTermCount

Out[412]:

defaultdict(int, {1: 23, 0: 24, 2: 20})

In [413]:

topicTermDistribution

Out[413]:

defaultdict(<function __main__.<lambda>()>,
            {1: defaultdict(int,
                         {'Hadoop': 2,
                          'Big Data': 3,
                          'Java': 3,
                          'Storm': 1,
                          'Cassandra': 2,
                          'NoSQL': 1,
                          'MongoDB': 2,
                          'scipy': 0,
                          'R': 0,
                          'machine learning': 0,
                          'programming languages': 0,
                          'statistics': 0,
                          'probability': 0,
                          'Mahout': 0,
                          'neural networks': 0,
                          'deep learning': 0,
                          'artificial intelligence': 0,
                          'Python': 0,
                          'HBase': 3,
                          'Spark': 1,
                          'Postgres': 2,
                          'scikit-learn': 0,
                          'numpy': 0,
                          'statsmodels': 0,
                          'pandas': 0,
                          'regression': 0,
                          'decision trees': 0,
                          'libsvm': 0,
                          'C++': 0,
                          'Haskell': 0,
                          'mathematics': 0,
                          'theory': 0,
                          'MapReduce': 1,
                          'databases': 1,
                          'MySQL': 1,
                          'support vector machines': 0}),
             0: defaultdict(int,
                         {'HBase': 0,
                          'Postgres': 0,
                          'scikit-learn': 2,
                          'numpy': 1,
                          'statsmodels': 0,
                          'probability': 0,
                          'regression': 3,
                          'libsvm': 2,
                          'Haskell': 1,
                          'machine learning': 2,
                          'Big Data': 0,
                          'Hadoop': 0,
                          'Java': 0,
                          'C++': 2,
                          'pandas': 0,
                          'MongoDB': 0,
                          'Spark': 0,
                          'Storm': 0,
                          'Cassandra': 0,
                          'NoSQL': 0,
                          'Python': 2,
                          'scipy': 0,
                          'R': 0,
                          'statistics': 0,
                          'decision trees': 1,
                          'programming languages': 0,
                          'mathematics': 0,
                          'theory': 0,
                          'Mahout': 1,
                          'neural networks': 2,
                          'deep learning': 2,
                          'artificial intelligence': 2,
                          'MapReduce': 0,
                          'databases': 0,
                          'MySQL': 0,
                          'support vector machines': 1}),
             2: defaultdict(int,
                         {'Spark': 0,
                          'HBase': 0,
                          'Python': 2,
                          'pandas': 2,
                          'statistics': 3,
                          'regression': 0,
                          'decision trees': 0,
                          'C++': 0,
                          'mathematics': 1,
                          'theory': 1,
                          'scikit-learn': 0,
                          'neural networks': 0,
                          'artificial intelligence': 0,
                          'MapReduce': 0,
                          'R': 4,
                          'statsmodels': 2,
                          'deep learning': 0,
                          'databases': 0,
                          'MySQL': 0,
                          'support vector machines': 0,
                          'Hadoop': 0,
                          'Big Data': 0,
                          'Java': 0,
                          'Storm': 0,
                          'Cassandra': 0,
                          'NoSQL': 0,
                          'MongoDB': 0,
                          'Postgres': 0,
                          'scipy': 1,
                          'numpy': 0,
                          'probability': 3,
                          'machine learning': 0,
                          'libsvm': 0,
                          'Haskell': 0,
                          'programming languages': 1,
                          'Mahout': 0})})

In [414]:

docTopicDistribution

Out[414]:

defaultdict(<function __main__.<lambda>()>,
            {0: defaultdict(int, {1: 7, 0: 0, 2: 0}),
             1: defaultdict(int, {1: 5, 2: 0, 0: 0}),
             2: defaultdict(int, {2: 3, 0: 3, 1: 0}),
             3: defaultdict(int, {1: 0, 2: 3, 0: 2}),
             4: defaultdict(int, {1: 0, 0: 4, 2: 0}),
             5: defaultdict(int, {2: 3, 1: 1, 0: 2}),
             6: defaultdict(int, {1: 0, 2: 4, 0: 0}),
             7: defaultdict(int, {0: 4, 2: 0, 1: 0}),
             8: defaultdict(int, {2: 0, 1: 1, 0: 3}),
             9: defaultdict(int, {0: 0, 2: 0, 1: 4}),
             10: defaultdict(int, {2: 3, 0: 0, 1: 0}),
             11: defaultdict(int, {0: 3, 2: 1, 1: 0}),
             12: defaultdict(int, {0: 0, 2: 3, 1: 0}),
             13: defaultdict(int, {2: 0, 0: 0, 1: 5}),
             14: defaultdict(int, {0: 3, 2: 0, 1: 0})})

In [415]:

termTopicAssignmentMatrix

Out[415]:

defaultdict(<function __main__.<lambda>()>,
            {0: defaultdict(int, {0: 1, 1: 1, 2: 1, 3: 1, 4: 1, 5: 1, 6: 1}),
             1: defaultdict(int, {0: 1, 1: 1, 2: 1, 3: 1, 4: 1}),
             2: defaultdict(int, {0: 0, 1: 0, 2: 2, 3: 0, 4: 2, 5: 2}),
             3: defaultdict(int, {0: 2, 1: 0, 2: 2, 3: 0, 4: 2}),
             4: defaultdict(int, {0: 0, 1: 0, 2: 0, 3: 0}),
             5: defaultdict(int, {0: 2, 1: 2, 2: 1, 3: 0, 4: 0, 5: 2}),
             6: defaultdict(int, {0: 2, 1: 2, 2: 2, 3: 2}),
             7: defaultdict(int, {0: 0, 1: 0, 2: 0, 3: 0}),
             8: defaultdict(int, {0: 0, 1: 0, 2: 1, 3: 0}),
             9: defaultdict(int, {0: 1, 1: 1, 2: 1, 3: 1}),
             10: defaultdict(int, {0: 2, 1: 2, 2: 2}),
             11: defaultdict(int, {0: 0, 1: 0, 2: 0, 3: 2}),
             12: defaultdict(int, {0: 2, 1: 2, 2: 2}),
             13: defaultdict(int, {0: 1, 1: 1, 2: 1, 3: 1, 4: 1}),
             14: defaultdict(int, {0: 0, 1: 0, 2: 0})})