Import Libraries 1 2 3 4 5 6 7 import requests from bs4 import BeautifulSoup import pandas as pd from datetime import datetime import time # time.sleep() import re 뉴스 검색 결과에서 네이버 뉴스 추출 네이버 뉴스 검색 결과 URL 분석 1 2 3 4 https://search.naver.com/search.naver? where=news& sm=tab_jum& <!-- 불필요 --> query=데이터분석 네이버 뉴스 검색 URL 불러오기 1 2 3 4 5 query = input() # 데이터분석 url = f'https://search.naver.com/search.naver?where=news&query={query}' web = requests....
Scikit-learn Library Traditional Machine Learning (vs DL, 인공신경을 썼는지의 여부) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 from sklearn import datasets, linear_model, model_selection, metrics data_total = datasets.load_boston() x = data_total.data y = data_total.target train_x, test_x, train_y, test_y = model_selection.train_test_split(x, y, test_size=0.3) # 학습 전의 모델 생성 model = linear_model.LinearRegression() # 모델에 학습 데이터를 넣으면서 학습 진행 model.fit(train_x, train_y) # 모델에게 새로운 데이터를 주면서 예측 요구 predictions = model....
Tokenizing Text Data Import Libraries 1 2 3 import nltk from nltk.corpus import stopwords from collections import Counter Set Stopwords 1 2 3 4 5 6 stop_words = stopwords.words("english") stop_words.append(',') stop_words.append('.') stop_words.append('’') stop_words.append('”') stop_words.append('—') Open Text Data 1 2 file = open('movie_review.txt', 'r', encoding="utf-8") lines = file.readlines() Tokenize 1 2 3 4 5 6 tokens = [] for line in lines: tokenized = nltk.word_tokenize(line) for token in tokenized: if token.lower() not in stop_words: tokens....
NLTK Library NLTK(Natural Language Toolkit)은 자연어 처리를 위한 라이브러리 1 2 3 import nltk nltk.download() 문장을 단어 수준에서 토큰화 1 2 3 sentence = 'NLTK is a leading platform for building Python programs to work with human language data. It provides easy-to-use interfaces to over 50 corpora and lexical resources such as WordNet, along with a suite of text processing libraries for classification, tokenization, stemming, tagging, parsing, and semantic reasoning, wrappers for industrial-strength NLP libraries, and an active discussion forum....
BeautifulSoup Library 1 2 from bs4 import BeautifulSoup from urllib.request import urlopen 단어의 검색 결과 출력 다음 어학사전 URL 불러오기 1 2 3 4 5 6 # 찾는 단어 입력 word = 'happiness' url = f'https://alldic.daum.net/search.do?q={word}' web = urlopen(url) web_page = BeautifulSoup(web, 'html.parser') 찾는 단어 출력 1 2 text_search = web_page.find('span', {'class': 'txt_emph1'}) print(f'찾는 단어: {text_search.get_text()}') 단어의 뜻 출력 1 2 3 4 list_search = web_page.find('ul', {'class': 'list_search'}) list_text = list_search....