¡Hola, Mundo!

Bag of words(BoW) It is a way of extracting features from text, a representation of text that describes the occurrence of words within a document, for example, pour the sentence into the bag and shuffling the bag, any information about the order or structure of words in the bag is discarded. Review 1 : This movie is very scary and long Review 2 : This movie is not scary and is slow Review 3 : Th..

LM Language Model, is a model assigns probability to sequence in order to modeling language, in other words, finding the most natural word sequence Language Modeling Prediction to unknown word from given words. Conditional Probability It is the probability of an event occurring given that another event has already occurred. In this theory, mutually exclusive events are events that cannot occur s..

word_index Grand index to each words. print(tokenizer.word_index) >>> {'barber': 1, 'secret': 2, 'huge': 3, 'kept': 4, 'person': 5, 'word': 6, 'keeping': 7, 'good': 8, 'knew': 9, 'driving': 10, 'crazy': 11, 'went': 12, 'mountain': 13} word_counts Count the each words. print(tokenizer.word_counts) >>> OrderedDict([('barber', 8), ('person', 3), ('good', 1), ('huge', 5), ('knew', 1), ('secret', 6),..

normalization Integrate different words to make them the same word-such as US is same as USA integrate them as US. 1. WordNetLemmatizer If words have different forms, find the root word-such as the root of 'am, are, is' is 'be'. from nltk.stem import WordNetLemmatizer lemmatizer=WordNetLemmatizer() words=[ 'have', 'going', 'loves', 'lives', 'flies', 'dies', 'watched', 'has', 'starting'] print('b..

Corpus Natural Language Data NLP - Natural Language Processing gensim - It is an open source library for unsupervised topic modeling and natural language processing, using modern statistical machine learning. Scikit-learn - SciPy Toolkit. It features various classification, regression and clustering algorithms including support vector machines. NLTK - The Natural Language ToolKit, is a suite of ..