Question

我在列表中有一堆句子，我想使用nltk库来阻止它。我一次只能抽出一个句子，但是在从列表中抽出句子并将它们重新组合在一起时遇到了问题。我缺少步骤了吗？ nltk库非常新。谢谢！

import nltk 
from nltk.stem import PorterStemmer 
ps = PorterStemmer()

# Success: one sentences at a time 
data = 'the gamers playing games'
words = word_tokenize(data)
for w in words:
    print(ps.stem(w))


# Fails: 

data_list = ['the gamers playing games',
            'higher scores',
            'sports']
words = word_tokenize(data_list)
for w in words:
    print(ps.stem(w))

# Error: TypeError: expected string or bytes-like object
# result should be: 
['the gamer play game',
 'higher score',
 'sport']

Answer 1

您正在将列表传递给word_tokenize，这是您无法做到的。

解决方案是将您的逻辑包装在另一个for-loop中，

data_list = ['the gamers playing games','higher scores','sports']
for words in data_list:
    words = tokenize.word_tokenize(words)
    for w in words:
        print(ps.stem(w))

>>>>the
gamer
play
game
higher
score
sport

Answer 2

import nltk
from nltk.tokenize import sent_tokenize
from nltk.stem import PorterStemmer

sentence = """At eight o'clock on Thursday morning, Arthur didn't feel very good. So i take him to hospital."""

sentence = sentence.lower()

word_tokens = nltk.word_tokenize(sentence)
sent_tokens = sent_tokenize(sentence)

stemmer = PorterStemmer()
stemmed_word = []
stemmed_sent = []
for token in word_tokens:
    stemmed_word.append(stemmer.stem(token))
    
for sent_token in sent_tokens:
    stemmed_sent.append(stemmer.stem(sent_token))
    
print(stemmed_word)
print(stemmed_sent)

python nltk-句子/短语的词干列表

2 个答案: