Hive 蜂巢句子的功能是如何打断每个句子的

在发布之前，我尝试了蜂巢句子功能并进行了一些搜索，但没有得到一个清晰的理解，我的问题是基于什么分隔符蜂巢句子功能打破了每个句子？蜂巢手册上说“适当的边界”这是什么意思？下面是我尝试的一个例子，我尝试在句子的不同点添加句号（.）和感叹号（！）。我得到了不同的结果，有人能解释一下吗

带句号（.） 输出-1阵列 用“！” 输出-2个阵列

---

如果你了解句子的功能（），它会消除你的疑虑

句子的定义（str）：

将str拆分为句子数组，其中每个句子都是一个数组 字里行间

例如：

SELECT sentences('Hello there! I am a UDF.') FROM src LIMIT 1;

[ ["Hello", "there"], ["I", "am", "a", "UDF"] ]



SELECT sentences('review . language') FROM movies;

[["review","language"]]

---

感叹号是一种位于句子末尾的标点符号。相关标点符号的其他示例包括句点和问号，它们也位于句子的末尾。但是根据

句子（）的定义，不必要的标点符号，例如英语中的句点和逗号，会自动被去除。因此，我们可以用！获得两个单词数组！。它完全涉及java.util.Locale.java

我不知道实际原因，但在句号（.）后观察到，如果您将空格和下一个单词的第一个字母作为大写字母，那么它就起作用了。 在这里，我改变了工作地点。然而，这是不需要的

Tokenizes a string of natural language text into words and sentences. Where each sentence is broken at the appropriate sentence boundary and returned as an array of words.

这是给下面的输出

[["Tokenizes","a","string","of","natural","language","text","into","words","and","sentences"],["Where","each","sentence","is","broken","at","the","appropriate","sentence","boundary","and","returned","as","an","array","of","words"]]

---

这毫无帮助。他的问题是，为什么它没有在这段时间分裂？

[["Tokenizes","a","string","of","natural","language","text","into","words","and","sentences"],["where","each","sentence","is","broken","at","the","appropriate","sentence","boundary","and","returned","as","an","array","of","words"]]

SELECT sentences('Hello there! I am a UDF.') FROM src LIMIT 1;

[ ["Hello", "there"], ["I", "am", "a", "UDF"] ]



SELECT sentences('review . language') FROM movies;

[["review","language"]]

Tokenizes a string of natural language text into words and sentences. Where each sentence is broken at the appropriate sentence boundary and returned as an array of words.

[["Tokenizes","a","string","of","natural","language","text","into","words","and","sentences"],["Where","each","sentence","is","broken","at","the","appropriate","sentence","boundary","and","returned","as","an","array","of","words"]]