Python pyparsing中的Group()是否需要后处理步骤来生成特定于所分析语言的结构
这是一个很好的例子。我已经把Python pyparsing中的Group()是否需要后处理步骤来生成特定于所分析语言的结构,python,pyparsing,Python,Pyparsing,这是一个很好的例子。我已经把pyparsing代码写成了语法的一对一翻译 我的DSL: response:success response:success AND extension:php OR extension:css response:sucess AND (extension:php OR extension:css) time >= 2020-01-09 time >= 2020-01-09 AND response:success OR os:windows NOT r
pyparsing
代码写成了语法的一对一翻译
我的DSL:
response:success
response:success AND extension:php OR extension:css
response:sucess AND (extension:php OR extension:css)
time >= 2020-01-09
time >= 2020-01-09 AND response:success OR os:windows
NOT reponse:success
response:success AND NOT os:windows
DSL以上的EBNF语法:
<expr> ::= <or>
<or> ::= <and> (" OR " <and>)*
<and> ::= <unary> ((" AND ") <unary>)*
<unary> ::= " NOT " <unary> | <equality>
<equality> ::= (<word> ":" <word>) | <comparison>
<comparison> ::= "(" <expr> ")" | (<word> (" > " | " >= " | " < " | " <= ") <word>)+
<word> ::= ("a" | "b" | "c" | "d" | "e" | "f" | "g"
| "h" | "i" | "j" | "k" | "l" | "m" | "n"
| "o" | "p" | "q" | "r" | "s" | "t" | "u"
| "v" | "w" | "x" | "y" | "z")+
这是我的输出:
[[<ExactMatchNode>, 'AND', <ExactMatchNode>, 'AND', <ExactMatchNode>], 'OR', ['NOT', <ExactMatchNode>]]
[0]:
[<ExactMatchNode>, 'AND', <ExactMatchNode>, 'AND', <ExactMatchNode>]
[1]:
OR
[2]:
['NOT', <ExactMatchNode>]
我现在可以使用.generate_query()
获取查询
但我在下面的输出中注意到的一点是:
[( AND-> [<ExactMatchNode response=200>, <ExactMatchNode time=22>, <ExactMatchNode rex=32>] ), 'OR', ( AND-> [( NOT-> ['NOT', <ExactMatchNode demo=good>] )] )]
[(和->[,]),'OR',(和->[(不->['NOT',]])]
第二个和->
附加在非
节点之前
我的问题仍然是一样的,这是使用pyparsing的正确方法,还是我错过了一些明显的错误方向?使用setParseAction附加节点类是我发现的从层次语法构建AST的最佳方法。如果使用此方法,则可能不需要组构造。之所以得到第二个,是因为解析器总是生成AndNode,即使只有一个操作数没有额外的
和操作数
如果存在操作数和操作数
(NOT和OR也是如此),则可以展开And表达式以仅附加AndNode parse操作类,例如:
And = (Unary + OneOrMore(AND + Unary)).addParseAction(AndNode) | Unary
Or = (And + OneOrMore(OR + And)).addParseAction(OrNode) | And
这就是pyparsing的infixNotation处理此类运算符的方式
我的解析器版本,使用infixNotation(我认为类几乎都是相同的,也许我调整了NotNode定义):
“”“
::=
:=(“或”)*
:=(“和”))*
:=“不是”|
::= ( ":" ) |
::=“(”)“|”((“>”|“>”=“<”|“[(和->[,]),]))]
响应:suces和(扩展名:php或扩展名:css)
[(和->[,(或->[,])]]
时间>=2020-01-09
[]
时间>=2020-01-09,响应:成功或操作系统:windows
[(或->[(和->[,]),]]
不回应:成功
[(非->)]
响应:成功,而不是操作系统:windows
[(和->[,(不->)]]
使用解析操作构建AST是我基本上决定执行此任务的方式。在执行此任务时,实际上不再需要组了-您的AST类正在为您处理组结构。非常感谢您提供如此详细的解释。过去几天我花了大部分时间学习解析概述、语法构造,以及如何在Github上玩pyparsing示例以理解位。我想最好是在添加一些注释或进一步询问任何内容之前,尽我所能理解所有内容。在我自己的机器上尝试并理解所有内容之前,我是否可以推迟接受答案?
[AND [<ExactMatchNode>, <ExactMatchNode>, <ExactMatchNode>]]
class NotNode:
def __init__(self, tokens):
self.negatearg = tokens
#print(f'**** \n {self.negatearg} \n +++')
def __repr__(self):
return f'( NOT-> {self.negatearg} )'
class AndNode:
def __init__(self, tokens):
self.conds = tokens[0][::2]
#print(f'**** \n {tokens} \n +++')
def __repr__(self):
return f'( AND-> {self.conds} )'
def generate_query(self):
result = [cond.generate_query() for cond in self.conds]
return result
class ExactMatchNode:
def __init__(self, tokens):
self.tokens = tokens[0]
#print(f'**** \n {tokens} \n +++')
def __repr__(self):
return f"<ExactMatchNode {self.tokens.searchKey}={self.tokens.searchValue}>"
def generate_query(self):
return {
'term' : { self.tokens[0]: self.tokens[2]}
}
unaryNot.setParseAction(NotNode)
Equality.setParseAction(ExactMatchNode)
And.setParseAction(AndNode)
[( AND-> [<ExactMatchNode response=200>, <ExactMatchNode time=22>, <ExactMatchNode rex=32>] ), 'OR', ( AND-> [( NOT-> ['NOT', <ExactMatchNode demo=good>] )] )]
And = (Unary + OneOrMore(AND + Unary)).addParseAction(AndNode) | Unary
Or = (And + OneOrMore(OR + And)).addParseAction(OrNode) | And
"""
<expr> ::= <or>
<or> ::= <and> (" OR " <and>)*
<and> ::= <unary> ((" AND ") <unary>)*
<unary> ::= " NOT " <unary> | <equality>
<equality> ::= (<word> ":" <word>) | <comparison>
<comparison> ::= "(" <expr> ")" | (<word> (" > " | " >= " | " < " | " <= ") <word>)+
<word> ::= ("a" | "b" | "c" | "d" | "e" | "f" | "g"
| "h" | "i" | "j" | "k" | "l" | "m" | "n"
| "o" | "p" | "q" | "r" | "s" | "t" | "u"
| "v" | "w" | "x" | "y" | "z")+
"""
import pyparsing as pp
NOT, AND, OR = map(pp.Keyword, "NOT AND OR".split())
word = ~(NOT | AND | OR) + pp.Word(pp.alphas.lower() + '-_')
date = pp.Regex(r"\d{4}-\d{2}-\d{2}")
operand = word | date
class ExactMatchNode:
def __init__(self, tokens):
self.tokens = tokens
def __repr__(self):
return "<ExactMatchNode>"
def query(self):
pass #generate the relevant elasticsearch query here?
class ComparisonNode:
def __init__(self, tokens):
self.tokens = tokens
def __repr__(self):
return "<ComparisonNode>"
def query(self):
pass #generate the relevant elasticsearch query here?
class NotNode:
def __init__(self, tokens):
self.negatearg = tokens[0][1]
#print(f'**** \n {self.negatearg} \n +++')
def __repr__(self):
return f'( NOT-> {self.negatearg} )'
class AndNode:
def __init__(self, tokens):
self.conds = tokens[0][::2]
#print(f'**** \n {tokens} \n +++')
def __repr__(self):
return f'( AND-> {self.conds} )'
def generate_query(self):
result = [cond.generate_query() for cond in self.conds]
return result
class OrNode:
def __init__(self, tokens):
self.conds = tokens[0][::2]
#print(f'**** \n {tokens} \n +++')
def __repr__(self):
return f'( OR-> {self.conds} )'
def generate_query(self):
result = [cond.generate_query() for cond in self.conds]
return result
expr = pp.infixNotation(operand,
[
(':', 2, pp.opAssoc.LEFT, ExactMatchNode),
(pp.oneOf("> >= < <="), 2, pp.opAssoc.LEFT, ComparisonNode),
(NOT, 1, pp.opAssoc.RIGHT, NotNode),
(AND, 2, pp.opAssoc.LEFT, AndNode),
(OR, 2, pp.opAssoc.LEFT, OrNode),
])
expr.runTests("""\
response:success
response:success AND extension:php OR extension:css
response:sucess AND (extension:php OR extension:css)
time >= 2020-01-09
time >= 2020-01-09 AND response:success OR os:windows
NOT reponse:success
response:success AND NOT os:windows
""")
response:success
[<ExactMatchNode>]
response:success AND extension:php OR extension:css
[( OR-> [( AND-> [<ExactMatchNode>, <ExactMatchNode>] ), <ExactMatchNode>] )]
response:sucess AND (extension:php OR extension:css)
[( AND-> [<ExactMatchNode>, ( OR-> [<ExactMatchNode>, <ExactMatchNode>] )] )]
time >= 2020-01-09
[<ComparisonNode>]
time >= 2020-01-09 AND response:success OR os:windows
[( OR-> [( AND-> [<ComparisonNode>, <ExactMatchNode>] ), <ExactMatchNode>] )]
NOT reponse:success
[( NOT-> <ExactMatchNode> )]
response:success AND NOT os:windows
[( AND-> [<ExactMatchNode>, ( NOT-> <ExactMatchNode> )] )]