搭建gos

Let’s explore the awesome builtin packages that ship with Go, and the cool stuff you can do with them. I will pick some of the more “obscure”, “complex”, and/or special purpose Go packages, especially ones that set the language’s builtin library apart from other languages.

让我们探索一下Go附带的很棒的内置软件包，以及您可以用它们完成的出色工作。 我将选择一些更“晦涩”，“复杂”和/或特殊用途的Go软件包，尤其是那些将语言的内置库与其他语言区分开的软件包。

The first stop is the go/ast package. This package is used to explore the syntax tree representation of a Go package, and can be used to perform static analysis, code linting, metaprogramming, and anything that requires a structured interpretation of Go source code.

第一站是打包包。 该软件包用于探索Go软件包的语法树表示形式，并且可以用于执行静态分析，代码整理，元编程以及任何需要对Go源代码进行结构化解释的事情。

This walkthrough is broken up into three parts. We will explore traversing the AST tree, type assertions, extracting literal values from the code, comment extraction, and some advanced struct reflection. By the end of it, we will have a utility capable of extracting documentation from a NATS publishing microservice that details the topics and message types that it produces. The output will be similar to a Swagger specification, but for event messaging systems.

本演练分为三个部分。 我们将探索遍历AST树，键入断言，从代码中提取文字值，注释提取以及一些高级结构反映。 到最后，我们将拥有一个实用程序，能够从NATS发布微服务中提取文档，其中详细说明了该文档所产生的主题和消息类型。 输出将类似于Swagger规范，但用于事件消息传递系统。

Part 1 of this article dives into the basics of the go/ast package. We will cover what an AST is, tree traversal, visitor functions, type assertions, and how the different types work.

本文的第1部分深入介绍了go / ast软件包的基础。 我们将介绍AST是什么，树遍历，访问者函数，类型断言以及不同类型如何工作。

Let’s start off simple. What is an AST, and what makes it a valuable tool?

让我们从简单开始。 什么是AST？什么使它成为有价值的工具？

抽象语法树 (Abstract Syntax Trees)

Abstract syntax trees, or AST for short, are tree representations of the syntax in a programming language’s source code. ASTs are used as a step in compilation, and are produced by the compilers syntax analysis phase. ASTs are similar to parse trees, however they are at a higher level in that they don’t include every detail of the syntax. For example, ASTs will provide more structure and context around parenthetical groupings, if/else statements, and looping constructs. In addition to this, ASTs will remove unnecessary information such as symbols, operators, and other tokens and instead, each node will represent a specific operation within the language. ASTs provide a semantic representation of a program, or a representation with meaning and contextual information, rather than just a structured representation of the tokens/text present in the source code.

抽象语法树 (简称AST)是编程语言源代码中语法的树表示形式。 AST被用作编译的步骤，并且由编译器语法分析阶段生成。 AST与解析树相似，但是它们 在较高级别上，因为它们没有包括语法的每个细节。 例如，AST将围绕括号分组，if / else语句和循环构造提供更多的结构和上下文。 除此之外，AST将删除不必要的信息，例如符号，运算符和其他标记，并且，每个节点将代表该语言内的特定操作。 AST提供了程序的语义表示或具有含义和上下文信息的表示，而不仅仅是源代码中存在的标记/文本的结构化表示。

Compilers will typically perform the following first steps when compiling source code. We will stop after the syntax/semantic analysis phase as this is the level at which Go’s ast package peers into.

编译源代码时，编译器通常会执行以下第一步。 我们将在语法/语义分析阶段之后停止，因为这是Go的ast软件包进入的级别。

Lexing: Given a table of all the tokens (symbols/words) that makeup the language, a lexer will tokens together and label accordingly.

乐兴：给定一个构成该语言的所有标记(符号/单词)的表，一个词法分析器将标记在一起并相应地进行标记。

Syntax Analysis: Each language has specific syntax rules it must follow. Given the output from the lexer, the syntax analyzer will build a parse tree that represents the operations and syntax of the language. For example, the following tokens “2,+,2” will be grouped into a parse tree as follows:

语法分析：每种语言都有必须遵循的特定语法规则。 给定词法分析器的输出，语法分析器将构建一个解析树，表示该语言的操作和语法。 例如，以下标记“ 2，+，2”将被分组为一个解析树，如下所示：

Simple Parse Tree for Addition 简单的加法解析树

Syntax Analysis/Semantic Analysis: The compiler will then verify the parse tree to make sure it has meaning in the context of the language, and will typically add semantics and context to the parse tree, as well as removing unnecessary constructs. This is where the AST is created. An AST of the above expression can be seen below:

语法分析/语义分析：然后，编译器将验证语法分析树以确保其在语言上下文中具有含义，并且通常将向语法分析树添加语义和上下文，并删除不必要的构造。 这是创建AST的地方。 上面的表达式的AST可以在下面看到：

Simple AST for Addition 简单的AST加法

Notice how the AST identifies the structure as a binary expression and labels the left-hand operand, the operator, and the right-hand operand accordingly?

注意AST如何将结构标识为二进制表达式，并相应地标记左操作数，运算符和右操作数？

Go’s AST package makes it very simple to traverse the language’s abstract syntax tree. These low level operations on the language constructs can be used to create powerful developer productivity tooling, and enhance testing and quality assurance efforts.

Go的AST软件包使遍历该语言的抽象语法树变得非常简单。 这些对语言构造的低级操作可用于创建功能强大的开发人员生产力工具，并增强测试和质量保证工作。

结构化文档提取 (Structured Documentation Extraction)

用例 (Use Case)

Being able to automatically extract documentation from source code is a powerful tool that can lead to increased developer productivity and end-user satisfaction. By using a tool that can auto-generate documentation from source code, developers can focus more on writing code and less on writing docs, while still providing knowledge for the end-user to consume. The below blog post dives into this concept on readability of source code if you are interested in the topic of self-documenting code.

能够从源代码自动提取文档是一个功能强大的工具，可以提高开发人员的工作效率和最终用户满意度。 通过使用可以从源代码自动生成文档的工具，开发人员可以将更多的精力放在编写代码上，而不必在编写文档上，而仍然为最终用户提供使用知识。 如果您对自助文档代码主题感兴趣，则以下博客文章深入探讨了有关源代码可读性的概念。

Go has a built-in tool called godoc that is able to automatically extract comments out of Go source code and build a web page that documents functions and types (example here). This tool is extremely powerful despite its simplicity, but what if we wanted something that provides more context and descriptions around functionality? Let’s leverage the go/ast package to document a NATS microservice.

Go具有一个称为godoc的内置工具，该工具能够自动从Go源代码中提取注释，并构建一个记录功能和类型的网页( 此处为示例)。 尽管该工具简单易用，但它仍然非常强大，但是如果我们想要一些可以提供更多上下文和功能描述的工具，该怎么办？ 让我们利用go / ast软件包来记录NATS微服务。

导航键 (NATS)

NATS is an event messaging system powered by Go that is lightweight, fault tolerant, and an absolute joy to use. One of the things that makes NATS so great is how simple it is to use. As such, NATS is used behind the scenes with microservices to perform message passing and request/response interactions between services. Applications can sometimes have upwards of hundreds of microservices, and so it can be easy to lose track of what microservices do what. Let’s build a utility that can parse a microservice’s source code, extract any NATS message publishers, and provide detail on the topics and messages they produce. We will essentially be building a “Swagger-like” specification for event producers. If you are interested in how NATS can be used, checkout the below post:

NATS是一种由Go驱动的事件消息传递系统，它轻巧，具有容错能力，并且绝对具有使用乐趣。 使NATS如此出色的原因之一是它的使用非常简单。 这样，NATS与微服务一起在后台使用，以执行消息传递以及服务之间的请求/响应交互。 应用程序有时可能具有数百个微服务，因此很容易失去对微服务功能的跟踪。 让我们构建一个实用程序，该实用程序可以解析微服务的源代码，提取任何NATS消息发布者，并提供有关它们产生的主题和消息的详细信息。 实际上，我们将为事件生产者建立一个“ 类似于Swagger ”的规范。 如果您对如何使用NATS感兴趣，请查看以下文章：

代码 (The Code)

We will use the following sample code as our microservice we want to document. This code will publish a message on a NATS topic every 5 seconds that says “Hello world!”. Probably the least exciting microservice ever created. But hey, it could be the basis for a heartbeat topic to guarantee service uptime right? :)

我们将使用以下示例代码作为我们要记录的微服务。 该代码将每5秒钟发布一次有关NATS主题的消息，消息为“ Hello world！”。 可能是有史以来最令人兴奋的微服务。 但是，这可能是确保服务正常运行的心跳话题的基础吗？ :)

package main import ( "log" "time" nats "github.com/nats-io/nats.go" ) type Msg struct { Subject string Body []byte } func main() { nc, _ := nats.Connect(nats.DefaultURL) ec, _ := nats.NewEncodedConn(nc, nats.JSON_ENCODER) msg := Msg{ Subject: "Hello World", Body: []byte("Hello!"), } log.Println("Running publisher") for { ec.Publish("heartbeat", msg) time.Sleep(time.Second * 5) } }

从简单开始 (Starting off Simple)

The first step is determining the topic the microserivce publishes to. We will start by traversing all function calls in this file since the “Publish” method is a function call, and the topic will be provided in the first argument of the call. With the contextual and semantic awareness of Go’s AST package, this is quite easy.

第一步是确定微服务发布到的主题。 我们将首先遍历此文件中的所有函数调用，因为“发布”方法是一个函数调用，并且主题将在调用的第一个参数中提供。 借助Go的AST软件包的上下文和语义意识，这非常容易。

package main import ( "fmt" "go/ast" "go/parser" "go/token" ) func main() { //Create a FileSet to work with fset := token.NewFileSet() //Parse the file and create an AST file, err := parser.ParseFile(fset, "../nats_publisher/main.go", nil, parser.ParseComments) if err != nil { panic(err) } ast.Inspect(file, func(n ast.Node) bool { // Find Function Call Statements funcCall, ok := n.(*ast.CallExpr) if ok { fmt.Println(funcCall.Fun) } return true }) }

Let’s break this code apart line-by-line. Starting off with the imports, we see that there are few more packages imported than initially advertised, however they are all part of Go’s compiler toolchain.

让我们逐行将这段代码分开。 从导入开始，我们看到导入的软件包比最初公布的要少，但是它们都是Go编译器工具链的一部分。

go/ast provides types and methods for exploring Go’s abstract syntax tree

go / ast提供了探索Go抽象语法树的类型和方法 go/parser provides methods for parsing source files and generating abstract syntax trees

go / parser提供了用于解析源文件和生成抽象语法树的方法 go/token provides types and methods for Go’s lexer process (tokenization)

go / token提供了Go的词法分析器过程(令牌化)的类型和方法

We start off by creating a FileSet. The FileSet provides facilities for tokenization and offsets (positions) in a group of source files. (https://golang.org/pkg/go/token/#FileSet).

我们首先创建一个FileSet。 FileSet为一组源文件中的标记化和偏移量(位置)提供了便利。 ( https://golang.org/pkg/go/token/#FileSet )。

//Create a FileSet to work with fset := token.NewFileSet()

We then use the go/parser package’s ParseFile function to parse the source code into an AST. In this case, the AST is available in the “file” variable, which is essentially a root node for the AST representing the entire source file.

然后，我们使用go / parser包的ParseFile函数将源代码解析为AST。 在这种情况下，AST在“文件”变量中可用，该变量实际上是AST的根节点，代表整个源文件。

file, err := parser.ParseFile(fset, "../nats_publisher/main.go", nil, parser.ParseComments)

The next part of the code is where things start to get interesting. The AST package exposes an “inspect” function that takes in a root node and a visitor function. A visitor function accepts a node as its parameter and returns a boolean. The inspect function allows us to traverse the entire AST of the source file, without having to worry about depth first or breadth first searching ourselves. Because let’s face it, the thing we hate the most about recursion is what we hate the most about recursion.

代码的下一部分是开始变得有趣的地方。 AST包公开了一个“检查”功能，该功能接受一个根节点和一个访问者功能。 访问者函数接受一个节点作为其参数，并返回一个布尔值。 检查功能使我们可以遍历源文件的整个AST，而不必担心深度优先或宽度优先搜索自己。 因为面对现实，我们最讨厌递归的就是最讨厌递归。

ast.Inspect(file, func(n ast.Node) bool { // Find Function Call Statements funcCall, ok := n.(*ast.CallExpr) if ok { fmt.Println(funcCall.Fun) } return true })

In the above visitor function, we attempt to cast the node to a “CallExpr” type which is a call expression. A call expression is when another function gets called from the current node. If the type matches, we print the function call.

在上面的visitor函数中，我们尝试将节点强制转换为“ CallExpr”类型，即调用表达式。 调用表达式是当从当前节点调用另一个函数时。 如果类型匹配，我们将打印函数调用。

增强 (Enhancing)

Now let’s enhance our visitor function to further inspect the function being called, as well as the parameters.

现在，让我们增强访问者函数，以进一步检查正在调用的函数以及参数。

package main import ( "fmt" "go/ast" "go/parser" "go/token" ) func main() { //Create a new token set for the file fset := token.NewFileSet() //Parse the file and create an AST file, err := parser.ParseFile(fset, "../nats_publisher/main.go", nil, parser.ParseComments) if err != nil { panic(err) } ast.Inspect(file, func(n ast.Node) bool { // Find Return Statements funcCall, ok := n.(*ast.CallExpr) if ok { mthd, ok := funcCall.Fun.(*ast.SelectorExpr) if ok { _, ok = mthd.X.(*ast.Ident) if ok { if mthd.Sel.Name == "Publish" && mthd.X.(*ast.Ident).Name == "ec" { fmt.Printf("Topic: %s \n", funcCall.Args[0].(*ast.BasicLit).Value) } } } } return true }) }

A lot going here now! But if we break apart the pieces, it becomes very simple. We can see that we are still checking if the node is a call expression, but then we start going a bit further.

现在要去这里很多！ 但是，如果我们将各个部分分开，它将变得非常简单。 我们可以看到，我们仍在检查该节点是否为调用表达式，但是接下来我们要进一步介绍。

On line 22 we check if the function call is a selector expression — an expression that selects an identifier from a base expression in the from <expression>.<identifier>. Since our calls to the NATS encoded connection are methods on an object, we want to make sure the function we are inspecting is a selector expression.

在第22行，我们检查函数调用是否为选择器表达式—一个从<expression>。<identifier>中的基本表达式中选择标识符的表达式。 由于对NATS编码的连接的调用是对象上的方法，因此我们要确保检查的函数是选择器表达式。

Line 24–26 will then make sure the expression we are selecting on is an identifier (so we don’t get type casting errors), and then we check to make sure we are working with a publishing function. This is done by checking the name of our selector to make sure it is the Publish method, and we also check to make sure the identifier we are selecting on is the “ec” variable, or our encoded connection. There are some assumptions made here, like the Publish method is always called directly (not wrapped) and the variable holding our encoded connection is named “ec”.

然后，第24–26行将确保我们选择的表达式是一个标识符(因此不会出现类型转换错误)，然后检查以确保我们正在使用发布功能。 这是通过检查选择器的名称以确保它是Publish方法来完成的，并且我们还检查以确保我们选择的标识符是“ ec”变量或编码连接。 这里有一些假设，例如总是直接调用Publish方法(不包装)，保存编码连接的变量称为“ ec”。

Once we are able to confirm the function is the “ec.Publish” method, we can print out the topic. Using a bit more node traversal and type assertion, we take the first arg and cast it to a basic literal and get the value. However, we could increase the robustness here by using a type switch like the following:

一旦我们能够确认该功能是“ ec.Publish”方法，就可以打印出该主题。 使用更多的节点遍历和类型断言，我们获取第一个arg并将其强制转换为基本文字并获取值。 但是，我们可以通过使用如下所示的类型开关来提高鲁棒性：

package main import ( "fmt" "go/ast" "go/parser" "go/token" ) func main() { //Create a new token set for the file fset := token.NewFileSet() //Parse the file and create an AST file, err := parser.ParseFile(fset, "../nats_publisher/main.go", nil, parser.ParseComments) if err != nil { panic(err) } ast.Inspect(file, func(n ast.Node) bool { // Find Return Statements funcCall, ok := n.(*ast.CallExpr) if ok { mthd, ok := funcCall.Fun.(*ast.SelectorExpr) if ok { _, ok = mthd.X.(*ast.Ident) if ok { if mthd.Sel.Name == "Publish" && mthd.X.(*ast.Ident).Name == "ec" { switch v := funcCall.Args[0].(type) { case *ast.BasicLit: fmt.Printf("Topic: %s \n", v.Value) case *ast.Ident: fmt.Printf("Topic is declared with: %s \n", v.Name) default: fmt.Println("Unrecognized type") } } } } } return true }) }

Although in this example we aren’t further exploring the case when the topic is specified by an identifier (variable, constant, object, etc), it is possible to explore further down the tree and get the actual value if derived from a literal. If you want to get really fancy, you can even use an additional visitor function along with the inspect method to further explore a node and dig deeper into the tree until you hit the node that has the literal topic value.

尽管在此示例中，我们不再进一步探讨由标识符(变量，常量，对象等)指定主题的情况，但可以进一步探索树，并从文字中获取实际值。 如果您真的想花哨的话，甚至可以使用附加的visitor函数以及inspect方法来进一步探索节点，并深入挖掘树，直到您找到具有文字主题值的节点。

下一步 (Next Steps)

Now that we have the topic of the Publisher extracted, we will move onto providing additional context around the microservice. We can accomplish this with structured code comments and the parser’s capability of associating groups of comments with Go’s various types. Stay tuned for the next part in this walkthrough.

现在我们已经提取了发布者的主题，我们将继续围绕微服务提供其他上下文。 我们可以通过结构化的代码注释以及解析器将注释组与Go的各种类型相关联的能力来完成此任务。 请继续关注本演练的下一部分。

翻译自: https://medium.com/@riptidedata/cool-stuff-with-gos-ast-package-pt-1-981460cddcd7

搭建gos

相关资源：四史答题软件安装包exe