python程序日志

Recently, I set about building a Python application to query a PostgreSQL database, process the data, and push subsequent trigger events to a Kafka queue. However, before tackling the interesting aspects, I knew I needed to get the basics right. And one of the basics for any application is logging!

最近，我着手构建一个Python应用程序来查询PostgreSQL数据库，处理数据并将后续触发事件推送到Kafka队列。 但是，在解决有趣的方面之前，我知道我需要正确了解基础知识。 记录任何应用程序的基础之一！

In this article, I’ll explain some of the key features of logging in Python and, more importantly, demonstrate how I implemented a logger in my application.

在本文中，我将解释Python日志记录的一些关键功能，更重要的是，演示如何在应用程序中实现记录器。

An example of an error log when connecting to PostgreSQL — Image from the author. 连接到PostgreSQL时的错误日志示例—来自作者。

打印(为什么不呢？) (print (Why Not?))

The first question to tackle: Why not just use print()? It’s tempting to take the easy route and add print statements all over your code. We’ve all written code that looks a bit like this:

要解决的第一个问题：为什么不只使用print() ？ 采取简单的方法并在代码中添加print语句很诱人。 我们都编写了看起来像这样的代码：

print("Getting some docs...")docs = getDocs()print("Doc count %s", len(docs))print("Finished")

Generally, this is fine for small scripts or if you need something quick and dirty while developing or debugging. However, print() is just not a viable logging solution for larger applications — especially if you are planning on promoting your code into production environments.

通常，这对于小型脚本或在开发或调试时需要快速而又肮脏的东西很好。 但是，对于大型应用程序而言， print()并不是可行的日志记录解决方案-特别是如果您打算将代码推广到生产环境中时，尤其如此。

Let’s touch on a few of the reasons why print should be avoided. Firstly, it only gives you the option of logging to stdout. This is going to be problematic if you are logging lots of data. Really, you want to be writing your logs somewhere that can be easily persisted, backed up, and queried at a later date. Another reason is that print statements are not configurable at runtime. To switch on/off specific logs, you will need to modify your code every single time. This will mean redeploying your code to production every time you need to turn on debug logging! On top of this, it’s also much more difficult to include valuable information and context, such as the line number and the time at which the log message was generated.

让我们来谈谈应避免print的一些原因。 首先，它仅使您可以选择登录到stdout 。 如果您要记录大量数据，这将是有问题的。 确实，您希望将日志写到可以轻松保存，备份和稍后查询的地方。 另一个原因是， print语句在运行时不可配置。 要打开/关闭特定的日志，您需要每次都修改代码。 这意味着您每次需要打开调试日志记录时，都将代码重新部署到生产环境中！ 最重要的是，包含有价值的信息和上下文(例如行号和生成日志消息的时间)也要困难得多。

I could go on, but hopefully, you are already convinced!

我可以继续，但是希望您已经说服了！

输入Python的日志记录模块 (Enter Python’s Logging Module)

Fortunately, the importance of logging is not a new phenomenon. Python ships with a ready-made logging solution as part of the Python standard library. It solves all the aforementioned problems with using print. For example:

幸运的是，测井的重要性并不是一个新现象。 Python随附了现成的日志记录解决方案，作为Python标准库的一部分。 它通过使用print解决了上述所有问题。 例如：

Automatically add context, such as line number and timestamps to logs.

自动将上下文(例如行号和时间戳)添加到日志中。 It’s possible to update our logger at runtime by passing a configuration file to the app.

通过将配置文件传递给应用程序，可以在运行时更新记录器。 It is easy to customise the log severity and configure different logging levels for different environments

自定义日志严重性并针对不同环境配置不同的日志记录级别很容易

Let’s try it out and set up a very basic logger:

让我们尝试一下，并设置一个非常基本的记录器：

Running this gives:

运行此命令可获得：

INFO:__main__:Getting some docs...INFO:__main__:Doc count 2INFO:__main__:Finished

Easy peasy!

十分简单！

Here, we have imported the logging module from the Python standard library. We then updated the default basic log level to log INFO messages. Next, logger = logging.getLogger(__name__) instantiates our logging instance. Finally, we passed an event to the logger with a log level of INFO by callinglogger.info("").

在这里，我们从Python标准库中导入了日志记录模块。 然后，我们更新了默认的基本日志级别以记录INFO消息。 接下来， logger = logging.getLogger(__name__)实例化我们的日志记录实例。 最后，我们通过调用logger.info("")将事件传递给日志级别为INFO的记录器。

At first glance, this output might appear suspiciously similar to using print(). Next, we’ll expand our example logger to demonstrate some of the more powerful features that the Python standard logging module provides.

乍一看，此输出看起来可疑地类似于使用print() 。 接下来，我们将扩展示例记录器，以演示Python标准记录模块提供的一些更强大的功能。

日志级别 (Log levels)

We can configure the severity of the logs being output and filter out unimportant ones. The module defines five constants throughout the spectrum, making it easy to differentiate between messages. The numeric values of logging levels are given in the following table:

我们可以配置输出日志的严重性，并过滤掉不重要的日志。 该模块在整个频谱中定义了五个常数，从而可以轻松区分消息。 下表中给出了日志记录级别的数值：

Python’s documentation. Python文档中记录级别。

It’s important not to flood your logs with lots of messages. To achieve concise logs, we should be careful to define the correct log level for each event:

重要的是不要在日志中充斥大量消息。 为了获得简洁的日志，我们应该为每个事件定义正确的日志级别：

logger.critical("Really bad event"logger.error("An error")logger.warning("An unexpected event")logger.info("Used for tracking normal application flow")logger.debug("Log data or variables for developing")

I tend to use the debug level to log the data being passed around the app. Here is an example of using three different log levels in the few lines of code responsible for sending events to Kafka:

我倾向于使用调试级别来记录在应用程序周围传递的数据。 这是在负责向Kafka发送事件的几行代码中使用三种不同日志级别的示例：

格式化日志 (Formatting logs)

The default formatter of the Python logging module doesn’t provide a great amount of detail. Fortunately, it is easy to configure the log format to add all the context we need to produce super-useful log messages.

Python日志记录模块的默认格式化程序未提供大量详细信息。 幸运的是，很容易配置日志格式以添加生成超级有用的日志消息所需的所有上下文。

For example, here we add a timestamp and the log level to the log message:

例如，在这里我们将时间戳和日志级别添加到日志消息中：

formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')

It’s best practice to add as much context as possible to your logs. This can easily be achieved by adding structured data to the log message’s metadata. For example, you may have scaled your application to run with multiple workers. In this case, it might be important to know which worker was logging each event when you’re debugging, so let’s add a worker ID context to the log metadata:

最佳做法是在日志中添加尽可能多的上下文。 通过将结构化数据添加到日志消息的元数据中，可以轻松实现这一点。 例如，您可能已缩放应用程序以与多个工作程序一起运行。 在这种情况下，调试时知道哪个工作人员正在记录每个事件可能很重要，因此让我们在日志元数据中添加一个工作人员ID上下文：

# Create the log formatterformatter = handler.setFormatter(formatter)logger.info('Querying database for docs...', extra={'worker': 'id_1'})

The output becomes:

输出变为：

2020-09-02 22:06:18,170 - id_1 - INFO - Querying database for docs...

日志处理程序 (Log handlers)

Now that we have perfectly formatted logs being fired at us from all over our application code, we need to consider where those logs are ending up. By default, the logs are being written to stdout, but Python’s logging module provides us with the functionality to push logs to alternative locations. For example, to save logs to the example.log file on disk:

现在，我们已经从所有应用程序代码中向我们发射了格式正确的日志，我们需要考虑这些日志在哪里结束。 默认情况下，日志被写入stdout ，但是Python的日志记录模块为我们提供了将日志推送到其他位置的功能。 例如，要将日志保存到磁盘上的example.log文件中：

# create a file handlerhandler = logging.FileHandler('example.log')handler.setLevel(logging.INFO)

There are several types of handlers that can be used. For the complete list, see the documentation for handlers. It is also possible to define custom logging handlers for different use cases. For example, this library defines a log handler for pushing logs to Slack!

可以使用几种类型的处理程序。 有关完整列表，请参见处理程序的文档 。 也可以为不同用例定义自定义日志记录处理程序。 例如， 此库定义了一个日志处理程序，用于将日志推送到Slack!。

To summarise. We’ve set up the Python standard logging module and configured it to log to different locations with custom log formats. You can find the final code for the example logger below:

总结一下。 我们已经设置了Python标准日志记录模块，并将其配置为使用自定义日志格式记录到其他位置。 您可以在下面找到示例记录器的最终代码：

实作 (Implementation)

By now, you should have a good idea of some of the key features of the Python logging module. That’s all well and good, but how should you implement all this different configuration and manage importing logging modules in your application files? It’s a point often overlooked by many guides. Here is how I decided to handle it.

到目前为止，您应该对Python日志记录模块的一些关键功能有所了解。 很好，但是您应该如何实现所有这些不同的配置并管理应用程序文件中的导入日志记录模块？ 这一点经常被许多指南所忽略。 这是我决定处理的方式。

First up, I decided to extract all the logging configuration into a config file called logging.ini. You can find all the information you need on how to format the log file in the official docs, so I won’t go over it all here. Currently, my logging.ini is super basic. When the application is out of development and ready for deployment, I’ll be extending the logging configuration to handle different environments, rotate logs on disk, and send alerts to a Slack channel.

首先，我决定将所有日志记录配置提取到一个名为logging.ini的配置文件中。 您可以在官方文档中找到有关如何格式化日志文件所需的所有信息，因此在此不再赘述。 目前，我的logging.ini超级基础。 当应用程序无法开发并准备部署时，我将扩展日志记录配置以处理不同的环境，旋转磁盘上的日志并将警报发送到Slack通道。

This really simplifies the implementation from a code point of view because you need much less boilerplate to set up log handlers and formatters. Above, we had 15 lines of code setting up just one formatter and one handler. It’s easy to imagine how this could become a lot bigger with more complex use cases. Now, all that complexity has been abstracted away to the config file. The result: To implement your logger, you just need two extra lines of code in your app!

这确实从一个代码点简化了实现，因为你需要少得多的样板设置日志处理程序和格式化。 上面，我们有15行代码仅设置了一个格式化程序和一个处理程序。 很难想象，如果使用更复杂的用例，情况可能会变得更大。 现在，所有这些复杂性已被抽象到配置文件中。 结果：要实现记录器，您只需要在应用程序中多添加两行代码即可！

import logging.configlogging.config.fileConfig(fname='logger.ini')

Pretty neat! But where should this code live exactly?

漂亮整齐！ 但是此代码应准确地存放在哪里？

My application is structured with a __main__.py as the entry point. I decided to implement my logger inside this file in order to not distract from the main functionality contained within app.py. The __main__.py below will handle reading the config file and applying the configuration to your logger:

我的应用程序的结构是一个__main__.py作为入口点。 我决定在此文件中实现我的记录器，以免分散app.py包含的主要功能。 下面的__main__.py将处理读取配置文件并将配置应用于记录器的操作：

Finally, we need to make use of our creation by firing off some logs! But these messages aren’t going to be generated by the__main__function, so how should we handle passing our logger to other files and functions within the application?

最后，我们需要通过释放一些日志来利用我们的创作！ 但是这些消息不会由__main__函数生成，因此我们应该如何处理将记录器传递给应用程序中的其他文件和函数呢？

Again, this becomes trivial. To use the logger in another file, we only need to import the logging module and instantiate it. It will automatically inherit the configuration that we passed to it in __main__. For example, to start logging messages in app.py:

再次，这变得微不足道。 要在另一个文件中使用记录器，我们只需要导入记录模块并将其实例化即可。 它将自动继承我们在__main__传递给它的配置。 例如，要开始在app.py记录消息：

The debug message above will be handled based on the logging configuration read fromlogging.ini. And the same will be true for every other Python file in your application as long as you import logging at the top. Yes, it is really that simple!

上面的调试消息将基于从logging.ini读取的日志配置进行处理。 只要您在顶部import logging ，应用程序中的所有其他Python文件都将如此。 是的，真的就是这么简单！

To recap, not only have you seen some of the features that make the Python’s logging module an awesome tool to have in your armoury, but you have also seen how easy it is to implement.

回顾一下，您不仅看到了使Python的日志记录模块成为您的精采工具的某些功能，而且还看到了实现的难易程度。

Thank you for reading. I hope you found this guide useful. Happy logging!

感谢您的阅读。 希望本指南对您有所帮助。 祝您登录愉快！

翻译自: https://medium.com/better-programming/how-to-implement-logging-in-your-python-application-1730315003c4

python程序日志

相关资源：微信小程序源码-合集6.rar