python 任务调度 celery_如果我使用celery作为任务调度程序,如何从...

经过数小时的努力,弄清楚了我的代码最终可能出什么问题,现在我得到了令我满意的结果.首先,我创建了一个文件loggingsetup.py,在其中使用dictConfig配置了python记录器：

LOGGING = {

'version': 1,

'disable_existing_loggers': True,

'formatters': { # Sets up the format of the logging output

'simple': {

'format': '%(asctime)s - %(name)s - %(levelname)s - %(message)s',

'datefmt': '%y %b %d, %H:%M:%S',

},

},

'filters': {

'filterForSplunk': { # custom loggingFilter, to not have Logs logged to Splunk that have the word celery in the name

'()': 'loggingsetup.RemoveCeleryLogs', # class on top of this file

'logsToSkip': 'celery' # word that it is filtered for

},

},

'handlers': {

'splunk': { # handler for splunk, level Warning. to not have many logs sent to splunk

'level': 'WARNING',

'class': 'splunk_logging_handler.SplunkLoggingHandler',

'url': os.getenv('SPLUNK_HTTP_COLLECTOR_URL'),

'splunk_key': os.getenv('SPLUNK_TOKEN'),

'splunk_index': os.getenv('SPLUNK_INDEX'),

'formatter': 'simple',

'filters': ['filterForSplunk']

},

'console': {

'level': 'DEBUG',

'class': 'logging.StreamHandler',

'stream': 'ext://sys.stdout',

'formatter': 'simple',

},

},

'loggers': { # the logger, root is used

'': {

'handlers': ['console', 'splunk'],

'level': 'DEBUG',

'propagate': 'False', # does not give logs to other logers

}

}

}

对于日志记录过滤器,我必须创建一个从logging.Filter类继承的类.该类还依赖于文件loggingsetup.py

class RemoveCeleryLogs(logging.Filter): # custom class to filter for celery logs (to not send them to Splunk)

def __init__(self, logsToSkip=None):

self.logsToSkip = logsToSkip

def filter(self, record):

if self.logsToSkip == None:

allow = True

else:

allow = self.logsToSkip not in record.name

return allow

之后,您可以像这样配置记录器：

logging.config.dictConfig(loggingsetup.LOGGING)

logger = logging.getLogger('')

而且由于芹菜重定向了它的日志,并且日志翻了一倍,我不得不更新app.conf：

app.conf.update({

'worker_hijack_root_logger': False, # so celery does not set up its loggers

'worker_redirect_stdouts': False, # so celery does not redirect its logs

})

我面临的下一个问题是,我选择的Splunk_Logging库将某些内容与URL混合在一起.因此,我必须创建自己的splunk_handler类,该类继承自logging.Handler类.这里的重要内容如下：

auth_header = {'Authorization': 'Splunk {0}'.format(self.splunk_key)}

json_message = {"index": str(self.splunk_index), "event": data}

r = requests.post(self.url, headers=auth_header, json=json_message)

我希望我能为遇到python,splunk和celery日志记录类似问题的人提供帮助！ ?