官术网_书友最值得收藏!

Loading parsers dynamically

This application is intended to be flexible and allow developers to create different parsers for different weather websites. We are going to create a parser loader that will dynamically discover files inside of the parsers directory, load them, and make them available to be used by the application without requiring changes to any other parts of the code. Here are the rules that our loader will require when implementing new parsers:

  • Create a file with a class implementing the methods for fetching the current weather forecast as well as five-day, ten-day, and weekend weather forecasts
  • The file name has to end with parser, for example, weather_com_parser.py
  • The file name can't start with double underscores

With that said, let's go ahead and create the parser loader. Create a file named parser_loader.py inside of the weatherterm/core directory and add the following content:

import os
import re
import inspect


def _get_parser_list(dirname):
files = [f.replace('.py', '')
for f in os.listdir(dirname)
if not f.startswith('__')]

return files


def _import_parsers(parserfiles):

m = re.compile('.+parser$', re.I)

_modules = __import__('weatherterm.parsers',
globals(),
locals(),
parserfiles,
0)

_parsers = [(k, v) for k, v in inspect.getmembers(_modules)
if inspect.ismodule(v) and m.match(k)]

_classes = dict()

for k, v in _parsers:
_classes.update({k: v for k, v in inspect.getmembers(v)
if inspect.isclass(v) and m.match(k)})

return _classes


def load(dirname):
parserfiles = _get_parser_list(dirname)
return _import_parsers(parserfiles)

First, the _get_parser_list function is executed and returns a list of all files located in weatherterm/parsers; it will filter the files based on the rules of the parser described previously. After returning a list of files, it is time to import the module. This is done by the _import_parsers function, which first imports the weatherterm.parsers module and makes use of the inspect package in the standard library to find the parser classes within the module.

The inspect.getmembers function returns a list of tuples where the first item is a key representing a property in the module, and the second item is the value, which can be of any type. In our scenario, we are interested in a property with a key ending with parser and with the value of type class.

Assuming that we already have a parser in place in the weatherterm/parsers  directory, the value returned by the inspect.getmembers(_modules) will look something like this:

[('WeatherComParser',
<class 'weatherterm.parsers.weather_com_parser.WeatherComParser'>),
...]
inspect.getmembers(_module)  returns many more items, but they have been omitted since it is not relevant to show all of them at this point.

Lastly, we loop through the items in the module and extract the parser classes, returning a dictionary containing the name of the class and the class object that will be later used to create instances of the parser.

主站蜘蛛池模板: 泰顺县| 嵊泗县| 朝阳县| 锦州市| 余姚市| 德惠市| 苏州市| 大安市| 浪卡子县| 闽清县| 武陟县| 梨树县| 广汉市| 中阳县| 东源县| 靖江市| 孟村| 弥渡县| 新河县| 剑川县| 泰州市| 安宁市| 石台县| 昌宁县| 顺昌县| 岗巴县| 宜兴市| 拉孜县| 寿阳县| 登封市| 兰坪| 启东市| 抚顺县| 海宁市| 寿宁县| 鞍山市| 怀柔区| 酒泉市| 铁岭县| 南部县| 缙云县|