官术网_书友最值得收藏!

Loading parsers dynamically

This application is intended to be flexible and allow developers to create different parsers for different weather websites. We are going to create a parser loader that will dynamically discover files inside of the parsers directory, load them, and make them available to be used by the application without requiring changes to any other parts of the code. Here are the rules that our loader will require when implementing new parsers:

  • Create a file with a class implementing the methods for fetching the current weather forecast as well as five-day, ten-day, and weekend weather forecasts
  • The file name has to end with parser, for example, weather_com_parser.py
  • The file name can't start with double underscores

With that said, let's go ahead and create the parser loader. Create a file named parser_loader.py inside of the weatherterm/core directory and add the following content:

import os
import re
import inspect


def _get_parser_list(dirname):
files = [f.replace('.py', '')
for f in os.listdir(dirname)
if not f.startswith('__')]

return files


def _import_parsers(parserfiles):

m = re.compile('.+parser$', re.I)

_modules = __import__('weatherterm.parsers',
globals(),
locals(),
parserfiles,
0)

_parsers = [(k, v) for k, v in inspect.getmembers(_modules)
if inspect.ismodule(v) and m.match(k)]

_classes = dict()

for k, v in _parsers:
_classes.update({k: v for k, v in inspect.getmembers(v)
if inspect.isclass(v) and m.match(k)})

return _classes


def load(dirname):
parserfiles = _get_parser_list(dirname)
return _import_parsers(parserfiles)

First, the _get_parser_list function is executed and returns a list of all files located in weatherterm/parsers; it will filter the files based on the rules of the parser described previously. After returning a list of files, it is time to import the module. This is done by the _import_parsers function, which first imports the weatherterm.parsers module and makes use of the inspect package in the standard library to find the parser classes within the module.

The inspect.getmembers function returns a list of tuples where the first item is a key representing a property in the module, and the second item is the value, which can be of any type. In our scenario, we are interested in a property with a key ending with parser and with the value of type class.

Assuming that we already have a parser in place in the weatherterm/parsers  directory, the value returned by the inspect.getmembers(_modules) will look something like this:

[('WeatherComParser',
<class 'weatherterm.parsers.weather_com_parser.WeatherComParser'>),
...]
inspect.getmembers(_module)  returns many more items, but they have been omitted since it is not relevant to show all of them at this point.

Lastly, we loop through the items in the module and extract the parser classes, returning a dictionary containing the name of the class and the class object that will be later used to create instances of the parser.

主站蜘蛛池模板: 麻栗坡县| 密云县| 尼勒克县| 鄱阳县| 临澧县| 思茅市| 静海县| 宁夏| 武陟县| 大埔县| 昌乐县| 广昌县| 凤城市| 隆化县| 石嘴山市| 清水县| 九龙县| 东安县| 莱西市| 镶黄旗| 隆昌县| 长宁区| 格尔木市| 博罗县| 贡觉县| 桦南县| 盐亭县| 教育| 石门县| 宣武区| 屏南县| 绵阳市| 二手房| 马鞍山市| 金华市| 陆川县| 闻喜县| 东台市| 来凤县| 安塞县| 阳泉市|