Source code for save_to_db.core.utils.dict_wrapper

import pprint


[docs]class DictWrapper(dict): """This class is used as a temporary wrapper around :py:class:`~save_to_db.core.item_base.ItemBase` instance for compatibility with Scrapy project. Scrapy is an open source and collaborative framework for extracting the data you need from websites. In a fast, simple, yet extensible way. [#DictWrapper_1]_ .. [#DictWrapper_1] Citation from https://scrapy.org site. When parsing a page with Scrapy you cannot yield instances of arbitrary classes, but you can yield an instance of a `dict` class which is treated as an instance of a Scrapy item and properly sent to item pipelines. Using :py:meth:`~save_to_db.core.item_base.ItemBase.to_dict` and then :py:meth:`~save_to_db.core.item_base.ItemBase.load_dict` of :py:class:`~save_to_db.core.item_base.ItemBase` instance is expensive, as it properly transforms an item into a `dict` and then `dict` to item. Using :py:meth:`~save_to_db.core.item_base.ItemBase.dict_wrapper` method of :py:class:`~save_to_db.core.item_base.ItemBase` will just wrap an item in a :py:class:`~.DictWrapper` instance (subclass of `dict` class), later you can get the wrapped item from the wrapper in a Scrapy pipeline. Instances of Scrapy items and of this library items are not completely compatible, so you need to use different item pipelines for this library items, the pipelines must accept and return instances of :py:class:`~.DictWrapper` if you want to use more then one pipe in a pipeline. Here is an example of a Scrapy pipeline that saves items to a database: .. code-block:: Python from django.db import transaction from save_to_db import Persister from save_to_db.adapters import DjangoAdapter persister = Persister(DjangoAdapter(adapter_settings={})) class DbPipeline(object): def process_item(self, item, spider): stdb_item = item.get_item() with transaction.atomic(): persister.persist(stdb_item) return item # return wrapped item for the next pipe in line .. note: Printing item wrapper to a console will output the wrapped item as a dictionary. :param item: An instance of :py:class:`~save_to_db.core.item_base.ItemBase` to be wrapped in a dictionary. """ def __init__(self, item): self["item"] = item
[docs] def get_item(self): """Returns an originally wrapped item instance. :returns: Instance of :py:class:`~save_to_db.core.item_base.ItemBase` class. """ return self["item"]
def __str__(self): return pprint.pformat(self.get_item().to_dict())