It's been a month since the start of GSoC coding, and I've finished the changes that I described in my previous post. The pull request is currently under evaluation, and can be seen in this url: https://github.com/scrapy/scrapy/pull/737.
The integration of the new settings interface with the rest of the code was made easier by the extensive set of unit tests on Scrapy that ensures that the main use cases are covered and remain errorless. I had to adjust some of the tests and their auxiliary routines so the settings configuration is updated with its present usage.
The old crawler settings class was deprecated, and every occurrence of this class was replaced by the new one, along with its methods and attributes. The documentation was fixed to explain the new settings API, and some of the internal details were added for developers that write Scrapy extensions.
I'll implement the spider settings on top of these changes, since now it's trivial to add new configuration sources, but we still have to test them individually and extend the documentation.
The last feature for the settings API will be freeze support, that is, to make settings immutable after they've been populated. We will issue a warning when trying to update settings after this point, so that users know that further changes have no effect.
I'll work on this next week, and as I'm ending my college semester on Thursday I hope that I can speed things up.
No comments:
Post a Comment