Skip to content

v0.6.37

Compare
Choose a tag to compare
@lorien lorien released this 10 May 09:08
· 528 commits to master since this release

Added

  • Add --spider-transport and --grab-transport options to crawl script
  • Add SOCKS5 proxy support in urllib3 transport

Fixed

  • Fix #237: urllib3 transport fails without pycurl installed
  • Fix bug: incorrect spider request logging when cache is enabled
  • Fix bug: crawl script fails while trying to process a lock key
  • Fix bug: urllib3 transport fails while trying to throw GrabConnectionError exception
  • Fix bug: Spider add_task method fails while trying to log invalid URL error

Removed

  • Remove obsoleted hammer_mode and hammer_timeout config options