Skip to content

W
webmagic
Switch branch/tag
  1. 21 Jan, 2016 10 commits
  3. 20 Jan, 2016 1 commit
  5. 19 Jan, 2016 1 commit
  7. 18 Jan, 2016 4 commits
  9. 16 Jan, 2016 2 commits
  11. 02 Dec, 2015 2 commits
  13. 17 Nov, 2015 2 commits
  15. 13 Nov, 2015 1 commit
  17. 12 Nov, 2015 1 commit
    • x1ny's avatar
      修正FileCacheQueueScheduler导致程序不能正常结束和未关闭流 · 90e14b31
      x1ny authored 
      FileCacheQueueScheduler中开启了一个线程周期运行来保存数据但在爬虫结束后没有关闭导致程序无法结束，以及没有关闭io流。

解决方法：
让FileCacheQueueScheduler实现Closable接口，在close方法中关闭线程以及流。
在Spider的close方法中添加对scheduler的关闭操作。
      90e14b31
  19. 18 Sep, 2015 2 commits
  21. 11 Jul, 2015 2 commits
    • bingoko's avatar
      update and validate pom.xml · 5d365f7b
      bingoko authored 
      Update selenium and GhostDriver (PhantomJSDriver) to latest version.
      5d365f7b
    • bingoko's avatar
      Add PhantomJS support for selenium · d3bbece2
      bingoko authored 
      The configuration file is config.ini
The dependencies are updated in pom.xml.
Update SeleniumDownloader and WebDriverPool to support PhantomJS. 
NOTE: The versions of GhostDriver, Selenium, and PhantomJS are stable
and validated.

A GooglePlay Example is under samples package: GooglePlayProcessor.java
      d3bbece2
  23. 15 Apr, 2015 3 commits
  25. 09 Mar, 2015 2 commits
  27. 18 Feb, 2015 1 commit
  29. 13 Feb, 2015 1 commit
  31. 22 Jan, 2015 1 commit
  33. 21 Jan, 2015 1 commit
  35. 13 Jan, 2015 1 commit
  37. 29 Sep, 2014 2 commits