爬虫之爬取小说

标签: 学习  nodejs  爬虫  后端  javascript

爬虫之爬取小说

展示

在这里插入图片描述
在这里插入图片描述
在这里插入图片描述

源码地址

https://github.com/Liar0320/spider

依赖

  • cherrio Fast, flexible & lean implementation of core jQuery designed specifically for the server.
  • inquirer A collection of common interactive command line user interfaces.

安装

Clone the repo using Git:

git clone https://github.com/Liar0320/spider.git

Alternatively you can download this repository and then:

npm i 
npm run start 

配置参数

请求的超时时间

Timeout for each request.

default value: 15000

最大同时下载多少章节

The maximum number of threads that exist at one time, that is, the number of asynchronous downloads that are started at the same time.

default value: 20

请求失败时,重新发起请求的次数

The number of times the request was re initiated if there was a request failure.

default value: 4

下载的目录

Root of download directory

default value: join(__dirname, ‘./download’)

版权声明:本文为m0_37740770原创文章,遵循 CC 4.0 BY-SA 版权协议,转载请附上原文出处链接和本声明。
本文链接:https://blog.csdn.net/m0_37740770/article/details/104614300