Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
W
webmagic
Project
Project
Details
Activity
Releases
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
沈俊林
webmagic
Commits
b539522c
Commit
b539522c
authored
Nov 29, 2017
by
yihua.huang
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
#701 support to tls1.2
parent
e276b11e
Changes
4
Hide whitespace changes
Inline
Side-by-side
Showing
4 changed files
with
36 additions
and
1 deletion
+36
-1
pom.xml
pom.xml
+5
-0
HttpClientGenerator.java
...us/codecraft/webmagic/downloader/HttpClientGenerator.java
+4
-1
HttpClientDownloaderTest.java
...decraft/webmagic/downloader/HttpClientDownloaderTest.java
+1
-0
SSLCompatibilityTest.java
...s/codecraft/webmagic/downloader/SSLCompatibilityTest.java
+26
-0
No files found.
pom.xml
View file @
b539522c
...
...
@@ -75,6 +75,11 @@
<artifactId>
httpclient
</artifactId>
<version>
4.5.2
</version>
</dependency>
<dependency>
<groupId>
org.apache.httpcomponents
</groupId>
<artifactId>
httpcore
</artifactId>
<version>
4.5.2
</version>
</dependency>
<dependency>
<groupId>
com.google.guava
</groupId>
<artifactId>
guava
</artifactId>
...
...
webmagic-core/src/main/java/us/codecraft/webmagic/downloader/HttpClientGenerator.java
View file @
b539522c
...
...
@@ -9,6 +9,7 @@ import org.apache.http.config.RegistryBuilder;
import
org.apache.http.config.SocketConfig
;
import
org.apache.http.conn.socket.ConnectionSocketFactory
;
import
org.apache.http.conn.socket.PlainConnectionSocketFactory
;
import
org.apache.http.conn.ssl.DefaultHostnameVerifier
;
import
org.apache.http.conn.ssl.SSLConnectionSocketFactory
;
import
org.apache.http.impl.client.*
;
import
org.apache.http.impl.conn.PoolingHttpClientConnectionManager
;
...
...
@@ -49,7 +50,9 @@ public class HttpClientGenerator {
private
SSLConnectionSocketFactory
buildSSLConnectionSocketFactory
()
{
try
{
return
new
SSLConnectionSocketFactory
(
createIgnoreVerifySSL
());
// 优先绕过安全证书
return
new
SSLConnectionSocketFactory
(
createIgnoreVerifySSL
(),
new
String
[]{
"SSLv3"
,
"TLSv1"
,
"TLSv1.1"
,
"TLSv1.2"
},
null
,
new
DefaultHostnameVerifier
());
// 优先绕过安全证书
}
catch
(
KeyManagementException
e
)
{
logger
.
error
(
"ssl connection fail"
,
e
);
}
catch
(
NoSuchAlgorithmException
e
)
{
...
...
webmagic-core/src/test/java/us/codecraft/webmagic/downloader/HttpClientDownloaderTest.java
View file @
b539522c
...
...
@@ -322,4 +322,5 @@ public class HttpClientDownloaderTest {
});
}
}
webmagic-core/src/test/java/us/codecraft/webmagic/downloader/SSLCompatibilityTest.java
0 → 100644
View file @
b539522c
package
us
.
codecraft
.
webmagic
.
downloader
;
import
org.junit.Test
;
import
us.codecraft.webmagic.Page
;
import
us.codecraft.webmagic.Request
;
import
us.codecraft.webmagic.Site
;
import
us.codecraft.webmagic.Task
;
import
static
org
.
assertj
.
core
.
api
.
Assertions
.
assertThat
;
/**
* @author code4crafter@gmail.com
* Date: 2017/11/29
* Time: 下午1:32
*/
public
class
SSLCompatibilityTest
{
@Test
public
void
test_tls12
()
throws
Exception
{
HttpClientDownloader
httpClientDownloader
=
new
HttpClientDownloader
();
Task
task
=
Site
.
me
().
setCycleRetryTimes
(
5
).
toTask
();
Request
request
=
new
Request
(
"https://juejin.im/"
);
Page
page
=
httpClientDownloader
.
download
(
request
,
task
);
assertThat
(
page
.
isDownloadSuccess
()).
isTrue
();
}
}
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment