Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
W
webmagic
Project
Project
Details
Activity
Releases
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
沈俊林
webmagic
Commits
acb63d55
Commit
acb63d55
authored
Apr 25, 2014
by
yihua.huang
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
some check and example #98
parent
11ba5beb
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
25 additions
and
5 deletions
+25
-5
MonitorExample.java
...in/java/us/codecraft/webmagic/example/MonitorExample.java
+3
-2
SpiderMonitor.java
...ain/java/us/codecraft/webmagic/monitor/SpiderMonitor.java
+22
-3
No files found.
webmagic-extension/src/main/java/us/codecraft/webmagic/example/MonitorExample.java
View file @
acb63d55
...
@@ -7,6 +7,7 @@ import us.codecraft.webmagic.processor.example.OschinaBlogPageProcessor;
...
@@ -7,6 +7,7 @@ import us.codecraft.webmagic.processor.example.OschinaBlogPageProcessor;
/**
/**
* @author code4crafer@gmail.com
* @author code4crafer@gmail.com
* @since 0.5.0
*/
*/
public
class
MonitorExample
{
public
class
MonitorExample
{
...
@@ -21,8 +22,8 @@ public class MonitorExample {
...
@@ -21,8 +22,8 @@ public class MonitorExample {
spiderMonitor
.
register
(
oschinaSpider
,
githubSpider
);
spiderMonitor
.
register
(
oschinaSpider
,
githubSpider
);
//If you want to connect it from remote, use spiderMonitor.server().jmxStart();
//If you want to connect it from remote, use spiderMonitor.server().jmxStart();
//ONLY ONE server can start for a machine.
//ONLY ONE server can start for a machine.
//Others will be registered
//Others will be registered
without start a server.
spiderMonitor
.
server
().
server
();
//You can also register a server by spiderMonitor.client(host,port).jmxStart().
spiderMonitor
.
jmxStart
();
spiderMonitor
.
jmxStart
();
oschinaSpider
.
start
();
oschinaSpider
.
start
();
githubSpider
.
start
();
githubSpider
.
start
();
...
...
webmagic-extension/src/main/java/us/codecraft/webmagic/monitor/SpiderMonitor.java
View file @
acb63d55
...
@@ -23,6 +23,7 @@ import java.rmi.server.ExportException;
...
@@ -23,6 +23,7 @@ import java.rmi.server.ExportException;
import
java.util.ArrayList
;
import
java.util.ArrayList
;
import
java.util.Collections
;
import
java.util.Collections
;
import
java.util.List
;
import
java.util.List
;
import
java.util.concurrent.atomic.AtomicBoolean
;
import
java.util.concurrent.atomic.AtomicInteger
;
import
java.util.concurrent.atomic.AtomicInteger
;
/**
/**
...
@@ -35,6 +36,10 @@ public class SpiderMonitor {
...
@@ -35,6 +36,10 @@ public class SpiderMonitor {
Server
,
Client
,
Local
;
Server
,
Client
,
Local
;
}
}
private
static
AtomicInteger
serialNumber
=
new
AtomicInteger
();
private
AtomicBoolean
started
=
new
AtomicBoolean
(
false
);
private
Logger
logger
=
LoggerFactory
.
getLogger
(
getClass
());
private
Logger
logger
=
LoggerFactory
.
getLogger
(
getClass
());
private
static
final
int
DEFAULT_SERVER_PORT
=
14721
;
private
static
final
int
DEFAULT_SERVER_PORT
=
14721
;
...
@@ -150,6 +155,17 @@ public class SpiderMonitor {
...
@@ -150,6 +155,17 @@ public class SpiderMonitor {
return
server
(
DEFAULT_SERVER_PORT
);
return
server
(
DEFAULT_SERVER_PORT
);
}
}
/**
* Local mode: the monitor will be bound to the JVM instance.<br></br>
* Use jconsole to check your application.
*
* @return
*/
public
SpiderMonitor
local
()
{
this
.
type
=
Type
.
Local
;
return
this
;
}
/**
/**
* Start monitor as client mode.
* Start monitor as client mode.
...
@@ -183,7 +199,11 @@ public class SpiderMonitor {
...
@@ -183,7 +199,11 @@ public class SpiderMonitor {
}
}
public
SpiderMonitor
jmxStart
(
String
jndiServer
,
int
rmiPort
)
throws
IOException
,
JMException
{
public
SpiderMonitor
jmxStart
(
String
jndiServer
,
int
rmiPort
)
throws
IOException
,
JMException
{
String
jmxServerName
=
"WebMagic-"
+
IPUtils
.
getFirstNoLoopbackIPAddresses
();
if
(!
started
.
compareAndSet
(
false
,
true
))
{
logger
.
error
(
"Monitor has already started!"
);
return
this
;
}
String
jmxServerName
=
"WebMagic-"
+
IPUtils
.
getFirstNoLoopbackIPAddresses
()
+
"-"
+
serialNumber
.
incrementAndGet
();
// start JNDI
// start JNDI
MBeanServer
localServer
=
ManagementFactory
.
getPlatformMBeanServer
();
MBeanServer
localServer
=
ManagementFactory
.
getPlatformMBeanServer
();
...
@@ -218,8 +238,7 @@ public class SpiderMonitor {
...
@@ -218,8 +238,7 @@ public class SpiderMonitor {
//If you want to connect it from remote, use spiderMonitor.server().jmxStart();
//If you want to connect it from remote, use spiderMonitor.server().jmxStart();
//ONLY ONE server can start for a machine.
//ONLY ONE server can start for a machine.
//Others will be registered
//Others will be registered
spiderMonitor
.
server
().
server
();
spiderMonitor
.
server
().
jmxStart
();
spiderMonitor
.
jmxStart
();
oschinaSpider
.
start
();
oschinaSpider
.
start
();
githubSpider
.
start
();
githubSpider
.
start
();
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment