博客
关于我
强烈建议你试试无所不能的chatGPT，快点击我
《Apache kafka实战》读书笔记-kafka集群监控工具
阅读量：6827 次
发布时间：2019-06-26

本文共 101061 字，大约阅读时间需要 336 分钟。

　　　　　　　　　　　　　　　《Apache kafka实战》读书笔记-kafka集群监控工具

　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　作者：尹正杰

版权声明：原创作品，谢绝转载！否则将追究法律责任。

 

　　如官网所述，Kafka使用基于yammer metrics的监控指标体系来统计broker端和clinets端的各种监控指标（metric）。说到yammer metrics，其官网过出了这样的一句话：

　　　　yammer metrics是一个java库，它使得你能够对生产环境代码所做之事具有无与伦比的洞察力。

　　当然，虽然该项目已经进化到3.x版本，但即使最新版本的kafka易燃在使用2.2版本的yammer metrics。目前，Kafka默认提供了超多的监控指标，无一例外，用户皆使用JMX接口访问这些指标。JMX，即Java Management Extension，Java管理扩展，是一套为各个应用程序，设备或系统等植入管理功能的框架。JMX本身是跨平台的，因此具有高度的灵活性，可以无缝集成进各种系统中。

　　虽然有一小部分功能是以Shell脚本直接允许的Java类来提供的，但是Kafka集群大部分的允许表现都可由JMX指标来表征，因此搭建一套能够访问JMX的监控框架或系统对于监控kafka集群而言至关重要。

　　Kafka的每个监控指标都是以JMX MBean的形式定义的。虽然JMX规范不再本书的讨论范围内，但这里依然简单介绍一些MBean的概念。MBean代表一个被管理的资源实例，他表示管理资源的Java对象。每个MBean的管理接口如下：

　　　　1>.属性值;

　　　　2>.能够执行的操作；

　　　　3>.能发出的通知事件；

　　　　4>.构建器；

　　对于kafka而言，我们只关系上述列表中的第一项，即MBean的属性值，它包含了Kafka各种监控指标的真实值。MBean需要注册之后才能使用，在注册时必须指定MBean的名称。在kafka中，所有MBean的命名规范有着统一的的格式：“xxx.type=xxx,{attr}=xxx”。第一个等式中的xxx通常表示所属的kafka组建，比如“kafka.server”,"kafka.producer","kafka.consumer"等；第二个等式的中的xxx和前面的attr通常表示该MBean的范围，比如topic=yinzhengjie-kafka表示该MBean的作用范围是名为“yinzhengjie-kafka”的topic，即它是一个topic级别的MBean，表示该特topic下的某个性能指标。

　　对于Java用户而言，访问JMX端口最简单直接方法就是使用JDK自带的JConsole工具（可以参考我之前的笔记：）。当然，你也可以直接查看本篇博客的末尾，我使用了jmxTool工具的一个案例。

 

一.安装Kafka Monitor

　　kafka Monitor 是LinkedIn公司开源的一个免费框架，用于在kafka集群上执行真实的kafka系统测试并实时监控测试度量结果。该框架由LinkedIn的kafka团队成员维护，维护的速度要比kafka-manager快一些。

 

　　如上所述，kafka monitor是在集群上执行系统测试的，以便让用户对Kafka集群的性能有大致的了解。他会自动创建一个监控topic，然后分配分区并触发preferred leader选举以确保每个broker都至少是一个分区的leader。这样做的好处在于可以全面地测试集群上所有broker的性能而无须用户手动干预。 

 

1>.安装git工具包

[root@node101 ~]# yum -y install gitLoaded plugins: fastestmirrorLoading mirror speeds from cached hostfile10gen                                                                                                                                                       | 2.5 kB  00:00:00     base                                                                                                                                                        | 3.6 kB  00:00:00     centosplus                                                                                                                                                  | 3.4 kB  00:00:00     epel                                                                                                                                                        | 3.2 kB  00:00:00     extras                                                                                                                                                      | 3.4 kB  00:00:00     mysql-connectors-community                                                                                                                                  | 2.5 kB  00:00:00     mysql-tools-community                                                                                                                                       | 2.5 kB  00:00:00     mysql56-community                                                                                                                                           | 2.5 kB  00:00:00     updates                                                                                                                                                     | 3.4 kB  00:00:00     (1/2): epel/x86_64/updateinfo                                                                                                                               | 934 kB  00:00:00     (2/2): epel/x86_64/primary                                                                                                                                  | 3.6 MB  00:00:01     epel                                                                                                                                                                   12720/12720Resolving Dependencies--> Running transaction check---> Package git.x86_64 0:1.8.3.1-14.el7_5 will be installed--> Processing Dependency: perl-Git = 1.8.3.1-14.el7_5 for package: git-1.8.3.1-14.el7_5.x86_64--> Processing Dependency: perl(Term::ReadKey) for package: git-1.8.3.1-14.el7_5.x86_64--> Processing Dependency: perl(Git) for package: git-1.8.3.1-14.el7_5.x86_64--> Processing Dependency: perl(Error) for package: git-1.8.3.1-14.el7_5.x86_64--> Processing Dependency: libgnome-keyring.so.0()(64bit) for package: git-1.8.3.1-14.el7_5.x86_64--> Running transaction check---> Package libgnome-keyring.x86_64 0:3.12.0-1.el7 will be installed---> Package perl-Error.noarch 1:0.17020-2.el7 will be installed---> Package perl-Git.noarch 0:1.8.3.1-14.el7_5 will be installed---> Package perl-TermReadKey.x86_64 0:2.30-20.el7 will be installed--> Finished Dependency ResolutionDependencies Resolved=================================================================================================================================================================================== Package                                        Arch                                 Version                                           Repository                             Size===================================================================================================================================================================================Installing: git                                            x86_64                               1.8.3.1-14.el7_5                                  updates                               4.4 MInstalling for dependencies: libgnome-keyring                               x86_64                               3.12.0-1.el7                                      base                                  109 k perl-Error                                     noarch                               1:0.17020-2.el7                                   base                                   32 k perl-Git                                       noarch                               1.8.3.1-14.el7_5                                  updates                                54 k perl-TermReadKey                               x86_64                               2.30-20.el7                                       base                                   31 kTransaction Summary===================================================================================================================================================================================Install  1 Package (+4 Dependent packages)Total download size: 4.6 MInstalled size: 23 MDownloading packages:(1/5): perl-Error-0.17020-2.el7.noarch.rpm                                                                                                                  |  32 kB  00:00:00     (2/5): libgnome-keyring-3.12.0-1.el7.x86_64.rpm                                                                                                             | 109 kB  00:00:00     (3/5): perl-TermReadKey-2.30-20.el7.x86_64.rpm                                                                                                              |  31 kB  00:00:00     (4/5): perl-Git-1.8.3.1-14.el7_5.noarch.rpm                                                                                                                 |  54 kB  00:00:00     (5/5): git-1.8.3.1-14.el7_5.x86_64.rpm                                                                                                                      | 4.4 MB  00:00:00     -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------Total                                                                                                                                               19 MB/s | 4.6 MB  00:00:00     Running transaction checkRunning transaction testTransaction test succeededRunning transaction  Installing : 1:perl-Error-0.17020-2.el7.noarch                                                                                                                               1/5   Installing : perl-TermReadKey-2.30-20.el7.x86_64                                                                                                                             2/5   Installing : libgnome-keyring-3.12.0-1.el7.x86_64                                                                                                                            3/5   Installing : git-1.8.3.1-14.el7_5.x86_64                                                                                                                                     4/5   Installing : perl-Git-1.8.3.1-14.el7_5.noarch                                                                                                                                5/5   Verifying  : 1:perl-Error-0.17020-2.el7.noarch                                                                                                                               1/5   Verifying  : git-1.8.3.1-14.el7_5.x86_64                                                                                                                                     2/5   Verifying  : libgnome-keyring-3.12.0-1.el7.x86_64                                                                                                                            3/5   Verifying  : perl-Git-1.8.3.1-14.el7_5.noarch                                                                                                                                4/5   Verifying  : perl-TermReadKey-2.30-20.el7.x86_64                                                                                                                             5/5 Installed:  git.x86_64 0:1.8.3.1-14.el7_5                                                                                                                                                    Dependency Installed:  libgnome-keyring.x86_64 0:3.12.0-1.el7         perl-Error.noarch 1:0.17020-2.el7         perl-Git.noarch 0:1.8.3.1-14.el7_5         perl-TermReadKey.x86_64 0:2.30-20.el7        Complete![root@node101 ~]#
[root@node101 ~]# yum -y install git

2>.克隆kafka-monitor 

[root@node101 ~]# cd /yinzhengjie/kafka-monitor/[root@node101 kafka-monitor]# [root@node101 kafka-monitor]# lltotal 0[root@node101 kafka-monitor]# [root@node101 kafka-monitor]# git clone https://github.com/linkedin/kafka-monitor.gitCloning into 'kafka-monitor'...remote: Enumerating objects: 16, done.remote: Counting objects: 100% (16/16), done.remote: Compressing objects: 100% (12/12), done.remote: Total 1199 (delta 1), reused 7 (delta 0), pack-reused 1183Receiving objects: 100% (1199/1199), 583.88 KiB | 248.00 KiB/s, done.Resolving deltas: 100% (479/479), done.[root@node101 kafka-monitor]#[root@node101 kafka-monitor]# lltotal 0drwxr-xr-x 11 root root 309 Dec  3 17:38 kafka-monitor[root@node101 kafka-monitor]#
[root@node101 kafka-monitor]# git clone https://github.com/linkedin/kafka-monitor.git

3>.构建gradlew 

[root@yinzhengjie kafka-monitor]# lltotal 64drwxr-xr-x 3 root root   129 Dec  3 17:35 bindrwxr-xr-x 7 root root    85 Dec  3 18:14 build-rw-r--r-- 1 root root  1797 Dec  3 17:35 build.gradledrwxr-xr-x 2 root root    50 Dec  3 17:35 checkstyledrwxr-xr-x 2 root root   130 Dec  3 17:35 config-rw-r--r-- 1 root root   906 Dec  3 17:35 CONTRIBUTING.mddrwxr-xr-x 2 root root    90 Dec  3 17:35 dockerdrwxr-xr-x 3 root root    19 Dec  3 17:35 docsdrwxr-xr-x 3 root root    20 Dec  3 17:35 gradle-rwxr-xr-x 1 root root  4971 Dec  3 17:35 gradlew-rw-r--r-- 1 root root  2404 Dec  3 17:35 gradlew.bat-rw-r--r-- 1 root root 28832 Dec  3 17:35 LICENSE-rw-r--r-- 1 root root  1371 Dec  3 17:35 NOTICE-rw-r--r-- 1 root root  6369 Dec  3 17:35 README.md-rw-r--r-- 1 root root     0 Dec  3 17:35 settings.gradledrwxr-xr-x 4 root root    28 Dec  3 17:35 srcdrwxr-xr-x 3 root root    32 Dec  3 17:35 webapp[root@yinzhengjie kafka-monitor]# ./gradlew jar

 　　下面是构建时他会自动联网下载一些maven依赖关系：

　　如果你安装成功的话，说明问题不大， 但是你的Java环境如果没有部署好的话，可能会抛异常如下：  

..................(一系列消息被我省略)18:12:29.045 [INFO] [org.gradle.api.internal.tasks.compile.JdkJavaCompiler] Compiling with JDK Java compiler API.18:12:29.045 [DEBUG] [org.gradle.api.internal.tasks.execution.ResolveTaskArtifactStateTaskExecuter] Removed task artifact state for {} from context.18:12:29.045 [DEBUG] [org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter] Finished executing task ':compileJava'18:12:29.046 [LIFECYCLE] [class org.gradle.internal.buildevents.TaskExecutionLogger] :compileJava FAILED18:12:29.046 [INFO] [org.gradle.execution.taskgraph.AbstractTaskPlanExecutor] :compileJava (Thread[Daemon worker Thread 3,5,main]) completed. Took 0.06 secs.18:12:29.046 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationWorkerRegistry] Worker root.1 completed (0 in use)18:12:29.046 [DEBUG] [org.gradle.execution.taskgraph.AbstractTaskPlanExecutor] Task worker [Thread[Daemon worker Thread 3,5,main]] finished, busy: 0.06 secs, idle: 0.0 secs18:12:29.047 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] 18:12:29.047 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] FAILURE: Build failed with an exception.18:12:29.047 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] 18:12:29.048 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] * What went wrong:18:12:29.048 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] Execution failed for task ':compileJava'.18:12:29.048 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] > Could not find tools.jar. Please check that /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-0.el7_5.x86_64/jre contains a valid JDK installation.18:12:29.048 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] 18:12:29.048 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] * Try:18:12:29.048 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] Run with --stacktrace option to get the stack trace. 18:12:29.048 [LIFECYCLE] [org.gradle.internal.buildevents.BuildResultLogger] 18:12:29.048 [LIFECYCLE] [org.gradle.internal.buildevents.BuildResultLogger] BUILD FAILED18:12:29.048 [LIFECYCLE] [org.gradle.internal.buildevents.BuildResultLogger] 18:12:29.048 [LIFECYCLE] [org.gradle.internal.buildevents.BuildResultLogger] Total time: 1.174 secs18:12:29.060 [DEBUG] [org.gradle.cache.internal.LockOnDemandCrossProcessCacheAccess] Releasing file lock for task history cache (/data/yinzhengjie/softwares/kafkasoft/kafka-monitor/kafka-monitor/.gradle/3.2.1/taskArtifacts)18:12:29.060 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on task history cache (/data/yinzhengjie/softwares/kafkasoft/kafka-monitor/kafka-monitor/.gradle/3.2.1/taskArtifacts).18:12:29.064 [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.resolveengine.store.CachedStoreFactory] Resolution result cache closed. Cache reads: 0, disk reads: 0 (avg: 0.0 secs, total: 0.0 secs)18:12:29.064 [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.resolveengine.store.CachedStoreFactory] Resolution result cache closed. Cache reads: 0, disk reads: 0 (avg: 0.0 secs, total: 0.0 secs)18:12:29.064 [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.resolveengine.store.ResolutionResultsStoreFactory] Deleted 2 resolution results binary files in 0.001 secs18:12:29.064 [DEBUG] [org.gradle.cache.internal.DefaultCacheAccess] Cache Plugin Resolution Cache (/root/.gradle/caches/3.2.1/plugin-resolution) was closed 0 times.18:12:29.064 [DEBUG] [org.gradle.cache.internal.DefaultCacheAccess] Cache Generated Gradle JARs cache (/root/.gradle/caches/3.2.1/generated-gradle-jars) was closed 0 times.18:12:29.064 [DEBUG] [org.gradle.api.internal.tasks.compile.daemon.CompilerDaemonManager] Stopping 0 compiler daemon(s).18:12:29.065 [INFO] [org.gradle.api.internal.tasks.compile.daemon.CompilerDaemonManager] Stopped 0 compiler daemon(s).18:12:29.065 [DEBUG] [org.gradle.cache.internal.LockOnDemandCrossProcessCacheAccess] Releasing file lock for artifact cache (/root/.gradle/caches/modules-2)18:12:29.065 [DEBUG] [org.gradle.cache.internal.btree.BTreePersistentIndexedCache] Closing cache module-metadata.bin (/root/.gradle/caches/modules-2/metadata-2.23/module-metadata.bin)18:12:29.065 [DEBUG] [org.gradle.cache.internal.btree.BTreePersistentIndexedCache] Closing cache artifact-at-repository.bin (/root/.gradle/caches/modules-2/metadata-2.23/artifact-at-repository.bin)18:12:29.065 [DEBUG] [org.gradle.cache.internal.btree.BTreePersistentIndexedCache] Closing cache module-artifacts.bin (/root/.gradle/caches/modules-2/metadata-2.23/module-artifacts.bin)18:12:29.065 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on artifact cache (/root/.gradle/caches/modules-2).18:12:29.065 [DEBUG] [org.gradle.launcher.daemon.server.exec.ExecuteBuild] The daemon has finished executing the build.18:12:29.275 [DEBUG] [org.gradle.launcher.daemon.client.DaemonClientInputForwarder] Dispatching close input message: org.gradle.launcher.daemon.protocol.CloseInput@1f6ddd9f18:12:29.275 [DEBUG] [org.gradle.launcher.daemon.client.DaemonClientConnection] thread 14: dispatching class org.gradle.launcher.daemon.protocol.CloseInput18:12:29.276 [INFO] [org.gradle.launcher.daemon.client.DaemonClient] Received result Failure[value=org.gradle.initialization.ReportedException: org.gradle.internal.exceptions.LocationAwareException: Execution failed for task ':compileJava'.] from daemon DaemonInfo{pid=18961, address=[d4fe0de0-9944-4ad9-a617-eae681b59f33 port:42738, addresses:[/127.0.0.1]], state=Idle, lastBusy=1543831940300, context=DefaultDaemonContext[uid=16910369-6f16-4b04-b799-3e7b93a3a0e6,javaHome=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-0.el7_5.x86_64/jre,daemonRegistryDir=/root/.gradle/daemon,pid=18961,idleTimeout=10800000,daemonOpts=-XX:MaxPermSize=256m,-XX:+HeapDumpOnOutOfMemoryError,-Xmx1024m,-Dfile.encoding=UTF-8,-Duser.country=US,-Duser.language=en,-Duser.variant]} (build should be done).18:12:29.276 [DEBUG] [org.gradle.launcher.daemon.client.DaemonClientConnection] thread 1: dispatching class org.gradle.launcher.daemon.protocol.Finished18:12:29.277 [DEBUG] [org.gradle.launcher.daemon.client.DaemonClientConnection] thread 1: connection stop[root@yinzhengjie kafka-monitor]#

　　报错分析：

　　　　根据上述报错信息，提示找不到tools.jar。可能是你的java是基于yum方式安装的，很多工具包可能不全。

　　解决方案：

　　　　将你下载好的jdk中存在的tools.jar包拷贝过去即可。

[root@yinzhengjie ~]# find / -name tools.jar/data/yinzhengjie/softwares/jdk/jdk1.8.0_131/lib/tools.jar[root@yinzhengjie ~]#[root@yinzhengjie ~]# cp /data/yinzhengjie/softwares/jdk/jdk1.8.0_131/lib/tools.jar /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-0.el7_5.x86_64/jre/lib/[root@yinzhengjie ~]#[root@yinzhengjie ~]# cp /data/yinzhengjie/softwares/jdk/jdk1.8.0_131/lib/tools.jar /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-0.el7_5.x86_64/jre/[root@yinzhengjie ~]# [root@yinzhengjie ~]# find / -name tools.jar/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-0.el7_5.x86_64/jre/lib/tools.jar/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-0.el7_5.x86_64/jre/tools.jar/data/yinzhengjie/softwares/jdk/jdk1.8.0_131/lib/tools.jar[root@yinzhengjie ~]# 

　　将tools.jar包拷贝到指定环境变量后，我们开始执行以下操作：（你会发现变异成功啦！）

[root@yinzhengjie kafka-monitor]#  lltotal 64drwxr-xr-x 3 root root   129 Dec  3 17:35 bindrwxr-xr-x 7 root root    85 Dec  3 18:14 build-rw-r--r-- 1 root root  1797 Dec  3 17:35 build.gradledrwxr-xr-x 2 root root    50 Dec  3 17:35 checkstyledrwxr-xr-x 2 root root   130 Dec  3 17:35 config-rw-r--r-- 1 root root   906 Dec  3 17:35 CONTRIBUTING.mddrwxr-xr-x 2 root root    90 Dec  3 17:35 dockerdrwxr-xr-x 3 root root    19 Dec  3 17:35 docsdrwxr-xr-x 3 root root    20 Dec  3 17:35 gradle-rwxr-xr-x 1 root root  4971 Dec  3 17:35 gradlew-rw-r--r-- 1 root root  2404 Dec  3 17:35 gradlew.bat-rw-r--r-- 1 root root 28832 Dec  3 17:35 LICENSE-rw-r--r-- 1 root root  1371 Dec  3 17:35 NOTICE-rw-r--r-- 1 root root  6369 Dec  3 17:35 README.md-rw-r--r-- 1 root root     0 Dec  3 17:35 settings.gradledrwxr-xr-x 4 root root    28 Dec  3 17:35 srcdrwxr-xr-x 3 root root    32 Dec  3 17:35 webapp[root@yinzhengjie kafka-monitor]# [root@yinzhengjie kafka-monitor]#  ./gradlew jar:compileJava UP-TO-DATE:processResources UP-TO-DATE:classes UP-TO-DATE:copyDependantLibs UP-TO-DATE:jar UP-TO-DATEBUILD SUCCESSFULTotal time: 1.073 secs[root@yinzhengjie kafka-monitor]#

4>.修改kafka-monitor.properties配置文件

[root@yinzhengjie kafka-monitor]# pwd/yinzhengjie/softwares/kafkasoft/kafka-monitor/kafka-monitor[root@yinzhengjie kafka-monitor]# [root@yinzhengjie kafka-monitor]# cat config/kafka-monitor.properties | grep -v ^# | grep -v ^${  "single-cluster-monitor": {    "class.name": "com.linkedin.kmf.apps.SingleClusterMonitor",    "topic": "kafka-monitor-topic",    "zookeeper.connect": "10.1.3.117:2181,10.1.3.118:2181,10.1.3.119:2181",　　　　　　　　　　　　　　　　　　　　　　　　#指定zookeeper地址    "bootstrap.servers": "10.1.3.116:9092,10.1.3.117:9092,10.1.3.118:9092,10.1.3.119:9092,10.1.3.120:9092",　　 　　#指定broker地址    "produce.record.delay.ms": 100,    "topic-management.topicCreationEnabled": true,    "topic-management.replicationFactor" : 1,    "topic-management.partitionsToBrokersRatio" : 2.0,    "topic-management.rebalance.interval.ms" : 600000,    "topic-management.topicFactory.props": {    },    "topic-management.topic.props": {      "retention.ms": "3600000"    },    "produce.producer.props": {      "client.id": "kmf-client-id"    },    "consume.latency.sla.ms": "20000",    "consume.consumer.props": {    }  },  "jetty-service": {    "class.name": "com.linkedin.kmf.services.JettyService",    "jetty.port": 8000  },  "jolokia-service": {    "class.name": "com.linkedin.kmf.services.JolokiaService"  },  "reporter-service": {    "class.name": "com.linkedin.kmf.services.DefaultMetricsReporterService",    "report.interval.sec": 1,    "report.metrics.list": [      "kmf:type=kafka-monitor:offline-runnable-count",      "kmf.services:type=produce-service,name=*:produce-availability-avg",      "kmf.services:type=consume-service,name=*:consume-availability-avg",      "kmf.services:type=produce-service,name=*:records-produced-total",      "kmf.services:type=consume-service,name=*:records-consumed-total",      "kmf.services:type=consume-service,name=*:records-lost-total",      "kmf.services:type=consume-service,name=*:records-duplicated-total",      "kmf.services:type=consume-service,name=*:records-delay-ms-avg",      "kmf.services:type=produce-service,name=*:records-produced-rate",      "kmf.services:type=produce-service,name=*:produce-error-rate",      "kmf.services:type=consume-service,name=*:consume-error-rate"    ]  }}[root@yinzhengjie kafka-monitor]#

5>.启动kafka monitor服务

[root@yinzhengjie kafka-monitor]# pwd/yinzhengjie/softwares/kafkasoft/kafka-monitor/kafka-monitor[root@yinzhengjie kafka-monitor]# [root@yinzhengjie kafka-monitor]# lltotal 64drwxr-xr-x 3 root root   129 Dec  3 17:35 bindrwxr-xr-x 7 root root    85 Dec  3 18:14 build-rw-r--r-- 1 root root  1797 Dec  3 17:35 build.gradledrwxr-xr-x 2 root root    50 Dec  3 17:35 checkstyledrwxr-xr-x 2 root root   130 Dec  3 19:26 config-rw-r--r-- 1 root root   906 Dec  3 17:35 CONTRIBUTING.mddrwxr-xr-x 2 root root    90 Dec  3 17:35 dockerdrwxr-xr-x 3 root root    19 Dec  3 17:35 docsdrwxr-xr-x 3 root root    20 Dec  3 17:35 gradle-rwxr-xr-x 1 root root  4971 Dec  3 17:35 gradlew-rw-r--r-- 1 root root  2404 Dec  3 17:35 gradlew.bat-rw-r--r-- 1 root root 28832 Dec  3 17:35 LICENSEdrwxr-xr-x 2 root root    29 Dec  3 19:24 logs-rw-r--r-- 1 root root  1371 Dec  3 17:35 NOTICE-rw-r--r-- 1 root root  6369 Dec  3 17:35 README.md-rw-r--r-- 1 root root     0 Dec  3 17:35 settings.gradledrwxr-xr-x 4 root root    28 Dec  3 17:35 srcdrwxr-xr-x 3 root root    32 Dec  3 17:35 webapp[root@yinzhengjie kafka-monitor]# yinzhengjie[root@yinzhengjie kafka-monitor]# mkdir -pv /yinzhengjie/softwares/kafkasoft/kafka-monitor/logsmkdir: created directory ‘/yinzhengjie/softwares/kafkasoft/kafka-monitor/logs’[root@yinzhengjie kafka-monitor]# [root@yinzhengjie kafka-monitor]# bin/kafka-monitor-start.sh config/kafka-monitor.properties >> /yinzhengjie/softwares/kafkasoft/kafka-monitor/logs/kafka-monitor-`date +%F`.log &[1] 16888[root@yinzhengjie kafka-monitor]# [root@yinzhengjie kafka-monitor]# jobs [1]+  Running                 bin/kafka-monitor-start.sh config/kafka-monitor.properties >> /yinzhengjie/softwares/kafkasoft/kafka-monitor/logs/kafka-monitor-`date +%F`.log &[root@yinzhengjie kafka-monitor]# [root@yinzhengjie kafka-monitor]# tail -1000f /yinzhengjie/softwares/kafkasoft/kafka-monitor/logs/kafka-monitor-2018-12-03.log  | more [2018-12-03 19:30:12,160] INFO MultiClusterTopicManagementServiceConfig values:         topic = kafka-monitor-topic        topic-management.rebalance.interval.ms = 600000 (com.linkedin.kmf.services.configs.MultiClusterTopicManagementServiceConfig)[2018-12-03 19:30:12,240] INFO TopicManagementServiceConfig values:         topic = kafka-monitor-topic        topic-management.minPartitionNum = 1        topic-management.partitionsToBrokersRatio = 2.0        topic-management.replicationFactor = 1        topic-management.topicCreationEnabled = true        topic-management.topicFactory.class.name = com.linkedin.kmf.topicfactory.DefaultTopicFactory        zookeeper.connect = 10.1.3.117:2181,10.1.3.118:2181,10.1.3.119:2181 (com.linkedin.kmf.services.configs.TopicManagementServiceConfig)[2018-12-03 19:30:12,244] INFO ProduceServiceConfig values:         bootstrap.servers = 10.1.3.116:9092,10.1.3.117:9092,10.1.3.118:9092,10.1.3.119:9092,10.1.3.120:9092        produce.latency.percentile.granularity.ms = 1        produce.latency.percentile.max.ms = 5000        produce.partitioner.class = class com.linkedin.kmf.partitioner.NewKMPartitioner        produce.producer.class = com.linkedin.kmf.producer.NewProducer        produce.producer.id = kmf-producer        produce.record.delay.ms = 100        produce.record.size.byte = 100        produce.sync = true        produce.thread.num = 5        produce.treat.zero.throughput.as.unavailable = true        topic = kafka-monitor-topic        zookeeper.connect = 10.1.3.117:2181,10.1.3.118:2181,10.1.3.119:2181 (com.linkedin.kmf.services.configs.ProduceServiceConfig)[2018-12-03 19:30:12,338] INFO single-cluster-monitor/ProduceService is initialized. (com.linkedin.kmf.services.ProduceService)[2018-12-03 19:30:12,347] INFO ConsumeServiceConfig values:         bootstrap.servers = 10.1.3.116:9092,10.1.3.117:9092,10.1.3.118:9092,10.1.3.119:9092,10.1.3.120:9092        consume.consumer.class = com.linkedin.kmf.consumer.NewConsumer        consume.latency.percentile.granularity.ms = 1        consume.latency.percentile.max.ms = 5000        consume.latency.sla.ms = 20000        topic = kafka-monitor-topic        zookeeper.connect = 10.1.3.117:2181,10.1.3.118:2181,10.1.3.119:2181 (com.linkedin.kmf.services.configs.ConsumeServiceConfig)--More--

6>.检查服务是否启动成功 

[root@yinzhengjie ~]# netstat -untalp | grep 8000tcp        0      0 0.0.0.0:8000            0.0.0.0:*               LISTEN      16888/java                      [root@yinzhengjie ~]# [root@yinzhengjie ~]# hostname -i10.19.104.189[root@yinzhengjie ~]#

　　我们可以通过webUI的方式访问kafka-monitor服务：

 

　　Kafka Monitor作为LinkedIn Kafka 团队维护的开源框架，从系统测试的角度给出了kafka集群的性能评测结果。从这个意义上来说，它不同于其他的监控框架。且平台目前提供的Web界面功能十分简单，因此实际使用用户数并不是很多。不过鉴于其代码贡献者基本上都是Kafka社区团队成员，其框架的发展还是值得期待的。 

 

二.安装Kafka Offset Monitor

 　　这款监控工具不推荐大家使用，因为这个项目已经不维护了啦。尤其是kafka0.9.0.0以后的kafka版本，不建议使用，因为我用kafka0.10.2.1这个版本时，它是无法监控到broker集群的信息的！这一点很让人蛋疼！令人欣喜的是，该项目又一个同名的“姊妹”项目，地址是：“https://github.com/Morningstar/kafka-offset-monitor”，感兴趣的小伙伴可以捣鼓一下。

　　我已经下载好相应的安装包了，感兴趣的小伙伴可以直接拿去用，我给出百度云链接:https://pan.baidu.com/s/1Qbgtxtdd_1B2hlqatA0GRQ  密码:u991

[root@yinzhengjie kafka-offset-monitor]# lltotal 54468-rw-r--r-- 1 root root 52937155 Dec  3 20:00 KafkaOffsetMonitor-assembly-0.2.1.jar-rw-r--r-- 1 root root  2830336 Dec  3 20:38 offsetapp.db[root@yinzhengjie kafka-offset-monitor]# [root@yinzhengjie kafka-offset-monitor]# java -cp KafkaOffsetMonitor-assembly-0.2.1.jar:kafka-offset-monitor-another-db-reporter.jar com.quantifind.kafka.offsetapp.OffsetGetterWeb --zk 10.1.3.117:2181,10.1.3.118:2181,10.1.3.119:2181 --port 1010 --refresh 10.seconds --retain 2.days serving resources from: jar:file:/data/yinzhengjie/softwares/kafkasoft/kafka-monitor/kafka-offset-monitor/KafkaOffsetMonitor-assembly-0.2.1.jar!/offsetappSLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".SLF4J: Defaulting to no-operation (NOP) logger implementationSLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.log4j:WARN No appenders could be found for logger (org.I0Itec.zkclient.ZkConnection).log4j:WARN Please initialize the log4j system properly.log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.log4j:WARN No appenders could be found for logger (org.I0Itec.zkclient.ZkEventThread).log4j:WARN Please initialize the log4j system properly.log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.2018-12-03 20:08:56.101:INFO:oejs.Server:jetty-7.x.y-SNAPSHOT2018-12-03 20:08:56.132:INFO:oejsh.ContextHandler:started o.e.j.s.ServletContextHandler{/,jar:file:/data/yinzhengjie/softwares/kafkasoft/kafka-monitor/kafka-offset-monitor/KafkaOffsetMonitor-assembly-0.2.1.jar!/offsetapp}2018-12-03 20:08:56.139:INFO:oejs.AbstractConnector:Started SocketConnector@0.0.0.0:1010

 　　然后你在webUI访问该主机的1010端口即可，如果你的kafka版本过高的话，估计够呛，你应该查不到相应的信息！

 

三.CruiseControl部署

　　我们聊聊LinkedIn公司于2017年8月底开源的Kafka监控框架，即CruiseControl。它主要用于管理查大规模的Kafka集群，提供的功能如下：

　　　　1>.实时监控资源使用率（broker，topic，分区）。

　　　　2>.提供多纬度的重平衡策略：

　　　　　　2.1>.机架感知（rack awareness）

　　　　　　2.2>.资源使用率再平衡（CPU，磁盘，网络I/O）

　　　　　　2.3>.leader副本流量分配

　　　　　　2.4>.副本均匀分布

　　　　3>.异常检测以及实时告警。

　　　　4>.开箱即用的运维操作：

　　　　　　4.1>.增加broker节点

　　　　　　4.2>.去除broker节点

　　　　　　4.3>.集群在平衡。

　　由于该框架刚刚开源，有很多功能尚在完善和演进中。打算使用CruiseControl的用户最好时刻关注该框架的发展，其官网地址是：。

　

　接下来我们掩饰一些如何安装，配置和启动CruiseControl的详细步骤。

1>.下载源码

[root@yinzhengjie kafka-cruiseControl]# lltotal 0[root@yinzhengjie kafka-cruiseControl]# [root@yinzhengjie kafka-cruiseControl]# git clone https://github.com/linkedin/cruise-control.gitCloning into 'cruise-control'...remote: Enumerating objects: 47, done.remote: Counting objects: 100% (47/47), done.remote: Compressing objects: 100% (32/32), done.remote: Total 7615 (delta 9), reused 16 (delta 0), pack-reused 7568Receiving objects: 100% (7615/7615), 2.41 MiB | 735.00 KiB/s, done.Resolving deltas: 100% (3537/3537), done.[root@yinzhengjie kafka-cruiseControl]# [root@yinzhengjie kafka-cruiseControl]# lltotal 4drwxr-xr-x 12 root root 4096 Dec  4 10:45 cruise-control[root@yinzhengjie kafka-cruiseControl]#
[root@yinzhengjie kafka-cruiseControl]# git clone https://github.com/linkedin/cruise-control.git

2>.进行编译

[root@yinzhengjie kafka-cruiseControl]# lltotal 4drwxr-xr-x 12 root root 4096 Dec  4 10:45 cruise-control[root@yinzhengjie kafka-cruiseControl]# [root@yinzhengjie kafka-cruiseControl]# cd cruise-control/[root@yinzhengjie cruise-control]# lltotal 68-rw-r--r-- 1 root root  8257 Dec  4 10:45 build.gradledrwxr-xr-x 3 root root    35 Dec  4 10:45 buildSrcdrwxr-xr-x 2 root root    50 Dec  4 10:45 checkstyledrwxr-xr-x 2 root root    80 Dec  4 10:45 config-rw-r--r-- 1 root root  1079 Dec  4 10:45 CONTRIBUTING.mddrwxr-xr-x 3 root root    16 Dec  4 10:45 cruise-controldrwxr-xr-x 3 root root    16 Dec  4 10:45 cruise-control-coredrwxr-xr-x 3 root root    16 Dec  4 10:45 cruise-control-metrics-reporterdrwxr-xr-x 3 root root    19 Dec  4 10:45 docsdrwxr-xr-x 3 root root    47 Dec  4 10:45 gradle-rw-r--r-- 1 root root    86 Dec  4 10:45 gradle.properties-rwxr-xr-x 1 root root  5299 Dec  4 10:45 gradlew-rw-r--r-- 1 root root  2260 Dec  4 10:45 gradlew.bat-rwxr-xr-x 1 root root  5129 Dec  4 10:45 kafka-cruise-control-start.sh-rw-r--r-- 1 root root  1332 Dec  4 10:45 LICENSE-rw-r--r-- 1 root root   304 Dec  4 10:45 NOTICE-rw-r--r-- 1 root root 11476 Dec  4 10:45 README.md-rw-r--r-- 1 root root   153 Dec  4 10:45 semantic-build-versioning.gradle-rw-r--r-- 1 root root   911 Dec  4 10:45 settings.gradle[root@yinzhengjie cruise-control]# [root@yinzhengjie cruise-control]# ./gradlew clean jarDownloading https://services.gradle.org/distributions/gradle-4.8-all.zip.....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................Unzipping /root/.gradle/wrapper/dists/gradle-4.8-all/41c9mvq2w0645xx6nsenaerjl/gradle-4.8-all.zip to /root/.gradle/wrapper/dists/gradle-4.8-all/41c9mvq2w0645xx6nsenaerjlSet executable permissions for: /root/.gradle/wrapper/dists/gradle-4.8-all/41c9mvq2w0645xx6nsenaerjl/gradle-4.8/bin/gradleWelcome to Gradle 4.8!Here are the highlights of this release: - Dependency locking - Maven Publish and Ivy Publish plugins improved and marked stable - Incremental annotation processing enhancements - APIs to configure tasks at creation timeFor more details see https://docs.gradle.org/4.8/release-notes.htmlTo honour the JVM settings for this build a new JVM will be forked. Please consider using the daemon: https://docs.gradle.org/4.8/userguide/gradle_daemon.html.Daemon will be stopped at the end of the build stopping after processingDownload https://jcenter.bintray.com/org/apache/httpcomponents/fluent-hc/4.5.2/fluent-hc-4.5.2.pomDownload https://jcenter.bintray.com/org/jfrog/buildinfo/build-info-extractor-gradle/4.7.3/build-info-extractor-gradle-4.7.3.pomDownload https://jcenter.bintray.com/org/ajoberstar/gradle-git/1.2.0/gradle-git-1.2.0.pomDownload https://jcenter.bintray.com/org/apache/httpcomponents/httpcomponents-client/4.5.2/httpcomponents-client-4.5.2.pomDownload https://jcenter.bintray.com/org/apache/httpcomponents/project/7/project-7.pomDownload https://jcenter.bintray.com/org/apache/apache/13/apache-13.pomDownload https://jcenter.bintray.com/org/ajoberstar/grgit/1.3.0/grgit-1.3.0.pomDownload https://jcenter.bintray.com/com/github/zafarkhaja/java-semver/0.9.0/java-semver-0.9.0.pomDownload https://jcenter.bintray.com/org/sonatype/oss/oss-parent/9/oss-parent-9.pomDownload https://jcenter.bintray.com/commons-logging/commons-logging/1.2/commons-logging-1.2.pomDownload https://jcenter.bintray.com/org/apache/httpcomponents/httpclient/4.5.2/httpclient-4.5.2.pomDownload https://jcenter.bintray.com/org/apache/commons/commons-parent/34/commons-parent-34.pomDownload https://jcenter.bintray.com/commons-lang/commons-lang/2.4/commons-lang-2.4.pomDownload https://jcenter.bintray.com/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.pomDownload https://jcenter.bintray.com/org/jfrog/buildinfo/build-info-extractor/2.11.2/build-info-extractor-2.11.2.pomDownload https://jcenter.bintray.com/org/apache/apache/7/apache-7.pomDownload https://jcenter.bintray.com/commons-io/commons-io/2.2/commons-io-2.2.pomDownload https://jcenter.bintray.com/org/apache/commons/commons-parent/24/commons-parent-24.pomDownload https://jcenter.bintray.com/org/apache/apache/9/apache-9.pomDownload https://jcenter.bintray.com/com/jcraft/jsch.agentproxy.usocket-jna/0.0.7/jsch.agentproxy.usocket-jna-0.0.7.pomDownload https://jcenter.bintray.com/org/eclipse/jgit/org.eclipse.jgit/3.7.0.201502260915-r/org.eclipse.jgit-3.7.0.201502260915-r.pomDownload https://jcenter.bintray.com/com/jcraft/jsch.agentproxy.pageant/0.0.7/jsch.agentproxy.pageant-0.0.7.pomDownload https://jcenter.bintray.com/com/jcraft/jsch.agentproxy.usocket-nc/0.0.7/jsch.agentproxy.usocket-nc-0.0.7.pomDownload https://jcenter.bintray.com/com/jcraft/jsch.agentproxy/0.0.7/jsch.agentproxy-0.0.7.pomDownload https://jcenter.bintray.com/org/eclipse/jgit/org.eclipse.jgit-parent/3.7.0.201502260915-r/org.eclipse.jgit-parent-3.7.0.201502260915-r.pomDownload https://jcenter.bintray.com/org/sonatype/oss/oss-parent/6/oss-parent-6.pomDownload https://jcenter.bintray.com/com/jcraft/jsch.agentproxy.jsch/0.0.7/jsch.agentproxy.jsch-0.0.7.pomDownload https://jcenter.bintray.com/org/eclipse/jgit/org.eclipse.jgit.ui/3.7.0.201502260915-r/org.eclipse.jgit.ui-3.7.0.201502260915-r.pomDownload https://jcenter.bintray.com/com/jcraft/jsch.agentproxy.sshagent/0.0.7/jsch.agentproxy.sshagent-0.0.7.pomDownload https://jcenter.bintray.com/org/slf4j/slf4j-api/1.7.7/slf4j-api-1.7.7.pomDownload https://jcenter.bintray.com/org/slf4j/slf4j-parent/1.7.7/slf4j-parent-1.7.7.pomDownload https://jcenter.bintray.com/org/apache/httpcomponents/httpcore/4.4.4/httpcore-4.4.4.pomDownload https://jcenter.bintray.com/commons-codec/commons-codec/1.9/commons-codec-1.9.pomDownload https://jcenter.bintray.com/org/apache/httpcomponents/httpcomponents-core/4.4.4/httpcomponents-core-4.4.4.pomDownload https://jcenter.bintray.com/org/apache/commons/commons-parent/32/commons-parent-32.pomDownload https://jcenter.bintray.com/com/google/guava/guava/18.0/guava-18.0.pomDownload https://jcenter.bintray.com/com/thoughtworks/xstream/xstream/1.4.10/xstream-1.4.10.pomDownload https://jcenter.bintray.com/org/jfrog/buildinfo/build-info-client/2.11.2/build-info-client-2.11.2.pomDownload https://jcenter.bintray.com/com/google/guava/guava-parent/18.0/guava-parent-18.0.pomDownload https://jcenter.bintray.com/com/thoughtworks/xstream/xstream-parent/1.4.10/xstream-parent-1.4.10.pomDownload https://jcenter.bintray.com/com/googlecode/javaewah/JavaEWAH/0.7.9/JavaEWAH-0.7.9.pomDownload https://jcenter.bintray.com/com/jcraft/jsch/0.1.50/jsch-0.1.50.pomDownload https://jcenter.bintray.com/org/sonatype/oss/oss-parent/5/oss-parent-5.pomDownload https://jcenter.bintray.com/com/jcraft/jsch.agentproxy.core/0.0.7/jsch.agentproxy.core-0.0.7.pomDownload https://jcenter.bintray.com/net/java/dev/jna/platform/3.4.0/platform-3.4.0.pomDownload https://jcenter.bintray.com/net/java/dev/jna/jna/3.4.0/jna-3.4.0.pomDownload https://jcenter.bintray.com/xmlpull/xmlpull/1.1.3.1/xmlpull-1.1.3.1.pomDownload https://jcenter.bintray.com/xpp3/xpp3_min/1.1.4c/xpp3_min-1.1.4c.pomDownload https://jcenter.bintray.com/org/apache/httpcomponents/httpcore/4.4.5/httpcore-4.4.5.pomDownload https://jcenter.bintray.com/org/apache/httpcomponents/httpcomponents-core/4.4.5/httpcomponents-core-4.4.5.pomDownload https://jcenter.bintray.com/com/fasterxml/jackson/core/jackson-core/2.9.5/jackson-core-2.9.5.pomDownload https://jcenter.bintray.com/org/jfrog/buildinfo/build-info-api/2.11.2/build-info-api-2.11.2.pomDownload https://jcenter.bintray.com/com/fasterxml/jackson/jackson-base/2.9.5/jackson-base-2.9.5.pomDownload https://jcenter.bintray.com/com/fasterxml/jackson/jackson-bom/2.9.5/jackson-bom-2.9.5.pomDownload https://jcenter.bintray.com/com/fasterxml/jackson/jackson-parent/2.9.1/jackson-parent-2.9.1.pomDownload https://jcenter.bintray.com/com/fasterxml/oss-parent/30/oss-parent-30.pomDownload https://jcenter.bintray.com/com/fasterxml/jackson/core/jackson-databind/2.9.5/jackson-databind-2.9.5.pomDownload https://jcenter.bintray.com/org/apache/commons/commons-compress/1.16/commons-compress-1.16.pomDownload https://jcenter.bintray.com/org/apache/commons/commons-parent/43/commons-parent-43.pomDownload https://jcenter.bintray.com/org/apache/apache/18/apache-18.pomDownload https://jcenter.bintray.com/org/objenesis/objenesis/2.6/objenesis-2.6.pomDownload https://jcenter.bintray.com/org/objenesis/objenesis-parent/2.6/objenesis-parent-2.6.pomGenerating JAR file 'gradle-api-4.8.jar'Download https://jcenter.bintray.com/org/jfrog/buildinfo/build-info-extractor-gradle/4.7.3/build-info-extractor-gradle-4.7.3.jarDownload https://jcenter.bintray.com/org/apache/httpcomponents/fluent-hc/4.5.2/fluent-hc-4.5.2.jarDownload https://jcenter.bintray.com/org/eclipse/jgit/org.eclipse.jgit.ui/3.7.0.201502260915-r/org.eclipse.jgit.ui-3.7.0.201502260915-r.jarDownload https://jcenter.bintray.com/org/jfrog/buildinfo/build-info-client/2.11.2/build-info-client-2.11.2.jarDownload https://jcenter.bintray.com/org/jfrog/buildinfo/build-info-extractor/2.11.2/build-info-extractor-2.11.2.jarDownload https://jcenter.bintray.com/org/jfrog/buildinfo/build-info-api/2.11.2/build-info-api-2.11.2.jarDownload https://jcenter.bintray.com/org/eclipse/jgit/org.eclipse.jgit/3.7.0.201502260915-r/org.eclipse.jgit-3.7.0.201502260915-r.jarDownload https://jcenter.bintray.com/commons-logging/commons-logging/1.2/commons-logging-1.2.jarDownload https://jcenter.bintray.com/org/ajoberstar/grgit/1.3.0/grgit-1.3.0.jarDownload https://jcenter.bintray.com/commons-io/commons-io/2.2/commons-io-2.2.jarDownload https://jcenter.bintray.com/org/ajoberstar/gradle-git/1.2.0/gradle-git-1.2.0.jarDownload https://jcenter.bintray.com/org/apache/httpcomponents/httpclient/4.5.2/httpclient-4.5.2.jarDownload https://jcenter.bintray.com/com/jcraft/jsch.agentproxy.jsch/0.0.7/jsch.agentproxy.jsch-0.0.7.jarDownload https://jcenter.bintray.com/com/jcraft/jsch.agentproxy.pageant/0.0.7/jsch.agentproxy.pageant-0.0.7.jarDownload https://jcenter.bintray.com/com/jcraft/jsch.agentproxy.sshagent/0.0.7/jsch.agentproxy.sshagent-0.0.7.jarDownload https://jcenter.bintray.com/commons-lang/commons-lang/2.4/commons-lang-2.4.jarDownload https://jcenter.bintray.com/com/jcraft/jsch.agentproxy.usocket-jna/0.0.7/jsch.agentproxy.usocket-jna-0.0.7.jarDownload https://jcenter.bintray.com/org/slf4j/slf4j-api/1.7.7/slf4j-api-1.7.7.jarDownload https://jcenter.bintray.com/com/jcraft/jsch.agentproxy.usocket-nc/0.0.7/jsch.agentproxy.usocket-nc-0.0.7.jarDownload https://jcenter.bintray.com/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jarDownload https://jcenter.bintray.com/org/apache/httpcomponents/httpcore/4.4.5/httpcore-4.4.5.jarDownload https://jcenter.bintray.com/commons-codec/commons-codec/1.9/commons-codec-1.9.jarDownload https://jcenter.bintray.com/com/google/guava/guava/18.0/guava-18.0.jarDownload https://jcenter.bintray.com/com/thoughtworks/xstream/xstream/1.4.10/xstream-1.4.10.jarDownload https://jcenter.bintray.com/com/googlecode/javaewah/JavaEWAH/0.7.9/JavaEWAH-0.7.9.jarDownload https://jcenter.bintray.com/com/jcraft/jsch.agentproxy.core/0.0.7/jsch.agentproxy.core-0.0.7.jarDownload https://jcenter.bintray.com/com/jcraft/jsch/0.1.50/jsch-0.1.50.jarDownload https://jcenter.bintray.com/net/java/dev/jna/platform/3.4.0/platform-3.4.0.jarDownload https://jcenter.bintray.com/xmlpull/xmlpull/1.1.3.1/xmlpull-1.1.3.1.jarDownload https://jcenter.bintray.com/net/java/dev/jna/jna/3.4.0/jna-3.4.0.jarDownload https://jcenter.bintray.com/xpp3/xpp3_min/1.1.4c/xpp3_min-1.1.4c.jarDownload https://jcenter.bintray.com/com/fasterxml/jackson/core/jackson-databind/2.9.5/jackson-databind-2.9.5.jarDownload https://jcenter.bintray.com/org/objenesis/objenesis/2.6/objenesis-2.6.jarDownload https://jcenter.bintray.com/com/fasterxml/jackson/core/jackson-core/2.9.5/jackson-core-2.9.5.jarDownload https://jcenter.bintray.com/org/apache/commons/commons-compress/1.16/commons-compress-1.16.jarDownload https://jcenter.bintray.com/com/github/zafarkhaja/java-semver/0.9.0/java-semver-0.9.0.jar> Task :buildSrc:jar:jar: No valid plugin descriptors were found in META-INF/gradle-pluginsDownload https://plugins.gradle.org/m2/gradle/plugin/net/vivin/gradle-semantic-build-versioning/4.0.0/gradle-semantic-build-versioning-4.0.0.pomDownload https://plugins.gradle.org/m2/org/eclipse/jgit/org.eclipse.jgit/4.8.0.201706111038-r/org.eclipse.jgit-4.8.0.201706111038-r.pomDownload https://plugins.gradle.org/m2/org/eclipse/jgit/org.eclipse.jgit-parent/4.8.0.201706111038-r/org.eclipse.jgit-parent-4.8.0.201706111038-r.pomDownload https://plugins.gradle.org/m2/com/jcraft/jsch/0.1.54/jsch-0.1.54.pomDownload https://plugins.gradle.org/m2/com/googlecode/javaewah/JavaEWAH/1.1.6/JavaEWAH-1.1.6.pomDownload https://plugins.gradle.org/m2/org/slf4j/slf4j-api/1.7.2/slf4j-api-1.7.2.pomDownload https://plugins.gradle.org/m2/org/apache/httpcomponents/httpclient/4.3.6/httpclient-4.3.6.pomDownload https://plugins.gradle.org/m2/org/slf4j/slf4j-parent/1.7.2/slf4j-parent-1.7.2.pomDownload https://plugins.gradle.org/m2/org/apache/httpcomponents/httpcomponents-client/4.3.6/httpcomponents-client-4.3.6.pomDownload https://plugins.gradle.org/m2/commons-codec/commons-codec/1.6/commons-codec-1.6.pomDownload https://plugins.gradle.org/m2/commons-logging/commons-logging/1.1.3/commons-logging-1.1.3.pomDownload https://plugins.gradle.org/m2/org/apache/httpcomponents/httpcore/4.3.3/httpcore-4.3.3.pomDownload https://plugins.gradle.org/m2/org/apache/commons/commons-parent/28/commons-parent-28.pomDownload https://plugins.gradle.org/m2/org/apache/httpcomponents/httpcomponents-core/4.3.3/httpcomponents-core-4.3.3.pomDownload https://plugins.gradle.org/m2/org/apache/commons/commons-parent/22/commons-parent-22.pomDownload https://plugins.gradle.org/m2/commons-logging/commons-logging/1.1.3/commons-logging-1.1.3.jarDownload https://plugins.gradle.org/m2/com/googlecode/javaewah/JavaEWAH/1.1.6/JavaEWAH-1.1.6.jarDownload https://plugins.gradle.org/m2/gradle/plugin/net/vivin/gradle-semantic-build-versioning/4.0.0/gradle-semantic-build-versioning-4.0.0.jarDownload https://plugins.gradle.org/m2/org/apache/httpcomponents/httpcore/4.3.3/httpcore-4.3.3.jarDownload https://plugins.gradle.org/m2/org/slf4j/slf4j-api/1.7.2/slf4j-api-1.7.2.jarDownload https://plugins.gradle.org/m2/com/jcraft/jsch/0.1.54/jsch-0.1.54.jarDownload https://plugins.gradle.org/m2/commons-codec/commons-codec/1.6/commons-codec-1.6.jarDownload https://plugins.gradle.org/m2/org/apache/httpcomponents/httpclient/4.3.6/httpclient-4.3.6.jarDownload https://plugins.gradle.org/m2/org/eclipse/jgit/org.eclipse.jgit/4.8.0.201706111038-r/org.eclipse.jgit-4.8.0.201706111038-r.jarDownload https://repo.maven.apache.org/maven2/junit/junit/4.12/junit-4.12.pomDownload https://repo.maven.apache.org/maven2/org/apache/commons/commons-math3/3.6.1/commons-math3-3.6.1.pomDownload https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/39/commons-parent-39.pomDownload https://repo.maven.apache.org/maven2/org/apache/apache/16/apache-16.pomDownload https://repo.maven.apache.org/maven2/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.pomDownload https://repo.maven.apache.org/maven2/org/hamcrest/hamcrest-parent/1.3/hamcrest-parent-1.3.pomDownload https://repo.maven.apache.org/maven2/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jarDownload https://repo.maven.apache.org/maven2/junit/junit/4.12/junit-4.12.jarDownload https://repo.maven.apache.org/maven2/org/apache/commons/commons-math3/3.6.1/commons-math3-3.6.1.jar> Task :cruise-control-core:compileJavaNote: /data/yinzhengjie/softwares/kafkasoft/kafka-cruiseControl/cruise-control/cruise-control-core/src/main/java/com/linkedin/cruisecontrol/common/config/ConfigDef.java uses unchecked or unsafe operations.Note: Recompile with -Xlint:unchecked for details.Download https://repo.maven.apache.org/maven2/org/apache/kafka/kafka_2.11/0.11.0.2/kafka_2.11-0.11.0.2.pomDownload https://repo.maven.apache.org/maven2/org/apache/kafka/kafka-clients/0.11.0.2/kafka-clients-0.11.0.2.pomDownload https://repo.maven.apache.org/maven2/org/scala-lang/modules/scala-parser-combinators_2.11/1.0.4/scala-parser-combinators_2.11-1.0.4.pomDownload https://repo.maven.apache.org/maven2/org/slf4j/slf4j-log4j12/1.7.25/slf4j-log4j12-1.7.25.pomDownload https://repo.maven.apache.org/maven2/org/apache/zookeeper/zookeeper/3.4.10/zookeeper-3.4.10.pomDownload https://repo.maven.apache.org/maven2/net/sf/jopt-simple/jopt-simple/5.0.3/jopt-simple-5.0.3.pomDownload https://repo.maven.apache.org/maven2/org/scala-lang/scala-library/2.11.11/scala-library-2.11.11.pomDownload https://repo.maven.apache.org/maven2/net/jpountz/lz4/lz4/1.3.0/lz4-1.3.0.pomDownload https://repo.maven.apache.org/maven2/org/xerial/snappy/snappy-java/1.1.2.6/snappy-java-1.1.2.6.pomDownload https://repo.maven.apache.org/maven2/org/apache/kafka/kafka_2.11/0.11.0.2/kafka_2.11-0.11.0.2.jarDownload https://repo.maven.apache.org/maven2/org/apache/kafka/kafka-clients/0.11.0.2/kafka-clients-0.11.0.2.jarDownload https://repo.maven.apache.org/maven2/net/sf/jopt-simple/jopt-simple/5.0.3/jopt-simple-5.0.3.jarDownload https://repo.maven.apache.org/maven2/org/slf4j/slf4j-log4j12/1.7.25/slf4j-log4j12-1.7.25.jarDownload https://repo.maven.apache.org/maven2/org/scala-lang/scala-library/2.11.11/scala-library-2.11.11.jarDownload https://repo.maven.apache.org/maven2/net/jpountz/lz4/lz4/1.3.0/lz4-1.3.0.jarDownload https://repo.maven.apache.org/maven2/org/xerial/snappy/snappy-java/1.1.2.6/snappy-java-1.1.2.6.jarDownload https://repo.maven.apache.org/maven2/org/apache/zookeeper/zookeeper/3.4.10/zookeeper-3.4.10.jarDownload https://repo.maven.apache.org/maven2/org/scala-lang/modules/scala-parser-combinators_2.11/1.0.4/scala-parser-combinators_2.11-1.0.4.jarDownload https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-server/9.4.6.v20170531/jetty-server-9.4.6.v20170531.pomDownload https://repo.maven.apache.org/maven2/org/apache/zookeeper/zookeeper/3.4.6/zookeeper-3.4.6.pomDownload https://repo.maven.apache.org/maven2/com/google/code/gson/gson/2.7/gson-2.7.pomDownload https://repo.maven.apache.org/maven2/io/dropwizard/metrics/metrics-core/3.2.2/metrics-core-3.2.2.pomDownload https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-servlet/9.4.6.v20170531/jetty-servlet-9.4.6.v20170531.pomDownload https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-project/9.4.6.v20170531/jetty-project-9.4.6.v20170531.pomDownload https://repo.maven.apache.org/maven2/com/google/code/gson/gson-parent/2.7/gson-parent-2.7.pomDownload https://repo.maven.apache.org/maven2/io/dropwizard/metrics/metrics-parent/3.2.2/metrics-parent-3.2.2.pomDownload https://repo.maven.apache.org/maven2/log4j/log4j/1.2.16/log4j-1.2.16.pomDownload https://repo.maven.apache.org/maven2/jline/jline/0.9.94/jline-0.9.94.pomDownload https://repo.maven.apache.org/maven2/io/netty/netty/3.7.0.Final/netty-3.7.0.Final.pomDownload https://repo.maven.apache.org/maven2/org/slf4j/slf4j-log4j12/1.6.1/slf4j-log4j12-1.6.1.pomDownload https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.6.1/slf4j-parent-1.6.1.pomDownload https://repo.maven.apache.org/maven2/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.pomDownload https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-io/9.4.6.v20170531/jetty-io-9.4.6.v20170531.pomDownload https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-http/9.4.6.v20170531/jetty-http-9.4.6.v20170531.pomDownload https://repo.maven.apache.org/maven2/net/java/jvnet-parent/3/jvnet-parent-3.pomDownload https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-security/9.4.6.v20170531/jetty-security-9.4.6.v20170531.pomDownload https://repo.maven.apache.org/maven2/io/netty/netty/3.10.5.Final/netty-3.10.5.Final.pomDownload https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-util/9.4.6.v20170531/jetty-util-9.4.6.v20170531.pomDownload https://repo.maven.apache.org/maven2/io/netty/netty/3.10.5.Final/netty-3.10.5.Final.jarDownload https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-security/9.4.6.v20170531/jetty-security-9.4.6.v20170531.jarDownload https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-servlet/9.4.6.v20170531/jetty-servlet-9.4.6.v20170531.jarDownload https://repo.maven.apache.org/maven2/com/google/code/gson/gson/2.7/gson-2.7.jarDownload https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-server/9.4.6.v20170531/jetty-server-9.4.6.v20170531.jarDownload https://repo.maven.apache.org/maven2/jline/jline/0.9.94/jline-0.9.94.jarDownload https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-http/9.4.6.v20170531/jetty-http-9.4.6.v20170531.jarDownload https://repo.maven.apache.org/maven2/io/dropwizard/metrics/metrics-core/3.2.2/metrics-core-3.2.2.jarDownload https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-io/9.4.6.v20170531/jetty-io-9.4.6.v20170531.jarDownload https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-util/9.4.6.v20170531/jetty-util-9.4.6.v20170531.jarDownload https://repo.maven.apache.org/maven2/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jarDownload https://repo.maven.apache.org/maven2/org/scala-lang/scala-compiler/2.11.11/scala-compiler-2.11.11.pomDownload https://repo.maven.apache.org/maven2/org/scala-lang/modules/scala-xml_2.11/1.0.5/scala-xml_2.11-1.0.5.pomDownload https://repo.maven.apache.org/maven2/org/scala-lang/scala-reflect/2.11.11/scala-reflect-2.11.11.pomDownload https://repo.maven.apache.org/maven2/org/scala-lang/scala-reflect/2.11.11/scala-reflect-2.11.11.jarDownload https://repo.maven.apache.org/maven2/org/scala-lang/scala-compiler/2.11.11/scala-compiler-2.11.11.jarDownload https://repo.maven.apache.org/maven2/org/scala-lang/modules/scala-xml_2.11/1.0.5/scala-xml_2.11-1.0.5.jarDownload https://repo.maven.apache.org/maven2/com/typesafe/zinc/zinc/0.3.15/zinc-0.3.15.pomDownload https://repo.maven.apache.org/maven2/org/scala-lang/scala-library/2.10.6/scala-library-2.10.6.pomDownload https://repo.maven.apache.org/maven2/com/typesafe/sbt/compiler-interface/0.13.15/compiler-interface-0.13.15.pomDownload https://repo.maven.apache.org/maven2/com/typesafe/sbt/incremental-compiler/0.13.15/incremental-compiler-0.13.15.pomDownload https://repo.maven.apache.org/maven2/com/typesafe/sbt/sbt-interface/0.13.15/sbt-interface-0.13.15.pomDownload https://repo.maven.apache.org/maven2/org/scala-lang/scala-compiler/2.10.6/scala-compiler-2.10.6.pomDownload https://repo.maven.apache.org/maven2/org/scala-lang/scala-reflect/2.10.6/scala-reflect-2.10.6.pomDownload https://repo.maven.apache.org/maven2/com/typesafe/sbt/incremental-compiler/0.13.15/incremental-compiler-0.13.15.jarDownload https://repo.maven.apache.org/maven2/com/typesafe/sbt/sbt-interface/0.13.15/sbt-interface-0.13.15.jarDownload https://repo.maven.apache.org/maven2/com/typesafe/zinc/zinc/0.3.15/zinc-0.3.15.jarDownload https://repo.maven.apache.org/maven2/org/scala-lang/scala-library/2.10.6/scala-library-2.10.6.jarDownload https://repo.maven.apache.org/maven2/org/scala-lang/scala-reflect/2.10.6/scala-reflect-2.10.6.jarDownload https://repo.maven.apache.org/maven2/org/scala-lang/scala-compiler/2.10.6/scala-compiler-2.10.6.jarDownload https://repo.maven.apache.org/maven2/com/typesafe/sbt/compiler-interface/0.13.15/compiler-interface-0.13.15-sources.jar> Task :cruise-control:compileScalaPruning sources from previous analysis, due to incompatible CompileSetup.………
[root@yinzhengjie cruise-control]# ./gradlew clean jar　　　　　　　　#进行构建

　　在构建的过程中，你可能会遇到如下的报错信息：

　　如果遇到上面的报错信息时，不要慌，根据报错信息提示缺少javac这个包，我们可以自行去oracle官网下载相应的jdk源码包，将对应的javac包拷贝到指定目录中即可。我服务器端已经安装好了对应的jdk源码包，于是我做了如下操作：

 

[root@yinzhengjie cruise-control]# find / -name javac/data/yinzhengjie/softwares/jdk/jdk1.8.0_131/bin/javac[root@yinzhengjie cruise-control]# cp /data/yinzhengjie/softwares/jdk/jdk1.8.0_131/bin/javac /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-0.el7_5.x86_64/jre/bin[root@yinzhengjie cruise-control]# [root@yinzhengjie cruise-control]# find / -name javac/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-0.el7_5.x86_64/jre/bin/javac/data/yinzhengjie/softwares/jdk/jdk1.8.0_131/bin/javac[root@yinzhengjie cruise-control]# [root@yinzhengjie cruise-control]# ./gradlew clean jarTo honour the JVM settings for this build a new JVM will be forked. Please consider using the daemon: https://docs.gradle.org/4.8/userguide/gradle_daemon.html.Daemon will be stopped at the end of the build stopping after processing> Task :cruise-control-core:compileJavaNote: /data/yinzhengjie/softwares/kafkasoft/kafka-cruiseControl/cruise-control/cruise-control-core/src/main/java/com/linkedin/cruisecontrol/common/config/ConfigDef.java uses unchecked or unsafe operations.Note: Recompile with -Xlint:unchecked for details.> Task :cruise-control:compileScalaPruning sources from previous analysis, due to incompatible CompileSetup.Note: /data/yinzhengjie/softwares/kafkasoft/kafka-cruiseControl/cruise-control/cruise-control/src/main/java/com/linkedin/kafka/cruisecontrol/analyzer/AnalyzerUtils.java uses or overrides a deprecated API.Note: Recompile with -Xlint:deprecation for details.Download https://repo.maven.apache.org/maven2/commons-io/commons-io/2.5/commons-io-2.5.pomDownload https://repo.maven.apache.org/maven2/org/easymock/easymock/3.4/easymock-3.4.pomDownload https://repo.maven.apache.org/maven2/org/easymock/easymock-parent/3.4/easymock-parent-3.4.pomDownload https://jcenter.bintray.com/com/linkedin/kafka/clients/li-apache-kafka-clients/0.0.12/li-apache-kafka-clients-0.0.12.pomDownload https://repo.maven.apache.org/maven2/org/objenesis/objenesis/2.2/objenesis-2.2.pomDownload https://repo.maven.apache.org/maven2/org/objenesis/objenesis-parent/2.2/objenesis-parent-2.2.pomDownload https://repo.maven.apache.org/maven2/commons-io/commons-io/2.5/commons-io-2.5.jarDownload https://repo.maven.apache.org/maven2/org/easymock/easymock/3.4/easymock-3.4.jarDownload https://repo.maven.apache.org/maven2/org/apache/kafka/kafka-clients/0.11.0.2/kafka-clients-0.11.0.2-test.jarDownload https://repo.maven.apache.org/maven2/org/apache/kafka/kafka_2.11/0.11.0.2/kafka_2.11-0.11.0.2-test.jarDownload https://jcenter.bintray.com/com/linkedin/kafka/clients/li-apache-kafka-clients/0.0.12/li-apache-kafka-clients-0.0.12.jarDownload https://repo.maven.apache.org/maven2/org/objenesis/objenesis/2.2/objenesis-2.2.jarDownload https://jcenter.bintray.com/com/linkedin/kafka/clients/li-apache-kafka-clients/0.0.12/li-apache-kafka-clients-0.0.12-tests.jarDeprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.See https://docs.gradle.org/4.8/userguide/command_line_interface.html#sec:command_line_warningsBUILD SUCCESSFUL in 38s10 actionable tasks: 10 executed[root@yinzhengjie cruise-control]#

3>.修改broker配置文件

　　构建成功后，我们可以在项目目录下的cruise-control-metrics-reporter/build/libs下会生成一个jar包，我们将这个jar包拷贝到broker安装目录的libs目录下，具体操作如下：

[root@yinzhengjie cruise-control]# lltotal 68-rw-r--r-- 1 root root  8257 Dec  4 10:45 build.gradledrwxr-xr-x 5 root root    61 Dec  4 10:47 buildSrcdrwxr-xr-x 2 root root    50 Dec  4 10:45 checkstyledrwxr-xr-x 2 root root    80 Dec  4 10:45 config-rw-r--r-- 1 root root  1079 Dec  4 10:45 CONTRIBUTING.mddrwxr-xr-x 4 root root    28 Dec  4 11:00 cruise-controldrwxr-xr-x 4 root root    28 Dec  4 11:00 cruise-control-coredrwxr-xr-x 4 root root    28 Dec  4 11:00 cruise-control-metrics-reporterdrwxr-xr-x 3 root root    19 Dec  4 10:45 docsdrwxr-xr-x 3 root root    47 Dec  4 10:45 gradle-rw-r--r-- 1 root root    86 Dec  4 10:45 gradle.properties-rwxr-xr-x 1 root root  5299 Dec  4 10:45 gradlew-rw-r--r-- 1 root root  2260 Dec  4 10:45 gradlew.bat-rwxr-xr-x 1 root root  5129 Dec  4 10:45 kafka-cruise-control-start.sh-rw-r--r-- 1 root root  1332 Dec  4 10:45 LICENSE-rw-r--r-- 1 root root   304 Dec  4 10:45 NOTICE-rw-r--r-- 1 root root 11476 Dec  4 10:45 README.md-rw-r--r-- 1 root root   153 Dec  4 10:45 semantic-build-versioning.gradle-rw-r--r-- 1 root root   911 Dec  4 10:45 settings.gradle[root@yinzhengjie cruise-control]# [root@yinzhengjie cruise-control]# ll cruise-control-metrics-reporter/build/libs/ total 36-rw-r--r-- 1 root root 36547 Dec  4 11:00 cruise-control-metrics-reporter-0.1.18-SNAPSHOT.jar[root@yinzhengjie cruise-control]# [root@yinzhengjie cruise-control]# scp cruise-control-metrics-reporter/build/libs/cruise-control-metrics-reporter-0.1.18-SNAPSHOT.jar 10.1.2.118:/soft/kafka/libs/root@10.1.2.118's password: cruise-control-metrics-reporter-0.1.18-SNAPSHOT.jar                                                                                              100%   36KB  35.7KB/s   00:00    [root@yinzhengjie cruise-control]#

　　注意，上面的操作需要在每台broker都需要做同样的操作，之后需要在每台broker端的配置文件“server.properties”中新增下列行，保存之后启动broker：

metric.reporters=com.linkedin.kafka.cruisecontrol.metricsreporter.CruiseControlMetricsReporter

4>.修改cruisecontrol项目目录下的conf/cruisecontrol.properties文件（根据你实际生产环境自定义相应的参数即可）

[root@yinzhengjie cruise-control]# pwd/yinzhengjie/softwares/kafkasoft/kafka-cruiseControl/cruise-control[root@yinzhengjie cruise-control]# [root@yinzhengjie cruise-control]# lltotal 68-rw-r--r-- 1 root root  8257 Dec  4 10:45 build.gradledrwxr-xr-x 5 root root    61 Dec  4 10:47 buildSrcdrwxr-xr-x 2 root root    50 Dec  4 10:45 checkstyledrwxr-xr-x 2 root root    80 Dec  4 10:45 config-rw-r--r-- 1 root root  1079 Dec  4 10:45 CONTRIBUTING.mddrwxr-xr-x 4 root root    28 Dec  4 11:00 cruise-controldrwxr-xr-x 4 root root    28 Dec  4 11:00 cruise-control-coredrwxr-xr-x 4 root root    28 Dec  4 11:00 cruise-control-metrics-reporterdrwxr-xr-x 3 root root    19 Dec  4 10:45 docsdrwxr-xr-x 3 root root    47 Dec  4 10:45 gradle-rw-r--r-- 1 root root    86 Dec  4 10:45 gradle.properties-rwxr-xr-x 1 root root  5299 Dec  4 10:45 gradlew-rw-r--r-- 1 root root  2260 Dec  4 10:45 gradlew.bat-rwxr-xr-x 1 root root  5129 Dec  4 10:45 kafka-cruise-control-start.sh-rw-r--r-- 1 root root  1332 Dec  4 10:45 LICENSE-rw-r--r-- 1 root root   304 Dec  4 10:45 NOTICE-rw-r--r-- 1 root root 11476 Dec  4 10:45 README.md-rw-r--r-- 1 root root   153 Dec  4 10:45 semantic-build-versioning.gradle-rw-r--r-- 1 root root   911 Dec  4 10:45 settings.gradle[root@yinzhengjie cruise-control]# [root@yinzhengjie cruise-control]# [root@yinzhengjie cruise-control]# [root@yinzhengjie cruise-control]# cat config/cruisecontrol.properties  | grep -v ^# | grep bootstrap.serversbootstrap.servers=10.1.2.118:9092,10.1.2.117:9092,10.1.2.116:9092,10.1.2.115:9092,10.1.2.114:9092[root@yinzhengjie cruise-control]# [root@yinzhengjie cruise-control]# cat config/cruisecontrol.properties  | grep -v ^# | grep zookeeper.connectzookeeper.connect=10.1.2.114:2181,10.1.2.115:2181,10.1.2.116:2181,10.1.2.117:2181,10.1.2.118:2181/kafka_yinzhengjie_cluster1[root@yinzhengjie cruise-control]# [root@yinzhengjie cruise-control]# [root@yinzhengjie cruise-control]# cat config/cruisecontrol.properties  | grep -v ^# | grep webserver.http.portwebserver.http.port=7070[root@yinzhengjie cruise-control]# [root@yinzhengjie cruise-control]# cat config/cruisecontrol.properties  | grep -v ^# | grep webserver.http.cors.originwebserver.http.cors.origin=http://localhost:7070/[root@yinzhengjie cruise-control]#
[root@node118 ~]# cat `which xcall.sh `    #!/bin/bash#@author :yinzhengjie#blog:http://www.cnblogs.com/yinzhengjie#EMAIL:y1053419035@qq.com#判断用户是否传参if [ $# -lt 1 ];then        echo "请输入参数"        exitfi#获取用户输入的命令cmd=$@for (( i=114;i<=118;i++ ))do        #使终端变绿色         tput setaf 2        echo ============= node${i}.aggrx : $cmd ============        #使终端变回原来的颜色，即白灰色        tput setaf 7        #远程执行命令        ssh node${i}.aggrx  $cmd        #判断命令是否执行成功        if [ $? == 0 ];then                echo "命令执行成功"        fidone[root@node118 ~]#
[root@node118 ~]# cat `which xcall.sh ` 
[root@node118 ~]# cat `which xkafka.sh `    #!/bin/bash#@author :yinzhengjie#blog:http://www.cnblogs.com/yinzhengjie#EMAIL:y1053419035@qq.com#判断用户是否传参if [ $# -ne 1 ];then    echo "无效参数，用法为: $0  {start|stop}"    exitfi#获取用户输入的命令cmd=$1for (( i=114 ; i<=118 ; i++ )) ; do    tput setaf 2    echo ========== node${i}.aggrx  $cmd ================    tput setaf 9    case $cmd in        start)            ssh node${i}.aggrx  "source /etc/profile ; kafka-server-start.sh -daemon /soft/kafka/config/server.properties"            echo node${i}.aggrx  "服务已启动"            ;;        stop)             ssh node${i}.aggrx  "source /etc/profile ; kafka-server-stop.sh"             echo node${i}.aggrx  "服务已停止"            ;;            *)             echo "无效参数，用法为: $0  {start|stop}"            exit             ;;     esacdone[root@node118 ~]#
[root@node118 ~]# cat `which xkafka.sh ` 
[root@node118 ~]# xcall.sh jps============= node114.aggrx : jps ============29930 QuorumPeerMain1900 Jps命令执行成功============= node115.aggrx : jps ============788 QuorumPeerMain5023 Jps命令执行成功============= node116.aggrx : jps ============19335 QuorumPeerMain23516 Jps命令执行成功============= node117.aggrx : jps ============26770 Jps22639 QuorumPeerMain命令执行成功============= node118.aggrx : jps ============20017 QuorumPeerMain24949 Jps30056 Bootstrap命令执行成功[root@node118 ~]# [root@node118 ~]# [root@node118 ~]# xkafka.sh start========== node114.aggrx start ================node114.aggrx 服务已启动========== node115.aggrx start ================node115.aggrx 服务已启动========== node116.aggrx start ================node116.aggrx 服务已启动========== node117.aggrx start ================node117.aggrx 服务已启动========== node118.aggrx start ================node118.aggrx 服务已启动[root@node118 ~]# [root@node118 ~]# [root@node118 ~]# xcall.sh jps   ============= node114.aggrx : jps ============2256 Kafka2480 Jps29930 QuorumPeerMain命令执行成功============= node115.aggrx : jps ============788 QuorumPeerMain5588 Jps5372 Kafka命令执行成功============= node116.aggrx : jps ============19335 QuorumPeerMain23865 Kafka24079 Jps命令执行成功============= node117.aggrx : jps ============27173 Kafka27387 Jps22639 QuorumPeerMain命令执行成功============= node118.aggrx : jps ============20017 QuorumPeerMain25556 Jps25317 Kafka30056 Bootstrap命令执行成功[root@node118 ~]#
[root@node118 ~]# xkafka.sh start 
[root@node118 ~]# xcall.sh lsof /soft/kafka/libs/cruise-control-metrics-reporter-0.1.18-SNAPSHOT.jar  ============= node114.aggrx : lsof /soft/kafka/libs/cruise-control-metrics-reporter-0.1.18-SNAPSHOT.jar ============COMMAND  PID USER   FD   TYPE DEVICE SIZE/OFF      NODE NAMEjava    2256 root  mem    REG  253,0    36547 100826940 /soft/kafka_2.11-1.1.0//libs/cruise-control-metrics-reporter-0.1.18-SNAPSHOT.jarjava    2256 root   13r   REG  253,0    36547 100826940 /soft/kafka_2.11-1.1.0//libs/cruise-control-metrics-reporter-0.1.18-SNAPSHOT.jar命令执行成功============= node115.aggrx : lsof /soft/kafka/libs/cruise-control-metrics-reporter-0.1.18-SNAPSHOT.jar ============COMMAND  PID USER   FD   TYPE DEVICE SIZE/OFF     NODE NAMEjava    5372 root  mem    REG  253,0    36547 67503111 /soft/kafka_2.11-1.1.0//libs/cruise-control-metrics-reporter-0.1.18-SNAPSHOT.jarjava    5372 root   13r   REG  253,0    36547 67503111 /soft/kafka_2.11-1.1.0//libs/cruise-control-metrics-reporter-0.1.18-SNAPSHOT.jar命令执行成功============= node116.aggrx : lsof /soft/kafka/libs/cruise-control-metrics-reporter-0.1.18-SNAPSHOT.jar ============COMMAND   PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAMEjava    23865 root  mem    REG  253,0    36547 1046 /soft/kafka_2.11-1.1.0//libs/cruise-control-metrics-reporter-0.1.18-SNAPSHOT.jarjava    23865 root   13r   REG  253,0    36547 1046 /soft/kafka_2.11-1.1.0//libs/cruise-control-metrics-reporter-0.1.18-SNAPSHOT.jar命令执行成功============= node117.aggrx : lsof /soft/kafka/libs/cruise-control-metrics-reporter-0.1.18-SNAPSHOT.jar ============COMMAND   PID USER   FD   TYPE DEVICE SIZE/OFF     NODE NAMEjava    27173 root  mem    REG  253,0    36547 83946551 /soft/kafka_2.11-1.1.0//libs/cruise-control-metrics-reporter-0.1.18-SNAPSHOT.jarjava    27173 root   13r   REG  253,0    36547 83946551 /soft/kafka_2.11-1.1.0//libs/cruise-control-metrics-reporter-0.1.18-SNAPSHOT.jar命令执行成功============= node118.aggrx : lsof /soft/kafka/libs/cruise-control-metrics-reporter-0.1.18-SNAPSHOT.jar ============COMMAND   PID USER   FD   TYPE DEVICE SIZE/OFF     NODE NAMEjava    25317 root  mem    REG  253,0    36547 59044906 /soft/kafka_2.11-1.1.0//libs/cruise-control-metrics-reporter-0.1.18-SNAPSHOT.jarjava    25317 root   13r   REG  253,0    36547 59044906 /soft/kafka_2.11-1.1.0//libs/cruise-control-metrics-reporter-0.1.18-SNAPSHOT.jar命令执行成功[root@node118 ~]#

 

5>.启动cruisecontrol服务

[root@yinzhengjie cruise-control]# lltotal 68-rw-r--r-- 1 root root  8257 Dec  4 10:45 build.gradledrwxr-xr-x 5 root root    61 Dec  4 10:47 buildSrcdrwxr-xr-x 2 root root    50 Dec  4 10:45 checkstyledrwxr-xr-x 2 root root    80 Dec  4 11:44 config-rw-r--r-- 1 root root  1079 Dec  4 10:45 CONTRIBUTING.mddrwxr-xr-x 4 root root    28 Dec  4 11:00 cruise-controldrwxr-xr-x 4 root root    28 Dec  4 11:00 cruise-control-coredrwxr-xr-x 4 root root    28 Dec  4 11:00 cruise-control-metrics-reporterdrwxr-xr-x 3 root root    19 Dec  4 10:45 docsdrwxr-xr-x 3 root root    47 Dec  4 10:45 gradle-rw-r--r-- 1 root root    86 Dec  4 10:45 gradle.properties-rwxr-xr-x 1 root root  5299 Dec  4 10:45 gradlew-rw-r--r-- 1 root root  2260 Dec  4 10:45 gradlew.bat-rwxr-xr-x 1 root root  5129 Dec  4 10:45 kafka-cruise-control-start.sh-rw-r--r-- 1 root root  1332 Dec  4 10:45 LICENSE-rw-r--r-- 1 root root   304 Dec  4 10:45 NOTICE-rw-r--r-- 1 root root 11476 Dec  4 10:45 README.md-rw-r--r-- 1 root root   153 Dec  4 10:45 semantic-build-versioning.gradle-rw-r--r-- 1 root root   911 Dec  4 10:45 settings.gradle[root@yinzhengjie cruise-control]# [root@yinzhengjie cruise-control]# [root@yinzhengjie cruise-control]# ./gradlew jar copyDependantLibsTo honour the JVM settings for this build a new JVM will be forked. Please consider using the daemon: https://docs.gradle.org/4.8/userguide/gradle_daemon.html.Daemon will be stopped at the end of the build stopping after processingDownload https://repo.maven.apache.org/maven2/org/bouncycastle/bcpkix-jdk15on/1.54/bcpkix-jdk15on-1.54.pomDownload https://repo.maven.apache.org/maven2/org/bouncycastle/bcprov-jdk15on/1.54/bcprov-jdk15on-1.54.pomDownload https://repo.maven.apache.org/maven2/org/bouncycastle/bcpkix-jdk15on/1.54/bcpkix-jdk15on-1.54.jarDownload https://repo.maven.apache.org/maven2/org/bouncycastle/bcprov-jdk15on/1.54/bcprov-jdk15on-1.54.jarDeprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.See https://docs.gradle.org/4.8/userguide/command_line_interface.html#sec:command_line_warningsBUILD SUCCESSFUL in 1m 59s11 actionable tasks: 4 executed, 7 up-to-date[root@yinzhengjie cruise-control]#
[root@yinzhengjie cruise-control]# ./gradlew jar copyDependantLibs  
[root@yinzhengjie cruise-control]# ./kafka-cruise-control-start.sh config/cruisecontrol.properties [2018-12-04 12:27:10,878] INFO KafkaCruiseControlConfig values:         anomaly.detection.allow.capacity.estimation = true        anomaly.detection.goals = [com.linkedin.kafka.cruisecontrol.analyzer.goals.RackAwareGoal, com.linkedin.kafka.cruisecontrol.analyzer.goals.ReplicaCapacityGoal, com.linkedin.kafka.cruisecontrol.analyzer.goals.DiskCapacityGoal, com.linkedin.kafka.cruisecontrol.analyzer.goals.NetworkInboundCapacityGoal, com.linkedin.kafka.cruisecontrol.analyzer.goals.NetworkOutboundCapacityGoal, com.linkedin.kafka.cruisecontrol.analyzer.goals.CpuCapacityGoal]        anomaly.detection.interval.ms = 10000        anomaly.notifier.class = class com.linkedin.kafka.cruisecontrol.detector.notifier.SelfHealingNotifier        bootstrap.servers = [10.1.2.118:9092, 10.1.2.117:9092, 10.1.2.116:9092, 10.1.2.115:9092, 10.1.2.114:9092]        broker.capacity.config.resolver.class = class com.linkedin.kafka.cruisecontrol.config.BrokerCapacityConfigFileResolver        broker.metric.sample.aggregator.completeness.cache.size = 5        broker.metrics.window.ms = 300000        client.id = kafka-cruise-control        completed.user.task.retention.time.ms = 21600000        connections.max.idle.ms = 540000        cpu.balance.threshold = 1.1        cpu.capacity.threshold = 0.8        cpu.low.utilization.threshold = 0.0        default.goals = [com.linkedin.kafka.cruisecontrol.analyzer.goals.RackAwareGoal, com.linkedin.kafka.cruisecontrol.analyzer.goals.ReplicaCapacityGoal, com.linkedin.kafka.cruisecontrol.analyzer.goals.DiskCapacityGoal, com.linkedin.kafka.cruisecontrol.analyzer.goals.NetworkInboundCapacityGoal, com.linkedin.kafka.cruisecontrol.analyzer.goals.NetworkOutboundCapacityGoal, com.linkedin.kafka.cruisecontrol.analyzer.goals.CpuCapacityGoal, com.linkedin.kafka.cruisecontrol.analyzer.goals.ReplicaDistributionGoal, com.linkedin.kafka.cruisecontrol.analyzer.goals.PotentialNwOutGoal, com.linkedin.kafka.cruisecontrol.analyzer.goals.DiskUsageDistributionGoal, com.linkedin.kafka.cruisecontrol.analyzer.goals.NetworkInboundUsageDistributionGoal, com.linkedin.kafka.cruisecontrol.analyzer.goals.NetworkOutboundUsageDistributionGoal, com.linkedin.kafka.cruisecontrol.analyzer.goals.CpuUsageDistributionGoal, com.linkedin.kafka.cruisecontrol.analyzer.goals.TopicReplicaDistributionGoal, com.linkedin.kafka.cruisecontrol.analyzer.goals.LeaderBytesInDistributionGoal]        disk.balance.threshold = 1.1        disk.capacity.threshold = 0.8        disk.low.utilization.threshold = 0.0        execution.progress.check.interval.ms = 10000        failed.brokers.zk.path = /CruiseControlBrokerList        follower.network.inbound.weight.for.cpu.util = 0.3        goals = [com.linkedin.kafka.cruisecontrol.analyzer.goals.RackAwareGoal, com.linkedin.kafka.cruisecontrol.analyzer.goals.ReplicaCapacityGoal, com.linkedin.kafka.cruisecontrol.analyzer.goals.DiskCapacityGoal, com.linkedin.kafka.cruisecontrol.analyzer.goals.NetworkInboundCapacityGoal, com.linkedin.kafka.cruisecontrol.analyzer.goals.NetworkOutboundCapacityGoal, com.linkedin.kafka.cruisecontrol.analyzer.goals.CpuCapacityGoal, com.linkedin.kafka.cruisecontrol.analyzer.goals.ReplicaDistributionGoal, com.linkedin.kafka.cruisecontrol.analyzer.goals.PotentialNwOutGoal, com.linkedin.kafka.cruisecontrol.analyzer.goals.DiskUsageDistributionGoal, com.linkedin.kafka.cruisecontrol.analyzer.goals.NetworkInboundUsageDistributionGoal, com.linkedin.kafka.cruisecontrol.analyzer.goals.NetworkOutboundUsageDistributionGoal, com.linkedin.kafka.cruisecontrol.analyzer.goals.CpuUsageDistributionGoal, com.linkedin.kafka.cruisecontrol.analyzer.goals.TopicReplicaDistributionGoal, com.linkedin.kafka.cruisecontrol.analyzer.goals.LeaderBytesInDistributionGoal, com.linkedin.kafka.cruisecontrol.analyzer.kafkaassigner.KafkaAssignerDiskUsageDistributionGoal, com.linkedin.kafka.cruisecontrol.analyzer.kafkaassigner.KafkaAssignerEvenRackAwareGoal, com.linkedin.kafka.cruisecontrol.analyzer.goals.PreferredLeaderElectionGoal]        hard.goals = [com.linkedin.kafka.cruisecontrol.analyzer.goals.RackAwareGoal, com.linkedin.kafka.cruisecontrol.analyzer.goals.ReplicaCapacityGoal, com.linkedin.kafka.cruisecontrol.analyzer.goals.DiskCapacityGoal, com.linkedin.kafka.cruisecontrol.analyzer.goals.NetworkInboundCapacityGoal, com.linkedin.kafka.cruisecontrol.analyzer.goals.NetworkOutboundCapacityGoal, com.linkedin.kafka.cruisecontrol.analyzer.goals.CpuCapacityGoal]        leader.network.inbound.weight.for.cpu.util = 0.6        leader.network.outbound.weight.for.cpu.util = 0.1        linear.regression.model.cpu.util.bucket.size = 5        linear.regression.model.min.num.cpu.util.buckets = 5        linear.regression.model.required.samples.per.bucket = 100        max.active.user.tasks = 5        max.allowed.extrapolations.per.broker = 5        max.allowed.extrapolations.per.partition = 5        max.cached.completed.user.tasks = 100        max.replicas.per.broker = 10000        metadata.max.age.ms = 55000        metric.anomaly.finder.class = [com.linkedin.kafka.cruisecontrol.detector.KafkaMetricAnomalyFinder]        metric.sampler.class = class com.linkedin.kafka.cruisecontrol.monitor.sampling.CruiseControlMetricsReporterSampler        metric.sampler.partition.assignor.class = class com.linkedin.kafka.cruisecontrol.monitor.sampling.DefaultMetricSamplerPartitionAssignor        metric.sampling.interval.ms = 120000        min.samples.per.broker.metrics.window = 1        min.samples.per.partition.metrics.window = 1        min.valid.partition.ratio = 0.995        network.inbound.balance.threshold = 1.1        network.inbound.capacity.threshold = 0.8        network.inbound.low.utilization.threshold = 0.0        network.outbound.balance.threshold = 1.1        network.outbound.capacity.threshold = 0.8        network.outbound.low.utilization.threshold = 0.0        num.broker.metrics.windows = 20        num.cached.recent.anomaly.states = 10        num.concurrent.leader.movements = 1000        num.concurrent.partition.movements.per.broker = 10        num.metric.fetchers = 1        num.partition.metrics.windows = 1        num.proposal.precompute.threads = 1        partition.metric.sample.aggregator.completeness.cache.size = 5        partition.metrics.window.ms = 300000        proposal.expiration.ms = 60000        receive.buffer.bytes = 32768        reconnect.backoff.ms = 50        replica.count.balance.threshold = 1.1        request.timeout.ms = 30000        sample.store.class = class com.linkedin.kafka.cruisecontrol.monitor.sampling.KafkaSampleStore        sasl.jaas.config = null        sasl.kerberos.kinit.cmd = /usr/bin/kinit        sasl.kerberos.min.time.before.relogin = 60000        sasl.kerberos.service.name = null        sasl.kerberos.ticket.renew.jitter = 0.05        sasl.kerberos.ticket.renew.window.factor = 0.8        sasl.mechanism = GSSAPI        security.protocol = PLAINTEXT        send.buffer.bytes = 131072        ssl.cipher.suites = null        ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]        ssl.endpoint.identification.algorithm = null        ssl.key.password = null        ssl.keymanager.algorithm = SunX509        ssl.keystore.location = null        ssl.keystore.password = null        ssl.keystore.type = JKS        ssl.protocol = TLS        ssl.provider = null        ssl.secure.random.implementation = null        ssl.trustmanager.algorithm = PKIX        ssl.truststore.location = null        ssl.truststore.password = null        ssl.truststore.type = JKS        topics.excluded.from.partition.movement =         use.linear.regression.model = false        webserver.accesslog.enabled = true        webserver.accesslog.path = access.log        webserver.accesslog.retention.days = 14        webserver.api.urlprefix = /kafkacruisecontrol/*        webserver.http.address = 0.0.0.0        webserver.http.cors.allowmethods = OPTIONS,GET,POST        webserver.http.cors.enabled = false        webserver.http.cors.exposeheaders = User-Task-ID        webserver.http.cors.origin = http://localhost:7070/        webserver.http.port = 7070        webserver.request.maxBlockTimeMs = 10000        webserver.session.maxExpiryTimeMs = 60000        webserver.session.path = /        webserver.ui.diskpath = ./cruise-control-ui/dist/        webserver.ui.urlprefix = /*        zookeeper.connect = 10.1.2.114:2181,10.1.2.115:2181,10.1.2.116:2181,10.1.2.117:2181,10.1.2.118:2181/kafka_yinzhengjie_cluster1/ (com.linkedin.kafka.cruisecontrol.config.KafkaCruiseControlConfig)[2018-12-04 12:27:11,102] INFO ConsumerConfig values:         auto.commit.interval.ms = 5000        auto.offset.reset = latest        bootstrap.servers = [10.1.2.118:9092, 10.1.2.117:9092, 10.1.2.116:9092, 10.1.2.115:9092, 10.1.2.114:9092]        check.crcs = true        client.id = CruiseControlMetricsReporterSampler-9001457965431329813-consumer--1607149276        connections.max.idle.ms = 540000        enable.auto.commit = false        exclude.internal.topics = true        fetch.max.bytes = 52428800        fetch.max.wait.ms = 500        fetch.min.bytes = 1        group.id = CruiseControlMetricsReporterSampler-9001457965431329813        heartbeat.interval.ms = 3000        interceptor.classes = null        internal.leave.group.on.close = true        isolation.level = read_uncommitted        key.deserializer = class org.apache.kafka.common.serialization.StringDeserializer        max.partition.fetch.bytes = 1048576        max.poll.interval.ms = 2147483647        max.poll.records = 2147483647        metadata.max.age.ms = 300000        metric.reporters = []        metrics.num.samples = 2        metrics.recording.level = INFO        metrics.sample.window.ms = 30000        partition.assignment.strategy = [class org.apache.kafka.clients.consumer.RangeAssignor]        receive.buffer.bytes = 65536        reconnect.backoff.max.ms = 1000        reconnect.backoff.ms = 50        request.timeout.ms = 305000        retry.backoff.ms = 100        sasl.jaas.config = null        sasl.kerberos.kinit.cmd = /usr/bin/kinit        sasl.kerberos.min.time.before.relogin = 60000        sasl.kerberos.service.name = null        sasl.kerberos.ticket.renew.jitter = 0.05        sasl.kerberos.ticket.renew.window.factor = 0.8        sasl.mechanism = GSSAPI        security.protocol = PLAINTEXT        send.buffer.bytes = 131072        session.timeout.ms = 10000        ssl.cipher.suites = null        ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]        ssl.endpoint.identification.algorithm = null        ssl.key.password = null        ssl.keymanager.algorithm = SunX509        ssl.keystore.location = null        ssl.keystore.password = null        ssl.keystore.type = JKS        ssl.protocol = TLS        ssl.provider = null        ssl.secure.random.implementation = null        ssl.trustmanager.algorithm = PKIX        ssl.truststore.location = null        ssl.truststore.password = null        ssl.truststore.type = JKS        value.deserializer = class com.linkedin.kafka.cruisecontrol.metricsreporter.metric.MetricSerde (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,152] WARN The configuration 'anomaly.detection.interval.ms' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,152] WARN The configuration 'network.outbound.balance.threshold' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,152] WARN The configuration 'disk.balance.threshold' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,152] WARN The configuration 'metric.anomaly.percentile.lower.threshold' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,152] WARN The configuration 'replica.count.balance.threshold' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,152] WARN The configuration 'min.monitored.partition.percentage' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,152] WARN The configuration 'cpu.low.utilization.threshold' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,152] WARN The configuration 'num.partition.metrics.windows' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,152] WARN The configuration 'execution.progress.check.interval.ms' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,152] WARN The configuration 'capacity.config.file' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,152] WARN The configuration 'webserver.http.cors.enabled' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,153] WARN The configuration 'partition.metric.sample.store.topic' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,153] WARN The configuration 'network.outbound.capacity.threshold' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,153] WARN The configuration 'cpu.capacity.threshold' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,153] WARN The configuration 'webserver.ui.urlprefix' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,153] WARN The configuration 'proposal.expiration.ms' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,153] WARN The configuration 'goals' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,153] WARN The configuration 'webserver.api.urlprefix' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,153] WARN The configuration 'metric.sampler.partition.assignor.class' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,153] WARN The configuration 'webserver.http.cors.origin' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,153] WARN The configuration 'metric.sampler.class' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,153] WARN The configuration 'hard.goals' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,153] WARN The configuration 'self.healing.enabled' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,153] WARN The configuration 'broker.metrics.window.ms' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,153] WARN The configuration 'cpu.balance.threshold' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,153] WARN The configuration 'metric.anomaly.finder.class' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,154] WARN The configuration 'max.cached.completed.user.tasks' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,154] WARN The configuration 'num.sample.loading.threads' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,154] WARN The configuration 'zookeeper.connect' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,154] WARN The configuration 'disk.capacity.threshold' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,154] WARN The configuration 'webserver.http.port' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,154] WARN The configuration 'partition.metrics.window.ms' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,154] WARN The configuration 'disk.low.utilization.threshold' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,154] WARN The configuration 'network.inbound.low.utilization.threshold' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,154] WARN The configuration 'network.inbound.capacity.threshold' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,154] WARN The configuration 'webserver.session.path' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,154] WARN The configuration 'failed.brokers.zk.path' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,154] WARN The configuration 'webserver.accesslog.enabled' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,154] WARN The configuration 'num.concurrent.partition.movements.per.broker' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,154] WARN The configuration 'webserver.ui.diskpath' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,155] WARN The configuration 'completed.user.task.retention.time.ms' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,155] WARN The configuration 'metric.reporter.topic.pattern' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,155] WARN The configuration 'min.samples.per.broker.metrics.window' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,155] WARN The configuration 'num.broker.metrics.windows' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,155] WARN The configuration 'webserver.accesslog.retention.days' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,155] WARN The configuration 'metric.sampling.interval.ms' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,155] WARN The configuration 'min.samples.per.partition.metrics.window' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,155] WARN The configuration 'anomaly.notifier.class' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,155] WARN The configuration 'webserver.http.cors.exposeheaders' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,155] WARN The configuration 'anomaly.detection.goals' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,155] WARN The configuration 'network.outbound.low.utilization.threshold' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,155] WARN The configuration 'num.proposal.precompute.threads' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,155] WARN The configuration 'webserver.http.address' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,155] WARN The configuration 'metric.anomaly.analyzer.metrics' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,155] WARN The configuration 'network.inbound.balance.threshold' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,155] WARN The configuration 'webserver.session.maxExpiryTimeMs' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,156] WARN The configuration 'webserver.accesslog.path' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,156] WARN The configuration 'webserver.request.maxBlockTimeMs' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,156] WARN The configuration 'metric.anomaly.percentile.upper.threshold' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,156] WARN The configuration 'sample.store.topic.replication.factor' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,156] WARN The configuration 'max.replicas.per.broker' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,156] WARN The configuration 'broker.metric.sample.store.topic' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,156] WARN The configuration 'webserver.http.cors.allowmethods' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,156] WARN The configuration 'num.metric.fetchers' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,156] WARN The configuration 'max.active.user.tasks' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,156] WARN The configuration 'sample.store.class' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,156] WARN The configuration 'default.goals' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)[2018-12-04 12:27:11,159] INFO Kafka version : 0.11.0.2 (org.apache.kafka.common.utils.AppInfoParser)[2018-12-04 12:27:11,159] INFO Kafka commitId : 73be1e1168f91ee2 (org.apache.kafka.common.utils.AppInfoParser)Exception in thread "main" java.lang.IllegalStateException: Cruise Control cannot find sampling topic matches __CruiseControlMetrics in the target cluster.        at com.linkedin.kafka.cruisecontrol.monitor.sampling.CruiseControlMetricsReporterSampler.configure(CruiseControlMetricsReporterSampler.java:203)        at com.linkedin.kafka.cruisecontrol.config.KafkaCruiseControlConfig.getConfiguredInstance(KafkaCruiseControlConfig.java:1065)        at com.linkedin.kafka.cruisecontrol.monitor.sampling.MetricFetcherManager.
     
      (MetricFetcherManager.java:129)        at com.linkedin.kafka.cruisecontrol.monitor.sampling.MetricFetcherManager.
      
       (MetricFetcherManager.java:69)        at com.linkedin.kafka.cruisecontrol.monitor.task.LoadMonitorTaskRunner.
       
        (LoadMonitorTaskRunner.java:72)        at com.linkedin.kafka.cruisecontrol.monitor.LoadMonitor.
        
         (LoadMonitor.java:152)        at com.linkedin.kafka.cruisecontrol.monitor.LoadMonitor.
         
          (LoadMonitor.java:110)        at com.linkedin.kafka.cruisecontrol.KafkaCruiseControl.
          
           (KafkaCruiseControl.java:89) at com.linkedin.kafka.cruisecontrol.async.AsyncKafkaCruiseControl.
           
            (AsyncKafkaCruiseControl.java:64) at com.linkedin.kafka.cruisecontrol.KafkaCruiseControlMain.main(KafkaCruiseControlMain.java:71)[root@yinzhengjie cruise-control]# [root@yinzhengjie cruise-control]#

　　启动的时候报错了，这就有点小尴尬了，问题还在排除中......

 

四.Kafka  Manager开源监控

　　我接触的第一款kafka监控工具，配置简单，大师不支持告警，详情请参考：。

 

五.Kafka Eagle

 　　我接触的第二款Kafka监控工具，配置也简单，想必Kafka Manager更优秀一点，支持告警功能，详情请参考：。

 

六.JmxTool

　　 严格的来说，JmxTools不是一个框架，而是kafka社区默认提供的工具，用于实时查看JMX监控指标。如果用户一时找不到合适的工具来监测JMX指标，可以考虑使用“临时救急”。打开一个终端并进入Kafka安装目录bin下，输入命令“kafka-run-class.sh kafka.tools.JmxTool”便可以得到JmxTool工具的帮助信息：

[root@yinzhengjie cruise-control]# kafka-run-class.sh kafka.tools.JmxToolDump JMX values to standard output.Option                              Description                                ------                              -----------                                --attributes 
     
               #指定 CSV格式的属性，如果不指定该参数则默认读取所有属性。查询属性的白名单。这个是逗号分隔的列表。如果没有属性指定所有对象将被询问。--date-format 
      
             # 设置时间格式，比如设置为 YYYY-MM-ss,则输出时间戳即为2018-12-01这样的格式        --help                                         #查询帮助信息             --jmx-url 
       
                 #指定要链接的JMX接口，默认格式是service.jmx.rmi:///jndi/rmi://:
        
         /jmxrmi--object-name 
         
               　　　　#指定JMX的Mbean名称--reporting-interval 
          
            　　#设置实时监测时间间隔，默认每2秒打印一次JMX指标值。[root@yinzhengjie cruise-control]#

　　假设我们要实时监控JMX MBean的消息入站速率。我们想要每个3秒输出一次该MBean过去15分钟的平均值，则可以执行下面的命令：

[root@yinzhengjie cruise-control]# kafka-run-class.sh kafka.tools.JmxTool --object-name kafka.server:type=BrokerTopicMetrics,name=BytesInPerSec --jmx-url service:jmx:rmi:///jndi/rmi://10.1.3.116:12345/jmxrmi --date-format "YYYY-MM-dd HH:mm:ss" --attributes FifteenMinuteRate --reporting-interval 3000Trying to connect to JMX url: service:jmx:rmi:///jndi/rmi://10.1.3.116:12345/jmxrmi."time","kafka.server:type=BrokerTopicMetrics,name=BytesInPerSec:FifteenMinuteRate"2018-12-04 13:21:25,1.0616180583446175E72018-12-04 13:21:28,1.0610205773377854E72018-12-04 13:21:31,1.0610205773377854E72018-12-04 13:21:34,1.0605810471779421E72018-12-04 13:21:37,1.0599882204038633E72018-12-04 13:21:40,1.0599882204038633E72018-12-04 13:21:43,1.059681434484897E72018-12-04 13:21:46,1.059681434484897E72018-12-04 13:21:49,1.0591635691401184E72018-12-04 13:21:52,1.058678141082473E72018-12-04 13:21:55,1.058678141082473E72018-12-04 13:21:58,1.0581291456978852E72018-12-04 13:22:01,1.0581291456978852E7

 

 

 

转载于:https://www.cnblogs.com/yinzhengjie/p/10051976.html

你可能感兴趣的文章
百万PV网站架构
查看>>
N26-第四周作业
查看>>
在vmware安装Ubuntu桌面软件
查看>>
MySQL之用户和权限管理
查看>>
常用的命令的使用方法
查看>>
使用HeartBeat实现高可用HA的配置过程详解
查看>>
最常用的四种大数据分析方法
查看>>
ajax https请求返回json数据
查看>>
convenience - 便利构造函数
查看>>
golang 碎片整理之 结构体
查看>>
查看oracle查看当前连接以及修改最大连接数
查看>>
docker安装mysql镜像
查看>>
java中的IO整理
查看>>
我的linux学习决心书
查看>>
python 之多线程加锁
查看>>
我的友情链接
查看>>
exchange快速将断开的邮箱显示出来
查看>>
linux 下查找文件或者内容常用命令
查看>>
Linux常用系统调用表
查看>>
linux x86_64要注意的问题
查看>>
喝酒易醉，品茶养心，人生如梦，品茶悟道，何以解忧？唯有杜康！-- 愿君每日到此一游！
当前时间: 2025-01-31 12:42:07   当前IP: 3.145.58.157   联系邮箱:javaeecc@qq.com     Copyright © 2020 - 2022 baihongyu.com 京ICP备2021015314号-2
强烈建议你试试无所不能的CHAT-GPT，快点击我
强烈建议你试试无所不能的CHAT-GPT，快点击我