16. Yarn資源調(diào)度器
16.5 Yarn常用命令
Yarn狀態(tài)的查詢(xún)，除了可以在hadoop103:8088頁(yè)面查看外，還可以通過(guò)命令操作。常見(jiàn)的命令操作如下所示：
需求：執(zhí)行WordCount案例，并用Yarn命令查看任務(wù)運(yùn)行情況。
-
[summer@hadoop102 ~]$ myhadoop.sh start
這個(gè)是之前寫(xiě)的腳本，想了解的可用看我之前寫(xiě)的文章
https://blog.csdn.net/Redamancy06/article/details/126234179 -
[summer@hadoop102 hadoop-3.1.3]$ hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-3.1.3.jar wordcount /testinput /testoutput/output1
16.5.1 yarn application查看任務(wù)
16.5.1.1 列出所有Application：
[summer@hadoop102 hadoop-3.1.3]$ yarn application -list
2022-10-10 16:57:24,620 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.159.103:8032
Total number of applications (application-types: [], states: [SUBMITTED, ACCEPTED, RUNNING] and tags: []):0
Application-Id Application-Name Application-Type User Queue State Final-State Progress Tracking-URL
這里因?yàn)槿蝿?wù)已經(jīng)執(zhí)行結(jié)束了，所以沒(méi)有顯示出來(lái)。
16.5.1.2 根據(jù)Application狀態(tài)過(guò)濾：yarn application -list -appStates （所有狀態(tài)：ALL、NEW、NEW_SAVING、SUBMITTED、ACCEPTED、RUNNING、FINISHED、FAILED、KILLED）
[summer@hadoop102 hadoop-3.1.3]$ yarn application -list -appStates FINISHED
2022-10-10 17:03:48,178 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.159.103:8032
Total number of applications (application-types: [], states: [FINISHED] and tags: []):1
Application-Id Application-Name Application-Type User Queue State Final-State Progress Tracking-URL
application_1665325770064_0003 word count MAPREDUCE summer default FINISHED SUCCEEDED 100% http://hadoop102:19888/jobhistory/job/job_1665325770064_0003
這里的任務(wù)application_1665325770064_0003就是剛剛我們執(zhí)行的任務(wù)
16.5.1.3 Kill掉Application：
[summer@hadoop102 hadoop-3.1.3]$ yarn application -kill application_1665325770064_0003
2022-10-10 17:05:56,667 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.159.103:8032
Application application_1665325770064_0003 has already finished
因?yàn)閍pplication_1665325770064_0003這個(gè)任務(wù)已經(jīng)完成了，所以不能再次被kill了
16.5.2 yarn logs查看日志
16.5.2.1 查詢(xún)Application日志：
yarn logs -applicationId <ApplicationId>
[summer@hadoop102 hadoop-3.1.3]$ yarn logs -applicationId application_1665325770064_0003
16.5.2.2 查詢(xún)Container日志：
yarn logs -applicationId <ApplicationId> -containerId <ContainerId>
[summer@hadoop102 hadoop-3.1.3]$ yarn logs -applicationId application_1665325770064_0003 -containerId container_1665325770064_0003_01_000001
containerId從16.5.3.1里獲取
16.5.3 yarn applicationattempt查看嘗試運(yùn)行的任務(wù)
16.5.3.1 列出所有Application嘗試的列表：
yarn applicationattempt -list <ApplicationId>
[summer@hadoop102 hadoop-3.1.3]$ yarn applicationattempt -list application_1665325770064_0003
2022-10-10 20:01:36,619 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.159.103:8032
Total number of application attempts :1
ApplicationAttempt-Id State AM-Container-Id Tracking-URL
appattempt_1665325770064_0003_000001 FINISHED container_1665325770064_0003_01_000001 http://hadoop103:8088/proxy/application_1665325770064_0003/
16.5.3.2 打印ApplicationAttemp狀態(tài)：
yarn applicationattempt -status <ApplicationAttemptId>
[summer@hadoop102 hadoop-3.1.3]$ yarn applicationattempt -status appattempt_1665325770064_0003_000001
2022-10-10 20:11:35,446 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.159.103:8032
Application Attempt Report :
ApplicationAttempt-Id : appattempt_1665325770064_0003_000001
State : FINISHED
AMContainer : container_1665325770064_0003_01_000001
Tracking-URL : http://hadoop103:8088/proxy/application_1665325770064_0003/
RPC Port : 42572
AM Host : hadoop103
Diagnostics :
ApplicationAttemptId從16.5.3.1里獲取
16.5.4 yarn container查看容器
16.5.4.1 列出所有Application 嘗試的列表
yarn container -list <ApplicationAttemptId>
[summer@hadoop102 hadoop-3.1.3]$ yarn container -list appattempt_1665325770064_0003_000001
2022-10-10 20:15:25,743 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.159.103:8032
Total number of containers :0
Container-Id Start Time Finish Time State Host Node Http Address LOG-URL
只有在任務(wù)跑的途中才能看到 container 的狀態(tài)，因?yàn)檫@個(gè)任務(wù)運(yùn)行完之后，container容器會(huì)立即釋放，我這里就沒(méi)有container容器，因?yàn)槿蝿?wù)已經(jīng)運(yùn)行結(jié)束了
[summer@hadoop102 hadoop-3.1.3]$ yarn applicationattempt -list application_1665325770064_0010
2022-10-10 20:25:08,250 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.159.103:8032
Total number of application attempts :1
ApplicationAttempt-Id State AM-Container-Id Tracking-URL
appattempt_1665325770064_0010_000001 RUNNING container_1665325770064_0010_01_000001 http://hadoop103:8088/proxy/application_1665325770064_0010/
這個(gè)靠手速弄出來(lái)一個(gè)結(jié)果，這個(gè)就是有容器的時(shí)候里面顯示的內(nèi)容
16.5.4.2 打印Container狀態(tài)：
yarn container -status <ContainerId>
[summer@hadoop102 hadoop-3.1.3]$ yarn container -status container_1665325770064_0003_01_000001
2022-10-10 20:20:13,494 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.159.103:8032
Container with id ‘container_1665325770064_0003_01_000001’ doesn’t exist in RM or Timeline Server.
注：只有在任務(wù)跑的途中才能看到 container 的狀態(tài)
[summer@hadoop102 hadoop-3.1.3]$ yarn container -status container_1665325770064_0010_01_000001
2022-10-10 20:25:14,642 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.159.103:8032
Container Report :
Container-Id : container_1665325770064_0010_01_000001
Start-Time : 1665404700780
Finish-Time : 0
State : RUNNING
Execution-Type : GUARANTEED
LOG-URL : http://hadoop104:8042/node/containerlogs/container_1665325770064_0010_01_000001/summer
Host : hadoop104:36495
NodeHttpAddress : http://hadoop104:8042
Diagnostics : null
這個(gè)靠手速弄了一個(gè)任務(wù)在運(yùn)行中的顯示結(jié)果
16.5.5 yarn node查看節(jié)點(diǎn)狀態(tài)
列出所有節(jié)點(diǎn)：yarn node -list -all
[summer@hadoop102 hadoop-3.1.3]$ yarn node -list -all
2022-10-10 20:33:40,711 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.159.103:8032
Total Nodes:3
Node-Id Node-State Node-Http-Address Number-of-Running-Containers
hadoop104:36495 RUNNING hadoop104:8042 0
hadoop103:38375 RUNNING hadoop103:8042 0
hadoop102:40891 RUNNING hadoop102:8042 0
16.5.6 yarn rmadmin更新配置
加載隊(duì)列配置：yarn rmadmin -refreshQueues
[summer@hadoop102 hadoop-3.1.3]$ yarn rmadmin -refreshQueues
2022-10-10 20:39:09,817 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.159.103:8033
16.5.7 yarn queue查看隊(duì)列
打印隊(duì)列信息：yarn queue -status
[summer@hadoop102 hadoop-3.1.3]$ yarn queue -status default
2022-10-10 20:43:10,934 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.159.103:8032
Queue Information :
Queue Name : default
State : RUNNING
Capacity : 100.0%
Current Capacity : .0%
Maximum Capacity : 100.0%
Default Node Label expression : <DEFAULT_PARTITION>
Accessible Node Labels : *
Preemption : disabled
Intra-queue Preemption : disabled
這里的值更詳細(xì)文章來(lái)源：http://www.zghlxwxcb.cn/news/detail-799516.html
16.6 Yarn生產(chǎn)環(huán)境核心參數(shù)
文章來(lái)源地址http://www.zghlxwxcb.cn/news/detail-799516.html
到了這里，關(guān)于HadoopYarn常用命令、yarn application查看任務(wù)、yarn logs查看日志、yarn applicationattempt查看嘗試運(yùn)行的任務(wù)、查看容器、Yarn生產(chǎn)環(huán)境核心參數(shù)的文章就介紹完了。如果您還想了解更多內(nèi)容，請(qǐng)?jiān)谟疑辖撬阉鱐OY模板網(wǎng)以前的文章或繼續(xù)瀏覽下面的相關(guān)文章，希望大家以后多多支持TOY模板網(wǎng)！
