1. docker-compose常用命令

詳見
https://blog.csdn.net/qq_42267173/article/details/124687804
https://blog.csdn.net/m0_72838865/article/details/126421685
https://www.cnblogs.com/cw2blog/p/16689584.html 容器啟停相關的

# 前臺啟動
docker-compose up
# 后臺啟動
docker-compose up -d

logs
# docker-compose logs查看服務容器的輸出日志。默認情況下，docker-compose將對不同的服務輸出使用不同的顏色來區(qū)分?？梢酝ㄟ^--no-color來關閉顏色。
# 輸出日志，不同的服務輸出使用不同的顏色來區(qū)分
docker-compose logs
# 跟蹤日志輸出
docker-compose logs -f
# 關閉顏色
docker-compose logs --no-color

run
docker-compose run在指定服務容器上執(zhí)行一個命令。
docker-compose run nginx echo "helloworld" # 在工程中指定服務的容器上執(zhí)行 echo "helloworld"

exec
docker-compose exec進入服務容器。
docker-compose exec nginx bash # 進入工程中指定服務的容器
docker-compose exec --index=1 nginx bash # 當一個服務擁有多個容器時，可通過 --index 參數(shù)進入到該服務下的任何容器

pause
docker-compose pause暫停服務容器
docker-compose pause # 暫停工程中所有服務的容器
docker-compose pause nginx # 暫停工程中指定服務的容器

unpause
docker-compose unpause恢復服務容器。
docker-compose unpause # 恢復工程中所有服務的容器
docker-compose unpause nginx # 恢復工程中指定服務的容器

restart
docker-compose restart重啟服務容器。
docker-compose restart # 重啟工程中所有服務的容器
docker-compose restart nginx # 重啟工程中指定服務的容器

start
docker-compose start啟動服務容器。
docker-compose start # 啟動工程中所有服務的容器
docker-compose start nginx # 啟動工程中指定服務的容器

stop
docker-compose stop停止服務容器。
docker-compose stop # 停止工程中所有服務的容器
docker-compose stop nginx # 停止工程中指定服務的容器


sudo docker-compose ps
sudo docker-compose stop jobmanager
sudo docker-compose stop taskmanager
sudo docker-compose rm -f jobmanager
sudo docker-compose rm -f taskmanager

sudo docker-compose exec jobmanager ./bin/sql-client.sh -f sql/flink_kafka1.sql
sudo docker-compose exec jobmanager ./bin/flink list
sudo docker logs -f -t --since="2023-05-08" --tail=200 flink_taskmanager_1
sudo docker-compose logs -f taskmanager

sudo docker-compose exec jobmanager bash
sudo docker-compose exec jobmanager ./bin/flink cancel 8d8cc94d73f7bd0c4cdc557264553a04

sudo docker-compose scale taskmanager=5

2. 準備工作

sudo yum install docker
sudo yum install docker-compose

3. 創(chuàng)建目錄

mkdir flink

4. vim docker-compose.yml

version: "2.1"
services:
  jobmanager:
    image: flink:1.14
    expose:
      - "6123"
    ports:
      - "8081:8081"
    command: jobmanager
    environment:
      - JOB_MANAGER_RPC_ADDRESS=jobmanager
  taskmanager:
    image: flink:1.14
    expose:
      - "6121"
      - "6122"
    depends_on:
      - jobmanager
    command: taskmanager
    links:
      - "jobmanager:jobmanager"
    environment:
      - JOB_MANAGER_RPC_ADDRESS=jobmanager
 

5. 啟動

啟動報錯

$ docker-compose up
ERROR: Couldn't connect to Docker daemon at http+docker://localunixsocket - is it running?

If it's at a non-standard location, specify the URL with the DOCKER_HOST environment variable.

解決參考：https://blog.csdn.net/xiojing825/article/details/79494408

# 加上sudo，這個會輸出詳細日志到窗口
flink]$ sudo docker-compose up
Creating network "flink_default" with the default driver
Pulling jobmanager (flink:1.14)...
1.14: Pulling from library/flink
301a8b74f71f: Pull complete
4688df200b56: Pull complete
317eff501ed4: Pull complete

docker-compose up -d 后臺運行，不輸出日志到窗口

5. 啟動多個taskmanager

# 啟用5個taskmanager
docker-compose scale taskmanager=5 

6. 瀏覽器觀看

http://IP:8082

想web訪問，但是只能訪問localhost，本地ip都無法訪問。
解決方法：https://blog.csdn.net/as350144/article/details/125654394

7. 停止容器

flink]$ sudo docker-compose ps
       Name                      Command               State                              Ports
---------------------------------------------------------------------------------------------------------------------------
flink_jobmanager_1    /docker-entrypoint.sh jobm ...   Up      6123/tcp, 8081/tcp, 0.0.0.0:8082->8082/tcp,:::8082->8082/tcp
flink_taskmanager_1   /docker-entrypoint.sh task ...   Up      6121/tcp, 6122/tcp, 6123/tcp, 8081/tcp

看下yml配置文件的服務名字，jobmanager和taskmanager
[xx@sandbox-test flink]$ sudo docker-compose stop jobmanager
Stopping flink_jobmanager_1 ... done
[xx@sandbox-test flink]$ sudo docker-compose stop taskmanager
Stopping flink_taskmanager_1 ... done

8. 完整的

# 啟動flink
[xx@sandbox-test flink]$ sudo docker-compose up -d
Starting flink_jobmanager_1 ... done
Starting flink_taskmanager_1 ... done

# 查看容器
[xx@sandbox-test flink]$ sudo docker-compose ps
       Name                      Command               State                              Ports
---------------------------------------------------------------------------------------------------------------------------
flink_jobmanager_1    /docker-entrypoint.sh jobm ...   Up      6123/tcp, 8081/tcp, 0.0.0.0:8082->8082/tcp,:::8082->8082/tcp
flink_taskmanager_1   /docker-entrypoint.sh task ...   Up      6121/tcp, 6122/tcp, 6123/tcp, 8081/tcp

# 查看日志
[xx@sandbox-test flink]$ sudo docker-compose logs
或者 docker-compose logs -f taskmanager

# 擴展taskmanager
[xx@sandbox-test flink]$ sudo docker-compose scale taskmanager=5
WARNING: The scale command is deprecated. Use the up command with the --scale flag instead.
Starting flink_taskmanager_1 ... done
Creating flink_taskmanager_2 ... done
Creating flink_taskmanager_3 ... done
Creating flink_taskmanager_4 ... done
Creating flink_taskmanager_5 ... done

9. 擴展（修改flink的conf）

如果要修改flink的conf呢？
參考:https://www.dandelioncloud.cn/article/details/1589983093958324225
https://www.clzg.cn/article/432005.html

flink下創(chuàng)建以下目錄并將flink-conf.yaml拷貝到兩個目錄下
├── conf
│   ├── job
│   │   └── flink-conf.yaml
│   └── task
│       └── flink-conf.yaml

docker-compose.yml修改如下：主要是增加了volumes，表示使用本地的conf文件，不使用容器里面自帶的conf。

version: "2.1"
services:
  jobmanager:
    image: flink:1.14
    expose:
      - "6123"
    ports:
      - "8082:8082"
    command: jobmanager
    volumes:
      - /home/disk1/xx/soft/flink/conf/job/flink-conf.yaml:/opt/flink/conf/flink-conf.yaml
    environment:
      - JOB_MANAGER_RPC_ADDRESS=jobmanager
  taskmanager:
    image: flink:1.14
    expose:
      - "6121"
      - "6122"
    depends_on:
      - jobmanager
    command: taskmanager
    links:
      - "jobmanager:jobmanager"
    environment:
      - JOB_MANAGER_RPC_ADDRESS=jobmanager
    volumes:
      - /home/disk1/xx/soft/flink/conf/task/flink-conf.yaml:/opt/flink/conf/flink-conf.yaml

flink-conf.yaml需要做一些修改，修改內容參考上面的鏈接。
就可以啟動flink了。

sudo docker-compose up -d
sudo docker-compose scale taskmanager=5

11. lib jar包

參考 （修改flink的conf）

12. 執(zhí)行sql 語句

flink sql執(zhí)行參考：https://www.cnblogs.com/isuning/p/16214356.html
報錯如下，解決參考：https://help.aliyun.com/document_detail/468412.html#section-jtg-wow-qey

sandbox-test flink]$ sudo docker-compose exec jobmanager ./bin/sql-client.sh -f flink_kafka1.sql


Exception in thread "main" org.apache.flink.table.client.SqlClientException: Unexpected exception. This is a bug. Please consider filing an issue.
	at org.apache.flink.table.client.SqlClient.startClient(SqlClient.java:201)
	at org.apache.flink.table.client.SqlClient.main(SqlClient.java:161)
Caused by: java.lang.NoSuchMethodError: org.apache.flink.table.catalog.FunctionCatalog.<init>(Lorg/apache/flink/configuration/ReadableConfig;Lorg/apache/flink/table/catalog/CatalogManager;Lorg/apache/flink/table/module/ModuleManager;)V
	at org.apache.flink.table.client.gateway.context.SessionContext.create(SessionContext.java:237)
	at org.apache.flink.table.client.gateway.local.LocalContextUtils.buildSessionContext(LocalContextUtils.java:87)
	at org.apache.flink.table.client.gateway.local.LocalExecutor.openSession(LocalExecutor.java:87)
	at org.apache.flink.table.client.SqlClient.start(SqlClient.java:88)
	at org.apache.flink.table.client.SqlClient.startClient(SqlClient.java:187)
	... 1 more

flink版本和jar包之間不兼容。
將flink版本改為1.16，重新啟動docker

sudo docker-compose exec jobmanager ./bin/sql-client.sh embedded -f sql/flink_kafka2.sql報錯如下

Flink SQL> [INFO] Submitting SQL update statement to the cluster...
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.apache.flink.api.java.ClosureCleaner (file:/opt/flink/lib/flink-dist-1.16.0.jar) to field java.lang.String.value
WARNING: Please consider reporting this to the maintainers of org.apache.flink.api.java.ClosureCleaner
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

解決方法：https://blog.csdn.net/LIZHONGPING00/article/details/127947431
https://blog.csdn.net/qq_41538097/article/details/129113866

docker容器中安裝包

0、首先更新源
# apt-get update
1、安裝wget
# apt-get install wget
2、安裝ping，ifconfig
# apt-get install inetutils-ping
# apt-get install net-tools
3、安裝gcc
# apt-get install gcc
4、安裝各種依賴包
# apt-get install zlib* bzip2-devel openssl-devel ncurses-devel sqlite-devel readline-devel tk-devel tcl-devel gdbm-devel db4-devel libpcap-devel xz-devel libffi-dev

詳見https://www.cnblogs.com/aliensir/articles/15838834.html文章來源地址http://www.zghlxwxcb.cn/news/detail-752959.html

到了這里，關于docker-compose 部署flink集群的文章就介紹完了。如果您還想了解更多內容，請在右上角搜索TOY模板網以前的文章或繼續(xù)瀏覽下面的相關文章，希望大家以后多多支持TOY模板網！