1. 基本概念
-
FE，F(xiàn)rontend，前端節(jié)點(diǎn)，接收用戶查詢請(qǐng)求，SQL解析，執(zhí)行計(jì)劃生成，元數(shù)據(jù)管理，節(jié)點(diǎn)管理等
-
BE，Backend，后端節(jié)點(diǎn)，數(shù)據(jù)存儲(chǔ)，執(zhí)行查詢計(jì)劃。
前端節(jié)點(diǎn)FE 和 后端節(jié)點(diǎn)BE 各自獨(dú)立運(yùn)行，互不影響。
-
broker：用來(lái)和外部文件系統(tǒng)打交道
2. 修改配置
DORIS_HOME=/export/server/doris-1.2.4.1
node1：安裝doris的機(jī)器名
| 配置文件 | 修改參數(shù) |
| $DORIS_HOME/fe/conf/fe.conf | priority_networks = node1/24 meta_dir = DORIS_HOME/fe/doris-meta |
| $DORIS_HOME/be/conf/be.conf | priority_networks = node1/24 storage_root_path = DORIS_HOME/be/storage1,10;DORIS_HOME/be/storage2 |
| $DORIS_HOME//extensions/apache_hdfs_broker /conf/apache_hdfs_broker.conf |
3. 啟動(dòng)腳本
| 啟動(dòng)腳本 | 運(yùn)行進(jìn)程 |
| $DORIS_HOME/fe/bin/start_fe.sh --daemon | PaloFe |
| ulimit -n?60000 sysctl -w vm.max_map_count=2000000 $DORIS_HOME/be/bin/start_be.sh --daemon |
ps -ef | grep be?? |
| $DORIS_HOME//extensions/apache_hdfs_broker/bin/start_broker.sh --daemon | BrokerBootstrap |
4. 停止腳本
| $DORIS_HOME/fe/bin/stop_fe.sh |
| $DORIS_HOME/be/bin/stop_be.sh? |
| $DORIS_HOME//extensions/apache_hdfs_broker/bin/stop_broker.sh? |
5. Doris端口
常用端口
webui: node1:8030
FE上的MySQL Server端口: 9030
6. MySQL客戶端連接Doris
mysql -uroot -P9030 -hnode1-u 此處使用的root用戶是doris內(nèi)置的默認(rèn)用戶，也是超級(jí)管理員用戶
-P:這是我們連接到Doris的查詢端口，默認(rèn)端口是9030，對(duì)應(yīng)于fe.conf中的query_port
-h:這是我們連接的FE的IP地址，如果你的客戶端和FE安裝在同一個(gè)節(jié)點(diǎn)上，可以使用127.0.0.1
查看FE、BE、Broker運(yùn)行狀態(tài)
show frontends\G;
show backends\G;
show broker\G;擴(kuò)縮容
#1.擴(kuò)容
#1.1添加前端
ALTER SYSTEM ADD FRONTEND "192.168.0.1:9050";
#1.2添加后端
ALTER SYSTEM ADD BACKEND "192.168.0.1:9050";
#2.縮容
#1.1剔除前端
ALTER SYSTEM DROP FRONTEND "host1:port", "host2:port";
#1.2剔除后端
ALTER SYSTEM DROP BACKEND "host1:port", "host2:port";清屏: CTRL + L
7. 創(chuàng)建表
建表
修改分區(qū)
mysql> ALTER TABLE demo.example_tb1 ADD PARTITION IF NOT EXISTS `p202005` VALUES LESS THAN ("2020-06-01");
Query OK, 0 rows affected (0.54 sec)
mysql> ALTER TABLE demo.example_list_tb2 ADD PARTITION IF NOT EXISTS p_uk VALUES IN ("London");
Query OK, 0 rows affected (0.25 sec)
mysql> ALTER TABLE demo.example_tb1 DROP PARTITION IF EXISTS p202005;
Query OK, 0 rows affected (0.09 sec)
mysql> ALTER TABLE demo.example_list_tb2 DROP PARTITION IF EXISTS p_uk;
Query OK, 0 rows affected (0.03 sec)
分區(qū)可以省略，如果省略的話，默認(rèn)Doris系統(tǒng)會(huì)創(chuàng)建一個(gè)分區(qū)，這個(gè)分區(qū)成為單分區(qū)，它的分區(qū)名字和表名一樣。這種很常用。
數(shù)據(jù)模型
Aggregate Key，相同的key，value會(huì)做聚合操作。按照給定的聚合函數(shù) (sum、max、min、replace) 進(jìn)行聚合。
Unique Key，保證key列的唯一性。只要key相同，新的值會(huì)覆蓋舊的值。
Duplicate Key，運(yùn)行數(shù)據(jù)冗余存儲(chǔ)，保留數(shù)據(jù)原始的樣子，不會(huì)對(duì)數(shù)據(jù)做任何操作。
建表時(shí)，可以省略，默認(rèn)是冗余模型。
8. 數(shù)據(jù)操作
數(shù)據(jù)導(dǎo)入
數(shù)據(jù)導(dǎo)出
數(shù)據(jù)刪除
數(shù)據(jù)修改
9. Rollup 和?物化視圖
Doris建表默認(rèn)是有順序的，這個(gè)順序就是字段的順序，可以認(rèn)為這就是它默認(rèn)的聚合索引。若根據(jù)某字段過(guò)濾數(shù)據(jù)，根據(jù)索引最左匹配原則，有可能索引失效，導(dǎo)致全表掃描。
rollup可以調(diào)整字段順序，使字段順序盡可能匹配過(guò)濾字段，以此增加前綴索引的匹配度，提升查詢效率。
查看表的Rollup：
desc table_name all;創(chuàng)建Rollup：
alter table table_name add rollup rollup_name (field1,field2...);#1.創(chuàng)建rollup
mysql> alter table example_site_visit add rollup rollup_cost_userid(user_id,cost);
#2.再創(chuàng)建rollup
mysql>alter table example_site_visit add rollup rollup_cost_userid2(age,date,city,user_id,sex,last_visit_date,cost,max_dwell_time,min_dwell_time);
mysql> desc example_site_visit all;
+---------------------+---------------+-----------------+-------------+--------------+------+-------+---------------------+---------+---------+
| IndexName | IndexKeysType | Field | Type | InternalType | Null | Key | Default | Extra | Visible |
+---------------------+---------------+-----------------+-------------+--------------+------+-------+---------------------+---------+---------+
| example_site_visit | AGG_KEYS | user_id | LARGEINT | LARGEINT | No | true | NULL | | true |
| | | date | DATE | DATE | No | true | NULL | | true |
| | | city | VARCHAR(20) | VARCHAR(20) | Yes | true | NULL | | true |
| | | age | SMALLINT | SMALLINT | Yes | true | NULL | | true |
| | | sex | TINYINT | TINYINT | Yes | true | NULL | | true |
| | | last_visit_date | DATETIME | DATETIME | Yes | false | 1970-01-01 00:00:00 | REPLACE | true |
| | | cost | BIGINT | BIGINT | Yes | false | 0 | SUM | true |
| | | max_dwell_time | INT | INT | Yes | false | 0 | MAX | true |
| | | min_dwell_time | INT | INT | Yes | false | 99999 | MIN | true |
| | | | | | | | | | |
| rollup_cost_userid | AGG_KEYS | user_id | LARGEINT | LARGEINT | No | true | NULL | | true |
| | | cost | BIGINT | BIGINT | Yes | false | 0 | SUM | true |
| | | | | | | | | | |
| rollup_cost_userid2 | AGG_KEYS | age | SMALLINT | SMALLINT | Yes | true | NULL | | true |
| | | date | DATE | DATE | No | true | NULL | | true |
| | | city | VARCHAR(20) | VARCHAR(20) | Yes | true | NULL | | true |
| | | user_id | LARGEINT | LARGEINT | No | true | NULL | | true |
| | | sex | TINYINT | TINYINT | Yes | true | NULL | | true |
| | | last_visit_date | DATETIME | DATETIME | Yes | false | 1970-01-01 00:00:00 | REPLACE | true |
| | | cost | BIGINT | BIGINT | Yes | false | 0 | SUM | true |
| | | max_dwell_time | INT | INT | Yes | false | 0 | MAX | true |
| | | min_dwell_time | INT | INT | Yes | false | 99999 | MIN | true |
+---------------------+---------------+-----------------+-------------+--------------+------+-------+---------------------+---------+---------+
22 rows in set (0.01 sec)
物化視圖
Doris的物化視圖，本質(zhì)上也是一個(gè)rollup，只是語(yǔ)法不一樣。
mysql> CREATE MATERIALIZED VIEW example_site_visit_mv AS
-> SELECT user_id,city,SUM(cost)
-> FROM example_site_visit
-> GROUP BY user_id,city;
Query OK, 0 rows affected (0.20 sec)
mysql> desc example_site_visit all
10. 內(nèi)置函數(shù)
語(yǔ)法:show builtin functions in database_name;
使用:show builtin functions in demo;
使用方式: help + 函數(shù)名;
請(qǐng)點(diǎn)擊查看更多函數(shù)，查看左側(cè)導(dǎo)航樹(shù)
11. 動(dòng)態(tài)分區(qū)
開(kāi)啟動(dòng)態(tài)分區(qū)
#1.開(kāi)啟動(dòng)態(tài)分區(qū),root用戶未設(shè)置密碼，所以為空
curl --location-trusted -u root: -XGET http://node1:8030/api/_set_config?dynamic_partition_enable=true
#2.設(shè)置動(dòng)態(tài)分區(qū)的檢測(cè)時(shí)間間隔，root用戶未設(shè)置密碼，所以為空
curl --location-trusted -u root: -XGET http://node1:8030/api/_set_config?dynamic_partition_check_interval_seconds=5動(dòng)態(tài)分區(qū)語(yǔ)法
PARTITION BY RANGE('分區(qū)字段')()
動(dòng)態(tài)分區(qū)不支持list分區(qū)
創(chuàng)建動(dòng)態(tài)分區(qū)表
-- 創(chuàng)建分區(qū)
CREATE TABLE order_dynamic_partition
(
id int,
time date,
money double,
areaName varchar(50)
)
duplicate key(id,time)
PARTITION BY RANGE(time)()
DISTRIBUTED BY HASH(id) buckets 10
PROPERTIES(
"dynamic_partition.enable" = "true",
"dynamic_partition.time_unit" = "DAY", --四種動(dòng)態(tài)分區(qū)類型：HOUR,DAY,WEEK,MONTH
"dynamic_partition.start" = "-7", -- 保留到7天前的分區(qū)
"dynamic_partition.end" = "3", --也創(chuàng)建后3天的分區(qū)
"dynamic_partition.prefix" = "p", --分區(qū)名稱前綴
"dynamic_partition.buckets" = "10",
"replication_num" = "1"
);
-- 查看分區(qū)
show partitions from order_dynamic_partition;查看動(dòng)態(tài)分區(qū)表
show dynamic partition tables;
動(dòng)態(tài)分區(qū)表? 與? 靜態(tài)分區(qū)表 的轉(zhuǎn)換開(kāi)關(guān)
"dynamic_partition.enable" = "true"文章來(lái)源：http://www.zghlxwxcb.cn/news/detail-730856.html
true為開(kāi)啟動(dòng)態(tài)分區(qū)，false為普通分區(qū)文章來(lái)源地址http://www.zghlxwxcb.cn/news/detail-730856.html
-- 創(chuàng)建靜態(tài)分區(qū)表
CREATE TABLE table_partition
(
id int,
time date,
money double,
areaName varchar(50)
)
duplicate key(id,time)
PARTITION BY RANGE(time)
(
PARTITION `p202001` VALUES LESS THAN ("2020-02-01"),
PARTITION `p202002` VALUES LESS THAN ("2020-03-01"),
PARTITION `p202003` VALUES LESS THAN ("2020-04-01")
)
DISTRIBUTED BY HASH(id) buckets 10
PROPERTIES
(
"dynamic_partition.enable" = "false",
"dynamic_partition.time_unit" = "DAY",
"dynamic_partition.prefix" = "p",
"dynamic_partition.end" = "3",
"dynamic_partition.buckets" = "10",
"replication_num" = "1"
);
-- 靜態(tài)分區(qū)表轉(zhuǎn)換為動(dòng)態(tài)分區(qū)表
ALTER TABLE table_partition set ("dynamic_partition.enable" = "true");
-- 動(dòng)態(tài)分區(qū)表轉(zhuǎn)換為靜態(tài)分區(qū)表
ALTER TABLE table_partition set ("dynamic_partition.enable" = "false");到了這里，關(guān)于Apache Doris 快速入門的文章就介紹完了。如果您還想了解更多內(nèi)容，請(qǐng)?jiān)谟疑辖撬阉鱐OY模板網(wǎng)以前的文章或繼續(xù)瀏覽下面的相關(guān)文章，希望大家以后多多支持TOY模板網(wǎng)！
