環(huán)境說明:
主機(jī)名:cmcc01為例?
操作系統(tǒng):centos7
kettle版本:9.3.0
1. 下載kettle
官網(wǎng):https://sourceforge.net/projects/pentaho/files/
2.解壓
unzip /opt/package/pdi-ce-9.3.0.0-428.zip -d /opt/software/
3、配置java環(huán)境變量
vim ~/.bash_profile
# 添加以下內(nèi)容
# JDK
export JAVA_HOME=/opt/software/jdk1.8.0_321
export PATH=$PATH:${JAVA_HOME}/bin
使配置生效
source /etc/profile
4.給同組用戶賦權(quán)
chmod g+x /opt/software/data-integration/kitchen.sh
5.執(zhí)行命令
[root@cmcc01 data-integration]#
[root@cmcc01 data-integration]#
[root@cmcc01 data-integration]# ./kitchen.sh
#######################################################################
WARNING: no libwebkitgtk-1.0 detected, some features will be unavailable
Consider installing the package with apt-get or yum.
e.g. 'sudo apt-get install libwebkitgtk-1.0-0'
#######################################################################
Options:
-rep = Repository name
-user = Repository username
-trustuser = !Kitchen.ComdLine.RepUsername!
-pass = Repository password
-job = The name of the job to launch
-dir = The directory (dont forget the leading /)
-file = The filename (Job XML) to launch
-level = The logging level (Basic, Detailed, Debug, Rowlevel, Error, Minimal, Nothing)
-logfile = The logging file to write to
-listdir = List the directories in the repository
-listjobs = List the jobs in the specified directory
-listrep = List the available repositories
-norep = Do not log into the repository
-version = show the version, revision and build date
-param = Set a named parameter <NAME>=<VALUE>. For example -param:FILE=customers.csv
-listparam = List information concerning the defined parameters in the specified job.
-export = Exports all linked resources of the specified job. The argument is the name of a ZIP file.
-custom = Set a custom plugin specific option as a String value in the job using <NAME>=<Value>, for example: -custom:COLOR=Red
-maxloglines = The maximum number of log lines that are kept internally by Kettle. Set to 0 to keep all rows (default)
-maxlogtimeout = The maximum age (in minutes) of a log line while being kept internally by Kettle. Set to 0 to keep all rows indefinitely (default)
[root@cmcc01 data-integration]#
[root@cmcc01 data-integration]#
此處有告警?
6.解決告警
wget ftp://ftp.pbone.net/mirror/ftp5.gwdg.de/pub/opensuse/repositories/home:/matthewdva:/build:/EPEL:/el7/RHEL_7/x86_64/webkitgtk-2.4.9-1.el7.x86_64.rpm
yum -y install webkitgtk-2.4.9-1.el7.x86_64.rpm
# 再次執(zhí)行命令,告警消除
[root@cmcc01 package]#
[root@cmcc01 package]# /opt/software/data-integration/kitchen.sh
Options:
-rep = Repository name
-user = Repository username
-trustuser = !Kitchen.ComdLine.RepUsername!
-pass = Repository password
-job = The name of the job to launch
-dir = The directory (dont forget the leading /)
-file = The filename (Job XML) to launch
-level = The logging level (Basic, Detailed, Debug, Rowlevel, Error, Minimal, Nothing)
-logfile = The logging file to write to
-listdir = List the directories in the repository
-listjobs = List the jobs in the specified directory
-listrep = List the available repositories
-norep = Do not log into the repository
-version = show the version, revision and build date
-param = Set a named parameter <NAME>=<VALUE>. For example -param:FILE=customers.csv
-listparam = List information concerning the defined parameters in the specified job.
-export = Exports all linked resources of the specified job. The argument is the name of a ZIP file.
-custom = Set a custom plugin specific option as a String value in the job using <NAME>=<Value>, for example: -custom:COLOR=Red
-maxloglines = The maximum number of log lines that are kept internally by Kettle. Set to 0 to keep all rows (default)
-maxlogtimeout = The maximum age (in minutes) of a log line while being kept internally by Kettle. Set to 0 to keep all rows indefinitely (default)
[root@cmcc01 package]#
[root@cmcc01 package]#
7.測試
# 執(zhí)行轉(zhuǎn)換
# 編寫測試轉(zhuǎn)換,執(zhí)行如下命令即可
/opt/software/data-integration/pan.sh -file=/opt/kettle-spoon/ktr/test/test1.ktr log=test1.log
# 執(zhí)行job
/opt/software/data-integration/kitchen.sh -file=/opt/kettle-spoon/ktr/test/SechuldUpdate.kjb log=timeLogUpdate.log
8.kettle整合mysql、oracle
?復(fù)制驅(qū)動到data-integration/lib下
cp /opt/package/mysql-connector-java-8.0.20.jar /opt/software/data-integration/lib
cp /opt/package/ojdbc6.jar /opt/software/data-integration/lib
?9.kettle整合hive
# 創(chuàng)建hive jar包軟連接
ln -s /opt/software/hive-3.1.3-bin/lib/*.jar /opt/software/data-integration/lib
可能會報(bào)錯(cuò):File exists,可忽略
?創(chuàng)建job測試
?
?10.修改windows端spoon.bat編碼
如不進(jìn)行此操作則在向子服務(wù)器提交任務(wù)時(shí)會報(bào)如下錯(cuò)誤:Invalid byte 1 of 1-byte UTF-8 sequence
使用文本編輯器打開:\pdi-ce-9.3.0.0-428\data-integration\spoon.bat
在下圖紅框處添加: "-Dfile.encoding=UTF-8"
??
?11.修改linux端Carte用戶名及密碼
? ? ? ?使用Carte服務(wù)執(zhí)行作業(yè)需要授權(quán)。默認(rèn)情況下,Carte只支持最基本的授權(quán)方式,就是將密碼保存在kettle.pwd文件中。kettle.pwd文件位于Kettle根目錄下的pwd目錄下。默認(rèn)情況下,kettle.pwd的內(nèi)容如下:
# Please note that the default password (cluster) is obfuscated using the Encr script provided in this release
# Passwords can also be entered in plain text as before
#?
cluster: OBF:1v8w1uh21z7k1ym71z7i1ugo1v9q
? ? ? ? ?最后一行是唯一有用的一行,定義了一個(gè)用戶cluster,以及混淆后的密碼(這個(gè)密碼也是cluster)。文件的注釋說明了這個(gè)混淆的密碼是由Encr.bat或encr.sh腳本生成的。
sh /data-integration/encr.sh -carte cluster
# 執(zhí)行結(jié)果
OBF:1v8w1uh21z7k1ym71z7i1ugo1v9q
注意:cluster是你需混淆的密碼
? ? ? ? 將新生成的混淆后的密碼填寫入kettle.pwd文件中,如修改后密碼未生效,則是carte服務(wù)還未重啟成功,查看相應(yīng)進(jìn)程殺掉重啟即可。
12.啟動Carte服務(wù)
? ? ? ?在linux服務(wù)器上切換到kettle跟目錄下:cd ?/kettle/data-integration/
sh carte.sh 本機(jī)ip 端口號
例如:sh carte.sh 192.168.12.250 8888
執(zhí)行顯示如下信息,則說明服務(wù)啟動成功
13.windows端kettle配置子服務(wù)器
?
14.創(chuàng)建子服務(wù)器運(yùn)行配置
?文章來源:http://www.zghlxwxcb.cn/news/detail-649148.html
文章來源地址http://www.zghlxwxcb.cn/news/detail-649148.html
到了這里,關(guān)于CentOS7部署kettle9.3.0并部署自服器遠(yuǎn)程提交任務(wù)的文章就介紹完了。如果您還想了解更多內(nèi)容,請?jiān)谟疑辖撬阉鱐OY模板網(wǎng)以前的文章或繼續(xù)瀏覽下面的相關(guān)文章,希望大家以后多多支持TOY模板網(wǎng)!