开发者社区> 樱桃味> 正文
阿里云
为了无法计算的价值
打开APP
阿里云APP内打开

linux kettle greenplum remote

简介: linux kettle greenplum 远程 两个参考文件中发现可以在etl服务器中加载gpload文件,通过文件将数据传送到远程的gp服务器中。
+关注继续查看
linux kettle greenplum 远程


两个参考文件中发现可以在etl服务器中加载gpload文件,通过文件将数据传送到远程的gp服务器中。

1、安装部署gpload以及相应的依赖文件

linux 操作系统:redhat 6.5

1-1)[root@localhost ~]# python
Python 2.6.6 (r266:84292, Sep  4 2013, 07:46:00)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> exit()


1-2)[root@localhost software]# tar -zxvf PyYAML-3.10.tar.gz
     [root@localhost PyYAML-3.10]# python setup.py install

1-3)[root@localhost software]# tar -zxvf yaml-0.1.4.tar.gz
[root@localhost yaml-0.1.4]# ./configure
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking for gcc... no
checking for cc... no
checking for cl.exe... no
configure: error: in `/software/yaml-0.1.4':
configure: error: no acceptable C compiler found in $PATH(缺少gcc依赖包http://blog.csdn.net/duguduchong/article/details/8699774
See `config.log' for more details
[root@localhost yaml-0.1.4]# make && make install
make: *** No targets specified and no makefile found.  Stop.

导致第4步,第5步,第6步执行

如果第3步不报错,那么后续3步可以不能执行

1-4)yum --update

1-4-1、cp -R /media/RHEL_6.5\ x86_64\ Disc\ 1/ /softwa
re/bao

1-4-2、[root@localhost yum.repos.d]# cat rhel-source.repo
[oel6]

name = Enterprise Linux 6.0 DVD

baseurl=file:///software/bao/Server

gpgcheck=0

enabled=1

1-4-3、[root@localhost yum.repos.d]# yum update
Loaded plugins: product-id, refresh-packagekit, security, subscription-
              : manager
This system is not registered to Red Hat Subscription Management. You can us
e subscription-manager to register.oel6                                                 | 3.9 kB     00:00 ...
oel6/primary_db                                      | 3.1 MB     00:00 ...
Setting up Update Process
No Packages marked for Update
1-4-4、yum install gcc

1-5)再次执行 [root@localhost yaml-0.1.4]# ./configure

checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for style of include used by make... GNU
checking dependency style of gcc... gcc3
checking how to run the C preprocessor... gcc -E
checking whether ln -s works... yes
checking whether make sets $(MAKE)... (cached) yes
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking for a sed that does not truncate output... /bin/sed
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for fgrep... /bin/grep -F
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking the maximum length of command line arguments... 1966080
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... yes
checking for /usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for ar... ar
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc object... ok
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc supports -fno-rtti -fno-exceptions... no
checking for gcc option to produce PIC... -fPIC -DPIC
checking if gcc PIC flag -fPIC -DPIC works... yes
checking if gcc static flag -static works... no
checking if gcc supports -c -o file.o... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared
libraries... yeschecking whether -lc should be explicitly linked in... no
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... yes
checking for doxygen... false
checking for ANSI C header files... (cached) yes
checking for stdlib.h... (cached) yes
checking for an ANSI C-conforming const... yes
checking for size_t... yes
configure: creating ./config.status
config.status: creating yaml-0.1.pc
config.status: creating include/Makefile
config.status: creating src/Makefile
config.status: creating Makefile
config.status: creating tests/Makefile
config.status: creating win32/Makefile
config.status: creating config.h
config.status: executing depfiles commands
config.status: executing libtool commands
[root@localhost yaml-0.1.4]#

1-6)再次执行[root@localhost yaml-0.1.4]# make && make install
make  all-recursive
make[1]: Entering directory `/software/yaml-0.1.4'
Making all in include
make[2]: Entering directory `/software/yaml-0.1.4/include'
make[2]: Nothing to be done for `all'.
make[2]: Leaving directory `/software/yaml-0.1.4/include'
Making all in src
make[2]: Entering directory `/software/yaml-0.1.4/src'
/bin/sh ../libtool --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I. -I..  -
I../include   -g -O2 -MT api.lo -MD -MP -MF .deps/api.Tpo -c -o api.lo api.clibtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I../include -g -O2 -MT api.
lo -MD -MP -MF .deps/api.Tpo -c api.c  -fPIC -DPIC -o .libs/api.olibtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I../include -g -O2 -MT api.
lo -MD -MP -MF .deps/api.Tpo -c api.c -o api.o >/dev/null 2>&1mv -f .deps/api.Tpo .deps/api.Plo
/bin/sh ../libtool --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I. -I..  -
I../include   -g -O2 -MT reader.lo -MD -MP -MF .deps/reader.Tpo -c -o reader.lo reader.clibtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I../include -g -O2 -MT read
er.lo -MD -MP -MF .deps/reader.Tpo -c reader.c  -fPIC -DPIC -o .libs/reader.olibtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I../include -g -O2 -MT read
er.lo -MD -MP -MF .deps/reader.Tpo -c reader.c -o reader.o >/dev/null 2>&1mv -f .deps/reader.Tpo .deps/reader.Plo
/bin/sh ../libtool --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I. -I..  -
I../include   -g -O2 -MT scanner.lo -MD -MP -MF .deps/scanner.Tpo -c -o scanner.lo scanner.clibtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I../include -g -O2 -MT scan
ner.lo -MD -MP -MF .deps/scanner.Tpo -c scanner.c  -fPIC -DPIC -o .libs/scanner.olibtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I../include -g -O2 -MT scan
ner.lo -MD -MP -MF .deps/scanner.Tpo -c scanner.c -o scanner.o >/dev/null 2>&1mv -f .deps/scanner.Tpo .deps/scanner.Plo
/bin/sh ../libtool --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I. -I..  -
I../include   -g -O2 -MT parser.lo -MD -MP -MF .deps/parser.Tpo -c -o parser.lo parser.clibtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I../include -g -O2 -MT pars
er.lo -MD -MP -MF .deps/parser.Tpo -c parser.c  -fPIC -DPIC -o .libs/parser.olibtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I../include -g -O2 -MT pars
er.lo -MD -MP -MF .deps/parser.Tpo -c parser.c -o parser.o >/dev/null 2>&1mv -f .deps/parser.Tpo .deps/parser.Plo
/bin/sh ../libtool --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I. -I..  -
I../include   -g -O2 -MT loader.lo -MD -MP -MF .deps/loader.Tpo -c -o loader.lo loader.clibtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I../include -g -O2 -MT load
er.lo -MD -MP -MF .deps/loader.Tpo -c loader.c  -fPIC -DPIC -o .libs/loader.olibtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I../include -g -O2 -MT load
er.lo -MD -MP -MF .deps/loader.Tpo -c loader.c -o loader.o >/dev/null 2>&1mv -f .deps/loader.Tpo .deps/loader.Plo
/bin/sh ../libtool --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I. -I..  -
I../include   -g -O2 -MT writer.lo -MD -MP -MF .deps/writer.Tpo -c -o writer.lo writer.clibtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I../include -g -O2 -MT writ
er.lo -MD -MP -MF .deps/writer.Tpo -c writer.c  -fPIC -DPIC -o .libs/writer.olibtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I../include -g -O2 -MT writ
er.lo -MD -MP -MF .deps/writer.Tpo -c writer.c -o writer.o >/dev/null 2>&1mv -f .deps/writer.Tpo .deps/writer.Plo
/bin/sh ../libtool --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I. -I..  -
I../include   -g -O2 -MT emitter.lo -MD -MP -MF .deps/emitter.Tpo -c -o emitter.lo emitter.clibtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I../include -g -O2 -MT emit
ter.lo -MD -MP -MF .deps/emitter.Tpo -c emitter.c  -fPIC -DPIC -o .libs/emitter.olibtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I../include -g -O2 -MT emit
ter.lo -MD -MP -MF .deps/emitter.Tpo -c emitter.c -o emitter.o >/dev/null 2>&1mv -f .deps/emitter.Tpo .deps/emitter.Plo
/bin/sh ../libtool --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I. -I..  -
I../include   -g -O2 -MT dumper.lo -MD -MP -MF .deps/dumper.Tpo -c -o dumper.lo dumper.clibtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I../include -g -O2 -MT dump
er.lo -MD -MP -MF .deps/dumper.Tpo -c dumper.c  -fPIC -DPIC -o .libs/dumper.olibtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I../include -g -O2 -MT dump
er.lo -MD -MP -MF .deps/dumper.Tpo -c dumper.c -o dumper.o >/dev/null 2>&1mv -f .deps/dumper.Tpo .deps/dumper.Plo
/bin/sh ../libtool --tag=CC   --mode=link gcc  -g -O2 -release 0 -version-in
fo 2:2:0  -o libyaml.la -rpath /usr/local/lib api.lo reader.lo scanner.lo parser.lo loader.lo writer.lo emitter.lo dumper.lo  libtool: link: gcc -shared  .libs/api.o .libs/reader.o .libs/scanner.o .libs
/parser.o .libs/loader.o .libs/writer.o .libs/emitter.o .libs/dumper.o      -Wl,-soname -Wl,libyaml-0.so.2 -o .libs/libyaml-0.so.2.0.2libtool: link: (cd ".libs" && rm -f "libyaml-0.so.2" && ln -s "libyaml-0.so.
2.0.2" "libyaml-0.so.2")libtool: link: (cd ".libs" && rm -f "libyaml.so" && ln -s "libyaml-0.so.2.0.
2" "libyaml.so")libtool: link: ar cru .libs/libyaml.a  api.o reader.o scanner.o parser.o loa
der.o writer.o emitter.o dumper.olibtool: link: ranlib .libs/libyaml.a
libtool: link: ( cd ".libs" && rm -f "libyaml.la" && ln -s "../libyaml.la" "
libyaml.la" )make[2]: Leaving directory `/software/yaml-0.1.4/src'
Making all in .
make[2]: Entering directory `/software/yaml-0.1.4'
make[2]: Leaving directory `/software/yaml-0.1.4'
Making all in tests
make[2]: Entering directory `/software/yaml-0.1.4/tests'
gcc -DHAVE_CONFIG_H -I. -I..  -I../include   -g -O2 -MT run-scanner.o -MD -M
P -MF .deps/run-scanner.Tpo -c -o run-scanner.o run-scanner.cmv -f .deps/run-scanner.Tpo .deps/run-scanner.Po
/bin/sh ../libtool --tag=CC   --mode=link gcc  -g -O2   -o run-scanner run-s
canner.o ../src/libyaml.la libtool: link: gcc -g -O2 -o .libs/run-scanner run-scanner.o  ../src/.libs/l
ibyaml.so -Wl,-rpath -Wl,/usr/local/libgcc -DHAVE_CONFIG_H -I. -I..  -I../include   -g -O2 -MT run-parser.o -MD -MP
-MF .deps/run-parser.Tpo -c -o run-parser.o run-parser.cmv -f .deps/run-parser.Tpo .deps/run-parser.Po
/bin/sh ../libtool --tag=CC   --mode=link gcc  -g -O2   -o run-parser run-pa
rser.o ../src/libyaml.la libtool: link: gcc -g -O2 -o .libs/run-parser run-parser.o  ../src/.libs/lib
yaml.so -Wl,-rpath -Wl,/usr/local/libgcc -DHAVE_CONFIG_H -I. -I..  -I../include   -g -O2 -MT run-loader.o -MD -MP
-MF .deps/run-loader.Tpo -c -o run-loader.o run-loader.cmv -f .deps/run-loader.Tpo .deps/run-loader.Po
/bin/sh ../libtool --tag=CC   --mode=link gcc  -g -O2   -o run-loader run-lo
ader.o ../src/libyaml.la libtool: link: gcc -g -O2 -o .libs/run-loader run-loader.o  ../src/.libs/lib
yaml.so -Wl,-rpath -Wl,/usr/local/libgcc -DHAVE_CONFIG_H -I. -I..  -I../include   -g -O2 -MT run-emitter.o -MD -M
P -MF .deps/run-emitter.Tpo -c -o run-emitter.o run-emitter.cmv -f .deps/run-emitter.Tpo .deps/run-emitter.Po
/bin/sh ../libtool --tag=CC   --mode=link gcc  -g -O2   -o run-emitter run-emitter.o ../src/libyaml.la
libtool: link: gcc -g -O2 -o .libs/run-emitter run-emitter.o  ../src/.libs/libyaml.so -Wl,-rpath -Wl,/usr/local/lib
gcc -DHAVE_CONFIG_H -I. -I..  -I../include   -g -O2 -MT run-dumper.o -MD -MP -MF .deps/run-dumper.Tpo -c -o run-dumper.o run-dumper.c
mv -f .deps/run-dumper.Tpo .deps/run-dumper.Po
/bin/sh ../libtool --tag=CC   --mode=link gcc  -g -O2   -o run-dumper run-dumper.o ../src/libyaml.la
libtool: link: gcc -g -O2 -o .libs/run-dumper run-dumper.o  ../src/.libs/libyaml.so -Wl,-rpath -Wl,/usr/local/lib
gcc -DHAVE_CONFIG_H -I. -I..  -I../include   -g -O2 -MT example-reformatter.o -MD -MP -MF .deps/example-reformatter.Tpo -c -o example-reformatter.o example-reformatter.c
mv -f .deps/example-reformatter.Tpo .deps/example-reformatter.Po
/bin/sh ../libtool --tag=CC   --mode=link gcc  -g -O2   -o example-reformatter example-reformatter.o ../src/libyaml.la
libtool: link: gcc -g -O2 -o .libs/example-reformatter example-reformatter.o  ../src/.libs/libyaml.so -Wl,-rpath -Wl,/usr/local/lib
gcc -DHAVE_CONFIG_H -I. -I..  -I../include   -g -O2 -MT example-reformatter-alt.o -MD -MP -MF .deps/example-reformatter-alt.Tpo -c -o example-reformatter-alt.o example-reformatter-alt.c
mv -f .deps/example-reformatter-alt.Tpo .deps/example-reformatter-alt.Po
/bin/sh ../libtool --tag=CC   --mode=link gcc  -g -O2   -o example-reformatter-alt example-reformatter-alt.o ../src/libyaml.la
libtool: link: gcc -g -O2 -o .libs/example-reformatter-alt example-reformatter-alt.o  ../src/.libs/libyaml.so -Wl,-rpath -Wl,/usr/local/lib
gcc -DHAVE_CONFIG_H -I. -I..  -I../include   -g -O2 -MT example-deconstructor.o -MD -MP -MF .deps/example-deconstructor.Tpo -c -o example-deconstructor.o example-deconstructor.c
mv -f .deps/example-deconstructor.Tpo .deps/example-deconstructor.Po
/bin/sh ../libtool --tag=CC   --mode=link gcc  -g -O2   -o example-deconstructor example-deconstructor.o ../src/libyaml.la
libtool: link: gcc -g -O2 -o .libs/example-deconstructor example-deconstructor.o  ../src/.libs/libyaml.so -Wl,-rpath -Wl,/usr/local/lib
gcc -DHAVE_CONFIG_H -I. -I..  -I../include   -g -O2 -MT example-deconstructor-alt.o -MD -MP -MF .deps/example-deconstructor-alt.Tpo -c -o example-deconstructor-alt.o example-deconstructor-alt.c
mv -f .deps/example-deconstructor-alt.Tpo .deps/example-deconstructor-alt.Po
/bin/sh ../libtool --tag=CC   --mode=link gcc  -g -O2   -o example-deconstructor-alt example-deconstructor-alt.o ../src/libyaml.la
libtool: link: gcc -g -O2 -o .libs/example-deconstructor-alt example-deconstructor-alt.o  ../src/.libs/libyaml.so -Wl,-rpath -Wl,/usr/local/lib
make[2]: Leaving directory `/software/yaml-0.1.4/tests'
Making all in win32
make[2]: Entering directory `/software/yaml-0.1.4/win32'
make[2]: Nothing to be done for `all'.
make[2]: Leaving directory `/software/yaml-0.1.4/win32'
make[1]: Leaving directory `/software/yaml-0.1.4'
Making install in include
make[1]: Entering directory `/software/yaml-0.1.4/include'
make[2]: Entering directory `/software/yaml-0.1.4/include'
make[2]: Nothing to be done for `install-exec-am'.
test -z "/usr/local/include" || /bin/mkdir -p "/usr/local/include"
/usr/bin/install -c -m 644  yaml.h '/usr/local/include/.'
make[2]: Leaving directory `/software/yaml-0.1.4/include'
make[1]: Leaving directory `/software/yaml-0.1.4/include'
Making install in src
make[1]: Entering directory `/software/yaml-0.1.4/src'
make[2]: Entering directory `/software/yaml-0.1.4/src'
test -z "/usr/local/lib" || /bin/mkdir -p "/usr/local/lib"
/bin/sh ../libtool   --mode=install /usr/bin/install -c   libyaml.la '/usr/local/lib'
libtool: install: /usr/bin/install -c .libs/libyaml-0.so.2.0.2 /usr/local/lib/libyaml-0.so.2.0.2
libtool: install: (cd /usr/local/lib && { ln -s -f libyaml-0.so.2.0.2 libyaml-0.so.2 || { rm -f libyaml-0.so.2 && ln -s libyaml-0.so.2.0.2 libyaml-0.so.2; }; })
libtool: install: (cd /usr/local/lib && { ln -s -f libyaml-0.so.2.0.2 libyaml.so || { rm -f libyaml.so && ln -s libyaml-0.so.2.0.2 libyaml.so; }; })
libtool: install: /usr/bin/install -c .libs/libyaml.lai /usr/local/lib/libyaml.la
libtool: install: /usr/bin/install -c .libs/libyaml.a /usr/local/lib/libyaml.a
libtool: install: chmod 644 /usr/local/lib/libyaml.a
libtool: install: ranlib /usr/local/lib/libyaml.a
libtool: finish: PATH="/usr/lib64/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin:/sbin" ldconfig -n /usr/local/lib
----------------------------------------------------------------------
Libraries have been installed in:
   /usr/local/lib

If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the `-LLIBDIR'
flag during linking and do at least one of the following:
   - add LIBDIR to the `LD_LIBRARY_PATH' environment variable
     during execution
   - add LIBDIR to the `LD_RUN_PATH' environment variable
     during linking
   - use the `-Wl,-rpath -Wl,LIBDIR' linker flag
   - have your system administrator add LIBDIR to `/etc/ld.so.conf'

See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.
----------------------------------------------------------------------
make[2]: Nothing to be done for `install-data-am'.
make[2]: Leaving directory `/software/yaml-0.1.4/src'
make[1]: Leaving directory `/software/yaml-0.1.4/src'
Making install in .
make[1]: Entering directory `/software/yaml-0.1.4'
make[2]: Entering directory `/software/yaml-0.1.4'
make[2]: Nothing to be done for `install-exec-am'.
test -z "/usr/local/lib/pkgconfig" || /bin/mkdir -p "/usr/local/lib/pkgconfig"
/usr/bin/install -c -m 644 yaml-0.1.pc '/usr/local/lib/pkgconfig'
make[2]: Leaving directory `/software/yaml-0.1.4'
make[1]: Leaving directory `/software/yaml-0.1.4'
Making install in tests
make[1]: Entering directory `/software/yaml-0.1.4/tests'
make[2]: Entering directory `/software/yaml-0.1.4/tests'
make[2]: Nothing to be done for `install-exec-am'.
make[2]: Nothing to be done for `install-data-am'.
make[2]: Leaving directory `/software/yaml-0.1.4/tests'
make[1]: Leaving directory `/software/yaml-0.1.4/tests'
Making install in win32
make[1]: Entering directory `/software/yaml-0.1.4/win32'
make[2]: Entering directory `/software/yaml-0.1.4/win32'
make[2]: Nothing to be done for `install-exec-am'.
make[2]: Nothing to be done for `install-data-am'.
make[2]: Leaving directory `/software/yaml-0.1.4/win32'
make[1]: Leaving directory `/software/yaml-0.1.4/win32'
[root@localhost yaml-0.1.4]#


1-7)[root@localhost software]# unzip greenplum-loaders-4.3.8.1-build-1-RHEL5-x86_64.zip
Archive:  greenplum-loaders-4.3.8.1-build-1-RHEL5-x86_64.zip
  inflating: greenplum-loaders-4.3.8.1-build-1-RHEL5-x86_64.bin 


1-8)[root@localhost software]# ./greenplum-loaders-4.3.8.1-build-1-RHEL5-x86_64.bin

....

*******************************************************************************
Install Greenplum Loaders into </usr/local/greenplum-loaders-4.3.8.1-build-1>? [yes | no]
********************************************************************************

yes

********************************************************************************
/usr/local/greenplum-loaders-4.3.8.1-build-1 does not exist.
Create /usr/local/greenplum-loaders-4.3.8.1-build-1 ? [ yes | no ]
(Selecting no will exit the installer)
********************************************************************************

yes

Extracting product to /usr/local/greenplum-loaders-4.3.8.1-build-1


********************************************************************************
Installation complete.
Greenplum Loaders is installed in:

  /usr/local/greenplum-loaders-4.3.8.1-build-1

Pivotal Greenplum documentation is available
for download at http://docs.gopivotal.com/gpdb

1-9)[root@localhost greenplum-loaders-4.3.8.1-build-1]# cat greenplum_loaders_pa
th.sh if [ `uname -s` = "HP-UX" ]; then
    set +o nounset
fi

GPHOME_LOADERS=/usr/local/greenplum-loaders-4.3.8.1-build-1
PATH=${GPHOME_LOADERS}/bin:${GPHOME_LOADERS}/ext/python/bin:${PATH}
PYTHONPATH=${GPHOME_LOADERS}/bin/ext:${PYTHONPATH}

export GPHOME_LOADERS
export PATH
export PYTHONPATH

# Mac OSX uses a different library path variable
if [ xDarwin = x`uname -s` ]; then
  DYLD_LIBRARY_PATH=${GPHOME_LOADERS}/lib:${GPHOME_LOADERS}/ext/python/lib:$
{DYLD_LIBRARY_PATH}  export DYLD_LIBRARY_PATH
else
  LD_LIBRARY_PATH=${GPHOME_LOADERS}/lib:${GPHOME_LOADERS}/ext/python/lib:${L
D_LIBRARY_PATH}  export LD_LIBRARY_PATH
fi

# AIX uses yet another library path variable
# Also, Python on AIX requires special copies of some libraries.  Hence, lib
/pware.if [ xAIX = x`uname -s` ]; then
  LIBPATH=${GPHOME_LOADERS}/lib/pware:${GPHOME_LOADERS}/lib:${GPHOME_LOADERS
}/ext/python/lib:/usr/lib/threads:${LIBPATH}  export LIBPATH
  GP_LIBPATH_FOR_PYTHON=${GPHOME_LOADERS}/lib/pware
  export GP_LIBPATH_FOR_PYTHON
fi

if [ "$1" != "-q" ]; then
  type python >/dev/null 2>&1
  if [ $? -ne 0 ]; then
    echo "Warning: Python not found.  Python-2.5.1 or better is required to
run gpload."  fi
fi
[root@localhost greenplum-loaders-4.3.8.1-build-1]#

如果第8步中都默认yes,那么不需要修改文件


1-10)[root@localhost ~]# vi .bash_profile


# .bash_profile

# Get the aliases and functions
if [ -f ~/.bashrc ]; then
        . ~/.bashrc
fi

# User specific environment and startup programs

PATH=$PATH:$HOME/bin

export PATH
#首先你的数据库,地址,端口,用户
export PGDATABASE=tutorial
export PGHOST=master
export PGPORT=5432
export PGUSER=user1
export PGPASSWORD=user1




1-11)[root@localhost ~]# source ~/.bash_profile


2、启动gpfdsit

1)[root@localhost bin]#  nohup /usr/local/greenplum-loaders-4.3.8.1-build-1/bin/gpfdist -d /home/admin/ -p 8888 > /tmp/gpfdist.log 2>&1 &
[1] 5474
[root@localhost bin]# cat /tmp/gpfdist.log
nohup: ignoring input
/usr/local/greenplum-loaders-4.3.8.1-build-1/bin/gpfdist: error while loading shared libraries: libyaml-0.so.1: cannot open shared object file: No such file or directory(《https://discuss.pivotal.io/hc/zh-cn/community/posts/206424348-Error-Message-gpfdist-error-while-loading-shared-libraries-libyaml-0-so-1》)
2)执行[root@localhost greenplum-loaders-4.3.8.1-build-1]# source greenplum_loaders_path.sh 

3)再次执行
[root@localhost greenplum-loaders-4.3.8.1-build-1]# nohup /usr/local/greenpl
um-loaders-4.3.8.1-build-1/bin/gpfdist -d /home/admin/ -p 8888 > /tmp/gpfdist.log 2>&1 &[1] 5535
[root@localhost greenplum-loaders-4.3.8.1-build-1]# cat /tmp/gpfdist.log
nohup: ignoring input
2016-12-14 23:57:28 5535 INFO Before opening listening sockets - following l
istening sockets are available:2016-12-14 23:57:28 5535 INFO IPV6 socket: [::]:8888
2016-12-14 23:57:28 5535 INFO IPV4 socket: 0.0.0.0:8888
2016-12-14 23:57:28 5535 INFO Trying to open listening socket:
2016-12-14 23:57:28 5535 INFO IPV6 socket: [::]:8888
2016-12-14 23:57:28 5535 INFO Opening listening socket succeeded
2016-12-14 23:57:28 5535 INFO Trying to open listening socket:
2016-12-14 23:57:28 5535 INFO IPV4 socket: 0.0.0.0:8888
Serving HTTP on port 8888, directory /home/admin

3、加载数据

[root@kettle bin]# gpload -f my_load.yml
2016-12-15 04:05:15|INFO|gpload session started 2016-12-15 04:05:15
2016-12-15 04:05:15|INFO|started gpfdist -p 8888 -P 8889 -f "/home/admin/gpe
xtdata/c.txt" -t 302016-12-15 04:05:17|ERROR|ERROR:  connection with gpfdist failed for gpfdist
://192.168.1.101:8889//home/admin/gpextdata/c.txt. effective url: http://192.168.1.101:8889//home/admin/gpextdata/c.txt. error code = 113 (No route to host(http://blog.sina.com.cn/s/blog_9151e7300101ksui.html))  (seg0 slice1 slave1:40000 pid=2566) encountered while running INSERT INTO "public"."test001" ("id","name") SELE
CT "id","name" FROM ext_gpload_97a037fe_c2a5_11e6_bb0b_08002741af212016-12-15 04:05:17|INFO|rows Inserted          = 0
2016-12-15 04:05:17|INFO|rows Updated           = 0
2016-12-15 04:05:17|INFO|data formatting errors = 0
2016-12-15 04:05:17|INFO|gpload failed

--关闭防火墙

[root@kettle bin]# gpload -f /home/admin/gpextdata/abc.yml
2016-12-15 04:34:29|INFO|gpload session started 2016-12-15 04:34:29
2016-12-15 04:34:29|INFO|started gpfdist -p 8877 -P 8878 -f "/home/admin/gpextdata/c.txt" -t 302016-12-15 04:34:30|INFO|running time: 0.31 seconds
2016-12-15 04:34:30|INFO|rows Inserted          = 2
2016-12-15 04:34:30|INFO|rows Updated           = 0
2016-12-15 04:34:30|INFO|data formatting errors = 0
2016-12-15 04:34:30|INFO|gpload succeeded
[root@kettle bin]#


[root@kettle bin]# cat abc.yml
VERSION: 1.0.0.1
DATABASE: tutorial
USER: user1
HOST: 192.168.1.80
PORT: 5432
GPLOAD:
    INPUT:
    - SOURCE:
        LOCAL_HOSTNAME:
         - 192.168.1.101
        PORT: 8888
        FILE: ['/home/admin/gpextdata/c.txt']
    - COLUMNS:
         - id:
         - "name":
    - FORMAT: TEXT
    - DELIMITER: ','
    - QUOTE: ''
    - HEADER: FALSE
    - ENCODING: UTF8
    - ERROR_LIMIT: 50
    OUTPUT:
    - TABLE: "public.test001"
    - MODE: insert

4、kettle调用

X9fgAAAAASUVORK5CYII=


2016/12/15 05:00:25 - Text file input.0 - Opening file: file:///home/admin/gpextdata/b.txt
2016/12/15 05:00:25 - Text file input.0 - Finished processing (I=7, O=0, R=0, W=7, U=1, E=0)
2016/12/15 05:00:25 - Greenplum Load.0 - Executing: /usr/local/greenplum-loaders-4.3.8.1-build-1/bin/gpload -f /home/admin/gpextdata/my_flag.yml
2016/12/15 05:00:25 - Greenplum Load.0 - Exit Value of psql: 127
2016/12/15 05:00:25 - Greenplum Load.0 - ERROR (version 5.4.0.1-130, build 1 from 2015-06-14_12-34-55 by buildguy) : Error in step, asking everyone to stop because of:
2016/12/15 05:00:25 - Greenplum Load.0 - Exit Value of psql: 127
2016/12/15 05:00:25 - Greenplum Load.0 - Finished processing (I=0, O=7, R=7, W=7, U=0, E=1)
2016/12/15 05:00:25 - Greenplum Load.0 - ERROR>/usr/local/greenplum-loaders-4.3.8.1-build-1/bin/gpload: line 14: exec: gpload.py: not found
2016/12/15 05:00:25 - greepload - ERROR (version 5.4.0.1-130, build 1 from 2015-06-14_12-34-55 by buildguy) : Errors detected!
2016/12/15 05:00:25 - Spoon - The transformation has finished!!

现在到了这一步更不能放弃

1)将主机的bin目录其他文件拷贝过来,重新执行报错

2)去主机中执行exec gpload.py

[gpadmin@master bin]$ exec gpload.py
COMMAND NAME: gpload

Runs a load job as defined in a YAML formatted control file.


*****************************************************
SYNOPSIS
*****************************************************

gpload -f <control_file> [-l <log_file>]
[-h <hostname>] [-p <port>] [-U <username>] [-d <database>] [-W]
[--gpfdist_timeout <seconds>] [[-v | -V] | [-q]] [-D]

gpload -?

gpload --version


*****************************************************
PREREQUISITES
*****************************************************

The client machine where gpload is executed must have the following:

* Python 2.6.2 or later, pygresql (the Python interface to PostgreSQL),
  and pyyaml. Note that Python and the required Python libraries
  are included with the Greenplum Database server installation, so if
  you have Greenplum Database installed on the machine where gpload
  is running, you do not need a separate Python installation.

  Note: Greenplum Loaders for Windows supports only Python 2.5
  (available from www.python.org).

* The gpfdist parallel file distribution program installed and in
  your $PATH. This program is located in $GPHOME/bin of your
  Greenplum Database server installation.

* Network access to and from all hosts in your Greenplum Database
  array (master and segments).

* Network access to and from the hosts where the data to be loaded
  resides (ETL servers).


*****************************************************
DESCRIPTION
*****************************************************

gpload is a data loading utility that acts as an interface to
Greenplum Database's parallel loading feature.
Using a load specification defined in a YAML formatted control file,
gpload executes a load by invoking the Greenplum parallel file
server (gpfdist), creating an external table definition based on
the source data defined, and executing an INSERT, UPDATE or MERGE
operation to load the source data into the target table in the
database.


*****************************************************
OPTIONS
*****************************************************

...
*****************************************************
SEE ALSO
*****************************************************


gpfdist, CREATE EXTERNAL TABLE

ERROR: configuration file required

[root@master ~]#

那么就是某些变量没有配置或者没有配对



重新查看官网文档配置环境路径

Configuring the Command-Line Load Tools

  
  •  
  •  
  •   

   










AAMjv8B6j5gfkbfbs0AAAAASUVORK5CYII=









>
>
>
>     
>     
>
>












版权声明:本文内容由阿里云实名注册用户自发贡献,版权归原作者所有,阿里云开发者社区不拥有其著作权,亦不承担相应法律责任。具体规则请查看《阿里云开发者社区用户服务协议》和《阿里云开发者社区知识产权保护指引》。如果您发现本社区中有涉嫌抄袭的内容,填写侵权投诉表单进行举报,一经查实,本社区将立刻删除涉嫌侵权内容。

相关文章
LINUX下载编译libpq(postgresql)
LINUX下载编译libpq(postgresql)
0 0
Linux下Azkaban安装
说明: CentOS 7JDK 1.8官方文档:https://azkaban.readthedocs.io/en/latest/getStarted.html#building-from-source 安装步骤 Clone the repo: git clone https://github.
5328 0
linux下为Hive安装mysql
1、安装了mysql server 查看是否安装rpm -qa|grep mysql 卸载 rpm -e --nodeps mysql-libs-5.
1397 0
HDFS的linux下开发环境准备
上传工具elicpse和maven 安装maven 解压命令: unzip apache-maven-3.0.5-bin.zip -d /opt/modules/ 配置maven环境变量 vi /opt/profile ...
688 0
oracle 同步数据 greenplum linux kettle
gh 加油 最近公司在做项目时,使用oracle采集底层数据,使用greenplum分析加工数据,数据交换使用的是kettle。
2785 0
greenplum install for linux
gh 加油   greenplum install for linux 1、实验环境 硬件环境 ORACLE virtualbox(5.
791 0
Linux下图形数据库Neo4j单机安装
Neo4j数据库简介   Neo4j 是一个NoSQL的图形数据库(Graph Database)。Neo4j使用图(graph)相关的概念来描述数据模型,把数据保存为图中的节点以及节点之间的关系。
2509 0
+关注
樱桃味
一个要努力的人
文章
问答
文章排行榜
最热
最新
相关电子书
更多
《3-Alibaba Dragonwell 在Alibaba Cloud Linux 3上的应用及优化》
立即下载
Decian GNU/Linux安全合规之路
立即下载
Alibaba Cloud Linux 3 发布
立即下载