一、Anaconda3
Anaconda，中文大蟒蛇，是一个开源的Python发行版本，其包含了conda、Python等180多个科学包及其依赖项。我当初安装这个是为了学习人工智能和算法。
1、安装
1、命令操作
安装好之后，可以使用命令conda，可以使用一系列命令参数，conda --help 或 conda -h 、 conda --version 或conda -V
例如，使用conda info命令如下所示信息：
$xxx:~ xxx$ conda info active environment : None shell level : 0 user config file : /Users/xxx/.condarc populated config files : /Users/xxx/.condarc conda version : 4.8.2 conda-build version : 3.18.11 python version : 3.7.6.final.0 virtual packages : __osx=10.15.7 base environment : /Users/xxx/opt/anaconda3 (writable) channel URLs : https://repo.anaconda.com/pkgs/main/osx-64 https://repo.anaconda.com/pkgs/main/noarch https://repo.anaconda.com/pkgs/r/osx-64 https://repo.anaconda.com/pkgs/r/noarch package cache : /Users/xxx/opt/anaconda3/pkgs /Users/xxx/.conda/pkgs envs directories : /Users/xxx/opt/anaconda3/envs /Users/xxx/.conda/envs platform : osx-64 user-agent : conda/4.8.2 requests/2.22.0 CPython/3.7.6 Darwin/19.6.0 OSX/10.15.7 UID:GID : 501:20 netrc file : None offline mode : False
2、导出所有存在依赖
1）导出所有依赖包到记事本
conda list -e > [fileName(.txt)]
例如，conda list -e > /usr/local/data/conda-lib-98.txt
1、Kafka 集群防火墙
2、运行情况，或者执行命令，查看status
3、手动执行消费者或生产者，订阅消息，查看消费情况
二、问题记录FQA
Q1、ModuleNotFoundError: No module named 'pymysql'
Q1：执行python程序，出现 ``ModuleNotFoundError: No module named 'pymysql'
A1：
1）查找确认依赖库
初步判定是安装阿naconda3缺少pymysql依赖包，执行命令`` conda list `` 分别查看85服务器和96服务器anaconda3的已经安装插件库，依赖包如下所示，85服务器已安装插件库295个插件，96服务器已安装插件库288个插件
2）查找、下载依赖库
85插件所在路径， /data/_software/_install_pack/_python/，anaconda3的home路径，/data/_software/python/anaconda3/
86插件所在路径，， /data/_software/_pack/_python/，anaconda3的home路径，/data/_software/python/anaconda3/
96插件所在路径，/data/_software/_package/，anaconda3的home路径，/data/_software/python/anaconda3/
经过插件列表比对并排查，发现96服务器缺失插件列表如下所示
85服务器有whl插件源件：
- PyNaCl-1.3.0-cp34-abi3-manylinux1_x86_64.whl
- bcrypt-3.1.7-cp34-abi3-manylinux1_x86_64.whl
- confluent_kafka-1.1.0-cp37-cp37m-manylinux1_x86_64.whl
- paramiko-2.6.0-py2.py3-none-any.whl
- PyMySQL-0.9.3-py2.py3-none-any.whl
- cx_Oracle-8.0.0-cp37-cp37m-manylinux1_x86_64.whl
85服务器未找到whl插件源件（其中avro-python3使用python安装，cx-oracle使用pip安装）：
- avro-python3=1.8.2=pypi_0
- cx-oracle=8.0.0=pypi_0
先下载所需要的插件包，whl格式本质上是一个压缩包，里面包含了py文件，以及经过编译的pyd文件，使得可以在不具备编译环境的情况下，选择合适自己的python环境进行安装，简单来说，适合内网环境。
其次有些时候在内网环境中并不可以使用pip命令来下载依赖包，这就出现了需要手动安装依赖包的情况，由于网络不通，采取手动安装插件库，查找85服务器是否存在插件库或网络下载插件库再传给96服务器。经过检查，发现原来85服务器已经下载插件。在85服务器找到所依赖的插件，打包上传到96服务器。
3）安装依赖库
安装插件的命令使用pip，此命令在已经内置在anaconda3的依赖插件中，具体位置为home/lib/python3.7/site-packages。
- 安装已有whl文件
安装bcrypt-3.1.7-cp34-abi3-manylinux1_x86_64.whl
$ pip install bcrypt-3.1.7-cp34-abi3-manylinux1_x86_64.whl Processing ./bcrypt-3.1.7-cp34-abi3-manylinux1_x86_64.whl Requirement already satisfied: cffi>=1.1 in /data/_software/python/anaconda3/lib/python3.7/site-packages (from bcrypt==3.1.7) (1.12.3) Requirement already satisfied: six>=1.4.1 in /data/_software/python/anaconda3/lib/python3.7/site-packages (from bcrypt==3.1.7) (1.12.0) Requirement already satisfied: pycparser in /data/_software/python/anaconda3/lib/python3.7/site-packages (from cffi>=1.1->bcrypt==3.1.7) (2.19) Installing collected packages: bcrypt Successfully installed bcrypt-3.1.7 $ pip install confluent_kafka-1.1.0-cp37-cp37m-manylinux1_x86_64.whl Processing ./confluent_kafka-1.1.0-cp37-cp37m-manylinux1_x86_64.whl Installing collected packages: confluent-kafka Successfully installed confluent-kafka-1.1.0
安装PyNaCl-1.3.0-cp34-abi3-manylinux1_x86_64.whl
先安装PyNaCl，再安装paramiko
$ pip install PyNaCl-1.3.0-cp34-abi3-manylinux1_x86_64.whl Processing ./PyNaCl-1.3.0-cp34-abi3-manylinux1_x86_64.whl Requirement already satisfied: cffi>=1.4.1 in /data/_software/python/anaconda3/lib/python3.7/site-packages (from PyNaCl==1.3.0) (1.12.3) Requirement already satisfied: six in /data/_software/python/anaconda3/lib/python3.7/site-packages (from PyNaCl==1.3.0) (1.12.0) Requirement already satisfied: pycparser in /data/_software/python/anaconda3/lib/python3.7/site-packages (from cffi>=1.4.1->PyNaCl==1.3.0) (2.19) Installing collected packages: PyNaCl Successfully installed PyNaCl-1.3.0
安装paramiko-2.6.0-py2.py3-none-any.whl
$ pip install paramiko-2.6.0-py2.py3-none-any.whl Processing ./paramiko-2.6.0-py2.py3-none-any.whl Requirement already satisfied: bcrypt>=3.1.3 in /data/_software/python/anaconda3/lib/python3.7/site-packages (from paramiko==2.6.0) (3.1.7) Requirement already satisfied: pynacl>=1.0.1 in /data/_software/python/anaconda3/lib/python3.7/site-packages (from paramiko==2.6.0) (1.3.0) Requirement already satisfied: cryptography>=2.5 in /data/_software/python/anaconda3/lib/python3.7/site-packages (from paramiko==2.6.0) (2.7) Requirement already satisfied: six>=1.4.1 in /data/_software/python/anaconda3/lib/python3.7/site-packages (from bcrypt>=3.1.3->paramiko==2.6.0) (1.12.0) Requirement already satisfied: cffi>=1.1 in /data/_software/python/anaconda3/lib/python3.7/site-packages (from bcrypt>=3.1.3->paramiko==2.6.0) (1.12.3) Requirement already satisfied: asn1crypto>=0.21.0 in /data/_software/python/anaconda3/lib/python3.7/site-packages (from cryptography>=2.5->paramiko==2.6.0) (0.24.0) Requirement already satisfied: pycparser in /data/_software/python/anaconda3/lib/python3.7/site-packages (from cffi>=1.1->bcrypt>=3.1.3->paramiko==2.6.0) (2.19) Installing collected packages: paramiko Successfully installed paramiko-2.6.0
安装PyMySQL-0.9.3-py2.py3-none-any.whl
$ pip install PyMySQL-0.9.3-py2.py3-none-any.whl Processing ./PyMySQL-0.9.3-py2.py3-none-any.whl Installing collected packages: PyMySQL Successfully installed PyMySQL-0.9.3
安装cx_Oracle-8.0.0-cp37-cp37m-manylinux1_x86_64.whl
$ pip install cx_Oracle-8.0.0-cp37-cp37m-manylinux1_x86_64.whl Processing ./cx_Oracle-8.0.0-cp37-cp37m-manylinux1_x86_64.whl Installing collected packages: cx-Oracle Successfully installed cx-Oracle-8.0.0
安装未有whl文件的tar.gz文件，进入到arvo_python3所在目录
cd /data/_software/_package
解压arvo_python3-1.8.2.tar.gz，会自动创建并解压到arvo_python3-1.8.2
tar -zxvf arvo_python3-1.8.2.tar.gz
进入arvo_python3-1.8.2包
cd arvo_python3-1.8.2
执行命令python setup.py install
$ python setup.py install running install running bdist_egg running egg_info writing avro_python3.egg-info/PKG-INFO writing dependency_links to avro_python3.egg-info/dependency_links.txt writing top-level names to avro_python3.egg-info/top_level.txt reading manifest file 'avro_python3.egg-info/SOURCES.txt'writing manifest file 'avro_python3.egg-info/SOURCES.txt'installing library code to build/bdist.linux-x86_64/egg running install_lib running build_py creating build creating build/lib creating build/lib/avro copying avro/__init__.py -> build/lib/avro copying avro/datafile.py -> build/lib/avro copying avro/io.py -> build/lib/avro copying avro/ipc.py -> build/lib/avro copying avro/protocol.py -> build/lib/avro copying avro/schema.py -> build/lib/avro copying avro/tool.py -> build/lib/avro copying avro/txipc.py -> build/lib/avro copying avro/HandshakeRequest.avsc -> build/lib/avro copying avro/HandshakeResponse.avsc -> build/lib/avro copying avro/VERSION.txt -> build/lib/avro copying avro/LICENSE -> build/lib/avro copying avro/NOTICE -> build/lib/avro creating build/bdist.linux-x86_64 creating build/bdist.linux-x86_64/egg creating build/bdist.linux-x86_64/egg/avro copying build/lib/avro/__init__.py -> build/bdist.linux-x86_64/egg/avro copying build/lib/avro/datafile.py -> build/bdist.linux-x86_64/egg/avro copying build/lib/avro/io.py -> build/bdist.linux-x86_64/egg/avro copying build/lib/avro/ipc.py -> build/bdist.linux-x86_64/egg/avro copying build/lib/avro/protocol.py -> build/bdist.linux-x86_64/egg/avro copying build/lib/avro/schema.py -> build/bdist.linux-x86_64/egg/avro copying build/lib/avro/tool.py -> build/bdist.linux-x86_64/egg/avro copying build/lib/avro/txipc.py -> build/bdist.linux-x86_64/egg/avro copying build/lib/avro/HandshakeRequest.avsc -> build/bdist.linux-x86_64/egg/avro copying build/lib/avro/HandshakeResponse.avsc -> build/bdist.linux-x86_64/egg/avro copying build/lib/avro/VERSION.txt -> build/bdist.linux-x86_64/egg/avro copying build/lib/avro/LICENSE -> build/bdist.linux-x86_64/egg/avro copying build/lib/avro/NOTICE -> build/bdist.linux-x86_64/egg/avro byte-compiling build/bdist.linux-x86_64/egg/avro/__init__.py to __init__.cpython-37.pyc byte-compiling build/bdist.linux-x86_64/egg/avro/datafile.py to datafile.cpython-37.pyc byte-compiling build/bdist.linux-x86_64/egg/avro/io.py to io.cpython-37.pyc byte-compiling build/bdist.linux-x86_64/egg/avro/ipc.py to ipc.cpython-37.pyc byte-compiling build/bdist.linux-x86_64/egg/avro/protocol.py to protocol.cpython-37.pyc byte-compiling build/bdist.linux-x86_64/egg/avro/schema.py to schema.cpython-37.pyc byte-compiling build/bdist.linux-x86_64/egg/avro/tool.py to tool.cpython-37.pyc byte-compiling build/bdist.linux-x86_64/egg/avro/txipc.py to txipc.cpython-37.pyc creating build/bdist.linux-x86_64/egg/EGG-INFO installing scripts to build/bdist.linux-x86_64/egg/EGG-INFO/scripts running install_scripts running build_scripts creating build/scripts-3.7 copying and adjusting scripts/avro -> build/scripts-3.7 changing mode of build/scripts-3.7/avro from 600 to 755creating build/bdist.linux-x86_64/egg/EGG-INFO/scripts copying build/scripts-3.7/avro -> build/bdist.linux-x86_64/egg/EGG-INFO/scripts changing mode of build/bdist.linux-x86_64/egg/EGG-INFO/scripts/avro to 755copying avro_python3.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO copying avro_python3.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying avro_python3.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying avro_python3.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO zip_safe flag not set; analyzing archive contents... avro.__pycache__.__init__.cpython-37: module references __file__ avro.__pycache__.ipc.cpython-37: module references __file__ creating dist creating 'dist/avro_python3-1.8.2-py3.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it removing 'build/bdist.linux-x86_64/egg' (and everything under it) Processing avro_python3-1.8.2-py3.7.egg creating /data/_software/python/anaconda3/lib/python3.7/site-packages/avro_python3-1.8.2-py3.7.egg Extracting avro_python3-1.8.2-py3.7.egg to /data/_software/python/anaconda3/lib/python3.7/site-packages Adding avro-python3 1.8.2 to easy-install.pth file Installing avro script to /data/_software/python/anaconda3/bin Installed /data/_software/python/anaconda3/lib/python3.7/site-packages/avro_python3-1.8.2-py3.7.egg Processing dependencies foravro-python3==1.8.2 Finished processing dependencies foravro-python3==1.8.2
4）检查确认依赖库
执行安装新插件后，96服务器插件列表为295个插件
执行命令conda list -e > [输出插件列表到文件.txt]
Q2、sshpass命令没有找到
Q2：执行python程序，提示sshpass命令没有找到
A2：
1）下载 sshpass-1.06.tar.gz
有网络情况下，执行命令
wget http://sourceforge.net/projects/sshpass/files/latest/download -O sshpass.tar.gz
2）解压
解压sshpass-1.06.tar.gz到当前目录（自动创建目录并写入解压文件）
tar -zxvf sshpass-1.06.tar.gz
3）编译
进入sshpass-1.06目录，进行编译
./configure
4）安装
执行安装命令
sudomake install
5）其他参考
https://linux.cn/article-8086-1.html
Q3、No such file or directory
Q3、执行python程序，控制台打印出来以下错误信息
Traceback (most recent call last): File "test_oracle_lyc.py", line 4, in <module> db = cx_Oracle.connect("xxx.xxx.xxx.xxx", "username", "password", "orclName", charset='utf8' ) cx_Oracle.DatabaseError: DPI-1047: Cannot locate a 64-bit Oracle Client library: "libnsl.so.1: cannot open shared object file: No such file or directory". See https://cx-oracle.readthedocs.io/en/latest/user_guide/installation.html for help
A3：
1）python、oracle依赖包
经过排查，发现缺少oracle_client，安装oracle-instantclient19.3-basic-19.3.0.0.0-1.x86_64.rpm
2）安装
使用root用户执行安装命令，安装到默认路径
rpm -Uvh oracle-instantclient19.3-basic-19.3.0.0.0-1.x86_64.rpm
3）环境变量
使用root用户登录进去配置相关环境变量（这个好像不用配置也可以，我在官方网站看到18+以下数据库客户端才需要配置环境变量，以及手动配置lib库地址和bin目录，现在我们使用的版本为19，已经自动创建libaio以及其他依赖包）。vim /etc/profile，输入环境变量信息
exportORACLE_HOME=/usr/lib/oracle/19.3/client64 exportORABIN=/usr/lib/oracle/19.3/client64/bin exportLD_LIBRARY_PATH=/usr/lib/oracle/19.3/client64/lib exportPATH=$PATH:$ORACLE_HOME/bin
让环境变量立即生效
source /etc/profile
4）安装系统依赖包
在root用户下，安装libnsl.so.1，进入到目录查看系统依赖包，由于我们都是使用的64bit操作系统，所有的软件工具也都是64bit
cd /usr/lib64
查看（ls）是否有libnsl依赖包，在85服务器上，我查询到libnsl* 的依赖包为
libnsl.so.2 libnsl.so.2.0.0
查看（ll）是否有libnsl依赖包，在85服务器上，我查询到libnsl* 的依赖包为
lrwxrwxrwx 1 root root 15 Mar 152020 libnsl.so.2 -> libnsl.so.2.0.0 -rwxr-xr-x1 root root 96480 Mar 152020 libnsl.so.2.0.0
然后进行拷贝，软连接设置
cp-r libnsl.so.2 libnsl.so.1
再次执行查看操作
lrwxrwxrwx 1 root root 15 Jan 2316:40 libnsl.so.1 -> libnsl.so.2.0.0 lrwxrwxrwx 1 root root 15 Mar 152020 libnsl.so.2 -> libnsl.so.2.0.0 -rwxr-xr-x1 root root 96480 Mar 152020 libnsl.so.2.0.0
5）切换测试
新打开一个窗口，使用appuser登录，或者重新登录到服务器，然后编写一个连接oracle的测试脚本，脚本如下
import cx_Oracle db = connect("userName", "password", "IP:Port/serviceName", "UTF-8") print(db.version)
保存代码为 test_oracle.py
执行python test_oracle.py，如果打印出来数据库服务器的版本号，则说明已经连接并成功读取数据。
