
bio2coder
conda channelswhat is conda channels ?Conda channels are the locations where packages are stored. They serve as the base for hosting and managing packages. Conda packages are downloaded from remote channels, which are URLs to directories containing conda packagesset up channelsThe conda-forge channel contains many general-purpose packages not already found in the defaults channel. The conda-forge is highest priority. use following commands to add channels :$ conda config --add channels defaults $ conda config --add channels bioconda $ conda config --add channels conda-forge (base) [biocodee@localhost ~]$ conda config --show channels channels: - conda-forge - bioconda - defaults #order matters, use following cmd to make sure `conda-forge` highest priority, then `bioconda` next (base) [biocodee@localhost ~]$ conda config --add channels bioconda (base) [biocodee@localhost ~]$ conda config --add channels conda-forge (base) [biocodee@localhost ~]$ conda config --show channels channels: - conda-forge - bioconda - defaults (base) [biocodee@localhost ~]$ conda config --prepend channels bioconda Warning: 'bioconda' already in 'channels' list, moving to the top (base) [biocodee@localhost ~]$ conda config --show channels channels: - bioconda - conda-forge - defaults (base) [biocodee@localhost ~]$ conda config --prepend channels conda-forge Warning: 'conda-forge' already in 'channels' list, moving to the top (base) [biocodee@localhost ~]$ conda config --show channels channels: - conda-forge - bioconda - defaultsreference : [1] Conda channels. website
how to check sha256sum value consistent with the reference value of the files downloaded( or copied) from somewhere ?1. usage of the command sha256sumUsage: sha256sum [OPTION]... [FILE]... Print or check SHA256 (256-bit) checksums. With no FILE, or when FILE is -, read standard input. -b, --binary read in binary mode -c, --check read SHA256 sums from the FILEs and check them --tag create a BSD-style checksum -t, --text read in text mode (default) Note: There is no difference between binary and text mode option on GNU system. The following four options are useful only when verifying checksums: --quiet don't print OK for each successfully verified file --status don't output anything, status code shows success --strict exit non-zero for improperly formatted checksum lines -w, --warn warn about improperly formatted checksum lines --help display this help and exit --version output version information and exit The sums are computed as described in FIPS-180-2. When checking, the input should be a former output of this program. The default mode is to print a line with checksum, a character indicating input mode ('*' for binary, space for text), and name for each FILE. GNU coreutils online help: <http://www.gnu.org/software/coreutils/> For complete documentation, run: info coreutils 'sha256sum invocation'2. An Example[biocodee@localhost Downloads]$ ls -al total 136640 drwxr-xr-x. 2 biocodee biocodee 231 Dec 25 09:44 . drwx------. 16 biocodee biocodee 4096 Dec 25 07:53 .. -rw-r--r--. 1 biocodee biocodee 40212 Dec 20 11:16 container-selinux-2.107-3.el7.noarch.rpm -rw-r--r--. 1 biocodee biocodee 36243572 Dec 20 10:55 docker-ce-18.03.0.ce-1.el7.centos.x86_64.rpm -rw-rw-r--. 1 biocodee biocodee 103553755 Dec 25 07:12 Miniconda3-py38_4.10.3-Linux-x86_64.sh -rw-rw-r--. 1 biocodee biocodee 65 Dec 25 07:53 Miniconda3-py38_4.10.3.sha256 -rw-r--r--. 1 biocodee biocodee 69596 Dec 20 11:23 pigz-2.3.3-1.el7.centos.x86_64.rpm [biocodee@localhost Downloads]$ echo "$(cat Miniconda3-py38_4.10.3.sha256) Miniconda3-py38_4.10.3-Linux-x86_64.sh" | sha256sum --check Miniconda3-py38_4.10.3-Linux-x86_64.sh: OK [biocodee@localhost Downloads]$ echo "$(cat Miniconda3-py38_4.10.3.sha256) container-selinux-2.107-3.el7.noarch.rpm" | sha256sum --check container-selinux-2.107-3.el7.noarch.rpm: FAILED sha256sum: WARNING: 1 computed checksum did NOT match [biocodee@localhost Downloads]$ cat Miniconda3-py38_4.10.3.sha256 935d72deb16e42739d69644977290395561b7a6db059b316958d97939e9bdf3d [biocodee@localhost Downloads]$ echo "935d72deb16e42739d69644977290395561b7a6db059b316958d97939e9bdf3d Miniconda3-py38_4.10.3-Linux-x86_64.sh" | sha256sum --check Miniconda3-py38_4.10.3-Linux-x86_64.sh: OK
以非根用户(non-root)管理 DockerDocker 守护进程(daemon)绑定到 Unix 套接字(Unix socket)而不是 TCP 端口。 默认情况下,Unix 套接字(Unix socket)由用户 root 拥有,其他用户只能使用 sudo 访问它。 Docker 守护进程始终以 root 用户身份运行。如果您不想在 docker 命令前使用 sudo,请创建一个名为 docker 的 Unix 组并向其中添加用户。 当 Docker 守护进程(docker daemon)启动时,它会创建一个可由 docker 组成员访问的 Unix 套接字(Unix socket)。Warning : docker组拥有和根用户(root user)同等权限。注意 : 以 non-root user (Rootless mode) 运行 Docker daemon , 可使当前用户不具根用户权限以non-root user (Rootless mode ) 运行 Docker daemonRootless 模式允许以非 root 用户身份运行 Docker 守护进程(daemon)和容器(container),以缓解守护进程和容器运行时中的潜在漏洞。 Rootless模式即使在安装 Docker 守护进程(docker daemon)的过程中也不需要 root 权限,只要满足先决条件即可。工作原理: Rootless 模式在用户命名空间(user namespace)内执行 Docker 守护进程(docker daemon)和容器(containers)。 这与 userns-remap 模式非常相似,除了在 userns-remap 模式下,守护进程(daemon)本身以 root 权限运行,而在Rootless 模式下,守护进程(daemon)和容器(containers)都在没有 root 权限的情况下运行。Rootless 模式不使用具有 SETUID 位或文件功能的二进制文件,除了 newuidmap 和 newgidmap,它们是允许在用户命名空间中使用多个 UID/GID 所必需的。reference : [1] Post-installation steps for Linux.Manage Docker as a non-root user.site [2] Run the Docker daemon as a non-root user (Rootless mode).site
Linux awk 命令awk命令概述 :awk可用于处理文本文件 (pattern scanning and processing language)$ awk --help Usage: awk [POSIX or GNU style options] -f progfile [--] file ... Usage: awk [POSIX or GNU style options] [--] 'program' file ... POSIX options: GNU long options: (standard) -f progfile --file=progfile -F fs --field-separator=fs -v var=val --assign=var=val Short options: GNU long options: (extensions) -b --characters-as-bytes -c --traditional -C --copyright -d[file] --dump-variables[=file] -e 'program-text' --source='program-text' -E file --exec=file -g --gen-pot -h --help -L [fatal] --lint[=fatal] -n --non-decimal-data -N --use-lc-numeric -O --optimize -p[file] --profile[=file] -P --posix -r --re-interval -S --sandbox -t --lint-old -V --version gawk is a pattern scanning and processing language. By default it reads standard input and writes standard output. Examples: gawk '{ sum += $1 }; END { print sum }' file gawk -F: '{ print $1 }' /etc/passwd实例 :################################################ #每行按空格或 tab 键分隔,输出第1,4项 (base) [biocodee@localhost ~]$ head log.txt 2 this is a test 3 Are you like awk This's a test 10 There are orange,apple,mongo (base) [biocodee@localhost ~]$ awk '{print $1,$4}' log.txt 2 a 3 like This's 10 orange,apple,mongo #格式化输出 (base) [biocodee@localhost ~]$ awk '{printf "%-8s %-10s\n",$1,$4}' log.txt 2 a 3 like This's 10 orange,apple,mongo #指定分隔符 -F (base) [biocodee@localhost ~]$ awk -F, '{print $1,$4}' log.txt 2 this is a test 3 Are you like awk This's a test 10 There are orange #使用内建变量 (base) [biocodee@localhost ~]$ awk 'BEGIN{FS=","} {print $1,$2}' log.txt 2 this is a test 3 Are you like awk This's a test 10 There are orange apple #多个分隔符:先用空格分割,然后对分割结果再使用","分割 (base) [biocodee@localhost ~]$ awk -F '[ ,]' '{print $1,$2,$5}' log.txt 2 this test 3 Are awk This's a 10 There apple #设置变量 (base) [biocodee@localhost ~]$ awk -va=1 '{print $1,$1+a}' log.txt 2 3 3 4 This's 1 10 11 (base) [biocodee@localhost ~]$ awk -va=1 -vb=s '{print $1,$1+a,$1b}' log.txt 2 3 2s 3 4 3s This's 1 This'ss 10 11 10s #执行脚本 $awk -f ***.awk log.txt #过滤第一列大于2的行 awk '$1>2' log.txt (base) [biocodee@localhost ~]$ awk '$1>2' log.txt 3 Are you like awk This's a test 10 There are orange,apple,mongo #过滤第一列等于2的行 awk '$1=2' log.txt (base) [biocodee@localhost ~]$ awk '$1=2' log.txt 2 this is a test 2 Are you like awk 2 a test 2 There are orange,apple,mongo #过滤第一列大于2并且第二列等于`Are`的行 awk '$1>2 && $2=="Are" {print $1,$2,$3}' log.txt (base) [biocodee@localhost ~]$ awk '$1>2 && $2=="Are" {print $1,$2,$3}' log.txt 3 Are you ############################################################## #####使用正则,字符串匹配 #输出第二列(包含"th"),并打印第二列与第四列 #awk '$2 ~ /th/ {print $2,$4}' log.txt ; '~'表示模式开始,'//' 中是模式 (base) [biocodee@localhost ~]$ awk '$2 ~ /th/ {print $2,$4}' log.txt this a #忽略大小写 #awk 'BEGIN{IGNORECASE=1} /this/' log.txt (base) [biocodee@localhost ~]$ awk 'BEGIN{IGNORECASE=1} /this/' log.txt 2 this is a test This's a test #模式取反 #awk '$2 !~ /th/ {print $2,$4}' log.txt (base) [biocodee@localhost ~]$ awk '$2 !~ /th/ {print $2,$4}' log.txt Are like a There orange,apple,mongo #awk '!/th/ {print $2,$4}' log.txt (base) [biocodee@localhost ~]$ awk '!/th/ {print $2,$4}' log.txt Are like a There orange,apple,mongo #`awk` 脚本 关键词 : BEGIN 和 END 执行前的语句 ----> BEGIN{***} 处理完所有的行后要执行的语句 -----> END{} 处理每一行时要执行的语句 -------> {} (base) [biocodee@localhost blog]$ awk -f score_cal.awk score.txt NAME NO. MATH ENGLISH COMPUTER TOTAL --------------------------------------------- Marry 2143 78 84 77 239 Jack 2321 66 78 45 189 Tom 2122 48 77 71 196 Mike 2537 87 97 95 279 Bob 2415 40 57 62 159 --------------------------------------------- TOTAL: 319 393 350 AVERAGE: 63.80 78.60 70.00 ################################################################## #九九乘法表 (base) [biocodee@localhost blog]$ seq 9 | sed 'H;g' | awk -v RS='' '{for(i=1;i<=NF;i++)printf("%dx%d=%d%s",i,NR,i*NR,i==NR?"\n":"\t")}' 1x1=1 1x2=2 2x2=4 1x3=3 2x3=6 3x3=9 1x4=4 2x4=8 3x4=12 4x4=16 1x5=5 2x5=10 3x5=15 4x5=20 5x5=25 1x6=6 2x6=12 3x6=18 4x6=24 5x6=30 6x6=36 1x7=7 2x7=14 3x7=21 4x7=28 5x7=35 6x7=42 7x7=49 1x8=8 2x8=16 3x8=24 4x8=32 5x8=40 6x8=48 7x8=56 8x8=64 1x9=9 2x9=18 3x9=27 4x9=36 5x9=45 6x9=54 7x9=63 8x9=72 9x9=81 awk运算符运算符 = += -= = /= %= ^= *=赋值?:C条件表达式\\ 逻辑或&&逻辑与~ 和 !~匹配正则表达式和不匹配正则表达式空格连接+ - !一元加,减 和逻辑非^ *求幂++ --增加或者减少,作为前缀或者后缀$字段引用in数组成员reference : runnob 菜鸟教程 awk
如果是通过source或者binaries 方式安装 Nodejs , 需要手动删除 Node executables和其他resources, 另外就是使用 Homebrew的方法 、nvm 方法 。 1. 手动方法:需要按如下步骤卸载node + npm :从/usr/local/lib 目录下删除全部 node 和node_modules之后,从/usr/local/include目录删除全部node和node_modules从/usr/local/bin 目录删除 node ,node-debug 和 node-gyp从home路径删除.npmrc,.npm,.node-gyp,.node_repl_history从/usr/local/share/man/man1路径删除node*,npm*从/opt/local/bin,opt/local/include,/usr/local/share/doc路径删除node从opt/local/lib 删除node_modules从/usr/local/share/systemtap/tapset路径删除node.stpsudo rm -rf /opt/local/bin/node /opt/local/include/node /opt/local/lib/node_modules sudo rm -rf /usr/local/bin/npm /usr/local/share/man/man1/node.1 /usr/local/lib/dtrace/node.d sudo rm -rf /usr/local/{lib/node{,/.npm,_modules},bin,share/man}/{npm*,node*,man1/node*} sudo rm -rf /usr/local/bin/npm /usr/local/share/man/man1/node* /usr/local/lib/dtrace/node.d ~/.npm ~/.node-gyp 2. Homebrew 和 nvm 方法 : 如果是通过$ brew install node 方式在Mac上安装Nodejs , 可通过如下方式:#unintall brew uninstall node; brew cleanup; rm -f /usr/local/bin/npm /usr/local/lib/dtrace/node.d; rm -rf ~/.npm; #install brew install node; which node export NODE_PATH='/usr/local/lib/node_modules' Node Version Manager(NVM) 和 Homebrew类似。可使用nvm管理node多个版本。如果你想从Macos卸载特定版本node ,可以执行:$nvm uninstall <version> $nvm list -> v6.14.4 v14.16.0 default -> v6.14.4 ... $nvm uninstall v6.14.4reference : [1] How do I completely uninstall Node.js, and reinstall from beginning (Mac OS X)
如何撤销命令make install的安装结果,或者如何卸载Linux 库library ?问题描述 : 使用指令make install 进行package安装,因参数设置错误或者依赖缺失安装终止等原因,需要撤销安装,或者希望卸载已经安装的库(library)解决方法 :方法 1 : make uninstall step 1 : 如果你已经删除或者改变了 build 目录,你需要重新下载source code, -->make-->make install (即重新下载并重新安装)step 2 : 运行make uninstall 命令 :$ cd ../build $ sudo make uninstall如果命令执行成功,即可成功卸载;若仍有顾虑,可参考方法 2 来确保卸载过程没有遗留文件方法 2 : (install_manifest.txt)文件install_manifest.txt包含安装过程中产生的文件名称 。 首先,检查文件列表及其修改时间 :sudo xargs -I{} stat -c "%z %n" "{}" < install_manifest.txt可以成功执行上述命令,并且文件修改时间为安装时间或者晚于安装时间,此时,我们可以删除这些安装过程中产生的文件 :mkdir deleted-by-uninstall sudo xargs -I{} mv -t deleted-by-uninstall "{}" < install_manifest.txt拓展Linux xargs 指令 : xargs , eXtended ARGuments. : 可以给命令传递参数,可以组合多个命令; 可将管道或标准输入数据转换成命令行参数,也可从文件输出中读取数据; 可捕获一个命令的输出,然后传递给另外一个命令。 许多命令不支持|管道传递参数,而 xargs具备此功能。 Reference : [1] What's the opposite of 'make install', i.e. how do you uninstall a library in Linux?.stackoverflow
Error : loading shared libraries: cannot open shared object file: No such file or directory1. 问题描述 :程序S运行需要库L1和库L2的支持。 库L1/L2完成安装后, 执行软件 S 时,库L2 shared object 无法加载, 提示文件或路径不存在2. 解决方法: 当库为动态库(dynamic library)时, 运行程序S时需要为操作系统指明动态库的路径。2.1 确定库(library)的路径:$ sudo find / -name library_name.so2.2 确定动态库路径是否存在于环境变量(LD_LIBRARY_PATH)$ echo $LD_LIBRARY_PATH 2.3 将动态库路径赋值给环境变量(LD_LIBRARY_PATH)$ LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/the/lib $ export LD_LIBRARY_PATH即可运行程序。拓展:1. Linux系统中的库文件 : 库是被称为函数的预编译代码段的集合。库包含公共函数,它们一起形成一个包,称为库。函数是在整个程序中重复使用的代码块。在程序中重复调用这些代码片段可以节省时间。可以避免多次重写代码。对于程序员来说,库提供可重用的函数、数据结构、类等等。库是不可执行的,这是进程和应用程序的关键区别。库在运行时或编译时发挥作用。在C编程语言中,有两种类型的库:动态库和静态库。动态库与静态库 : 动态或共享库(dynamic or shared libraries)作为可执行程序文件之外的独立文件出现。因此,程序在运行时只需要动态/共享库文件的一个副本。而静态库(static libraries)被锁定在程序中。2. Linux 系统环境变量: 环境变量与shell 变量 : 环境变量由shell管理。环境变量与常规shell变量的区别在于,shell变量是shell的特定实例(比如shell脚本)的局部变量,而环境变量是由您启动的任何程序“继承”的,包括另一个shell。也就是说,新进程获得自己的这些变量的副本,它可以读取、修改这些变量,并依次传递给自己的子进程。事实上,每个UNIX进程(不仅仅是shell)都将其环境变量传递给其子进程。在程序及其子程序中,环境变量是全局可用的。shell变量仅在当前shell中可用。像$SHELL这样的环境变量在系统范围内是有效的。在当前的Bash shell中,$ Bash指向Bash的执行路径,而$ shell指向定义为default的shell(其值可能相同)。#定义shell变量 VAR 及其 value [biocodee@localhost ~]$ VAR="This is a variable" [biocodee@localhost ~]$ echo $VAR This is a variable #查看shell变量是否存在于环境变量 env 当中 [biocodee@localhost ~]$ env | grep VAR #将shell 变量转变为 环境变量 [biocodee@localhost ~]$ export VAR [biocodee@localhost ~]$ env | grep VAR VAR=This is a variable参考资料: [1] Linux error while loading shared libraries: cannot open shared object file: No such file or directory [2] Linux Basics: Static Libraries vs. Dynamic Libraries[3] Shell and Environment Variables.UNIX POWER TOOLS.[4] Environment variable vs Shell variable, what's the difference?
不需要使用 dockfile 创建 docker images 的时候,需要从 base docker image 开始. 主机上的容器(container) 共享主机内核(kernel), 但是每个容器必须提供其自身运行需要的操作系统(OS)。 在 windows系统中, 1 : 1 mapping of kernel : OS ; 但是在 Linux 系统当中, kernel 包括多种类型 OSs : Debian, Ubuntu, Alpine, SuSE, CoreOS 等语句FROM 通常指定操作系统类型,但是不需要或者没有必要将操作系统 bundle进一个容器。容器应该仅包括它所需要的。 FROM 提供初始的文件系统,例如,文件、目录等。 同样可以可以使用 Docker 保留的最小的镜像scratch 作为容器构建的起点。使用 镜像scratch将传递给创建过程如下信息: Dockerfile 文件中下一个命令将作为镜像第一个文件系统层。尽管scratch 出现在 Docker repository 当中,但是你不能 pull 或者 run 这个镜像, 也不能用scratch来命名任何镜像。你可以在 Dockfile 当中refer它。使用 FROM scratch ,从一个空的 images 开始构建。 镜像scratch 在创建 base images(比如 debian 和 busybox) 的时候非常重要,或者 super minimal images (仅包含 a single binary 和其他它需要的) .使用 scratch 创建一个简易容器 :FROM scratch COPY hello / CMD ["/hello"]镜像NGINX 使用 Debian 操作系统:FROM debian:buster-slim LABEL maintainer="NGINX Docker Maintainers <docker-maint@nginx.com>" ENV NGINX_VERSION 1.17.6 ENV NJS_VERSION 0.3.7 ENV PKG_RELEASE 1~buster RUN set -x \ # create nginx user/group first, to be consistent throughout docker variants && addgroup --system --gid 101 nginx \ && adduser --system --disabled-login --ingroup nginx --no-create-home --home /nonexistent --gecos "nginx user" --shell /bin/false --uid 101 nginx \ && apt-get update \ && apt-get install --no-install-recommends --no-install-suggests -y gnupg1 ca-certificates \ && \ NGINX_GPGKEY=573BFD6B3D8FBC641079A6ABABF5BD827BD9BF62; \reference : [1] scratch. website [2] does docker always need an operating system as base image.Stack Overflow[3] Create a simple parent image using scratch. website
查看目录结构:[arthur@localhost /]$ ls -al total 2664 dr-xr-xr-x. 18 root root 273 Sep 5 09:25 . dr-xr-xr-x. 18 root root 273 Sep 5 09:25 .. #目录`/bin`为binaries缩写,存放常用命令 lrwxrwxrwx. 1 root root 7 Nov 3 2020 bin -> usr/bin #目录`/boot`启动Linux时所使用的核心文件 dr-xr-xr-x. 6 root root 4096 Sep 5 09:34 boot drwxr-xr-x. 2 root root 47 Apr 15 02:16 bugreport #目录`/dev`为device 缩写,存放外部设备 drwxr-xr-x. 21 root root 3460 Sep 15 11:52 dev drwxr-xr-x. 156 root root 8192 Sep 14 21:22 etc #用户主目录,系统每个用户都有自己一个目录,目录名称同用户账户名 drwxr-xr-x. 3 root root 20 Nov 3 2020 home #存储系统最基本的动态连接共享库,作用类似 windows 中的 dll 文件。几乎所有程序都需要使用这些共享库。 lrwxrwxrwx. 1 root root 7 Nov 3 2020 lib -> usr/lib lrwxrwxrwx. 1 root root 9 Nov 3 2020 lib64 -> usr/lib64 #系统自动识别的一些设备,如 U 盘、光驱等,会将其挂载在该目录下 drwxr-xr-x. 3 root root 18 Sep 2 04:24 media #用于临时挂载其他文件系统 drwxr-xr-x. 2 root root 6 Nov 3 2020 mnt #用于为当前主机额外安装软件 drwxr-xr-x. 4 root root 38 Aug 21 05:42 opt #伪文件系统(虚拟文件系统),存储当前内核运行状态的一系列特殊文件,是系统内存的映射 dr-xr-xr-x. 294 root root 0 Sep 14 21:21 proc #用户root 主目录: dr-xr-x---. 12 root root 4096 Sep 12 07:41 root #临时文件系统,存储系统启动以来的信息。当系统重启时,该目录下文件被删除或清除 drwxr-xr-x. 48 root root 1420 Sep 15 11:52 run #系统管理员所使用的系统管理程序superuser binaries lrwxrwxrwx. 1 root root 8 Nov 3 2020 sbin -> usr/sbin lrwxrwxrwx. 1 root root 19 Aug 31 10:10 snap -> /var/lib/snapd/snap #存放服务启动之后需要提取的数据 drwxr-xr-x. 2 root root 6 Nov 3 2020 srv #内核 dr-xr-xr-x. 13 root root 0 Sep 14 21:21 sys -rw-r--r--. 1 root root 2696952 Apr 8 02:49 @System.solv drwxrwxrwt. 16 root root 4096 Sep 15 12:37 tmp #共享资源 unix shared resources,用户许多应用程序和文件存放于该目录下 drwxr-xr-x. 12 root root 144 Apr 11 20:30 usr #变量 variable, 经常被修改的目录、日志文件存放于该目录下 drwxr-xr-x. 22 root root 4096 Aug 31 10:06 varreference:runoob.com
查看 top man#man top information $man top NAME top - display Linux processes SYNOPSIS top -hv|-bcEHiOSs1 -d secs -n max -u|U user -p pid -o fld -w [cols] The traditional switches `-' and whitespace are optional. DESCRIPTION The top program provides a dynamic real-time view of a running system. It can display system summary information as well as a list of pro‐ cesses or threads currently being managed by the Linux kernel. The types of system summary information shown and the types, order and size of information displayed for processes are all user configurable and that configuration can be made persistent across restarts. The program provides a limited interactive interface for process manipulation as well as a much more extensive interface for personal con‐ figuration -- encompassing every aspect of its operation. And while top is referred to throughout this document, you are free to name the program anything you wish. That new name, possibly an alias, will then be reflected on top's display and used when reading and writing a configuration file.查看top -h[arthur@localhost ~]$ top -help procps-ng 3.3.15 Usage: top -hv | -bcEHiOSs1 -d secs -n max -u|U user -p pid(s) -o field -w [cols]查看系统进程动态信息#example: $top top - 11:55:48 up 14:34, 1 user, load average: 0.54, 0.30, 0.13 Tasks: 276 total, 2 running, 274 sleeping, 0 stopped, 0 zombie %Cpu(s): 3.1 us, 1.5 sy, 0.0 ni, 94.8 id, 0.0 wa, 0.4 hi, 0.2 si, 0.0 st MiB Mem : 9356.3 total, 4824.7 free, 1896.4 used, 2635.2 buff/cache MiB Swap: 4848.0 total, 4848.0 free, 0.0 used. 6934.0 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 2887 arthur 20 0 4487204 204476 109484 S 6.0 2.1 0:59.54 gnome-shell 1021 root -51 0 0 0 0 D 3.3 0.0 0:10.28 irq/37-rmi4_smb 3670 arthur 20 0 767400 50208 36212 S 2.7 0.5 0:09.45 gnome-terminal- 32549 arthur 20 0 16.8g 108524 74992 S 2.7 1.1 0:07.81 chrome 32719 arthur 20 0 20.8g 216424 130188 S 2.7 2.3 0:17.09 chrome 注释: PID (Process ID) : 进程标识号 ; USER: 进程使用者; PR(Priority):优先级别,数值越小优先级越高; NI (Nice): Nice value of task. 优先级别数值(数值越小越先执行); VIRT : 进程占用的虚拟内存值; RES : 进程占用的物理内存值; SHR : 进程使用的共享内存值; S : 进程状态(S : sleep; D: 不可中断睡眠状态; R : 正在运行 ); %CPU : 进程占用的CPU 使用率; %MEM : 进程占用物理内存的百分比 ; TIME+ : 进程启用后占用的CPU 时间其他:top 显示当前最近运行的所有进程。 top -i 可以将空闲进程隐藏。
8.2.2. LISTING PACKAGESyum list 和其它相关的命令提供packages, package groups, repositories 相关信息。 通过拓展一个或多个glob expressions 作为yum's list commands 的参数,可以实现对结果的过滤。 Glob expression 是普通字符串,它包含一个或多个通配符(*,可以多次匹配任意字符; ?,可以匹配任意一个字符)。Note : 当将 glob expressions 作为参数传递给 yum命令时,需要注意 glob expressions 表达式的转义,否则 Bash shell 会将这些表达式当作 pathname expansions, 进而会将当前路径下所有匹配 glob 的文件传递给 yum. 为了glob expressions能够按照我们的意图传递给yum,有两种方式可供选择:+ 通过在通配符前添加反斜杠字符实现转义 + 将整个`glob expression` 用双引号或单引号引起来1.1 yum list glob_expression :能够和glob expressions匹配的并且已经安装的和available packages 信息将会list :#example: 使用 glob expressions将`ABRT add-ons`和`plug-ins` 列出: #注释:`ABRT add-ons`和`plug-ins` packages 以`abrt-addon- ` 或者 `abrt-plugin- `开头 [arthur@localhost ~]$ yum list abrt-addon\* abrt-plugin\* CentOS-8 - AppStream 4.2 MB/s | 8.8 MB 00:02 CentOS-8 - Base 4.4 MB/s | 6.5 MB 00:01 CentOS-8 - Extras 14 kB/s | 10 kB 00:00 CentOS-8 - PowerTools 2.5 MB/s | 2.3 MB 00:00 Extra Packages for Enterprise Linux Modular 8 - x86_64 474 kB/s | 939 kB 00:01 Extra Packages for Enterprise Linux 8 - x86_64 4.6 MB/s | 10 MB 00:02 Available Packages abrt-addon-ccpp.x86_64 2.10.9-20.el8 AppStream abrt-addon-coredump-helper.x86_64 2.10.9-20.el8 AppStream abrt-addon-kerneloops.x86_64 2.10.9-20.el8 AppStream abrt-addon-pstoreoops.x86_64 2.10.9-20.el8 AppStream abrt-addon-vmcore.x86_64 2.10.9-20.el8 AppStream abrt-addon-xorg.x86_64 2.10.9-20.el8 AppStream abrt-plugin-machine-id.x86_64 2.10.9-20.el8 AppStream abrt-plugin-sosreport.x86_64 2.10.9-20.el8 AppStream1.2 yum list all将所有installed and available packages list :[arthur@localhost ~]$ yum list all Last metadata expiration check: 0:10:39 ago on Mon 13 Sep 2021 09:21:32 AM EDT. ###Installed Packages Last metadata expiration check: 0:10:39 ago on Mon 13 Sep 2021 09:21:32 AM EDT. Installed Packages GConf2.x86_64 3.2.6-22.el8 @AppStream ModemManager.x86_64 1.10.4-1.el8 @anaconda ModemManager-glib.x86_64 1.10.4-1.el8 @anaconda NetworkManager.x86_64 1:1.20.0-3.el8 @anaconda NetworkManager-adsl.x86_64 1:1.20.0-3.el8 @anaconda NetworkManager-bluetooth.x86_64 1:1.20.0-3.el8 @anaconda NetworkManager-libnm.x86_64 1:1.20.0-3.el8 @anaconda NetworkManager-libreswan.x86_64 1.2.10-4.el8 @AppStream NetworkManager-team.x86_64 1:1.20.0-3.el8 @anaconda NetworkManager-tui.x86_64 1:1.20.0-3.el8 @anaconda NetworkManager-wifi.x86_64 1:1.20.0-3.el8 @anaconda NetworkManager-wwan.x86_64 1:1.20.0-3.el8 @anaconda PackageKit.x86_64 1.1.12-3.el8 @AppStream PackageKit-command-not-found.x86_64 1.1.12-3.el8 @AppStream PackageKit-glib.x86_64 1.1.12-3.el8 @AppStream PackageKit-gstreamer-plugin.x86_64 1.1.12-3.el8 @AppStream PackageKit-gtk3-module.x86_64 1.1.12-3.el8 @AppStream R.x86_64 4.0.3-1.el8 @epel R-core.x86_64 4.0.3-1.el8 @epel R-core-devel.x86_64 4.0.3-1.el8 @epel ###Available Packages 3proxy.x86_64 0.8.13-1.el8 epel AusweisApp2.x86_64 1.22.2-3.el8 epel AusweisApp2-data.noarch 1.22.2-3.el8 epel AusweisApp2-doc.noarch 1.22.2-3.el8 epel BackupPC.x86_64 4.4.0-1.el8 epel BackupPC-XS.x86_64 0.62-1.el8 epel BibTool.x86_64 2.68-1.el8 epel CCfits.x86_64 2.5-14.el8 epel CCfits-devel.x86_64 2.5-14.el8 epel CCfits-doc.noarch 2.5-14.el8 epel CFR.noarch 0.151-4.el8 epel CFR-javadoc.noarch 0.151-4.el8 epel CGSI-gSOAP.x86_64 1.3.11-7.el8 epel CGSI-gSOAP-devel.x86_64 1.3.11-7.el8 epel CUnit.i686 2.1.3-17.el8 AppStream 1.3 yum list installed list 所有系统已安装的 packages , 输出中最右边的列列出了从其中检索包的存储库。#列出以`krb`开头,后随一个字符和一个连字符的所有 installed packages [arthur@localhost ~]$ yum list "krb?-*" Last metadata expiration check: 0:34:44 ago on Mon 13 Sep 2021 09:21:32 AM EDT. Installed Packages krb5-devel.x86_64 1.18.2-5.el8 @BaseOS krb5-libs.x86_64 1.18.2-5.el8 @BaseOS Available Packages krb5-devel.i686 1.18.2-8.el8 BaseOS krb5-devel.x86_64 1.18.2-8.el8 BaseOS krb5-libs.i686 1.18.2-8.el8 BaseOS krb5-libs.x86_64 1.18.2-8.el8 BaseOS krb5-pkinit.i686 1.18.2-8.el8 BaseOS krb5-pkinit.x86_64 1.18.2-8.el8 BaseOS krb5-server.i686 1.18.2-8.el8 BaseOS krb5-server.x86_64 1.18.2-8.el8 BaseOS krb5-server-ldap.i686 1.18.2-8.el8 BaseOS krb5-server-ldap.x86_64 1.18.2-8.el8 BaseOS krb5-workstation.x86_64 1.18.2-8.el8 BaseOS [arthur@localhost ~]$ 1.4 yum list available将all enabled repositories 中的available packages 列出:#列出名称中包含“gstreamer”和“plugin”的所有可用包 arthur@localhost ~]$ yum list available "*gstreamer*plugin*" Last metadata expiration check: 0:42:50 ago on Mon 13 Sep 2021 09:21:32 AM EDT. ###Available Packages PackageKit-gstreamer-plugin.x86_64 1.1.12-6.el8 AppStream gstreamer-plugin-crystalhd.x86_64 3.10.0-22.el8 epel gstreamer1-plugins-bad-free.i686 1.16.1-1.el8 AppStream gstreamer1-plugins-bad-free.x86_64 1.16.1-1.el8 AppStream gstreamer1-plugins-bad-free-devel.i686 1.16.1-1.el8 PowerTools gstreamer1-plugins-bad-free-devel.x86_64 1.16.1-1.el8 PowerTools gstreamer1-plugins-base.i686 1.16.1-2.el8 AppStream gstreamer1-plugins-base.x86_64 1.16.1-2.el8 AppStream gstreamer1-plugins-base-devel.i686 1.16.1-2.el8 AppStream gstreamer1-plugins-base-devel.x86_64 1.16.1-2.el8 AppStream gstreamer1-plugins-good.i686 1.16.1-2.el8 AppStream gstreamer1-plugins-good.x86_64 1.16.1-2.el8 AppStream gstreamer1-plugins-good-gtk.i686 1.16.1-2.el8 AppStream gstreamer1-plugins-good-gtk.x86_64 1.16.1-2.el8 AppStream gstreamer1-plugins-ugly-free.i686 1.16.1-1.el8 AppStream gstreamer1-plugins-ugly-free.x86_64 1.16.1-1.el8 AppStream [arthur@localhost ~]$ yum list available gstreamer\*plugin\* Last metadata expiration check: 0:43:42 ago on Mon 13 Sep 2021 09:21:32 AM EDT. ###Available Packages gstreamer-plugin-crystalhd.x86_64 3.10.0-22.el8 epel gstreamer1-plugins-bad-free.i686 1.16.1-1.el8 AppStream gstreamer1-plugins-bad-free.x86_64 1.16.1-1.el8 AppStream gstreamer1-plugins-bad-free-devel.i686 1.16.1-1.el8 PowerTools gstreamer1-plugins-bad-free-devel.x86_64 1.16.1-1.el8 PowerTools gstreamer1-plugins-base.i686 1.16.1-2.el8 AppStream gstreamer1-plugins-base.x86_64 1.16.1-2.el8 AppStream gstreamer1-plugins-base-devel.i686 1.16.1-2.el8 AppStream gstreamer1-plugins-base-devel.x86_64 1.16.1-2.el8 AppStream gstreamer1-plugins-good.i686 1.16.1-2.el8 AppStream gstreamer1-plugins-good.x86_64 1.16.1-2.el8 AppStream gstreamer1-plugins-good-gtk.i686 1.16.1-2.el8 AppStream gstreamer1-plugins-good-gtk.x86_64 1.16.1-2.el8 AppStream gstreamer1-plugins-ugly-free.i686 1.16.1-1.el8 AppStream gstreamer1-plugins-ugly-free.x86_64 1.16.1-1.el8 AppStream [arthur@localhost ~]$ yum list available gstreamer\*plugin Last metadata expiration check: 0:43:58 ago on Mon 13 Sep 2021 09:21:32 AM EDT. Error: No matching Packages to list [arthur@localhost ~]$ 1.5 yum group list列出所有的package groups.[arthur@localhost ~]$ yum group list Last metadata expiration check: 0:48:56 ago on Mon 13 Sep 2021 09:21:32 AM EDT. ###Available Environment Groups: Server with GUI Server Minimal Install KDE Plasma Workspaces Virtualization Host Custom Operating System ###Installed Environment Groups: Workstation ###Installed Groups: Container Management RPM Development Tools Development Tools Graphical Administration Tools Headless Management Legacy UNIX Compatibility Network Servers Scientific Support Security Tools Smart Card Support System Tools ###Available Groups: .NET Core Development Fedora Packager Xfce [arthur@localhost ~]$ 1.6 yum repolist列出已经启用的存储库的存储库ID、名称和提供的包的数量。[arthur@localhost ~]$ yum repolist Last metadata expiration check: 0:48:15 ago on Mon 13 Sep 2021 09:21:32 AM EDT. repo id repo name status AppStream CentOS-8 - AppStream 5,802 BaseOS CentOS-8 - Base 2,076 PowerTools CentOS-8 - PowerTools 1,632 *epel Extra Packages for Enterprise Linux 8 - x86_64 7,760 *epel-modular Extra Packages for Enterprise Linux Modular 8 - x86_64 0 extras CentOS-8 - Extras 37reference: 8.2.2. Listing Packages - Red Hat Customer Portal website
目的:下载 docker images 之后,了解 images 内部目录、文件结构,以及容器内部程序运行情况方法:1. 使用docker exec (最简单)和nsenter命令类似,docker exec 命令可以在已经运行的容器运行新的进程(容器必须已经运行才可以,否则会报错),可以运行/bin/bash进入容器:[arthur@localhost Downloads]$ sudo docker ps [sudo] password for arthur: CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 09b6673f175a hicstuff:v1.0 "/bin/bash" 2 hours ago Up 2 hours quizzical_chatelet c67a7e98646a koszullab/hicstuff "/bin/bash" 4 hours ago Up 4 hours sleepy_bhabha [arthur@localhost Downloads]$ sudo docker exec -it quizzical_chatelet /bin/bash (base) root@09b6673f175a:/app# pwd /app (base) root@09b6673f175a:/app# ls LICENSE __init__.py distance_law.py io.py pipeline.py test_commands.py test_filter.py tests MANIFEST.in commands.py filter.py iteralign.py requirements.txt test_digest.py test_hicstuff.py version.py Makefile cutsite.py hicstuff log.py setup.cfg test_distance_law.py test_io.py view.py README.md digest.py hicstuff.py main.py setup.py test_doctests.py test_pipeline.py2. 使用快照(snapshotting)#查看已经运行的容器,记录要查看容器的 ID $ docker ps #创建新的 images $ docker commit ID NewImagesName #通过 `bash` 进入新镜像 $ docker run -it NewImagesName /bin/bash #删除新的镜像 NewImagesName $ docker rmi NewIamgesName3. 使用 ssh如果需要持续性连接到容器,可以在容器内安装sshd,并运行sshd#查看 容器 port $ docker ps # $ docker run -d -p PortNumber ContainerName /usr/sbin/sshd -D4. 使用nsenter5. 覆盖默认的 ENTRYPOINT$ docker run --rm -it --entrypoint=/bin/bash NameofImage ######################################################################### --entrypoint string Overwrite the default ENTRYPOINT of the image #覆盖默认 entrypoint [arthur@localhost Downloads]$ sudo docker run --rm -it --entrypoint=/bin/bash koszullab/hicstuff [sudo] password for arthur: (base) root@1f60e1415681:/app# pwd /app (base) root@1f60e1415681:/app# ls LICENSE __init__.py distance_law.py io.py pipeline.py test_commands.py test_filter.py tests MANIFEST.in commands.py filter.py iteralign.py requirements.txt test_digest.py test_hicstuff.py version.py Makefile cutsite.py hicstuff log.py setup.cfg test_distance_law.py test_io.py view.py README.md digest.py hicstuff.py main.py setup.py test_doctests.py test_pipeline.py (base) root@1f60e1415681:/app# #未覆盖 [arthur@localhost Downloads]$ sudo docker run -it koszullab/hicstuff [sudo] password for arthur: INFO :: generated new fontManager usage: hicstuff [-hv] <command> [<args>...] [arthur@localhost Downloads]$ sudo docker run -it koszullab/hicstuff /bin/bash INFO :: generated new fontManager Unknown command. usage: hicstuff [-hv] <command> [<args>...] [arthur@localhost Downloads]$ #运行其他 images [arthur@localhost Downloads]$ sudo docker run -it hicpro:homo.v1 root@5bf6508e2b7c:/# exit exit [arthur@localhost Downloads]$ sudo docker run -it nservant/hicpro:v3.01 root@fb00a89cb150:/# exit exit #原镜像运行 commit 之后的 [arthur@localhost Downloads]$ [arthur@localhost Downloads]$ sudo docker run -it hicstuff:v1.0 [sudo] password for arthur: (base) root@4de5fd7f7bae:/app# pwd /app (base) root@4de5fd7f7bae:/app# ls LICENSE __init__.py distance_law.py io.py pipeline.py test_commands.py test_filter.py tests MANIFEST.in commands.py filter.py iteralign.py requirements.txt test_digest.py test_hicstuff.py version.py Makefile cutsite.py hicstuff log.py setup.cfg test_distance_law.py test_io.py view.py README.md digest.py hicstuff.py main.py setup.py test_doctests.py test_pipeline.py 拓展:镜像构建 dockfile 存在差别: koszullab/hicstuff 镜像为 from conda , nservant/hicpro from ubuntu原镜像 koszullab/hicstuff commit 之后与原镜像的差别:登陆方式出现差别Reference : [1] exploring Docker container's file system.stack overflow.
1. 查看内核版本(kernel version)信息:[arthur@localhost genome]$ uname -a Linux localhost.localdomain 4.18.0-240.22.1.el8_3.x86_64 #1 SMP Thu Apr 8 19:01:30 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux2. 查看发行版本信息(distribution information)不同的系统发行版本需要使用不同的命令。#centos 默认不支持 Linux Standard Base,需要安装 [arthur@localhost genome]$ lsb_release -a bash: lsb_release: command not found... Install package 'redhat-lsb-core' to provide command 'lsb_release'? [N/y] N [arthur@localhost genome]$ ll /etc/lsb-release ls: cannot access '/etc/lsb-release': No such file or directory #/etc/issue 文件包含信息或者系统识别信息,这些信息在登入提示之前会进行输出 #`r` : insert the release number of the OS #`m` : insert the architecture identifier of the machine [arthur@localhost genome]$ cat /etc/issue.net \S Kernel \r on an \m #对于 centos 或者 redhat linux发行版本,可以使用如下命令: [arthur@localhost genome]$ cat /etc/redhat-release CentOS Linux release 8.3.2011 #同样,可以采用如下方式: [arthur@localhost genome]$ cat /etc/os-release NAME="CentOS Linux" VERSION="8" ID="centos" ID_LIKE="rhel fedora" VERSION_ID="8" PLATFORM_ID="platform:el8" PRETTY_NAME="CentOS Linux 8" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:centos:centos:8" HOME_URL="https://centos.org/" BUG_REPORT_URL="https://bugs.centos.org/" CENTOS_MANTISBT_PROJECT="CentOS-8" CENTOS_MANTISBT_PROJECT_VERSION="8" #或者: [arthur@localhost genome]$ grep '^VERSION' /etc/os-release VERSION="8" VERSION_ID="8" [arthur@localhost genome]$ egrep '^(VERSION|NAME)=' /etc/os-release NAME="CentOS Linux" VERSION="8" #或者: [arthur@localhost genome]$ cat /proc/version Linux version 4.18.0-240.22.1.el8_3.x86_64 (mockbuild@kbuilder.bsys.centos.org) (gcc version 8.3.1 20191121 (Red Hat 8.3.1-5) (GCC)) #1 SMP Thu Apr 8 19:01:30 UTC 2021 reference : [1] man7.org/linux/man-pages/man8/agetty.8.html[2] how to check os version in Linux command line.cyberciti.
1. gz文件类型:解压:[arthur@localhost Documents]$ gzip --help Usage: gzip [OPTION]... [FILE]... Compress or uncompress FILEs (by default, compress FILES in-place). -d, --decompress decompress ... -1, --fast compress faster -9, --best compress better [arthur@localhost Documents]$ ll total 484 drwxrwxr-x. 2 arthur arthur 4096 Sep 3 08:50 blog drwxrwxr-x. 2 arthur arthur 42 Apr 7 04:11 log drwxr-xr-x. 2 arthur arthur 20 Apr 18 23:12 notes -rwxrwxr-x. 1 arthur arthur 487578 Sep 3 09:30 sample.reads_for.fastq.gz drwxrwxr-x. 2 arthur arthur 41 Apr 15 09:41 script [arthur@localhost Documents]$ gzip -d sample.reads_for.fastq.gz [arthur@localhost Documents]$ ll total 1988 drwxrwxr-x. 2 arthur arthur 4096 Sep 3 08:50 blog drwxrwxr-x. 2 arthur arthur 42 Apr 7 04:11 log drwxr-xr-x. 2 arthur arthur 20 Apr 18 23:12 notes -rwxrwxr-x. 1 arthur arthur 1861842 Sep 3 09:30 sample.reads_for.fastq drwxrwxr-x. 2 arthur arthur 41 Apr 15 09:41 script 2. bz2文件类型[arthur@localhost Documents]$ bzip2 --help bzip2, a block-sorting file compressor. Version 1.0.6, 6-Sept-2010. usage: bzip2 [flags and input files in any order] -h --help print this message -d --decompress force decompression ... If invoked as `bzip2', default action is to compress. as `bunzip2', default action is to decompress. as `bzcat', default action is to decompress to stdout. [arthur@localhost Documents]$ ll total 1824 drwxrwxr-x. 2 arthur arthur 4096 Sep 3 08:50 blog drwxrwxr-x. 2 arthur arthur 42 Apr 7 04:11 log drwxr-xr-x. 2 arthur arthur 20 Apr 18 23:12 notes -rwxrwxr-x. 1 arthur arthur 1861842 Sep 3 09:30 sample.reads_for.fastq drwxrwxr-x. 2 arthur arthur 41 Apr 15 09:41 script #压缩 [arthur@localhost Documents]$ bzip2 sample.reads_for.fastq [arthur@localhost Documents]$ ll total 320 drwxrwxr-x. 2 arthur arthur 4096 Sep 3 08:50 blog drwxrwxr-x. 2 arthur arthur 42 Apr 7 04:11 log drwxr-xr-x. 2 arthur arthur 20 Apr 18 23:12 notes -rwxrwxr-x. 1 arthur arthur 322712 Sep 3 09:30 sample.reads_for.fastq.bz2 drwxrwxr-x. 2 arthur arthur 41 Apr 15 09:41 script #解压 [arthur@localhost Documents]$ bunzip2 sample.reads_for.fastq.bz2 [arthur@localhost Documents]$ ll total 1824 drwxrwxr-x. 2 arthur arthur 4096 Sep 3 08:50 blog drwxrwxr-x. 2 arthur arthur 42 Apr 7 04:11 log drwxr-xr-x. 2 arthur arthur 20 Apr 18 23:12 notes -rwxrwxr-x. 1 arthur arthur 1861842 Sep 3 09:30 sample.reads_for.fastq drwxrwxr-x. 2 arthur arthur 41 Apr 15 09:41 script #压缩: (添加 管道 命令) [arthur@localhost Documents]$ bzip2 sample.reads_for.fastq | ll total 1824 drwxrwxr-x. 2 arthur arthur 4096 Sep 3 08:50 blog drwxrwxr-x. 2 arthur arthur 42 Apr 7 04:11 log drwxr-xr-x. 2 arthur arthur 20 Apr 18 23:12 notes -rwxrwxr-x. 1 arthur arthur 1861842 Sep 3 09:30 sample.reads_for.fastq -rw-------. 1 arthur arthur 0 Sep 3 09:38 sample.reads_for.fastq.bz2 drwxrwxr-x. 2 arthur arthur 41 Apr 15 09:41 script [arthur@localhost Documents]$ ll total 320 drwxrwxr-x. 2 arthur arthur 4096 Sep 3 08:50 blog drwxrwxr-x. 2 arthur arthur 42 Apr 7 04:11 log drwxr-xr-x. 2 arthur arthur 20 Apr 18 23:12 notes -rwxrwxr-x. 1 arthur arthur 322712 Sep 3 09:30 sample.reads_for.fastq.bz2 drwxrwxr-x. 2 arthur arthur 41 Apr 15 09:41 script #解压:(添加 管道 命令) [arthur@localhost Documents]$ bzip2 -d sample.reads_for.fastq.bz2 | ll total 320 drwxrwxr-x. 2 arthur arthur 4096 Sep 3 08:50 blog drwxrwxr-x. 2 arthur arthur 42 Apr 7 04:11 log drwxr-xr-x. 2 arthur arthur 20 Apr 18 23:12 notes -rw-------. 1 arthur arthur 0 Sep 3 09:39 sample.reads_for.fastq -rwxrwxr-x. 1 arthur arthur 322712 Sep 3 09:30 sample.reads_for.fastq.bz2 drwxrwxr-x. 2 arthur arthur 41 Apr 15 09:41 script [arthur@localhost Documents]$ ll total 1824 drwxrwxr-x. 2 arthur arthur 4096 Sep 3 08:50 blog drwxrwxr-x. 2 arthur arthur 42 Apr 7 04:11 log drwxr-xr-x. 2 arthur arthur 20 Apr 18 23:12 notes -rwxrwxr-x. 1 arthur arthur 1861842 Sep 3 09:30 sample.reads_for.fastq drwxrwxr-x. 2 arthur arthur 41 Apr 15 09:41 script 3. xz文件类型:[arthur@localhost Documents]$ xz --help Usage: xz [OPTION]... [FILE]... Compress or decompress FILEs in the .xz format. -z, --compress force compression -d, --decompress force decompression -t, --test test compressed file integrity -l, --list list information about .xz files4. tar.gz, tar.bz2文件类型:[arthur@localhost Documents]$ tar --help Usage: tar [OPTION...] [FILE]... GNU 'tar' saves many files together into a single tape or disk archive, and can restore individual files from the archive. Examples: tar -cf archive.tar foo bar # Create archive.tar from files foo and bar. tar -tvf archive.tar # List all files in archive.tar verbosely. tar -xf archive.tar # Extract all files from archive.tar. ######################################################################## 文件打包与解包 #查看当前目录与文件 [arthur@localhost Documents]$ ll total 5472 drwxrwxr-x. 2 arthur arthur 4096 Sep 3 08:50 blog drwxrwxr-x. 2 arthur arthur 42 Apr 7 04:11 log drwxr-xr-x. 2 arthur arthur 20 Apr 18 23:12 notes -rwxrwxr-x. 1 arthur arthur 1861842 Sep 3 09:53 sample.reads_for.fastq -rwxrwxr-x. 1 arthur arthur 1861842 Sep 3 09:30 sample.reads_rev.fastq drwxrwxr-x. 2 arthur arthur 41 Apr 15 09:41 script #将文件 for 和 rev 打包为 seq.tar [arthur@localhost Documents]$ tar -cf seq.tar sample.reads_for.fastq sample.reads_rev.fastq && ll total 9112 drwxrwxr-x. 2 arthur arthur 4096 Sep 3 08:50 blog drwxrwxr-x. 2 arthur arthur 42 Apr 7 04:11 log drwxr-xr-x. 2 arthur arthur 20 Apr 18 23:12 notes -rwxrwxr-x. 1 arthur arthur 1861842 Sep 3 09:53 sample.reads_for.fastq -rwxrwxr-x. 1 arthur arthur 1861842 Sep 3 09:30 sample.reads_rev.fastq drwxrwxr-x. 2 arthur arthur 41 Apr 15 09:41 script -rw-rw-r--. 1 arthur arthur 3727360 Sep 3 09:54 seq.tar #查看包内文件: [arthur@localhost Documents]$ tar -tvf seq.tar -rwxrwxr-x arthur/arthur 1861842 2021-09-03 09:53 sample.reads_for.fastq -rwxrwxr-x arthur/arthur 1861842 2021-09-03 09:30 sample.reads_rev.fastq #解开包并将包内文件输出到新的路径下 [arthur@localhost Documents]$ tar -xf seq.tar -C ../Desktop/ [arthur@localhost Documents]$ ll ../Desktop/ total 5420 -rw-rw-r--. 1 arthur arthur 162525 Apr 14 11:37 20210414log -rw-rw-r--. 1 arthur arthur 1441063 Apr 15 02:44 20210415.log -rw-rw-r--. 1 arthur arthur 46384 Apr 18 10:25 20210418log -rwxrwxr-x. 1 arthur arthur 1861842 Sep 3 09:53 sample.reads_for.fastq -rwxrwxr-x. 1 arthur arthur 1861842 Sep 3 09:30 sample.reads_rev.fastq [arthur@localhost Documents]$ ll total 7284 drwxrwxr-x. 2 arthur arthur 4096 Sep 3 08:50 blog drwxrwxr-x. 2 arthur arthur 42 Apr 7 04:11 log drwxr-xr-x. 2 arthur arthur 20 Apr 18 23:12 notes -rwxrwxr-x. 1 arthur arthur 1861842 Sep 3 09:53 sample.reads_for.fastq -rwxrwxr-x. 1 arthur arthur 1861842 Sep 3 09:30 sample.reads_rev.fastq drwxrwxr-x. 2 arthur arthur 41 Apr 15 09:41 script -rw-rw-r--. 1 arthur arthur 3727360 Sep 3 09:54 seq.tar ######################################################################## 文件打包压缩与文件解包解压缩 tar 可以使用 -z 参数来调用gzip进行压缩;使用 -j 参数来调用 bzip2 进行压缩; tar 调用 compress , compress 也是一个压缩程序, `.Z`结尾的文件就是 bzip2 压缩的结果。相对应解压程序为 uncompress. tar 中使用 -Z 参数调用 compress #文件打包压缩为 seq.tar.gz, 参数 `-Z` #此处输出文件命名不规范,应为 seq.tar.Z [arthur@localhost Documents]$ tar -Zcf seq.tar.gz sample.reads_for.fastq sample.reads_rev.fastq [arthur@localhost Documents]$ ll total 8108 drwxrwxr-x. 2 arthur arthur 4096 Sep 3 08:50 blog drwxrwxr-x. 2 arthur arthur 42 Apr 7 04:11 log drwxr-xr-x. 2 arthur arthur 20 Apr 18 23:12 notes -rwxrwxr-x. 1 arthur arthur 1861842 Sep 3 09:53 sample.reads_for.fastq -rwxrwxr-x. 1 arthur arthur 1861842 Sep 3 09:30 sample.reads_rev.fastq drwxrwxr-x. 2 arthur arthur 41 Apr 15 09:41 script -rw-rw-r--. 1 arthur arthur 3727360 Sep 3 09:54 seq.tar -rw-rw-r--. 1 arthur arthur 841611 Sep 3 10:16 seq.tar.gz #文件打包压缩为 seq2.tar.gz, 参数 `z` [arthur@localhost Documents]$ tar -zcf seq2.tar.gz sample.reads_for.fastq sample.reads_rev.fastq [arthur@localhost Documents]$ ll total 9060 drwxrwxr-x. 2 arthur arthur 4096 Sep 3 08:50 blog drwxrwxr-x. 2 arthur arthur 42 Apr 7 04:11 log drwxr-xr-x. 2 arthur arthur 20 Apr 18 23:12 notes -rwxrwxr-x. 1 arthur arthur 1861842 Sep 3 09:53 sample.reads_for.fastq -rwxrwxr-x. 1 arthur arthur 1861842 Sep 3 09:30 sample.reads_rev.fastq drwxrwxr-x. 2 arthur arthur 41 Apr 15 09:41 script -rw-rw-r--. 1 arthur arthur 974587 Sep 3 10:19 seq2.tar.gz -rw-rw-r--. 1 arthur arthur 3727360 Sep 3 09:54 seq.tar -rw-rw-r--. 1 arthur arthur 841611 Sep 3 10:16 seq.tar.gz #创建目录存放解包解压缩文件 [arthur@localhost Documents]$ mkdir seq2 seq1 [arthur@localhost Documents]$ ll total 9060 drwxrwxr-x. 2 arthur arthur 4096 Sep 3 08:50 blog drwxrwxr-x. 2 arthur arthur 42 Apr 7 04:11 log drwxr-xr-x. 2 arthur arthur 20 Apr 18 23:12 notes -rwxrwxr-x. 1 arthur arthur 1861842 Sep 3 09:53 sample.reads_for.fastq -rwxrwxr-x. 1 arthur arthur 1861842 Sep 3 09:30 sample.reads_rev.fastq drwxrwxr-x. 2 arthur arthur 41 Apr 15 09:41 script drwxrwxr-x. 2 arthur arthur 6 Sep 3 10:21 seq1 drwxrwxr-x. 2 arthur arthur 6 Sep 3 10:21 seq2 -rw-rw-r--. 1 arthur arthur 974587 Sep 3 10:19 seq2.tar.gz -rw-rw-r--. 1 arthur arthur 3727360 Sep 3 09:54 seq.tar -rw-rw-r--. 1 arthur arthur 841611 Sep 3 10:16 seq.tar.gz # [arthur@localhost seq2]$ tar -zxvf ../seq2.tar.gz sample.reads_for.fastq sample.reads_rev.fastq [arthur@localhost seq2]$ cd ../ [arthur@localhost Documents]$ cd seq1 # [arthur@localhost seq1]$ tar -Zxvf ../seq.tar.gz sample.reads_for.fastq sample.reads_rev.fastq5. 解压缩文件总结:对于.tar结尾的文件:tar -xf file.tar对于.gz结尾的文件: -gzip -d file.gz-gunzip file.gz对于.tgz,.tar.gz结尾的文件-tar -zxvf file.tar.gz-tar -zxvf file.tgz对于.bz2结尾的文件:-bzip2 -d file.bz2-bunzip2 file.bz2对于tar.bz2结尾的文件: -tar -jxvf file.tar.bz2对于.Z结尾的文件:-uncompress file.Z对于.tar.Z结尾的文件:-tar -xZf file.tar.Zreference : [1] 鸟哥私房菜[2] 解压缩、压缩命令.runoob.
Linux系统CentOS8 检测USB、创建挂载点、挂载、存取USB数据、取消挂载当主机接入 usb 设备时, linux 系统会在 /dev 路径下增加一个新的 block设备。在和 usb 设备进行数据传输之前,需要将 usb 文件系统 进行挂载1. 检测:#查看磁盘信息,根据磁盘文件系统类型和大小查找 usb 驱动器 [arthur@localhost ~]$ sudo fdisk -l Disk /dev/sdb: 119.2 GiB, 128035676160 bytes, 250069680 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: dos Disk identifier: 0x522664d6 Device Boot Start End Sectors Size Id Type /dev/sdb1 * 2048 2099199 2097152 1G 83 Linux /dev/sdb2 2099200 250068991 247969792 118.2G 8e Linux LVM ... Disk /dev/sdc: 29.3 GiB, 31457280000 bytes, 61440000 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: dos Disk identifier: 0x50886156 Device Boot Start End Sectors Size Id Type /dev/sdc1 * 63 61439999 61439937 29.3G c W95 FAT32 (LBA)2. 创建挂载点: 挂载点可以是主机文件系统内已经存在的或新创建的路径[arthur@localhost ~]$ sudo mkdir /media/usbC [arthur@localhost ~]$ ll /media/usbC total 0 3. 挂载:[arthur@localhost ~]$ sudo mount /dev/sdc1 /media/usbC/ #查看挂载后 usb disk 内的目录文件信息 [arthur@localhost ~]$ ll /media/usbC/ total 926512 -rw-r--r--. 1 arthur arthur 948731419 Mar 10 04:00 hg19.fa.gz drwxr-xr-x. 2 arthur arthur 16384 Aug 3 07:15 'System Volume Information' #查看挂载信息 [arthur@localhost ~]$ mount | grep sdc1 /dev/sdc1 on /run/media/arthur/disk type vfat (rw,nosuid,nodev,relatime,uid=1000,gid=1000,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,showexec,utf8,flush,errors=remount-ro,uhelper=udisks2) /dev/sdc1 on /media/usbC type vfat (rw,relatime,uid=1000,gid=1000,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,showexec,utf8,flush,errors=remount-ro) 4. 存取数据4.1 写入数据:[arthur@localhost usbC]$ echo Hello,Aliyun! >test.txt [arthur@localhost usbC]$ ll total 926528 -rw-r--r--. 1 arthur arthur 948731419 Mar 10 04:00 hg19.fa.gz drwxr-xr-x. 2 arthur arthur 16384 Aug 3 07:15 'System Volume Information' -rw-r--r--. 1 arthur arthur 14 Sep 2 04:40 test.txt [arthur@localhost usbC]$ cat test.txt Hello,Aliyun!4.2 拷贝数据至主机:[arthur@localhost usbC]$ cp test.txt /home/arthur/Downloads/ [arthur@localhost usbC]$ ll /home/arthur/Downloads/ total 926500 -rw-r--r--. 1 arthur arthur 948731419 Mar 10 04:00 hg19.fa.gz -rw-r--r--. 1 arthur arthur 14 Sep 2 04:40 test.txt4.3 查看磁盘使用情况[arthur@localhost usbC]$ df -h Filesystem Size Used Avail Use% Mounted on devtmpfs 4.6G 0 4.6G 0% /dev ... /dev/sdc1 30G 905M 29G 4% /media/usbC5. 取消挂载#确保没有进程正在使用或者占用挂载点路径,否则会出现设备占用的警告 umount: target is busy #命令`umount`取消挂载 [arthur@localhost ~]$ sudo umount /media/usbC [sudo] password for arthur: #查看当前系统挂载文件系统 ##手动创建的挂载点已无设备,此处出现的挂载信息为系统检测到的 usb disk,当手动弹出 usb disk 之后,该条信息消失 [arthur@localhost ~]$ mount | grep sdc1 /dev/sdc1 on /run/media/arthur/disk type vfat (rw,nosuid,nodev,relatime,uid=1000,gid=1000,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,showexec,utf8,flush,errors=remount-ro,uhelper=udisks2) [arthur@localhost ~]$ ll /media/usbC/ total 0 [arthur@localhost ~]$ sudo umount /media/usbC umount: /media/usbC: not mounted. #未手动弹出 usb disk [arthur@localhost ~]$ ll /run/media/arthur/disk/ total 926528 -rw-r--r--. 1 arthur arthur 948731419 Mar 10 04:00 hg19.fa.gz drwxr-xr-x. 2 arthur arthur 16384 Aug 3 07:15 'System Volume Information' -rw-r--r--. 1 arthur arthur 14 Sep 2 04:40 test.txt #弹出 usb disk 之后 [arthur@localhost ~]$ ll /run/media/arthur/ total 0 [arthur@localhost ~]$ mount | grep sdc1 [arthur@localhost ~]$ 6. 在 Linux 中永久性添加挂载点编辑 fstab 文件, 可以使系统重启后仍然保留此文件挂载点[arthur@localhost ~]$ tail /etc/fstab # Accessible filesystems, by reference, are maintained under '/dev/disk/'. # See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. # # After editing this file, run 'systemctl daemon-reload' to update systemd # units generated from this file. # /dev/mapper/cl-root / xfs defaults 0 0 UUID=bf580455-80bf-4ca2-98fc-1a663e9a73e0 /boot ext4 defaults 1 2 /dev/mapper/cl-home /home xfs defaults 0 0 /dev/mapper/cl-swap swap swap defaults 0 0 添加如下:/dev/sdc1 /media/usbC vfat default 0 0拓展:从etc/fstab获取的 block device 名字来指代 usb 设备可能并不是一个好的长久的方案。随着 linux 系统中 USB drives 数量的变化, block device 名称可能会发生改变。尽管使用 UUID raw block device name 同样是临时性方案,但 使用 UUID 会相对更佳。reference:[1]How to mount usb drive in linux.linuxconfig
区别:命令set,env,export 均在 bash 中执行。set : 改变 shell 属性和定位参数值; 显示本地变量、当前shell 的变量,包括当前用户变量 export是bash的内建指令;显示和设置环境变量。VAR=whatever是变量定义的bash语法; env显示环境变量,显示当前用户变量;本身是一个程序,当env被调用/执行时,实际触发以下过程:命令env作为一个新的进程被执行env修改环境调用被用作参数的命令(command),env进程被命令(command)进程取代 举例:[arthur@localhost blog]$ env GREP_OPTIONS='-v' grep one test.txt grep: warning: GREP_OPTIONS is deprecated; please use an alias or script #this is the test file to analyse the difference between env,set and export line two上述命令将启动两个新的进程:(i) env 和 (ii) grep (事实上第二个进程会取代第一个进程)。从grep进程的角度来看,命令执行的结果等同于:$ GREP_OPTIONS='-v' grep one test.txt然而,如果你在 bash 之外或者不想启动另一个 shell ,可以使用这个习惯用法(例如,当你使用exec()系列函数而不是system()调用时)场景:命令grep 使用称为GREP_OPTIONS 的环境变量来设置命令默认选项。1. 创建测试文件 test.txt[arthur@localhost blog]$ vi test.txt [arthur@localhost blog]$ cat test.txt #this is the test file to analyse the difference between env,set and export line one line two2. 运行grep 命令$ grep --help Usage: grep [OPTION]... PATTERN [FILE]... Search for PATTERN in each FILE. Example: grep -i 'hello world' menu.h main.c -v, --invert-match select non-matching lines[arthur@localhost blog]$ grep one test.txt line one [arthur@localhost blog]$ grep -v one test.txt #this is the test file to analyse the difference between env,set and export line two3. 使用环境变量设置选项:3.1 若不使用export命令,设置环境变量时,该变量将不能被继承[arthur@localhost blog]$ GREP_OPTIONS='-v' [arthur@localhost blog]$ grep one test.txt line one可以发现,选项 -v 并没有传递给命令grep。 使用场景:当你设置一个变量仅仅是为了供shell使用的时候,你可以用上述变量命名方式。例如,当你仅仅是为了实现以下功能时:for i in *;do, 你并不想使用export $i来设置变量。3.2 传递变量至特定的命令行环境[arthur@localhost blog]$ GREP_OPTIONS='-v' grep one test.txt grep: warning: GREP_OPTIONS is deprecated; please use an alias or script #this is the test file to analyse the difference between env,set and export line two使用场景: 临时性改变所启动程序的特定实例的环境3.3 使用export命令使变量可被子程序继承[arthur@localhost blog]$ export GREP_OPTIONS='-v' [arthur@localhost blog]$ grep one test.txt grep: warning: GREP_OPTIONS is deprecated; please use an alias or script #this is the test file to analyse the difference between env,set and export line two使用场景: 为shell 中随后启动的进程设置变量的最常见方法拓展: #!/usr/bin/envenv 不需要全路径指向一个程序,是因为它使用execvp()函数,这个函数类似shell的工作原理,通过PATH变量来寻找路径,然后,用命令的运行来取代自身。因此,它可以用于找出解释器(如 perl 或者 python)在路径 中的位置。因而经常使用 #!/usr/bin/env interpreter 而不是 #!/usr/bin/interpreter 此外, 通过修改当前路径,可以影响到具体哪个 python 变量的调用,这将会使如下成为可能:echo -e `#!/usr/bin/bash\n\necho I am an evil interpreter!` > python chmod a+x ./python export PATH=. python并不会触发 python 的运行,上述代码将输出结果:I am an evil interpreter!reference:[1] what's the difference between set,export, and env and when should I use each ?.stackExchange.
磁盘管理Part1 :1. 磁盘或者笔记本设备与Linux系统的适配性 :不同厂商硬件设备对Linux系统的支持与否,可以查看以下链接: RedHat 硬件支持 :Red Hat certified hardware. hereRed Hat certified software. hereRed Hat certified cloud and service providers. hereCertified workstations. here不同品牌型号笔记本对Linux系统的支持与否,可查看here2. Linux系统中的硬件设备 在Linux系统中,硬件设备均被视为“文件”,而大部分硬件设备文件存放于目录/dev之下。以SATA接口的硬盘为例,硬盘的名称形式为/dev/sd[a-d] . 另外,the linux kernel archives 的详细信息可以查看 here另外,可以借助虚拟机创建虚拟化环境用于linux系统的运行,虚拟化技术可以参考 Fedora Document3. 磁盘分区实体机器使用的磁盘名,形如/dev/sd[a-d], 而虚拟机环境下,可能会使用形如/dev/vd[a-p]磁盘名,虚拟机以virtlO接口接入磁盘时,使用virtlO界面时,磁盘文件名形如/dev/vd[a-p], 而SATA/USB/SAS等磁盘接口都是使用SCSI模块来驱动的,因此这些接口的磁盘设备名称都为/dev/sd[a-p]的格式。根据Linux检测到的磁盘顺序对磁盘设备加以命名。磁盘第一个扇区存储整个磁盘重要信息,这种分区格式称为MBR(Master Boot Record)。随着磁盘容量的增加,部分操作系统已经不能读取大于2G的磁盘文件,因此出现了新的磁盘分区格式GPT(GUID partition table)。磁盘结构图示: 包括: platter/ actuator arm/disk sector/ track/ track sector/ cluster 分区表的两种格式:MSDOS(MBR) 分区表格式: GUID partition table, GPT 磁盘分区表4. 磁盘操作命令1.fdisk : Display or manipulate a disk partition table. # fdisk --help Usage: fdisk [options] <disk> change partition table fdisk [options] -l [<disk>] list partition table(s) Display or manipulate a disk partition table. Options: -b, --sector-size <size> physical and logical sector size -B, --protect-boot don't erase bootbits when creating a new label -c, --compatibility[=<mode>] mode is 'dos' or 'nondos' (default) -L, --color[=<when>] colorize output (auto, always or never) colors are enabled by default -l, --list display partitions and exit -o, --output <list> output columns -t, --type <type> recognize specified partition table type only -u, --units[=<unit>] display units: 'cylinders' or 'sectors' (default) -s, --getsz display device size in 512-byte sectors [DEPRECATED] --bytes print SIZE in bytes rather than in human readable format -w, --wipe <mode> wipe signatures (auto, always or never) -W, --wipe-partitions <mode> wipe signatures from new partitions (auto, always or never) -C, --cylinders <number> specify the number of cylinders -H, --heads <number> specify the number of heads -S, --sectors <number> specify the number of sectors per track -h, --help display this help -V, --version display version # fdisk -l Disk /dev/sda: 45 GiB, 48318382080 bytes, 94371840 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: dos Disk identifier: 0x0f689d0d Device Boot Start End Sectors Size Id Type /dev/sda1 * 2048 2099199 2097152 1G 83 Linux /dev/sda2 2099200 41943039 39843840 19G 8e Linux LVM /dev/sda3 41943040 94371839 52428800 25G 8e Linux LVM Disk /dev/mapper/cl-root: 37 GiB, 39724253184 bytes, 77586432 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk /dev/mapper/cl-swap: 2 GiB, 2147483648 bytes, 4194304 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes df : Show information about the file system on which each FILE resides,or all file systems by default.$ df --help Mandatory arguments to long options are mandatory for short options too. -a, --all include pseudo, duplicate, inaccessible file systems -B, --block-size=SIZE scale sizes by SIZE before printing them; e.g., '-BM' prints sizes in units of 1,048,576 bytes; see SIZE format below --direct show statistics for a file instead of mount point -h, --human-readable print sizes in powers of 1024 (e.g., 1023M) -H, --si print sizes in powers of 1000 (e.g., 1.1G) -i, --inodes list inode information instead of block usage -k like --block-size=1K -l, --local limit listing to local file systems --no-sync do not invoke sync before getting usage info (default) --output[=FIELD_LIST] use the output format defined by FIELD_LIST, or print all fields if FIELD_LIST is omitted. -P, --portability use the POSIX output format --sync invoke sync before getting usage info --total elide all entries insignificant to available space, and produce a grand total -t, --type=TYPE limit listing to file systems of type TYPE -T, --print-type print file system type -x, --exclude-type=TYPE limit listing to file systems not of type TYPE -v (ignored) --help display this help and exit --version output version information and exit $ df -a Filesystem 1K-blocks Used Available Use% Mounted on sysfs 0 0 0 - /sys proc 0 0 0 - /proc devtmpfs 5002356 0 5002356 0% /dev securityfs 0 0 0 - /sys/kernel/security tmpfs 5020460 0 5020460 0% /dev/shm devpts 0 0 0 - /dev/pts tmpfs 5020460 10204 5010256 1% /run tmpfs 5020460 0 5020460 0% /sys/fs/cgroup cgroup 0 0 0 - /sys/fs/cgroup/systemd pstore 0 0 0 - /sys/fs/pstore bpf 0 0 0 - /sys/fs/bpf cgroup 0 0 0 - /sys/fs/cgroup/perf_event ... cgroup 0 0 0 - /sys/fs/cgroup/devices cgroup 0 0 0 - /sys/fs/cgroup/net_cls,net_prio cgroup 0 0 0 - /sys/fs/cgroup/pids ... configfs 0 0 0 - /sys/kernel/config /dev/mapper/cl-root 38782976 23065508 15717468 60% / selinuxfs 0 0 0 - /sys/fs/selinux debugfs 0 0 0 - /sys/kernel/debug hugetlbfs 0 0 0 - /dev/hugepages mqueue 0 0 0 - /dev/mqueue systemd-1 0 0 0 - /proc/sys/fs/binfmt_misc /dev/sda1 999320 146144 784364 16% /boot sunrpc 0 0 0 - /var/lib/nfs/rpc_pipefs tmpfs 1004092 1180 1002912 1% /run/user/42 tmpfs 1004092 6956 997136 1% /run/user/1000 gvfsd-fuse 0 0 0 - /run/user/1000/gvfs fusectl 0 0 0 - /sys/fs/fuse/connections lvs : Display information about logical volumesparted :# parted Usage: parted [OPTION]... [DEVICE [COMMAND [PARAMETERS]...]...] Apply COMMANDs with PARAMETERS to DEVICE. If no COMMAND(s) are given, run in interactive mode. OPTIONs: -h, --help displays this help message -l, --list lists partition layout on all block devices -m, --machine displays machine parseable output -s, --script never prompts for user intervention -v, --version displays the version -a, --align=[none|cyl|min|opt] alignment for new partitions COMMANDs: align-check TYPE N check partition N for TYPE(min|opt) alignment help [COMMAND] print general help, or help on COMMAND mklabel,mktable LABEL-TYPE create a new disklabel (partition table) mkpart PART-TYPE [FS-TYPE] START END make a partition name NUMBER NAME name partition NUMBER as NAME print [devices|free|list,all|NUMBER] display the partition table, available devices, free space, all found partitions, or a particular partition quit exit program rescue START END rescue a lost partition near START and END resizepart NUMBER END resize partition NUMBER rm NUMBER delete partition NUMBER select DEVICE choose the device to edit disk_set FLAG STATE change the FLAG on selected device disk_toggle [FLAG] toggle the state of FLAG on selected device set NUMBER FLAG STATE change the FLAG on partition NUMBER toggle [NUMBER [FLAG]] toggle the state of FLAG on partition NUMBER unit UNIT set the default unit to UNIT version display the version number and copyright information of GNU Parted lsblk : List information about block devices.# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 45G 0 disk ├─sda1 8:1 0 1G 0 part /boot ├─sda2 8:2 0 19G 0 part │ ├─cl-root 253:0 0 37G 0 lvm / │ └─cl-swap 253:1 0 2G 0 lvm [SWAP] └─sda3 8:3 0 25G 0 part └─cl-root 253:0 0 37G 0 lvm / sr0 11:0 1 1024M 0 rom pvs : Display information about physical volumes # pvs PV VG Fmt Attr PSize PFree /dev/sda2 cl lvm2 a-- <19.00g 0 /dev/sda3 cl lvm2 a-- <25.00g <5.00g resize2fs :# resize2fs resize2fs 1.45.6 (20-Mar-2020) Usage: resize2fs [-d debug_flags] [-f] [-F] [-M] [-P] [-p] device [-b|-s|new_size] [-S RAID-stride] [-z undo_file] xfs_growfs :Usage: xfs_growfs [options] mountpoint Options: -d grow data/metadata section -l grow log section -r grow realtime section -n don't change anything, just show geometry -i convert log from external to internal format -t alternate location for mount table (/etc/mtab) -x convert log from internal to external format -D size grow data/metadata section to size blks -L size grow/shrink log section to size blks -R size grow realtime section to size blks -e size set realtime extent size to size blks -m imaxpct set inode max percent to imaxpct -V print version information vgdisplay : Display volume group information vgdisplay [ -A|--activevolumegroups ] [ -c|--colon ] [ -C|--columns ] [ -o|--options String ] [ -S|--select String ] [ -s|--short ] [ -O|--sort String ] ...vgextend : Add physical volumes to a volume group vgextend VG PV ... [ -A|--autobackup y|n ] [ -f|--force ] [ -Z|--zero y|n ] [ -M|--metadatatype lvm2 ] [ --labelsector Number ] [ --metadatasize Size[m|UNIT] ] ... `lvdisplay : Display information about a logical volume lvdisplay [ -a|--all ] [ -c|--colon ] [ -C|--columns ] [ -H|--history ] [ -m|--maps ] [ -o|--options String ] [ -O|--sort String ] [ -S|--select String ] ...lvextend : Add space to a logical volume Extend an LV by a specified size. lvextend -L|--size [+]Size[m|UNIT] LV [ -l|--extents [+]Number[PERCENT] ] [ -r|--resizefs ] [ -i|--stripes Number ] [ -I|--stripesize Size[k|UNIT] ] [ --poolmetadatasize [+]Size[m|UNIT] ] [ COMMON_OPTIONS ] [ PV ... ] Extend an LV by specified PV extents. lvextend LV PV ... [ -r|--resizefs ] [ -i|--stripes Number ] [ -I|--stripesize Size[k|UNIT] ] [ COMMON_OPTIONS ] Extend a pool metadata SubLV by a specified size. lvextend --poolmetadatasize [+]Size[m|UNIT] LV_thinpool [ -i|--stripes Number ] [ -I|--stripesize Size[k|UNIT] ] [ COMMON_OPTIONS ] [ PV ... ] Extend an LV according to a predefined policy. lvextend --usepolicies LV_snapshot_thinpool [ -r|--resizefs ] [ COMMON_OPTIONS ] [ PV ... ] du -sch *Usage: du [OPTION]... [FILE]... or: du [OPTION]... --files0-from=F Summarize disk usage of the set of FILEs, recursively for directories. 虚拟机中磁盘管理 :Issue : CentOS8 in the VMware: No space left on device1. Issue Description :Initially, 20G Hard Disk(SCSI) was portioned to the CentOS8 in the virtual machine. For disk room occupied by the genome index file , genome reference ,rawdata and so on, and larger room would be required in the downstream analysis, so the disk resize would be rather necessary.2. Solutions :extend the logical volume : NOTE : backup of the virtual machine through snapshots or commands (e.g. cp )identify the device and confirm the size : fdisk -l :create a new primary partition :(1) operate on the new device :fdisk /dev/sd[a-p](2) print the partition table : press p(3) create a new primary partition: press n(4) primary type : press p(5) partition number , depending on the output of the partition table print. :(6) change system's partition ID : t(7) select the newly creation partition :(8) change the Hex Code of the partition for Linux LVM : type 8e(9) write the change to the partition table : press wreboot the system CentOS8verify the changes has been written into the partition table and the type of the new partition is 8e : fdisk -lconvert the new partition to a physical volume : pvcreate /dev/sd* extend the physical volume :(1) df : show information about the file system , identify the high used , low available and Mountpoint(2) vgdisplay: to check the volume group(3) extend physical volume : vgextend VolumeGroup /dev/sd*verify how many physical extents are available to the VolumeGroup : vgdisplay VolumeGroup | grep "Free"extend the logical volume :lvdisplay : to determine which logical volume to extend lvextend -L+*G /dev/VolumeGroup/LogicalVolume :expand the ext3 filesystem online , inside of logical volume :xfs_growfs dir : dir is the volume group mountpointverify that the dir file system has the new space available : df -h dir ----reference : 1.Extending a logical volume in a virtual machine running Red Hat or Cent OS 2.CentOS Manual Partitioning
centos8 系统通过 snaps 安装markdown编辑器 typoraSnaps 是打包了程序所有依赖的应用,可应用于桌面系统、云端和物联网,具有安装简易、安全、跨平台以及无需手动解决程序依赖问题。 Snaps 可以从 Snap Store 获取下载和安装。Snap 是一项应用程序及其依赖的捆绑集成包bundle,可在linux 发行版中无需修改就可以直接运行。 Snapd 是一项后台服务,可以自动管理和维护Snap 。Snapcraft 是用于构建和发布 Snap 的命令和框架。1. enable snapdSnap 适用于 Centos 7.6 版本及以上、 RHEL 7.6版本及以上,可以通过 Extra Packages for Enterprise Linux(EPEL) repository 仓库获取安装。EPEL 是一个 Fedora Special Interest Group , 负责创建、维护和管理Enterprise Linux一系列高质量additional packages, 包括但不限于 RHEL、CentOS、Scientific Linux(SL), Oracle Linux(OL).EPEL包通常基于它们的 Fedora 对应包,永远不会与 base Enterprise Linux 发行版中的包冲突或发生替换。 EPEL 使用与 Fedora 基本相同的infrastructures, 包括 buildsystem, bugzilla instance, update manager, mirror manager and more.1.1 添加 EPEL 仓库[arthur@localhost ~]$ yum repolist | grep epel epel Extra Packages for Enterprise Linux 8 - x86_64 epel-modular Extra Packages for Enterprise Linux Modular 8 - x86_64此处,显示当前系统已经添加 epel 仓库,若无,可以通过以下方式添加:[arthur@localhost ~]$ sudo yum install epel-release [sudo] password for arthur: Last metadata expiration check: 2:41:41 ago on Tue 31 Aug 2021 07:22:49 AM EDT. Package epel-release-8-10.el8.noarch is already installed. Dependencies resolved. ... Upgraded: epel-release-8-11.el8.noarch Complete!1.2 安装 Snap[arthur@localhost ~]$ sudo yum install snapd Last metadata expiration check: 2:42:12 ago on Tue 31 Aug 2021 07:22:49 AM EDT. ... Installing: snapd x86_64 2.51-1.el8 epel 17 M ... Installing dependencies: snap-confine x86_64 2.51-1.el8 epel 3.0 M snapd-selinux noarch 2.51-1.el8 epel 424 k ... Downloading Packages: (1/5): snapd-selinux-2.51-1.el8.noarch.rpm 952 kB/s | 424 kB 00:00 (2/5): selinux-policy-3.14.3-67.el8_4.1.noarch.rpm 1.7 MB/s | 628 kB 00:00 (3/5): snap-confine-2.51-1.el8.x86_64.rpm 1.6 MB/s | 3.0 MB 00:01 (4/5): snapd-2.51-1.el8.x86_64.rpm 3.9 MB/s | 17 MB 00:04 (5/5): selinux-policy-targeted-3.14.3-67.el8_4.1.noarch.rpm 3.1 MB/s | 15 MB 00:04 \------------------------------------------------------------------------------------------------------------------------------------------------------ Total 5.1 MB/s | 36 MB 00:06 ... Installed: snap-confine-2.51-1.el8.x86_64 snapd-2.51-1.el8.x86_64 snapd-selinux-2.51-1.el8.noarch Complete!1.3 启用 systemd 单元systemd单元负责管理 snap communication socket .systemd 是 Linux 操作系统中系统和服务的 manager. 它被设计为向后兼容 SysV init 脚本,并提供许多特性,比如在引导时并行启动系统服务、按需激活守护进程(daemons)或基于依赖关系的服务控制逻辑。 socket unit是 systemd unit 的一种类型,文件拓展名为.socket, 是进程间通信套接字。[arthur@localhost ~]$ sudo systemctl enable --now snapd.socket Created symlink /etc/systemd/system/sockets.target.wants/snapd.socket → /usr/lib/systemd/system/snapd.socket.1.4 创建/var/lib/snapd/snap 与 /snap 之间的符号连接symbolic link$ ln --help Usage: ln [OPTION]... [-T] TARGET LINK_NAME or: ln [OPTION]... TARGET or: ln [OPTION]... TARGET... DIRECTORY or: ln [OPTION]... -t DIRECTORY TARGET... In the 1st form, create a link to TARGET with the name LINK_NAME. In the 2nd form, create a link to TARGET in the current directory. In the 3rd and 4th forms, create links to each TARGET in DIRECTORY. Create hard links by default, symbolic links with --symbolic. By default, each destination (name of new link) should not already exist. When creating hard links, each TARGET must exist. Symbolic links can hold arbitrary text; if later resolved, a relative link is interpreted in relation to its parent directory. [arthur@localhost ~]$ sudo ln -s /var/lib/snapd/snap /snap 完成上述步骤之后,虽然官网说明文档提到可以退出登陆后再重新登入,但还是推荐重启系统,以确保 snap 路径更新成功。2. 安装 typora[arthur@localhost ~]$ sudo snap install typora 此后,就会一路自动化安装 typora . 安装完成后,需要在命令行启动 typora :$typorareference :[1] install typora on centos.snapcraft.[2] EPEL.fedoraproject.org.[3] managing services with systemd.redhat.com
shell 与 环境变量1. 查看系统可用的shell 类型 :[arthur@localhost ~]$ cat /etc/shells /bin/sh /bin/bash /usr/bin/sh /usr/bin/bash /usr/bin/tmux /bin/tmux /usr/bin/zsh /bin/zsh2. 查看用户登陆时获得的shell类型:[arthur@localhost ~]$ cat /etc/passwd root:x:0:0:root:/root:/bin/bash bin:x:1:1:bin:/bin:/sbin/nologin daemon:x:2:2:daemon:/sbin:/sbin/nologin adm:x:3:4:adm:/var/adm:/sbin/nologin ... rngd:x:973:973:Random Number Generator Daemon:/var/lib/rngd:/sbin/nologin [arthur@localhost ~]$ cat /etc/passwd | grep arthur arthur:x:1000:1000:arthur:/home/arthur:/bin/bash3. shell 的变量功能:3.1 可以使用echo来显示变量内容:[arthur@localhost ~]$ echo $PATH /home/arthur/.local/bin:/home/arthur/bin:/home/arthur/.local/bin:/home/arthur/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/var/lib/snapd/snap/bin [arthur@localhost ~]$ echo ${HOME} /home/arthur3.2 设置和修改变量内容:可对变量赋值、修改、增添、取消[arthur@localhost ~]$ vname1="This's is variable 1" [arthur@localhost ~]$ echo $vname1 This's is variable 1 #单引号识别错误 [arthur@localhost ~]$ vname2='This\'s\ is\ variable\ 2' | echo $vname2 > ^C #单引号识别错误 [arthur@localhost ~]$ vname2='This\'s\ is\ variable\ 2' > ^C [arthur@localhost ~]$ vname2=This\'s\ is\ variable\ 2 [arthur@localhost ~]$ echo $vname2 This's is variable 2 [arthur@localhost ~]$ echo $vname1 $vname2 This's is variable 1 This's is variable 2 #修改变量 [arthur@localhost ~]$ vname=variable [arthur@localhost ~]$ echo $vname variable [arthur@localhost ~]$ vname=variables [arthur@localhost ~]$ echo $vname variables #取消变量 [arthur@localhost ~]$ unset vname1 vname2 [arthur@localhost ~]$ echo $vname1 $vname2 变量内容删除、取代与替换:[arthur@localhost ~]$ echo $PATH /home/arthur/.local/bin:/home/arthur/bin:/home/arthur/.local/bin:/home/arthur/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/var/lib/snapd/snap/bin #变量path 定义 [arthur@localhost ~]$ path=${PATH} [arthur@localhost ~]$ echo $path /home/arthur/.local/bin:/home/arthur/bin:/home/arthur/.local/bin:/home/arthur/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/var/lib/snapd/snap/bin ####################################################################### ####################################################################### **删除** ##将首个含 /local/bin 的路径移除,`#`代表从变量内容的最前面开始向右删除,且仅删除最短的那个; `/`代表要被删除的部分,`#`代表由最前面开始向右删除,所以这里由`/`开始写起; [arthur@localhost ~]$ echo ${path#/*local/bin:} /home/arthur/bin:/home/arthur/.local/bin:/home/arthur/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/var/lib/snapd/snap/bin [arthur@localhost ~]$ echo $PATH /home/arthur/.local/bin:/home/arthur/bin:/home/arthur/.local/bin:/home/arthur/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/var/lib/snapd/snap/bin #可以发现,上述删除操作并未改变变量`path`内容 [arthur@localhost ~]$ echo $path /home/arthur/.local/bin:/home/arthur/bin:/home/arthur/.local/bin:/home/arthur/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/var/lib/snapd/snap/bin ####################################################################### ##删除前面所有目录,仅保留最后一个目录. 一个`#`代表仅删除最短的目录,两个`#`代表删除最长的目录 [arthur@localhost ~]$ echo ${path#/*:} /home/arthur/bin:/home/arthur/.local/bin:/home/arthur/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/var/lib/snapd/snap/bin [arthur@localhost ~]$ echo ${path} /home/arthur/.local/bin:/home/arthur/bin:/home/arthur/.local/bin:/home/arthur/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/var/lib/snapd/snap/bin [arthur@localhost ~]$ echo ${path##/*:} /var/lib/snapd/snap/bin [arthur@localhost ~]$ echo $path /home/arthur/.local/bin:/home/arthur/bin:/home/arthur/.local/bin:/home/arthur/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/var/lib/snapd/snap/bin [arthur@localhost ~]$ echo ${path#home*} /home/arthur/.local/bin:/home/arthur/bin:/home/arthur/.local/bin:/home/arthur/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/var/lib/snapd/snap/bin:home/arthur/bin [arthur@localhost ~]$ echo ${path} /home/arthur/.local/bin:/home/arthur/bin:/home/arthur/.local/bin:/home/arthur/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/var/lib/snapd/snap/bin:home/arthur/bin ######################################################################### #从后往前删除变量名称 `%` [arthur@localhost ~]$ echo $path /home/arthur/.local/bin:/home/arthur/bin:/home/arthur/.local/bin:/home/arthur/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/var/lib/snapd/snap/bin #删除最后一个目录,从`:` 到`bin`为止 [arthur@localhost ~]$ echo ${path%:*bin} /home/arthur/.local/bin:/home/arthur/bin:/home/arthur/.local/bin:/home/arthur/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin #只保留第一个目录, `%%` 代表最长的符合字串 [arthur@localhost ~]$ echo $path /home/arthur/.local/bin:/home/arthur/bin:/home/arthur/.local/bin:/home/arthur/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/var/lib/snapd/snap/bin [arthur@localhost ~]$ echo ${path%%:*bin} /home/arthur/.local/bin ######################################################################### ######################################################################### **取代** #将 `path` 变量内容 `sbin` 取代成大写 `SBIN` [arthur@localhost ~]$ echo ${path} /home/arthur/.local/bin:/home/arthur/bin:/home/arthur/.local/bin:/home/arthur/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/var/lib/snapd/snap/bin #两条斜线中间的为旧字符串 [arthur@localhost ~]$ echo ${path/sbin/SBIN} /home/arthur/.local/bin:/home/arthur/bin:/home/arthur/.local/bin:/home/arthur/bin:/usr/local/bin:/usr/local/SBIN:/usr/bin:/usr/sbin:/var/lib/snapd/snap/bin [arthur@localhost ~]$ echo ${path} /home/arthur/.local/bin:/home/arthur/bin:/home/arthur/.local/bin:/home/arthur/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/var/lib/snapd/snap/bin #仅一条斜线时,仅首个变量内容会发生取代 [arthur@localhost ~]$ echo ${path/bin/BIN} /home/arthur/.local/BIN:/home/arthur/bin:/home/arthur/.local/bin:/home/arthur/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/var/lib/snapd/snap/bin #两条斜线,所有匹配的内容都会发生取代 [arthur@localhost ~]$ echo ${path} /home/arthur/.local/bin:/home/arthur/bin:/home/arthur/.local/bin:/home/arthur/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/var/lib/snapd/snap/bin [arthur@localhost ~]$ echo ${path//bin/BIN} /home/arthur/.local/BIN:/home/arthur/BIN:/home/arthur/.local/BIN:/home/arthur/BIN:/usr/local/BIN:/usr/local/sBIN:/usr/BIN:/usr/sBIN:/var/lib/snapd/snap/BIN3.3 环境变量查看当前shell环境下的环境变量: env 或者export[arthur@localhost ~]$ env LANG=en_US.UTF-8 ... USER=arthur PWD=/home/arthur SSH_ASKPASS=/usr/libexec/openssh/gnome-ssh-askpass HOME=/home/arthur LOGNAME=arthur PATH=/home/arthur/.local/bin:/home/arthur/bin:/home/arthur/.local/bin:/home/arthur/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/var/lib/snapd/snap/bin自定义变量转为环境变量:export#export 变量 export PATH=${PATH}:/dir/to/des/bin3.4 变量的有效范围环境变量等价于全局变量(global variable); 自定义变量等价于局部变量(local variable)
2021年12月
2021年11月
2021年10月
2021年09月