记录关于Devops运维,虚拟化容器云计算,数据库,网络安全等各方面问题。
harbor2.0.1安装部署一、下载安装包wget https://github.com/goharbor/harbor/releases/download/v2.0.1/harbor-online-installer-v2.0.1.tgz二、解压tar xf harbor-online-installer-v2.0.1.tgz -C /usr/local/三、配置https访问1.创建目录mkdir /data/harbor/cert -p
cd /data/harbor/cert配置证书颁发机构1.生成CA证书私钥openssl genrsa -out ca.key 40962.生成CA证书openssl req -x509 -new -nodes -sha512 -days 3650 \
-subj "/C=CN/ST=Shanghai/L=Shanghai/O=soulchild/OU=myharbor/CN=registry.com" \
-key ca.key \
-out ca.crt字段含义:C:国家ST:省份L:城市O:组织单位OU:其他内容CN:一般填写域名配置服务器证书生成私钥openssl genrsa -out registry.com.key 40962.生成证书签名请求(CSR)openssl req -sha512 -new \
-subj "/C=CN/ST=Shanghai/L=Shanghai/O=soulchild/OU=myharbor/CN=registry.com" \
-key registry.com.key \
-out registry.com.csr生成x509 v3扩展文件cat > v3.ext <<-EOF
authorityKeyIdentifier=keyid,issuer
basicConstraints=CA:FALSE
keyUsage = digitalSignature, nonRepudiat...
redis哨兵模式vip三个哨兵配置如下port 26379
daemonize yes
pidfile "/var/run/redis-sentinel.pid"
logfile "/var/log/redis/redis-sentinel.log"
sentinel monitor mymaster 10.0.0.30 6379 2
sentinel down-after-milliseconds mymaster 5000
sentinel failover-timeout mymaster 20000
sentinel parallel-syncs mymaster 1
sentinel client-reconfig-script mymaster /server/scripts/redis_sentinel.shIP漂移脚本每个sentinel节点都需要添加vim /server/scripts/redis_sentinel.sh#!/bin/bash
MASTER_IP=${6}
VIP='10.0.0.25'
NETMASK='24'
INTERFACE='eth0'
MY_IP=`ip a s dev ${INTERFACE} | awk 'NR==3{split($2,ip,"/");print ip[1]}'`
if [ ${MASTER_IP} = ${MY_IP} ]; then
/sbin/ip addr add ${VIP}/${NETMASK} dev ${INTERFACE}
/sbin/arping -q -c 3 -A ${VIP} -I ${INTERFACE}
exit 0
else
/sbin/ip addr del ${VIP}/${NETMASK} dev ${INTERFACE}
exit 0
fi
exit 1redis-sentinel会向脚本传参mymaster observer start 旧主ip 6379 新主ip 6379添加执行权限c...
redis哨兵模式配置Redis 的 Sentinel 系统用于管理多个 Redis 服务器(instance), 该系统执行以下三个任务:监控(Monitoring): Sentinel 会不断地检查你的主服务器和从服务器是否运作正常。提醒(Notification): 当被监控的某个 Redis 服务器出现问题时, Sentinel 可以通过 API 向管理员或者其他应用程序发送通知。自动故障迁移(Automatic failover): 当一个主服务器不能正常工作时, Sentinel 会开始一次自动故障迁移操作, 它会将失效主服务器的其中一个从服务器升级为新的主服务器, 并让失效主服务器的其他从服务器改为复制新的主服务器; 当客户端试图连接失效的主服务器时, 集群也会向客户端返回新主服务器的地址, 使得集群可以使用新主服务器代替失效服务器。环境准备:redis-server:10.0.0.30:6379 主10.0.0.30:6389 从10.0.0.31:6379 从10.0.0.31:6380 从redis-sentinel:10.0.0.30:2637910.0.0.30:2638010.0.0.31:26379配置redis主从参考链接配置sentinelsentinel配置文件:daemonize yes
port 26379
logfile /var/log/redis/redis-sentinel.log
pidfile /var/run/redis-sentinel.pid
sentinel monitor mymaster 10.0.0.30 6379 2
sentinel d...
kafka2.0.1集群部署1.下载安装: wget https://archive.apache.org/dist/kafka/2.0.1/kafka_2.12-2.0.1.tgz
tar xf kafka_2.12-2.0.1.tgz
mv kafka_2.12-2.0.1 /usr/local/kafka 2.配置zookeeper:可参考https://soulchild.cn/1663.html/usr/local/kafka/config/zookeeper.properties3. 创建 /tmp/zookeeper/myid文件echo 1 > /tmp/zookeeper/myid4. 启动zookeepercd /usr/local/kafka/bin
./zookeeper-server-start.sh -daemon ../config/zookeeper.properties 5.配置kafka:文件路径:/usr/local/kafka/config/server.properties修改配置文件:#kafka集群的唯一标识,在改变IP地址,不改变broker.id的话不会影响消费者。
#Kafka在启动时会在zookeeper中/brokers/ids路径下创建一个以broker的id为名称的虚节点,Kafka的健康状态检查就依赖于此节点。
#当broker下线时,该虚节点会自动删除,其他broker或者客户端通过判断/brokers/ids路径下是否有此broker的id来确定该broker的健康状态。
broker.id=0
#配置监听地址和端口,listener_name://host_name:port
listeners=PLAINTEXT://elk1:9092
#topic不存在时,是否自动创建topic
auto.c...
redis主从复制和配置1.安装redis参考链接2.配置redismaster:port 6379
daemonize yes
bind 0.0.0.0
pidfile /var/run/redis_6379-master.pid
logfile "/var/log/redis/redis_6379-master.log"slave:port 6379
daemonize yes
bind 0.0.0.0
pidfile /var/run/redis_6379-slave.pid
logfile "/var/log/redis/redis_6379-slave.log"
# 指定redis-master的地址和端口
slaveof 10.0.0.30 63793.启动两台redisredis-server redis.conf4.测试登陆主redis[root@redis01 etc]# redis-cli
127.0.0.1:6379> set name soulchild
OK登陆从redis[root@redis02 ~]# redis-cli
127.0.0.1:6379> keys *
1) "name"
127.0.0.1:6379> get name
"soulchild"如果在主从复制架构中出现宕机的情况,需要分情况看:1)从Redis宕机这个相对而言比较简单,在Redis中从库重新启动后会自动加入到主从架构中,自动完成同步数据,这是因为在Redis2.8版本后就新增了增量复制功能,主从断线后恢复是通过增量复制实现的。所以这种情况无需担心。2)主Redis宕机这个情况相对而言就会复杂一些,需要以下2步才能完成:第一步,在从数据库中执行SL...
mongodb一主一从一Arbiter复制集部署安装mongodb可参考:https://soulchild.cn/1279.html 环境:10.0.0.40 mongodb-0110.0.0.41 mongodb-0210.0.0.42 mongodb-arb 修改配置文件:# 系统日志相关
systemLog:
destination: file
logAppend: true
path: /application/mongodb/log/mongod.log
# 数据存储相关
storage:
dbPath: /application/mongodb/data
journal:
enabled: true
# 网络相关
net:
port: 27017
bindIp: 0.0.0.0
# 进程控制相关
processManagement:
fork: true
# pidFilePath: /var/run/mongod.pid
# 安全配置
#security:
# authorization: enable
# keyFile: /application/mongodb/data/keyfile
#复制集配置
replication:
oplogSizeMB: 2048
replSetName: app_1 配置复制集#登陆主mongo
mongo
conf = {
_id: 'app_1',
members: [
{_id: 0, host: '10.0.0.40:27017',priority:10},
{_id: 1, host: '10.0.0.41:27017',priority:9},
...
hadoop3.2.1集群部署HDFS的守护进程分别是NameNode, SecondaryNameNode,DataNode.YARN的守护进程分别是ResourceManager, NodeManager,WebAppProxy.一、安装前准备:主机名IP角色hadoop-0110.0.0.150namenode、datanode、resourcemanager、nodemanagerhadoop-0210.0.0.151datanode、nodemanagerhadoop-0310.0.0.152datanode、nodemanager1.hosts解析:三台机器配置cat >> /etc/hosts <<EOF
10.0.0.150 hadoop-01
10.0.0.151 hadoop-02
10.0.0.152 hadoop-03
EOF2.配置免密登陆# 创建用户
useradd hadoop
su hadoop
ssh-keygen -P '' -t rsa -f ~/.ssh/id_rsa
cat ~/.ssh/id_rsa.pub >> authorized_keys
chmod 600 ~/.ssh/authorized_keys
# 发送配置到其他机器
for i in {151..152};do scp -rp ~/.ssh/ root@10.0.0.$i:/home/hadoop/;done
for i in {151..152};do ssh root@10.0.0.$i chown -R hadoop:hadoop /home/hadoop/;done3.准备安装包:三台机器配置mkdir /server/packages -p
cd /server/packages
[root@hadoop-01 packages]# ls /server/packages/
hadoop-3.2.1.tar.gz jdk-8u221-linux-x64.tar.gz
# 发送软件包到其他...
mysql主主复制+keepalived部署环境:mysql-master-01:10.0.0.30mysql-master-02:10.0.0.35vip:10.0.0.39 安装mysql5.7.20可参考:https://soulchild.cn/266.html 开始配置主主环境一、修改mysql配置master-01:[mysqld]
basedir=/application/mysql
datadir=/data/mysql
socket=/tmp/mysql.sock
server_id=1
port=3306
log-bin=mysql-bin
relay-log = mysql-relay-bin
replicate-wild-ignore-table=mysql.%
replicate-wild-ignore-table=information_schema.%
[mysql]
socket=/tmp/mysql.sock
prompt=master-01[\\d]>
[mysqld_safe]
log-error=/var/log/mysql.log master-02:[mysqld]
basedir=/application/mysql
datadir=/data/mysql
socket=/tmp/mysql.sock
server_id=11
port=3306
log-bin=mysql-bin
relay-log = mysql-relay-bin
replicate-wild-ignore-table=mysql.%
replicate-wild-ignore-table=information_schema.%
[mysql]
socket=/tmp/mysql.sock
prompt=master-02[\\d]>
[mysqld_safe]
log-error=/var/log/mysql.log 二、配置msater-01主,master-02从1.添加主从复制用户,master-01执行grant...
KVM利用nfs热迁移环境:KVM01:安装kvm、nfs、/opt/目录为磁盘文件目录KVM02:安装kvm、nfs host解析:10.0.0.11 kvm0110.0.0.12 kvm02 1.安装nfs(两台KVM都装)yum install -y nfs-utils
#服务端安装rpcbind
yum install -y rpcbind 2.kvm01配置nfs[root@kvm01 ~]# vim /etc/exports
#共享目录写虚拟磁盘文件目录
/opt 10.0.0.0/24(rw,rsync,no_root_squash)
[root@kvm01 ~]# systemctl start rpcbind
[root@kvm01 ~]# systemctl start nfs 3.KVM02挂载目录#注意挂载的目录要和原来的一致,否则会找不到磁盘文件
[root@kvm02 ~]# mount -t nfs 10.0.0.11:/opt /opt/ 4.开始迁移#查看当前虚拟机状态
[root@kvm01 ~]# virsh list
Id Name State
----------------------------------------------------
16 centos7 running --live 实时迁移--verbose 显示进度--unsafe 忽略安全#迁移
virsh migrate --live --verbose centos7 qemu+ssh://10.0.0.12/system --unsafe
sersync实现实时同步备份sersync github下载地址:https://github.com/wsgzao/sersync/(下载这个sersync2.5.4_64bit_binary_stable_final.tar.gz)sersync:监控目录的变化,推送到rsync服务器上这里检测到指定目录文件有变化时会自动调用rsync同步到backup服务器中(backup服务器安装rsync服务)1、安装解压后得到两个文件:sersync,confxml.xml,移动到/usr/local/sersync目录下(目录结构可以自己创建)[root@nfs01 ~]# tree /usr/local/sersync/
/usr/local/sersync/
├── bin
│ └── sersync
├── conf
│ └── confxml.xml
└── logs
#创建软连接,方便以后使用
[root@nfs01 sersync]# ln -s /usr/local/sersync/bin/sersync /bin/
[root@nfs01 sersync]# chmod +x /usr/local/sersync/bin/sersync2、修改配置文件打开confxml.xml配置文件<?xml version="1.0" encoding="ISO-8859-1" ?>
- <head version="2.5">
<host hostip="localhost" port="8008" />
<debug start="false" />
<fileSystem xfs="true" /> # 文件系统类...
最新评论