ZooKeeper 安装使用

前言

相信大家对 ZooKeeper 应该不算陌生，ZooKeeper 的使用和 ZooKeeper 集群的搭建并不复杂，这里分享自己看到的一些好文章以及使用 Ansible 快速部署和手动部署 ZooKeeper 的经验分享。

ZooKeeper 分布式环境的协调利器

更新历史

2018 年 09 月 27 日 - 初稿

阅读原文 - https://liaojiaxin158.github.io/post/zookeeper/

扩展阅读

ZooKeeper - https://zookeeper.apache.org/

ZooKeeper 简介

ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. All of these kinds of services are used in some form or another by distributed applications. Each time they are implemented there is a lot of work that goes into fixing the bugs and race conditions that are inevitable. Because of the difficulty of implementing these kinds of services, applications initially usually skimp on them ,which make them brittle in the presence of change and difficult to manage. Even when done correctly, different implementations of these services lead to management complexity when the applications are deployed.

ZooKeeper 是一个开源的为分布式应用提供分布式协调的服务。它公开了一组简单的原语，分布式应用程序可以基于这些原语实现更高级别的服务，包括同步、维护配置、组和命名。它的设计易于编程，它使用一个遵循文件系统中常见的目录树结构的数据模型。它在 Java 环境中运行，对 Java 和 C 都有绑定。
协调服务是出了名的难。它们特别容易出错，如竞态条件和死锁。ZooKeeper 背后的动机是让分布式应用从零开始实现一站式协调服务。

翻译：ZooKeeper OverView
https://www.cnblogs.com/f-ck-need-u/p/9231153.html

可能是全网把 ZooKeeper 概念讲的最清楚的一篇文章
https://github.com/Snailclimb/JavaGuide/blob/master/docs/system-design/framework/ZooKeeper.md

ZooKeeper 相关概念总结
 ZooKeeper 数据模型和常见命令

jdk

install jdk by ansible

https://galaxy.ansible.com/geerlingguy/java

# download
ansible-galaxy install geerlingguy.java
# create yaml file and config java version
vi install_jdk.yml

---
- hosts: all
  roles:
    - role: geerlingguy.java
      when: "ansible_os_family =='RedHat'"
      java_packages:
        - java-1.8.0-openjdk

install jdk manually

http://www.oracle.com/technetwork/java/javase/downloads/index.html

# yum
yum install java-1.8.0-openjdk
# rpm
yum localinstall -y jdk-8u181-linux-x64.rpm
# set env
vi /etc/profile

export JAVA_HOME=/usr/java/jdk1.8.0_181
export CLASSPATH=$CLASSPATH:$JAVA_HOME/lib:$JAVA_HOME/jre/lib 
export PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$PATH:$HOME/bin 

source /etc/profile

zookeeper

install zookeeper by ansible

ansible-zookeeper

zookeeper stand-alone

# install jdk
yum -y install java-1.8.0-openjdk
# download and install zookeeper
wget https://archive.apache.org/dist/zookeeper/zookeeper-3.4.14/zookeeper-3.4.14.tar.gz
tar -zxvf zookeeper-3.4.14.tar.gz -C /opt
ln -s /opt/zookeeper-3.4.14 /opt/zookeeper
cd /opt/zookeeper
# create zoo.cfg
cd /opt/zookeeper/conf
cp zoo_sample.cfg zoo.cfg
# start zookeeper
/opt/zookeeper/bin/zkServer.sh start
/opt/zookeeper/bin/zkCli.sh -server 127.0.0.1:2181

install zookeeper manually

https://zookeeper.apache.org/releases.html#download
https://archive.apache.org/dist/zookeeper/stable/

# install jdk
yum -y install java-1.8.0-openjdk
# download and install zookeeper
tar -zxvf zookeeper-3.4.14.tar.gz —C /opt
ln -s /opt/zookeeper-3.4.14 /opt/zookeeper
cd /opt/zookeeper
# create zoo.cfg
cd /opt/zookeeper/conf
cp zoo_sample.cfg zoo.cfg
vi zoo.cfg

tickTime=2000
dataDir=/var/lib/zookeeper
dataLogDir=/var/log/zookeeper
clientPort=2181
initLimit=5
syncLimit=2

server.1=zoo1:2888:3888
server.2=zoo2:2888:3888
server.3=zoo3:2888:3888

# create directory
mkdir /var/lib/zookeeper
mkdir /var/log/zookeeper
# create myid file in different zookeeper
cd /var/lib/zookeeper/

echo 1 > /var/lib/zookeeper/myid
echo 2 > /var/lib/zookeeper/myid
echo 3 > /var/lib/zookeeper/myid

# live
cat << EOF > /opt/zookeeper/conf/zoo.cfg
tickTime=2000
initLimit=5
syncLimit=2
dataDir=/data/zookeeper/data
dataLogDir=/var/log/zookeeper
clientPort=2181
server.1=192.168.188.120:2888:3888
server.2=192.168.188.121:2888:3888
server.3=192.168.188.122:2888:3888
EOF

# autostart
echo '/opt/zookeeper/bin/zkServer.sh start'  >> /etc/rc.local

# start
cd /usr/local/zookeeper/bin
./zkServer.sh start
# check
./zkServer.sh status
# create znode
./zkCli.sh -server 127.0.0.1:2181
create /codis codis
ls /
[zookeeper, codis]

Zookeeper Configuration

Platform: RHEL / CentOS 7
Java: Oracle JDK

Variables

zookeeper_version: 3.4.14
zookeeper_group: zookeeper
zookeeper_user: zookeeper
zookeeper_dir: /opt/zookeeper-{{zookeeper_version}} # or /opt/zookeeper
zookeeper_conf_dir: {{zookeeper_dir}} # or /etc/zookeeper
zookeeper_tarball_dir: /opt/src
data_dir: /var/lib/zookeeper
log_dir: /var/log/zookeeper
zookeeper_client_port: 2181
zookeeper_id: 1
zookeeper_leader_port: 2888
zookeeper_election_port: 3888

Default Ports

Port	Description
2181	Client connection port
2888	Quorum port for clustering
3888	Leader election port for clustering

conf/zoo.cfg

tickTime=2000
dataDir=/var/lib/zookeeper
dataLogDir=/var/log/zookeeper
clientPort=2181
initLimit=5
syncLimit=2
server.1=zoo1:2888:3888
server.2=zoo2:2888:3888
server.3=zoo3:2888:3888

tickTime
the basic time unit in milliseconds used by ZooKeeper. It is used to do heartbeats and the minimum session timeout will be twice the tickTime.

dataDir
the location to store the in-memory database snapshots and, unless specified otherwise, the transaction log of updates to the database.

clientPort
the port to listen for client connections

initLimit
is timeouts ZooKeeper uses to limit the length of time the ZooKeeper servers in quorum have to connect to a leader.

syncLimit
limits how far out of date a server can be from a leader.

In this example, the timeout for initLimit is 5 ticks at 2000 milleseconds a tick, or 10 seconds.

参考资料

ZooKeeper 系列文章
https://www.cnblogs.com/f-ck-need-u/p/7576137.html#zk

ZooKeeper 系列 (1)：安装搭建 ZooKeeper 环境
https://www.cnblogs.com/f-ck-need-u/p/9235308.html

Administrator’s Guide - a guide for system administrators and anyone else who might deploy ZooKeeper
https://zookeeper.apache.org/doc/current/zookeeperAdmin.html

HelloDog

Keep Calm and Carry On