By

前言

本文介绍了如何快速搭建一个 pypiserver，通过自建 pypiserver，我们可以解决网络环境不好，或者离线无法安装 python 包的问题。如果结合最新的 GitLab CI/CD 和 pipenv 我相信各位还可以玩出更多的花样。

pypiserver - minimal PyPI server for use with pip/easy_install

更新记录

2018 年 04 月 12 日 - 初稿

阅读原文 - https://liaojiaxin158.github.io/post/pypiserver/

扩展阅读

pypiserver - https://github.com/pypiserver/pypiserver

pypiserver 简介

pypiserver is a minimal PyPI compatible server for pip or easy_install. It is based on bottle and serves packages from regular directories. Wheels, bdists, eggs and accompanying PGP-signatures can be uploaded either with pip, setuptools, twine, pypi-uploader, or simply copied with scp.

pypiserver 服务端配置

如果你的 Linux 环境缺少 Python 2.7 可以参考我的文章直接离线升级至最新版本

Python 2.6 升级至 Python 2.7 的实践心得
https://liaojiaxin158.github.io/post/python-2-6-update-to-2-7/

pypiserver > 1.2.x works with python 2.7 and 3.3+ or pypy. Older python-versions may still work, but they are not tested. For legacy python versions, use pypiserver-1.1.x series.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
# 替换 pip 为阿里云，感概豆瓣的时代已经过去 
tee ~/.pip/pip.conf <<-'EOF'
[global]
index-url = https://mirrors.aliyun.com/pypi/simple/
[install]
trusted-host= mirrors.aliyun.com
EOF

# 直接在线安装 pypiserver
pip install pypiserver
# 离线下载 pypiserver
mkdir /tmp/pypiserver
pip install -d /tmp/pypiserver/ pypiserver

# Copy packages into this directory.
mkdir ~/packages
# Copy some packages into your ~/packages folder and then get your pypiserver up and running:
pypi-server -p 8080 ~/packages &

pypiserver 客户端配置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
## Download and Install hosted packages.
pip install  --extra-index-url http://localhost:8080/simple/ ...

# or
pip install --extra-index-url http://localhost:8080

## Search hosted packages
pip search --index http://localhost:8080 ...

# 个人推荐的配置 
tee ~/.pip/pip.conf <<-'EOF'
[global]
index-url = http://172.28.70.126/simple
extra-index-url = https://mirrors.aliyun.com/pypi/simple/
[install]
trusted-host = 172.28.70.126
EOF

pypiserver 进阶配置

pypiserver Running as a systemd service

https://github.com/pypiserver/pypiserver#running-as-a-systemd-service

Adjusting the paths and adding this file as pypiserver.service into your systemd/system directory will allow management of the pypiserver process with systemctl, e.g. systemctl start pypiserver.

More useful information about systemd can be found at https://www.digitalocean.com/community/tutorials/how-to-use-systemctl-to-manage-systemd-services-and-units

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
# 安装需要的包 
yum install nginx -y
pip install passlib pypiserver gunicorn

# 创建 pypiserver 服务方便服务启停管理 
tee /usr/lib/systemd/system/pypiserver.service <<-'EOF'
[Unit]
Description=gunicorn daemon
After=network.target

[Service]
PIDFile=/run/pypiserver.pid
ExecStart=/usr/local/bin/gunicorn -w16 \
    --pid /run/pypiserver.pid \
    -b :10012 \
    'pypiserver:app(root="/var/www/pypi")'
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/bin/kill -s TERM $MAINPID

[Install]
WantedBy=multi-user.target 
EOF

# Warning: pypiserver.service changed on disk. Run 'systemctl daemon-reload' to reload units.
systemctl daemon-reload

# 启动 pypiserver 服务 
systemctl enable pypiserver.service
systemctl start pypiserver.service
systemctl status pypiserver.service

# 停止 pypiserver 服务 
systemctl disable pypiserver.service
systemctl stop pypiserver.service
systemctl status pypiserver.service

[root@centos7 run]# systemctl status pypiserver.service
● pypiserver.service - gunicorn daemon
   Loaded: loaded (/usr/lib/systemd/system/pypiserver.service; disabled; vendor preset: disabled)
   Active: active (running) since Fri 2018-04-13 15:14:08 CST; 859ms ago
 Main PID: 22524 (gunicorn)
   CGroup: /system.slice/pypiserver.service
           ├─22524 /usr/local/bin/python /usr/local/bin/gunicorn -w16 --pid /run/pypiserver.pid -b :10012 pypiserver:app(root="/var/www/pypi")
           ├─22530 /usr/local/bin/python /usr/local/bin/gunicorn -w16 --pid /run/pypiserver.pid -b :10012 pypiserver:app(root="/var/www/pypi")
           ├─22531 /usr/local/bin/python /usr/local/bin/gunicorn -w16 --pid /run/pypiserver.pid -b :10012 pypiserver:app(root="/var/www/pypi")
           ├─22532 /usr/local/bin/python /usr/local/bin/gunicorn -w16 --pid /run/pypiserver.pid -b :10012 pypiserver:app(root="/var/www/pypi")
           ├─22533 /usr/local/bin/python /usr/local/bin/gunicorn -w16 --pid /run/pypiserver.pid -b :10012 pypiserver:app(root="/var/www/pypi")
           ├─22534 /usr/local/bin/python /usr/local/bin/gunicorn -w16 --pid /run/pypiserver.pid -b :10012 pypiserver:app(root="/var/www/pypi")
           ├─22535 /usr/local/bin/python /usr/local/bin/gunicorn -w16 --pid /run/pypiserver.pid -b :10012 pypiserver:app(root="/var/www/pypi")
           ├─22536 /usr/local/bin/python /usr/local/bin/gunicorn -w16 --pid /run/pypiserver.pid -b :10012 pypiserver:app(root="/var/www/pypi")
           ├─22537 /usr/local/bin/python /usr/local/bin/gunicorn -w16 --pid /run/pypiserver.pid -b :10012 pypiserver:app(root="/var/www/pypi")
           ├─22538 /usr/local/bin/python /usr/local/bin/gunicorn -w16 --pid /run/pypiserver.pid -b :10012 pypiserver:app(root="/var/www/pypi")
           ├─22539 /usr/local/bin/python /usr/local/bin/gunicorn -w16 --pid /run/pypiserver.pid -b :10012 pypiserver:app(root="/var/www/pypi")
           └─22540 /usr/local/bin/python /usr/local/bin/gunicorn -w16 --pid /run/pypiserver.pid -b :10012 pypiserver:app(root="/var/www/pypi")

Apr 13 15:14:08 centos7 gunicorn[22524]: [2018-04-13 15:14:08 +0000] [22531] [INFO] Booting worker with pid: 22531
Apr 13 15:14:08 centos7 gunicorn[22524]: [2018-04-13 15:14:08 +0000] [22532] [INFO] Booting worker with pid: 22532
Apr 13 15:14:08 centos7 gunicorn[22524]: [2018-04-13 15:14:08 +0000] [22533] [INFO] Booting worker with pid: 22533
Apr 13 15:14:08 centos7 gunicorn[22524]: [2018-04-13 15:14:08 +0000] [22534] [INFO] Booting worker with pid: 22534
Apr 13 15:14:08 centos7 gunicorn[22524]: [2018-04-13 15:14:08 +0000] [22535] [INFO] Booting worker with pid: 22535
Apr 13 15:14:08 centos7 gunicorn[22524]: [2018-04-13 15:14:08 +0000] [22536] [INFO] Booting worker with pid: 22536
Apr 13 15:14:08 centos7 gunicorn[22524]: [2018-04-13 15:14:08 +0000] [22537] [INFO] Booting worker with pid: 22537
Apr 13 15:14:08 centos7 gunicorn[22524]: [2018-04-13 15:14:08 +0000] [22538] [INFO] Booting worker with pid: 22538
Apr 13 15:14:08 centos7 gunicorn[22524]: [2018-04-13 15:14:08 +0000] [22539] [INFO] Booting worker with pid: 22539
Apr 13 15:14:08 centos7 gunicorn[22524]: [2018-04-13 15:14:08 +0000] [22540] [INFO] Booting worker with pid: 22540
[root@centos7 run]# netstat -tnlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:111             0.0.0.0:*               LISTEN      1/systemd           
tcp        0      0 192.168.122.1:53        0.0.0.0:*               LISTEN      1517/dnsmasq        
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      977/sshd            
tcp        0      0 127.0.0.1:631           0.0.0.0:*               LISTEN      978/cupsd           
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      1383/master         
tcp        0      0 127.0.0.1:6011          0.0.0.0:*               LISTEN      19378/sshd: root@pt 
tcp        0      0 0.0.0.0:10012           0.0.0.0:*               LISTEN      22524/python        
tcp6       0      0 :::111                  :::*                    LISTEN      1/systemd           
tcp6       0      0 :::22                   :::*                    LISTEN      977/sshd            
tcp6       0      0 ::1:631                 :::*                    LISTEN      978/cupsd           
tcp6       0      0 ::1:25                  :::*                    LISTEN      1383/master         
tcp6       0      0 ::1:6011                :::*                    LISTEN      19378/sshd: root@pt 

# 检查 pypiserver 服务 
cd /var/www/pypi
# 向仓库中添加 python package
[root@centos7 pypi]# pip download pypiserver
Collecting pypiserver
  Downloading https://mirrors.aliyun.com/pypi/packages/d7/78/5772432dad2b9e754ab92f4d301fa507069b9decc8c943c1b18c2043ff4f/pypiserver-1.2.1-py2.py3-none-any.whl (83kB)
    100% |████████████████████████████████| 92kB 643kB/s 
  Saved ./pypiserver-1.2.1-py2.py3-none-any.whl
Successfully downloaded pypiserver

[root@centos7 pypi]# ll
total 84
-rw-r--r-- 1 root root 83529 Apr 13 14:55 pypiserver-1.2.1-py2.py3-none-any.whl

# 搜索刚才下载的 package
[root@centos7 pypi]# pip search -i http://127.0.0.1:10012 pypiserver
pypiserver (1.2.1)  - 1.2.1
  INSTALLED: 1.2.1 (latest)

# 配置 nginx 做反向代理 
tee /etc/nginx/conf.d/pypi.conf <<-'EOF'
upstream pypiserver {
  server 127.0.0.1:10012;
}

server {
  listen 10087;

  # disable any limits to avoid HTTP 413 for large package uploads
  client_max_body_size 0;

  location / {
    proxy_pass http://pypiserver/;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

    # When setting up pypiserver behind other proxy, such as an Nginx instance, remove the below line if the proxy already has similar settings.
    proxy_set_header X-Forwarded-Proto $scheme;

    proxy_buffering off;
    proxy_request_buffering off;
  }

  location /packages/ {
    alias /var/www/pypi; # static file
  }
}
EOF

# 启动 nginx
systemctl enable nginx
systemctl start nginx
systemctl status nginx

# 检查 nginx 服务 
pip search -i http://172.28.79.126:10087 pypiserver
pypiserver (1.2.1)  - 1.2.1
  INSTALLED: 1.2.1 (latest)
文章目录
  1. 1. 前言
  2. 2. 更新记录
  3. 3. pypiserver 简介
  4. 4. pypiserver 服务端配置
  5. 5. pypiserver 客户端配置
  6. 6. pypiserver 进阶配置