在CodeStar中使用Django開發（Python3.6.8、Django2.2.9）
让我们使用CodeStar创建Django应用，并通过CodePipeline进行持续集成/持续交付。
开发环境
本地计算机 dì jì jī)
-
- git for windows
-
- anaconda 2019
-
- python 3.6.9
- Django 1.11.18 / 2.2.9
亚马逊云服务器 (AWS EC2)
-
- python 3.6.8
- Django 1.11.18 / 2.2.9
创建项目
在俄勒冈地区遇到错误
Your requested instance type (t2.micro) is not supported in your requested Availability Zone (us-west-2d). Please retry your request by not specifying an Availability Zone or choosing us-west-2a, us-west-2b, us-west-2c. (Service: AmazonEC2; Status Code: 400; Error Code: Unsupported; Request ID: 7ee63da9-fc85-4e2a-999c-629fe3997dbe)
只需一個選項，在中文中改述以下內容：
因為停止/啟動EC2會使頁面無法顯示，所以需要在啟動時設定腳本來啟動Django應用程序。後續解釋。
编辑项目（练习CodeCommit）。
3. 安装Windows版的Git。
打开Anaconda提示符。
创建django1和django2的虚拟环境。
conda create -n django1 python=3.6
conda create -n django2 python=3.6
6. IAM的CodeCommit配置
我在IAM组的访问权限如下所示。
AmazonEC2FullAccess
AWSCodeCommitFullAccess
AWSLambdaFullAccess
AmazonS3FullAccess
AmazonDynamoDBFullAccess
AmazonSageMakerFullAccess
AmazonSageMaker-ExecutionPolicy-20191208Txxxxxx
AWSCloudFormationFullAccess
7. 激活django1环境并克隆项目。询问设置好的用户名和密码。然后进行git配置。
(base) $ conda activate django1
(django1) $ git config --global user.email "s-fujimoto@knowledgecommunication.jp"
(django1) $ git config --global user.name "s-fujimoto"
(django1) $ git clone https://git-codecommit.ap-northeast-1.amazonaws.com/v1/repos/helloworld
(django1) $ cd helloworld
8. 创建develop分支 *不要直接提交到master。一旦创建项目，立即创建develop分支。
(django1) $ git branch develop
(django1) $ git checkout develop
Switched to branch 'develop'
切一个feature分支，但不直接将更改提交到develop分支。每次添加或更改新功能时都切一个feature分支。
(django1) $ git branch feature
(django1) $ git checkout feature
Switched to branch 'feature'
安装项目所需的库。
(django1) $ pip install -r requirements/dev.txt
Collecting Django==1.11.18
Using cached https://files.pythonhosted.org/packages/e0/eb/6dc122c6d0a82263bd26bebae3cdbafeb99a7281aa1dae57ca1f645a9872/Django-1.11.18-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Installing collected packages: pytz, Django
Successfully installed Django-1.11.18 pytz-2019.3
安装 Django 1.11.18。
11. 在本地启动 Django。
(django1) $ python manage.py runserver
Performing system checks...
System check identified no issues (0 silenced).
You have 13 unapplied migration(s). Your project may not work properly until you apply the migrations for app(s): admin, auth, contenttypes, sessions.
Run 'python manage.py migrate' to apply them.
December 23, 2019 - 18:15:46
Django version 1.11.18, using settings 'ec2django.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CTRL-BREAK.
[23/Dec/2019 18:15:54] "GET / HTTP/1.1" 200 6652
[23/Dec/2019 18:15:54] "GET /static/helloworld/css/styles.css HTTP/1.1" 200 2690
[23/Dec/2019 18:15:54] "GET /static/helloworld/js/set-background.js HTTP/1.1" 200 137
[23/Dec/2019 18:15:54] "GET /static/helloworld/css/gradients.css HTTP/1.1" 200 2133
[23/Dec/2019 18:15:54] "GET /static/helloworld/img/tweet.svg HTTP/1.1" 200 1418
[23/Dec/2019 18:15:54] "GET /favicon.ico HTTP/1.1" 404 77
13．使用 git status 查看程序的变更
(django1) $ git status
On branch feature
Untracked files:
(use "git add <file>..." to include in what will be committed)
db.sqlite3
ec2django/__pycache__/
helloworld/__pycache__/
helloworld/migrations/__pycache__/
nothing added to commit but untracked files present (use "git add" to track)
14. 由于存在pycache会导致CodePipline的部署失败，所以不需要它，请添加以下.gitignore文件（db.sqlite3是可选的）。
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
# C extensions
*.so
# Distribution / packaging
.Python
env/
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
.hypothesis/
# Translations
*.mo
*.pot
# Django stuff:
*.log
local_settings.py
# Flask stuff:
instance/
.webassets-cache
# Scrapy stuff:
.scrapy
# Sphinx documentation
docs/_build/
# PyBuilder
target/
# Jupyter Notebook
.ipynb_checkpoints
# pyenv
.python-version
# celery beat schedule file
celerybeat-schedule
# SageMath parsed files
*.sage.py
# dotenv
.env
# virtualenv
.venv
venv/
ENV/
# Spyder project settings
.spyderproject
.spyproject
# Rope project settings
.ropeproject
# mkdocs documentation
/site
# mypy
.mypy_cache/
.idea/
db.sqlite3
migrations/
15. feature 提交至特性
(django1) $ git add .
(django1) $ git commit -m "add .gitignore"
[feature 6d9a594] add .gitignore
1 file changed, 106 insertions(+)
create mode 100644 .gitignore
(django1) $ git push origin feature
Enumerating objects: 4, done.
Counting objects: 100% (4/4), done.
Delta compression using up to 8 threads
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 939 bytes | 939.00 KiB/s, done.
Total 3 (delta 1), reused 0 (delta 0)
To https://git-codecommit.ap-northeast-1.amazonaws.com/v1/repos/helloworld
* [new branch] feature -> feature
确认页面可见。
编辑项目（从Django1更新至Django2）
离开django1环境并激活django2环境。
(django1) $ conda deactivate
(base) $ conda activate django2
(django2) $
2. 编辑 requirements/common.txt
编辑之前
# dependencies common to all environments
Django==1.11.18
修改后
# dependencies common to all environments
Django==2.2.9
2. 安装图书馆
(django2) $ pip install -r requirements/dev.txt
Collecting Django==2.2.9
Using cached https://files.pythonhosted.org/packages/cb/c9/ef1e25bdd092749dae74c95c2707dff892fde36e4053c4a2354b2303be10/Django-2.2.9-py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting sqlparse
Using cached https://files.pythonhosted.org/packages/ef/53/900f7d2a54557c6a37886585a91336520e5539e3ae2423ff1102daf4f3a7/sqlparse-0.3.0-py2.py3-none-any.whl
Installing collected packages: pytz, sqlparse, Django
Successfully installed Django-2.2.9 pytz-2019.3 sqlparse-0.3.0
安装Django版本2.2.9
3．编辑ec2django/setting.py文件
修改前
MIDDLEWARE_CLASSES = [
'django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
]
LANGUAGE_CODE = 'en-us'
TIME_ZONE = 'UTC'
USE_I18N = True
USE_L10N = True
USE_TZ = True
改变后
MIDDLEWARE = [
'django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
]
LANGUAGE_CODE = 'ja'
TIME_ZONE = 'Asia/Tokyo'
USE_I18N = True
USE_L10N = True
USE_TZ = True
4. 编辑ec2django/urls.py文件。
修改之前
from django.conf import settings
from django.conf.urls.static import static
from django.conf.urls import include, url
from django.contrib import admin
from django.contrib.staticfiles.urls import staticfiles_urlpatterns
urlpatterns = [
url(r'^admin/', admin.site.urls),
url(r'^', include('helloworld.urls')),
] + static(settings.STATIC_URL, document_root=settings.STATIC_ROOT)
更改后
from django.conf import settings
from django.conf.urls.static import static
from django.conf.urls import include, url
from django.contrib import admin
from django.contrib.staticfiles.urls import staticfiles_urlpatterns
from django.urls import path, include
urlpatterns = [
path('admin/', admin.site.urls),
path('', include('helloworld.urls')),
] + static(settings.STATIC_URL, document_root=settings.STATIC_ROOT)
将helloworld/templates/index.html移动到helloworld/templates/helloworld/index.html中。
6. 世界你好/urls.py
前一次变更
# helloworld/urls.py
from django.conf.urls import url
from django.conf.urls.static import static
from helloworld import views
urlpatterns = [
url(r'^$', views.HomePageView.as_view()),
]
改变之后 zhī
# helloworld/urls.py
from django.urls import path
from django.conf.urls import url
from django.conf.urls.static import static
from helloworld import views
urlpatterns = [
path('', views.HomePageView.as_view(), name='index'),
]
七. helloworld/views.py (请提供一个选项)
变更之前
# helloworld/views.py
from django.shortcuts import render
from django.views.generic import TemplateView
# Create your views here.
class HomePageView(TemplateView):
def get(self, request, **kwargs):
return render(request, 'index.html', context=None)
改动后
# helloworld/views.py
from django.shortcuts import render
from django.views.generic import TemplateView
from django.views import generic
# Create your views here.
class HomePageView(generic.TemplateView):
template_name = 'helloworld/index.html'
8. 在本地启动Django，并且通过浏览器访问 http://127.0.0.1:8000/，如果能够看到相同的页面，则表示正常。
(django2) $ python manage.py runserver
Performing system checks...
System check identified no issues (0 silenced).
You have 17 unapplied migration(s). Your project may not work properly until you apply the migrations for app(s): admin, auth, contenttypes, sessions.
Run 'python manage.py migrate' to apply them.
December 23, 2019 - 19:27:46
Django version 2.2.9, using settings 'ec2django.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CTRL-BREAK.
[23/Dec/2019 19:27:49] "GET / HTTP/1.1" 200 6652
[23/Dec/2019 19:27:49] "GET /static/helloworld/js/set-background.js HTTP/1.1" 200 137
[23/Dec/2019 19:27:49] "GET /static/helloworld/css/gradients.css HTTP/1.1" 200 2133
[23/Dec/2019 19:27:49] "GET /static/helloworld/css/styles.css HTTP/1.1" 200 2690
[23/Dec/2019 19:27:49] "GET /static/helloworld/img/tweet.svg HTTP/1.1" 200 1418
9．将功能提交到feature分支中，然后合并至develop分支，最后再合并至master分支。
(django2) $ git status
On branch feature
Changes not staged for commit:
(use "git add/rm <file>..." to update what will be committed)
(use "git restore <file>..." to discard changes in working directory)
modified: ec2django/settings.py
modified: ec2django/urls.py
deleted: helloworld/templates/index.html
modified: helloworld/urls.py
modified: helloworld/views.py
modified: requirements/common.txt
Untracked files:
(use "git add <file>..." to include in what will be committed)
helloworld/templates/helloworld/
no changes added to commit (use "git add" and/or "git commit -a")
(django2) $ git add .
(django2) $ git commit -m "django1.11.18 -> django2.2.9"
[feature 2226d88] django1.11.18 -> django2.2.9
6 files changed, 12 insertions(+), 12 deletions(-)
rename helloworld/templates/{ => helloworld}/index.html (100%)
(django2) $ git push origin feature
Enumerating objects: 21, done.
Counting objects: 100% (21/21), done.
Delta compression using up to 8 threads
Compressing objects: 100% (10/10), done.
Writing objects: 100% (11/11), 1.27 KiB | 648.00 KiB/s, done.
Total 11 (delta 5), reused 0 (delta 0)
To https://git-codecommit.ap-northeast-1.amazonaws.com/v1/repos/helloworld
* [new branch] feature -> feature
14. 请继续点击以进行安全警告。
17．启动Django
[ec2-user@ip-172-31-14-214 ~]$ sudo su
[root@ip-172-31-14-214 ec2-user]# source /home/ec2-user/environment/bin/activate
(environment) [root@ip-172-31-14-214 ec2-user]# /usr/local/bin/supervisord -c /home/ec2-user/supervisord.conf
Error: Another program is already listening on a port that one of our HTTP servers is configured to use. Shut this program down first before starting supervisord.
For help, use /usr/local/bin/supervisord -h
(environment) [root@ip-172-31-14-214 ec2-user]# pkill supervisord
(environment) [root@ip-172-31-14-214 ec2-user]# /usr/local/bin/supervisord -c /home/ec2-user/supervisord.conf
由于无法访问第18页，因此需要检查日志。
(environment) [root@ip-172-31-14-214 ec2-user]# cat /var/log/django-application-stderr.log
・・・
File "/home/ec2-user/environment/local/lib/python3.6/site-packages/django/db/backends/sqlite3/base.py", line 63, in check_sqlite_version
raise ImproperlyConfigured('SQLite 3.8.3 or later is required (found %s).' % Database.sqlite_version)
django.core.exceptions.ImproperlyConfigured: SQLite 3.8.3 or later is required (found 3.7.17).
[2019-12-23 19:54:59 +0900] [25438] [INFO] Worker exiting (pid: 25438)
[2019-12-23 10:54:59 +0000] [25435] [INFO] Shutting down: Master
[2019-12-23 10:54:59 +0000] [25435] [INFO] Reason: Worker failed to boot.
由于升级到Django2，需要更新SQLite3的版本。
19. SQLite3 的更新
如果在Django2.2开发服务器启动时遇到SQLite3错误，可以参考以下文章解决。
(environment) [root@ip-172-31-14-214 ec2-user]# wget https://www.sqlite.org/2019/sqlite-autoconf-3280000.tar.gz
(environment) [root@ip-172-31-14-214 ec2-user]# tar xvfz sqlite-autoconf-3280000.tar.gz
(environment) [root@ip-172-31-14-214 ec2-user]# cd sqlite-autoconf-3280000
(environment) [root@ip-172-31-14-214 ec2-user]# ./configure --prefix=/usr/local
(environment) [root@ip-172-31-14-214 ec2-user]# make
(environment) [root@ip-172-31-14-214 ec2-user]# sudo make install
(environment) [root@ip-172-31-14-214 ec2-user]# sudo find /usr/ -name sqlite3
(environment) [root@ip-172-31-14-214 ec2-user]# cd ../
(environment) [root@ip-172-31-14-214 ec2-user]# rm sqlite-autoconf-3280000.tar.gz
(environment) [root@ip-172-31-14-214 ec2-user]# rm -rf ./sqlite-autoconf-3280000
(environment) [root@ip-172-31-14-214 ec2-user]# /usr/local/bin/sqlite3 --version
3.28.0 2019-04-16 19:49:53 884b4b7e502b4e991677b53971277adfaf0a04a284f8e483e2553d0f83156b50
(environment) [root@ip-172-31-14-214 ec2-user]# /usr/bin/sqlite3 --version
3.7.17 2013-05-20 00:56:22 118a3b35693b134d56ebd780123b7fd6f1497668
(environment) [root@ip-172-31-14-214 ec2-user]# sqlite3 --version
3.7.17 2013-05-20 00:56:22 118a3b35693b134d56ebd780123b7fd6f1497668
(environment) [root@ip-172-31-14-214 ec2-user]# sudo mv /usr/bin/sqlite3 /usr/bin/sqlite3_old
(environment) [root@ip-172-31-14-214 ec2-user]# sudo ln -s /usr/local/bin/sqlite3 /usr/bin/sqlite3
(environment) [root@ip-172-31-14-214 ec2-user]# export LD_LIBRARY_PATH="/usr/local/lib"
(environment) [root@ip-172-31-14-214 ec2-user]# python
Python 3.6.8 (default, Oct 14 2019, 21:22:53)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
>>> sqlite3.sqlite_version
'3.28.0'
>>> exit()
(environment) [root@ip-172-31-14-214 ec2-user]#
只要能启动Django并能查看页面，就算OK。
(environment) [root@ip-172-31-14-214 ec2-user]# pkill supervisord
(environment) [root@ip-172-31-14-214 ec2-user]# /usr/local/bin/supervisord -c /home/ec2-user/supervisord.conf
在EC2重新启动时，设置启动Django。
1. 修改 supervisord.conf 文件（在本地进行编辑，然后将更改合并到主服务器上）。
修改前
[program:djangoproject]
command = environment/bin/gunicorn -b 0.0.0.0:80 ec2django.wsgi
更改后
[program:djangoproject]
command = /home/ec2-user/environment/bin/gunicorn -b 0.0.0.0:80 ec2django.wsgi
2．服务的注册
(rephrased in Chinese)
(environment) [root@ip-172-31-14-214 ec2-user]# vi /etc/init.d/helloworld
#!/bin/sh
# chkconfig: 2345 99 10
# description: start helloworld
# processname: helloworld
start() {
echo "start"
source /home/ec2-user/environment/bin/activate
export LD_LIBRARY_PATH="/usr/local/lib"
/usr/local/bin/supervisord -c /home/ec2-user/supervisord.conf
}
stop() {
echo "stop"
pkill supervisord
}
case "$1" in
start)
start
;;
stop)
stop
;;
restart)
stop
start
;;
esac
exit 0
(environment) [root@ip-172-31-14-214 ec2-user]# chkconfig --add helloworld
(environment) [root@ip-172-31-14-214 ec2-user]# chkconfig helloworld on
(environment) [root@ip-172-31-14-214 ec2-user]# chmod u+x /etc/init.d/helloworld
(environment) [root@ip-172-31-14-214 ec2-user]# service helloworld restart
如果页面可见，无论是重新启动还是EC2的停止和启动，都可以。
(environment) [root@ip-172-31-14-214 ec2-user]# sudo reboot
当停止或启动EC2时，请注意IP地址会发生变化。
4. 修改 feature 的 supervisord.conf 文件并提交，将 feature 分支合并到 develop 分支，再将 develop 分支合并到 master 分支。
(django2) $ git status
On branch feature
Changes not staged for commit:
(use "git add <file>..." to update what will be committed)
(use "git restore <file>..." to discard changes in working directory)
modified: supervisord.conf
no changes added to commit (use "git add" and/or "git commit -a")
(django2) $ git add .
(django2) D:\Users\s-fujimoto\CodeStar\helloworld>git commit -m "modified supervisord.conf"
[feature 38c0e8c] modified supervisord.conf
1 file changed, 1 insertion(+), 1 deletion(-)
(django2) D:\Users\s-fujimoto\CodeStar\helloworld>git push origin feature
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Delta compression using up to 8 threads
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 325 bytes | 325.00 KiB/s, done.
Total 3 (delta 2), reused 0 (delta 0)
To https://git-codecommit.ap-northeast-1.amazonaws.com/v1/repos/helloworld
* [new branch] feature -> feature
5. 当CodePipline完成且页面可见时，就可以了。
当无法看到时，请重新启动EC2或通过SSH连接并检查日志并重新启动服务。
(environment) [root@ip-172-31-14-214 ec2-user]# cat /var/log/django-application-stderr.log
(environment) [root@ip-172-31-14-214 ec2-user]# cat /var/log/django-application-stdout.log
(environment) [root@ip-172-31-14-214 ec2-user]# service helloworld restart
强制部署
由于一次部署失败的状态再次尝试仍然失败。因此，我们决定强行部署，然后将其合并到主分支，这样就成功了。
由于未在appspec.yml和buildspec.yml中添加文件，导致陷入泥潭的故事
使用自定义用户模型（略）
由於 db.sqlite3 被 gitignore，所以透過 Tera Term 進行傳送。
關於 AWS EC2（Linux 系統）的連線方式與文件傳輸方法。
需要更新数据库
python manage.py makemigrations helloworld
python manage.py migrate
刪除專案
总结
最近我觉得Django应用开发变得轻松多了，我们要加油进行CICD。
