Tango with Django 1 Chapter 1: Overview 1. Overview Aim of this tutorial: provides a pratical guide to web development using Django. What you will learn: ● Setup a development environment ● Setup the Django project ● Configure the Django project to serve static media ● Work with Django's ModelViewTemplate (MTV) design pattern ● Create database models ● Create forms ● Use the user authentication service ● Incorporate external services ● Include CSS and JavaScript ● Design and apply CSS ● Work with cookies and sessions ● Include more advanced functionality like AJAX ● Deploy your application Technoligies and services involved: ● Python ● Pip ● Django ● Git and github ● HTML ● CSS ● JavaScript, jQuery ● Postgres ● Twitter Bootstrap ● Heroku 2. The Ntier architecture of web applications The client tier: a web browser 1 Source: http://www.tangowithdjango.com/ 1
Transcript
Tango with Django 1
Chapter 1: Overview
1. Overview
Aim of this tutorial: provides a pratical guide to web development using Django.
What you will learn: Setup a development environment Setup the Django project Configure the Django project to serve static media Work with Django's ModelViewTemplate (MTV) design
pattern Create database models Create forms Use the user authentication service Incorporate external services Include CSS and JavaScript Design and apply CSS Work with cookies and sessions Include more advanced functionality like AJAX Deploy your application
Technoligies and services involved: Python Pip Django Git and github HTML CSS JavaScript, jQuery Postgres Twitter Bootstrap Heroku
A framework is a layered structure indicating what kind of programs can or should be built and how they would interrelate. It forces programmers to implement code in a way that promotes consistent coding. It also provides common libraries to facilitate/speed up web development.
Conceptually, a framework is different from libraries: our programs call the libraries; however, the framework calls our programs.
For a web framework, the framework do most of the work for handling an HTTP request and calls our program when necessary.
Such an architecture is also called Handler Callback Pattern" (very common in objectoriented programming), in which we hand over most of the work to the framework programs and let the framework call our program back to handle some of the tasks.
4. Initial design and specification of the website
The rango project will provide categorized information of programming languages and tutorials. The followings are the main features.
(a) The website Let users browse through userdefined categories to access
various web pages.
(b) Main page: the visitor is able to see the 5 most viewed pages (or all pages); the 5 most rango'ed categories (or all categories); and some way for visitors to browse or search through
categories
(c) When a user views a category page, they would like to display: the category name, the number of visits, the number of
likes; along with the list of associated pages in that category
(showing the page’s title and linking to its url); and some search functionality (via Bing’s Search API) to find
other pages that can be linked to this category
(d) For a particular category, the client would like the name of the category to be recorded, the number of times each category page has been visited, and how many users have clicked a “like” button (i.e. the page gets rango’ed, and voted up the social hierarchy).
(e) Each category should be accessible via a readable URL for example, /rango/booksaboutdjango/
(f) Only registered users will be able to search and add pages to categories. And so, visitors to the site should be able to register for an account.
5. Wireframes
Wireframe: provides clients with some idea of what the application should look like when complete. The main page of the rango project will look like Fig. 1.1, and the category page is shown in Fig. 1.2.
3
Fig. 1.1
Fig. 1.2
6. Pages and URL mappings
The URL and page mappings are as follows: /rango/: main page view /rango/about/: about page view /rango/category/<categoryName>/: category page view,
<categoryName> might be games, python recipies, or code and compilers
/rango/etc/: other page views, etc could be replaced with another URL
7. Models
There are two entities, a category and a page. A category can contain many pages, and one page may be assigned to only one category. The fields and types of the Category Table and Page Table are as follows
Category: name
(String) views
(Intege) likes
(Integer)
Page: category (ForeignKey)
title (String)
url (URL)
views (Integer)
in which Str, Int, FK, and URL are the string, integer, foriegn key, and URL types, respectively. There is another table, User Table, which will be discussed later.
4
8. The font convention used in this tutorial
This font means userentered texts or commands
This font means system messages
This font means directories or files This font means codes This font means modified codes This font means emphasized codes
5
Chapter 2: Getting Ready to Tango
In this project, the software stack we will be using is as follows: Ubuntu 14.04 LTS Python 3.4 Django 1.8 Postgres 9.3.6 Gunicorn 19.3 Nginx 1.4.6
1. Creating a virtual environment and installing libraries:
We would like to layout our project as follows (under some directory): project/ deploy/ git/ virtualenv/ workspace/ where
deploy: contains projects for deploying to Heroku git: the Git repository virtualenv: contains the installed virtual environment workspace: contains projects created from Eclipse
Before developing the project, we have to create a virtual environment for our project. A virtual environment contains all the libraries required for the Django project, and its main purpose is to specify the libraries and their versions in the execution
Create the directory strucutre and virtual environment, tangoVenv: $ cd <...> # change directory to the parent of directory project $ mkdir project $ cd project $ mkdir virtualenv $ cd virtualenv $ virtualenv p /usr/bin/python3 tangoVenv
The following shorthand names will be used: <ursername>: the username of the computer <projRoot>: the project root (<...>/project/workspace/tango ) <projGitRoot>: the project root under Git (<...>/project/git/tango/tango) <venv>: <...>/project/virtualenv/tangoVenv
2. Installing Postgress on the operating system
Install postgres on Ubuntu using the native package: $ sudo aptget update $ sudo aptget y upgrade $ sudo aptget install libpqdev postgresql postgresqlcontrib
After installaton, the postgres daemon will be running.
The default database is postgres. To create a database newDB: $ sudo i u postgres [sudo] password for <username>: xxxxxxxx postgres@<username> $ createdb newDB
Drop a database: postgres@<username> $ dropdb newDB
Create a database user: postgres@<username> $ createuser P newuser Enter password for new role: xxxxxxxx Enter it again: xxxxxxxx
Delete a user: postgres@<username>$ dropuser newuser
To grant privileges to a user: postgres@<username>$ psql postgres=# grant all privileges on database newDB to newuser;
An error message will show up because postgres change every character into lower case. ERROR: database "newDB" does not exist
Solution: add double quotes: postgres=# grant all privileges on database "newDB" to newuser;
List all databases: postgres=# \list
Connect to a database: postgres=# \c <databaseName> (The prompt becomes <databaseName>=#)
List all tables in the current database: <databaseName>=# \dt
Change a user's password: postgres=# \password newuser
Exit from postgres=# prompt: postgres=# \q
Exit from postgres=# prompt: postgres=# exit
8
3. Installing Django and Postgres in the virtual environment
Now we install Django and Postgres using the command pip3 installed in <venv>. $ cd <venv> $ source bin/activate (venv)$ pip3 install django (venv)$ pip3 install psycopg2
List all the installed libraries: (venv)$ pip3 freeze
Django==1.8.2 psycopg2==2.6
Chapter 3: Django Basics and Version Control
In this chapter, we will create and setup a new project with Django, and it will be be put in version control with Github.
1. Installing and setting up Eclipse
(a) Installing Eclpse
Skip...
(b) Setting up Eclipse
Run Eclipse and choose the workspace as <...>/project/workspace. Install PyDev in Eclipse: Help → Install New Software … → Add → Name: PyDev, Location: http://pydev.org/updates → OK → Check PyDev → Next → Finish (afterwards: check Brainy Software; PyDev; Brainwy)
Create a new python interpreter from the venv: Window → Preferences → (Left panel) Expand PyDev → Expand Interpreters → Python Interpreter → (Right panel) New ... → Interpreter Name: tangoPython3, Interpreter Executable: <venv>/bin/python3 → OK → Check everything → OK → OK
Install HTML and JavaScript editors Help → Install New Software … → Work with: http://download.eclipse.org/releases/luna <Enter> → (Left panel) Expand Web, XML, Java EE and OSGi Enterprise Development
9
→ Check Eclipse Web Developer Tools, JavaScript Development Tools and JSF Tools Web Page Editor → Next → Next → Agree with the terms → Finish
Set file associatiion Window → Preferences → Expand General and Editor → Select File Association
Set monospace font Window → Preference → General → Editors → Text Editors → (Right panel) Color and Fonts → Edit → Fonts: DejaVu Sans Mono, Size: 12 → OK
Show line number in the editor Window → Preference → General → Editors → Text Editors → (Right panel) Check Show line numbers → OK
Set character encoding as UTF8 Window → Preference → General → Editors → Text Editors → Spelling → Encoding: UTF8
2. Creating a Django project in Eclipse
(a) Creating a new django project in Eclipse File → New → Project ... → PyDev → PyDev Django Project → Next → Project Name: tango, Grammar Version: 3.0, Interpreter: tangoPython3 → Next → Next → Finish
Note: If there is an error message Django not found, exit eclipse
and restart again If the Next and Finish buttons are grayed out, create a
dummy PeDev project first and then create the django project. Do not delete the PyDev project. (This is a PyDev bug)
CLI (Command line interface) for creating a django project: $ source <venv>/bin/activate (venv)$ djangoadmin startproject tango
Project thus created will have the following file structure (under <...>/project/workspace/): tango/ manage.py tango/ __init__.py settings.py urls.py wsgi.py
10
where manage.py: main Django execution command __init__.py: indicates that the directory is a python
package settings.py: project's settings urls.py: URL patterns wsgi.py: runs the development server and deploy to
production environment
(b) Setting up the project
Edit <projRoot>/tango/settings.py and make the following changes: ALLOWED_HOSTS = ['*'] LANGUAGE_CODE = 'zhtw' TIME_ZONE = 'Asia/Taipei' ... DATABASES = 'default': 'ENGINE': 'django.db.backends.postgresql_psycopg2', 'NAME': 'xxxxxxxx', 'USER': 'xxxxxx', 'PASSWORD': 'xxxxxxxx', 'HOST': 'localhost', 'PORT': '',
Note: the database name and user will be set to tangoDB and tango, respectively. Create a tangoDB database and grant all privileges to user tango: $ sudo i u postgres [sudo] password for <username>: xxxxxxxx postgres@<username> $ createdb tangoDB postgres@<username> $ createuser P tango Enter password for new role: xxxxxxxx Enter it again: xxxxxxxx postgres@<username>$ psql postgres=# grant all privileges on database "tangoDB" to tango; GRANT postgres=# \q postgres@<username> $ exit
Note: The double quotes for tangoDB is necessary because psql
lowercases the command. Otherwise, an error message will show up: ERROR: database "tangodb" does not exist.
If the username contains capital letters, double quotes are
11
also necessary. There is a semicolon at the end of the command. Reset the password:
After creating the database, it's time to migrate it. Database migration is the process of taking the schema specified in the application and actually creating the corresponding data tables in the database.
Django (or South) provides two commands for database migration: makemigrations and migrate. The purpose of makemigrations is to prepare everything for database migration.
Further readings: What are migrations? The basics of south
To perform makemigration:
Right click project → Django → Make Migrations → Specify the name of the app
Because we have not created/altered any models, there is no need to perform makemigrations. Now, migrate the database:
Right click project → Django → Migrate Operations to perform: Synchronize unmigrated apps: messages, staticfiles Apply all migrations: sessions, admin, contenttypes, auth Synchronizing apps without migrations: Creating tables... Running deferred SQL... Installing custom SQL... Running migrations: Rendering model states... DONE Applying contenttypes.0001_initial... OK Applying auth.0001_initial... OK Applying admin.0001_initial... OK Applying contenttypes.0002_remove_content_type_name... OK Applying auth.0002_alter_permission_name_max_length... OK Applying auth.0003_alter_user_email_max_length... OK Applying auth.0004_alter_user_username_opts... OK Applying auth.0005_alter_user_last_login_null... OK Applying auth.0006_require_contenttypes_0002... OK Applying sessions.0001_initial... OK
Right click project → Run As → PyDev: Django Performing system checks... System check identified no issues (0 silenced). June 16, 2015 11:57:53 Django version 1.8.2, using settings 'tango.settings' Starting development server at http://127.0.0.1:8000/ Quit the server with CONTROLC.
Open a browser and test: localhost:8000
Setup the run configuration:
Right click project → Run As → Run Configurations ... → (Left panel) PyDev Django → tango tango (note: there are usually repeated two names, this is a bug of Eclipse) → (Right panel) Name: tango, Main tab, Project: tango, Main Module: $workspace_loc:tango/$DJANGO_MANAGE_LOCATION, Arguments tab, Program arguments: runserver → Apply → Close
CLI for running the project: (venv)$ cd <projRoot> (venv)$ python3 manage.py runserver
4. Creating Django applications
Create a main app: Right click project → Django → Create Application → Name of the appdjango: main → OK
CLI for creating an app: (venv)$ python3 manage.py startapp <appName>
Under the proejct's root folder, a new folder main will be created with the following files:
main/ migrations/ __init__.py admin.py models.py tests.py views.py where
migrations: contains migration information for this app __init__.py: same as before admin.py: contains the models that are avaiable in the
13
admin's page models.py: stores the app's data models test.py: stores functions for testing the app views.py: stores functions for handling requests
Register the app so that Django knows this app has been installed. Edit settings.py and add main into INSTALLED_APPS:
ModelViewController (MVC) is a software architecure pattern with the main purpose to separate the following three main building blocks of a software application:
Model: the data (i.e., the database schema) View: the page (i.e., the HTML files) Controller: the program (business logic)
Django practices MVC, too, but it is called modeltamplateview (MTV):
Model: the data (i.e., the database schema) Tamplate: the page (i.e., the HTML files) View: the program (business logic)
Best practice: Fat model, thin view, stupid template.
Now we are ready to create a view which print out the string "Hello world!" on the web page.
Edit /main/views.py with the following content: 1 2 3 4 5
from django.shortcuts import render from django.http import HttpResponse def main(request): return HttpResponse('Hello world!')
In the above: Lines 1, 2: import objects from django.shortcuts and
django.http for handing HTTP requests (render is not used now and will be used later)
Line 4: the view function, main, which handles the HTTP request. It takes an input argument request. Each view takes at least one argument an HTTPRequest object, with the conventional name request. But other names may be used.
Line 5: each view must return an HTTPResponse object. A
simple HTTPResponse object takes a string parameter, e.g. "Hello world!", to send to the client.
Good practice: use the same names for all objects within the same app, e.g., request: /main/, template: main.html, view function: main(), CSS: main.css, ...
With the view created, next thing is to specify which kind of requests will be handled by this view.
6. Mapping URLs
It is necessary to specify which URL is handled by which app, and such mappings are stored in the urls.py file. Create a file main/urls.py with the following contents:
1 2 3 4 5 6
from django.conf.urls import patterns, url from main import views urlpatterns = patterns('', url(r'$', views.main, name='main'), )
In the above: Line 1: import Django's URL pattern Line 2: import the view function from package main Lines 4~6: Django uses Python tuples to set up URL
mappings. The name of the tuple must be urlpatterns, which contains a series of calls to the django.conf.urls.url() function (we have only one mapping here). The arguments to the url() function: r'...': a raw string (i.e., there is no escape characters
in the string) in which a regular expression is used to match the HTTP request
: begin with the expression to the right of $: end with the expression to the left of $ $: means an empty string views.main: the matched request will be handle my views.main module
name='main': this URL mapping is given the name main for later reference
In the proejct's URL mapping <projRoot>/urls.py, insert the main's URL mapping:
from django.conf.urls import patterns, include, url from django.contrib import admin urlpatterns = patterns('', ... url(r'admin/', include(admin.site.urls)), url(r'.*', include('main.urls')), )
15
In the above: r'.*': the request format is http://localhost:8000/
followed by anything. It uses the regular expression (explained later), in which dot means any character and star means the character to its left may occur zero or many times.
The request will be handled according to the settings in the main.urls module.
For example, we may enter http://localhost:8000/whatever/andwhatever/ in the browser's url field. The string, whatever/andwhatever/ matches r'.*'. The rest of the string (i.e., an empty string) will be sent to main.urls for further matching. Finally, an empty string matches r'$' in the url mapping of the main app, hence the request will be handled by the main.view.main() function. A separate urls.py file for each application allows you to set URLs for individual applications, which maintains minimal coupling among apps.
Enter http://localhost:8000/ in the browser's URL field, the result:
7. Creating a superuser
Every project needs a superuser to manage its data, and Django provides powerful admin functions to manipulate the data. To create a superuser:
CLI: $ source <venv>/bin/activate (venv)$ cd <projRoot> (venv)$ python3 manage.py createsuperuser Username (leave blank to use '<userName>'): xxxxxx Email address: [email protected] Password: xxxxxxxx Password (again): xxxxxxxx Superuser created successfully.
Note: we will use admin as the name of the superuser.
The admin page: http://localhost:8000/admin/
16
8. Distributed version control with Git
A project is usually developed by a group of team members. Version control is a good tool for team collaboration. We will use the Git version control on GitHub. Eclipse has a builtin Git tool called EGit.
(a) Setting up Git repository
Set up egit folder: Window → Preferences → (Left panel) Expand Team → Git → (Right panel) Default repository folder: <...>/project/git → OK
Note: Eclipse suggests not to set the git repository inside the project in order to prevent interference.
(b) Creating a local Git repository
Right click project → Team → Share Project → Select Git → Next → Click Create button → Parent directory (or Repository directory): <...>/project/git/tango → Finish → Finish
The project will be moved to the <...>/project/git/tango directory (one more level down) and a .git directory will be created: project/ git/ tango/ .git/ tango/ manage.py
Note: after commiting the project, the development server needs to be restarted.
(c) Creating a GitHub account
Go to https://github.com
Create a GitHub account (The username will be referenced by <gitUsername>) → Create a Repository → Repository name: tango → Create repository (Registration confirm in email)
You may add collaborators of the project:
(Right panel) Settings → (Left panel) Collaborators → Enter the username and then select it → Add collaborator
(d) Pushing to GItHub
Right click project → Team → Commit... → (Eclipse may ask for name and email) Name: xxxxxxxx, Email: [email protected] (Don't show this dialog again) → OK → Commit message: xxxxxxxx → Select all files by clicking the check box at the right → Commit and Push
Destination ref: refs/heads/master → + Add Spec → Finish → User: xxxxxxxx (<gitUsername>), Password: xxxxxxxx → OK → OK
(f) Creating a .gitignore file Some files/directories do not need version control and they can be specified in the .gitignore file. Create .gitignore file in the project root directory which lists the file or directory patterns that will not be committed into version control: .* *~ *.pyc
Note: Don't create .gitignore before the first push because the
Eclipse's .project and .pydevproject files have to be there for other collaborators to pull. Other cllaborators may create their own .gitignore file after first pull.
(g) Collabortors import the project
File → Import... → (Left) Expand Git → Projects from Git → Next → Clone URI → Next → URI: https://github.com/<gitUsername>/tango.git → Next → Check master → Next → Make sure the destination directory is correct → Next → Check Import existing Eclipse projects → Next → Check tango → Finish
(h) Configuring push and pull
Right click project → Team → Push Branch ... → Remote name: origin, URI: https://github.com/<gitUsername>/tango.git, Host: github.com, Repository path: /<gitUsername>/tango.git → Next → Branch name: master, Check "Configure upstream for push and pull", When doing a pull, check "Merge upstream commits into local branch" → Next → Input user and PSW → OK → Finish → Input user and PSW → OK → OK
Right click project → Team → Commit ... → Commit Message: xxxxxxxx, Check the files to push → Commit and Push → Enter user and PSW → Next → OK
(j) Followup pull from GitHub
Right click project → Team → Pull → OK
20
Chapter 4: Templates and Static Media
In this chapter, we'll discuss the template engine as well as how to serve static media within your web pages.
1. Responding with the HTML format
In the previous chapter, the main view only returns a string to the user's browser. Normally, we would like to return a complete HTML page, and the HTML page may be coded in the program and sent directly to the user. Change the content of main/views.py as follows and run the app again:
from django.http import HttpResponse def main(request): html = ''' <!doctype html> <html> <head> <title>Tango with Django</title> <meta charset=utf8> </head> <body> <p>This is the HTML version of Hello World!</p> </body> </html> ''' return HttpResponse(html)
Such a method is not practical because Writing HTML codes in the program is a bad practice, which
violates the paradigm of separation the view from the template.
The HTML could have a great number of codes
Solution: use HTML files.
2. Creating a template folder and add a template
The best way of handling HTML codes is to use HTML files, which are called templates in Django. The most important function of a template is that we may create dynamic web pages, in which the view provide variable values to the template.
Django requires the structure of templates in an app as follows: <appName>/ templates/ <appName>/ <someName>.html
21
For our project, it will be: main/ templates/ main/ main.html Note:
Why should there be one more folder also called main? Because prefixing every HTML file with its app is a good idea, so the format of the render() function that returns a page to the user is preferred to be render(request, '<appName>/<someName>.html', ...).
Now create the folder structure:
Right click main app → New → Folder → Folder name: templates
Right click the main/templates folder → New → Folder → Folder name: main
Right click the main/templates/main folder → New → File → File name: main.html → Finish. Add the following contents in main.html: <!doctype html> <html> <head> <title>Tango with Django</title> <meta charset=utf8> </head> <body> <h1>Django says ... hello world!</h1> <p><strong>message</strong></p> </body> </html>
In the above: message is a Django template variable which will be substituted by the value supplied by the view. A template variable begins with and ends with , with the variable name between them.
Modify main/views.py such that it will render an HTML template and supply the value of the template variable message: from django.shortcuts import render from django.http import HttpResponse def main(request): context = 'message':'Django is great.' return render(request, 'main/main.html', context)
In the above: context: Django uses Python's dictionary to map the
template variable and its value
22
render(...): a function to respond to the request, which has three arguments: the HTTP request object: request the template: main.html a dictionary which contains template variables and their values: context
render() function replace all template variables with the corresponding values specified in the context dictionary and returns the resulting page to the user:
Result:
3. Creating another app 'rango'
Create another app rango: Right click project → Django → Create Application → Name of the django app to be created: rango → OK
Register the apps: edit tango/settings.py and add rango into INSTALLED_APPS:
from django.shortcuts import render def rango(request): return render(request, 'rango/rango.html')
Create folders rango/templates and
23
rango/templates/rango. And then, create file rango/templates/rango/rango.html with the following contents: <!doctype html> <html> <head> <title>Tango with Django</title> <meta charset=utf8> </head> <body> <h1>Rango says ... hello world!</h1> <a href=/rango/about/>About</a><br><br> </body> </html>
Create file rango/urls.py: from django.conf.urls import patterns, url from rango import views urlpatterns = patterns('', url(r'$', views.rango, name='rango'), )
Register the urls in tango/urls.py: url(r'admin/', include(admin.site.urls)), url(r'rango/', include('rango.urls')), url(r'.*', include('main.urls'))
Note: the main app's url has to be at the last line. Now, test the rango app by entering localhost:8000/rango/ in the browser's URL field.
Note: the About link in rango.html currently has no corresponding view, hence is handled by main view due to url(r'.*', ...).
We may add a link in the main page, which goes to the rango page. Add the following line to main.html: ... <p><strong>message</strong></p> <br> <p><a href=/rango/>Go to the rango page</a></p> </body> </html>
The contents of static data will not change, therefore they are handled differently from dynamic templates. To get static media up and running, you will need to set up a directory in which static media files are stored. Now, we will add an image into the template.
First we need to configure the static media directory: create folder /rango/static. Django will look for a static subdirectory in each app in INSTALLED_APPS. Create another two folders: /rango/static/rango/ and /rango/static/rango/img/, and then put the image rango.png in the folder. So, we have the following folder structure: rango/ static/ rango/ img/ rango.png Edit rango/templates/rango/rango.html file and make the following changes: <!doctype html> % load staticfiles % <html> ... <a href=/rango/about/>About</a><br><br> <img src="% static 'rango/img/rango.png' %" alt="Picture of Rango"> </body> </html>
Note: HTML requrires <!doctype html> to be at the first
line, so % load staticfiles % is at the second line.
% ... % is a template statement, which will be explained further later.
The resulting page is on the right:
The function of % load staticfiles % and % static 'rango/img/rango.png' % is to use STATIC_URL in settings.py to compose the correct path of the static file. Because STATIC_URL is /static/, so the resulting path is /static/rango/img/rango.png. Now add a CSS file in main app: create the folders main/static , main/static/main, and main/static/main/css and create file main/static/main/css/main.css with the follow contents: body backgroundcolor: lightblue;
25
Add the CSS link in main.html: % load staticfiles % <html> <head> ... <link rel=stylesheet href="% static 'main/css/main.css' %"> </head>
After adding the CSS link, the server may need to restart.
Also add CSS link in rango.html: ... <head> ... <link rel=stylesheet href="% static 'main/css/main.css' %"> </head> ...
The resulting page:
Request localhost:8000/static/main/css/main.css can access the CSS file. Same as image file localhost:8000/static/rango/img/rango.png.
See: Deploying static files into production.
Another type of static media is JavaScript. T locate a JavaScript is similar: create folder main/static/main/js and file main/static/main/js/main.js, and add the following line in the template. <script src="% static 'main/js/main.js' %"></script>
5. Exercises
Now, fix the About link by adding a view and a template. Create a new view about in rango/views.py:
Create a new template rango/templates/rango/about.html. <!doctype html> % load staticfiles % <html> <head> <title>Tango with Django</title> <meta charset=utf8> </head> <body> <h1>Rango says: Here is the about page.</h1> </body> </html>
Create a new URL mapping in rango/urls.py: url(r'$', views.rango, name='rango'), url(r'about/$', views.about, name='about'),
Now test the system by clicking the About link in the rango.html page:
Now, we may push the new version of this project to Github.
27
Chapter 5: Models and Databases
Django's database module: objectrelational mapping (ORM). ORM is a programming technique for converting data between incompatible type systems in objectoriented programming languages. SQL languages are no longer needed when ORM is introduced.
A Django model is a Python object that describes the data model/table. Instead of directly working on the database table via SQL, all you have to do is manipulate the corresponding Python object.
1. Rango's requirement
Rango's data requirements: Rango is a web page directory a site containing links to
other websites There are a number of different webpage categories, and
each category contains a number of links A category has a name, number of visits, and number of
likes A page refers to a category and has a title, URL, and a
number of views
2. Creating models
About our database, we have already set the following in settings.py: DATABASES = 'default': 'ENGINE':
The database engine is Postgres backend. We don't need to alter our application if later we decided to use another database, say MySQL. All we have to do is to change the database engine. For example, MySQL: DATABASES = 'default': 'ENGINE': 'mysql.connector.django', 'NAME': 'xxxxxxxx', 'USER': 'xxxxxx',
For the Rango site, we need two initial models: Category and Page.
Category: name
Page: category title url views
Edit rango/models.py and add the following: from django.db import models class Category(models.Model): name = models.CharField(max_length=128, unique=True) def __str__(self): # __unicode__(self): for python2 return self.name
class Page(models.Model): category = models.ForeignKey(Category) title = models.CharField(max_length=128) url = models.URLField() views = models.IntegerField(default=0) def __str__(self): # __unicode__(self): for python2 return self.title
Note: It is a good practice to use the lowercase of the model
name as a foreign key. For example, category = models.ForeignKey(Category)
A Django model is a Python class and it subclasses Django's module models.Models.
29
Django provides a number of builtin fields. Some of the most commonly used are listed below.
CharField: a field for storing character data (e.g. strings). Specify max_length to provide a maximum number of characters the field can store
URLField: much like a CharField, but designed for storing resource URLs. You may also specify a max_length parameter
IntegerField: stores an integes DateField: stores a Python datetime.date
If a field value has to be unique, you can specify unique=True Django also provides simple mechanisms to relate models/database tables together. These mechanisms are encapsulated in three further field types:.
ForeignKey: a field type that allows to create a onetomany relationship
OneToOneField: afield type that allows to define a strict onetoone relationship
ManyToManyField: a field type which allows to define a manytomany relationship
In the Rango application, several pages may belong to one catogory, hence, it's a onetomany relationship, and the ForeignKey is used to designate such relationship. The __str__(self): method is used in the admin page, return specifies which field should be displayed.
3. Creating and migrating the database
Now, let Django create the database tables. Migration lets you alter the model fields later without deleting the entire model and recreate it again. We've done the intial migrate, and whenever we make changes to the models, we need to register the changes, via makemigrations command for the particular app.
Right click project → Django → Make Migrations → Name of the app: rango → OK
Migrations for 'rango': 0001_initial.py: Create model Category Create model Page
CLI for make migrations: (venv)$ python3 manage.py makemigrations rango
Makemigrations command creates file 0001_initial.py in rango/migrations folder
Now, migrate the data models:
30
Right click project → Django → Migrate Operations to perform: Synchronize unmigrated apps: staticfiles, messages Apply all migrations: contenttypes, rango, sessions, auth, admin Synchronizing apps without migrations: Creating tables... Running deferred SQL... Installing custom SQL... Running migrations: Rendering model states... DONE Applying rango.0001_initial... OK
CLI for migrate: (venv)$ python3 manage.py migrate
Add one more item 00*_* in .gitignore: ... 00*_*
Note: 00*_* files are migration information and they may be
different for each team member. Hence, they don't need version control during development. However, when the system goes production, migration information need to be retained; therefore, the 00*_* item in .gitignore has to be removed.
To delete all data in the database: drop the database and delete all 00* files: $ sudo i u postgress # and then enter password $ dropdb tangoDB $ find . type f name 00* exec rm \;
4. Configure the admin interface
Django provides a buildin, webbased administrative interface for manipulating databases.
To be able to edit rango's data, make sure that the item 'django.contrib.admin' is in the INSTALLED_APPS in file settings.py, and there is the item url(r'admin/', include(admin.site.urls)), in urlpatterns in the project's urls.py. Register rango's models such they may be manipulated by Django's admin interface. Edit rango/admin.py and add the following contents: from django.contrib import admin from rango.models import Category, Page admin.site.register(Category) admin.site.register(Page)
The admin page: http://localhost:8000/admin/
31
It can be seen that Django changes the name of models by adding s to make them plural.
5. Creating a popluating script
Entering data into the database one by one is time consuming. We would like to create a population script to automatically populate the database with initial test data.
First, define the popCategory() function to retrieve or create a category record: def popCategory(name): category = Category.objects.get_or_create(name=name)[0] return category
This function takes one arguments (the name of the category: name) and uses get_or_create() method to retrieve or create a record in the model. And then it returns the record.
Second, define the popCategory() function to retrieve or add a page record: def popPage(category, title, url, views=0): page = Page.objects.get_or_create(category=category, title=title, url=url, views=views)[0] page.save()
This function takes several arguments (category, title, url, and views) and also uses get_or_create() method to retrieve or create a record in the model.
With the above two functions, we may perform a series of calls to create the initial data. For example, adding a Python category and an Official Python Tutorial page can be done as follows: pythonCategory = popCategory('Python') popPage(category=pythonCategory, title='Official Python Tutorial', url='http://docs.python.org/2/tutorial/')
Edit rango/views.py and add the following:
32
from rango.models import Category, Page ... def populate(request): # Python pythonCategory = popCategory('Python') popPage(category=pythonCategory, title='Official Python Tutorial', url='http://docs.python.org/2/tutorial/') popPage(category=pythonCategory, title='How to Think like a Computer Scientist', url='http://www.greenteapress.com/thinkpython/') popPage(category=pythonCategory, title='Learn Python in 10 Minutes', url='http://www.korokithakis.net/tutorials/python/') # Django djangoCategory = popCategory('Django') popPage(category=djangoCategory, title='Official Django Tutorial', url='https://docs.djangoproject.com/en/1.5/intro/tutorial01/') popPage(category=djangoCategory, title='Django Rocks', url='http://www.djangorocks.com/') popPage(category=djangoCategory, title='How to Tango with Django', url='http://www.tangowithdjango.com/')
In the admin page, clicking Categorys or Pages, the category names or page titles will appear, respectively, because we specified such in the model declaration. Category Page
def __str__(self): return self.name
def __str__(self): return self.title
6. Django ORM
An objectrelational mapping (ORM) is a tool that lets you query and manipulate data from a database using an object paradigm. That is, map objectoriented language to relational database. With ORM, SQL is not needed anymore.
Common database operations: CRUD → create, read, upate, and delete.
(b) Read: Category.object.all() # Retrieve all data Category.objects.get(...) # Retrieve only one record Category.objects.filter(...) # Retrieve one or more records as a list
1. Configure DATABASES in settings.py 2. Add a model:
(a) Create the new model in the app's models.py (b) Register the model in the app's admin.py (c) Migrate DB (d) Enter data or run the populate script
8. Exercises
There are two fields missing in class Category: views and likes. Now, fix it. Add two more fields in class Category: class Category(models.Model):
name = models.CharField(max_length=128, unique=True) views = models.IntegerField(default=0) likes = models.IntegerField(default=0)
And then populate the two fields by modifying rango/views.py: import random def popCategory(name): category = Category.objects.get_or_create(name=name)[0] category.views = random.randint(0, 20) category.likes = random.randint(0, 20) category.save() return category
Perform makemigrations and migrate (there will be an additional file 0002_*.py in the rango/migrations folder). Now, populate the data again: http://localhost:8000/rango/populate/
Note: When performing migration, eclipse may terminate the
development server. However, running the server again, you may encounter the error message: Error: That port is already in use. It means the server was not terminated properly and the port 8000 is occupied. Click Terminate
to stop server and run the server again. Or use the command $ fuser k 8000/tcp to clear the port:
It's time to push the project to Github.
35
Chapter 6: Models, Templates, and Views
Now that we have the models set up and populated with some data, we are ready to to retrieve the data from the database and present the data within the templates.
1. Basic workflow: data driven pages
Django's steps to create a data driven webpage. 1. Import the models in the views.py 2. Query the model to get data 3. Pass the results into the template's context 4. Setup the template to present the data to the user
2. Showing categories on Rango's page
Now, we would like to show the categories on the page. All records in model Category will be retrieved via database query and listed in rango.html as follows:
Django Python Other Frameworks
Edit rango/view.py: retrieve data form Category and pass the data to template rango.html as a python list def rango(request): categories = Category.objects.order_by('likes') context = 'categories':categories return render(request, 'rango/rango.html', context)
Note: It is a good practice to use the lowercase, plural form of the
model name as the name of the retrieved list. For example, categories = Category.object... So, later we may iterate each of the item as follows: for category in categories: ...
It is also a good practice to use the same name for a template variable and its value. For example, context = 'categories'=categories So, later we may iterate each of the item in the template as follows: % for category in categories % ...
A template statement is enclosed by % and %, and there should be one space after % and before %. A template variable is enclosed by and ; however, if the variable is in a statement, it does not need to be enclosed.
We will use the decision (if) and iteration (for loop) statements
36
to list the data. The format of the if statement: % if <condition> % ... % endif %
The format of the for statement: % for <item> in <itemList> % ... % endfor %
% for i in 'xxxxxxxxxx' % ... forloop.counter0 ... % endfor %
views: render(..., ..., 'range':range(100))
template: % for i in range % ... % endfor %
Edit rango.html: display the category list and remove the image element ... <h1>Rango says ... hello world!</h1> % if categories %
37
<ul> % for category in categories % <li>category.name</li> % endfor % </ul> % else % <p><strong>There are no categories present.</strong></p> % endif % <a href=/rango/about/>About</a><br><br> <img src="% static 'rango/img/rango.png' %" alt="Picture of Rango"> ...
Result of http://localhost:8000/rango/
3. Creating a details page
According to Rango’s specification, we also need to show a list of pages that are associated with each category.
A new view must be created, which should be parameterised, i.e., there will be parameters in the request. We also need to create URL patterns and URL strings that encode category names.
(a) URL design and mapping There should be a way for the user to tell Rango which category he/she wants to see.
Solution 1: Design the URL patterns such as /rango/category/1/ or /rango/category/2/, in which the numbers 1 and 2 are the category's ID. However, the user wouldn't know what it means by 1 or 2.
Solution 2: Use the category name as part of the URL pattern such as /rango/category/Python/ or /rango/category/Django/. This patter is simple, readable, and meaningful. Problem: the category name may have multiple words with white spaces, like 'Other Frameworks', which is inappropriate because URLs does not like spaces. This can be solved by Django's slugify function. Also see the solution.
To make clean urls, we are going to include a slug field in the Category model. First import the slugify function, which will replace white space with hyphens, e.g., "how do i create a slug in django" turns into "howdoicreateaslugindjango".
Second, override the save method of Category model, which we will call the slugify method and update the slug field.
Modify rango/models.py as follows: ... from django.template.defaultfilters import slugify class Category(models.Model): name = models.CharField(max_length=128, unique=True) views = models.IntegerField(default=0) likes = models.IntegerField(default=0) slug = models.SlugField(unique=True) def save(self, *args, **kwargs): self.slug = slugify(self.name) super(Category, self).save(*args, **kwargs) def __str__(self): return self.name
In the above: One more field, slug, is added with field type SlugField. Django models has a default save method, but here we
override it with our own save method. The save method:
Use Slugify() function to replace white spaces with hyphens in the name field and store the result in slug field Save the model using the save method, which calls the save mothod of the parent class (models.Model)
Python function arguments: *args: positional arguments, for examples:
# Example 1 def foo(x, y, z): print(x, y, z) foo(1, 2, 3) # → 1 2 3 (x, y, z take values 1, 2, 3, respectively) # Example 2 def foo(x, y, z=3): # Default arg may not follow nondefault arg print(x, y, z)
39
foo(1, 2) # x, y take values 1, 2,respectively; z is 3 by default # Example 3 def foo(*args): print(args) foo(1, 2, 3) # → (1, 2, 3), a tuple
**kwargs: keyword arguments, for example: def bar(**kwargs): print(kwargs) foo(x=1, y=2, z=3) # → 'x':1, 'y':2, 'z':3, a dictionary
Now make migration for the model:
Click project → Django → Make Migrations → Name of the app: rango → OK You are trying to add a nonnullable field 'slug' to category without a default; we can't do that (the database needs something to populate existing rows). Please select a fix: 1) Provide a oneoff default now (will be set on all existing rows) 2) Quit, and let me add a default in models.py Select an option: 1 Please enter the default value now, as valid Python The datetime and django.utils.timezone modules are available, so you can do e.g. timezone.now() >>> '' Migrations for 'rango': 0003_category_slug.py: Add field slug to category
In the above, because we have added a new field and the existing records don't have any value in that field. We have two options: to provide a default value once for all, or quit this makemigration. We choose to enter an empty string as a default value.
"<venv>/lib/python3.4/sitepackages/django/db/backends/utils.py", line 64, in execute return self.cursor.execute(sql, params) psycopg2.IntegrityError: could not create unique index "rango_category_slug_key" DETAIL: Key (slug)=() is duplicated. ...
Migrate failed because we specified unique=True in the slug field; however, we provided an empty string for all existing records.
Try the following solutions: Remove unique=True and migrate again → Fail Remove slug field and migrate again → Fail Enter admin page to set values → Fail
The only solution: delete the database? During development, it's OK to delete the database; but in production, you cannot do this!
Correct solution: 1. Remove unique=True in the slug field 2. Delete the failed migration files in rango (in our case, files
with the starting number greater than 0002) 3. Perform makemigrations and migrate 4. Run localhost:8000/rango/populate/ to fill in the slug field
for all existing records (because name is unique, the derived slug will be unique) → You may want to check in the admin page to see if everything is OK
5. Set unique=True 6. Perform makemigrations and migrate again
Lesson learned: When adding a new field in a model, never set any constraints.
Constraints: unique=True, blank=False, null=False, ... Set the constraint after correct data have been entered to the new field for each existing records.
(c) Display categories and pages
After designing the slug URL, we will start to write a view to retrieve all data in Category and Page models and display them in a template. The steps:
1. Create a new view in rango/view.py, called category, which will take an additional parameter, categoryNameURL which will store the encoded category name
2. Create a new template templates/rango/category.html
3. Update Rango's urlpatterns to map the new category view to a URL pattern in rango/urls.py
41
(c.1) Create a category() view to retrieve the data Edit rango/views.py: add a new view category() ... def category(request, categoryNameSlug): context = try: category = Category.objects.get(slug=categoryNameSlug) context['category'] = category pages = Page.objects.filter(category=category) context['pages'] = pages except Category.DoesNotExist: context['categoryName'] = categoryNameSlug.replace('', ' ') return render(request, 'rango/category.html', context)
In the above, categoryNameSlug is an argument obtained from the
URL request context = or context = dict initializes an empty
dictionary. context['...'] = ... or
context.update('...':...) adds or modifies an item in the dictionary
try: retrieve data from models Category and Page, and put them into template variables. If the category is not found, except part will takeover without crashing the program.
except: replace hyphens with white spaces in the categoryNameSlug such that the name looks normal in the template
(c.2) Create a category.html template to display the data Create template rango/templates/rango/category.html and enter the following contents: <!doctype html> % load staticfiles % <html> <head> <title>Tango with Django</title> <meta charset=utf8> <link rel=stylesheet href="% static 'main/css/main.css' %"> </head> <body> % if category % <h1>category.name</h1> % if pages % <ul> % for page in pages % <li><a href=page.url>page.title</a></li>
42
% endfor % </ul> % else % <p><strong>No pages currently in category.</strong></p> % endif % % else % <p>The specified category "categoryName" does not exist.</p> % endif % </body> </html>
In the above: First print out the category name If categories is not empty
If there are pages associated with it then print the pages, otherwise "no pages ..." else print "... category does not exist. "
In the above: (?P<name>pattern) is a named regular expression
group in which name is the name of the argument passed to the view (remember we have such an argument in category() view) and pattern is some pattern to match. For example, (?P<categoryNameSlug>Python) will pass the argument categoryNameSlug with the value Python to the request handler.
\w matches any character, number, or underscore (i.e., a~z, A~Z, 0~9, _)
\ matches a hyphen + is a "one or more" quentifier. Hence, [\w\]+ means
one or more character, number, underscore, or hyphen The followings are some of the regular expresions'
symbols/expressions and their meanings:
Symbol/ Expression
Matched String
. Any character
Start of string
43
$ End of string
* 0 or more repetitions
+ 1 or more repetitions
? 0 or 1 repetitions
| A | B means A or B
[az] [AZ] [09]
Any lowercase character Any uppdrcase character Any onedigit number
\w Any alphanumeric character or _
\d Any digit
(c.3) Add a category link in rango.html Finally, modify the rango.html template: add a link to a category <ul> % for category in categories % <li><a href=/rango/category/category.slug/>category.name</a></li> % endfor % </ul>
Result of http://localhost:8000/rango/
→ Click Python →
Also note, the link of "Other Frameworks" is actually "otherframeworks".
4. Exercises
We would like to see the populated data immediately after the request localhost:8000/rango/populate/ as follows. Make it happen.
44
It's time to push the project to Github.
45
Chapter 7: Fun with Forms
Forms are used to gather information from users.
Django's form handling functionality allows you to: display an HTML form with automatically generated form
widgets (like a text field or date picker) check submitted data against a set of validation rules redisplay a form in case of validation errors convert submitted form data to the relevant Python data
types
Using Django's form functionality saves a lot of time and you don't have to write too much HTML codes because forms are generated automatically.
Some of the form fields: Text input: <input type=text name=...>
Password input: <input type=password name=...>
Text area: <textarea rows=... cols=...> ... </textarea>
Radio choice: <input type=radio name=... value=... checked> ... <input type=radio name=... value=...> ... <input type=radio name=... value=...> ... In the above, the values of the name has to be the same
File upload form <form ... enctype=multipart/formdata> <input type=file value=...>
Image button: <input type=image src=...>
Hidden: <input type=hidden name=... value=...>
Button: <button> ... </button>
Form submit: <input type=submit value=...>
1. Basic workflow
Steps of creating a form: 1. Create a file forms.py in the application folder 2. Create a ModelForm class for each model that you wish to
represent as a form 3. Customise the forms as you desire 4. Create or update a view to handle the form including
displaying the form, saving the form data, and flagging up errors which may occur when the user enters incorrect data (or no data at all) in the form
5. Create or update a template to display the form 6. Add a urlpattern to map to the new view (if you created
a new one)
2. Page and Category Forms
Create the file rango/forms.py and enter the following contents: from django import forms from rango.models import Page, Category class CategoryForm(forms.ModelForm): name = forms.CharField(max_length=128, help_text="Please enter the category name.") views = forms.IntegerField(widget=forms.HiddenInput(), initial=0) likes = forms.IntegerField(widget=forms.HiddenInput(), initial=0) slug = forms.CharField(widget=forms.HiddenInput(), required=False) class Meta: model = Category fields = ('name',)
47
class PageForm(forms.ModelForm): title = forms.CharField(max_length=128, help_text="Please enter the title of the page.") url = forms.URLField(max_length=200, help_text="Please enter the URL of the page.") views = forms.IntegerField(widget=forms.HiddenInput(), initial=0) class Meta: model = Page exclude = ('category',)
In the above, In CategoryForm, fields views and likes are set to be
hidden (forms.HiddenInput()) because users won't have to fill them or even have to see them. They are here because their values cannot be NULL in the model, so we have to use form fields to populate their values (initial=0). Question: their values are set default ( = 0) in the model, we don't seem to have to include them in the form.
Inner class Meta is a class container with some options (metadata) attached to the model. It is “anything that’s not a field”, such as ordering options, available permissions, associated database table name, whether the model is abstract or not, singular and plural versions of the name etc. A model is specified first. Including a model in the
Meta class is important, because thus Django may check if the model and the form is consistent
Some fields may allow NULL in the model definition, so we don't have to include them in the form
It is required to put fields = (...) or exclude = (...) in Meta
Use fields = '__all__' to include all fields, fields = (...) to include the needed fields, or exclude = (...) to exclude certain fields
Besides the CharField, IntegerField widget, others are also available, such as EmailField, ChoiceField, DateField, etc. Check: the official Django documentation on widgets.
Check out the official Django documentation on forms for further information.
3. Creating an addCategory view, template, and URL mapping
With the CategoryForm defined, we are ready to create a new view to display the form and handle the posting of the form data.
... from rango.forms import CategoryForm ... def addCategory(request): template = 'rango/addCategory.html' if request.method=='GET': return render(request, template, 'form':CategoryForm()) # request.method=='POST' form = CategoryForm(request.POST) if not form.is_valid(): return render(request, template, 'form':form) form.save(commit=True) return rango(request) # Call function rango()
In the above, First import CategoryForm class and define the template If the HTTP method is GET, return with an empty form
CategoryForm(), which is also called an unbound form. Else (the HTTP method is POST) set form to
CategoryForm(request.POST), which is a bound form populated wiith userinput data (request.POST).
If the form is not valid, display a form with error messages automatically. Django's form handling not only save the form data, but also check if the form is valid. It will not save an incomplete form.
Else (a valid form) save the form data into the model. commit=True means save the data immediately.
Finally, invoke the rango view (i.e., return to the rango page).
(b) Create a template for adding categories
Create rango/templates/rango/addCategory.html: <!doctype html> % load staticfiles % <html> <head> <title>Tango with Django</title> <meta charset=utf8> <link rel=stylesheet href=% static 'main/css/main.css' %> </head> <body> <h1>Add a Category</h1> <form id=categoryForm method=post action=/rango/addCategory/> % csrf_token % % for hidden in form.hidden_fields % hidden % endfor % % for field in form.visible_fields % field.errors
In the above, % csrf_token %: Django form security to prevent
CrossSite Request Forgery % for hidden ... %: output the hidden fields % for field ... %: output the visible fields defined
in class Meta: fields(...) Hidden form fields are necessary because HTTP is a
stateless protocol, and you can’t persist state between different HTTP requests. To overcome this limitation, hidden HTML form fields allow to pass information to a client (which cannot be seen on the rendered page), and to be sent back to the server when the user submits the form.
(c) Create a URL mapping
Add the following line into rango/urls.py: url(r'addCategory/$', views.addCategory, name='addCategory'),
Now test with localhost:8000/rango/addCategory/. Django generates the following form: <form id=categoryForm method=post action=/rango/addCategory/> <input type='hidden' name='csrfmiddlewaretoken' value='34GJRzmxX4XZfBPJOJLtUP46ZZaedugK' /> <input id="id_views" name="views" type="hidden" value="0" /> <input id="id_likes" name="likes" type="hidden" value="0" /> <input id="id_slug" name="slug" type="hidden" /> Please enter the category name. <input id="id_name" maxlength="128" name="name" type="text" /> <input type=submit value="Create Category"> </form> ...
Django form actions: Generate form automatically Generate CSRF middleware token Loop through invisible and visible fields and ceneratereate
corresponding input fields Add an id for each field with the format id_<fieldName>
Now, put a link on the rango page so that we can easily add
50
categories.
Edit rango/templates/rango/rango.html and add the following link before About link: % else % <strong>There are no categories present.</strong> % endif % <br> <a href=/rango/addCategory/>Add a New Category</a><br><br> <a href=/rango/about/>About</a><br><br> ...
Result: click Add a New Category → Enter Ruby on Rails and click Create Category
→
4. Cleaner forms
Purpose: to help users to enter form data with correct formats.
Add a clean() function to class PageForm() in rango/forms.py file to check if the URL is in the correct format (should start with http://) class PageForm(forms.ModelForm): ... class Meta: ... def clean(self): cleanedData = self.cleaned_data url = cleanedData.get('url')
51
if url and not url.startswith('http://'): url = 'http://' + url cleanedData['url'] = url return cleanedData
In the above: Our clean() method overrides Django form's default
clean() method Form data is obtained from the ModelForm dictionary
attribute cleaned_data Use get() method to obtain a particular field Use startswith() methd to detect if the string matches
what we specified If the data format is incorrect, correct it Finally return the cleaned result
5. Exercise: creating an addPage view, template, and URL mapping
(a) Create a view for adding pages
Create a new view addPage() in /rango/views.py, which takes two parameters as inputs: categoryName and pageName.
commit=False means don't actually save the object (i.e., page) into the database (something more needs to be set), just obtain an object for now. Django will delay saving it until page.save() is called. After performing necessary setting (page.category,
52
page.views), finally call page.save() to actually save it.
(b) Create a template for adding pages
Create a new template
/rango/templates/rango/addPage.html: <!doctype html> % load staticfiles % <html> <head> <title>Tango with Django</title> <meta charset=utf8> <link rel=stylesheet href=% static 'main/css/main.css' %> </head> <body> <h1>Add a new page in category.name</h1> <form method=post action=/rango/category.slug/addPage/> % csrf_token % % for hidden in form.hidden_fields % hidden % endfor % % for field in form.visible_fields % field.errors field.help_text field<br> % endfor % <input type=submit value="Create Page"> </form> </body> </html>
(c) Create a URL mapping
Add the following lines in category.html for the user to add a new page: ... <p><strong>No pages currently in category.</strong></p> % endif % <br><br> <a href="/rango/category.slug/addPage/">Add a New Page</a><br><br> % else % <p>The specified category "categoryName" does not exist.</p> % endif % ...
Add the following line into rango/urls.py: url(r'(?P<categoryNameSlug>[\w\]+)/addPage/$', views.addPage,
name='addPage'),
Now, test the app http://localhost:8000/rango/
53
Click Django →
Click Add a New Page →
Eenter Writing you first Django app and click Create Page →
The auth application in Django's django.contrib.auth package handles user authentication. It consists of the following aspects:
Users Permissions: binary flags (e.g. yes/no) determining what a
user may or may not do Groups: a method of applying permissions to more than
one user A configurable password hashing system: a must for
ensuring data security Forms and view tools for logging in users, or restricting
content A pluggable backend system, allowing you to provide your
own authenticationrelated functionality
1. Setting up authentication
Make sure that django.contrib.auth and django.contrib.conttenttypes are in the INSTALLED_APP in settings.py. In Django, passwords are stored using PBKDF2 algorithm
2. The User model
The main object for authentication is the User model, which contains the following fields:
For this project, we need two additional fields: URLField and ImageField for a user. Modify rango/models.py: from django.contrib.auth.models import User ... class UserProfile(models.Model): user = models.OneToOneField(User) # The additional attributes we wish to include.
One UserProfile model maps to only one User model, based on the onetoone relationship.
Setting blank=True allows the field to be empty. To use ImageField, we have to install Pillow: (venv)$ pip3 install Pillow
The upload_to attribute takes the value of MEDIA_ROOT specified in settings.py. Add the UserProfile model into rango/admin.py such that it is modifiable in the administration interface. from rango.models import Category, Page, UserProfile ... admin.site.register(UserProfile)
Now, perform makemigrations: Migrations for 'rango': 0005_userprofile.py: Create model UserProfile
and migrate: Operations to perform: Synchronize unmigrated apps: messages, staticfiles Apply all migrations: sessions, auth, admin, rango, contenttypes Synchronizing apps without migrations: Creating tables... Running deferred SQL... Installing custom SQL... Running migrations: Rendering model states... DONE Applying rango.0005_userprofile... OK
3. Creating a user registration view and template
We would like to let users register to our site.
Steps: 1. Create a UserForm and UserProfileForm 2. Add a view to handle the creation of a user 3. Create a template that displays the UserForm and
UserProfileForm 4. Map a URL 5. Link the rango page to the register page
56
(a) Create UserForm and UserProfileForm Create two classes in rango/forms.py: ... from django.contrib.auth.models import User from rango.models import Page, Category, UserProfile ... class UserForm(forms.ModelForm): password = forms.CharField(widget=forms.PasswordInput()) class Meta: model = User fields = ('username', 'email', 'password') class UserProfileForm(forms.ModelForm): class Meta: model = UserProfile fields = ('website', 'picture')
The UserForm takes three fields, username, email, and password, from the User model. And a widget (forms.PasswordInput()) is specified for field password. The UserProfileForm takes two fields, website and picture, from the UserProfile model.
(b) Create the register() view There are two form class in our registration form.
In rango/views.py, add the following:
from rango.forms import UserForm, UserProfileForm ... def register(request): template = 'rango/register.html' if request.method=='GET': return render(request, template, 'userForm': UserForm(), 'profileForm': UserProfileForm(), 'registered': False) # request.method == 'POST': userForm = UserForm(request.POST) profileForm = UserProfileForm(request.POST) if not (userForm.is_valid() and profileForm.is_valid()): #print(userForm.errors, profileForm.errors) return render(request, template,
57
'userForm': userForm, 'profileForm': profileForm, 'registered': False) # Save the user's form data to the database user = userForm.save() user.set_password(user.password) user.save() profile = profileForm.save(commit=False) profile.user = user if 'picture' in request.FILES: profile.picture = request.FILES['picture'] profile.save() return render(request, template, 'userForm': userForm, 'profileForm': profileForm, 'registered': True)
In the above: If the request method is GET, return an unbound form
containing two form classes, UserForm and UserProfileForm, and a flag registered.
Else, bound the two form classes. Test if both of the bound forms are valid, if not return the bound forms
Else, save the userForm form, following by encypting the password by method set_password() and another save.
Next, save userProfileForm without commit, followed by setting the OneToOne field and uploaded file if necessary. And then, save again.
Finally, return the bound forms.
(c) Create the registration template
Create a new template rango/templates/rango/register.html: <!doctype html> % load staticfiles % <html> <head> <title>Tango with Django</title> <meta charset=utf8> <link rel=stylesheet href=% static 'main/css/main.css' %> </head> <body> <h1>Register with Tango</h1> % if registered % <p><strong>Thank you for registering!</strong><br> <a href=/>Return to the homepage.</a></p> % else % <p><strong>Register here!</strong></p> <form id=userForm method=post action=/rango/register/ enctype=multipart/formdata> % csrf_token % userForm.as_p
In the above: If the registered successfully, print "thank you" message
and provide a link to the main page Else, provide a registration form, the user may upload an
image, so the form has to have the enctype=multipart/formdata attribute. The method .as_p sets the form in a <p> element so every field is in a paragraph.
(d) Map the URL
In rango/urls.py, add the new mapping: url(r'register/$', views.register, name='register'),
(e) Create a link in rango.html: <a href=/rango/addCategory/>Add a New Category</a><br><br> <a href=/rango/register/>Register Here</a><br><br> <a href=/rango/about/>About</a><br><br>
Now test the system http://localhost:8000/rango/ and click Register Here.
There are Chinese characters because we set LANGUAGE_CODE = 'zhtw' in settings.py and Django's model User has internationalization (I18N) property. If we set LANGUAGE_CODE = 'en', then everything will be English.
4. Add login functionality
Once users may register our site, we need to provide a login page.
Steps:
59
1. Create a login view 2. Create a login template 3. Map the URL 4. Provide a login link
(a) Create the login view
Add the userLogin() view in rango/views.py:
from django.http import HttpResponse, HttpResponseRedirect from django.contrib.auth import authenticate, login ... def userLogin(request): if request.method=='GET': return render(request, 'rango/userLogin.html') # request.method=='POST': username = request.POST['username'] password = request.POST['password'] user = authenticate(username=username, password=password) if not user: return HttpResponse('Invalid login details supplied.') if not user.is_active: return HttpResponse('Your Tango account is disabled.') login(request, user) return HttpResponseRedirect('/')
In the above, If the request method is GET, return the template Get the POST data: username and password Authenticate the user with function authentication():
see if the user has already registered and if the password is correct
Something wrong with logging in: return and show the corresponding message
Everything is OK: login the user using function login() and redirect to the main page
(b) Create the template rango/templates/rango/userLogin.html: <!doctype html> % load staticfiles % <html> <head> <! Is anyone getting tired of repeatedly entering the header over and over?? > <title>Tango with Django</title> <meta charset=utf8> <link rel=stylesheet href=% static 'main/css/main.css' %> </head> <body> <h1>Login to Tango</h1> <form id=loginForm method=post action=/rango/userLogin/>
In the above, Display a login form with two input fields: username and
password.
(c) Map the URL in rango/urls/py: url(r'userLogin/$', views.userLogin, name='userLogin'),
(d) Create a login link in rango.thml and change the greeting message at the top: <body> % if user.is_authenticated % <h1>Rango says... hello user.username!</h1> % else % <h1>Rango says... hello world!</h1> % endif % ... <a href=/rango/register/>Register Here</a><br><br> <a href=/rango/userLogin/>Login</a><br><br> <a href=/rango/about/>About</a><br><br>
In the above, Use Django template statement
user.is_authenticated() to test if the user is logged in and display different welcome message
Provide a link for login
Now test the system http://localhost:8000/rango/, and click Login.
5. Restrict access
Restrict some part of the application as per the specification: only registered users can add categories and pages. There are two ways to check if the user is authenticated: examining the request directly or using a decorator function.
61
(a) Examine the request object to check if the user is authenticated
Check to see if the user is logged in via the user.is_authenticated() method. For example: def someView(request): if request.user.is_authenticated(): return HttpResponse('You are logged in.') return HttpResponse('You are not logged in.')
(b) Use Python decorator to check if the user is authenticated
Python decorators can dynamically alter the functionality of a function, method, or class without having to change the source code. Django provides a decorator called login_required() which can be attached to a view that requires the user to be logged in.
Add the following to rango() in rango/views.py: ... from django.contrib.auth.decorators import login_required @login_required def restricted(request): return HttpResponse("Since you're logged in, you can see this text!")
The decorator is placed directly above the function signature, and a @ is put before the decorator name. Python will execute the decorator before executing the decorated function/method.
Add an url pattern in rango's urls.py: urlpatterns = patterns('', ... url(r'restricted/', views.restricted, name='restricted'), )
For an unloggend user who attemps to acces the restricted() view, we will redirect him/her to the login page. Add the following line in settings.py so that login_required() decorator will redirect notlogged user to the URL /rango/userLogin/. LOGIN_URL = '/rango/userLogin/'
6. Log out
Django has a builtin logout() function. Add a view called userLogout() in rango/views.py file with the following: from django.contrib.auth import authenticate, login, logout ... @login_required
Map the URL: url(r'userLogout/', views.userLogout, name='userLogout'),
Now provide a logout link for the loggedin user. Modify rango/templates/rango/rango.html: to check if user is logged in. If yes, provide a logout link, otherwise, provide a login link.
<a href=/rango/addCategory/>Add a New Category</a><br><br> % if user.is_authenticated % <a href=/rango/restricted/>Restricted Page</a><br><br> <a href=/rango/userLogout/>Logout</a><br><br> % else % <a href="/rango/register/">Register Here</a><br><br> <a href=/rango/userLogin/>Login</a><br><br> % endif % <a href=/rango/about/>About</a><br><br>
In the above, If user is logged in, present Restricted Page and Logout
links Else, present Register Here and Login links
Now test the system: (1) Login and enter http://localhost:8000/rango/restricted/ in the
browser's URL field. (2) Logout and enter http://localhost:8000/rango/restricted/ in
the browser's URL field.
7. Create a superuser with an initialization view
Normally, a superuser has to be created to manage the data and this can be done by the following as we have done before:
$ python manage.py createsuperuser
The superuser can also be created during the system deployment. The procedure of system deployment could be as follows:
(a) Copy all system programs and data to the production server
(b) Collect static data and copy them to the destination for the static file server
(c) Run the init() app to create necessary initial data, including the admin account
(d) Set DEBUG = False in settings.py Implement the init() app:
63
In main.views.py, add the init() fucntion: import os from django.contrib.auth.models import User from rango.models import Category ... def main(request): ... def init(request): if User.objects.filter(username='admin'): return render(request, 'main/init.html', 'abort':True) # Create 'admin' account User.objects.all().delete() user = User() user.username = 'admin' user.set_password('admin') user.is_staff = True user.is_superuser = True user.is_active = True user.save() # Init Category Category.objects.all().delete() infile = open(os.path.dirname(os.path.realpath(__file__))+'/category.txt', 'r') for line in infile: category = Category() category.name = line category.save() infile.close() # Init other data ... # Retrieve everything users = User.objects.all() categories = Category.objects.all() ... context = 'users':users, 'categories':categories return render(request, 'main/init.html', context) # end of init()
In the above, Check if admin is already in the User model, if yes, don't
initialize the project Delete everything in User model and then create an admin
user Delete everything in the models that we would like to
initialize, e.g., Category, and then populate the initial data using the data from a file we provide (category.txt). The format of the categories in category.txt should be one category per line, for example:
64
Django Python Other frameworks ...
Finally, after initialization, retrieve everything and display them on the page init.html.
Create a template main/templates/init.html: <!doctype html> % load staticfiles % <html> <head> <title>Tango with Django</title> <meta charset=utf8> <link rel=stylesheet href=% static 'main/css/main.css' %> </head> <body> % if abort % <h1>Initialization aborted.</h1> % else % <h1>Initial data:</h1> <h1>Users:</h1> % for user in users % user.username<br> % endfor % <h1>Categories:</h1> % for category in categories % category.name<br> % endfor % % endif % </body> </html>
Add another URL mapping in the project's tango/urls.py (again, url(r'.*', ...) has to be at the last line): url(r'rango/', include('rango.urls')), url(r'main/', include('main.urls')), url(r'.*', include('main.urls')),
Add a URL mapping in main/urls.py: urlpatterns = patterns('', url(r'init/$', views.init, name='init'), url(r'.*', views.main, name='main'), )
Test by entering the url in the browser: http://localhost:8000/main/init/
8. Exercises
65
(a) Write codes such that only registered users can add/edit, while nonregistered can only view/use the categories/pages.
Add the @login_required decorator on addPage() and addCategory() views. In category.html, check if the user is logged in. If yes, show the Add a Page link; otherwise show nothing: % if user.is_authenticated % <a href="/rango/ categoryName /add_page/">Add a Page</a> % endif %
(b) Provide informative error messages when users incorrectly enter their username or password.
We shouldn't really want to do this! However, since the tutorial ask us to do it, here we go. from django.contrib.auth.models import User ... # The following try: user = User.objects.get(username=username) except User.DoesNotExist: return HttpResponse('Invalid username: 0'.format(username)) user = authenticate(username=username, password=password) if not user: return HttpResponse('Invalid password: 0'.format(password)) if not user.is_active: return HttpResponse("Your Rango account is disabled.") login(request, user) return HttpResponseRedirect('/rango/')
(c) Visit https://djangoregistration.readthedocs.org/en/latest/ to find out more about using the user registration package.
Now push your project.
66
Chapter 9: Working with Templates
There are a lot of same HTML codes in various HTML files. We will use templates to minimize the copyandpaste of HTML codes.
Steps
1. Identify the reoccurring parts of each HTML pages (i.e. header bar, sidebar, footer, content pane).
2. In a base template, provide the skeleton structure of a standard page along with any common content. And then define a number of blocks which are subject to change depending on which page the user is viewing.
3. Create specific templates which inherit from the base tempalte, and specify the contents of each block.
1. Reoccuring HTML and the base template
The common part (color blue) of various templates in the project: <!doctype html> % load staticfiles % <html> <head> <title>Tango with Django</title> <meta charset=utf8> <link rel=stylesheet href=% static 'main/css/main.css' %> </head> <body> <! Page specific content goes here > </body> </html>
These common part should not be inherited. Supposing a template base.html has the following contents: <!doctype html> % load staticfiles % <html> <head> <title>Tango with Django</title> <meta charset=utf8> <link rel=stylesheet href=% static 'main/css/main.css' %> </head> <body> % block content %% endblock % </body> </html>
In the above, a template block is denoted by % block <blockName> %, where <blockName> is the name of the block, and % endblock % is an ending tag. You may also include default content of the block, for example:
67
% block content %This is the default content.% endblock %
If the content of the % block content % is not replaced with anything, it's result will be empty: ... <body> </body> </html>
If a template inherits from base.html, it can supply the content of block content. For example, the content of index.html: % extends "main/base.html" % % block content %<p>This is the text to insert.</p>% endblock %
The resulting page would be: <!doctype html> % load staticfiles % <html> <head> <title>Tango with Django</title> <meta charset=utf8> <link rel=stylesheet href=% static 'main/css/main.css' %> </head> <body> <p>This is the text to insert.</p> </body> </html>
2. More blocks
Modify main/templates/main/main.html with the following contents: <!doctype html> % load staticfiles % <html> <head> <title>Tango with Django</title> <meta charset=utf8> <link rel=stylesheet href=% static 'main/css/main.css' %> </head> <body> <h1>Tango with Django% block heading %% endblock %</h1> <div> % block content %% endblock % </div> <br> <hr> <div> <ul> % if user.is_authenticated % <li><a href=/rango/restricted/>Restricted Page</a></li>
68
<li><a href=/rango/userLogout/>Logout</a></li> <li><a href=/rango/addCategory/>Add a New Category</a></li> % else % <li><a href=/rango/register/>Register Here</a></li> <li><a href=/rango/userLogin/>Login</a></li> % endif % <li><a href=/rango/about/>About</a></li> <li><a href=/rango/>Back to Rango</a></li> </ul> </div> </body> </html>
In the above: The first block is called heading without default content The second block is called content without default content A series of links are at the bottom of the page
3. Template inheritance
Since main.html is ready, other templates can inherit from it. For example, we can modify category.html as follows. The beginning part: <!doctype html> % load staticfiles % <html> <head> <title>Tango with Django</title> <meta charset=utf8> <link rel=stylesheet href=% static
The resulting code becomes: % extends "main/main.html" %
69
% block heading % List Pages% endblock % % block content % % if category % <h1>category.name</h1> % if pages % <ul> % for page in pages % <li><a href=page.url>page.title</a></li> % endfor % </ul> % else % <p><strong>No pages currently in category.</strong></p> % endif % <br><br> <a href="/rango/category.slug/addPage/">Add a New Page</a><br><br> % else % <p>The specified category "categoryName" does not exist.</p> % endif % % endblock %
Important!
In main.html, there is user.is_authenticated statement, which means main.html has to have access to the content of the request. Since all other templates now inherit from main.html, that means all HTMLs have to have access to the request. Using render(request, ...) in the views can achieve that, however, HttpResponse(...) will not. Therefore, HttpResponse(...) should not be used anymore. Modify userLogin() in rango/views.py:
def userLogin(request): template = 'rango/userLogin.html' if request.method == 'GET': return render(request, template) ... if not user: return render(request, template, 'errorMessage':'Incorrect login information. Try again.') return HttpResponse("Invalid login details supplied.") if not user.is_active: return render(request, template, 'errorMessage':'The user is diabled. Please contact the site manager.') return HttpResponse("Your Rango account is disabled.") Add a line for displaying an error message in userLogin.html:
... <h1>Login to Rango</h1> errorMessage<br><br> <form id=loginForm method=post action=/rango/userLogin/> ...
70
4. Further works
(a) Update all other templates to extends from main/main.html. about.html: % extends 'main/main.html' % % block heading % About% endblock % % block content % <h2>Rango Says: Here is the about page.</h2> % endblock %
AddCategory.html: % extends 'main/main.html' % % block heading % Add a Category% endblock % % block content % <h2>Add a Category</h2> <form id=categoryForm method=post action=/rango/addCategory/> % csrf_token % % for hidden in form.hidden_fields % hidden % endfor % % for field in form.visible_fields % field.errors field.help_text field % endfor % <input type=submit value="Create Category"> </form> % endblock % addPage.html:
% extends 'main/main.html' % % block heading % Add a Page% endblock % % block content % <h2>Add a new page in category.name</h2> <form method=post action=/rango/category.slug/addPage/> % csrf_token % % for hidden in form.hidden_fields % hidden % endfor % % for field in form.visible_fields % field.errors field.help_text field<br> % endfor % <input type=submit value="Create Page"> </form>
71
% endblock %
category.html:
% extends "main/main.html" % % block heading % List Pages% endblock % % block content % % if category % <h2>category.name</h2> % if pages % <ul> % for page in pages % <li><a href=page.url>page.title</a></li> % endfor % </ul> % else % <p><strong>No pages currently in category.</strong></p> % endif % <br><br> <a href="/rango/category.slug/addPage/">Add a New Page</a><br><br> % else % <p>The specified category "categoryName" does not exist.</p> % endif % % endblock %
The current project layout under GitHub's version control is as follows: project/ git/ tango/ .git/ tango/ tango/ .gitignore manage.py virtualenv/ tangoVenv/ workspace/
1. Install djangotoolbelt for Heroku
Install system toolbelt package on Ubuntu:
$ wget qO https://toolbelt.heroku.com/installubuntu.sh | sh
Create a new virtual environment herokuVenv: $ cd <...>/project/virtualenv
Interpreters → Python Interpreter → (Right panel) New → Interpreter Name: herokuPython3, Interpreter Executable: <...>/project/virtualenv/herokuVenv/bin/python3 → OK → OK → OK
Set the new interpreter for the tango project: Right click project → Properties → (Left panel) PyDev Interpreter/Grammar → (Right panel) Interpreter: herokuPython3 → OK
2. Create files to specify the runtime environment for Heroku
Create the following 3 files in the project root directory:
Specify the application server: Procfile web: gunicorn tango.wsgi logfile
Database migrate will be performed remotely on Heroku.
6. Create a deployment folder, create a new Git repository, and deploy the project
Because Heroku also uses Git for file upload, we need to clone the project and create a new Git repository in order not to interfere with the original.
1. Create a new folder <...>/project/deploy 2. Clone the project 3. Create a git repository in the Git project root
$ cd <...>/project $ mkdir deploy $ cd deploy $ cp r <projGitRoot> .
Make sure the content of .gitignore is as follows:
.*
77
*~ *.pyc
Now the project layout becomes: <...>/project/ git/ tango/ .git/ tango/ manage.py deploy/ tango/ tango/ .git/ manage.py virtualenv/ herokuVenv/ workspace/
(E.g., <appName>=tango12345 as tango may not be available) This command also creates a config file in .git directory with the content: [core] repositoryformatversion = 0 filemode = true bare = false logallrefupdates = true [remote "heroku"] url = https://git.heroku.com/<appName>.git fetch = +refs/heads/*:refs/remotes/heroku/*
Push to Heroku: $ git push heroku master
78
On the Heroku's Resources page, check to see if a postgres database is created.
Perform migrate remotely on Heroku: $ heroku run python3 manage.py migrate
Set Heroku dyno (Heroku may have done this automatically): $ heroku ps:scale web=1
Create the superuser: $ heroku run python3 manage.py createsuperuser
Test: http://<appName>.herokuapp.com
7. Get more addons
On Heroku's Resources page, click "Get more addons" and add the following addons: (1) Data Store Utilities: PostgreSQL Studio (beta) (2) Monitoring: New Relic APM (3) Logging: Papertrail (4) PostgreSQL Studio
8. Other Heroku commands
(a) Clear database $ heroku pg:reset DATABASE_URL
(b) Check logs $ heroku logs tail
(b) Run bash $ heroku run bash
9. Deployment scripts
The process of deploying to Heroku is very complex, so it's better to automate it using a script.
79
There are two types of deployment:
Clean deploy: deletes all migration information and data tables on Heroku
Followup deploy: retains all migration information and data tables on Heroku
Before deployment:
1. Create the app on Heroku if it is the first deploy 2. Login heroku 3. Make sure the contents of .gitignore,
requirements.txt, and runtime.txt are correct .gitignore
Clean Deploy Followup deploy
.* *~ *.pyc 00*_*
.* *~ *.pyc
Run the following deploy script (set file mode to executable: $ chmod +x deploy) #!/bin/bash # # Deploy the project to Heroku. # # Prerequisites: # 1. The heroku app has already been created # 2. Login heroku # 3. Make sure the contents of ".gitignore", "requirements.txt", and # "runtime.txt" are correct # # Project layout: # project/ # deploy/ # tango/ # tango/ # .git/ # tango/ # manage.py # deploy # git/ # tango/ # .git/ # tango/ # tango/ # manage.py # virtualenv/ # herokuVenv/ # bin/ # activate
80
# workspace/ # # Variables (no spaces before or after "="): # projName=<name of the project> # appName=<name of the app on Heroku> # root=<...>/project # projRoot=$root/git/$projName/$projName # activate=<path of the activate file> # projName=tango appName=tango12345 root=<...>/project deploy=$root/deploy/$projName projRoot=$root/git/$projName/$projName activate=$root/virtual.env/herokuVenv/bin/activate read p "Perform (1)followup or (2)clean deployment? (1/2)? " deployType if [ "$deployType" = "1" ]; then cleanDeploy=false else if [ "$deployType" = "2" ]; then echo e "\nDANGER!! All data on Heroku will be deleted!" read p "Are you ABSOLUTELY sure (yes)? " yes if [ "$yes" = "yes" ]; then cleanDeploy=true else echo e "Abort.\n" exit fi else echo e "Abort.\n" exit fi fi if [ ! f requirements.txt ]; then echo e "File 'requirements.txt' does not exit." exit fi if [ ! f runtime.txt ]; then echo e "File 'runtime.txt' does not exit." exit fi if [ ! f gitignore ]; then echo e "File 'gitignore' does not exit." exit fi echo e "\nClone the project." if [ ! d $projRoot ]; then echo e "$projRoot does not exist.\n" exit fi rm rf ./$projName cp r $projRoot .
81
cd $projName cp ../requirements.txt . cp ../runtime.txt . cp ../gitignore .gitignore if [ "$cleanDeploy" = true ]; then echo e "\nDelete '00*_*' files." find . name "00*_*" exec rm \; fi echo e "\nRun 'makemigrations'." source $activate python3 manage.py makemigrations echo e "\nCreate git repository ..." rm rf .git git init git remote add heroku https://git.heroku.com/$appName.git echo e "\nCommit all files ..." git add . git commit m "Deploying" echo e "\nPush to heroku ..." git push force heroku master if [ "$cleanDeploy" = true ]; then echo e "\nClear Heroku's database ..." echo $appName | heroku pg:reset DATABASE_URL fi echo e "\nRun 'migrate' on Heroku ..." heroku run python3 manage.py migrate if [ "$cleanDeploy" = true ]; then echo e "\nSet 'web dyno'=1. Adjust if necessary." heroku ps:scale web=1 fi echo e "\nDeployment completed."
After the system goes production, when a model needs to be changed, make sure only one member may make the change at a time. Follow the steps below:
1. Make sure no other member is making any model changes 2. Change the model and perform makemigrations 3. Push to Git as soon as possible (include all 00*_*
migration files) 4. Other members pull form Git (also include all 00*_*
migration files) 5. Members perform migrate on his/her machine 6. And then proceed to work on other code
82
83
Chapter 10: Deploying to an Ubuntu server 3
1. Install database and vsftpd on the production server
Install postgres on Ubuntu using the native package: $ sudo aptget install libpqdev postgresql postgresqlcontrib
Install vsftpd: $ sudo aptget install vsftpd
Edit the setting file: /etc/vsftpd.conf anonymous_enable=NO local_enable=YES write_enable=YES chroot_local_user=YES
2. Why use Nginx as a reverseproxy in front of an application server (Gunicorn)?
Many frameworks and application servers (including Gunicorn) can serve static files (e.g. javascript, css, images etc.) together with responses. However, the better thing to do is to let a (reverseproxy) server such as Nginx handle the task of serving these files and managing connections (requests). This relieves a lot of the load from the application servers, granting you a much better overall performance.
As your application grows, you will want to optimize it— and when the time comes, distribute it across servers (VPS) to be able to handle more connections simultaneously (and have a generally more robust architecture). Having a reverseproxy in front of your application server(s) helps you with this from the very beginning.
Now, copy the entire project to /webapps/tango/tango (or ftp the project files).
Turn off debugging in /webapps/tango/tango/tango/settings.py: DEBUG = False TEMPLATE_DEBUG = False
4. Set up the database
$ sudo i u postgres postgres@<username> $ createdb "tangoDB" postgres@<username> $ createuser P tango Enter password for new role: xxxxxxxx Enter it again: xxxxxxxx postgres@<username>$ psql postgres=# grant all privileges on database "tangoDB" to tango;
5. Serving Python Web Applications with Gunicorn Web Server
Gunicorn is a WSGI compliant server and you need to pass your application object to it. Django provides an application object wsgi.py. Run gunicorn by passing it the wsgi application and set the port to 8001: $ source <venv>/bin/activate (venv)$ cd /webapps/tango/tango (venv)$ gunicorn tango.wsgi:application bind=localhost:8001 &
To stop Gunicorn, use the following command: $ killall gunicorn
To further enhance the server, we may create multiple workers. We don't want the server to be trapped in I/O bound processes. Therefore, when a worker is waiting for I/O, another server may take over other requests (see here). For example: (venv)$ gunicorn tango.wsgi:application bind=localhost:8001
workers=9 &
The suggested amount of workers is set according to the following simple formula:
Total workers = (2 workers * CPU cores) + 1 For example: For 1 core → (2*1) + 1 = 3
Note, the page is served without static files because we haven't run nginx yet.
6. Install and configure Nginx
(Read this)
Install Nginx: $ sudo aptget install nginx
Nginx acts as a reverse proxy. All the requests will initially come to Nginx and it will decide what requests it wants to serve and what requests it wants to pass to some other server. In our case, we
want Nginx to serve requests for static resources and to pass any other request to Gunicorn.
We need to tell nginx the location of our static resources and for that we need to make sure all our static resources are at a single location:
/webapps/tango/static: /main /css /img /js /rango /css /img /js Create a file /etc/nginx/sitesenabled/tango: $ cd /etc/nginx/sitesenabled $ sudo vi tango
Enter the following content: access_log /webapps/tango/logs/nginxAccess.log; error_log /webapps/tango/logs/nginxError.log; server listen xxx.xxx.xxx:80; location / proxy_pass http://127.0.0.1:8001;
location /static/ root /webapps/tango/;
where xxxx.xxx.xxx is a real domain name.
Or, enter the following content (not tested yet): access_log /webapps/tango/logs/nginxAccess.log; error_log /webapps/tango/logs/nginxError.log; server listen 80; server_name my.domain.name; location / proxy_pass http://127.0.0.1:8001;
(d) Comment out access_log and error_log in /etc/nginx/nginx.conf: $ cd /etc/nginx $ sudo vi nginx.conf #access_log /var/log/nginx/access.log; #error_log /var/log/nginx/error.log;
Django's django.contrib.staticfiles provides a command for gathering static files in a single directory. Run the collectstatic management command $ source <venv>/bin/activate (venv)$ cd /webapps/tango/tango (venv)$ python3 manage.py collectstatic You have requested to collect static files at the destination location as specified in your settings:
/webapps/tango/static This will overwrite existing files! Are you sure you want to do this? Type 'yes' to continue, or 'no' to cancel: yes
...
<n> static files copied to '/webapps/tango/static'.
C. Migrate the database (venv)$ cd /webapps/tango/tango (venv)$ python3 manage.py makemigrations (venv)$ python3 manage.py migrate
Now test: http://xxx.xxx.xxx/rango/
90
Summary Steps of firsttime system deployment:
1. Install nginx, gunicorn, and vsftpd
2. Install Postgres
3. Create a database (tangoDB) and the database user (tango) and grant all privileges
4. Create the whole file structure
5. FTP all application files
6. Create the virtual environment according to requirements.txt
7. Execute: source bin/activate and python3 manage.py collectstatic
8. Make database migrations (makemigrations, migrate)
