Following the SQLite3 series, this post is about some advanced topics when we are working with the SQLite3 module. If you missed the first part, you can find it here.

Using SQLite's date and datetime Types

Sometimes we need to insert and retrieve some date and datetime types in our SQLite3 database. When you execute the insert query with a date or datetime object, the sqlite3 module calls the default adapter and converts them to an ISO format. When you execute a query in order to retrieve those values, the sqlite3 module is going to return a string object:

[python]
>>> import sqlite3
>>> from datetime import date, datetime
>>>
>>> db = sqlite3.connect(':memory:')
>>> c = db.cursor()
>>> c.execute('''CREATE TABLE example(id INTEGER PRIMARY KEY, created_at DATE)''')
>>>
>>> # Insert a date object into the database
>>> today = date.today()
>>> c.execute('''INSERT INTO example(created_at) VALUES(?)''', (today,))
>>> db.commit()
>>>
>>> # Retrieve the inserted object
>>> c.execute('''SELECT created_at FROM example''')
>>> row = c.fetchone()
>>> print('The date is {0} and the datatype is {1}'.format(row[0], type(row[0])))
# The date is 2013-04-14 and the datatype is <class 'str'>
>>> db.close()
[/python]

The problem is that if you inserted a date object in the database, most of the time you are expecting a date object when you retrieve it, not a string object. This problem can be solved passing PARSE_DECLTYPES and PARSE_COLNAMES to the connect method:

[python]
>>> import sqlite3
>>> from datetime import date, datetime
>>>
>>> db = sqlite3.connect(':memory:', detect_types=sqlite3.PARSE_DECLTYPES|sqlite3.PARSE_COLNAMES)
>>> c = db.cursor()
>>> c.execute('''CREATE TABLE example(id INTEGER PRIMARY KEY, created_at DATE)''')
>>> # Insert a date object into the database
>>> today = date.today()
>>> c.execute('''INSERT INTO example(created_at) VALUES(?)''', (today,))
>>> db.commit()
>>>
>>> # Retrieve the inserted object
>>> c.execute('''SELECT created_at FROM example''')
>>> row = c.fetchone()
>>> print('The date is {0} and the datatype is {1}'.format(row[0], type(row[0])))
# The date is 2013-04-14 and the datatype is <class 'datetime.date'>
>>> db.close()
[/python]

Changing the connect method, the database now is returning a date object. The sqlite3 module uses the column's type to return the correct type of object. So, if we need to work with a datetime object, we must declare the column in the table as a timestamp type:

[python]
>>> c.execute('''CREATE TABLE example(id INTEGER PRIMARY KEY, created_at timestamp)''')
>>> # Insert a datetime object
>>> now = datetime.now()
>>> c.execute('''INSERT INTO example(created_at) VALUES(?)''', (now,))
>>> db.commit()
>>>
>>> # Retrieve the inserted object
>>> c.execute('''SELECT created_at FROM example''')
>>> row = c.fetchone()
>>> print('The date is {0} and the datatype is {1}'.format(row[0], type(row[0])))
# The date is 2013-04-14 16:29:11.666274 and the datatype is <class 'datetime.datetime'>
[/python]

In case you have declared a column type as DATE, but you need to work with a datetime object, it is necessary to modify your query in order to parse the object correctly:

[python]
c.execute('''CREATE TABLE example(id INTEGER PRIMARY KEY, created_at DATE)''')
# We are going to insert a datetime object into a DATE column
now = datetime.now()
c.execute('''INSERT INTO example(created_at) VALUES(?)''', (now,))
db.commit()

# Retrieve the inserted object
c.execute('''SELECT created_at as "created_at [timestamp]" FROM example''')
[/python]

Using as "created_at [timestamp]" in the SQL query will make the adapter to parse the object correctly.

Insert Multiple Rows with SQLite's executemany

Sometimes we need to insert a sequence of objects in the database, the sqlite3 module provides the executemany method to execute a SQL query against a sequence.

[python]
# Import the SQLite3 module
import sqlite3
db = sqlite3.connect(':memory:')
c = db.cursor()
c.execute('''CREATE TABLE users(id INTEGER PRIMARY KEY, name TEXT, phone TEXT)''')
users = [
('John', '5557241'),
('Adam', '5547874'),
('Jack', '5484522'),
('Monthy',' 6656565')
]

c.executemany('''INSERT INTO users(name, phone) VALUES(?,?)''', users)
db.commit()

# Print the users
c.execute('''SELECT * FROM users''')
for row in c:
print(row)

db.close()
[/python]

Please note that each element of the sequence must be a tuple.

Execute SQL File with SQLite's executescript

The execute method only allows you to execute a single SQL sentence. If you need to execute several different SQL sentences you should use executescript method:

[python]
# Import the SQLite3 module
import sqlite3
db = sqlite3.connect(':memory:')
c = db.cursor()
script = '''CREATE TABLE users(id INTEGER PRIMARY KEY, name TEXT, phone TEXT);
CREATE TABLE accounts(id INTEGER PRIMARY KEY, description TEXT);

INSERT INTO users(name, phone) VALUES ('John', '5557241'),
('Adam', '5547874'), ('Jack', '5484522');'''
c.executescript(script)

# Print the results
c.execute('''SELECT * FROM users''')
for row in c:
print(row)

db.close()
[/python]

If you need to read the script from a file:
[python]
fd = open('myscript.sql', 'r')
script = fd.read()
c.executescript(script)
fd.close()
[/python]

Please remember that it is a good idea to surround your code with a try/except/else clause in order to catch the exceptions. To learn more about the try/except/else keywords, checkout the Catching Python Exceptions – The try/except/else keywords article.

Defining SQLite SQL Functions

Sometimes we need to use our own functions in a statement, specially when we are inserting data in order to accomplish some specific task. A good example of this is when we are storing passwords in the database and we need to encrypt those passwords:

[python]
import sqlite3 #Import the SQLite3 module
import hashlib

def encrypt_password(password):
# Do not use this algorithm in a real environment
encrypted_pass = hashlib.sha1(password.encode('utf-8')).hexdigest()
return encrypted_pass

db = sqlite3.connect(':memory:')
# Register the function
db.create_function('encrypt', 1, encrypt_password)
c = db.cursor()
c.execute('''CREATE TABLE users(id INTEGER PRIMARY KEY, email TEXT, password TEXT)''')
user = ('[email protected]', '12345678')
c.execute('''INSERT INTO users(email, password) VALUES (?,encrypt(?))''', user)
[/python]

The create_function takes 3 parameters: name (the name used to call the function inside the statement), the number of parameters the function expects (1 parameter in this case) and a callable object (the function itself). To use our registered function, we called it using encrypt() in the statement.

Finally, PLEASE use a true encryption algorithm when you are storing passwords!