Skip to content

Commit c14d86a

Browse files
new release
- Add support for logging - Improve the way connection arguments are passed - Add some DEBUG logging to make debugging easier
1 parent 0eb23b0 commit c14d86a

183 files changed

Lines changed: 40487 additions & 4455 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,3 @@
11
*.pyc
22
dbconnector.egg-info/
3+
example_db_options.secret

README.md

Lines changed: 76 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -1,33 +1,66 @@
1-
# MySQL-DBConnector #
1+
# MySQL-DBConnector
22

33
A wrapper for the MySQL Python connector that provides additional resilience and functionality.
44

5-
## What is this repository for? ##
5+
**Latest version: 1.3.1**
6+
7+
## About this repository
68

79
* A wrapper for the MySQL Python connector that provides additional resilience and functionality.
810
* Implements connection pooling that's more reliable.
9-
* Version 1.2.1
10-
* Works with Python 2.7+ or 3.5+
11+
* Developed and tested with Python 3.9, should work with 3.5+
1112

12-
## How do I get set up? ##
13+
## How do I get set up?
1314

1415
* Make sure you have Git installed - [Download Git](https://git-scm.com/downloads)
1516
* Run `pip install git+https://github.com/SheffieldSolar/MySQL-DBConnector/`
1617

17-
## Getting started ##
18-
I recommend using a MySQL options file to pass the connection parameters rather than hard-coding the DB credentials:
18+
## Usage
19+
Start by creating an instance of the `DBConnector` class, ideally using a context manager as this will ensure all connections are closed if an error is encountered or the code is interrupted:
20+
21+
```Python
22+
from dbconnector import DBConnector
23+
```
24+
25+
The `DBConnector` class requires at least one argument: `connector_args`. This should be a dictionary of connection arguments, the full list of which can be found [here](https://dev.mysql.com/doc/connector-python/en/connector-python-connectargs.html). Most of these are optional, but you'll need to specify at least:
26+
27+
```Python
28+
connector_args = {
29+
"user": "<your-username>",
30+
"password": "<your-password>",
31+
"database": "<your-database>",
32+
}
33+
```
34+
35+
Some of the connector args have default values which override the defaults in the mysql-connector-python library, they are:
36+
37+
- `time_zone`: "UTC"
38+
- `connection_timeout`: 60
39+
- `buffered`: True
40+
- `get_warnings`: True
41+
- `raise_on_warnings`: False
42+
- `use_pure`: True
43+
44+
You can override these yourself if you prefer.
45+
46+
Loading your password into your Python code is usually insecure, so I recommend using a MySQL options file to pass the connection parameters rather than hard-coding the DB credentials:
1947
```Python
2048
from dbconnector import DBConnector
2149

2250
mysql_options_file = "mysql_options.txt"
2351

24-
# Use the context manager to ensure DB connections are always safely closed
25-
# Always specify the session_tz to avoid issues with MySQL's timezone-naive datetimes or unwanted TZ conversions of timestamp fields!
26-
with DBConnector(mysql_options_file, session_tz="UTC") as dbc:
52+
# HINT: Use the context manager to ensure DB connections are always safely closed
53+
54+
with DBConnector(connector_args={"option_files": mysql_options_file}) as dbc:
2755
# Selecting from the database (returns a list of tuples):
2856
# See https://dev.mysql.com/doc/connector-python/en/connector-python-api-mysqlcursor-fetchall.html
57+
# Note that using `select * from` is not recommended since the returned columns or their order
58+
# might change.
2959
data = dbc.query("SELECT * FROM test;")
3060

61+
# Select from the database and convert to a Pandas DataFrame:
62+
data = dbc.query("SELECT col2, col2 FROM test;", df=True)
63+
3164
# Inserting one row into the database:
3265
dbc.iud_query("insert into test (col1, col2, col3) values ('val1', 'val2', 'val3');")
3366

@@ -39,7 +72,9 @@ with DBConnector(mysql_options_file, session_tz="UTC") as dbc:
3972
args = (1, 2, 3)
4073
result = dbc.proc("myProcedure", args)
4174
```
75+
4276
The file _'mysql_options.txt'_ should look something like this:
77+
4378
```
4479
[client]
4580
host = <your-server-hostname>
@@ -49,32 +84,50 @@ database = <your-database>
4984
```
5085

5186
It is also possible to pass connection parameters directly to the DBConnector class. This is not recommended but can be useful for testing/debugging:
87+
5288
```Python
5389
from dbconnector import DBConnector
5490

55-
db_config={"host": "<your-server-hostname>", "user": "<your-username>",
56-
"password": "<your-password>", "database": "<your-database>"}
57-
58-
with DBConnector(db_config=db_config, session_tz="UTC") as dbc:
91+
# HINT: Always specify the time_zone arg to avoid issues with MySQL's timezone-naive datetimes or
92+
# unwanted TZ conversions of timestamp fields!
93+
connector_args = {
94+
"host": "<your-server-hostname>",
95+
"user": "<your-username>",
96+
"password": "<your-password>",
97+
"database": "<your-database>",
98+
"time_zone": "UTC"
99+
}
100+
101+
with DBConnector(connector_args=connector_args) as dbc:
59102
# Do some DB stuff
60103
```
61104

62-
## Documentation ##
105+
The `dbconnector` module uses the Python `logging` module to log useful debugging messages, warnings and errors. You can set your preferred logging level using something like:
106+
107+
```Python
108+
import os
109+
import logging
110+
111+
fmt = "%(asctime)s %(module)s %(levelname)s %(message)s"
112+
logging.basicConfig(format=fmt, level=os.environ.get("LOGLEVEL", "DEBUG"))
113+
```
114+
115+
## Documentation
63116

64-
* [https://sheffieldsolar.github.io/MySQL-DBConnector/](https://sheffieldsolar.github.io/MySQL-DBConnector/)
117+
- [https://sheffieldsolar.github.io/MySQL-DBConnector/](https://sheffieldsolar.github.io/MySQL-DBConnector/)
65118

66-
## How do I update? ##
119+
## How do I update?
67120

68-
* Run `pip install --upgrade git+https://github.com/SheffieldSolar/MySQL-DBConnector/`
121+
- Run `pip install --upgrade git+https://github.com/SheffieldSolar/MySQL-DBConnector/`
69122

70-
## Who do I talk to? ##
123+
## Who do I talk to?
71124

72-
* Jamie Taylor - [jamie.taylor@sheffield.ac.uk](mailto:jamie.taylor@sheffield.ac.uk "Email Jamie") - [SheffieldSolar](https://github.com/SheffieldSolar)
125+
- [Jamie Taylor](https://github.com/JamieTaylor-TUOS) - [jamie.taylor@sheffield.ac.uk](mailto:jamie.taylor@sheffield.ac.uk "Email Jamie") - [SheffieldSolar](https://github.com/SheffieldSolar)
73126

74-
## Authors ##
127+
## Authors
75128

76-
* **Jamie Taylor** - *Initial work* - [SheffieldSolar](https://github.com/SheffieldSolar)
129+
- [Jamie Taylor](https://github.com/JamieTaylor-TUOS) - [SheffieldSolar](https://github.com/SheffieldSolar)
77130

78-
## License ##
131+
## License
79132

80133
This project is licensed under the MIT License - see the [LICENSE.md](LICENSE.md) file for details

0 commit comments

Comments
 (0)