Python and databases. Good enough is sometimes best

With Django sub forum.
Site Admin
Posts: 178
Joined: Tue Sep 29, 2009 6:25 pm
Location: Moss Norway

Python and databases. Good enough is sometimes best

Postby KBleivik » Sat Mar 03, 2012 12:21 pm

1. Make it simple.

If you have some experience in using Python, you may know that the minimalistic SQL database platform sqlite comes as a standalone file with your Python installation. For some simple soultions SQLite is good enough. If you are mining text documents, there may be even better solutions like CouchDB.

Apache CouchDB is a document-oriented database that can be queried and indexed using JavaScript in a MapReduce fashion. CouchDB also offers incremental replication with bi-directional conflict detection and resolution.

CouchDB provides a RESTful JSON API than can be accessed from any environment that allows HTTP requests. There are myriad third-party client libraries that make this even easier from your programming language of choice. CouchDB’s built in Web administration console speaks directly to the database using HTTP requests issued from your browser.


or even simpler Redis

Redis is an open source, advanced key-value store. It is often referred to as a data structure server since keys can contain strings, hashes, lists, sets and sorted sets.


A key - value combinantion? Can that be all you need? Ask yourself.

2. Don't use a canon to shoot sparrows. But security ...

The canons are of course proprietary data base platforms like Oracle, Sybase, MS SQL and similar proprietary platfroms. Many companies prefer free platforms like MySQL and PostGreSQL. Security is of course a concern, but don't fool yourself to think that your solution is secure enough:

Is your online password protected database on a secure server really secure?

No platform can compensate for bad programming and insecure code. An obvious example is php where a use of prepared statements and the PDO data base object can be sufficient for some tasks. Study the code in this document where an effecient (object relational) mapping between a PostGreSQL database and your web application is described.

Of course, if you need a large and efficient, secure platform where monitoring the network traffic to and from the database is important, the best solution may be a proprietary platform.

3. Litterature and links ... ith_Python ... in-Python/

4. Measure 10 times and saw one.

The top of laziness is to do everything correct from the beginning. Think ahead. Think 5 - 10 years into the future. Ask yourself. What do I need? Is a key value combination that can be loaded into json structure all I need? Is security important? Do I need a database that can be run on the same server as your site? Or do I need one or more stand alone data base servers? Do I even need multiple data senters. The solution you choose is very important, so if you can not answer the above questions alone, contact a database expert that don't shoot sparrows with canons.

Example: If you need to run a big board you may soon run into problems if you don't know the bottlenecks. But for text mining, like it is described in this book a simple Python / Redis / CouchDB solution may be good enough. Sometimes good enough is best. And SQLite comes with Python and is implemented in your file system. It is minimalistic and easy to use on a your laptop that you can carry with you on a holiday.

Return to “Python”

Who is online

Users browsing this forum: No registered users and 1 guest