Home » Python » Python SQL query string formatting

Python SQL query string formatting

Posted by: admin November 30, 2017 Leave a comment

Questions:

I’m trying to find the best way to format an sql query string. When I’m debugging
my application I’d like to log to file all the sql query strings, and it is
important that the string is properly formated.

Option 1

def myquery():
    sql = "select field1, field2, field3, field4 from table where condition1=1 and condition2=2"
    con = mymodule.get_connection()
    ...
  • This is good for printing the sql string.
  • It is not a good solution if the string is long and not fits the standard width
    of 80 characters.

Option 2

def query():
    sql = """
        select field1, field2, field3, field4
        from table
        where condition1=1
        and condition2=2"""
    con = mymodule.get_connection()
    ...
  • Here the code is clear but when you print the sql query string you get all these annoying white spaces.

    u’\nselect field1, field2, field3, field4\n_____from table\n____where condition1=1 \n_____and condition2=2′

Note: I have replaced white spaces with underscore _, because they are trimmed by the editor

Option 3

def query():
    sql = """select field1, field2, field3, field4
from table
where condition1=1
and condition2=2"""
    con = mymodule.get_connection()
    ...
  • I don’t like this option because it breaks the clearness of the well tabulated code.

Option 4

def query():
    sql = "select field1, field2, field3, field4 " \
          "from table " \
          "where condition1=1 " \
          "and condition2=2 "
    con = mymodule.get_connection()    
    ...
  • I don’t like this option because all the extra typing in each line
    and is difficult to edit the query also.

For me the best solution would be Option 2 but I don’t like the extra whitespaces when I print the sql string.

Do you know of any other options?

Answers:

Sorry for posting to such an old thread — but as someone who also shares a passion for pythonic ‘best’, I thought I’d share our solution.

The solution is to build SQL statements using python’s String Literal Concatenation (http://docs.python.org/), which could be qualified a somewhere between Option 2 and Option 4

Code Sample:

sql = ('select field1, field2, field3, field4 '
       'from table '
       'where condition1=1 '
       'and condition2=2 ')

Pros:

  1. It retains the pythonic ‘well tabulated’ format, but does not add extraneous space characters (which pollutes logging).
  2. It avoids the backslash continuation ugliness of Option 4, which makes it difficult to add statements (not to mention white-space blindness).
  3. And further, it’s really simple to expand the statement in VIM (just position the cursor to the insert point, and press SHIFT-O to open a new line).
Questions:
Answers:

You’ve obviously considered lots of ways to write the SQL such that it prints out okay, but how about changing the ‘print’ statement you use for debug logging, rather than writing your SQL in ways you don’t like? Using your favourite option above, how about a logging function such as this:

def debugLogSQL(sql):
     print ' '.join([line.strip() for line in sql.splitlines()]).strip()

sql = """
    select field1, field2, field3, field4
    from table"""
if debug:
    debugLogSQL(sql)

This would also make it trivial to add additional logic to split the logged string across multiple lines if the line is longer than your desired length.

Questions:
Answers:
sql = ("select field1, field2, field3, field4 "
       "from table "
       "where condition1={} "
       "and condition2={}").format(1, 2)

Output: 'select field1, field2, field3, field4 from table 
         where condition1=1 and condition2=2'

if the value of condition should be a string, you can do like this:

sql = ("select field1, field2, field3, field4 "
       "from table "
       "where condition1='{0}' "
       "and condition2='{1}'").format('2016-10-12', '2017-10-12')

Output: "select field1, field2, field3, field4 from table where
         condition1='2016-10-12' and condition2='2017-10-12'"

Questions:
Answers:

Cleanest way I have come across is inspired by the sql style guide.

sql = """
    SELECT field1, field2, field3, field4
      FROM table
     WHERE condition1 = 1
       AND condition2 = 2; """

Essentially, the keywords that begin a clause should be right-aligned and the field names etc, should be left aligned. This looks very neat and is easier to debug as well.

Questions:
Answers:

you could put the field names into an array “fields”, and then:


sql = 'select %s from table where condition1=1 and condition2=2' % (
 ', '.join(fields))

Questions:
Answers:

I would suggest sticking to option 2 (I’m always using it for queries any more complex than SELECT * FROM table) and if you want to print it in a nice way you may always use a separate module.

Questions:
Answers:
sql = """\
select field1, field2, field3, field4
from table
where condition1=1
and condition2=2
"""
[edit in responese to comment] Having an SQL string inside a method does NOT mean that you have to “tabulate” it:

>>> class Foo:
...     def fubar(self):
...         sql = """\
... select *
... from frobozz
... where zorkmids > 10
... ;"""
...         print sql
...
>>> Foo().fubar()
select *
from frobozz
where zorkmids > 10
;
>>>

Questions:
Answers:

For short queries that can fit on one or two lines, I use the string literal solution in the top-voted solution above. For longer queries, I break them out to .sql files. I then use a wrapper function to load the file and execute the script, something like:

script_cache = {}
def execute_script(cursor,script,*args,**kwargs):
    if not script in script_cache:
        with open(script,'r') as s:
            script_cache[script] = s
    return cursor.execute(script_cache[script],*args,**kwargs)

Of course this often lives inside a class so I don’t usually have to pass cursor explicitly. I also generally use codecs.open(), but this gets the general idea across. Then SQL scripts are completely self-contained in their own files with their own syntax highlighting.

Questions:
Answers:

In Addition to @user590028 :

Using format was helpful for what I was working on like so:

statement = (ins
            "(name,standard_price,list_price,mes_type,uom_id,uom_po_id,type,procure_method,cost_method_categ_id,supply_method,sale_ok) "
            "VALUE ('{0}','{1}','{2}'".format(row[1],str(row[2]),str(row[2])) + ",'fixed',1,1,'product','make_to_stock','standard',1,'buy',True) RETURNING id"
            )

And:

statement = ("INSERT INTO product_product "
             "(product_tmpl_id,default_code,active,valuation) "
             "VALUE "
             "('{0}','{1}',True,'manual_periodic')".format(str(row[0]), row[1])
             )

Questions:
Answers:

I would suggest a very easy option. just put an r before the string
You can use it like below:

query=(r'SELECT f1,f2,f3 
     r'FROM table1 '
     r'WHERE f4=cond1 '
     r'AND f5=cond2 ')
cursor.execute(str(query))
results=cursor.fetchall()
cursor.close()