Home » Mysql » UnicodeEncodeError: 'latin-1' codec can't encode character

UnicodeEncodeError: 'latin-1' codec can't encode character

Posted by: admin November 1, 2017 Leave a comment


What could be causing this error when I try to insert a foreign character into the database?

>>UnicodeEncodeError: 'latin-1' codec can't encode character u'\u201c' in position 0: ordinal not in range(256)

And how do I resolve it?



Character U+201C Left Double Quotation Mark is not present in the Latin-1 (ISO-8859-1) encoding.

It is present in code page 1252 (Western European). This is a Windows-specific encoding that is based on ISO-8859-1 but which puts extra characters into the range 0x80-0x9F. Code page 1252 is often confused with ISO-8859-1, and it’s an annoying but now-standard web browser behaviour that if you serve your pages as ISO-8859-1, the browser will treat them as cp1252 instead. However, they really are two distinct encodings:

>>> u'He said \u201CHello\u201D'.encode('iso-8859-1')
>>> u'He said \u201CHello\u201D'.encode('cp1252')
'He said \x93Hello\x94'

If you are using your database only as a byte store, you can use cp1252 to encode and other characters present in the Windows Western code page. But still other Unicode characters which are not present in cp1252 will cause errors.

You can use encode(..., 'ignore') to suppress the errors by getting rid of the characters, but really in this century you should be using UTF-8 in both your database and your pages. This encoding allows any character to be used. You should also ideally tell MySQL you are using UTF-8 strings (by setting the database connection and the collation on string columns), so it can get case-insensitive comparison and sorting right.


I ran into this same issue when using the Python MySQLdb module. Since MySQL will let you store just about any binary data you want in a text field regardless of character set, I found my solution here:

Using UTF8 with Python MySQLdb

Edit: Quote from the above URL to satisfy the request in the first comment…

“UnicodeEncodeError:’latin-1′ codec can’t encode character …”

This is because MySQLdb normally tries to encode everythin to latin-1.
This can be fixed by executing the following commands right after
you’ve etablished the connection:

dbc.execute('SET NAMES utf8;')
dbc.execute('SET CHARACTER SET utf8;')
dbc.execute('SET character_set_connection=utf8;')

“db” is the result of MySQLdb.connect(), and “dbc” is the result of


I hope your database is at least UTF-8. Then you will need to run yourstring.encode('utf-8') before you try putting it into the database.


The best solution is

  1. set mysql’s charset to ‘utf-8’
  2. do like this comment(add use_unicode=True and charset="utf8")

    db = MySQLdb.connect(host=”localhost”, user = “root”, passwd = “”, db = “testdb”, use_unicode=True, charset=”utf8″) – KyungHoon Kim Mar
    13 ’14 at 17:04

detail see :

class Connection(_mysql.connection):

    """MySQL Database Connection Object"""

    default_cursor = cursors.Cursor

    def __init__(self, *args, **kwargs):

        Create a connection to the database. It is strongly recommended
        that you only use keyword parameters. Consult the MySQL C API
        documentation for more information.

          string, host to connect

          string, user to connect as

          string, password to use

          string, database to use

          integer, TCP/IP port to connect to

          string, location of unix_socket to use

          conversion dictionary, see MySQLdb.converters

          number of seconds to wait before the connection attempt

          if set, compression is enabled

          if set, a named pipe is used to connect (Windows only)

          command which is run once the connection is created

          file from which default client values are read

          configuration group to use from the default file

          class object, used to create cursors (keyword only)

          If True, text-like columns are returned as unicode objects
          using the connection's character set.  Otherwise, text-like
          columns are returned as strings.  columns are returned as
          normal strings. Unicode objects will always be encoded to
          the connection's character set regardless of this setting.

          If supplied, the connection character set will be changed
          to this character set (MySQL-4.1 and newer). This implies

          If supplied, the session SQL mode will be changed to this
          setting (MySQL-4.1 and newer). For more details and legal
          values, see the MySQL documentation.

          integer, flags to use or 0
          (see MySQL docs or constants/CLIENTS.py)

          dictionary or mapping, contains SSL connection parameters;
          see the MySQL documentation for more details
          (mysql_ssl_set()).  If this is set, and the client does not
          support SSL, NotSupportedError will be raised.

          integer, non-zero enables LOAD LOCAL INFILE; zero disables

          If False (default), autocommit is disabled.
          If True, autocommit is enabled.
          If None, autocommit isn't set and server default is used.

        There are a number of undocumented, non-standard methods. See the
        documentation for the MySQL C API for some hints on what they do.



You are trying to store a Unicode codepoint \u201c using an encoding ISO-8859-1 / Latin-1 that can’t describe that codepoint. Either you might need to alter the database to use utf-8, and store the string data using an appropriate encoding, or you might want to sanitise your inputs prior to storing the content; i.e. using something like Sam Ruby’s excellent i18n guide. That talks about the issues that windows-1252 can cause, and suggests how to process it, plus links to sample code!


Latin-1 (aka ISO 8859-1) is a single octet character encoding scheme, and you can’t fit \u201c () into a byte.

Did you mean to use UTF-8 encoding?


SQLAlchemy users can simply specify their field as convert_unicode=True.

sqlalchemy.String(1000, convert_unicode=True)

SQLAlchemy will simply accept unicode objects and return them back, handling the encoding itself.



Python: You will need to add
# – * – coding: UTF-8 – * – (remove the spaces around * )
to the first line of the python file. and then add the following to the text to encode: .encode(‘ascii’, ‘xmlcharrefreplace’). This will replace all the unicode characters with it’s ASCII equivalent.