I want to I check whether a string is in ASCII or not.
I am aware of
ord(), however when I try
ord('é'), I have
TypeError: ord() expected a character, but string of length 2 found. I understood it is caused by the way I built Python (as explained in
Is there another way to check?
def is_ascii(s): return all(ord(c) < 128 for c in s)
I think you are not asking the right question–
A string in python has no property corresponding to ‘ascii’, utf-8, or any other encoding. The source of your string (whether you read it from a file, input from a keyboard, etc.) may have encoded a unicode string in ascii to produce your string, but that’s where you need to go for an answer.
Perhaps the question you can ask is: “Is this string the result of encoding a unicode string in ascii?” — This you can answer
try: mystring.decode('ascii') except UnicodeDecodeError: print "it was not a ascii-encoded unicode string" else: print "It may have been an ascii-encoded unicode string"
Python 3 way:
isascii = lambda s: len(s) == len(s.encode())
Ran into something like this recently – for future reference
import chardet encoding = chardet.detect(string) if encoding['encoding'] == 'ascii': print 'string is in ascii'
which you could use with:
string_ascii = string.decode(encoding['encoding']).encode('ascii')
Your question is incorrect; the error you see is not a result of how you built python, but of a confusion between byte strings and unicode strings.
Byte strings (e.g. “foo”, or ‘bar’, in python syntax) are sequences of octets; numbers from 0-255. Unicode strings (e.g. u”foo” or u’bar’) are sequences of unicode code points; numbers from 0-1112064. But you appear to be interested in the character é, which (in your terminal) is a multi-byte sequence that represents a single character.
ord(u'é'), try this:
>>> [ord(x) for x in u'é']
That tells you which sequence of code points “é” represents. It may give you , or it may give you [101, 770].
chr() to reverse this, there is
>>> unichr(233) u'\xe9'
This character may actually be represented either a single or multiple unicode “code points”, which themselves represent either graphemes or characters. It’s either “e with an acute accent (i.e., code point 233)”, or “e” (code point 101), followed by “an acute accent on the previous character” (code point 770). So this exact same character may be presented as the Python data structure
Most of the time you shouldn’t have to care about this, but it can become an issue if you are iterating over a unicode string, as iteration works by code point, not by decomposable character. In other words,
len(u'e\u0301') == 2 and
len(u'\u00e9') == 1. If this matters to you, you can convert between composed and decomposed forms by using
The Unicode Glossary can be a helpful guide to understanding some of these issues, by pointing how how each specific term refers to a different part of the representation of text, which is far more complicated than many programmers realize.
How about doing this?
import string def isAscii(s): for c in s: if c not in string.ascii_letters: return False return True
I found this question while trying determine how to use/encode/decode a string whose encoding I wasn’t sure of (and how to escape/convert special characters in that string).
My first step should have been to check the type of the string- I didn’t realize there I could get good data about its formatting from type(s). This answer was very helpful and got to the real root of my issues.
If you’re getting a rude and persistent
UnicodeDecodeError: ‘ascii’ codec can’t decode byte 0xc3 in position 263: ordinal not in range(128)
particularly when you’re ENCODING, make sure you’re not trying to unicode() a string that already IS unicode- for some terrible reason, you get ascii codec errors. (See also the Python Kitchen recipe, and the Python docs tutorials for better understanding of how terrible this can be.)
Eventually I determined that what I wanted to do was this:
escaped_string = unicode(original_string.encode('ascii','xmlcharrefreplace'))
Also helpful in debugging was setting the default coding in my file to utf-8 (put this at the beginning of your python file):
# -*- coding: utf-8 -*-
That allows you to test special characters (‘àéç’) without having to use their unicode escapes (u’\xe0\xe9\xe7′).
>>> specials='àéç' >>> specials.decode('latin-1').encode('ascii','xmlcharrefreplace') 'àéç'
Vincent Marchetti has the right idea, but
str.decode has been deprecated in Python 3. In Python 3 you can make the same test with
try: mystring.encode('ascii') except UnicodeEncodeError: pass # string is not ascii else: pass # string is ascii
Note the exception you want to catch has also changed from
To improve Alexander’s solution from the Python 2.6 (and in Python 3.x) you can use helper module curses.ascii and use curses.ascii.isascii() function or various other: https://docs.python.org/2.6/library/curses.ascii.html
from curses import ascii def isascii(s): return all(ascii.isascii(c) for c in s)
You could use the regular expression library which accepts the Posix standard [[:ASCII:]] definition.
A sting (
str-type) in Python is a series of bytes. There is no way of telling just from looking at the string whether this series of bytes represent an ascii string, a string in a 8-bit charset like ISO-8859-1 or a string encoded with UTF-8 or UTF-16 or whatever.
However if you know the encoding used, then you can
decode the str into a unicode string and then use a regular expression (or a loop) to check if it contains characters outside of the range you are concerned about.
To prevent your code from crashes, you maybe want to use a
try-except to catch
>>> ord("¶") Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: ord() expected a character, but string of length 2 found
def is_ascii(s): try: return all(ord(c) < 128 for c in s) except TypeError: return False
import re def is_ascii(s): return bool(re.match(r'[\x00-\x7F]+$', s))
To include an empty string as ASCII, change the
Like @RogerDahl’s answer but it’s more efficient to short-circuit by negating the character class and using search instead of
>>> import re >>> re.search('[^\x00-\x7F]', 'Did you catch that \x00?') is not None False >>> re.search('[^\x00-\x7F]', 'Did you catch that \xFF?') is not None True
I imagine a regular expression is well-optimized for this.
I use the following to determine if the string is ascii or unicode:
>> print 'test string'.__class__.__name__ str >>> print u'test string'.__class__.__name__ unicode >>>
Then just use a conditional block to define the function:
def is_ascii(input): if input.__class__.__name__ == "str": return True return False