Home » Python » What is the maximum recursion depth in Python, and how to increase it?

What is the maximum recursion depth in Python, and how to increase it?

Posted by: admin November 1, 2017 Leave a comment

Questions:

I have this tail recursive function here:

def fib(n, sum):
    if n < 1:
        return sum
    else:
        return fib(n-1, sum+n)

c = 998
print(fib(c, 0))

It works up to n=997, then it just breaks and spits a “maximum recursion depth exceeded in comparison” RuntimeError. Is this just a stack overflow? Is there a way to get around it?

Answers:

It is a guard against a stack overflow, yes. Python (or rather, the CPython implementation) doesn’t optimize tail recursion, and unbridled recursion causes stack overflows. You can change the recursion limit with sys.setrecursionlimit, but doing so is dangerous — the standard limit is a little conservative, but Python stackframes can be quite big.

Python isn’t a functional language and tail recursion is not a particularly efficient technique. Rewriting the algorithm iteratively, if possible, is generally a better idea.

Questions:
Answers:

Looks like you just need to set a higher recursion depth

sys.setrecursionlimit(1500)

Questions:
Answers:

It’s to avoid a stack overflow. The Python interpreter limits the depths of recursion to help you avoid infinite recursions, resulting in stack overflows.
Try increasing the recursion limit (sys.setrecursionlimit) or re-writing your code without recursion.

from python website:

sys.getrecursionlimit()

Return the current value of the recursion limit, the maximum depth of the Python interpreter stack. This limit prevents infinite recursion from causing an overflow of the C stack and crashing Python. It can be set by setrecursionlimit().

Questions:
Answers:

Use a language that guarantees tail-call optimisation. Or use iteration. Alternatively, get cute with decorators.

Questions:
Answers:

I realize this is an old question but for those reading, I would recommend against using recursion for problems such as this – lists are much faster and avoid recursion entirely. I would implement this as:

def fibonacci(n):
    f = [0,1,1]
    for i in xrange(3,n):
        f.append(f[i-1] + f[i-2])
    return 'The %.0fth fibonacci number is: %.0f' % (n,f[-1])

(Use n+1 in xrange if you start counting your fibonacci sequence from 0 instead of 1.)

Questions:
Answers:

I had a similar issue with the error “Max recursion depth exceeded”. I discovered the error was being triggered by a corrupt file in the directory I was looping over with os.walk. If you have trouble solving this issue and you are working with file paths, be sure to narrow it down, as it might be a corrupt file.

Questions:
Answers:

Of course Fibonacci numbers can be computed in O(1) by applying the Binet formula:

from math import floor, sqrt

def fib(n):                                                     
    return int(floor(((1+sqrt(5))**n-(1-sqrt(5))**n)/(2**n*sqrt(5))+0.5))

You will need big integers from numpy when the results no longer fit in a long.

Questions:
Answers:

Use generators?

def fib():
    a, b = 0, 1
    while True:
        yield a
        a, b = b, a + b

fibs = fib() #seems to be the only way to get the following line to work is to
             #assign the infinite generator to a variable

f = [fibs.next() for x in xrange(1001)]

for num in f:
        print num

above fib() function adapted from: http://intermediatepythonista.com/python-generators

Questions:
Answers:

Many recommend that increasing recursion limit is a good solution however it is not because there will be always limit. Instead use an iterative solution.

def fib(n):
    a,b = 1,1
    for i in range(n-1):
        a,b = b,a+b
    return a
print fib(5)

Questions:
Answers:

resource.setrlimit must also be used to increase the stack size and prevent segfault

The Linux kernel limits the stack of processes.

Recursion takes up stack space.

If Python tries to go over the stack limit, the Linux kernel segfaults it.

The stack limit size is controlled with the getrlimit and setrlimit system calls.

Python offers access to those system calls through the resource module.

import resource
import sys

print resource.getrlimit(resource.RLIMIT_STACK)
print sys.getrecursionlimit()
print

# Will segfault without this line.
resource.setrlimit(resource.RLIMIT_STACK, [0x10000000, resource.RLIM_INFINITY])
sys.setrecursionlimit(0x100000)

def f(i):
    print i
    sys.stdout.flush()
    f(i + 1)
f(0)

Of course, if you keep increasing ulimit, your RAM will run out, which will either slow your computer to a halt due to swap madness, or kill Python via the OOM Killer.

From bash, you can see and set the stack limit (in kb) with:

ulimit -s
ulimit -s 10000

Default value for me is 8Mb.

See also:

Tested on Ubuntu 16.10, Python 2.7.12.

Questions:
Answers:

As @alex suggested, you could use a generator function to do this. Here’s the equivalent of the code in your question:

def fib(n):
    def fibseq(n):
        """ Iteratively return the first n Fibonacci numbers, starting from 0 """
        a, b = 0, 1
        for _ in xrange(n):
            yield a
            a, b = b, a + b

    return sum(v for v in fibseq(n))

print format(fib(100000), ',d')  # -> no recursion depth error

Leave a Reply

Your email address will not be published. Required fields are marked *