Home » C++ » Sequence-zip function for c++11?

Sequence-zip function for c++11?

Posted by: admin November 29, 2017 Leave a comment

Questions:

With the new range-based for loop we can write code like

for(auto x: Y) {}

Which IMO is a huge improvement from (for ex.)

for(std::vector<int>::iterator x=Y.begin(); x!=Y.end(); ++x) {}

Can it be used to loop over two simultaneous loops, like Pythons zip function? For those unfamiliar with Python, the code:

Y1 = [1,2,3]
Y2 = [4,5,6,7]
for x1,x2 in zip(Y1,Y2):
    print x1,x2

Gives as output (1,4) (2,5) (3,6)

Answers:

Warning: boost::zip_iterator and boost::combine as of Boost 1.63.0 (2016 Dec 26) will cause undefined behavior if the length of the input containers are not the same (it may crash or iterate beyond the end).


Starting from Boost 1.56.0 (2014 Aug 7) you could use boost::combine (the function exists in earlier versions but undocumented):

#include <boost/range/combine.hpp>
#include <vector>
#include <list>
#include <string>

int main() {
    std::vector<int> a {4, 5, 6};
    double b[] = {7, 8, 9};
    std::list<std::string> c {"a", "b", "c"};
    for (auto tup : boost::combine(a, b, c, a)) {    // <---
        int x, w;
        double y;
        std::string z;
        boost::tie(x, y, z, w) = tup;
        printf("%d %g %s %d\n", x, y, z.c_str(), w);
    }
}

This would print

4 7 a 4
5 8 b 5
6 9 c 6

In earlier versions, you could define a range yourself like this:

#include <boost/iterator/zip_iterator.hpp>
#include <boost/range.hpp>

template <typename... T>
auto zip(T&&... containers) -> boost::iterator_range<boost::zip_iterator<decltype(boost::make_tuple(std::begin(containers)...))>>
{
    auto zip_begin = boost::make_zip_iterator(boost::make_tuple(std::begin(containers)...));
    auto zip_end = boost::make_zip_iterator(boost::make_tuple(std::end(containers)...));
    return boost::make_iterator_range(zip_begin, zip_end);
}

The usage is the same.

Questions:
Answers:

You can use a solution based on boost::zip_iterator. Make a phony container class maintaining references to your containers, and which return zip_iterator from the begin and end member functions. Now you can write

for (auto p: zip(c1, c2)) { ... }

Example implementation (please test):

#include <iterator>
#include <boost/iterator/zip_iterator.hpp>

template <typename C1, typename C2>
class zip_container
{
    C1* c1; C2* c2;

    typedef boost::tuple<
        decltype(std::begin(*c1)), 
        decltype(std::begin(*c2))
    > tuple;

public:
    zip_container(C1& c1, C2& c2) : c1(&c1), c2(&c2) {}

    typedef boost::zip_iterator<tuple> iterator;

    iterator begin() const
    {
         return iterator(std::begin(*c1), std::begin(*c2));
    }

    iterator end() const
    {
         return iterator(std::end(*c1), std::end(*c2));
    }
};

template <typename C1, typename C2>
zip_container<C1, C2> zip(C1& c1, C2& c2)
{
    return zip_container<C1, C2>(c1, c2);
}

I leave the variadic version as an excellent exercise to the reader.

Questions:
Answers:

See <redi/zip.h> for a zip function which works with range-base for and accepts any number of ranges, which can be rvalues or lvalues and can be different lengths (iteration will stop at the end of the shortest range).

std::vector<int> vi{ 0, 2, 4 };
std::vector<std::string> vs{ "1", "3", "5", "7" };
for (auto i : redi::zip(vi, vs))
  std::cout << i.get<0>() << ' ' << i.get<1>() << ' ';

Prints 0 1 2 3 4 5

Questions:
Answers:

So I wrote this zip before when I was bored, I decided to post it because it’s different than the others in that it doesn’t use boost and looks more like the c++ stdlib.

template <typename Iterator>
    void advance_all (Iterator & iterator) {
        ++iterator;
    }
template <typename Iterator, typename ... Iterators>
    void advance_all (Iterator & iterator, Iterators& ... iterators) {
        ++iterator;
        advance_all(iterators...);
    } 
template <typename Function, typename Iterator, typename ... Iterators>
    Function zip (Function func, Iterator begin, 
            Iterator end, 
            Iterators ... iterators)
    {
        for(;begin != end; ++begin, advance_all(iterators...))
            func(*begin, *(iterators)... );
        //could also make this a tuple
        return func;
    }

Example use:

int main () {
    std::vector<int> v1{1,2,3};
    std::vector<int> v2{3,2,1};
    std::vector<float> v3{1.2,2.4,9.0};
    std::vector<float> v4{1.2,2.4,9.0};
     zip (
            [](int i,int j,float k,float l){
                std::cout << i << " " << j << " " << k << " " << l << std::endl;
            },
            v1.begin(),v1.end(),v2.begin(),v3.begin(),v4.begin());
}

Questions:
Answers:

I ran into this same question independently and didn’t like the syntax of any of the above. So, I have a short header file that essentially does the same as the boost zip_iterator but has a few macros to make the syntax more palatable to me:

https://github.com/cshelton/zipfor

For example you can do

vector<int> a {1,2,3};
array<string,3> b {"hello","there","coders"};

zipfor(i,s eachin a,b)
    cout << i << " => " << s << endl;

The main syntactic sugar is that I can name the elements from each container. I also include a “mapfor” that does the same, but for maps (to name the “.first” and “.second” of the element).

Questions:
Answers:
// declare a, b
BOOST_FOREACH(boost::tie(a, b), boost::combine(list_of_a, list_of_b)){
    // your code here.
}

Questions:
Answers:

If you have a C++14 compliant compiler (e.g. gcc5) you can use zip provided in the cppitertools library by Ryan Haining, which looks really promising:

array<int,4> i{{1,2,3,4}};
vector<float> f{1.2,1.4,12.3,4.5,9.9};
vector<string> s{"i","like","apples","alot","dude"};
array<double,5> d{{1.2,1.2,1.2,1.2,1.2}};

for (auto&& e : zip(i,f,s,d)) {
    cout << std::get<0>(e) << ' '
         << std::get<1>(e) << ' '
         << std::get<2>(e) << ' '
         << std::get<3>(e) << '\n';
    std::get<1>(e)=2.2f; // modifies the underlying 'f' array
}

Questions:
Answers:

If you like operator overloading, here are three possibilities. The first two are using std::pair<> and std::tuple<>, respectively, as iterators; the third extends this to range-based for. Note that not everyone will like these definitions of the operators, so it’s best to keep them in a separate namespace and have a using namespace in the functions (not files!) where you’d like to use these.

#include <iostream>
#include <utility>
#include <vector>
#include <tuple>

// put these in namespaces so we don't pollute global
namespace pair_iterators
{
    template<typename T1, typename T2>
    std::pair<T1, T2> operator++(std::pair<T1, T2>& it)
    {
        ++it.first;
        ++it.second;
        return it;
    }
}

namespace tuple_iterators
{
    // you might want to make this generic (via param pack)
    template<typename T1, typename T2, typename T3>
    auto operator++(std::tuple<T1, T2, T3>& it)
    {
        ++( std::get<0>( it ) );
        ++( std::get<1>( it ) );
        ++( std::get<2>( it ) );
        return it;
    }

    template<typename T1, typename T2, typename T3>
    auto operator*(const std::tuple<T1, T2, T3>& it)
    {
        return std::tie( *( std::get<0>( it ) ),
                         *( std::get<1>( it ) ),
                         *( std::get<2>( it ) ) );
    }

    // needed due to ADL-only lookup
    template<typename... Args>
    struct tuple_c
    {
        std::tuple<Args...> containers;
    };

    template<typename... Args>
    auto tie_c( const Args&... args )
    {
        tuple_c<Args...> ret = { std::tie(args...) };
        return ret;
    }

    template<typename T1, typename T2, typename T3>
    auto begin( const tuple_c<T1, T2, T3>& c )
    {
        return std::make_tuple( std::get<0>( c.containers ).begin(),
                                std::get<1>( c.containers ).begin(),
                                std::get<2>( c.containers ).begin() );
    }

    template<typename T1, typename T2, typename T3>
    auto end( const tuple_c<T1, T2, T3>& c )
    {
        return std::make_tuple( std::get<0>( c.containers ).end(),
                                std::get<1>( c.containers ).end(),
                                std::get<2>( c.containers ).end() );
    }

    // implement cbegin(), cend() as needed
}

int main()
{
    using namespace pair_iterators;
    using namespace tuple_iterators;

    std::vector<double> ds = { 0.0, 0.1, 0.2 };
    std::vector<int   > is = {   1,   2,   3 };
    std::vector<char  > cs = { 'a', 'b', 'c' };

    // classical, iterator-style using pairs
    for( auto its  = std::make_pair(ds.begin(), is.begin()),
              end  = std::make_pair(ds.end(),   is.end()  ); its != end; ++its )
    {
        std::cout << "1. " << *(its.first ) + *(its.second) << " " << std::endl;
    }

    // classical, iterator-style using tuples
    for( auto its  = std::make_tuple(ds.begin(), is.begin(), cs.begin()),
              end  = std::make_tuple(ds.end(),   is.end(),   cs.end()  ); its != end; ++its )
    {
        std::cout << "2. " << *(std::get<0>(its)) + *(std::get<1>(its)) << " "
                           << *(std::get<2>(its)) << " " << std::endl;
    }

    // range for using tuples
    for( const auto& d_i_c : tie_c( ds, is, cs ) )
    {
        std::cout << "3. " << std::get<0>(d_i_c) + std::get<1>(d_i_c) << " "
                           << std::get<2>(d_i_c) << " " << std::endl;
    }
}

Questions:
Answers:

With range-v3:

#include <range/v3/all.hpp>
#include <vector>
#include <iostream>

namespace ranges {
    template <class T, class U>
    std::ostream& operator << (std::ostream& os, common_pair<T, U> const& p)
    {
      return os << '(' << p.first << ", " << p.second << ')';
    }
}

using namespace ranges::v3;

int main()
{
    std::vector<int> a {4, 5, 6};
    double b[] = {7, 8, 9};
    std::cout << view::zip(a, b) << std::endl; 
}

The output:

[(4, 7),(5, 8),(6, 9)]
Questions:
Answers:

Boost.Iterators has zip_iterator you can use (example’s in the docs). It won’t work with range for, but you can use std::for_each and a lambda.

Questions:
Answers:

Here is a simple version that does not require boost. It won’t be particularly efficient as it creates temporary values, and it does not generalise over containers other than lists, but it has no dependencies and it solves the most common case for zipping.

template<class L, class R>
std::list< std::pair<L,R> >  zip(std::list<L> left, std::list<R> right)
{
auto l = left.begin();
auto r = right.begin();
std::list< std::pair<L,R> > result;
  while( l!=left.end() && r!=right.end() )
    result.push_back( std::pair<L,R>( *(l++), *(r++) ) );
  return result;
}

Although the other versions are more flexible, often the point of using a list operator is make a simple one-liner. This version has the benefit that the common-case is simple.

Questions:
Answers:

For a C++ stream processing library I’m writing I was looking for a solution that doesn’t rely on third party libraries and works with an arbitrary number of containers. I ended up with this solution. It’s similar to the accepted solution which uses boost (and also results in undefined behavior if the container lengths are not equal)

#include <utility>

namespace impl {

template <typename Iter, typename... Iters>
class zip_iterator {
public:
  using value_type = std::tuple<const typename Iter::value_type&,
                                const typename Iters::value_type&...>;

  zip_iterator(const Iter &head, const Iters&... tail)
      : head_(head), tail_(tail...) { }

  value_type operator*() const {
    return std::tuple_cat(std::tuple<const typename Iter::value_type&>(*head_), *tail_);
  }

  zip_iterator& operator++() {
    ++head_; ++tail_;
    return *this;
  }

  bool operator==(const zip_iterator &rhs) const {
    return head_ == rhs.head_ && tail_ == rhs.tail_;
  }

  bool operator!=(const zip_iterator &rhs) const {
    return !(*this == rhs);
  }

private:
  Iter head_;
  zip_iterator<Iters...> tail_;
};

template <typename Iter>
class zip_iterator<Iter> {
public:
  using value_type = std::tuple<const typename Iter::value_type&>;

  zip_iterator(const Iter &head) : head_(head) { }

  value_type operator*() const {
    return value_type(*head_);
  }

  zip_iterator& operator++() { ++head_; return *this; }

  bool operator==(const zip_iterator &rhs) const { return head_ == rhs.head_; }

  bool operator!=(const zip_iterator &rhs) const { return !(*this == rhs); }

private:
  Iter head_;
};

}  // namespace impl

template <typename Iter>
class seq {
public:
  using iterator = Iter;
  seq(const Iter &begin, const Iter &end) : begin_(begin), end_(end) { }
  iterator begin() const { return begin_; }
  iterator end() const { return end_; }
private:
  Iter begin_, end_;
};

/* WARNING: Undefined behavior if iterator lengths are different.
 */
template <typename... Seqs>
seq<impl::zip_iterator<typename Seqs::iterator...>>
zip(const Seqs&... seqs) {
  return seq<impl::zip_iterator<typename Seqs::iterator...>>(
      impl::zip_iterator<typename Seqs::iterator...>(std::begin(seqs)...),
      impl::zip_iterator<typename Seqs::iterator...>(std::end(seqs)...));
}