Currently my num reduce task is set to
So my final output directory is in S3 and looks like the following
/output/part-r-00000.gz /output/part-r-00001.gz ... etc
in order to count all the lines I have to manually download and unzip all files and go through each file to count the total lines.
Is there a total line metric store somewhere in hadoop context?