Home » Php » php – Fastest way to read a csv file

php – Fastest way to read a csv file

Posted by: admin July 12, 2020 Leave a comment

Questions:

I’m looking for a very fast method to read a csv file. My data structure looks like this:

timestamp ,float     , string    ,ip          ,string
1318190061,1640851625, lore ipsum,84.169.42.48,appname

and I’m using fgetcsv to read this data into arrays.

The problem: Performance. On a regular basis the script has to read (and process) more than 10,000 entries.

My first attempt is very simple:

//Performance: 0,141 seconds / 13.5 MB

while(!feof($statisticsfile)) 
    {
    $temp = fgetcsv($statisticsfile);
    $timestamp[] = $temp[0];
    $value[] = $temp[1];
    $text[] = $temp[2];
    $ip[] = $temp[3];
    $app[] = $temp[4];
    }

My second attempt:

//Performance: 0,125 seconds / 10.8 MB

while (($userinfo = fgetcsv($statisticsfile)) !== FALSE) {
   list ($timestamp[], $value[], $text, $ip, $app) = $userinfo;
}
  1. Is there any way to improve performance even further, or is my method as fast as it could get?
  2. Probably more important: Is there any way to define what columns are read, e.g. sometimes only the timestamp, float columns are needed. Is there any better way than my way (have a look at my second attempt 🙂

Thanks 🙂

How to&Answers:

How long is the longest line? Pass that as the second parameter to fgetcsv() and you’ll see the greatest improvement.

Answer:

Check time that PHP read this file:

If is bigg move file to ramdisk or SSD

  1. [..]sometimes only the timestamp

Somthing like that

preg_match_all('#\d{10},\d{10}, (.*?),\d.\d.\d.\d,appname#',$f,$res);

print_r($res);