Home » excel » vba – Fastest way to update large data in Excel Table

vba – Fastest way to update large data in Excel Table

Posted by: admin May 14, 2020 Leave a comment

Questions:

I have a large table in Excel that has to be updated from a JSON source.
The data is fetched and available to me in the form of a dictionary after parsing the JSON.
I am iterating over all the fields in the data and updating relevant columns in a table.

Public Function GetFields(ByVal sApiEndpoint As String, ByVal sSheetName As String, ByVal sTableName As String)

    .........

    'Parse the Json Response and Update Table
    Dim dicParsed As Dictionary

    With ActiveWorkbook.Sheets(sSheetName).ListObjects(sTableName)

        Dim iCount As Integer

        iCount = 1

        Set dicParsed = JsonConverter.ParseJson(sRestResponse)

        For Each Item In dicParsed("data")

            iCount = iCount + 1

        Next Item

        If .ListRows.Count >= 1 Then
            .DataBodyRange.Delete
        End If

        Set Rng = .Range.Resize(iCount, .HeaderRowRange.Columns.Count)
        .Resize Rng

        Dim iRow As Integer
        iRow = 0

        For Each Item In dicParsed("data")

            On Error Resume Next


            .DataBodyRange.Cells(iRow, .ListColumns("name").Index) = Item("name")
            .DataBodyRange.Cells(iRow, .ListColumns("id").Index) = Item("id")
            .DataBodyRange.Cells(iRow, .ListColumns("type").Index) = Item("schema")("type")

            iRow = iRow + 1

        Next Item

    End With

    .........

End Function

It takes around 5 minutes to update a table of 500 rows with 15 columns with the calculations and updates turned off.

Is there any faster way to update the data in this scenario?

How to&Answers:

You can push the updates to an array and then bulk update the table.

  1. Dim an array of any type without giving a size. Use Variant if you have columns of different type like numbers and strings.
  2. ReDim the array when you know final dimensions.
  3. Update the array with the data.
  4. Set the tables databody equal to the array.

I was able to reduce update time from 5 minutes to less than 5 seconds with the below code.

Public Function GetFields(ByVal sApiEndpoint As String, ByVal sSheetName As String, ByVal sTableName As String)

    .........

    'Parse the Json Response and Update Table
    Dim dicParsed As Dictionary

    With ActiveWorkbook.Sheets(sSheetName).ListObjects(sTableName)

        Dim iCount As Integer
        Dim arrDataBuffer() As Variant

        iCount = 1

        Set dicParsed = JsonConverter.ParseJson(sRestResponse)

        For Each Item In dicParsed("data")

            iCount = iCount + 1

        Next Item

        If .ListRows.Count >= 1 Then
            .DataBodyRange.Delete
        End If

        Set Rng = .Range.Resize(iCount, .HeaderRowRange.Columns.Count)
        .Resize Rng

        ReDim arrDataBuffer(iCount, .HeaderRowRange.Columns.Count)

        Dim iRow As Integer
        iRow = 0

        For Each Item In dicParsed("data")

            On Error Resume Next

            arrDataBuffer(iRow, .ListColumns("name").Index - 1) = Item("name")
            arrDataBuffer(iRow, .ListColumns("id").Index - 1) = Item("id")
            arrDataBuffer(iRow, .ListColumns("type").Index - 1) = Item("schema")("type")

            iRow = iRow + 1

        Next Item

        .DataBodyRange = arrDataBuffer

    End With

    .........

End Function