Home » excel » excel – Perform an operation on multiple tables in a list

excel – Perform an operation on multiple tables in a list

Posted by: admin May 14, 2020 Leave a comment

Questions:

Trying to teach myself Power Query and can’t get my head around this particular process.

I understand there may be other, perhaps more efficient methods of attaining the desired result, but my purpose here is to understand the process of performing multiple operations on each table in a list of tables, where the number of entries in the list is unknown and/or large.

  • My original data has pairs of information [Credit, Name] in adjacent columns with an unknown number of pairs.

    • Since it is a table, the different column/pairs have different Names.

      Credit|Name|Credit1|Name1|...

  • If I demote the headers and transpose the table, the column headers will wind up in Column 1, and I can strip off the differentiating digit.

  • Using Table.Split, I can then create a number of tables where each pair of columns has the Identical headers.
  • I can then combine these tables to create a single, two column table, where I can Group and aggregate to get the results.

My problem is that I have not been able to figure out how to do the:

Table.PromoteHeaders(Table.Transpose(table))

operation on each table.

This M-code produces the desired result for the four pairs of columns in the provided data, but is clearly not scalable since the number of tables needs to be known in advance.

let 
   //Create the table
   Tbl1= Table.FromRecords({
      [Credit = 1, Name = "Bob", Credit2 = 2, Name2 = "Jim", Credit3 = 1, Name3 = "George", Credit4 = 1.75, Name4="Phil"],
      [Credit = 2, Name = "Phil", Credit2 = 4, Name2="George", Credit3 = 2.5, Name3 = "Stephen",Credit4 = 6, Name4="Bob"]
      }),

   //Demote headers and transpose
   transpose1 = Table.Transpose( Table.DemoteHeaders(Tbl1)),

   //Create matching names for what will eventually be the final Column Headers
    #"Split Column by Character Transition" = Table.SplitColumn(transpose1, "Column1", Splitter.SplitTextByCharacterTransition((c) => not List.Contains({"0".."9"}, c), {"0".."9"}), {"Column1.1", "Column1.2"}),
    #"Removed Columns" = Table.RemoveColumns(#"Split Column by Character Transition",{"Column1.2"}),

   //Create multiple tables from above
    multTables = Table.Split(#"Removed Columns",2),

   /*transpose and promote the headers for each table
   HOW can I do this in a single step when I don't know how many tables there might be???
   */

   tbl0 = Table.PromoteHeaders(Table.Transpose(multTables{0}),[PromoteAllScalars=true]),
   tbl1 = Table.PromoteHeaders(Table.Transpose(multTables{1}),[PromoteAllScalars=true]),
   tbl2  = Table.PromoteHeaders(Table.Transpose(multTables{2}),[PromoteAllScalars=true]),
   tbl3 = Table.PromoteHeaders(Table.Transpose(multTables{3}),[PromoteAllScalars=true]),

   combTable = Table.Combine({tbl0,tbl1,tbl2,tbl3})


in
    combTable

Original Table

enter image description here

Demoted headers / Transposed table

enter image description here

Desired Result

enter image description here

Any help would be appreciated.

How to&Answers:

You could also try replacing this part of your code:

   tbl0 = Table.PromoteHeaders(Table.Transpose(multTables{0}),[PromoteAllScalars=true]),
   tbl1 = Table.PromoteHeaders(Table.Transpose(multTables{1}),[PromoteAllScalars=true]),
   tbl2  = Table.PromoteHeaders(Table.Transpose(multTables{2}),[PromoteAllScalars=true]),
   tbl3 = Table.PromoteHeaders(Table.Transpose(multTables{3}),[PromoteAllScalars=true]),

   combTable = Table.Combine({tbl0,tbl1,tbl2,tbl3})


in
   combTable

with this:

    Custom1 = List.Transform(multTables, each Table.PromoteHeaders( Table.Transpose(_),[PromoteAllScalars=true])),
    #"Converted to Table" = Table.FromList(Custom1, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
    #"Expanded Column1" = Table.ExpandTableColumn(#"Converted to Table", "Column1", {"Credit", "Name"}, {"Credit", "Name"})
in
    #"Expanded Column1"

Answer:

A bit clunky but seems to work with any number of rows and any number of 2 column paired columns.

First, a bunch of indexes modified in different ways, then a filter into two table, unpivot and merge

let  Tbl1= Table.FromRecords({
  [Credit = 1, Name = "Bob", Credit2 = 2, Name2 = "Jim", Credit3 = 1, Name3 = "George", Credit4 = 1.75, Name4="Phil"],
  [Credit = 2, Name = "Phil", Credit2 = 4, Name2="George", Credit3 = 2.5, Name3 = "Stephen",Credit4 = 6, Name4="Bob"],
  [Credit = 3, Name = "Sam", Credit2 = 5, Name2="Allen", Credit3 = 3.5, Name3 = "Ralph",Credit4 = 7, Name4="Nance"]
  }),

#"Transposed Table" = Table.Transpose(Tbl1),
#"Added Index" = Table.AddIndexColumn(#"Transposed Table", "Index", 0, .5),
#"Added Custom" = Table.AddColumn(#"Added Index", "Index2", each Number.RoundDown([Index])),
#"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"Index"}),
#"Added Index1" = Table.AddIndexColumn(#"Removed Columns", "Index", 0, 1),
#"Added Custom1" = Table.AddColumn(#"Added Index1", "Custom", each Number.Mod([Index],2)),
#"Removed Columns1" = Table.RemoveColumns(#"Added Custom1",{"Index"}),
#"Unpivoted Other Columns" = Table.UnpivotOtherColumns(#"Removed Columns1", {"Index2", "Custom"}, "Attribute", "Value"),
#"Filtered Rows" = Table.SelectRows(#"Unpivoted Other Columns", each ([Custom] = 0)),
#"Filtered Rows2" = Table.SelectRows(#"Unpivoted Other Columns", each ([Custom] = 1)),
#"Merged Queries" = Table.NestedJoin(#"Filtered Rows2",{"Index2", "Attribute"},#"Filtered Rows",{"Index2", "Attribute"},"Filtered Rows",JoinKind.LeftOuter),
#"Expanded Filtered Rows" = Table.ExpandTableColumn(#"Merged Queries", "Filtered Rows", {"Value"}, {"Value.1"}),
#"Removed Columns2" = Table.RemoveColumns(#"Expanded Filtered Rows",{"Index2", "Custom", "Attribute"})
in #"Removed Columns2"

Another way of doing it is to create two tables by selecting and unpivoting groups of columns based on their column names, then combining them using a custom column referring to the row index in each table

  let  Tbl1= Table.FromRecords({
  [Credit = 1, Name = "Bob", Credit2 = 2, Name2 = "Jim", Credit3 = 1, Name3 = "George", Credit4 = 1.75, Name4="Phil"],
  [Credit = 2, Name = "Phil", Credit2 = 4, Name2="George", Credit3 = 2.5, Name3 = "Stephen",Credit4 = 6, Name4="Bob"],
  [Credit = 3, Name = "Sam", Credit2 = 5, Name2="Allen", Credit3 = 3.5, Name3 = "Ralph",Credit4 = 7, Name4="Nance"]
  }),
  Credit = List.Select(Table.ColumnNames(Tbl1), each Text.Contains(_, "Credit")),
  Name = List.Select(Table.ColumnNames(Tbl1), each Text.Contains(_, "Name")),

  // create table of just Names with index
  #"Removed Columns1" = Table.RemoveColumns(Tbl1,Credit),
  #"Added Index" = Table.AddIndexColumn(#"Removed Columns1", "Index", 0, 1),
  #"Unpivoted Other Columns" = Table.UnpivotOtherColumns(#"Added Index", {"Index"}, "Attribute", "Value"),

  // create table of just Credits with index
  #"Removed Columns2" = Table.RemoveColumns(Tbl1,Name),
  #"Added Index2" = Table.AddIndexColumn(#"Removed Columns2", "Index", 0, 1),
  #"Unpivoted Other Columns2" = Table.UnpivotOtherColumns(#"Added Index2", {"Index"}, "Attribute", "Value"),
  #"Added Index1" = Table.AddIndexColumn(#"Unpivoted Other Columns2", "Index.1", 0, 1),

  //merge two table together and remove excess columns
  #"Added Custom" = Table.AddColumn(#"Added Index1", "Custom", each #"Unpivoted Other Columns"{[Index.1]}[Value]),
  #"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"Index", "Attribute", "Index.1"})
  in #"Removed Columns"