Performance: VertiPaq, query plans, aggregations, incremental refresh
FactSales WebEvents- Why it matters
- Slow reports stop being used, and a model that is too big costs real capacity money every hour.
- Typical production failure
- An executive page takes 18 seconds every Monday because one measure iterates the whole fact table per cell.
- When to use it
- Measure first: Performance Analyzer, DAX query view and Server Timings tell you whether the time is in visuals, the formula engine or the storage engine.
- When not to
- Optimising without a baseline. Every change needs a before and after number.
Assignments
Make a big fact and measure it
- In Power Query, reference FactSales and cross-join it with a 1,000-row list (List.Numbers) to get 100,000 rows; add a random offset to dates. Name it FactSalesBig.
- Open DAX Studio → Advanced → View Metrics (VertiPaq Analyzer). Record total model size and the top 3 columns by size.
- Change SalesKey to not loaded (remove it), and change UnitPrice from Decimal to Fixed Decimal. Re-run metrics. Record the size difference.
- Split OrderDate/ShipDate into Date only (remove time). Record again.
Read a query plan
- Write the slow version: Customers Over 500 Slow = COUNTROWS(FILTER(FactSalesBig, [Net Sales] > 500)).
- In DAX Studio, Server Timings on, run a matrix query using it. Record FE ms, SE ms, number of SE queries, and whether CallbackDataID appears.
- Rewrite: COUNTROWS(FILTER(VALUES(DimCustomer[CustomerKey]), [Net Sales] > 500)). Re-run and compare.
- Rewrite once more with SUMMARIZE + a filter on a pre-computed column; compare.
Aggregation table
- Create AggSalesMonthCat in Power Query: group FactSalesBig by YearMonth, Category, RegionKey with Sum Quantity, Sum Amount, Count rows.
- Set FactSalesBig to DirectQuery (or keep Import and just practise the mapping), AggSalesMonthCat to Import. Manage aggregations: map each column.
- In DAX Studio, run a query by Category and check the "Aggregation hit" in the query plan event; then run by ProductName and confirm it misses.
Incremental refresh end to end
- RangeStart/RangeEnd parameters; filter step that folds.
- Incremental refresh policy: archive 2 years, refresh 3 days, detect data changes on a ModifiedDate column if you have one.
- Publish, refresh, then in SSMS or Tabular Editor via XMLA endpoint (needs capacity/PPU) look at the partitions created.
- Simulate a late-arriving fact and confirm it is not picked up outside the refresh window. Then fix with a longer window.
Storage engine friendly DAX
- Write Sessions With Purchase = CALCULATE(DISTINCTCOUNT(WebEvents[SessionID]), WebEvents[EventType] = "Purchase").
- Write Conversion Rate = DIVIDE([Sessions With Purchase], DISTINCTCOUNT(WebEvents[SessionID])).
- Write Avg Session Minutes two ways: (a) AVERAGEX(VALUES(SessionID), DATEDIFF(CALCULATE(MIN(EventTime)), CALCULATE(MAX(EventTime)), MINUTE)) and (b) with a calculated column SessionStart via EARLIER, then a measure. Compare the query plan cost and model size.
Interview questions
- How does VertiPaq compress data, and what is the single biggest lever?
- Formula engine vs storage engine: what runs where?
- What is a CallbackDataID and why is it bad?
- Aggregations: when do they not help?
- Incremental refresh: what happens on the first refresh after publishing, and how do you avoid a timeout?
- Give three model-level performance rules you enforce in code review.
Assessment
Which change reduces model size most?
- Rename columns
- Remove a unique integer key column from a 100k-row fact
- Change text to uppercase
- Add a hierarchy
FILTER(FactSales, [Net Sales] > 500) versus FILTER(VALUES(DimCustomer[CustomerKey]), [Net Sales] > 500):
- Same cost
- First is far more expensive due to per-row context transition
- Second is more expensive
- Both are storage-engine only
Detect data changes in incremental refresh requires:
- A date column
- A datetime column such as LastModified that folds
- Nothing
- A primary key
Produce a VertiPaq Analyzer before/after table for FactSalesBig with at least three optimisations and the % reduction of each.
Key removal, Fixed Decimal, date/time split, removing unused columns.