3.7.13 · D1Algorithm Paradigms

Foundations — DP problems — matrix chain multiplication

2,741 words12 min readBack to topic

This page assumes you have seen nothing. Before you can read the parent MCM note, every symbol it throws at you must first be earned. We build them one at a time, each on top of the last.


0. What is a matrix, really?

Figure — DP problems — matrix chain multiplication

Look at the figure. The magenta grid has rows and columns, so it is a matrix. The little labels on the outside are the dimensions. That is the entire vocabulary you need — we never look inside the grid on this page, only at its shape.

Why shape and not contents? Because MCM asks "how much work is the multiplication?", and — as we will see — the workload depends only on the shape, never on the actual numbers inside.


1. Multiplying two matrices — the shape rule

You cannot multiply just any two matrices. There is one rule about their shapes.

Figure — DP problems — matrix chain multiplication

In the picture the inner numbers (violet, touching in the middle) must match — they are the shared dimension . The outer numbers (orange) survive to become the shape of the answer. A handy way to remember it:


2. The cost of ONE multiplication — where comes from

Now the heart of it: how much work is one matrix multiply? We keep the exact same from §1 — rows on the left, shared middle , columns on the right.

Figure — DP problems — matrix chain multiplication

The figure shows one highlighted output cell (orange). To compute it, we slide along the shared strip of length (violet), doing multiplications. There are such cells, so we pay that many times.

Why count multiplications and not additions? By long tradition, and because on real hardware multiplications historically dominated the cost. The parent note fixes this convention; we follow it.


3. Why one dimension array describes the whole chain

The parent note never lists the matrices' shapes separately. Instead it uses a single array . Here is why that is enough.

Figure — DP problems — matrix chain multiplication

Look at the figure: each straddles two adjacent -values. sits on , on , and they share — which is exactly the matching-inner-dimensions rule automatically satisfied.


4. The subscript and the index

The little number under or is a subscript — a nametag, not arithmetic. means "the -th matrix"; means "the -value one step to the left of position ".

We will meet three at once in the recurrence: (start of a sub-chain), (end of a sub-chain), and (the place we cut). Keep them straight: and are the ends; roams between them.


5. Parenthesization — pairing up with brackets

Figure — DP problems — matrix chain multiplication

Both trees in the figure produce the same final matrix — that is a property called associativity (rearranging the brackets in a product does not change the answer). But, as §2 showed, the two orders can cost wildly different amounts. That gap is the entire reason MCM exists, and it connects to Catalan Numbers, which count how many bracketings are possible.


6. The two words that make DP work

The parent note leans on two phrases. Here is what each one means as a picture.


7. The DP table, its base case, and how it fills

Now we can name the storage and — finally — assemble the formula.

Figure — DP problems — matrix chain multiplication

The figure shows why the loop is ordered by length: to fill a length-3 entry (top), the arrows show it reaches down into length-2 and length-1 entries — which only exist if we already filled them. Fill row-by-row and you would ask for numbers that aren't there yet. Fill by length, shortest first, and every value you need is guaranteed ready.


8. Assembling the recurrence — all symbols now earned

Every piece is defined: (), the cut index , the cost of one combine, the table , the over all cuts, the base case. Here is how they click together.

Everything you defined above feeds exactly one slot in this formula. That is the payoff: not a single symbol here is new.


The prerequisite map (how the pieces feed the formula)

Read this map as a flow: the shape rule and the cost formula (top) both feed the cost of a bracketing; the base case and length-ordering feed the table; and everything converges on the single MCM recurrence at the bottom. Trace any arrow and you should be able to say why that dependency exists.

n equals number of matrices

Dimension array p

Matrix a by b grid

Shape matching rule

Cost equals a times b times c

Matrix A i is p left times p right

Cost of a bracketing

Parenthesization brackets

Associativity same result

Minimize scalar multiplications

Base case m i i equals 0

DP table m i j

Optimal substructure

Overlapping subproblems

Fill by chain length L

The min over all cuts k

MCM recurrence


Equipment checklist

Read the term, answer aloud, then reveal.

What does stand for in MCM?
The number of matrices in the chain, .
What does " matrix" mean, and which number is rows?
A grid with rows and columns; the first number is always rows.
What shape rule lets you multiply an by a matrix, and what is the result's shape?
The inner dimensions must match (both ); the result is .
How many scalar multiplications does an times multiply cost, and why?
— there are output cells, each costing multiplications.
What is a scalar, and does MCM count additions?
A scalar is one ordinary number; MCM counts only multiplications, ignoring additions.
In the array , what are the dimensions of matrix ?
(borrows the number on its left and its right).
Why does an array of numbers describe matrices?
Neighbours share one boundary dimension, so there is exactly one more -value than matrices.
What does "parenthesization" choose, and why doesn't it change the final matrix?
Where the brackets go (pairing two at a time); the result is unchanged because matrix multiplication is associative.
What is optimal substructure in one sentence?
The best overall plan is made from best plans for its independent left and right sub-chains.
Give a concrete overlapping subproblem in .
The sub-chain (or ) is reused across several different whole-chain bracketings.
What does store, and where is the final answer?
The cheapest cost to multiply ; the answer is at .
What is the base case of the table and why?
, because a single matrix needs no multiplication; it is initialized first.
What does instruct you to do?
Try every cut point from to and keep the smallest cost.
Why fill the table by chain length ?
A longer chain needs shorter sub-chains already computed, so shorter lengths must come first.
State the full MCM recurrence.
, with .

Connections

  • Parent topic — full MCM note
  • Dynamic Programming — the paradigm these foundations feed into
  • Catalan Numbers — how many parenthesizations exist
  • Memoization vs Tabulation — two ways to exploit overlapping subproblems
  • Optimal Binary Search Tree · Burst Balloons — same interval-DP shape
  • Time Complexity Analysis — why brute force is hopeless