Luke from St Patricks suggested that

1. The answer is $5\mathrm{cm}$. The answer is $2.5\mathrm{cm}$. For every tile you add on the overhang will have to be halved to have equal balance with both of the other tiles.

Yuliang Djanogly City Academy identified that the best overhang in the three tile case is $6.6\mathrm{cm}$ over the base tile. The explanation for the sort of reasoning for this was given by Jason, by using centre of mass considerations.

Whatever amount of tiles there are, when they are placed on another tile, half the mass must be on the tile for them to just balance. So, for the first two, the top tile may sit $10\mathrm{cm}$ on and $10\mathrm{cm}$ off the bottom one. This gives a pivot point for the mass at the halfway point, ie. $15\mathrm{cm}$. Ergo these two may be placed $15\mathrm{cm}$ on and $15\mathrm{cm}$ off the bottom one. Now the pivotal point of the mass of these three tiles is about $16.6\mathrm{cm}$.

An algebraic extension was given by Lindsay from Collyer's Sixth Form College.

So we've established that the maximum overhang for two tiles is $10\mathrm{cm}$. Now we can consider three tiles: The centre of mass of the first two tiles must sit on the edge of the third tiles to achieve maximum overhang.

Using the idea of centre of mass we can produce the following equation, where $x$ is the new centre of mass.
$$10\mathrm{cm} + 20\mathrm{cm} = 2x$$
Rearranging this gives
$$ \frac{10\mathrm{cm} + 20\mathrm{cm}}{2} = x = 15\mathrm{cm}\;.$$
The two tiles can therefore be placed $5\mathrm{cm}$ from the end of the third, giving an additional overhang of $5\mathrm{cm}$ and an overall overhang of $10\mathrm{cm} + 5\mathrm{cm} = 15\mathrm{cm}$.

This process can be repeated to add a fourth tile: Centre of mass of three tiles
$$
\frac{10\mathrm{cm} + 2\times20\mathrm{cm}}{3} = \frac{50\mathrm{cm}}{3} = 16 \frac{2}{3}\mathrm{cm}\;.
$$
So they can be placed $20\mathrm{cm} - 16\frac{2}{3}\mathrm{cm} = 3\frac{1}{3}\mathrm{cm}$ from the end of the fourth tile, creating an overall overhang of $10\mathrm{cm} + 5\mathrm{cm} + 3\frac{1}{3}\mathrm{cm} = 18\frac{1}{3}\mathrm{cm}$.

There seems to be a pattern in these results, which becomes clearer once we tabulate them. This can be generalised for $n$ tiles in the following way:
$$
10\mathrm{cm} + (n-1)\times20\mathrm{cm} = nx\;,
$$
where $x$ is the centre of mass. This rearranges to $$\frac{10\mathrm{cm}+(n-1)\times 20\mathrm{cm}}{n} = x\;.$$ Because the edge of the bottom tile must be directly below the common centre of mass of the stack of $n$ tiles above, $x$ represents the total overhang of a stack of $n$ tiles.

We can see this in a table

Number of tiles being added	Additional overhang (cm)	Total overhang (cm)
$1$	$10$	$10$	$10/1$
$2$	$5$	$10 +5 = 15$	$10/1+10/2$
$3$	$3\frac{1}{3}$	$10+5+3\frac{1}{3}$	$10/1+10/2+10/3$