Congratulations to Curt, Reigate College and to Andrei, Tudor Vianu National College, Bucharest, Romania for you solutions to this problem

When AB is a diameter angle ACB is 90 degrees so we can use Pythagoras' Theorem. The area of the triangle is given by 1 2 ab= 1 2 ch where h is the length of the perpendicular from C to AB. Then
(a+b )2 = a2 + b2 +2ab= c2 +2ch.

Thus (a+b )2 is a maximum when h is a maximum and equal to the radius of the circle 1 2 c . So the maximum value of a+b is c2.

triangle inscribed in circle with altitude h
When AB is not a diameter we have (using the Cosine Rule):
(a+b )2 = a2 + b2 +2ab = c2 +2abcosACB+2ab = c2 +2ab(1+cosACB) = c2 +4ab cos2 1 2 ACB. .

As the area of triangle ACB is given by Δ= 1 2 absinACB we have
(a+b )2 = c2 + 8Δ cos2 1 2 ACB sinACB = c2 +4Δcot 1 2 ACB .

If we keep A and B fixed and vary C then, as c and ACB are constant, and the area of the triangle Δ is a maximum when h is a maximum, it follows that a+b is a maximum when h is a maximum, that is when a=b and the altitude of the triangle drawn from C to AB is a line of symmetry of the triangle. In this case
a=b= c 2sin 1 2 ACB .

Conjecture: Let Q be a variable cyclic quadrilateral in a circle of radius r. Then the area and the perimeter of Q will be a maximum when Q is a square; that is when each of the diagonals of the quadrilateral is a diameter and each diagonal bisects of the other diagonal at right angles.

cyclic quadrilaterals
Labelling Q as ABCD, consider triangle ABC with AC fixed and B varying. Let B' be the position of B when, by the previous result AB+BC is a maximum, that is when AB=BC Note that this also gives the maximum area of triangle ABC. Similarly by considering triangle ADC with AC fixed, we find D' where AD+DC is a maximum and AD=DC. Now B'D' is a diameter, keep this fixed and consider triangles B'CD' and B'AD'. The positions of C and A that maximise the perimeter and area are C' and A' where A'C' is a diameter. Hence, for the perimeter and area of Q to be a maximum all the sides of the quadrilateral must be equal making Q a square. Hence the maximum perimeter is 4r2 and the maximum area is 2 r2 .