Another approach could be as follows:
let ABC be a triangle inscribed in the circle with center O and radius r.
If the area of this triangle is maximum, then vertex C should be at a maximum distance from the base AB i.e. CD must be perpendicular to AB.
Hence, ABC is an isosceles triangle.
If

BCD =

, where D is the mid-point of BC, then

BOD = 2

so, AB = 2 BD
= 2r sin 2

CD = CO + OD = r + r cos 2

If S be the area of the triangle ABC, then
S = (1/2) AB x CD
= (1/2) x 2r sin 2

(r + r cos 2

)
ds/d

= r
2[sin2

(-2 sin2

) + (1 + cos2

)(2 cos2

)]
= 2r
2[cos
22

- sin
22

+ cos2

] = 2r
2(cos4

+ cos2

)
For maximum and mimimum
ds/d

= 0
or, cos4

+ cos2

= 0
or, 2 cos3

cos

= 0
so, Either cos3

= 0 or, cos

=0
If cos

= 0 or 3

=

/2 or

=

/6
(d
2s/d
2) = -ve, for

=

/6

ACB = 2

= 2(

/6) =

/3 =

ABC =

BAC
so

ABC is an equilateral triangle.