As per your requirement the detail solution as follows:

As  in your question  (1+x+2x2)20 = a0 + a1x + a2x2 +..........+a40x40   ...............(1)

By putting x=1 and x = -1 then the (1) becomes

a0 + a1 + a2 + a3+ a4+......+ a40  = 420 ..............(2)

a0 - a1 + a2 - a3+ a4+......+ a40  = 220 ............(3)    [Note: Odd sub-script terms are negative]

By adding (2) into (3) odd sub-script terms are canceled and you get:

2(a0 +  a2 +  a4+.....+ a38+ a40 ) = 420 + 220 = 240 + 220 = 220 (220+1)

Þ a0 +  a2 +  a4+.....+ a38+ a40 = [220 (220+1)]/2

Þ a0 +  a2 +  a4+.....+ a38+ a40 = 219 (220+1)

Þ a0 +  a2 +  a4+.....+ a38 = 219 (220+1) - a40   ................(4)

You can get a40 = Coefficient of x40 in (1+x+2x2)20 

                              = Coefficient of x40 in [1+x(1+2x))20

                                        = Coefficient of x40 in S 20Cr xr(1+2x)r where r=0 to 20

                              = Coefficient of x40 in  20C20 x20(1+2x)20 

                              = Coefficient of x20 in  20C20 (1+2x)20  = 20C20 *220  =220

Using it in (4) 

a0 +  a2 +  a4+.....+ a38 = 219 (220+1) - 220  =  219 (220+1) ? (219 *2 ) = 219 (220+1-2)

                                       =  219 (220-1)

So (b) is the answer.