Ur question is not so clear to me(the figure)... so changed some things (Let the triangular block be M, smaller block on the wedge be m)
Let the mass of hanging block be m`
acceleration of system is a=(m`g)/(m+M+m`)
The mass m will not slide over M if 
substituting,
=gsin\emptyset)
cross substitute and simplify for m`
u get
/(cot\phi-1))
cheers !! rate if u like