Actually the refraction takes place as soon as there is change in medium,i.e immediately the ray shall bend towards the normal as it enters the glass from air......

But because in our terms the deflection between the incident ray and refracted ray(within the glass) isnt much significant (as in the diagram) so we take that bending takes place from the pole.But nonetheless,we must remember that the ray has already deviated and is already moving in a deviated path right since the time it entered the glass,which bends towards the normal....

try drawing ray diagrams yourself to understand more.....