Of course there's no reason why the function should just be continuous on [a,b], or just be differentiable on (a,b). In fact, usually, f will be differentiable (and hence continuous) on a much larger set E, and a and b are just going to be two points that we pick in E (with the additional property that the whole interval [a,b] lies inside E.)