Check out the Telltale Dozen in Phases of Recovery from a Restrictive Eating Disorder.
Usually I am asked this question because the person asking is not clinically underweight. However, 2/3 of those with active restrictive eating disorders are not clinically underweight, nor have they ever been.
There is also a common theme of "not being sick enough" that those with restrictive eating disorders seem to think delineates whether they need to recover or not.
You need to recover when the Telltale Dozen rings true for you.