There is a difference between the righteous who work on themselves and those who have no inner struggle. There are those who by nature or habit are not drawn to physical pleasures. Even if in action and behavior the two may be identical, one who does the inner work is called an "Oved Elokim" one who worships G-d.