You are seeing the larvae, which do the most damage - eating carpets and other fabrics, especially in hidden places such as in corners and under bookcases stood on carpet. You probably have more of them than you think (sorry to break that to you). You need to:
- break the life cycle, by destroying any larvae or adults you find (you would have difficulty finding the eggs),
- be more diligent in your cleaning (to reduce the chances of re-infestation in future)
- perhaps use chemicals as a means of killing the existing presence and/or preventing future ones
You can also consult your local council pest control officer.