Before you assume the worst, consider:
Potted plant soil – Did someone spill a plant and not clean up properly?
Dried play-doh or craft material – Kids get creative.
Crumbled cookies or crackers – Late-night snacks leave evidence.
Crayon shavings – Sharpening crayons makes colorful piles.
Dirt from shoes – Kicked off under the bed.
Sometimes the explanation is far less dramatic than we fear.