welcome! in a small collection, it may be easier to try and get rid of them than with a big one. I would repot everything. either use new pots or get a water spray wand that you can hook up to a sink or if it's warm outdoors. Gently spray out every crevice in each plant after taking off any dead or dying leaf/root material. set the plants aside to dry, after a little while you'll probably see more white spots moving around; blast them again. do a few times and then use new potting material and pots or blast clean the pots as well. a key thing to do is to clean or change the tray or shelving that your plants are sitting in/on, and they can be cleaned with a household spray cleanser that will kill any crawlies that are moving around away from the plants. they are very good at moving around and hiding where there aren't any plants. if the trays and shelves can be blasted with water as well then that can be done instead of chemicals, just trying to eliminate a backup population of bugs. if you see tiny white cottony masses here and there near where your plants are, those are usually egg masses. a mix of mineral oil, a few drops of mild dish soap and water can be used for contact spraying of the bugs, provided that the plants won't be in bright light or extreme temps which can allow the mix to burn plants. this won't poison the bugs but can help to smother them. I think it is a few teaspoons of oil, a few drops of soap and a liter/quart of water, shaken periodically to keep the mix together (think salad dressing). using lukewarm water can help the oil mix into the water better.
after that, if you would like you can use a systemic insecticide, some come in granule form that can be applied to the top of the pot or drenched into the pot. mealybugs have that white, waxy material that often makes it tough for chemicals to get to their body which can keep them from being exposed to the insecticide.
when you get new plants, it may be a good idea to keep it isolated from your hopefully clean collection, repot and give it the same treatment of blasting and cleaning that you did with your other plants. even if you don't think there are any bugs it's a good practice to put plants into potting material that works in your collection, because mix that grows good plants in greenhouse conditions may not work well under lights or on a windowsill.
hope this helps, good luck!
charles
I first noticed the problem when I saw the bottom leaf turning yellow, and when I peeled back the leaves to look in-between, I noticed a white substance that looked like toothpaste. There was a bug - I think it was a bug, it wasn't really moving around much - with white, hairy whiskers. 
