What limits cake in the fridge is almost never the crumb, it is whatever is sitting on top of it. Buttercream is mostly sugar and fat, so it holds for days. Whipped cream, cream cheese, and custard fillings are wet and protein rich, and they set a much shorter clock.
Sugar and fat preserve, water spoils
Microbes need available water to grow, and food scientists track that as water activity. Dissolve enough sugar into a frosting and most of the water is chemically tied up, leaving very little for anything to live on. That is why an American buttercream sitting under a cake dome behaves so differently from a stabilized whipped cream that is mostly water held in a foam.
Fat helps in a second way: a coating of frosting seals the cut faces of the cake, slowing moisture loss from the crumb. An unfrosted layer left exposed goes stale faster than the same layer buried under icing.

Quality windows by what is on the cake
| Type | Refrigerated, wrapped | Frozen, wrapped tight |
|---|---|---|
| Unfrosted layers or pound cake | About a week, though it dries out first | Two to three months |
| Buttercream or ganache | Roughly four to five days | Two to three months |
| Cream cheese frosting | Three to four days, refrigeration required | About a month |
| Whipped cream or custard filling | One to two days at most | Not recommended, texture breaks |
| Fresh fruit filling or topping | One to two days, the fruit weeps | Not recommended |
These are quality ranges for home kitchens, not a safety ruling. For the official cold storage guidance, FoodSafety.gov and the USDA publish charts worth checking against whatever you are storing.
The refrigerator is also drying your cake out
Cold does not just preserve, it stales. Starch retrogradation peaks near refrigerator temperature, so a cake three days into the fridge can feel older than one left covered on the counter overnight. Two habits fix most of it: wrap the cut faces directly against plastic rather than trusting a loose container, and take slices out twenty to thirty minutes before serving so the fat softens and the crumb reads moist again.
When to stop guessing
Visible mold anywhere means the whole cake goes, not just the fuzzy corner, because the threads run deeper than the spot you can see. Sour or yeasty smells, a slick surface on the frosting, or fruit filling that has turned watery and cloudy are all done signals. A dry, crumbly cake with no off smell is stale rather than spoiled, and it makes excellent trifle. If a frosted cake with dairy in it sat out at a party for more than about two hours, the safe move is to let it go and check the USDA guidance next time.