That's also not quite true. Matter is "destroyed", or rather, converted into other forms of energy, when undergoing energy-positive fission or fusion.

The bonds that keep atoms together have mass due to the mass energy equivalence (binding energy). When those chains break or get rearranged, the amount of mass can change. A uranium nucleus has more mass than the sum of its individual parts. The energy you get out of fission does come from mass.

So when uranium atoms split into two daughter atoms, you do end up with less mass than you started out with.

Matter in general is not a conserved quantity, and we break this symmetry every second in our particle accelerators which both destroy and create new matter particles all the time. What is closer to a conserved quantity is energy (although even that isn't quite true on universe scales).