Yes, you are technically right. Little too much hyperbole on my part, but the bulk modulus of solids in typically in the tens to hundreds of gigapascals

Static pressure on it’s own doesn’t really “do” anything though. You could have the thinest piece of solid material with 10 million atmospheres on either side and the material doesn’t care since forces cancel out. Difference in pressure is where shit goes pop or crush depending on the direction.