Microscopic symmetries impose strong constraints on the elasticity of a crystalline solid. In addition to the usual spatial symmetries captured by the tensorial character of the elastic tensor, hidden nonspatial symmetries can occur microscopically in special classes of mechanical structures. Examples of such nonspatial symmetries occur in families of mechanical metamaterials where a duality transformation relates pairs of different configurations. We show on general grounds how the existence of nonspatial symmetries further constrains the elastic tensor, reducing the number of independent moduli. In systems exhibiting a duality transformation, the resulting constraints on the number of moduli are particularly stringent at the self-dual point but persist even away from it, in a way reminiscent of critical phenomena.