You're absolutely correct.
Though pressure is distributed evenly in a space given time, at any specific moment - due to the movement of the cone - the front and the back of the box experience the greatest outward pressure, so double baffling can help to aleviate some of the flexing of the wood that's caused by this.
In a ported box you can generally get away with doubling just the front (if flexing is a problem) because there is a port - generally- originating near the back of the box, which allows the pressure to escape, lowering the stress on that portion of the box. The front of the box still experiences an inward pressure from the temporary "vacuum-like" moment when the air is pushed out of the box, then experiences an opposite, outward, pressure when the cone moves the opposite direction allowing air back in. In a properly designed enclosure though, port velocity is minimized and this effect is therefore minimized to the momentary pressure differential within the enclosure.
In a sealed enclosure, no air is leaving or entering the box so by having instantaneous moments of increased outward pressure on the front or back you get an inward pressure on the opposite side, making flexing a bit more prevalent. In a sealed box, if flexing becomes an issue, you should double up more than just the front.
With all of that said, in general flexing does not USUALLY become a problem. You've got to be moving a lot of air and have long stretches of wood that are not braced, which any well constructed box will not have.
