The pill’s main mechanism is suppressing ovulation. Basically, it shuts down your ovaries. As soon as you’ve taken seven consecutive pills, that’s done and they probably won’t wake up enough to release an egg if you take your break week. Basically, your break week is the window where the ovarian suppression is sufficient to prevent ovulation. However, Some research shows that around 2% of pill users ovulate after the break week (then the pill uses its “backup” mechanisms), while pretty much none ovulate if the break is shortened to 4 days (I’ll find the article and link it in a sec).
This means that you theoretically can take a break anytime when you’ve taken 7 consecutive pills correctly, but this information can be really confusing, so pamphlets really give you the guidance on missed pills to keep it simple.