It is safe as long as the next pack is started on time.
Basically the way the (combined) pill works is by suppressing ovulation. After start taking it for the first time, it takes 7 full days of pill taking for your body to reach hormone levels that stop you from ovulating. Once you get to that point, when you stop taking the pill (break week), there's enough hormones in your system to ensure that you don't ovulate for 7 days. If you extend your break by a few days, that creates a chance for ovulation to occur, meaning your protection "resets" and any sex during your break week may have been unprotected (by the pill) because sperm may still be present in the vagina and fertilise the egg that was just released. If you start your next pill pack on time, then there's no change in protection and there's nothing really to worry about. No egg = no baby.
If you were unprotected by the pill during break week that means the pill would work for 2 out of 4 weeks of the month which would be pointless.
Hope that clears it up for you!