Someone cancelled an order after delivery and the whole managers group got an alert — can I turn that off?

No, by design. Cancelling an already-delivered order is the one action that can quietly rewrite money, so the group is told immediately: invoice, amount, who, and the typed reason. Stock and any coupon are returned automatically; if cash was collected, the refund is yours to arrange.