It can be tricky to come back downstairs without being spotted (even when using tricks like timestop and blink people seem to see me). If anyone sees you while you're trespassing (upstairs) everyone will turn hostile (one person seeing you = everyone, oddly enough), I'd bet that's what's happening. So, any idea what's going on? I've already finished the mission by slowing time, choking the appropriate sister, then Blinking her down to the basement, that allowed me to get away without killing anyone but I was still Detected even though I wouldn't have been otherwise. However, I've only talked to each sister once. It seems to be one of the Boyle sisters who notices me first each time but I can't say for sure. I've re-loaded the same save several times, and sometimes everyone gets scared as soon as I come out of the service stair doorway, sometimes it doesn't happen until a few minutes later.

I figured one of them must have seen me upstairs somehow, but after reloading my save a few times, I'm not so sure. However, when I come back downstairs, people start freaking out at the sight of me.

As far as I know, I was never spotted while I was up there, and I stashed all the watchmen's bodies in the attic where they couldn't be found. After taking great pains to blend in with the guests, and quietly incapacitating most of the waitstaff and the watchman guarding the service stairs (by the rum fountain), I snuck upstairs to loot the joint. I am at Lady Boyle's Last Party, I have an invitation and everything.

Dishonored 2 mission 7 stuck in room