In one of the cells I'm creating, I want part of it to be open for the player to walk on, but if they go into certain areas, it will be considered tresspassing. I know how to set a cell to be owned by a certain faction. Maybe jails use a similar method? i.e You're allowed to walk around in the cell, but if you escape, you are considered hostile. Any ideas?

Are you talking about exterior or interior cells?