Take videos of/record them. Explain to them that you need sleep/when you go to bed/have to get up (politely, no threats) and suggest a time that you'd appreciate if they'd be quieter on days you work (midnight perhaps). You could start with a note on the door, then a knock on the door. If they don't lessen, call the cops next time (and have the video/s to prove it's not a one time occurrence). The idea of asking other neighbors/maybe have block complaint is also a good one, since then you'd have support.
Generally speaking, I find such neighbors will never change on their own. People have patterns to their lives and they don't always mix with our own patterns - who's to say whose has more importance. We have to find compromises. Don't know what your local laws are, but 11pm-1am is often used for noise. That's assuming it's truly excessive noise. If you're a light sleeper it could be other neighbors haven't noticed much. If you have extra sensitive hearing/don't like loud voices in general, same thing. So another good reason to ask your neighbors.
If you're afraid of confrontation/repercussions to the point of not wanting to rock the boat in a way that they'll know it's you, you may have to live with it a while. I know how you feel - I'm not very confrontational either. But sometimes it's the only way.
I have a completely different pattern to my life then most, especially when I don't go to bed before 2am most nights. But I am never hanging out outside, drinking and being loud, and just plain being rude. (It's the nights I have to get to bed by 11pm that the issue really gets to me.) The noise control hours here I guess are 10 or 11pm til 6am. And honestly, those are the hours they usually hang out. lol I'm not telling them they can't have people over, but they sure need to keep their get together inside.
I am extremely sensitive to sound (crazy sensitive to talking when I'm trying to sleep) so I'm going to ask my neighbors if they've been hearing them, but I just might be extremely unlucky that my windows face directly towards their patio, and my other neighbors all have solid walls towards them. I guess I'm just annoyed that I have enough respect for my neighbors to keep it quiet even when coming home at 4am, but these guys just can't be bothered with it. They're just really unpleasant people to live near...
Though they did go to bed early last night so I didn't hear them, but I still didn't sleep. lol Thank you time change!