I need advice 😣

Hey guys this has nothing to do with sex or periods or anything like that I just need some advice 
So me and my boyfriend have been together for just over 3 months (I'm 16 and he's 17, almost 18) we knew each other for just over a week before we started dating we met threw my best friends uncle anyway, he calls me every night when we are apart but whenever he's with his mates they are always drinking and when he calls me he hardly talk to me, he will always be talking to his mates in the background than get so deep into conversation with his friends that he will ask if he can call me back in an hour. At first it didn't bother me because I didn't want to me the controlling girlfriend and stop him hanging out with his friends but lately it's really starting to get to me and I don't know what to do. His replays to my texts have also been getting slower, at frist they were instead but now it's taking him longer and longer it text back, like today it took him 5 hours to reply and when I asked why he said "sorry I was out with my mates"
Am I just reading to much into it? I have no idea what I should do😞