I would also make an appointment to see my PCP -today!
I had some dull pains a couple of years ago in my right side that were bothersome, so I went to the doctor. They didn't find anything wrong, and soon after my apointment, they went away.
My FIL used to have pains in his gut all the time. One night, he went to bed in pain and feeling miserable, and by the middle of the night, he thought he was having a heart attack. He was rushed to the hospital in an ambulance, and it was discovered he had pancreatitis. They think a gall stone blocked the duct shared by the gall bladder and the pancreas, an caused the pancreas to become inflammed. He spent 4 months in ICU, 9 months in the hospital / rehab facility, almost died several times, has had multiple surgeries (10 or more) to address the issue, and is now on disability. He does still have some of his pancreas, so he does not have to take insulin, but he does have to take medication every day to help with digestion.
Your pancreas is nothing to mess around with.
Hope you are feeling better soon.
Denae