Are stimulants dangerous for panic attack sufferers?

Patient: Hello, can stimulants prescribed for ADHD cause a panic attack to be dangerous? At normal doses? I get occasional panic attacks and am uncertain about taking the stimulant. If I have a panic attack while on my Adderal, does it have the potential to make a panic attack worse such as a dangerous arrhythmia? My doctor seems unconcerned and told me to take one of my PRN Xanax for any panic attack that occurs. MyPanic attacks can be short but severe w/heart racing, etc.

Symptoms: Panic attack….heart racing