At what point do the benefits of taking Adderall during pregnancy outweigh the risks?

I was taking 15mg of Adderall twice daily for 4-5 months until I found out I was pregnant 2 weeks ago (about 7 weeks , now). Since I stopped taking my ADHD meds, I have been very irritable, having bouts of depression and mood swings, trouble sleeping/sleeping too long, distant from loved ones, fatigued, etc. I was also taking .3 Clonidine low blood pressure med to counteract Adderall at night, and help me relax for sleep, as well as Restoril 30 mg at bedtime for sleep. I stopped everything cold turkey, and I know the Restoril is absolutely OFF the list while being pregnant, and my Psychiatrist and I decided together to see how well I would do without my other meds, but I don't feel like I'm doing so well. They are considered a Class "C" for pregnant women, which could be potentially harmful to the fetus...unless the benefits outweigh the risks. I don't want to cause any harm to my baby, but I'm really not handling things well. I have many stress factors in my life right now, and I know that could be dangerous, as well. When I was pregnant with my daughter 4 years ago, I was not on any meds and I did fine, but then my Dad committed suicide on my 8th month of pregnancy with her and I lost both Grandparents 4 months later. I no longer have a mother and sister....they want my daughter, not me. I have a loving fiance, his two awesome teenage kids and my beautiful daughter. I have much to be grateful for, but a lot that is causing me stress. I haven't felt like I was able to concentrate for 30 years of my life until the Adderral. I'm wondering if it is best I just take a smaller dosage once a day. I know I need to talk to my doctor, but I just wanted to hear from anyone here who's experienced anything similar being pregnant. Thank you.