Low testosterones levels for Male

Patient: Hi Sir, I am 39 years old Male. when I was 30 years old, I had tested for low testosterone levels. I had carried out all tests including MRI but nothing found wrong. Then doctor at that time gave me injections of pregnyl1500 three times a week. He didnt give me the testosterone boosts due to the reason that it impacts the fertility. My first child was born in 2006 and second in 2009. I stopped those pregnyl injections somewhere around mid 2007 or so.I never checked my testosterone levels in the mean time.In 2011, i was feeling sexually week and was again diagnosed with low testosterone. It was at another place and doctor started me injections for low testosterone. Later in 2013 , i came to know through my endocrinologist that testosterone injections impacts the fertility and you have to stop it as I was tring for 3rd baby. I stopped it in start of 2013.In order to boost my testosterone levels without impacting my fertility, doctor gave me clomid and i am taking it for almost 1.5 years. Myself and my wife tests are clear and nothing wrong but unfortunately my wife is not pregnant yet.I want to know if I am on the right path?Did the testosterone injections I got in 2011-2012 may have done some harm to my fertility and should I continue using clomid tablets?I did all my tests recently including testosterone levels and sperms too and all are fine.

Symptoms: I am taking clomid to enhance my testosterone levels.