Board Thread:General Discussion/@comment-4427410-20160805014450/@comment-25322628-20160930182654

Probably because of the Stone being used to do many different things at once. Also, in The Avengers, the Stone couldn't have done anything to the heroes during their arguments; there were seeds of doubt already there. Tony didn't trust Fury with what he knew, Banner didn't believe that S.H.I.E.L.D. sought out his help only and needed a contingency plan, Steve was beginning to wonder why they were all looking for the Tesseract.

When the Stone was incorporated into Vision, it must've been suppressed, enabling the android to refrain from using it to control others, since free will is a blessing and curse.