For those currently invested in the stock market, there is one thing that has investors walking on egg shells more than anything else: a possible rate hike implemented by the Federal Reserve. Further, the possible rate hike seems to be just as scary as the real thing. For those figuring that the Fed still might hike rates in September are getting one more piece of evidence that this is so.

As this week was drawing to an end and the Fed’s “quiet period” before meetings was about to settle in, investors recoiled over news that the central bank’s most dovish official, Governor Lael Brainard, would be delivering a previously unannounced speech Monday at the Chicago Council on Global Affairs.

This news sent shivers through markets Friday, with major stock market averages taking a beating and short-term government bond yields and the U.S. dollar moving higher, and it set off yet another round of speculation over whether the Fed is ready to come off its historically loose monetary policy. The S&P 500 was down more than 1 percent Friday afternoon, on track to close with its biggest percentage move since July 8. The Dow Jones industrial average fell about 300 points lower.

“When a market is quiet, it’s susceptible to rumors, whether we’re talking about a path to freeze oil production or whether the Fed is going to raise rates in September,” said Quincy Krosby, market strategist at Prudential Financial. “This may be a market that has too much time on its hands right now.”

This is not something new, this game of possibilities has been going on ever since the stock market started rebounding after the recession. The Federal Reserve placed interest rates at record lows with the intention of allowing investors to be more profitable and to allure more potential investors with that. The only thing is the Fed did so in a time when they had to do so but were likely unable to forecast how things would turn out. One thing that is likely is that the game of investor fear over the Feds decision will exist until there is an actual rate hike.