But he recovered from his awful start in strong, gusty winds at Muirfield Village Golf Club - on a day in which only three players broke 70.
Perry fired an eagle from 35 yards to get back in touch and eventually carded a one-under-par round of 71 to finish at -7.
"I hit the shot of my life to keep the ship from sinking," he said.
"I knew I could chip the ball down within 15 feet, and it came out perfect and just fell in, like a slow-motion deal.
"It was brutal out there, so I'm mentally tired now. That was a stressful round to try to hang in there and keep it together."
Overnight leader Goggin went the other way, getting off to a great start with four birdies in his first five holes before falling away to a level-par round of 72.
"I didn't know what was going on, but it just got windier and windier and tougher and tougher," Goggin said.
"Not having the experience on this course, I made a few mental mistakes and left a couple of shots in bad positions or misjudged the wind."
Jerry Kelly (72) lies clear in third, one shot adrift at -6, with Luke Donald (71) a further shot back, ahead of a group of four including Geoff Ogilvy at -4.



