The best regular season record to not make the playoffs was 11-1-2 by the 1967 Baltimore Colts. They played in the Coastal Division of the NFL and ended the season tied with the Los Angeles Rams. Coming into the final game of the season, which was against the Rams, the Colts were 11-0-2 with the one of the ties being against the Rams. The Rams won the season finale, 34-10, to win the Division. There were no wild card teams back then so the Colts didn't make the playoffs even though they tied for the best record in the NFL. No other team won more than 9 games that season.
The 1963 Green Bay Packers went 11-2-1 and didn't make the playoffs, losing out to the 11-1-2 Chicago Bears.