After only eleven games with the Selkirk Fishermen of the MHL Sr., where he collected twelve points, Johnny Sheppard signed with the WCHL's Edmonton Eskimos as a free agent for the 1922-23 season. After four seasons with Edmonton, Sheppard was traded to the Detroit Cougars for cash in 1926, and that is where he began his NHL career.
Sheppard played in Detroit for two seasons before being dealt to the New York Americans in 1928. In his first season with his new team, Sheppard saw his first NHL playoff experience only to be ousted in the first round by their cross-town rival Rangers. The 1929-30 season saw Sheppard collect his NHL career high in points with 29 in 43 games though the team missed the playoffs for the next few years. After five seasons in New York, Sheppard was traded once again, this time to the Boston Bruins, in 1933. His stay in Boston lasted only four games before he was released, though he signed as a free agent with the Chicago Blackhawks.
The Hawks finished the season behind the Red Wings, but would have the last laugh in the playoffs. After going through both the Montreal teams, the Canadiens and the Maroons, the Hawks found themselves in the Stanley Cup final against the Wings. It took only four games to bring Chicago its first Stanley Cup championship, and for Sheppard his first and only Cup. He left the NHL with his championship and played for two more seasons with the NWHL's Seattle Seahawks before hanging up his skates for good in 1936.