Rocco was born Alexander Federico Petricone, Jr. in Cambridge, Massachusetts. He is of Italian descent.
According to organized crime turncoat Vincent Teresa, Alex was a hanger-on with the Winter Hill Gang of the Boston area. An unwanted advance toward Petricone's then girlfriend on Labor Day, 1961 touched off the Boston Irish Gang War of the 1960s.
Howie Carr, a Boston-area journalist and radio personality who has written extensively about the Boston underworld, has written that the young Petricone (whose nickname was "Bobo") was arrested in Charlestown in November 1961 along with Winter Hill boss Buddy McLean for questioning following the death of Bernie McLaughlin of the McLaughlin gang, the first murder of the war.
Petricone moved to California in 1962 and began using the name Alex Rocco. He first worked as a bartender in Santa Monica, California and took acting lessons from actor Leonard Nimoy, a fellow Boston native. Nimoy was not impressed with Rocco's heavy Boston accent and told him to take speech lessons. Rocco followed through with Nimoy's instructions and after ridding himself of the accent came back to study under Nimoy and character actor and teacher Jeff Corey.