Maryland was the earliest Catholic part of the United States.
Louisiana was French (and Catholic) until the Louisiana Purchase in 1803.
The Spanish established settlements in St. Augustine, Florida, in 1565, and in New Mexico starting in 1598. There were earlier French (Fort Caroline, 1564) and Spanish (Pensacola, 1559) settlements that did not survive long.
Florida was added to the United States in 1819.
Most of New Mexico did not become part of the United States until 1848.