Jim's coaching career began at Morningside High School in Inglewood, California where he served as an assistant coach from 1964-1969 and as head coach from 1970-1973. He was then hired as an assistant coach at Utah State from 1974-1977. Harrick then spent two seasons as an assistant coach at UCLA from 1978-1979. He got his first collegiate head coaching job at Pepperdine University in 1979, where he led the school to four NCAA Tournament appearances and was a conference coach of the year four times.

​

In 1988, Jim returned to UCLA to assume head coaching duties after the firing of Walt Hazzard. During his first year, he recruited Don MacLean which helped turn the UCLA program around. During the 1994-1995 season, he led UCLA to the school's eleventh national championship in school history and its first since the 1974-1975 season.