While browsing through old LOSSAN meeting agendas, I found out that Amtrak used to give ridership information by station pair. Unfortunately, FY2005 (September 2004-September 2005) appears to be the last year that it was given for, in this meeting.
One important note is that Los Angeles may be overrepresented due to a need to transfer there for a number of trains. Acknowledging that as a potential issue, a quick run through the data shows that the Surfliner Express' schedule would suffice for 974,840 of the 2,520,444 passengers that year, of whom 2,100,954 were traveling solely on the southern portion of LOSSAN between Los Angeles and San Diego (though what looks like a significant majority of those traveling to points north do so from or to stations south of Los Angeles).
Beyond that I don't have any remarks right at the moment other than to wish that this information was still made public on a regular basis.