Reading the questions here and here has given me some insight into the situation, and it seems like using the AsEnumerable is memory consuming. Is there a better way to do this LINQ and the way it is done now, is the data that comes out reliable?
Removing the AsEnumerable results in a "Local sequence cannot be used in LINQ to SQL implementations of query operators except the Contains operator."
var results = from p in pollcards.AsEnumerable()
join s in spoils.AsEnumerable() on new { Ocr = p.OCR, fileName = p.PrintFilename } equals new { Ocr = s.seq, fileName = s.inputFileName }
where p.Version == null
orderby s.fileOrdering, s.seq
select new ReportSpoilsEntity
{
seq = s.seq,
fileOrdering = s.fileOrdering,
inputFileName = s.inputFileName,
Ocr = p.OCR,
ElectorName = p.ElectorName
};