While answering a question in a recent video, the head of Google’s web spam team Matt Cutts has said that the page speed ranking factor does not carry more weightage in case of mobile search compared with the desktop search.
Matt Cutts clearly said:
It is not that in mobile, we apply that [page speed ranking factor] any more or less than desktop search.
However, in the same he also went on to say that there is no doubt that users of mobile expect faster page landing speeds and Google is aware of it. And if Google feels that search algorithms need to be tweaked to reflect this, Google has every right to do so.
You can watch the full video here:
It seems very likely that going into future, Google will make some kind of adjustments to its algorithms with respect to page speed ranking factor for mobile search.