We would have to find a way to recognise if a file is an LFS object. Currently the only way to do so is by reading the file contents and look for the version;oid;size lines that the link contains. The only information that git has when it comes to lfs objects is the .gitattribute file which contains info on what files are considered lfs object.
So the main obstacle is figuring out how to recognise the file.
After that it shouldn't be that difficult to just ask Carrierwave to tell us where the file is and serve it via gitlab-workhorse.
@marin I think verifying it's a plain text file (using the existing method for that purpose) and then checking if it matches the version;oid;size format is enough. The smudge filter does the same thing per the docs.