This does not work as expected. We should not attempt to be clever here. Varnish will do the right thing if we just teach Pleroma to set headers properly for content we know can be cached.
Someday Pleroma will learn this skill :-)