This is the second try (original post: https://feddit.de/post/426890) of me trying to get an answer, this time I’ll be more specific of what I am thinking to do. I thought a more generalized question would be enough. Sorry for that.

A peertube server needs lots of storage. Many of the videos will hardly get any views. Storage space on a vps is pretty expensive, storage space in general isn’t cheap. So my thought was to

have a disk at home (maybe external disk on a raspberry pi) and a VPS.

The VPS only has a very limited amount of storage, but is otherwise totally able to run peertube well. So why not have a virtual file system on the VPS, which looks like it has the size of the HDD and it uses a specified amount of the vps storage for caching. So if someone watches a popular part of a popular video, the vps can serve the video content from the local disk. If someone wants to watch the video that nobody ever watches, it’s not a problem since the uplink from home can easily deliver that as well, without the video taking the precious storage. Block caching would be best, since file caching wouldn’t be ideal with video files being really big in some cases. So a very long video would fill the cache, even if only parts of it are needed.

The remote storage doesn’t need to be from home of course, could be cheap cloud storage. I know that peertube works with s3, but it will only move transcoded videos into a bucket and then serve them directly from there. I don’t want that from home, it would also not use the upload performance of the VPS for popular videos.

Any thoughts? Good idea or not?

I have worked with bcache in the past and was always very impressed with the performance, I think my scenario could really work.

  • @holdengreen@lemmygrad.ml
    link
    fedilink
    1
    edit-2
    1 year ago

    I was thinking bcache too. I want to see how to do this.

    personally I bought two multi year VPS deals from Hostens and Time4VPS. A Linux VPS and a 512GB storage VPS. You can use NFS and and get lots of free bandwidth because they are on the same LAN. storage vps is a OpenVZ6 (kernel 2.6.32) which sucks its old debian and no functioning cgroups or loop devices.

    RAM and CPU is still too expensive for me on the Linux VPS so that sucks. I want to see if I can find a service somewhere that will let me send them a Rock Pi 5 16GB with 1TB Netac NV3000 strapped on that and see if they will let me host it for cheaper than I’m paying hostens with decent bandwidth…

    unixsurplusnet (ebay) seems to have good deals on hard drives. I found out you can make server encasings from cheap aliexpress aluminum extrusions (wip).

  • sj_zero
    link
    fedilink
    11 year ago

    I used remote storage on my peertube by just mounting a location on the filesystem and pointing the data directory there. It worked ok, but the throughput was pretty low even on a LAN so I’d spend a lot of time buffering on videos at too high a bitrate.

    I think a thing to think about is what is a popular video on peertube? Yeah, there are some videos by big channels that get thousands of views, but most of the ones I see are maybe 100 views, usually single digit.

    Looking at how bcache is setup, my intuition is that you couldn’t use it to cache a remote filesystem since it looks like it works on the block device level and would need a bunch of stuff…

    I’m sort of surprised that nobody has created a transparent caching solution like that before. It seems like something straightforward, and an easy win for certain workloads.

  • Helix 🧬
    link
    fedilink
    1
    edit-2
    1 year ago

    Try it out. Maybe start with sshfs+bcache and see if it’s viable at all (decrease encryption and remove compression for performance). Then you already have a worst case estimate. Performance tuning can be done with something like IPFS, NFS or glusterfs via wireguard afterwards.

  • @sexy_peach@feddit.deOP
    link
    fedilink
    1
    edit-2
    1 year ago

    Okay so I managed to do this with rclone and a simple sftp connection. It’s pretty slow though, I think rclone isn’t a good solution for caching data through a pretty slow connection. So I’d call this a partial success, at best.

    This is my rclone.service file
    # /etc/systemd/system/rclone.service
    [Unit]
    Description=externalRemote (rclone)
    AssertPathIsDirectory=/mounting/directory
    
    [Service]
    Type=simple
    ExecStart=/usr/bin/rclone mount \
            --config=/root/.config/rclone/rclone.conf \
            --allow-other \
    	--vfs-fast-fingerprint \
    	--dir-cache-time 1h\
    	--vfs-cache-mode full\
    	--vfs-cache-max-age 240000h\
    	--vfs-cache-max-size 15G\
    	--vfs-cache-poll-interval 5m\
    	--vfs-read-chunk-size 1M \
    	--vfs-read-chunk-size-limit 10M \
    	--buffer-size 5M \
    	rcloneRemote:/remote/directory /mounting/directory
    ExecStop=/bin/fusermount -u /mounting/directory
    Restart=always
    RestartSec=10
    
    [Install]
    WantedBy=default.target
    

    If I have the time I am interested in trying nfs with FS-Cache. I think that might work better.