Top all-time · #7243
Nikolaus Rath
Personal blog · rath.orgsystemsprogramming
Nikolaus Rath's personal blog at rath.org covers systems and programming. Since 2015, 11 different Hacker News readers have linked it 11 times, most often “On the Beauty of Python's ExitStack (2015)”. The last post was in 2022, but the archive is still widely read.
Too few sampled comments (7) to say how often HN recommends it.?
Why HN links it
HN readers link rath.org mainly for the S3QL filesystem documentation, which covers caching, metadata storage and encryption details. Commenters cite it as a source on how S3QL works.
Summarised by Claude from a sample of 7 HN comments that link here. Each comment is marked recommending, critical or neutral (citing without judging); how this works.
Most-linked posts?
- /s3ql-docs/impl_details.html3 people · 3 mentions · first linked 2015-02-22
- On the Beauty of Python's ExitStack (2015)2 people · 2 mentions · first linked 2020-11-09
What Hacker News says?
You can experiment with something kind of like this yourself pretty cheaply. Here's an example [0] user-space filesystem that stores its data in an in-memory sqlite file. You could just as easily replace those in-memory calls with a networked DB (perhaps with speculative pre-fetching or something, I dunno, I probably wouldn't try to make a python filesystem too performant). The salient detail here is that as far as…
Comment linking /pyfuse3-docs/example.html
on “SQLite the only database you will ever need in most cases”Submitted On the Beauty of Python's ExitStack (2015) as a story · 135 points
HN discussion"S3QL splits file contents into smaller blocks and caches blocks locally." -- http://www.rath.org/s3ql-docs/about.html#features It allows you to configure an arbitrary cache size, I've been using it with 60GB local cache.
Comment linking /s3ql-docs/about.html
on “Show HN: Zero – Local file system transparently swapping to the cloud”Submitted Matlab is a terrible programming language as a story · 3 points
HN discussionPydis code is awesome: https://github.com/boramalper/pydis/blob/0a51b6e31475dab083d... I've been a Python dev for roughly a decade and had no idea about asyncio.Protocol, asyncio.transports.Transport, or that asyncio's event loops could be used to create a TCP listening server. I always thought of asyncio as synonymous with async def and websockets. But perhaps the websockets should have clued me in that it's an act…
Comment linking /on-the-beauty-of-pythons-exitstack.html
on “Pydis – Redis clone in 250 lines of Python, for performance comparison”Submitted SSD Caching under Linux as a story · 2 points
HN discussionThis looks cool and I am going to give this a try. The problem for me is, as is usually the case with such project, the packaging. If this thing is production-ready, then why must I check for installed dependencies by running random commands [1]? If it's a Python project, why isn't it distributed on PyPI? I don't want to download stuff from BitBucket manually and install it by executing setup.py. I understand that t…
Comment linking /s3ql-docs/installation.html
on “S3QL: an S3 FS with encryption, de-dup, immutable trees and snapshotting”I believe it uploads the whole thing each time. I think it may even upload it with a new index counter in the name to version it (but I can't find that in the docs now). http://www.rath.org/s3ql-docs/impl_details.html#metadata-sto...
Comment linking /s3ql-docs/impl_details.html
on “S3QL: an S3 FS with encryption, de-dup, immutable trees and snapshotting”I was recently looking into options for offsite, encrypted (at-rest & in-transit), deduplicated, snapshotted, backups to S3. S3QL was one of the options I investigated, but unfortunately their crypto is MAC-then-encrypt [0], which is no good. A small sample of the other options considered: * EncFS + rsnapshot/rdiff-backup * Weak cryptography in EncFS [1] * Duplicity * Uses forward snapshots, so recovery takes longer…
Comment linking /s3ql-docs/impl_details.html
on “S3QL: an S3 FS with encryption, de-dup, immutable trees and snapshotting”Sure looks interesting. It seems like a good idea for specific use cases, like for instance backup storage. The SQLite usage for the inode table is a neat hack. Implementation details here: http://www.rath.org/s3ql-docs/impl_details.html
Comment linking /s3ql-docs/impl_details.html
on “S3QL: an S3 FS with encryption, de-dup, immutable trees and snapshotting”Unfortunately S3QL uses MAC-then-encrypt[1], which is pretty strongly discouraged[2]. Very nice there's a detailed writeup on the details in the docs though, wish more projects did that. 1: http://www.rath.org/s3ql-docs/impl_details.html (last paragraph) 2: http://www.daemonology.net/blog/2009-06-24-encrypt-then-mac....
Comment linking /s3ql-docs/impl_details.html
on “Bup – An efficient backup system based on the git packfile format”