Top all-time · #14998
D
docs.fluxninja.com
Company blog · docs.fluxninja.com/blogsystemsprogramming
docs.fluxninja.com is the company blog at docs.fluxninja.com, writing about systems and programming. Since 2022, 5 different Hacker News readers have linked it 22 times, most often “Why Your Service Needs Adaptive Concurrency Limits”. No feed was found, so read it on the site itself.
No feed foundVisit the blog
22mentions?
5HN people?
2022–2024first – last linked?
Most-linked posts?
- Why Your Service Needs Adaptive Concurrency Limits2 people · 3 mentions · first linked 2022-12-11
What Hacker News says?
And this understanding is extremely important when you are working on rate limiting. Instead of controlling the rate of requests, one must control the number of concurrent requests - https://docs.fluxninja.com/concepts/concurrency-limiter Throughput (number of requests processed) is different from capacity (number of requests that can be handled). Managing capacity is more practical and optimum solution than managin…
Comment linking /concepts/concurrency-limiter
on “Queues don't fix overload (2014)”Submitted Reliability “Spectrum” as a story · 1 points
HN discussionI agree that queues can cause problems especially when misconfigured. But some amount of queuing is necessary, to absorb short spikes in demand vs capacity. Also, queues can be helpful to re-order requests based on criticality which won't be possible with zero queue size - in which case we have to immediately drop a request or admit it without considering it's priority. I think it is beneficial to re-think how we tu…
Comment linking /concepts/scheduler
on “Queues don't fix overload (2014)”Submitted Protect Your GraphQL Services with Distributed Rate Limiter as a story · 1 points
HN discussionI am in the same school of thought as you, fair queuing is the most optimal solution while capacity is constrained. This is the approach we took in Aperture, an open-source load management system. Read more about our Scheduler which implements a variant of WFQ (Weighted Fair Queuing) to ensure desired capacity allocation across workloads (group of requests at the same priority level) and SWFQ (Stochastic Weighted Fa…
Comment linking /concepts/scheduler
on “Queues don't fix overload (2014)”Submitted Why Your Service Needs Adaptive Concurrency Limits as a story · 1 points
HN discussionI work at FluxNinja CodeRabbit is a customer of ours - https://docs.fluxninja.com/blog/coderabbit-openai-rate-limit...
Comment linking /blog/coderabbit-openai-rate-limits
on “Ask HN: Do tools like Copilot or ChatGPT4 help you code?”Submitted Why Your Service Needs Adaptive Concurrency Limits as a story · 1 points
HN discussionPrioritized load shedding works well as a last resort [0]. The idea is simple - - Detect overload/congestion build-up at the database - Apply queueing at the gateway service and schedule requests based on their priority - Shed excess requests after a timeout [0]: https://docs.fluxninja.com/blog/protecting-postgresql-with-a...
Comment linking /blog/protecting-postgresql-with-adaptive-rate-limiting
on “Squeeze the hell out of the system you have”Submitted Why Your Service Needs Adaptive Concurrency Limits as a story · 1 points
HN discussionWe use Jsonnet extensively in Aperture project for generating control policies. - Policy spec is expressed in protobuf format - GRPC gateway plugin is used to generate Swagger (OpenAPI v2) spec from proto files - Jsonnet bindings are generated from Swagger spec. - Blueprints are implemented using Jsonnet bindings. The users generate policies from blueprints by providing configuration in yaml (using aperturectl CLI)…
Comment linking /development/reference/policies/spec
on “Jsonnet – The Data Templating Language”Quite interesting, thank you for sharing. In one of your tutorials ( https://docs.fluxninja.com/tutorials/integrations/flow-contr... ), you've mentioned that the policy uses latency-based AIMD concurrency limiting; is there a plan to implement a PID (something similar) component in the future? This would be useful to smooth out the adjustments.
Comment linking /tutorials/integrations/flow-control/concurrency-limiting/basic-concurrency-limiting
on “Show HN: Aperture – Unified Reliability Management for Microservices”Fascinating historical insight! We have a team of 20 engineers currently working on solving this problem in the context of API requests and service chains. Do you know JMS @ Penn? Asking because he did some work in ATM networks, QoS etc. He is advising us on the project. This is the link to the project: https://github.com/fluxninja/aperture We built a weighted-fair queueing scheduler as well - https://docs.fluxninja…
Comment linking /concepts/scheduler/load-scheduler
on “Graceful Behavior at Capacity”We built a fair scheduler for APIs and it's in the open source - https://docs.fluxninja.com/concepts/scheduler/load-scheduler I wish more people knew about this project!
Comment linking /concepts/scheduler/load-scheduler
on “Graceful Behavior at Capacity”You should check out the Aperture flow control system - https://github.com/fluxninja/aperture We built a weighted fair queueing system in the Aperture flow control system to help alleviate the API load pressure - https://docs.fluxninja.com/concepts/scheduler/load-scheduler And in addition, we are investing in the graceful-js library to handle 429 and 523 codes returned by the Aperture system - https://github.com/flu…
Comment linking /concepts/scheduler/load-scheduler
on “Graceful Behavior at Capacity”Aperture project's policy language is indeed inspired by Control Systems, especially PID Control. See: https://docs.fluxninja.com/concepts/advanced/circuit
Comment linking /concepts/advanced/circuit
on “Why Adaptive Rate Limiting Is a Game-Changer”It's a blog on an open-source project that precisely tells you how to implement adaptive rate limiting. Just click around a bit: - https://github.com/fluxninja/aperture - https://docs.fluxninja.com/use-cases/adaptive-service-protec... Note: I am one of the authors' of this project.
Comment linking /use-cases/adaptive-service-protection/average-latency-feedback
on “Why Adaptive Rate Limiting Is a Game-Changer”HN: our team has been working on an advanced API queue/load management system called Aperture[0]. It has following capabilities - 1. Limits concurrency based on latency gradient or queue depths etc. 2. Weighted fair queueing scheduler - prioritized load shedding based on workload priorities and latency estimation. Anyone interested in Little’s Law, token buckets, network schedulers, control theory (PID controllers)…
Comment linking /
on “Queues Should Be Empty”Service meshes make it easier to roll out advanced load management/reliability features such as prioritized load shedding, which would otherwise need to be implemented within each language/framework. For instance, Aperture[0] open-source flow control system is built on service meshes. [0]: https://github.com/fluxninja/aperture [1]: https://docs.fluxninja.com
Comment linking /
on “Service Mesh Use Cases”DoorDash team talks about types of metastable failures in microservices and how observability-based control techniques can be used to mitigate them. They introduce a project called Aperture[0] that provides techniques such as adaptive concurrency limiting, prioritized load shedding and graceful degradation to mitigate cascading failures. [0] https://github.com/fluxninja/aperture [1] https://docs.fluxninja.com/
Comment linking /
on a deleted thread