Machine-Scale Problem, Meet Human-Scale Solution

Greetings fellow humans! It is now 2025 and while we still don’t have flying cars, we do have self-driving cars—that has got to count for something. Some 2.6 million years ago humans began using tools. Today is a different day because, while we are still using machines as tools, machines have surpassed human ability on three important dimensions: The ability to observe change beyond what is humanly possible, efficacy beyond what is humanly possible, and the performance of analyses beyond what is humanly possible and summing it up at a level for human comprehension. I’m talking about problems and solutions which exist at machine-scale—which is exactly the topic for this article.  

Machine-scale is defined as : “The capacity or level at which processes, systems, or operations can be efficiently executed or automated at the scale of machine capabilities, typically involving high speed, vast data processing, and extensive computational power beyond human ability.”

This topic is near and dear to my heart because it elegantly explains the value of DNS, as well as what we do here at DNSFilter. The domain name system (DNS) exists because all the addresses which are easily handled by machines are not well handled by humans. Thus a domain name—a name easily remembered and used by a human—was invented to ensure that the machine-scale nature of Internet addresses had a human interface. One does not have to remember to type in 2001:4860:4802:32::78 when they can just remember to type in www.google.com, if you get what I mean. DNS delivers a human-scale solution to a machine-scale problem. 

Human-scale understanding relies on machine-scale solutions

The observational domain we operate in is at the rate today of 170 billion DNS requests per day and it is growing at about 16% each quarter. No amount of human staffing could deliver the outcomes we need in the time we need it. So here again, we are having to deal with a machine-scale problem, requiring a machine-scale solution, so that we can deliver human-scale understanding.  

As CTO of DNSFilter, I am always looking for the right tool for the right job. Nothing makes me happier than when a customer experiences value from the hard work myself and my team put in day and night. On that note, I’d like to talk about some areas of technology that we are quite proud of, and explain why we invest so much effort in them to ensure our customers have an awesome experience.  

DNSFilter is a very fast—if not the fastest—resolver on the planet.  People sometimes ask why we care about that so much. We are a security company, isn’t it our job to block bad things?  I quickly remind them that when we are not blocking bad things, 99.999% of the time we are just resolving DNS names that users ask us to resolve. Being the fastest, no matter where you are operating from on the planet, means we are never in your way—until we need to be.  

No one, and I mean no one, wants to run a security product that slows them down. That is true for humans and machines.  One of the tenets we practice in our Platform Engineering team is: “We strive to be the fastest”. We measure this daily through tools like DNSperf, where we are always very comfortably in one of the top spots. We achieve this level of speed because of our understanding of Internet architecture, the laws of physics, and our ability to operate at machine-scale.  

Efficacy matters

OK, we are fast but to what end? I already said that most of the time we are fast because we just want to stay out of the way and make sure you have a great Internet experience. But when we block, we better be right. 

This is where efficacy matters. Again, we can only do this because we are ready to deal with a machine-scale problem—being, at the time of writing this, to process approximately 1.8 million DNS requests per second. We’re constantly analyzing which one of those requests is intending to connect to a malicious site, which one of those will be violating a corporate policy, and so on.  Every domain name we observe is analyzed and categorized into labels that humans can understand, and in turn they can create policies that keep their communities safe.

DNSFilter’s global edge network is working hard all day and night processing on a global scale.  I already mentioned that this amounts to 170 billion requests per day. At this growth rate, we are on pace to bypass the 228 billion request-per-day mark by summer 2025.  

It is one thing to just service these requests, but it is entirely another to accurately analyze them. This is once again a great example of machine-scale problems requiring machine-scale solutions.

Machine-scale is limited by human cognition

By this point, you may have put together a drinking game every time TK says machine-scale. But in all seriousness, none of this machine-scale stuff has any value if you cannot bring it down to human-scale understanding. The limits of human cognition are still the limits of how far we can push machines. The UI/UX of anything we do must deliver value and utility to a human.  

We are watching GenerativeAI grow from a simple chat interface to interacting with humans in many different conversational ways. My point being, we cannot forget that if we are to provide value to our customers, we must design for the human experience. And ultimately, that is which sets our constraints on how far we can push machines. 

Wait, are you saying that humans are holding back machines from progressing at machine-scale? Yes, that is exactly what I am saying. 

A notable example is when in 2017 Facebook researchers discovered their chatbots had developed their own language while negotiating with each other, prompting them to shut down the experiment. There are other examples like this one, but my point is that there is so much we can do for humanity in well designed machine-scale solutions.   

I will conclude here with an emphasis that we should measure our machine-scale solutions through the utility they bring to us as humans. We must ask the end user if what they just experienced was useful or not. Plain and simple. With all that we do with machines, we do it for the good of humanity. At least, that is how we think about it here at DNSFilter.