Technology
The RuleSpace Backend Systems
For over 11 years RuleSpace has been scanning, defining, categorizing, and storing category ratings for the Internet. In that time we’ve built hundreds of systems that help us do this in an automated, scalable, and highly reliable way.
This system scans the Internet, categorize sites and pages, receive and process customer feedback, automatically discovers previously unknown sites, and send updates every day to customers all over the world.
Some statistics about the RuleSpace Backend Systems:
- The Backend Systems add over 20,000 site categorizations each day to the Master Database.
- The RuleSpace Master Database contains over 60 million URL: category ratings. RuleSpace receives over 30 billion lookup requests each month on its hosted service.
The RuleSpace back-end supports 45 million subscribers through its partnerships with ISPs and MSOs, and over 30 million subscribers through it partnerships with Mobile Carriers. - The RuleSpace Backend Systems throw away more URLs every year than other lists companies have in their entire database.
- The RuleSpace Backend Systems scan the entire Internet approximately 3 times per year, and do focused scanning of the Internet on an ongoing basis.
- All the automated systems in the RuleSpace Backend are subjected to regular quality analysis by our editorial staff and engineers to make certain that all automated systems are at least as accurate as trained human reviewers.
To learn more about how the powerful, scalable, and accurate RuleSpace Backend Systems can be put to work for you, we invite you to review some of the Case Studies on our site, and to contact RuleSpace directly today to discuss you application.
1] Where the “rating node” for a certain URL is made for a given URL is important. Rating nodes can be at the hostname lever, the domain, the directory, or the page. hostname.domain.com/directory/page.html. The RuleSpace Backend Systems support resolution of ratings at each of these nodes.
2] When URLs are removed from the zone file on the Top Level Domain (TLD) registrars for over 6 months, they are removed from the RuleSpace database. Many other companies keep these URLs in their lists for years to prop up the size of their database even though these sites no longer exist. When the domains do come back online, RuleSpace automatically categorizes the site as soon as one of our customers browses to it or when our internal scans re-discover it. .
3] Humans make value judgments. They get tired. They make mistakes. Our experience has shown that when judging a website for one category and language pair, e.g. Spanish: Gambling, a human reviewer is approximately 98.7% accurate over the course of a day. When given more than one category and language pair to judge, the accuracy of human judgments drop quickly to 85% accuracy with just three pairs. The RuleSpace Backend Systems are trained to be over 99.5% accurate on each pair, and do not decrease in accuracy when given multiple pair definitions. They also do not get tired.
