About our crawler

MegaMETA Supplier Survey — about our crawler

MegaMETA, MB operates an automated crawler that visits manufacturing company websites to build an internal directory of potential suppliers. If you found this page, our crawler visited your site.

What it is. User-Agent:
MegaMETA-SupplierSurveyBot/2.0 (+https://megameta.lt/bot; mailto:crawler@megameta.lt)

What it does. It reads publicly available pages — typically your home page, company profile, capabilities or machine list, quality and certification pages, and contact page — to understand what you manufacture. It reads at most 12 pages per company. It does not submit forms, does not attempt to log in, does not follow links behind authentication, and does not collect anything a visitor could not read.

How it behaves. It obeys robots.txt, including Crawl-delay. It waits at least two seconds between requests to the same site and never opens more than one connection to you at a time. If your server returns 429 it slows down and comes back later; if it returns 403 it stops and does not retry that day.

Why. MegaMETA is a contract-manufacturing purchaser. We are looking for machining, casting, forging, sheet-metal, welding and gear suppliers. Being in our directory means we may contact you with a request for quotation. It carries no obligation and costs nothing.

How to stop it. Add to your robots.txt:

User-agent: MegaMETA-SupplierSurveyBot
Disallow: /

We honour this on the next visit. Or write to crawler@megameta.lt and we will remove your company and block future visits.

Your data. We store the public text of the pages we read and a summary of your manufacturing capabilities. Where a page shows a company e-mail address such as info@ or sales@ we store it so we can send an enquiry. To have your company removed, or any personal data erased, write to crawler@megameta.lt — we will delete it and record the domain so it is not visited again. Controller: MegaMETA, MB, Lithuania.