oxffaa/gopher-parse-sitemap
A high effective golang library for parsing big-sized sitemaps and avoiding high memory usage. The sitemap parser was written on golang without external dependencies.
This library helps Go developers efficiently process large website sitemaps without exhausting system memory. It takes a sitemap file as input and allows developers to access individual URL entries as they are read, rather than loading the entire file into memory at once. It's designed for Go developers building web crawlers, SEO tools, or content management systems that need to handle extensive site structures.
No commits in the last 6 months.
Use this if you are a Go developer building an application that needs to parse very large sitemap files reliably and without high memory consumption.
Not ideal if you prefer a simpler API for small to medium-sized sitemaps, as other Go libraries might offer a more straightforward approach for those cases.
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/perception/oxffaa/gopher-parse-sitemap"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
scrapy/scrapy
Scrapy, a fast high-level web crawling & scraping framework for Python.
Altimis/Scweet
A simple and unlimited twitter scraper : scrape tweets, likes, retweets, following, followers,...
lexiforest/curl_cffi
Python binding for curl-impersonate fork via cffi. A http client that can impersonate browser...
plabayo/rama
modular service framework to move and transform network packets
scrapinghub/spidermon
Scrapy Extension for monitoring spiders execution.