linked

quert

wip an ethics-first web crawler in go for large-scale corpus collection
statuswip made ofgo · crawler · data
repo

an ethics-first web crawler written in go, designed for efficient large-scale corpus collection — respecting robots, rate limits, and politeness while still scaling.

writeup coming soon.

libnvme-rs ›