Crawl Links
Enter a URL, include or exclude paths, fetch links.
Find it in ChatBaser
- Agent
- Sources
- Website
- Crawl links
/agents/:agentId/sources/websiteThe chain
From what it does to where it takes you.
-
01
FeatureWhat it does
Aim the crawl at the parts that matter.
-
02
AdvantageWhy it beats the usual way
Skip junk pages.
-
03
BenefitWhat you get
Cleaner knowledge.
-
04
Promised LandWhere it takes you
Your bot quotes your help docs, not your cookie banner.
Try it yourself
Put Crawl Links to work.
The AI agent builder grounded in your approved knowledge.