Semantic search using the Open Crawler and Semantic Text

Learn how to use the Open Crawler in combination with Semantic Text to easily crawl web sites and make them semantically searchable.

New to Elasticsearch? Join our getting started with Elasticsearch webinar. You can also start a free cloud trial or try Elastic on your machine now.

What are we going to do here?

Elastic Open Crawler is the successor to Elastic’s hosted crawler.

Semantic Text is Elastic’s easy on-ramp data type for getting up and running with semantic search.

Together, they are The New(ish) Kids on the Block.

Follow along to learn how to easily configure and run the Open Crawler to crawl a website (Search Labs’ blogs in this case), automatically chunk and generate sparse vector embeddings with ELSER for that web content (blog text), and run a few sample searches to ensure everything worked out.

Choose your own adventure