POST
/
api
/
v1
/
collection
/
insert-url
curl --request POST \
  --url https://app.twilix.io/api/v1/collection/insert-url \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '{
  "collection": "<string>",
  "metadata": {}
}'

Insert Url contents into your collection.

This endpoint will automatically scrapes the web content and parse the url into html and store them in the vectorestore.

Some websites have strict anti-scraping mechanisms in place. For those, we recommand using the Insert Html endpoint directly.

collection
string

The collection to insert the data into.

url
The url link to insert.
metadata
object
default: "{}"

Optional metadata when inserting the HTML

Note that this requires an object. For example, an example metadata would be:

{
  "name": 'my_html.html',
  "url": 'https://www.example.com'
}
Content-Type
default: "application/json"required

Requires JSON Content Type

Insert HTMLInsert Bulk Objects