AgentOven

Data Connectors

Ingest documents from external data sources into your RAG pipeline.

Data Connectors allow AgentOven to ingest documents from external data sources — data warehouses, object stores, and data lakes — into your RAG pipeline. This is an enterprise feature available in AgentOven Pro.

Architecture

Data connectors use a pluggable driver architecture. Each connector implements the DataConnectorDriver interface, which provides connect, read, and health check operations.

go

type DataConnectorDriver interface {
    Kind() string
    Connect(ctx context.Context, config map[string]interface{}) error
    ReadDocuments(ctx context.Context, query string, limit int) ([]models.RawDocument, error)
    HealthCheck(ctx context.Context) error
}
Supported Connectors (Pro)
Snowflake

Connect to Snowflake data warehouses to pull structured and unstructured data for RAG ingestion. Supports SQL queries against your Snowflake tables.

Config: account · warehouse · database · schema · role

Databricks

Read from Databricks Delta Lake tables and Unity Catalog. Supports SQL queries and direct table reads for large-scale data ingestion.

Config: host · token · warehouse_id · catalog · schema

S3 / ADLS / GCS

Read documents directly from cloud object stores. Supports multiple file formats (text, JSON, CSV, PDF) with configurable prefix filtering.

Config: provider (s3|adls|gcs) · bucket · prefix · credentials

Ingestion Flow

Data connectors feed into the existing RAG ingestion pipeline:

  1. Configure a data connector with credentials
  2. Connector reads raw documents from the data source
  3. Documents are passed to the ingestion engine
  4. Text is chunked, embedded, and stored in the vector store
  5. Data is available for RAG queries
API Endpoints
MethodPathDescription
GET/api/v1/connectorsList configured data connectors
POST/api/v1/connectorsRegister a new data connector
POST/api/v1/connectors/{id}/testTest connector connection
POST/api/v1/connectors/{id}/ingestTrigger document ingestion
DELETE/api/v1/connectors/{id}Remove a data connector
OSS vs Pro

The data connector framework (registry, driver interface, API endpoints) ships with OSS. The actual connector drivers (Snowflake, Databricks, S3/ADLS/GCS) are Pro-only and require an AgentOven Pro license.