EdenGuideReference

Pinecone

Pinecone Endpoint Documentation

OverviewCopied!

Pinecone endpoints provide a comprehensive interface for interacting with Pinecone vector databases. These endpoints support the full range of Pinecone operations, including vector search, upsert, delete, and management functions, making it ideal for AI applications requiring vector similarity search.

Key FeaturesCopied!

  • Vector Search: High-performance similarity search for embeddings

  • Vector Management: Complete CRUD operations for vector data

  • Namespace Support: Logical partitioning of your vector database

  • Index Statistics: Monitoring and inspection of your vector indexes

  • Metadata Filtering: Advanced filtering based on vector metadata

API Request StructureCopied!

Pinecone API requests follow this general structure:

{
  "kind": "pinecone",
  "type": "[operation_type]",
  "[additional_parameters]": "[values]"
}

OperationsCopied!

Query

The Query operation performs vector similarity search to find vectors similar to input vectors.

{
  "kind": "pinecone",
  "type": "query",
  "body": "{\"vector\": [0.1, 0.2, 0.3, ...], \"topK\": 10, \"filter\": {\"genre\": \"comedy\"}, \"includeMetadata\": true, \"includeValues\": false, \"namespace\": \"movies\"}"
}

Query Parameters:

  • vector: The query vector (required)

  • topK: Number of results to return (default: 10)

  • filter: Metadata filters to apply

  • includeMetadata: Whether to include metadata in results (default: true)

  • includeValues: Whether to include vector values in results (default: false)

  • namespace: The namespace to query (optional)

Upsert

The Upsert operation inserts or updates vectors in the index.

{
  "kind": "pinecone",
  "type": "upsert",
  "body": "{\"vectors\": [{\"id\": \"vec1\", \"values\": [0.1, 0.2, 0.3, ...], \"metadata\": {\"title\": \"Example Document\", \"category\": \"reference\"}}, ...], \"namespace\": \"documents\"}"
}

Upsert Parameters:

  • vectors: Array of vectors to insert or update (required)

    • id: Unique identifier for the vector (required)

    • values: Vector embeddings (required)

    • metadata: Optional metadata associated with the vector

  • namespace: The namespace to upsert into (optional)

Update

The Update operation updates values or metadata for existing vectors.

{
  "kind": "pinecone",
  "type": "update",
  "body": "{\"id\": \"vec1\", \"values\": [0.1, 0.2, 0.3, ...], \"setMetadata\": {\"status\": \"reviewed\"}, \"namespace\": \"documents\"}"
}

Update Parameters:

  • id: ID of the vector to update (required)

  • values: New vector values (optional)

  • setMetadata: Metadata to set or update (optional)

  • namespace: The namespace containing the vector (optional)

Delete

The Delete operation removes vectors from the index.

{
  "kind": "pinecone",
  "type": "delete",
  "body": "{\"ids\": [\"vec1\", \"vec2\", \"vec3\"], \"deleteAll\": false, \"namespace\": \"documents\", \"filter\": {\"status\": \"archived\"}}"
}

Delete Parameters:

  • ids: Array of vector IDs to delete (optional if using filter or deleteAll)

  • deleteAll: Whether to delete all vectors in the namespace (default: false)

  • namespace: The namespace to delete from (optional)

  • filter: Metadata filter for vectors to delete (optional)

Fetch

The Fetch operation retrieves vectors by their IDs.

{
  "kind": "pinecone",
  "type": "fetch",
  "ids": ["vec1", "vec2", "vec3"],
  "namespace": "documents"
}

Fetch Parameters:

  • ids: Array of vector IDs to fetch (required)

  • namespace: The namespace to fetch from (required, can be null)

List

The List operation lists vector IDs in the index.

{
  "kind": "pinecone",
  "type": "list",
  "prefix": "doc-",
  "namespace": "documents",
  "limit": "100",
  "pagination_token": null
}

List Parameters:

  • prefix: Filter results to IDs with this prefix (optional)

  • namespace: The namespace to list from (required, can be null)

  • limit: Maximum number of IDs to return (optional)

  • pagination_token: Token for paginated results (optional)

Describe Index Stats

The Describe Index Stats operation retrieves statistics about the index.

{
  "kind": "pinecone",
  "type": "describe_index_stats"
}

Best PracticesCopied!

Performance Optimization

  1. Vector Dimension:

    • Use consistent vector dimensions within indexes

    • Choose appropriate dimensions to balance accuracy and performance

    • Typical dimensions range from 128 to 1536, depending on the embedding model

  2. Batch Operations:

    • Use batch upsert for inserting multiple vectors (up to 100 per request)

    • Group related operations to minimize API calls

    • Consider chunking large batches into smaller groups

  3. Query Optimization:

    • Use appropriate topK values to limit result sets

    • Apply metadata filters to narrow search space

    • Only request vector values when needed with includeValues

    • Use namespaces to partition data for faster queries

Data Management

  1. Namespaces:

    • Use namespaces to logically partition data

    • Consider namespace limits in your index design

    • Organize by content type, time period, or other logical divisions

  2. Metadata:

    • Design metadata schema to support your filtering needs

    • Use metadata to store contextual information about vectors

    • Be mindful of metadata size limitations

    • Index frequently filtered fields

  3. Vector IDs:

    • Use consistent and meaningful ID schemes

    • Consider prefix strategies for organizing vectors

    • Avoid special characters in IDs

Error Handling

  1. Retry Logic:

    • Implement appropriate retry mechanisms for transient errors

    • Use exponential backoff for retries

    • Set appropriate timeouts

    • Handle rate limiting gracefully

  2. Error Recovery:

    • Implement reconciliation strategies for failed batch operations

    • Log vector operations for auditability and recovery

    • Consider using transactions when possible

Security

  1. Access Control:

    • Manage API keys securely

    • Implement the principle of least privilege

    • Consider data isolation requirements

    • Rotate credentials regularly

  2. Data Validation:

    • Validate vector dimensions before upsert

    • Sanitize metadata inputs

    • Validate query parameters

    • Implement proper error handling

Monitoring and Maintenance

  1. Index Statistics:

    • Regularly monitor index statistics

    • Track vector counts by namespace

    • Monitor dimension and metadata usage

    • Set up alerts for abnormal patterns

  2. Resource Management:

    • Plan for index scaling requirements

    • Monitor query latency and throughput

    • Implement appropriate alerting for index health

    • Schedule maintenance during low-traffic periods

Common Use CasesCopied!

  • Semantic search applications

  • Recommendation systems

  • Image similarity search

  • Duplicate detection

  • Anomaly detection

  • Natural language processing applications

  • AI-powered chatbots and question-answering systems

Related DocumentationCopied!