• James Bartlett's avatar
    Add UDFs/UDAs for k-means, embedding model, and sentencepiece. · 796c78d7
    James Bartlett authored
    Summary:
    Adds a UDF for the transformer model
    Adds a UDF for the sentencepiece model
    Adds a UDA for fitting kmeans
    Adds a UDF for taking fitted kmeans centroids, and mapping to cluster ids.
    
    Test Plan: Added tests, and also tried end2end
    
    Reviewers: #engineering, zasgar
    
    Reviewed By: #engineering, zasgar
    
    Subscribers: philkuz, zasgar
    
    Differential Revision: https://phab.corp.pixielabs.ai/D6295
    
    GitOrigin-RevId: 81c1a0da8b5dbd7051c11caf5caafae5568e70bb
    796c78d7