• Natalie Serrino's avatar
    PP-1928 pt 2: Add two new rules to refactor existing approach to metadata in the compiler. · f41bb865
    Natalie Serrino authored
    Summary:
    Depends on D4868. This is part of a series of changes to make the way the compiler handles metadata work a bit more easily with agent metadata pruning.
    The new flow will remove the _attr columns created by maps, and MetadataIRs will now be considered intermediate nodes that are compiled to a func that generates them.
    
    New analyzer flow:
    1. MetadataIR has its MetadataProperty set in ResolveMetadataPropertyRule
    2. MetadataIR is converted to the func that creates it (upid_to_pod_name(upid) for example) in ConvertMetadataRule. ConvertMetadataRule also sets the metadata_type annotation on the output func.
    3. PropagateExpressionAnnotationsRule (D4868) will propagate the metadata_type annotation in the generated func to all of the places that generated func is renamed or set to
    a column. for example, if that func is used to produce a column, which is then renamed and used as a group by column, the annotations will follow it as long as the column is intact.
    
    Then, consumers of the IR that need to know about metadata type of an expression (such as metadata agent pruning) only have to look at the metadata type annotation. This avoids the problem
    of those consumers needing to check for the various cases that exist today.
    
    Test Plan: added
    
    Reviewers: philkuz, jamesbartlett, zasgar, #engineering
    
    Reviewed By: philkuz, #engineering
    
    JIRA Issues: PP-1928
    
    Differential Revision: https://phab.corp.pixielabs.ai/D4877
    
    GitOrigin-RevId: 67e5ca2faa13e816b39f18d668e9cbd1766d893a
    f41bb865