• Mahmood Ali's avatar
    dev: avoid codecgen code in downstream projects · 3a517614
    Mahmood Ali authored
    This is an attempt to ease dependency management for external driver
    plugins, by avoiding requiring them to compile ugorji/go generated
    files.  Plugin developers reported some pain with the brittleness of
    ugorji/go dependency in particular, specially when using go mod, the
    default go mod manager in golang 1.13.
    
    Context
    --------
    
    Nomad uses msgpack to persist and serialize internal structs, using
    ugorji/go library.  As an optimization, we use ugorji/go code generation
    to speedup process and aovid the relection-based slow path.
    
    We commit these generated files in repository when we cut and tag the
    release to ease reproducability and debugging old releases.  Thus,
    downstream projects that depend on release tag, indirectly depends on
    ugorji/go generated code.
    
    Sadly, the generated code is brittle and specific to the version of
    ugorji/go being used.  When go mod picks another version of ugorji/go
    then nomad (go mod by default uses release according to semver),
    downstream projects face compilation errors.
    
    Interestingly, downstream projects don't commonly serialize nomad
    internal structs.  Drivers and device plugins use grpc instead of
    msgpack for the most part.  In the few cases where they use msgpag (e.g.
    decoding task config), they do without codegen path as they run on
    driver specific structs not the nomad internal structs.  Also, the
    ugorji/go serialization through reflection is generally backward
    compatible (mod some ugorji/go regression bugs that get introduced every
    now and then :( ).
    
    Proposal
    ---------
    
    The proposal here is to keep committing ugorji/go codec generated files
    for releases but to use a go tag for them.
    
    All nomad development through the makefile, including releasing, CI and
    dev flow, has the tag enabled.
    
    Downstream plugin projects, by default, will skip these files and life
    proceed as normal for them.
    
    The downside is that nomad developers who use generated code but avoid
    using make must start passing additional go tag argument.  Though this
    is not a blessed configuration.
    3a517614