This project is mirrored from https://gitee.com/mirrors/Mizu.git. Pull mirroring failed .
Repository mirroring has been paused due to too many failed attempts. It can be resumed by a project maintainer.
  1. 15 Nov, 2021 2 commits
    • M. Mert Yıldıran's avatar
    • M. Mert Yıldıran's avatar
      Optimize UI entry feed performance (#452) · 618cb3a4
      M. Mert Yıldıran authored
      * Optimize the React code for feeding the entries
      
      By building `EntryItem` only once and updating the `entries` state on meta query messages.
      
      * Upgrade `react-scrollable-feed-virtualized` version from `1.4.3` to `1.4.8`
      
      * Fix the `isSelected` state
      
      * Set the query text before deciding the background to prevent lags while typing
      
      * Upgrade Basenine version from `0.2.6` to `0.2.7`
      
      * Set the query background color only if the query is same after the HTTP request and use `useEffect` instead
      
      * Upgrade Basenine version from `0.2.7` to `0.2.8`
      
      * Use `CancelToken` of `axios` instead of trying to check the query state
      
      * Turn `updateQuery` function into a state hook
      
      * Update the macro for `http`
      
      * Do the `source.cancel()` call in `axios.CancelToken`
      
      * Reduce client-side logging
      618cb3a4
  2. 09 Nov, 2021 1 commit
    • M. Mert Yıldıran's avatar
      Migrate from SQLite to Basenine and introduce a new filtering syntax (#279) · d2fe3f66
      M. Mert Yıldıran authored
      * Fix the OOMKilled error by calling `debug.FreeOSMemory` periodically
      
      * Remove `MAX_NUMBER_OF_GOROUTINES` environment variable
      
      * Change the line
      
      * Increase the default value of `TCP_STREAM_CHANNEL_TIMEOUT_MS` to `10000`
      
      * Write the client and integrate to the new real-time database
      
      * Refactor the WebSocket implementaiton for `/ws`
      
      * Adapt the UI to the new filtering system
      
      * Fix the rest of the issues in the UI
      
      * Increase the buffer of the scanner
      
      * Implement accessing single records
      
      * Increase the buffer of another scanner
      
      * Populate `Request` and `Response` fields of `MizuEntry`
      
      * Add syntax highlighting for the query
      
      * Add database to `Dockerfile`
      
      * Fix some issues
      
      * Update the `realtime_dbms` Git module commit hash
      
      * Upgrade Gin version and print the query string
      
      * Revert "Upgrade Gin version and print the query string"
      
      This reverts commit aa09f904.
      
      * Use WebSocket's itself to query instead of the query string
      
      * Fix some errors related to conversion to HAR
      
      * Fix the issues caused by the latest merge
      
      * Fix the build error
      
      * Fix PR validation GitHub workflow
      
      * Replace the git submodule with latest Basenine version `0.1.0`
      
      Remove `realtime_client.go` and use the official client library `github.com/up9inc/basenine/client/go` instead.
      
      * Move Basenine host and port constants to `shared` module
      
      * Reliably execute and wait for Basenine to become available
      
      * Upgrade Basenine version
      
      * Properly close WebSocket and data channel
      
      * Fix the issues caused by the recent merge commit
      
      * Clean up the TypeScript code
      
      * Update `.gitignore`
      
      * Limit the database size
      
      * Add `Macros` method signature to `Dissector` interface and set the macros provided by the protocol extensions
      
      * Run `go mod tidy` on `agent`
      
      * Upgrade `github.com/up9inc/basenine/client/go` version
      
      * Implement a mechanism to update the query using click events in the UI and use it for protocol macros
      
      * Update the query on click to timestamps
      
      * Fix some issues in the WebSocket and channel handling
      
      * Update the query on clicks to status code
      
      * Update the query on clicks to method, path and service
      
      * Update the query on clicks to is outgoing, source and destination ports
      
      * Add an API endpoint to validate the query against syntax errors
      
      * Move the query background color state into `TrafficPage`
      
      * Fix the logic in `setQuery`
      
      * Display a toast message in case of a syntax error in the query
      
      * Remove a call to `fmt.Printf`
      
      * Upgrade Basenine version to `0.1.3`
      
      * Fix an issue related to getting `MAX_ENTRIES_DB_BYTES` environment variable
      
      * Have the `path` key in request details, in HTTP
      
      * Rearrange the HTTP headers for the querying
      
      * Do the same thing for `cookies` and `queryString`
      
      * Update the query on click to table elements
      
      Add the selectors for `TABLE` type representations in HTTP extension.
      
      * Update the query on click to `bodySize` and `elapsedTime` in `EntryTitle`
      
      * Add the selectors for `TABLE` type representations in AMQP extension
      
      * Add the selectors for `TABLE` type representations in Kafka extension
      
      * Add the selectors for `TABLE` type representations in Redis extension
      
      * Define a struct in `tap/api.go` for the section representation data
      
      * Add the selectors for `BODY` type representations
      
      * Add `request.path` to the HTTP request details
      
      * Change the summary string's field name from `path` to `summary`
      
      * Introduce `queryable` CSS class for queryable UI elements and underline them on hover
      
      * Instead of `N requests` at the bottom, make it `Displaying N results (queried X/Y)` and live update the values
      
      Upgrade Basenine version to `0.2.0`.
      
      * Verify the sha256sum of Basenine executable inside `Dockerfile`
      
      * Pass the start time to web UI through WebSocket and always show the `EntriesList` footer
      
      * Pipe the `stderr` of Basenine as well
      
      * Fix the layout issues related to `CodeEditor` in the UI
      
      * Use the correct `shasum` command in `Dockerfile`
      
      * Upgrade Basenine version to `0.2.1`
      
      * Limit the height of `CodeEditor` container
      
      * Remove `Paused` enum `ConnectionStatus` in UI
      
      * Fix the issue caused by the recent merge
      
      * Add the filtering guide (cheatsheet)
      
      * Update open cheatsheet button's title
      
      * Update cheatsheet content
      
      * Remove the old SQLite code, adapt the `--analyze` related code to Basenine
      
      * Change the method signature of `NewEntry`
      
      * Change the method signature of `Represent`
      
      * Introduce `HTTPPair` field in `MizuEntry` specific to HTTP
      
      * Remove `Entry`, `EntryId` and `EstimatedSizeBytes` fields from `MizuEntry`
      
      Also remove the `getEstimatedEntrySizeBytes` method.
      
      * Remove `gorm.io/gorm` dependency
      
      * Remove unused `sensitiveDataFiltering` folder
      
      * Increase the left margin of open cheatsheet button
      
      * Add `overflow: auto` to the cheatsheet `Modal`
      
      * Fix `GetEntry` method
      
      * Fix the macro for gRPC
      
      * Fix an interface conversion in case of AMQP
      
      * Fix two more interface conversion errors in AMQP
      
      * Make the `syncEntriesImpl` method blocking
      
      * Fix a grammar mistake in the cheatsheet
      
      * Adapt to the changes in the recent merge commit
      
      * Improve the cheatsheet text
      
      * Always display the timestamp in `en-US`
      
      * Upgrade Basenine version to `0.2.2`
      
      * Fix the order of closing Basenine connections and channels
      
      * Don't close the Basenine channels at all
      
      * Upgrade Basenine version to `0.2.3`
      
      * Set the initial filter to `rlimit(100)`
      
      * Make Basenine persistent
      
      * Upgrade Basenine version to `0.2.4`
      
      * Update `debug.Dockerfile`
      
      * Fix a failing test
      
      * Upgrade Basenine version to `0.2.5`
      
      * Revert "Do not show play icon when disconnected (#428)"
      
      This reverts commit 8af2e562.
      
      * Upgrade Basenine version to `0.2.6`
      
      * Make all non-informative things informative
      
      * Make `100` a constant
      
      * Use `===` in JavaScript no matter what
      
      * Remove a forgotten `console.log`
      
      * Add a comment and update the `query` in `syncEntriesImpl`
      
      * Don't call `panic` in `GetEntry`
      
      * Replace `panic` calls in `startBasenineServer` with `logger.Log.Panicf`
      
      * Remove unnecessary `\n` characters in the logs
      d2fe3f66
  3. 07 Nov, 2021 1 commit
  4. 26 Oct, 2021 1 commit
  5. 25 Oct, 2021 1 commit
    • David Levanon's avatar
      Tapper Refactor (#396) · e6968512
      David Levanon authored
      
      * introduce tcp_assembler and tcp_packet_source - the motivation is to … (#380)
      
      * add passive-tapper main tester (#353)
      
      * add passive-tapper main tester
      
      * add errors to go.sum of mizu agent
      
      * disable host mode for tester - to avoid filterAuthorities
      
      * rename main to tester
      
      * build extenssions as part of the tester launch
      
      * add a README to the tester
      
      * solving go.mod and .sum conflicts with addition of go-errors
      
      * trivial warning fixes (#354)
      
      * add passive-tapper main tester
      
      * trivial warning fixes
      
      * add errors to go.sum of mizu agent
      
      * disable host mode for tester - to avoid filterAuthorities
      
      * tcp streams map (#355)
      
      * add passive-tapper main tester
      
      * trivial warning fixes
      
      * add errors to go.sum of mizu agent
      
      * tcp streams map
      
      * disable host mode for tester - to avoid filterAuthorities
      
      * set tcp streams map for tcp stream factory
      
      * change rlog to mizu logger
      
      * errors map (#356)
      
      * add passive-tapper main tester
      
      * trivial warning fixes
      
      * add errors to go.sum of mizu agent
      
      * tcp streams map
      
      * disable host mode for tester - to avoid filterAuthorities
      
      * set tcp streams map for tcp stream factory
      
      * errors map
      
      * change int to uint - errorsmap
      
      * change from int to uint
      
      * Change errorsMap.nErrors to uint.
      
      * change errors map to mizu logger instead of rlog
      
      * init mizu logger in tester + fix errormap declaration
      Co-authored-by: default avatarNimrod Gilboa Markevich <nimrod@up9.com>
      
      * move own ips to tcp stream factory (#358)
      
      * add passive-tapper main tester
      
      * trivial warning fixes
      
      * add errors to go.sum of mizu agent
      
      * tcp streams map
      
      * disable host mode for tester - to avoid filterAuthorities
      
      * set tcp streams map for tcp stream factory
      
      * errors map
      
      * move own ips to tcp stream factory
      
      * Feature/tapper refactor i/move own ips to tcp stream factory (#379)
      
      * add passive-tapper main tester
      
      * trivial warning fixes
      
      * add errors to go.sum of mizu agent
      
      * tcp streams map
      
      * disable host mode for tester - to avoid filterAuthorities
      
      * set tcp streams map for tcp stream factory
      
      * errors map
      
      * move own ips to tcp stream factory
      
      * fix ownips compilation issue
      
      * introduce tcp_assembler and tcp_packet_source - the motivation is to split the actual packet sources from the assembler, so we can have a single thread for the assembly which is separated from packet source threads
      
      * make struts private at this point - planning to move some packages to their own package so we can utilize encapsulation
      
      * move context to tcp_assembly + fix error check of tcp source packet
      
      * use param instead of gloab flag for ipdefrag
      
      * fix a typo
      
      * remove unused pid param
      Co-authored-by: default avatarNimrod Gilboa Markevich <nimrod@up9.com>
      
      * extract stats functions out of the main tapping function (#381)
      
      * add passive-tapper main tester (#353)
      
      * add passive-tapper main tester
      
      * add errors to go.sum of mizu agent
      
      * disable host mode for tester - to avoid filterAuthorities
      
      * rename main to tester
      
      * build extenssions as part of the tester launch
      
      * add a README to the tester
      
      * solving go.mod and .sum conflicts with addition of go-errors
      
      * trivial warning fixes (#354)
      
      * add passive-tapper main tester
      
      * trivial warning fixes
      
      * add errors to go.sum of mizu agent
      
      * disable host mode for tester - to avoid filterAuthorities
      
      * tcp streams map (#355)
      
      * add passive-tapper main tester
      
      * trivial warning fixes
      
      * add errors to go.sum of mizu agent
      
      * tcp streams map
      
      * disable host mode for tester - to avoid filterAuthorities
      
      * set tcp streams map for tcp stream factory
      
      * change rlog to mizu logger
      
      * errors map (#356)
      
      * add passive-tapper main tester
      
      * trivial warning fixes
      
      * add errors to go.sum of mizu agent
      
      * tcp streams map
      
      * disable host mode for tester - to avoid filterAuthorities
      
      * set tcp streams map for tcp stream factory
      
      * errors map
      
      * change int to uint - errorsmap
      
      * change from int to uint
      
      * Change errorsMap.nErrors to uint.
      
      * change errors map to mizu logger instead of rlog
      
      * init mizu logger in tester + fix errormap declaration
      Co-authored-by: default avatarNimrod Gilboa Markevich <nimrod@up9.com>
      
      * move own ips to tcp stream factory (#358)
      
      * add passive-tapper main tester
      
      * trivial warning fixes
      
      * add errors to go.sum of mizu agent
      
      * tcp streams map
      
      * disable host mode for tester - to avoid filterAuthorities
      
      * set tcp streams map for tcp stream factory
      
      * errors map
      
      * move own ips to tcp stream factory
      
      * Feature/tapper refactor i/move own ips to tcp stream factory (#379)
      
      * add passive-tapper main tester
      
      * trivial warning fixes
      
      * add errors to go.sum of mizu agent
      
      * tcp streams map
      
      * disable host mode for tester - to avoid filterAuthorities
      
      * set tcp streams map for tcp stream factory
      
      * errors map
      
      * move own ips to tcp stream factory
      
      * fix ownips compilation issue
      
      * introduce tcp_assembler and tcp_packet_source - the motivation is to split the actual packet sources from the assembler, so we can have a single thread for the assembly which is separated from packet source threads
      
      * make struts private at this point - planning to move some packages to their own package so we can utilize encapsulation
      
      * extract stats functions out of the main tapping function
      
      * move context to tcp_assembly + fix error check of tcp source packet
      
      * use param instead of gloab flag for ipdefrag
      Co-authored-by: default avatarNimrod Gilboa Markevich <nimrod@up9.com>
      
      * Feature/tapper refactor i/internal tapper stats (#384)
      
      * add passive-tapper main tester (#353)
      
      * add passive-tapper main tester
      
      * add errors to go.sum of mizu agent
      
      * disable host mode for tester - to avoid filterAuthorities
      
      * rename main to tester
      
      * build extenssions as part of the tester launch
      
      * add a README to the tester
      
      * solving go.mod and .sum conflicts with addition of go-errors
      
      * trivial warning fixes (#354)
      
      * add passive-tapper main tester
      
      * trivial warning fixes
      
      * add errors to go.sum of mizu agent
      
      * disable host mode for tester - to avoid filterAuthorities
      
      * tcp streams map (#355)
      
      * add passive-tapper main tester
      
      * trivial warning fixes
      
      * add errors to go.sum of mizu agent
      
      * tcp streams map
      
      * disable host mode for tester - to avoid filterAuthorities
      
      * set tcp streams map for tcp stream factory
      
      * change rlog to mizu logger
      
      * errors map (#356)
      
      * add passive-tapper main tester
      
      * trivial warning fixes
      
      * add errors to go.sum of mizu agent
      
      * tcp streams map
      
      * disable host mode for tester - to avoid filterAuthorities
      
      * set tcp streams map for tcp stream factory
      
      * errors map
      
      * change int to uint - errorsmap
      
      * change from int to uint
      
      * Change errorsMap.nErrors to uint.
      
      * change errors map to mizu logger instead of rlog
      
      * init mizu logger in tester + fix errormap declaration
      Co-authored-by: default avatarNimrod Gilboa Markevich <nimrod@up9.com>
      
      * move own ips to tcp stream factory (#358)
      
      * add passive-tapper main tester
      
      * trivial warning fixes
      
      * add errors to go.sum of mizu agent
      
      * tcp streams map
      
      * disable host mode for tester - to avoid filterAuthorities
      
      * set tcp streams map for tcp stream factory
      
      * errors map
      
      * move own ips to tcp stream factory
      
      * Feature/tapper refactor i/move own ips to tcp stream factory (#379)
      
      * add passive-tapper main tester
      
      * trivial warning fixes
      
      * add errors to go.sum of mizu agent
      
      * tcp streams map
      
      * disable host mode for tester - to avoid filterAuthorities
      
      * set tcp streams map for tcp stream factory
      
      * errors map
      
      * move own ips to tcp stream factory
      
      * fix ownips compilation issue
      
      * introduce tcp_assembler and tcp_packet_source - the motivation is to split the actual packet sources from the assembler, so we can have a single thread for the assembly which is separated from packet source threads
      
      * make struts private at this point - planning to move some packages to their own package so we can utilize encapsulation
      
      * extract stats functions out of the main tapping function
      
      * move context to tcp_assembly + fix error check of tcp source packet
      
      * use param instead of gloab flag for ipdefrag
      
      * introduce internal tapper stats
      
      * minor fixes for errors map and internal stats
      Co-authored-by: default avatarNimrod Gilboa Markevich <nimrod@up9.com>
      
      * Feature/tapper refactor i/diagnose package (#386)
      
      * add passive-tapper main tester (#353)
      
      * add passive-tapper main tester
      
      * add errors to go.sum of mizu agent
      
      * disable host mode for tester - to avoid filterAuthorities
      
      * rename main to tester
      
      * build extenssions as part of the tester launch
      
      * add a README to the tester
      
      * solving go.mod and .sum conflicts with addition of go-errors
      
      * trivial warning fixes (#354)
      
      * add passive-tapper main tester
      
      * trivial warning fixes
      
      * add errors to go.sum of mizu agent
      
      * disable host mode for tester - to avoid filterAuthorities
      
      * tcp streams map (#355)
      
      * add passive-tapper main tester
      
      * trivial warning fixes
      
      * add errors to go.sum of mizu agent
      
      * tcp streams map
      
      * disable host mode for tester - to avoid filterAuthorities
      
      * set tcp streams map for tcp stream factory
      
      * change rlog to mizu logger
      
      * errors map (#356)
      
      * add passive-tapper main tester
      
      * trivial warning fixes
      
      * add errors to go.sum of mizu agent
      
      * tcp streams map
      
      * disable host mode for tester - to avoid filterAuthorities
      
      * set tcp streams map for tcp stream factory
      
      * errors map
      
      * change int to uint - errorsmap
      
      * change from int to uint
      
      * Change errorsMap.nErrors to uint.
      
      * change errors map to mizu logger instead of rlog
      
      * init mizu logger in tester + fix errormap declaration
      Co-authored-by: default avatarNimrod Gilboa Markevich <nimrod@up9.com>
      
      * move own ips to tcp stream factory (#358)
      
      * add passive-tapper main tester
      
      * trivial warning fixes
      
      * add errors to go.sum of mizu agent
      
      * tcp streams map
      
      * disable host mode for tester - to avoid filterAuthorities
      
      * set tcp streams map for tcp stream factory
      
      * errors map
      
      * move own ips to tcp stream factory
      
      * Feature/tapper refactor i/move own ips to tcp stream factory (#379)
      
      * add passive-tapper main tester
      
      * trivial warning fixes
      
      * add errors to go.sum of mizu agent
      
      * tcp streams map
      
      * disable host mode for tester - to avoid filterAuthorities
      
      * set tcp streams map for tcp stream factory
      
      * errors map
      
      * move own ips to tcp stream factory
      
      * fix ownips compilation issue
      
      * introduce tcp_assembler and tcp_packet_source - the motivation is to split the actual packet sources from the assembler, so we can have a single thread for the assembly which is separated from packet source threads
      
      * make struts private at this point - planning to move some packages to their own package so we can utilize encapsulation
      
      * extract stats functions out of the main tapping function
      
      * move context to tcp_assembly + fix error check of tcp source packet
      
      * use param instead of gloab flag for ipdefrag
      
      * introduce internal tapper stats
      
      * minor fixes for errors map and internal stats
      
      * move errors map + app stats + internal stats + periodic tasks to diagnose package
      
      * initialize tapper internal stats
      Co-authored-by: default avatarNimrod Gilboa Markevich <nimrod@up9.com>
      
      * move tcp packet source to its packet (#387)
      
      * add passive-tapper main tester (#353)
      
      * add passive-tapper main tester
      
      * add errors to go.sum of mizu agent
      
      * disable host mode for tester - to avoid filterAuthorities
      
      * rename main to tester
      
      * build extenssions as part of the tester launch
      
      * add a README to the tester
      
      * solving go.mod and .sum conflicts with addition of go-errors
      
      * trivial warning fixes (#354)
      
      * add passive-tapper main tester
      
      * trivial warning fixes
      
      * add errors to go.sum of mizu agent
      
      * disable host mode for tester - to avoid filterAuthorities
      
      * tcp streams map (#355)
      
      * add passive-tapper main tester
      
      * trivial warning fixes
      
      * add errors to go.sum of mizu agent
      
      * tcp streams map
      
      * disable host mode for tester - to avoid filterAuthorities
      
      * set tcp streams map for tcp stream factory
      
      * change rlog to mizu logger
      
      * errors map (#356)
      
      * add passive-tapper main tester
      
      * trivial warning fixes
      
      * add errors to go.sum of mizu agent
      
      * tcp streams map
      
      * disable host mode for tester - to avoid filterAuthorities
      
      * set tcp streams map for tcp stream factory
      
      * errors map
      
      * change int to uint - errorsmap
      
      * change from int to uint
      
      * Change errorsMap.nErrors to uint.
      
      * change errors map to mizu logger instead of rlog
      
      * init mizu logger in tester + fix errormap declaration
      Co-authored-by: default avatarNimrod Gilboa Markevich <nimrod@up9.com>
      
      * move own ips to tcp stream factory (#358)
      
      * add passive-tapper main tester
      
      * trivial warning fixes
      
      * add errors to go.sum of mizu agent
      
      * tcp streams map
      
      * disable host mode for tester - to avoid filterAuthorities
      
      * set tcp streams map for tcp stream factory
      
      * errors map
      
      * move own ips to tcp stream factory
      
      * Feature/tapper refactor i/move own ips to tcp stream factory (#379)
      
      * add passive-tapper main tester
      
      * trivial warning fixes
      
      * add errors to go.sum of mizu agent
      
      * tcp streams map
      
      * disable host mode for tester - to avoid filterAuthorities
      
      * set tcp streams map for tcp stream factory
      
      * errors map
      
      * move own ips to tcp stream factory
      
      * fix ownips compilation issue
      
      * introduce tcp_assembler and tcp_packet_source - the motivation is to split the actual packet sources from the assembler, so we can have a single thread for the assembly which is separated from packet source threads
      
      * make struts private at this point - planning to move some packages to their own package so we can utilize encapsulation
      
      * extract stats functions out of the main tapping function
      
      * move context to tcp_assembly + fix error check of tcp source packet
      
      * use param instead of gloab flag for ipdefrag
      
      * introduce internal tapper stats
      
      * minor fixes for errors map and internal stats
      
      * move errors map + app stats + internal stats + periodic tasks to diagnose package
      
      * move tcp packet source to its packet
      
      * initialize tapper internal stats
      Co-authored-by: default avatarNimrod Gilboa Markevich <nimrod@up9.com>
      
      * Fix coding style
      
      * Remove `tap/internal_stats.go`
      
      * make channel between input and assembler blocking - to preserve the same behaviour we have before the refactor
      Co-authored-by: default avatarNimrod Gilboa Markevich <nimrod@up9.com>
      Co-authored-by: default avatarM. Mert Yildiran <mehmet@up9.com>
      e6968512
  6. 20 Oct, 2021 2 commits
    • David Levanon's avatar
      remove main dir (#385) · e2e69a3d
      David Levanon authored
      e2e69a3d
    • David Levanon's avatar
      passive-tapper refactor - first phase · 2944493e
      David Levanon authored
      
      * add passive-tapper main tester (#353)
      
      * add passive-tapper main tester
      
      * add errors to go.sum of mizu agent
      
      * disable host mode for tester - to avoid filterAuthorities
      
      * rename main to tester
      
      * build extenssions as part of the tester launch
      
      * add a README to the tester
      
      * solving go.mod and .sum conflicts with addition of go-errors
      
      * trivial warning fixes (#354)
      
      * add passive-tapper main tester
      
      * trivial warning fixes
      
      * add errors to go.sum of mizu agent
      
      * disable host mode for tester - to avoid filterAuthorities
      
      * tcp streams map (#355)
      
      * add passive-tapper main tester
      
      * trivial warning fixes
      
      * add errors to go.sum of mizu agent
      
      * tcp streams map
      
      * disable host mode for tester - to avoid filterAuthorities
      
      * set tcp streams map for tcp stream factory
      
      * change rlog to mizu logger
      
      * errors map (#356)
      
      * add passive-tapper main tester
      
      * trivial warning fixes
      
      * add errors to go.sum of mizu agent
      
      * tcp streams map
      
      * disable host mode for tester - to avoid filterAuthorities
      
      * set tcp streams map for tcp stream factory
      
      * errors map
      
      * change int to uint - errorsmap
      
      * change from int to uint
      
      * Change errorsMap.nErrors to uint.
      
      * change errors map to mizu logger instead of rlog
      
      * init mizu logger in tester + fix errormap declaration
      Co-authored-by: default avatarNimrod Gilboa Markevich <nimrod@up9.com>
      
      * move own ips to tcp stream factory (#358)
      
      * add passive-tapper main tester
      
      * trivial warning fixes
      
      * add errors to go.sum of mizu agent
      
      * tcp streams map
      
      * disable host mode for tester - to avoid filterAuthorities
      
      * set tcp streams map for tcp stream factory
      
      * errors map
      
      * move own ips to tcp stream factory
      
      * Feature/tapper refactor i/move own ips to tcp stream factory (#379)
      
      * add passive-tapper main tester
      
      * trivial warning fixes
      
      * add errors to go.sum of mizu agent
      
      * tcp streams map
      
      * disable host mode for tester - to avoid filterAuthorities
      
      * set tcp streams map for tcp stream factory
      
      * errors map
      
      * move own ips to tcp stream factory
      
      * fix ownips compilation issue
      Co-authored-by: default avatarNimrod Gilboa Markevich <nimrod@up9.com>
      2944493e
  7. 19 Oct, 2021 5 commits
    • RoyUP9's avatar
      47f2e69b
    • M. Mert Yıldıran's avatar
    • M. Mert Yıldıran's avatar
      Remove `github.com/romana/rlog` dependency completely (#374) · 29ba963c
      M. Mert Yıldıran authored
      * Remove `github.com/romana/rlog` dependency completely
      
      * Comment out all the unnecessary logging in the protocol extensions
      
      * Remove commented out all the unnecessary logging lines
      
      * Remove two more lines related to logging
      29ba963c
    • M. Mert Yıldıran's avatar
      Add OAS contract monitoring support (#325) · 145e7cda
      M. Mert Yıldıran authored
      * Add OAS contract monitoring support
      
      * Pass the contract failure reason to UI
      
      * Fix the issues related to contract validation
      
      * Fix rest of the issues in the UI
      
      * Add documentation related to contract monitoring feature
      
      * Fix a typo in the docs
      
      * Unmarshal to `HTTPRequestResponsePair` only if the OAS validation is enabled
      
      * Fix an issue caused by the merge commit
      
      * Slightly change the logic in the `validateOAS` method
      
      Change the `contractText` value to `No Breaches` or `Breach` and make the text `white-space: nowrap`.
      
      * Retrieve and display the failure reason for both request and response
      
      Also display the content of the contract/OAS file in the UI.
      
      * Display the OAS under `CONTRACT` tab with syntax highlighting
      
      Also fix the styling in the entry feed.
      
      * Remove `EnforcePolicyFileDeprecated` constant
      
      * Log the other errors as well
      
      * Get context from caller instead
      
      * Define a type for the contract status and make it...
      145e7cda
    • M. Mert Yıldıran's avatar
      Set the default log level for Agent to `INFO` and raise it to `DEBUG` if... · b7ff0765
      M. Mert Yıldıran authored
      Set the default log level for Agent to `INFO` and raise it to `DEBUG` if `dump-logs=true` is provided (#373)
      
      * Set the default log level for Agent to `INFO` and change it to `DEBUG` if `dump-logs=true` is provided
      
      * Remove `Trace` method and replace its calls with `Debug`
      
      * Export logging levels from `logger` by defining functions
      
      * Revert "Export logging levels from `logger` by defining functions"
      
      This reverts commit e554e40f4a16f07bf7b1f2ee4a7d2544a5a5a19d.
      
      * Run `go mod tidy` on agent
      
      * Define a method named `determineLogLevel`
      b7ff0765
  8. 18 Oct, 2021 1 commit
  9. 10 Oct, 2021 1 commit
  10. 08 Oct, 2021 1 commit
  11. 06 Oct, 2021 2 commits
  12. 29 Sep, 2021 1 commit
  13. 28 Sep, 2021 1 commit
  14. 25 Sep, 2021 1 commit
  15. 23 Sep, 2021 3 commits
  16. 19 Sep, 2021 2 commits
  17. 18 Sep, 2021 2 commits
  18. 13 Sep, 2021 3 commits
  19. 12 Sep, 2021 1 commit
    • M. Mert Yıldıran's avatar
      Stop the hanging Goroutines by dropping the old, unidentified TCP streams (#260) · 858a6468
      M. Mert Yıldıran authored
      * Close the hanging TCP message channels after a dynamically aligned timeout (base `10000` milliseconds)
      
      * Bring back `source.Lazy`
      
      * Add a one more `sync.Map.Delete` call
      
      * Improve the formula by taking base Goroutine count into account
      
      * Reduce duplication
      
      * Include the dropped TCP streams count into the stats tracker and print a debug log whenever it happens
      
      * Add `superIdentifier` field to `tcpStream` to check if it has identified
      
      Also stop the other protocol dissectors if a TCP stream identified by a protocol.
      
      * Take one step forward in fixing the channel closing issue (WIP)
      
      Add `sync.Mutex` to `tcpReader` and make the loops reference based.
      
      * Fix the channel closing issue
      
      * Improve the accuracy of the formula, log better and multiply `baseStreamChannelTimeoutMs` by 100
      
      * Remove `fmt.Printf`
      
      * Replace `runtime.Gosched()` with `time.Sleep(1 * time.Millisecond)`
      
      * Close the channels of other protocols in case of an identification
      
      * Simplify the logic
      
      * Replace the formula with hard timeout 5000 milliseconds and 4000 maximum number of Goroutines
      858a6468
  20. 09 Sep, 2021 1 commit
    • M. Mert Yıldıran's avatar
      Fix the memory exhaustion by optimizing max. AMQP message size and `GOGC` (#257) · 7cc077c8
      M. Mert Yıldıran authored
      * Permanently resolve the memory exhaustion in AMQP
      
      Introduce;
      - `MEMORY_PROFILING_DUMP_PATH`
      - `MEMORY_PROFILING_TIME_INTERVAL`
      environment variables and make `startMemoryProfiler` method more parameterized.
      
      * Fix a leak in HTTP
      
      * Revert "Fix a leak in HTTP"
      
      This reverts commit 9d46820ff3730bc5ef80f90243a42b12e8f695fb.
      
      * Set maximum AMQP message size to 16MB
      
      * Set `GOGC` to 12800
      
      * Remove some commented out lines and an unnecessary `else if`
      7cc077c8
  21. 07 Sep, 2021 1 commit
  22. 05 Sep, 2021 1 commit
  23. 04 Sep, 2021 1 commit
  24. 02 Sep, 2021 1 commit
    • M. Mert Yıldıran's avatar
      Refactor Mizu, define an extension API and add new protocols: AMQP, Kafka (#224) · 366c1d0c
      M. Mert Yıldıran authored
      * Separate HTTP related code into `extensions/http` as a Go plugin
      
      * Move `extensions` folder into `tap` folder
      
      * Move HTTP files into `tap/extensions/lib` for now
      
      * Replace `orcaman/concurrent-map` with `sync.Map`
      
      * Remove `grpc_assembler.go`
      
      * Remove `github.com/up9inc/mizu/tap/extensions/http/lib`
      
      * Add a build script to automatically build extensions from a known path and load them
      
      * Start to define the extension API
      
      * Implement the `run()` function for the TCP stream
      
      * Add support of defining multiple ports to the extension API
      
      * Set the extension name inside the extension
      
      * Declare the `Dissect` function in the extension API
      
      * Dissect HTTP request from inside the HTTP extension
      
      * Make the distinction of outbound and inbound ports
      
      * Dissect HTTP response from inside the HTTP extension
      
      * Bring back the HTTP request-response pair matcher
      
      * Return a `*api.RequestResponsePair` from the dissection
      
      * Bring back the gRPC-HTTP/2 parser
      
      * Fix the issues in `handleHTTP1ClientStream` and `handleHTTP1ServerStream`
      
      * Call a function pointer to emit dissected data back to the `tap` package
      
      * roee changes -
      trying to fix agent to work with the "api" object) - ***still not working***
      
      * small mistake in the conflicts
      
      * Fix the issues that are introduced by the merge conflict
      
      * Add `Emitter` interface to the API and send `OutputChannelItem`(s) to `OutputChannel`
      
      * Fix the `HTTP1` handlers
      
      * Set `ConnectionInfo` in HTTP handlers
      
      * Fix the `Dockerfile` to build the extensions
      
      * remove some unwanted code
      
      * no message
      
      * Re-enable `getStreamProps` function
      
      * Migrate back from `gopacket/tcpassembly` to `gopacket/reassembly`
      
      * Introduce `HTTPPayload` struct and `HTTPPayloader` interface to `MarshalJSON()` all the data structures that are returned by the HTTP protocol
      
      * Read `socketHarOutChannel` instead of `filteredHarChannel`
      
      * Connect `OutputChannelItem` to the last WebSocket means that finally the web UI started to work again
      
      * Add `.env.example` to React app
      
      * Marshal and unmarshal `*http.Request`, `*http.Response` pairs
      
      * Move `loadExtensions` into `main.go` and map extensions into `extensionsMap`
      
      * Add `Summarize()` method to the `Dissector` interface
      
      * Add `Analyze` method to the `Dissector` interface and `MizuEntry` to the extension API
      
      * Add `Protocol` struct and make it effect the UI
      
      * Refactor `BaseEntryDetails` struct and display the source and destination ports in the UI
      
      * Display the protocol name inside the details layout
      
      * Add `Represent` method to the `Dissector` interface and manipulate the UI through this method
      
      * Make the protocol color affect the details layout color and write protocol abbreviation vertically
      
      * Remove everything HTTP related from the `tap` package and make the extension system fully functional
      
      * Fix the TypeScript warnings
      
      * Bring in the files related AMQP into `amqp` directory
      
      * Add `--nodefrag` flag to the tapper and bring in the main AMQP code
      
      * Implement the AMQP `BasicPublish` and fix some issues in the UI when the response payload is missing
      
      * Implement `representBasicPublish` method
      
      * Fix several minor issues
      
      * Implement the AMQP `BasicDeliver`
      
      * Implement the AMQP `QueueDeclare`
      
      * Implement the AMQP `ExchangeDeclare`
      
      * Implement the AMQP `ConnectionStart`
      
      * Implement the AMQP `ConnectionClose`
      
      * Implement the AMQP `QueueBind`
      
      * Implement the AMQP `BasicConsume`
      
      * Fix an issue in `ConnectionStart`
      
      * Fix a linter error
      
      * Bring in the files related Kafka into `kafka` directory
      
      * Fix the build errors in Kafka Go files
      
      * Implement `Dissect` method of Kafka and adapt request-response pair matcher to asynchronous client-server stream
      
      * Do the "Is reversed?" checked inside `getStreamProps` and fix an issue in Kafka `Dissect` method
      
      * Implement `Analyze`, `Summarize` methods of Kafka
      
      * Implement the representations for Kafka `Metadata`, `RequestHeader` and `ResponseHeader`
      
      * Refactor the AMQP and Kafka implementations to create the summary string only inside the `Analyze` method
      
      * Implement the representations for Kafka `ApiVersions`
      
      * Implement the representations for Kafka `Produce`
      
      * Implement the representations for Kafka `Fetch`
      
      * Implement the representations for Kafka `ListOffsets`, `CreateTopics` and `DeleteTopics`
      
      * Fix the encoding of AMQP `BasicPublish` and `BasicDeliver` body
      
      * Remove the unnecessary logging
      
      * Remove more logging
      
      * Introduce `Version` field to `Protocol` struct for dynamically switching the HTTP protocol to HTTP/2
      
      * Fix the issues in analysis and representation of HTTP/2 (gRPC) protocol
      
      * Fix the issues in summary section of details layout for HTTP/2 (gRPC) protocol
      
      * Fix the read errors that freezes the sniffer in HTTP and Kafka
      
      * Fix the issues in HTTP POST data
      
      * Fix one more issue in HTTP POST data
      
      * Fix an infinite loop in Kafka
      
      * Fix another freezing issue in Kafka
      
      * Revert "UI Infra - Support multiple entry types + refactoring (#211)"
      
      This reverts commit f74a52d4.
      
      * Fix more issues that are introduced by the merge
      
      * Fix the status code in the summary section
      
      * adding the cleaner again (why we removed it?).
      add TODO: on the extension loop .
      
      * fix dockerfile (remove deleting .env file) - it is found in dockerignore and fails to build if the file not exists
      
      * fix GetEntrties ("/entries" endpoint) - working with "tapApi.BaseEntryDetail" (moved from shared)
      
      * Fix an issue in the UI summary section
      
      * Refactor the protocol payload structs
      
      * Fix a log message in the passive tapper
      
      * Adapt `APP_PORTS` environment variable to the new extension system and change its format to `APP_PORTS='{"http": ["8001"]}' `
      
      * Revert "fix dockerfile (remove deleting .env file) - it is found in dockerignore and fails to build if the file not exists"
      
      This reverts commit 4f514ae1.
      
      * Bring in the necessary changes from f74a52d4
      
      * Open the API server URL in the web browser as soon as Mizu is ready
      
      * Make the TCP reader consists of a single Go routine (instead of two) and try to dissect in both client and server mode by rewinding
      
      * Swap `TcpID` without overwriting it
      
      * Sort extension by priority
      
      * Try to dissect with looping through all the extensions
      
      * fix getStreamProps function.
      (it should be passed from CLI as it was before).
      
      * Turn TCP reader back into two Goroutines (client and server)
      
      * typo
      
      * Learn `isClient` from the TCP stream
      
      * Set `viewer` style `overflow: "auto"`
      
      * Fix the memory leaks in AMQP and Kafka dissectors
      
      * Revert some of the changes in be7c65eb
      
      * Remove `allExtensionPorts` since it's no longer needed
      
      * Remove `APP_PORTS` since it's no longer needed
      
      * Fix all of the minor issues in the React code
      
      * Check Kafka header size and fail-fast
      
      * Break the dissectors loop upon a successful dissection
      
      * Don't break the dissector loop. Protocols might collide
      
      * Improve the HTTP request-response counter (still not perfect)
      
      * Make the HTTP request-response counter perfect
      
      * Revert "Revert some of the changes in be7c65eb"
      
      This reverts commit 08e7d786.
      
      * Bring back `filterItems` and `isHealthCheckByUserAgent` functions
      
      * Remove some development artifacts
      
      * remove unused and commented lines that are not relevant
      
      * Fix the performance in TCP stream factory. Make it create two `tcpReader`(s) per extension
      
      * Change a log to debug
      
      * Make `*api.CounterPair` a field of `tcpReader`
      
      * Set `isTapTarget` to always `true` again since `filterAuthorities` implementation has problems
      
      * Remove a variable that's only used for logging even though not introduced by this branch
      
      * Bring back the `NumberOfRules` field of `ApplicableRules` struct
      
      * Remove the unused `NewEntry` function
      
      * Move `k8sResolver == nil` check to a more appropriate place
      
      * default healthChecksUserAgentHeaders should be empty array (like the default config value)
      
      * remove spam console.log
      
      * Rules button cause app to crash (access the service via incorrect property)
      
      * Ignore all .env* files in docker build.
      
      * Better caching in dockerfile: only copy go.mod before go mod download.
      
      * Check for errors while loading an extension
      
      * Add a comment about why `Protocol` is not a pointer
      
      * Bring back the call to `deleteOlderThan`
      
      * Remove the `nil` check
      
      * Reduce the maximum allowed AMQP message from 128MB to 1MB
      
      * Fix an error that only occurs when a Kafka broker is initiating
      
      * Revert the change in b2abd7b9
      
      
      
      * Fix the service name resolution in all protocols
      
      * Remove the `anydirection` flag and fix the issue in `filterAuthorities`
      
      * Pass `sync.Map` by reference to `deleteOlderThan` method
      
      * Fix the packet capture issue in standalone mode that's introduced by the removal of `anydirection`
      
      * Temporarily resolve the memory exhaustion in AMQP
      
      * Fix a nil pointer dereference error
      
      * Fix the CLI build error
      
      * Fix a memory leak that's identified by `pprof`
      Co-authored-by: default avatarRoee Gadot <roee.gadot@up9.com>
      Co-authored-by: default avatarNimrod Gilboa Markevich <nimrod@up9.com>
      366c1d0c
  25. 19 Aug, 2021 1 commit
  26. 16 Aug, 2021 1 commit
  27. 08 Aug, 2021 1 commit