7b67994a05
use zstandard library ( #33110 )
...
* use zstandard
* run 10
* bye bye
* this was a little overzealous
* clean up
old-commit-hash: 1b3b5ab758
9 months ago
539c828861
tools: keep re pattern file small ( #33084 )
...
remove bz2 from re file
old-commit-hash: e68bb26e14
9 months ago
309ccf6482
LogReader: support multiple log extensions in route ( #33066 )
...
* support rlog.zst
* TODO
old-commit-hash: e38a1428aa
9 months ago
5333e8ba20
LogReader: show total segment count in missing rlog prompt
...
old-commit-hash: 3e41b1b287
10 months ago
5887093eed
LogReader: print number of missing rlogs ( #32830 )
...
* count rlogs
* minimization changes
* cache to avoid duplicate checks
* no real point in checking qlogs to just show the warning
---------
Co-authored-by: Shane Smiskol <shane@smiskol.com>
old-commit-hash: 504dd4543d
10 months ago
9f50db15c0
LogReader: try zst on internal source ( #32751 )
...
* internal source list files like azure api
* messy but works
* no limit
* simpler
* clean up
* clean up
* clean up
* that's obvious
* better
* we need to unfortunately return a url, so best to take a naive approach for now
* todo
* fix
* clean up
old-commit-hash: b45caf4033
11 months ago
644156420b
LogReader: revert broken check
...
it returns files OR None
old-commit-hash: 68e22faa29
11 months ago
8f402f29a3
SegmentRange: remove timestamp ( #32750 )
...
* add deprecation warning
* remove timestamp
* Update logreader.py
old-commit-hash: d5ce4b3085
11 months ago
ed80e9357a
LogReader: improve error messages ( #32747 )
...
* better error messages
* clean up
old-commit-hash: 3ede1e2a7c
11 months ago
e88cbb0162
add zstd Python library ( #32731 )
...
* add zstd
* fix
* uploader
* logreader: zst support
* caps
* only logreader
old-commit-hash: 04a93dd849
11 months ago
450d012166
logreader.py: optimize event loading and sorting in _LogFileReader ( #32729 )
...
* improve logfilereader
* less changes
old-commit-hash: c64bca97ba
11 months ago
529699f3fe
logreader: fix auto source + interactive modes ( #31847 )
...
* interactive modes
* these exceptions don't matter
old-commit-hash: a819292073
1 year ago
bc7664aa2b
LogReader: fix sort by time and union types ( #31565 )
...
* fix :(
* test_sort_by_time
* this isn't required
* not slow, and just compare sorted
* messy
* works
* clean up
* clean up
* not here
* clean up
* clean up
* clean up
* makes network call
---------
Co-authored-by: Justin Newberry <justin@comma.ai>
old-commit-hash: e964c5944d
1 year ago
9bd90112d0
use pyupgrade to update to new typing syntax ( #31580 )
...
* add pyupgrade hook
* run pyupgrade (pre-commit run -a)
* ruff --fix
* Revert "add pyupgrade hook"
This reverts commit 56ec18bb6b995250ae49
1 year ago
8dbdd4736c
test_logreader: test internal scenarios ( #31484 )
...
* test source scenario
* test source scenario
* fix
old-commit-hash: 16d1339553
1 year ago
42f292b3ce
test_logreader: test interactive mode + fix typo in AUTO_INTERACTIVE ( #31481 )
...
* fix spelling
* test interactive
* remove that
* test taht
* move that
old-commit-hash: 8aee0d2af2
1 year ago
edd4649d24
LogReader: add typing hints ( #31464 )
...
logreader typing
old-commit-hash: 33cf6bda9e
1 year ago
38fc6f9dd6
LogReader: retain old behavior for direct parsing of files ( #31419 )
...
* maintain exception
* test that head is not called
* annoying mock
* test with cache
old-commit-hash: 1436f576df
1 year ago
5c16ae62d8
SegmentRange: simplify slice ( #31455 )
...
* simplify slicing
* rm
old-commit-hash: 8fe9bc7a69
1 year ago
4c7ad7f3b7
tools/lib: format code ( #31454 )
...
easier to read. pylint used to catch all this stuff, but it's mostly missing in ruff :'(
old-commit-hash: 0846175f44
1 year ago
8b5f1235f4
live_logreader to its own file ( #31436 )
...
own file
old-commit-hash: d6762c3035
1 year ago
fbf5ada69f
LogReader: live_logreader helpers ( #31416 )
...
live helper
old-commit-hash: 0a92c5bf96
1 year ago
1384c10fb0
logreader: skip internal source if not available ( #31400 )
...
* logreader: skip internal source if not available
* raise exception
* but only when appropriate
old-commit-hash: 667693b8c2
1 year ago
9b4b5d3cf5
LogReader: skip file_exists check for comma_api ( #31251 )
...
* valid file
* missed this one
old-commit-hash: 6010a39bf8
1 year ago
16e0907129
LogReader: add ```.filter``` ( #31222 )
...
add filter
old-commit-hash: c782fbc34e
1 year ago
223b1ed065
LogReader: add tqdm back to run_across_segments ( #31216 )
...
tqdm
old-commit-hash: 7835f9cce0
1 year ago
abfa8fcda9
CommaCarSegments: BIG Database! ( #31169 )
...
* sanetized selector
* god damn
* add to re
* slog branch
* slog
* source
* big
* not slog
* cleanup
* no space
* works!
* trigger
* main!
old-commit-hash: 824ddffec8
1 year ago
9edc902106
LogReader: fail-fast on invalid segments ( #31152 )
...
* don't check all
* is
old-commit-hash: edd2428b60
1 year ago
7d76ac8eba
LogReader: replace get_first_message with LogReader.first ( #31146 )
...
* first
* str
old-commit-hash: dd0c474e6c
1 year ago
67bb21219c
LogReader: no redownloading on multiple iterations ( #31141 )
...
* no redownload
* sort
old-commit-hash: 88dcaa51c4
1 year ago
0d558d02ee
auto_source: add commaCarSegments source ( #31137 )
...
add source
old-commit-hash: ac71c30ad5
1 year ago
6a1d7fd732
comma_api_source: fix qlogs ( #31136 )
...
fix qlogs
old-commit-hash: 3d9b483e10
1 year ago
8768be968b
LogReader: don't use route for max_segment ( #31129 )
...
* dont use route
* fix and cleanup
old-commit-hash: d574013d7b
1 year ago
c5c522c984
plotjuggler: auto fallback to qlogs with prompt ( #31110 )
...
* qlog fallback
* show a warning for that
* qlog fallback
* fix
* wording
old-commit-hash: 65490bb3c5
1 year ago
c03fe6bb13
add example of using the comma car segments database ( #31117 )
...
* add vin fingerprinting example
* perhaps!
* remove segment
* easier
old-commit-hash: 389b8ca30d
1 year ago
f447062b66
CommaCarSegments: public database of segments for each platform ( #31114 )
...
* comma car segments
* comma car segments for test_models
* oneliner
Co-authored-by: Shane Smiskol <shane@smiskol.com>
* better name
* not used here
* sort
* remove print
* better comment
---------
Co-authored-by: Shane Smiskol <shane@smiskol.com>
old-commit-hash: caa9153974
1 year ago
b29eea47ce
sanitizer: sanitize vin number ( #31090 )
...
old-commit-hash: 2c86c023fe
1 year ago
a9676e7086
auto_source: optimize api calls and use head to determine if file exists ( #31025 )
...
* fast
* catch all
* source
* fix file_exists
* remove duplicate reset
* test multiple loops
* iterations
* cleanup imports
old-commit-hash: 2967cada71
1 year ago
5df01b3d22
LogReader: fix plotjuggler not reading properly ( #31023 )
...
* fix plotjuggler
* simpler
* fix juggler
* fix that
old-commit-hash: 61fe705160
1 year ago
4b7edec2b0
LogReader: move opci to tools/lib ( #31021 )
...
move opci
old-commit-hash: 7d400112df
1 year ago
35dcd7ccd8
LogReader: support lists of identifiers ( #31020 )
...
* support lists of logs
* mark as slow instead
old-commit-hash: 0739d5d7b2
1 year ago
13e7366b6d
LogReader: support only_union_types ( #31019 )
...
support only-union-types
old-commit-hash: 4da8d16514
1 year ago
c8cc30112d
rename segmentrangereader to logreader ( #30981 )
...
* Replace multilogiterator
* replace logreader
* update readme
* fix from_Bytes
* new section
* reset before iter
* selector
* fix internal
* whitespace
old-commit-hash: 3d73512c07
1 year ago
a729e102c7
Replace multilogiterator ( #30980 )
...
old-commit-hash: 9f1b72ac79
1 year ago
ef1acd4e6c
Proc Replay: cleanup iterable log typing ( #30261 )
...
* cleanup log typing
* use iterable
* fix those
* Cleanup
* cleanup
* missed that one
old-commit-hash: 1820b5f259
2 years ago
16ec56398c
add openpilot prefix to imports ( #29498 )
...
* add openpilot prefix to imports
* more
* more
* fix docs
* fix linter
* bump submodules
* fix patched tests
* update dynamic imports
* debug
* Revert "debug"
This reverts commit db5e13b991a9626f95b6
2 years ago
d487c0501f
Ruff: enable most of bugbear ( #29320 )
...
* added mutable default args
* most of the Bs
* add comment about lrucache
old-commit-hash: 62c1e65924
2 years ago
a16a20005b
process replay: logreader as bytes ( #24610 )
...
* willem's changes
* classmethod for bytes
* submodules
* submodules
* Update tools/lib/logreader.py
Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
* add back files
* little cleanup
Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: 86f73a507e
3 years ago
f957c16a7b
make normal logreader more robust ( #24577 )
...
old-commit-hash: 194e5fdf1c
3 years ago
2d6ae65ea7
Latency logger bad data robustness ( #24343 )
...
* robustness against bad data
* sort by time logreader
* loop until last frameid
old-commit-hash: 1439867d16
3 years ago
Shane Smiskol
StefanLekanic-SYRMIA
Dean Lee
Justin Newberry
Cameron Clough
Greg Hogan
Adeeb Shihadeh
Lukas Petersson
Willem Melching