* switch logging from raw file format to zstd compressed files
* more zst suffix
* compress bootlog
* remove class RawFile
* Optimize ZstdFileWriter by adding input caching
* use ZSTD_compressStream2
* cleanup
* LOG_COMPRESSION_LEVEL=10
* space
* add zst suffix to LOGS_SIZE_RATE
* use OP prefix for logmessage
* cleanup paths too
* cleanup the paths too
* add hw.py to release
* fix those issues
* fix unittests
* fix unittests
* fix unittests
* do swaglog_ipc properly across all the files
* fix that
* fix swaglog in c++
* review suggestions
old-commit-hash: bfe990b112
* enable encoderd
* correct enable line
* fix loggerd tests
* fix power draw and cpu tests
* correct cpu for encoderd
* fix a bug, video_writer is shared
* fix issue with not recording dcam
* add recording state
* wooo tests pass. encode id keeps counting
* core 3
* loggerd then encoderd
* stop loggerd first
* core 3 always online
* see the camera when we see encoder packet
* encoderd on small core uses 37%
* remove encoder logic from loggerd
* delete unit test that doesn't really make sense anymore
Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: 94b9972eb7
* remove log_name
* log without compression
* fix tests
* remove extension for bootlog
* another test fix
* uploader compresses
* also compress in athena
* only compress qlog
* more generic check in do_upload
* fix bootlog compression
* lower loggerd cpu usage
* dont link against bz2
* set core affinity to little cluster
* handle old files
old-commit-hash: 77a6f3d034
* start v4l encoder
* v4l encoder starts
* start and stop
* fill in proper controls
* it dequeued a buffer
* getting bytes
* it made a video
* it does make files
* getting close
* ahh, so that's how dequeue works
* qcam works (no remuxing)
* remuxing works
* we just need to make shutdown and rollover graceful
* graceful destruction
* switch to polling
* should work now
* fix pc build
* refactors, stop properly
* touchups, remove a copy
* add v4l encoder to release
* inlcude file
* move writing to it's own thread
* fix minor memory leak
* block instead of dropping frames
* add counter, fix tests maybe
* better debugging and test print
* print file path in assert
* format string in test
* no more oversized qlogs
* match qcam
* touchups, remove omx encoder
* remove omx include files
* checked ioctl, better debugging, open by name
* unused import
* move linux includes to third_party/linux/include
* simple encoderd
* full packet
* encoderd should be complete
* lagging print
* updates
* name dq thread
* subset idx
* video file writing works
* debug
* potential bugfix
* rotation works
* iframe
* keep writing support
* ci should pass
* loggerd, not encoderd
* remote encoder code
* support remote encoder
* cereal to master, add encoderd
* header no longer required
* put that back there
* realtime
* lower decoder latency
* don't use queue for VisionIpcBufExtra, disable realtime again
* assert all written
* hmm simpler
* only push to to_write if we are writing
* assert timestamp is right
* use at and remove assert
* revert to queue
Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: 0baa4c3e2a
* trigger rotate in eoncode thread
* rotate in time
* lgtm
* dcam trigger rotate on C3
* check trigger rotate field
* Use >=
* add rotator thread
* set cnt to 0
* test encodeIdx is increasing across segments
* test both segmentId and encodeId
* fix encodeIdx
* no thread needed
* add log for failed to encode
* stricter C3 test, dont check start encodeId on C2 dcam
* only update last_camera_seen_tms when considered for rotate
Co-authored-by: deanlee <deanlee3@gmail.com>
old-commit-hash: a39873872d
* cleanup test first
* wait for other encoder threads
* fixes initial crash
* no print
* unused
* fix up tests
* need that
Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: d4489fbf91
* loggerd:refactor rotate encoder
* change notify_all to notify_one
* finish
* cancelWait
* start front_encoder_thread on need
* rebase master to resolve conflict
* tici qcam works, but
* ifs are ugly
* synced rotate off frame ids
* only start counting frames after modeld alive
* typo
* fix some bugs
* never over rotate
* works
* missing space
* zero frame count tolerance
* lol
* add back omx race protection
* not need that
* not to confuse different locks
* playing with locks is playing with fire
* tici now has qcam
* size is overrated
* make cppcheck happy
* fall back if no camera
* warn
* missing condition
* first is freestyle
* no stream id before connecting
* wait for increment
* first will be imperfect
* wait is futile
* block frames when writing state
* just log write time
* exit loops when exit
* tici qcamera
* fixed
* add atomics
* incl
Co-authored-by: deanlee <deanlee3@gmail.com>
Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: 2b974150c4
* run loggerd tests in jenkins
* fast
* check segs on the fly
* missing import
* wait for dir
* no thread
* relax FILE_SIZE_TOLERANCE
* cleanup
* unused
* set ci flag
* wait longer for first seg
* fix race condition with setting RecordFront
Co-authored-by: ZwX1616 <zwx1616@gmail.com>
Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: 427d4a5a9e
* should only count after the check
* include no dcam
* no need to copy
* run with both param
* independent idx
* not break test if no device
* remove unused
* EON D is half
* needs test for qcamera too
Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: 6a8cd6e054
* loggerd test
* delete afterwards
* add ccc
* fix pylint
* cleanup rotation test
* this works
* bump tolerance
* clear the data after test
Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: 87473fbc41
Dean Lee
Maxime Desroches
Adeeb Shihadeh
cl0cks4fe
Justin Newberry
George Hotz
Willem Melching
ZwX1616
Greg Hogan