* switch logging from raw file format to zstd compressed files
* more zst suffix
* compress bootlog
* remove class RawFile
* Optimize ZstdFileWriter by adding input caching
* use ZSTD_compressStream2
* cleanup
* LOG_COMPRESSION_LEVEL=10
* space
* add zst suffix to LOGS_SIZE_RATE
* enable encoderd
* correct enable line
* fix loggerd tests
* fix power draw and cpu tests
* correct cpu for encoderd
* fix a bug, video_writer is shared
* fix issue with not recording dcam
* add recording state
* wooo tests pass. encode id keeps counting
* core 3
* loggerd then encoderd
* stop loggerd first
* core 3 always online
* see the camera when we see encoder packet
* encoderd on small core uses 37%
* remove encoder logic from loggerd
* delete unit test that doesn't really make sense anymore
Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: 94b9972eb7
* don't use the codec in video_writer
* this produces broken videos for some reason
* bugfix
* refactor on the class
* works on device
* fix codec
* no codec enum
* fix pc
* move into dirs
* these includes also
* rename it ffmpegencoder
* add avcodec_close
Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: 2008db47c2
* remove log_name
* log without compression
* fix tests
* remove extension for bootlog
* another test fix
* uploader compresses
* also compress in athena
* only compress qlog
* more generic check in do_upload
* fix bootlog compression
* lower loggerd cpu usage
* dont link against bz2
* set core affinity to little cluster
* handle old files
old-commit-hash: 77a6f3d034
* start v4l encoder
* v4l encoder starts
* start and stop
* fill in proper controls
* it dequeued a buffer
* getting bytes
* it made a video
* it does make files
* getting close
* ahh, so that's how dequeue works
* qcam works (no remuxing)
* remuxing works
* we just need to make shutdown and rollover graceful
* graceful destruction
* switch to polling
* should work now
* fix pc build
* refactors, stop properly
* touchups, remove a copy
* add v4l encoder to release
* inlcude file
* move writing to it's own thread
* fix minor memory leak
* block instead of dropping frames
* add counter, fix tests maybe
* better debugging and test print
* print file path in assert
* format string in test
* no more oversized qlogs
* match qcam
* touchups, remove omx encoder
* remove omx include files
* checked ioctl, better debugging, open by name
* unused import
* move linux includes to third_party/linux/include
* simple encoderd
* full packet
* encoderd should be complete
* lagging print
* updates
* name dq thread
* subset idx
* video file writing works
* debug
* potential bugfix
* rotation works
* iframe
* keep writing support
* ci should pass
* loggerd, not encoderd
* remote encoder code
* support remote encoder
* cereal to master, add encoderd
* header no longer required
* put that back there
* realtime
* lower decoder latency
* don't use queue for VisionIpcBufExtra, disable realtime again
* assert all written
* hmm simpler
* only push to to_write if we are writing
* assert timestamp is right
* use at and remove assert
* revert to queue
Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: 0baa4c3e2a
* seperate writer from encoder
* video writer in release
* confirm loggerd on PC works, add YUV option to compressed_vipc
* make raw_logger use video_writer
* put this back to master
* close codec
* put this back
* use unique_ptr
Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: 8d24655787
* no-cache mode
* fix test cases build error
* space
* don't create cache dir in no-cache mode
* fix errors in test cases
* no_local_cache_
* set the number of connections by chunk_size
* use size_t instead of int64_t
* add test case for no-cache mode
* rename variables
* fix SIGSEGV
* cleanup
* faster decompressBZ2
* always decompress bz2
* add test cases
* prepare for python interface
* fix test cases build error
* continue
* camera_replay: cache remote file
* protected inheritance
* single option name
* TODO
* test_case for LogReader&FrameReader
* fix wrong require
* test case for FileReader
* cleanup test
* test:fix wrong filename
* check cached file's checksum
* fix mkdir permissions err
cleanup filereader
* remove initialize libav network libraries.
dd
* abort all loading if one failed
* cleanup tests
* use threadpool to limit concurrent downloads
* cache more segments
* merge 3 segments for replay
* one segment uses about 100M of memory
* use segments_need_merge.size()
* shutdown
* fix stuck if exit replay before keyboard thread started
* load one segment at a time
* small cleanup
* cleanup filereader
* space
* tiny cleanup
* merge master
* cleanup test cases
* use util:create_directories
* cleanup framereader
old-commit-hash: 2b4a477fbc
* test multiple threads writing to log
* fix typo
* use util::getenv
* check INIT_DATA and SENTINEL
* test END_OF_SEGMENT
* add comment
* test multiple threads logging and rotation
* cleanup
* update
* LoggerHandle:quick fix
* cleanup
* revert test_loggerd.py
* refactor test case
* check lock file
* check refcnt after close
* test_runner.cc
d
* int eixt_signal
old-commit-hash: 935cbd3139
* only link as needed
* a few more
* need to fix order
* everywhre
* fix mac
* update spinner + text
* fix c2 build
* another fix
Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: 36d8c89ebf
* Squashed vipc
* Update release files
* Remove else
* add visionipc to release files
* use poller in vipc receive
* opencl framework instead of lib on macos
* Fix camera webcam
* Fix opencl on mac in ui
* more webcam fixes
* typo in ui sconsfile
* Use cur_yuv_buf
* visionbuf c++ class
* Camera qcom was still using visionbuf_allocate
* Turn loggerd back on
* fix snapshot
* No build needed
* update test camerad
* no more release callback
* make encoder c++
* Revert "no more release callback"
This reverts commit e5707b07002fee665d0483d90713154efc2d70d4.
* fix exit handlers
* No need to check errno
* move release callback call
* s/VIPCBufExtra/VisionIpcBufExtra/g
* use non blocking connect
* ui use non blocking connect
* Lower condition variable wait time
* Snapshot cleanup
* bump cereal
* bump cereal
old-commit-hash: fb496c692a
* add PubMaster & SubMaster
remove 'delete msg'
remove headers
* use constructor to initial all submster
* modify drain sockets
* fix typo in ssconscript.remove lines
no checkValid in loggerd
last modify
handle_message:event->&event
fix type
remove heads
event to auto
* new interface
* api changed
* Revert "use constructor to initial all submster"
This reverts commit 73be7ea46250a325ce41d3a0445e34395a2ae692.
* change to new api
* revert loggerd
* dd
* use new PubSub api
* update to new interface
don't modify loggerd
reset panda
reset opendbc
remove empty lines
* switch to new pubMaster
* update to the new inteface
change
remove error code
. to ->
merge paramsd.cc
update panda
fix typo
simplify
fix typo
* Fix build
* always conflate
Co-authored-by: deanlee <deanlee3@gmail.com>
old-commit-hash: ab5af232b2
* sentinel WIP
* logger is now a cc, not a c
* simpler code doesn't touch loggerd.cc
* oops, double lock. and logreader is nicer
old-commit-hash: 2f2b85581c
Dean Lee
Harald Schäfer
Adeeb Shihadeh
martinl
George Hotz
Willem Melching
Shane Smiskol
George Hotz
Adeeb
Vehicle Researcher