jyoung

13114 Commits

108 Branches

34 Tags

1.6 GiB

Tag: Branch: Tree: bf68eeb596

Commit Graph

Author	SHA1	Message	Date
Shane Smiskol	6a15c42143	selfdrive/car: ban cereal and capnp (#33208 ) * ban cereal and msgq * common too * do toyota/values.py * do all fingerprints * example without builder * this still works, but no type checking anymore * stash * wtf, how does this work * okay actually not bad * safe * epic! * stash data_structures.py * some clean up * hell yeah * clean up old file * add to delete * delete This reverts commit 90239b7797ace31ee647a2fdbd67e0c3faa98dcf. * switch more CarParams stuff over remove unused * fix car tests by removing cereal! mypy forgets about dataclass if we wrap it :( * fix this too * fix this too * remove more cereal and add some good hyundai tests * bunch more typing * override default with 20hz radar * temp capnp converter helper * more lateralTuning * small union replicator is better than what i was trying, and fixes mypy dynamic typing issues * can keep all this the same now! * type ret: CarParams, add more missing structs, revert lateralTuning changes (smaller diff!) * revert more * get first enum automatically, but ofc mypy doesn't pick up the new metaclass so can't use :( would have been `CarParams.NetworkLocation()` * Revert "get first enum automatically, but ofc mypy doesn't pick up the new metaclass so can't use :(" This reverts commit bb28b228becba932052d2fc5a4389784027435b1. * remove cereal from car_helpers (TODO: caching) * remove a bunch of temp lines * use dataclass_transform! * remove some car.CarParams from the interfaces * remove rest of car.CarParams from the interfaces * same which() API * sort * from cereal/cache from fingerprinting! * more typing * dataclass to capnp helper for CarParams, cached it since it's kinda slow * (partial) fix process replay fingerprintig for new API * latcontrollers take capnp * forgot this * fix test_models * fix unit tests * not here * VehicleModel and controller still takes capnp CP since they get it from Params() * fix modeld test * more fix * need to namespace to structs, since CarState is both class and struct * this was never in the base class?! * clean that up again * fix import error fix import error * cmts and more structs * remove some more cereal from toyota + convert CarState to capnp * bruh this was wrong * replace more cereal * EventName is one of the last things... * replace a bunch more cereal.car * missing imports * more * can fix this typing now * proper toyota+others CS typing! * mypy can detect return type of CS.update() now * fix redeclaration of cruise_buttons type * mypy is only complaining about events now * temp fix * add carControl struct * replace CarControl i hope there's no circular imports in hyundai's CC * fine now * lol this was wrong too * fix crash * include my failed attempts at recursively converting to dataclass (doesn't implicitly convert types/recursively :( ) but attrs does, maybe will switch in the future * clean up * try out attr.s for its converter (doesn't work recursively yet, but interesting!) * Revert "try out attr.s for its converter (doesn't work recursively yet, but interesting!)" This reverts commit ff2434f7bbd45a4d4bfb21f7d6712d1f1c3bcde9. * test processes doesn't fail anymore (on toyota)! * fix honda crash * stash * Revert "stash" This reverts commit c1762af4e776790e4ad1322ad4ce0610157346e0. * remove a bunch more cereal! * LET'S GOOO * fix these tests * and these * and that * stash, something is wrong with hyundai enable * Revert "stash, something is wrong with hyundai enable" This reverts commit 39cf327def258e2959fe23cd7a550a858f6d8f03. * forgot these * remove cereal from fw_versions * Revert "remove cereal from fw_versions" This reverts commit 232b37cd409b55d04b1afc90d4a80c49e710eb56. * remove rest of the cereal exceptions! * fix that * add typing to radard since I didn't realize RI.update() switched from cereal to structs * and here too! * add TODO for slots * needed CS to be capnp, fix comparisons, and type hint car_specific so it's easier to catch type issues (capnp isn't detected by mypy :( ) * remove the struct converter * save ~4-5% CPU at 100hz, we don't modify after so no need to deepcopy btw pickle.loads(pickle.dumps()) is faster by ~1% CPU * deepcopy -> copy: we can technically make a reference, but copy is almost free and less error-prone saves ~1% CPU * add non-copying asdict function * should save ~3% CPU (still 4% above baseline) * fix that, no dict support * ~27% decrease in time for 20k iterations on 3X (3.37857 -> 2.4821s) * give a better name * fix * dont support none, capitalize * sheesh, this called type() on every field * remove CS.events, clean up * bump card % * this was a bug on master! * add a which enum * default to pid * revert * update refs * not needed, but consistent * just Ecu * don't need to do this in this pr * clean up * no cast * consistent typing * rm * fix * can do this if we're desperate for the last few % * Revert "can do this if we're desperate for the last few %" This reverts commit 18e11ac7883a0a56583750b1cc5a2b13011e7299. * type this * don't need to convert carControl * i guess don't support set either * fix CP type hint * simplify that	1 year ago
Cameron Clough	85c9fea5d8	Ford: use platform codes to fuzzy fingerprint (#31124 ) * Ford: use platform codes to fuzzy fingerprint TODO: write scripts/tests (print platform codes and version ranges etc.) May close #31052 * get_platform_codes: fix return type * add print_platform_codes.py script * print_platform_codes: sort versions * match_fw_to_car_fuzzy: use set comprehension, and fix typo * Ford: add missing Mach-E fw From the route 83a4e056c7072678/2023-11-13--16-51-33 (which is already in selfdrive/car/tests/routes.py, added in #30691). * add ford_fuzzy_fingerprint.ipynb notebook * get_platform_codes: use regex to parse firmware * test_ford: test_platform_codes_fuzzy_fw * test_ford: use get_platform_codes in test_fw_versions * match_fw_to_car_fuzzy: improve comments * test_ford: add test_platform_codes_spot_check * test_ford: add test_match_fw_fuzzy * remove comment from notebook * TestFordFW: remove engine ECU FW * update print_platform_codes.py * remove part number (unecessary) * platform codes can just use platform hint and model year - software revision not useful * fuzzy FP on the platform hint and model year hint range * fix platform codes test * update notebook * add notebook * explain model year hint better * test part numbers again * cleanup notebooks * remove notebook * cleanup match_fw_to_car_fuzzy and add comments * update comment * . * Revert "remove notebook" This reverts commit 5d4ca202f2a23601d5c829204119f36a58f2b451. * add notebook back * remove PSCM from PLATFORM_CODE_ECUS ABS and IPMA are the best for uniquely matching, and the radar is always required * Revert "remove PSCM from PLATFORM_CODE_ECUS" This reverts commit b7baeac19c18b5aa0c31da52f12054f4bae6e1ff. * fix from merge * more fixes revert * FW_RE -> FW_PATTERN * this can actually be set * conventions * just add * convention * refactor matcher, this brings it more in line with Hyundai. IMPORTANT NOTE: NOTE THAT WE remove the separation for the different platform code model year hint ranges, I don't see that being a problem * better/smaller test * add test to catch overlapping platform codes * remove nb * not now --------- Co-authored-by: Shane Smiskol <shane@smiskol.com>	1 year ago

Author

SHA1

Message

Date

Shane Smiskol

6a15c42143

selfdrive/car: ban cereal and capnp (#33208 )

* ban cereal and msgq

* common too

* do toyota/values.py

* do all fingerprints

* example without builder

* this still works, but no type checking anymore

* stash

* wtf, how does this work

* okay actually not bad

* safe

* epic!

* stash data_structures.py

* some clean up

* hell yeah

* clean up old file

* add to delete

* delete

This reverts commit 90239b7797ace31ee647a2fdbd67e0c3faa98dcf.

* switch more CarParams stuff over

remove unused

* fix car tests by removing cereal! mypy forgets about dataclass if we wrap it :(

* fix this too

* fix this too

* remove more cereal and add some good hyundai tests

* bunch more typing

* override default with 20hz radar

* temp capnp converter helper

* more lateralTuning

* small union replicator is better than what i was trying, and fixes mypy dynamic typing issues

* can keep all this the same now!

* type ret: CarParams, add more missing structs, revert lateralTuning changes (smaller diff!)

* revert more

* get first enum automatically, but ofc mypy doesn't pick up the new metaclass so can't use :(

would have been `CarParams.NetworkLocation()`

* Revert "get first enum automatically, but ofc mypy doesn't pick up the new metaclass so can't use :("

This reverts commit bb28b228becba932052d2fc5a4389784027435b1.

* remove cereal from car_helpers (TODO: caching)

* remove a bunch of temp lines

* use dataclass_transform!

* remove some car.CarParams from the interfaces

* remove rest of car.CarParams from the interfaces

* same which() API

* sort

* from cereal/cache from fingerprinting!

* more typing

* dataclass to capnp helper for CarParams, cached it since it's kinda slow

* (partial) fix process replay fingerprintig for new API

* latcontrollers take capnp

* forgot this

* fix test_models

* fix unit tests

* not here

* VehicleModel and controller still takes capnp CP since they get it from Params()

* fix modeld test

* more fix

* need to namespace to structs, since CarState is both class and struct

* this was never in the base class?!

* clean that up again

* fix import error

fix import error

* cmts and more structs

* remove some more cereal from toyota + convert CarState to capnp

* bruh this was wrong

* replace more cereal

* EventName is one of the last things...

* replace a bunch more cereal.car

* missing imports

* more

* can fix this typing now

* proper toyota+others CS typing!

* mypy can detect return type of CS.update() now

* fix redeclaration of cruise_buttons type

* mypy is only complaining about events now

* temp fix

* add carControl struct

* replace CarControl

i hope there's no circular imports in hyundai's CC

* fine now

* lol this was wrong too

* fix crash

* include my failed attempts at recursively converting to dataclass (doesn't implicitly convert types/recursively :( )

but attrs does, maybe will switch in the future

* clean up

* try out attr.s for its converter (doesn't work recursively yet, but interesting!)

* Revert "try out attr.s for its converter (doesn't work recursively yet, but interesting!)"

This reverts commit ff2434f7bbd45a4d4bfb21f7d6712d1f1c3bcde9.

* test processes doesn't fail anymore (on toyota)!

* fix honda crash

* stash

* Revert "stash"

This reverts commit c1762af4e776790e4ad1322ad4ce0610157346e0.

* remove a bunch more cereal!

* LET'S GOOO

* fix these tests

* and these

* and that

* stash, something is wrong with hyundai enable

* Revert "stash, something is wrong with hyundai enable"

This reverts commit 39cf327def258e2959fe23cd7a550a858f6d8f03.

* forgot these

* remove cereal from fw_versions

* Revert "remove cereal from fw_versions"

This reverts commit 232b37cd409b55d04b1afc90d4a80c49e710eb56.

* remove rest of the cereal exceptions!

* fix that

* add typing to radard since I didn't realize RI.update() switched from cereal to structs

* and here too!

* add TODO for slots

* needed CS to be capnp, fix comparisons, and type hint car_specific so it's easier to catch type issues (capnp isn't detected by mypy :( )

* remove the struct converter

* save ~4-5% CPU at 100hz, we don't modify after so no need to deepcopy

btw pickle.loads(pickle.dumps()) is faster by ~1% CPU

* deepcopy -> copy: we can technically make a reference, but copy is almost free and less error-prone

saves ~1% CPU

* add non-copying asdict function

* should save ~3% CPU (still 4% above baseline)

* fix that, no dict support

* ~27% decrease in time for 20k iterations on 3X (3.37857 -> 2.4821s)

* give a better name

* fix

* dont support none, capitalize

* sheesh, this called type() on every field

* remove CS.events, clean up

* bump card %

* this was a bug on master!

* add a which enum

* default to pid

* revert

* update refs

* not needed, but consistent

* just Ecu

* don't need to do this in this pr

* clean up

* no cast

* consistent typing

* rm

* fix

* can do this if we're desperate for the last few %

* Revert "can do this if we're desperate for the last few %"

This reverts commit 18e11ac7883a0a56583750b1cc5a2b13011e7299.

* type this

* don't need to convert carControl

* i guess don't support set either

* fix CP type hint

* simplify that

1 year ago

Cameron Clough

85c9fea5d8

Ford: use platform codes to fuzzy fingerprint (#31124 )

* Ford: use platform codes to fuzzy fingerprint

TODO: write scripts/tests (print platform codes and version ranges etc.)

May close #31052

* get_platform_codes: fix return type

* add print_platform_codes.py script

* print_platform_codes: sort versions

* match_fw_to_car_fuzzy: use set comprehension, and fix typo

* Ford: add missing Mach-E fw

From the route 83a4e056c7072678/2023-11-13--16-51-33 (which is already
in selfdrive/car/tests/routes.py, added in #30691).

* add ford_fuzzy_fingerprint.ipynb notebook

* get_platform_codes: use regex to parse firmware

* test_ford: test_platform_codes_fuzzy_fw

* test_ford: use get_platform_codes in test_fw_versions

* match_fw_to_car_fuzzy: improve comments

* test_ford: add test_platform_codes_spot_check

* test_ford: add test_match_fw_fuzzy

* remove comment from notebook

* TestFordFW: remove engine ECU FW

* update print_platform_codes.py

* remove part number (unecessary)

* platform codes can just use platform hint and model year - software revision not useful

* fuzzy FP on the platform hint and model year hint range

* fix platform codes test

* update notebook

* add notebook

* explain model year hint better

* test part numbers again

* cleanup notebooks

* remove notebook

* cleanup match_fw_to_car_fuzzy and add comments

* update comment

* .

* Revert "remove notebook"

This reverts commit 5d4ca202f2a23601d5c829204119f36a58f2b451.

* add notebook back

* remove PSCM from PLATFORM_CODE_ECUS

ABS and IPMA are the best for uniquely matching, and the radar is always required

* Revert "remove PSCM from PLATFORM_CODE_ECUS"

This reverts commit b7baeac19c18b5aa0c31da52f12054f4bae6e1ff.

* fix from merge

* more fixes

revert

* FW_RE -> FW_PATTERN

* this can actually be set

* conventions

* just add

* convention

* refactor matcher, this brings it more in line with Hyundai. IMPORTANT NOTE:

NOTE THAT WE remove the separation for the different platform code model year hint ranges, I don't see that being a problem

* better/smaller test

* add test to catch overlapping platform codes

* remove nb

* not now

---------

Co-authored-by: Shane Smiskol <shane@smiskol.com>

1 year ago

2 Commits (bf68eeb596f873f3e6ab497ea42a079c98835f97)