open source driving agent
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

29 lines
958 B

Ford: use platform codes to fuzzy fingerprint (#31124) * Ford: use platform codes to fuzzy fingerprint TODO: write scripts/tests (print platform codes and version ranges etc.) May close #31052 * get_platform_codes: fix return type * add print_platform_codes.py script * print_platform_codes: sort versions * match_fw_to_car_fuzzy: use set comprehension, and fix typo * Ford: add missing Mach-E fw From the route 83a4e056c7072678/2023-11-13--16-51-33 (which is already in selfdrive/car/tests/routes.py, added in #30691). * add ford_fuzzy_fingerprint.ipynb notebook * get_platform_codes: use regex to parse firmware * test_ford: test_platform_codes_fuzzy_fw * test_ford: use get_platform_codes in test_fw_versions * match_fw_to_car_fuzzy: improve comments * test_ford: add test_platform_codes_spot_check * test_ford: add test_match_fw_fuzzy * remove comment from notebook * TestFordFW: remove engine ECU FW * update print_platform_codes.py * remove part number (unecessary) * platform codes can just use platform hint and model year - software revision not useful * fuzzy FP on the platform hint and model year hint range * fix platform codes test * update notebook * add notebook * explain model year hint better * test part numbers again * cleanup notebooks * remove notebook * cleanup match_fw_to_car_fuzzy and add comments * update comment * . * Revert "remove notebook" This reverts commit 5d4ca202f2a23601d5c829204119f36a58f2b451. * add notebook back * remove PSCM from PLATFORM_CODE_ECUS ABS and IPMA are the best for uniquely matching, and the radar is always required * Revert "remove PSCM from PLATFORM_CODE_ECUS" This reverts commit b7baeac19c18b5aa0c31da52f12054f4bae6e1ff. * fix from merge * more fixes revert * FW_RE -> FW_PATTERN * this can actually be set * conventions * just add * convention * refactor matcher, this brings it more in line with Hyundai. IMPORTANT NOTE: NOTE THAT WE remove the separation for the different platform code model year hint ranges, I don't see that being a problem * better/smaller test * add test to catch overlapping platform codes * remove nb * not now --------- Co-authored-by: Shane Smiskol <shane@smiskol.com>
1 year ago
#!/usr/bin/env python3
from collections import defaultdict
selfdrive/car: ban cereal and capnp (#33208) * ban cereal and msgq * common too * do toyota/values.py * do all fingerprints * example without builder * this still works, but no type checking anymore * stash * wtf, how does this work * okay actually not bad * safe * epic! * stash data_structures.py * some clean up * hell yeah * clean up old file * add to delete * delete This reverts commit 90239b7797ace31ee647a2fdbd67e0c3faa98dcf. * switch more CarParams stuff over remove unused * fix car tests by removing cereal! mypy forgets about dataclass if we wrap it :( * fix this too * fix this too * remove more cereal and add some good hyundai tests * bunch more typing * override default with 20hz radar * temp capnp converter helper * more lateralTuning * small union replicator is better than what i was trying, and fixes mypy dynamic typing issues * can keep all this the same now! * type ret: CarParams, add more missing structs, revert lateralTuning changes (smaller diff!) * revert more * get first enum automatically, but ofc mypy doesn't pick up the new metaclass so can't use :( would have been `CarParams.NetworkLocation()` * Revert "get first enum automatically, but ofc mypy doesn't pick up the new metaclass so can't use :(" This reverts commit bb28b228becba932052d2fc5a4389784027435b1. * remove cereal from car_helpers (TODO: caching) * remove a bunch of temp lines * use dataclass_transform! * remove some car.CarParams from the interfaces * remove rest of car.CarParams from the interfaces * same which() API * sort * from cereal/cache from fingerprinting! * more typing * dataclass to capnp helper for CarParams, cached it since it's kinda slow * (partial) fix process replay fingerprintig for new API * latcontrollers take capnp * forgot this * fix test_models * fix unit tests * not here * VehicleModel and controller still takes capnp CP since they get it from Params() * fix modeld test * more fix * need to namespace to structs, since CarState is both class and struct * this was never in the base class?! * clean that up again * fix import error fix import error * cmts and more structs * remove some more cereal from toyota + convert CarState to capnp * bruh this was wrong * replace more cereal * EventName is one of the last things... * replace a bunch more cereal.car * missing imports * more * can fix this typing now * proper toyota+others CS typing! * mypy can detect return type of CS.update() now * fix redeclaration of cruise_buttons type * mypy is only complaining about events now * temp fix * add carControl struct * replace CarControl i hope there's no circular imports in hyundai's CC * fine now * lol this was wrong too * fix crash * include my failed attempts at recursively converting to dataclass (doesn't implicitly convert types/recursively :( ) but attrs does, maybe will switch in the future * clean up * try out attr.s for its converter (doesn't work recursively yet, but interesting!) * Revert "try out attr.s for its converter (doesn't work recursively yet, but interesting!)" This reverts commit ff2434f7bbd45a4d4bfb21f7d6712d1f1c3bcde9. * test processes doesn't fail anymore (on toyota)! * fix honda crash * stash * Revert "stash" This reverts commit c1762af4e776790e4ad1322ad4ce0610157346e0. * remove a bunch more cereal! * LET'S GOOO * fix these tests * and these * and that * stash, something is wrong with hyundai enable * Revert "stash, something is wrong with hyundai enable" This reverts commit 39cf327def258e2959fe23cd7a550a858f6d8f03. * forgot these * remove cereal from fw_versions * Revert "remove cereal from fw_versions" This reverts commit 232b37cd409b55d04b1afc90d4a80c49e710eb56. * remove rest of the cereal exceptions! * fix that * add typing to radard since I didn't realize RI.update() switched from cereal to structs * and here too! * add TODO for slots * needed CS to be capnp, fix comparisons, and type hint car_specific so it's easier to catch type issues (capnp isn't detected by mypy :( ) * remove the struct converter * save ~4-5% CPU at 100hz, we don't modify after so no need to deepcopy btw pickle.loads(pickle.dumps()) is faster by ~1% CPU * deepcopy -> copy: we can technically make a reference, but copy is almost free and less error-prone saves ~1% CPU * add non-copying asdict function * should save ~3% CPU (still 4% above baseline) * fix that, no dict support * ~27% decrease in time for 20k iterations on 3X (3.37857 -> 2.4821s) * give a better name * fix * dont support none, capitalize * sheesh, this called type() on every field * remove CS.events, clean up * bump card % * this was a bug on master! * add a which enum * default to pid * revert * update refs * not needed, but consistent * just Ecu * don't need to do this in this pr * clean up * no cast * consistent typing * rm * fix * can do this if we're desperate for the last few % * Revert "can do this if we're desperate for the last few %" This reverts commit 18e11ac7883a0a56583750b1cc5a2b13011e7299. * type this * don't need to convert carControl * i guess don't support set either * fix CP type hint * simplify that
1 year ago
from openpilot.selfdrive.car.structs import CarParams
Ford: use platform codes to fuzzy fingerprint (#31124) * Ford: use platform codes to fuzzy fingerprint TODO: write scripts/tests (print platform codes and version ranges etc.) May close #31052 * get_platform_codes: fix return type * add print_platform_codes.py script * print_platform_codes: sort versions * match_fw_to_car_fuzzy: use set comprehension, and fix typo * Ford: add missing Mach-E fw From the route 83a4e056c7072678/2023-11-13--16-51-33 (which is already in selfdrive/car/tests/routes.py, added in #30691). * add ford_fuzzy_fingerprint.ipynb notebook * get_platform_codes: use regex to parse firmware * test_ford: test_platform_codes_fuzzy_fw * test_ford: use get_platform_codes in test_fw_versions * match_fw_to_car_fuzzy: improve comments * test_ford: add test_platform_codes_spot_check * test_ford: add test_match_fw_fuzzy * remove comment from notebook * TestFordFW: remove engine ECU FW * update print_platform_codes.py * remove part number (unecessary) * platform codes can just use platform hint and model year - software revision not useful * fuzzy FP on the platform hint and model year hint range * fix platform codes test * update notebook * add notebook * explain model year hint better * test part numbers again * cleanup notebooks * remove notebook * cleanup match_fw_to_car_fuzzy and add comments * update comment * . * Revert "remove notebook" This reverts commit 5d4ca202f2a23601d5c829204119f36a58f2b451. * add notebook back * remove PSCM from PLATFORM_CODE_ECUS ABS and IPMA are the best for uniquely matching, and the radar is always required * Revert "remove PSCM from PLATFORM_CODE_ECUS" This reverts commit b7baeac19c18b5aa0c31da52f12054f4bae6e1ff. * fix from merge * more fixes revert * FW_RE -> FW_PATTERN * this can actually be set * conventions * just add * convention * refactor matcher, this brings it more in line with Hyundai. IMPORTANT NOTE: NOTE THAT WE remove the separation for the different platform code model year hint ranges, I don't see that being a problem * better/smaller test * add test to catch overlapping platform codes * remove nb * not now --------- Co-authored-by: Shane Smiskol <shane@smiskol.com>
1 year ago
from openpilot.selfdrive.car.ford.values import get_platform_codes
from openpilot.selfdrive.car.ford.fingerprints import FW_VERSIONS
selfdrive/car: ban cereal and capnp (#33208) * ban cereal and msgq * common too * do toyota/values.py * do all fingerprints * example without builder * this still works, but no type checking anymore * stash * wtf, how does this work * okay actually not bad * safe * epic! * stash data_structures.py * some clean up * hell yeah * clean up old file * add to delete * delete This reverts commit 90239b7797ace31ee647a2fdbd67e0c3faa98dcf. * switch more CarParams stuff over remove unused * fix car tests by removing cereal! mypy forgets about dataclass if we wrap it :( * fix this too * fix this too * remove more cereal and add some good hyundai tests * bunch more typing * override default with 20hz radar * temp capnp converter helper * more lateralTuning * small union replicator is better than what i was trying, and fixes mypy dynamic typing issues * can keep all this the same now! * type ret: CarParams, add more missing structs, revert lateralTuning changes (smaller diff!) * revert more * get first enum automatically, but ofc mypy doesn't pick up the new metaclass so can't use :( would have been `CarParams.NetworkLocation()` * Revert "get first enum automatically, but ofc mypy doesn't pick up the new metaclass so can't use :(" This reverts commit bb28b228becba932052d2fc5a4389784027435b1. * remove cereal from car_helpers (TODO: caching) * remove a bunch of temp lines * use dataclass_transform! * remove some car.CarParams from the interfaces * remove rest of car.CarParams from the interfaces * same which() API * sort * from cereal/cache from fingerprinting! * more typing * dataclass to capnp helper for CarParams, cached it since it's kinda slow * (partial) fix process replay fingerprintig for new API * latcontrollers take capnp * forgot this * fix test_models * fix unit tests * not here * VehicleModel and controller still takes capnp CP since they get it from Params() * fix modeld test * more fix * need to namespace to structs, since CarState is both class and struct * this was never in the base class?! * clean that up again * fix import error fix import error * cmts and more structs * remove some more cereal from toyota + convert CarState to capnp * bruh this was wrong * replace more cereal * EventName is one of the last things... * replace a bunch more cereal.car * missing imports * more * can fix this typing now * proper toyota+others CS typing! * mypy can detect return type of CS.update() now * fix redeclaration of cruise_buttons type * mypy is only complaining about events now * temp fix * add carControl struct * replace CarControl i hope there's no circular imports in hyundai's CC * fine now * lol this was wrong too * fix crash * include my failed attempts at recursively converting to dataclass (doesn't implicitly convert types/recursively :( ) but attrs does, maybe will switch in the future * clean up * try out attr.s for its converter (doesn't work recursively yet, but interesting!) * Revert "try out attr.s for its converter (doesn't work recursively yet, but interesting!)" This reverts commit ff2434f7bbd45a4d4bfb21f7d6712d1f1c3bcde9. * test processes doesn't fail anymore (on toyota)! * fix honda crash * stash * Revert "stash" This reverts commit c1762af4e776790e4ad1322ad4ce0610157346e0. * remove a bunch more cereal! * LET'S GOOO * fix these tests * and these * and that * stash, something is wrong with hyundai enable * Revert "stash, something is wrong with hyundai enable" This reverts commit 39cf327def258e2959fe23cd7a550a858f6d8f03. * forgot these * remove cereal from fw_versions * Revert "remove cereal from fw_versions" This reverts commit 232b37cd409b55d04b1afc90d4a80c49e710eb56. * remove rest of the cereal exceptions! * fix that * add typing to radard since I didn't realize RI.update() switched from cereal to structs * and here too! * add TODO for slots * needed CS to be capnp, fix comparisons, and type hint car_specific so it's easier to catch type issues (capnp isn't detected by mypy :( ) * remove the struct converter * save ~4-5% CPU at 100hz, we don't modify after so no need to deepcopy btw pickle.loads(pickle.dumps()) is faster by ~1% CPU * deepcopy -> copy: we can technically make a reference, but copy is almost free and less error-prone saves ~1% CPU * add non-copying asdict function * should save ~3% CPU (still 4% above baseline) * fix that, no dict support * ~27% decrease in time for 20k iterations on 3X (3.37857 -> 2.4821s) * give a better name * fix * dont support none, capitalize * sheesh, this called type() on every field * remove CS.events, clean up * bump card % * this was a bug on master! * add a which enum * default to pid * revert * update refs * not needed, but consistent * just Ecu * don't need to do this in this pr * clean up * no cast * consistent typing * rm * fix * can do this if we're desperate for the last few % * Revert "can do this if we're desperate for the last few %" This reverts commit 18e11ac7883a0a56583750b1cc5a2b13011e7299. * type this * don't need to convert carControl * i guess don't support set either * fix CP type hint * simplify that
1 year ago
Ecu = CarParams.Ecu
Ford: use platform codes to fuzzy fingerprint (#31124) * Ford: use platform codes to fuzzy fingerprint TODO: write scripts/tests (print platform codes and version ranges etc.) May close #31052 * get_platform_codes: fix return type * add print_platform_codes.py script * print_platform_codes: sort versions * match_fw_to_car_fuzzy: use set comprehension, and fix typo * Ford: add missing Mach-E fw From the route 83a4e056c7072678/2023-11-13--16-51-33 (which is already in selfdrive/car/tests/routes.py, added in #30691). * add ford_fuzzy_fingerprint.ipynb notebook * get_platform_codes: use regex to parse firmware * test_ford: test_platform_codes_fuzzy_fw * test_ford: use get_platform_codes in test_fw_versions * match_fw_to_car_fuzzy: improve comments * test_ford: add test_platform_codes_spot_check * test_ford: add test_match_fw_fuzzy * remove comment from notebook * TestFordFW: remove engine ECU FW * update print_platform_codes.py * remove part number (unecessary) * platform codes can just use platform hint and model year - software revision not useful * fuzzy FP on the platform hint and model year hint range * fix platform codes test * update notebook * add notebook * explain model year hint better * test part numbers again * cleanup notebooks * remove notebook * cleanup match_fw_to_car_fuzzy and add comments * update comment * . * Revert "remove notebook" This reverts commit 5d4ca202f2a23601d5c829204119f36a58f2b451. * add notebook back * remove PSCM from PLATFORM_CODE_ECUS ABS and IPMA are the best for uniquely matching, and the radar is always required * Revert "remove PSCM from PLATFORM_CODE_ECUS" This reverts commit b7baeac19c18b5aa0c31da52f12054f4bae6e1ff. * fix from merge * more fixes revert * FW_RE -> FW_PATTERN * this can actually be set * conventions * just add * convention * refactor matcher, this brings it more in line with Hyundai. IMPORTANT NOTE: NOTE THAT WE remove the separation for the different platform code model year hint ranges, I don't see that being a problem * better/smaller test * add test to catch overlapping platform codes * remove nb * not now --------- Co-authored-by: Shane Smiskol <shane@smiskol.com>
1 year ago
if __name__ == "__main__":
cars_for_code: defaultdict = defaultdict(lambda: defaultdict(set))
for car_model, ecus in FW_VERSIONS.items():
print(car_model)
selfdrive/car: ban cereal and capnp (#33208) * ban cereal and msgq * common too * do toyota/values.py * do all fingerprints * example without builder * this still works, but no type checking anymore * stash * wtf, how does this work * okay actually not bad * safe * epic! * stash data_structures.py * some clean up * hell yeah * clean up old file * add to delete * delete This reverts commit 90239b7797ace31ee647a2fdbd67e0c3faa98dcf. * switch more CarParams stuff over remove unused * fix car tests by removing cereal! mypy forgets about dataclass if we wrap it :( * fix this too * fix this too * remove more cereal and add some good hyundai tests * bunch more typing * override default with 20hz radar * temp capnp converter helper * more lateralTuning * small union replicator is better than what i was trying, and fixes mypy dynamic typing issues * can keep all this the same now! * type ret: CarParams, add more missing structs, revert lateralTuning changes (smaller diff!) * revert more * get first enum automatically, but ofc mypy doesn't pick up the new metaclass so can't use :( would have been `CarParams.NetworkLocation()` * Revert "get first enum automatically, but ofc mypy doesn't pick up the new metaclass so can't use :(" This reverts commit bb28b228becba932052d2fc5a4389784027435b1. * remove cereal from car_helpers (TODO: caching) * remove a bunch of temp lines * use dataclass_transform! * remove some car.CarParams from the interfaces * remove rest of car.CarParams from the interfaces * same which() API * sort * from cereal/cache from fingerprinting! * more typing * dataclass to capnp helper for CarParams, cached it since it's kinda slow * (partial) fix process replay fingerprintig for new API * latcontrollers take capnp * forgot this * fix test_models * fix unit tests * not here * VehicleModel and controller still takes capnp CP since they get it from Params() * fix modeld test * more fix * need to namespace to structs, since CarState is both class and struct * this was never in the base class?! * clean that up again * fix import error fix import error * cmts and more structs * remove some more cereal from toyota + convert CarState to capnp * bruh this was wrong * replace more cereal * EventName is one of the last things... * replace a bunch more cereal.car * missing imports * more * can fix this typing now * proper toyota+others CS typing! * mypy can detect return type of CS.update() now * fix redeclaration of cruise_buttons type * mypy is only complaining about events now * temp fix * add carControl struct * replace CarControl i hope there's no circular imports in hyundai's CC * fine now * lol this was wrong too * fix crash * include my failed attempts at recursively converting to dataclass (doesn't implicitly convert types/recursively :( ) but attrs does, maybe will switch in the future * clean up * try out attr.s for its converter (doesn't work recursively yet, but interesting!) * Revert "try out attr.s for its converter (doesn't work recursively yet, but interesting!)" This reverts commit ff2434f7bbd45a4d4bfb21f7d6712d1f1c3bcde9. * test processes doesn't fail anymore (on toyota)! * fix honda crash * stash * Revert "stash" This reverts commit c1762af4e776790e4ad1322ad4ce0610157346e0. * remove a bunch more cereal! * LET'S GOOO * fix these tests * and these * and that * stash, something is wrong with hyundai enable * Revert "stash, something is wrong with hyundai enable" This reverts commit 39cf327def258e2959fe23cd7a550a858f6d8f03. * forgot these * remove cereal from fw_versions * Revert "remove cereal from fw_versions" This reverts commit 232b37cd409b55d04b1afc90d4a80c49e710eb56. * remove rest of the cereal exceptions! * fix that * add typing to radard since I didn't realize RI.update() switched from cereal to structs * and here too! * add TODO for slots * needed CS to be capnp, fix comparisons, and type hint car_specific so it's easier to catch type issues (capnp isn't detected by mypy :( ) * remove the struct converter * save ~4-5% CPU at 100hz, we don't modify after so no need to deepcopy btw pickle.loads(pickle.dumps()) is faster by ~1% CPU * deepcopy -> copy: we can technically make a reference, but copy is almost free and less error-prone saves ~1% CPU * add non-copying asdict function * should save ~3% CPU (still 4% above baseline) * fix that, no dict support * ~27% decrease in time for 20k iterations on 3X (3.37857 -> 2.4821s) * give a better name * fix * dont support none, capitalize * sheesh, this called type() on every field * remove CS.events, clean up * bump card % * this was a bug on master! * add a which enum * default to pid * revert * update refs * not needed, but consistent * just Ecu * don't need to do this in this pr * clean up * no cast * consistent typing * rm * fix * can do this if we're desperate for the last few % * Revert "can do this if we're desperate for the last few %" This reverts commit 18e11ac7883a0a56583750b1cc5a2b13011e7299. * type this * don't need to convert carControl * i guess don't support set either * fix CP type hint * simplify that
1 year ago
for ecu in sorted(ecus):
Ford: use platform codes to fuzzy fingerprint (#31124) * Ford: use platform codes to fuzzy fingerprint TODO: write scripts/tests (print platform codes and version ranges etc.) May close #31052 * get_platform_codes: fix return type * add print_platform_codes.py script * print_platform_codes: sort versions * match_fw_to_car_fuzzy: use set comprehension, and fix typo * Ford: add missing Mach-E fw From the route 83a4e056c7072678/2023-11-13--16-51-33 (which is already in selfdrive/car/tests/routes.py, added in #30691). * add ford_fuzzy_fingerprint.ipynb notebook * get_platform_codes: use regex to parse firmware * test_ford: test_platform_codes_fuzzy_fw * test_ford: use get_platform_codes in test_fw_versions * match_fw_to_car_fuzzy: improve comments * test_ford: add test_platform_codes_spot_check * test_ford: add test_match_fw_fuzzy * remove comment from notebook * TestFordFW: remove engine ECU FW * update print_platform_codes.py * remove part number (unecessary) * platform codes can just use platform hint and model year - software revision not useful * fuzzy FP on the platform hint and model year hint range * fix platform codes test * update notebook * add notebook * explain model year hint better * test part numbers again * cleanup notebooks * remove notebook * cleanup match_fw_to_car_fuzzy and add comments * update comment * . * Revert "remove notebook" This reverts commit 5d4ca202f2a23601d5c829204119f36a58f2b451. * add notebook back * remove PSCM from PLATFORM_CODE_ECUS ABS and IPMA are the best for uniquely matching, and the radar is always required * Revert "remove PSCM from PLATFORM_CODE_ECUS" This reverts commit b7baeac19c18b5aa0c31da52f12054f4bae6e1ff. * fix from merge * more fixes revert * FW_RE -> FW_PATTERN * this can actually be set * conventions * just add * convention * refactor matcher, this brings it more in line with Hyundai. IMPORTANT NOTE: NOTE THAT WE remove the separation for the different platform code model year hint ranges, I don't see that being a problem * better/smaller test * add test to catch overlapping platform codes * remove nb * not now --------- Co-authored-by: Shane Smiskol <shane@smiskol.com>
1 year ago
platform_codes = get_platform_codes(ecus[ecu])
for code in platform_codes:
cars_for_code[ecu][code].add(car_model)
selfdrive/car: ban cereal and capnp (#33208) * ban cereal and msgq * common too * do toyota/values.py * do all fingerprints * example without builder * this still works, but no type checking anymore * stash * wtf, how does this work * okay actually not bad * safe * epic! * stash data_structures.py * some clean up * hell yeah * clean up old file * add to delete * delete This reverts commit 90239b7797ace31ee647a2fdbd67e0c3faa98dcf. * switch more CarParams stuff over remove unused * fix car tests by removing cereal! mypy forgets about dataclass if we wrap it :( * fix this too * fix this too * remove more cereal and add some good hyundai tests * bunch more typing * override default with 20hz radar * temp capnp converter helper * more lateralTuning * small union replicator is better than what i was trying, and fixes mypy dynamic typing issues * can keep all this the same now! * type ret: CarParams, add more missing structs, revert lateralTuning changes (smaller diff!) * revert more * get first enum automatically, but ofc mypy doesn't pick up the new metaclass so can't use :( would have been `CarParams.NetworkLocation()` * Revert "get first enum automatically, but ofc mypy doesn't pick up the new metaclass so can't use :(" This reverts commit bb28b228becba932052d2fc5a4389784027435b1. * remove cereal from car_helpers (TODO: caching) * remove a bunch of temp lines * use dataclass_transform! * remove some car.CarParams from the interfaces * remove rest of car.CarParams from the interfaces * same which() API * sort * from cereal/cache from fingerprinting! * more typing * dataclass to capnp helper for CarParams, cached it since it's kinda slow * (partial) fix process replay fingerprintig for new API * latcontrollers take capnp * forgot this * fix test_models * fix unit tests * not here * VehicleModel and controller still takes capnp CP since they get it from Params() * fix modeld test * more fix * need to namespace to structs, since CarState is both class and struct * this was never in the base class?! * clean that up again * fix import error fix import error * cmts and more structs * remove some more cereal from toyota + convert CarState to capnp * bruh this was wrong * replace more cereal * EventName is one of the last things... * replace a bunch more cereal.car * missing imports * more * can fix this typing now * proper toyota+others CS typing! * mypy can detect return type of CS.update() now * fix redeclaration of cruise_buttons type * mypy is only complaining about events now * temp fix * add carControl struct * replace CarControl i hope there's no circular imports in hyundai's CC * fine now * lol this was wrong too * fix crash * include my failed attempts at recursively converting to dataclass (doesn't implicitly convert types/recursively :( ) but attrs does, maybe will switch in the future * clean up * try out attr.s for its converter (doesn't work recursively yet, but interesting!) * Revert "try out attr.s for its converter (doesn't work recursively yet, but interesting!)" This reverts commit ff2434f7bbd45a4d4bfb21f7d6712d1f1c3bcde9. * test processes doesn't fail anymore (on toyota)! * fix honda crash * stash * Revert "stash" This reverts commit c1762af4e776790e4ad1322ad4ce0610157346e0. * remove a bunch more cereal! * LET'S GOOO * fix these tests * and these * and that * stash, something is wrong with hyundai enable * Revert "stash, something is wrong with hyundai enable" This reverts commit 39cf327def258e2959fe23cd7a550a858f6d8f03. * forgot these * remove cereal from fw_versions * Revert "remove cereal from fw_versions" This reverts commit 232b37cd409b55d04b1afc90d4a80c49e710eb56. * remove rest of the cereal exceptions! * fix that * add typing to radard since I didn't realize RI.update() switched from cereal to structs * and here too! * add TODO for slots * needed CS to be capnp, fix comparisons, and type hint car_specific so it's easier to catch type issues (capnp isn't detected by mypy :( ) * remove the struct converter * save ~4-5% CPU at 100hz, we don't modify after so no need to deepcopy btw pickle.loads(pickle.dumps()) is faster by ~1% CPU * deepcopy -> copy: we can technically make a reference, but copy is almost free and less error-prone saves ~1% CPU * add non-copying asdict function * should save ~3% CPU (still 4% above baseline) * fix that, no dict support * ~27% decrease in time for 20k iterations on 3X (3.37857 -> 2.4821s) * give a better name * fix * dont support none, capitalize * sheesh, this called type() on every field * remove CS.events, clean up * bump card % * this was a bug on master! * add a which enum * default to pid * revert * update refs * not needed, but consistent * just Ecu * don't need to do this in this pr * clean up * no cast * consistent typing * rm * fix * can do this if we're desperate for the last few % * Revert "can do this if we're desperate for the last few %" This reverts commit 18e11ac7883a0a56583750b1cc5a2b13011e7299. * type this * don't need to convert carControl * i guess don't support set either * fix CP type hint * simplify that
1 year ago
print(f' (Ecu.{ecu[0]}, {hex(ecu[1])}, {ecu[2]}):')
Ford: use platform codes to fuzzy fingerprint (#31124) * Ford: use platform codes to fuzzy fingerprint TODO: write scripts/tests (print platform codes and version ranges etc.) May close #31052 * get_platform_codes: fix return type * add print_platform_codes.py script * print_platform_codes: sort versions * match_fw_to_car_fuzzy: use set comprehension, and fix typo * Ford: add missing Mach-E fw From the route 83a4e056c7072678/2023-11-13--16-51-33 (which is already in selfdrive/car/tests/routes.py, added in #30691). * add ford_fuzzy_fingerprint.ipynb notebook * get_platform_codes: use regex to parse firmware * test_ford: test_platform_codes_fuzzy_fw * test_ford: use get_platform_codes in test_fw_versions * match_fw_to_car_fuzzy: improve comments * test_ford: add test_platform_codes_spot_check * test_ford: add test_match_fw_fuzzy * remove comment from notebook * TestFordFW: remove engine ECU FW * update print_platform_codes.py * remove part number (unecessary) * platform codes can just use platform hint and model year - software revision not useful * fuzzy FP on the platform hint and model year hint range * fix platform codes test * update notebook * add notebook * explain model year hint better * test part numbers again * cleanup notebooks * remove notebook * cleanup match_fw_to_car_fuzzy and add comments * update comment * . * Revert "remove notebook" This reverts commit 5d4ca202f2a23601d5c829204119f36a58f2b451. * add notebook back * remove PSCM from PLATFORM_CODE_ECUS ABS and IPMA are the best for uniquely matching, and the radar is always required * Revert "remove PSCM from PLATFORM_CODE_ECUS" This reverts commit b7baeac19c18b5aa0c31da52f12054f4bae6e1ff. * fix from merge * more fixes revert * FW_RE -> FW_PATTERN * this can actually be set * conventions * just add * convention * refactor matcher, this brings it more in line with Hyundai. IMPORTANT NOTE: NOTE THAT WE remove the separation for the different platform code model year hint ranges, I don't see that being a problem * better/smaller test * add test to catch overlapping platform codes * remove nb * not now --------- Co-authored-by: Shane Smiskol <shane@smiskol.com>
1 year ago
print(f' Codes: {sorted(platform_codes)}')
print()
print('\nCar models vs. platform codes:')
for ecu, codes in cars_for_code.items():
selfdrive/car: ban cereal and capnp (#33208) * ban cereal and msgq * common too * do toyota/values.py * do all fingerprints * example without builder * this still works, but no type checking anymore * stash * wtf, how does this work * okay actually not bad * safe * epic! * stash data_structures.py * some clean up * hell yeah * clean up old file * add to delete * delete This reverts commit 90239b7797ace31ee647a2fdbd67e0c3faa98dcf. * switch more CarParams stuff over remove unused * fix car tests by removing cereal! mypy forgets about dataclass if we wrap it :( * fix this too * fix this too * remove more cereal and add some good hyundai tests * bunch more typing * override default with 20hz radar * temp capnp converter helper * more lateralTuning * small union replicator is better than what i was trying, and fixes mypy dynamic typing issues * can keep all this the same now! * type ret: CarParams, add more missing structs, revert lateralTuning changes (smaller diff!) * revert more * get first enum automatically, but ofc mypy doesn't pick up the new metaclass so can't use :( would have been `CarParams.NetworkLocation()` * Revert "get first enum automatically, but ofc mypy doesn't pick up the new metaclass so can't use :(" This reverts commit bb28b228becba932052d2fc5a4389784027435b1. * remove cereal from car_helpers (TODO: caching) * remove a bunch of temp lines * use dataclass_transform! * remove some car.CarParams from the interfaces * remove rest of car.CarParams from the interfaces * same which() API * sort * from cereal/cache from fingerprinting! * more typing * dataclass to capnp helper for CarParams, cached it since it's kinda slow * (partial) fix process replay fingerprintig for new API * latcontrollers take capnp * forgot this * fix test_models * fix unit tests * not here * VehicleModel and controller still takes capnp CP since they get it from Params() * fix modeld test * more fix * need to namespace to structs, since CarState is both class and struct * this was never in the base class?! * clean that up again * fix import error fix import error * cmts and more structs * remove some more cereal from toyota + convert CarState to capnp * bruh this was wrong * replace more cereal * EventName is one of the last things... * replace a bunch more cereal.car * missing imports * more * can fix this typing now * proper toyota+others CS typing! * mypy can detect return type of CS.update() now * fix redeclaration of cruise_buttons type * mypy is only complaining about events now * temp fix * add carControl struct * replace CarControl i hope there's no circular imports in hyundai's CC * fine now * lol this was wrong too * fix crash * include my failed attempts at recursively converting to dataclass (doesn't implicitly convert types/recursively :( ) but attrs does, maybe will switch in the future * clean up * try out attr.s for its converter (doesn't work recursively yet, but interesting!) * Revert "try out attr.s for its converter (doesn't work recursively yet, but interesting!)" This reverts commit ff2434f7bbd45a4d4bfb21f7d6712d1f1c3bcde9. * test processes doesn't fail anymore (on toyota)! * fix honda crash * stash * Revert "stash" This reverts commit c1762af4e776790e4ad1322ad4ce0610157346e0. * remove a bunch more cereal! * LET'S GOOO * fix these tests * and these * and that * stash, something is wrong with hyundai enable * Revert "stash, something is wrong with hyundai enable" This reverts commit 39cf327def258e2959fe23cd7a550a858f6d8f03. * forgot these * remove cereal from fw_versions * Revert "remove cereal from fw_versions" This reverts commit 232b37cd409b55d04b1afc90d4a80c49e710eb56. * remove rest of the cereal exceptions! * fix that * add typing to radard since I didn't realize RI.update() switched from cereal to structs * and here too! * add TODO for slots * needed CS to be capnp, fix comparisons, and type hint car_specific so it's easier to catch type issues (capnp isn't detected by mypy :( ) * remove the struct converter * save ~4-5% CPU at 100hz, we don't modify after so no need to deepcopy btw pickle.loads(pickle.dumps()) is faster by ~1% CPU * deepcopy -> copy: we can technically make a reference, but copy is almost free and less error-prone saves ~1% CPU * add non-copying asdict function * should save ~3% CPU (still 4% above baseline) * fix that, no dict support * ~27% decrease in time for 20k iterations on 3X (3.37857 -> 2.4821s) * give a better name * fix * dont support none, capitalize * sheesh, this called type() on every field * remove CS.events, clean up * bump card % * this was a bug on master! * add a which enum * default to pid * revert * update refs * not needed, but consistent * just Ecu * don't need to do this in this pr * clean up * no cast * consistent typing * rm * fix * can do this if we're desperate for the last few % * Revert "can do this if we're desperate for the last few %" This reverts commit 18e11ac7883a0a56583750b1cc5a2b13011e7299. * type this * don't need to convert carControl * i guess don't support set either * fix CP type hint * simplify that
1 year ago
print(f' (Ecu.{ecu[0]}, {hex(ecu[1])}, {ecu[2]}):')
Ford: use platform codes to fuzzy fingerprint (#31124) * Ford: use platform codes to fuzzy fingerprint TODO: write scripts/tests (print platform codes and version ranges etc.) May close #31052 * get_platform_codes: fix return type * add print_platform_codes.py script * print_platform_codes: sort versions * match_fw_to_car_fuzzy: use set comprehension, and fix typo * Ford: add missing Mach-E fw From the route 83a4e056c7072678/2023-11-13--16-51-33 (which is already in selfdrive/car/tests/routes.py, added in #30691). * add ford_fuzzy_fingerprint.ipynb notebook * get_platform_codes: use regex to parse firmware * test_ford: test_platform_codes_fuzzy_fw * test_ford: use get_platform_codes in test_fw_versions * match_fw_to_car_fuzzy: improve comments * test_ford: add test_platform_codes_spot_check * test_ford: add test_match_fw_fuzzy * remove comment from notebook * TestFordFW: remove engine ECU FW * update print_platform_codes.py * remove part number (unecessary) * platform codes can just use platform hint and model year - software revision not useful * fuzzy FP on the platform hint and model year hint range * fix platform codes test * update notebook * add notebook * explain model year hint better * test part numbers again * cleanup notebooks * remove notebook * cleanup match_fw_to_car_fuzzy and add comments * update comment * . * Revert "remove notebook" This reverts commit 5d4ca202f2a23601d5c829204119f36a58f2b451. * add notebook back * remove PSCM from PLATFORM_CODE_ECUS ABS and IPMA are the best for uniquely matching, and the radar is always required * Revert "remove PSCM from PLATFORM_CODE_ECUS" This reverts commit b7baeac19c18b5aa0c31da52f12054f4bae6e1ff. * fix from merge * more fixes revert * FW_RE -> FW_PATTERN * this can actually be set * conventions * just add * convention * refactor matcher, this brings it more in line with Hyundai. IMPORTANT NOTE: NOTE THAT WE remove the separation for the different platform code model year hint ranges, I don't see that being a problem * better/smaller test * add test to catch overlapping platform codes * remove nb * not now --------- Co-authored-by: Shane Smiskol <shane@smiskol.com>
1 year ago
for code, cars in codes.items():
print(f' {code!r}: {sorted(map(str, cars))}')