Frigate gives you a lot of fine tune control on the detection, so with aggressive filtering you could have very limited object detection (for cases when somebody very close to it i.e. trying to peal of the camera).

I would also suggest enabling audio detection if you cameras have mics in them. Audio detection needs a negligible amounts of processing. Fire alarm audio detection should be on by default. You can add things like breaking glass as extra triggers.

Not enabling detection should work for you but to retain video records you would have to setup config globally or for individual cams:

    record:
  enabled: true
  retain:
    days: 10
    mode: all

I would run both or if your cameras can not handle doubling streams just setup a demo in parallels on one of the cams, so you can test it.

I also find https://github.com/bairhys/prometheus-frigate-exporter very useful for longer term performance tuning.