Skip to content

user_custom.yaml

config/user_custom.yaml holds your personal, uncommitted settings — where your outputs go, which physics model to use, and a handful of options. It is loaded on top of the merged configuration, so it overrides the defaults for your runs without changing anything for anyone else. It is git-ignored: it never gets committed.

A minimal file to get started

# Where outputs go (your EOS / CERNBox user area):
fs_default: davs://eoshome-<initial>.cern.ch:8444/eos/user/<initial>/<user>/FLAF/HH_bbtautau/

# Use the small, fast set of processes while testing:
phys_model: TestModel

# Standard options:
analysis_config_area: config
compute_unc_variations: true
compute_unc_histograms: true
store_noncentral: true

Replace <initial>/<user> with yours (e.g. k / kandroso). With just this, you can run the first-run smoke test.

Fields

Field Type Meaning
fs_default string or list Required. Default storage for all outputs. The fallback for every other fs_*. See Storage.
fs_anaTuple, fs_HistTuple, fs_anaCacheTuple, fs_plots, … string/list Optional per-output-type storage. Unset ⇒ uses fs_default.
phys_model string Which physics model to run: TestModel (small, for testing/CI) or the analysis's production model (e.g. BaseModel).
analysis_config_area string The analysis config directory, relative to the checkout — normally config.
compute_unc_variations bool Whether to compute systematic (up/down) variations during production.
compute_unc_histograms bool Whether to also fill histograms for those variations. Prefer setting this per histTuple flavor in global.yaml (histTuple_flavors.<flavor>.compute_unc_histograms) — uncertainties are usually only needed for the limit-setting shape flavor, so the flavor should dictate it. The value here (or in global.yaml) is used as the fallback when the active flavor does not set it.
store_noncentral bool Whether to keep the non-central (systematic-shift) outputs, not just the central one.
remove_merged_inputs bool If true, HistMergerTask deletes each variable's per-chunk split histograms (HistFromNtupleProducerTask outputs) after merging, to save space, leaving a tiny per-chunk .merged marker in place of each. Safe: the producer stays "complete" for exactly the chunks that were merged (it finds the split or its marker), so the task graph stays consistent (no re-run); a chunk that was never produced has no marker and is still produced. Default false — intermediates are kept.
variables list Restrict which variables are produced/plotted (applied to the active histTuple_flavor list). If that flavor's variable list is empty (e.g. H_mumu default), this list is used as the active set. Omit for the full flavor set.
histTuple_flavor string Optional. Selects which histTuple_flavors entry drives the variable lists (e.g. CI for the short H_mumu CI set).
hist_from_ntuple_max_hists int Max histograms HistFromNtupleProducerTask books in one RDataFrame pass. The count is variables × selections × (Central + every Up/Down). Default 4000; 0 disables batching. Lower this (do not raise CI memory) if a job OOMs.
anaTuple_scheduling map Tunes how AnaTupleFileTask branches are composed into HTCondor jobs. Every key has a default; see below.

anaTuple_scheduling

Optional. Controls the cost-based job composition of AnaTuple production. Sensible defaults apply when the block is absent.

Key Default Meaning
target_job_hours 6.0 Wall time a job is packed up to. A branch that costs more than this is submitted on its own.
max_units_per_job 50 Upper bound on branches per job, whatever their cost.
runtime_safety 2.5 The packing capacity never exceeds max_runtime / runtime_safety, so a packed job cannot be built into the wall clock.
parallel_jobs 2000 Default queue footprint (also what creates the submission waves that let estimates improve mid-run). --parallel-jobs overrides it.
probe_enabled true Whether to run AnaTupleCostProbeTask before production.
probe_events 5000 Events a probe scans per dataset.
overhead_sec 300 Fixed per-job cost (worker setup, JIT, corrections); re-measured from the probes.
default_sec_per_event 0.02 Per-event prior, used only until a dataset has been measured.
default_events_per_byte 4.5e-4 Events-per-byte prior, used when neither an event count nor a measurement is available.
tier_safety see below Divides the packing capacity according to how well the estimate is known: job 1.0, probe/catalogue 1.3, process 2.0, group 3.0, default 4.0.
retry_runtime_factor / retry_memory_factor / retry_max_factor 1.5 / 1.25 / 3.0 Per-attempt escalation of a resubmitted job's runtime and memory, and the cap on both.
request_memory_mb (unset) When set, requests this much memory explicitly instead of letting the site derive it from --n-cpus.

The calibration is stored in data/<version>/AnaTupleCost/cost_model.json and is keyed by version, not era — a version fixes the physics selection by convention, so a cost measured while producing one era is reused by the others. Delete the file to force a full recalibration.

TestModel is the fast path

TestModel selects a reduced set of processes so the pipeline runs quickly end-to-end. Use it for development and local testing; switch to the production model only when you need full results. This is exactly what CI does.

A production-style example

fs_default: davs://eoshome-k.cern.ch:8444/eos/user/k/kandroso/FLAF/HH_bbtautau/
# A separate, roomier site for the big ntuples:
fs_anaTuple: T3_US_FNALLPC:/store/user/lpcflaf/HH_bbtautau/

phys_model: BaseModel
analysis_config_area: config
compute_unc_variations: true
compute_unc_histograms: true
store_noncentral: true

Per-run overrides (--user-custom)

To change settings for a single run without editing your committed file, pass an extra YAML with --user-custom. It is loaded after user_custom.yaml, so its values win:

law run FLAF.Analysis.tasks.HistPlotTask \
  --version my_test --period Run3_2022 --workflow local --branches 0 --test 1000 \
  --user-custom /path/to/extra.yaml

The path may be absolute or relative to $ANALYSIS_PATH. This is the preferred way to run one-off variants (a different model, a different storage area, a short variables: list) — it keeps your user_custom.yaml clean and is reproducible.

The CI uses a dedicated file

The integration pipeline supplies its own ci_custom.yaml (local storage, TestModel, a short variables: list) instead of a personal file, so tests never touch real storage. See Integration pipeline.