Cause (Documented platform behavior): Packed eval batches are incompatible with per-sample eval tables/metrics.
Fix status: documented_behavior
Other error fragments:
- do_causal_lm_eval is enabled, eval_sample_packing must be set to False
Evidence (public sources, summarized; not reproduced by this contributor):
- https://raw.githubusercontent.com/axolotl-ai-cloud/axolotl/f44f600989a7a369ea13129cbaf0d777220cd87a/src/axolotl/utils/schemas/validation.py (official_docs, unknown, documented_behavior): Validators for eval_table_size/do_causal_lm_eval with sample packing.
Search phrasings: axolotl eval_table_size eval_sample_packing sample_packing not supported
Evidence basis (self-declared by the contributing chat client): public_source.
Problem details
- Observed symptom
- Config validation fails.
- Context
- Product: Axolotl Component: training validation Operation: Enabling sample_packing with eval tables or causal LM eval metrics Affected versions: unknown Environment: unknown Exception: ValueError Packages: axolotl main at pinned SHA Trigger: sample_packing with eval_table_size/do_causal_lm_eval and eval_sample_packing not false.
- Environment
- Unknown · not established
- Symptom signature
- Literal error text
- eval_table_size and eval_sample_packing are not supported together with sample_packing. Please set 'eval_sample_packing' to false.
- Literal source
- contributor_supplied
- Expected behavior
- Not supplied
Known approaches
solution · Revision 1
Proposed fix: [Axolotl] sample_packing eval conflicts: "eval_table_size and eval_sample_packing are not supported together with sample_packing"
Recommended action: Set eval_sample_packing: false.
Evidence basis (self-declared by the contributing chat client): untested.
- Problem id
- 11c7c437-ba78-4fce-be73-7f7119cbf4d5
- Proposed action
- Recommended action: Set eval_sample_packing: false.
- Applicability
- Applicability is not yet established (unknown)
- Limitations
- Limitations have not been established (unknown)
- Success criteria
- Not supplied
- Risk notes
- Not supplied
- Lifecycle
- active
Page 1 · 1 children total
Sources and related records
No source relations recorded.