{"id":77127,"plugin_id":"plugins_6ab424ad5b448191b8fede2ebf569333","kind":"skill","collection_source":"plugin_package","comparison_source":null,"observed_at":"2026-11-03T00:12:27.526Z","digest":"fea80d1111189aef5992dbc365955729f1999e3292a37a89be72d09d508101cb","against":null,"payload":{"description":"Diagnose Spark and Databricks performance using actual plans, stage metrics, skew, spill and storage layout evidence.","included_files":[],"name":"tune-spark-databricks","skill_md_contents":"---\nname: tune-spark-databricks\ndescription: Diagnose Spark and Databricks performance using actual plans, stage metrics, skew, spill and storage layout evidence.\n---\n\n\n# Spark / Databricks tuning\n**Trigger:** Slow Spark, sudden runtime regression, skewed joins, excessive shuffle, small files or high compute costs.\n\n1. Compare current and known-good run: dataset size, partition distribution, changed configs/runtime, physical plan and stage statistics.\n2. Locate time spent: scan, shuffle, join, skew, spill, executor/driver pressure, GC, state store, sink or remote I/O.\n3. Inspect final/adaptive physical plan and task-duration percentiles before recommending repartition/broadcast/caching/cluster resize.\n4. Consider stats, predicate pushdown, join grain, file layout and AQE. Platform/runtime defaults vary; check official docs.\n5. Propose one change at a time with expected direction, cost, correctness risk, test scope and rollback.\n6. Validate output equality and representative throughput/cost, not only speed on toy data.\n\nRead `../../references/spark-performance.md` and `../../references/original/spark_databricks_tuning_guide.md`.\n"},"changes":[],"summary":"First saved snapshot. No earlier version is available for comparison.","summary_kind":"deterministic","summary_metadata":{}}