Update to astronomer-data

{
  "name": "profiling-tables",
  "description": "Deep-dive data profiling for a specific table. Use when the user asks to profile a table, wants statistics about a dataset, asks about data quality, or needs to understand a table's structure and content. Requires a table name.",
  "included_files": [],
  "skill_md_contents": "---\nname: profiling-tables\ndescription: Deep-dive data profiling for a specific table. Use when the user asks to profile a table, wants statistics about a dataset, asks about data quality, or needs to understand a table's structure and content. Requires a table name.\n---\n\n# Data Profile\n\nGenerate a comprehensive profile of a table that a new team member could use to understand the data.\n\n## Step 1: Basic Metadata\n\nQuery column metadata:\n\n```sql\nSELECT COLUMN_NAME, DATA_TYPE, COMMENT\nFROM <database>.INFORMATION_SCHEMA.COLUMNS\nWHERE TABLE_SCHEMA = '<schema>' AND TABLE_NAME = '<table>'\nORDER BY ORDINAL_POSITION\n```\n\nIf the table name isn't fully qualified, search INFORMATION_SCHEMA.TABLES to locate it first.\n\n## Step 2: Size and Shape\n\nRun via `run_sql`:\n\n```sql\nSELECT\n    COUNT(*) as total_rows,\n    COUNT(*) / 1000000.0 as millions_of_rows\nFROM <table>\n```\n\n## Step 3: Column-Level Statistics\n\nFor each column, gather appropriate statistics based on data type:\n\n### Numeric Columns\n```sql\nSELECT\n    MIN(column_name) as min_val,\n    MAX(column_name) as max_val,\n    AVG(column_name) as avg_val,\n    STDDEV(column_name) as std_dev,\n    PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY column_name) as median,\n    SUM(CASE WHEN column_name IS NULL THEN 1 ELSE 0 END) as null_count,\n    COUNT(DISTINCT column_name) as distinct_count\nFROM <table>\n```\n\n### String Columns\n```sql\nSELECT\n    MIN(LEN(column_name)) as min_length,\n    MAX(LEN(column_name)) as max_length,\n    AVG(LEN(column_name)) as avg_length,\n    SUM(CASE WHEN column_name IS NULL OR column_name = '' THEN 1 ELSE 0 END) as empty_count,\n    COUNT(DISTINCT column_name) as distinct_count\nFROM <table>\n```\n\n### Date/Timestamp Columns\n```sql\nSELECT\n    MIN(column_name) as earliest,\n    MAX(column_name) as latest,\n    DATEDIFF('day', MIN(column_name), MAX(column_name)) as date_range_days,\n    SUM(CASE WHEN column_name IS NULL THEN 1 ELSE 0 END) as null_count\nFROM <table>\n```\n\n## Step 4: Cardinality Analysis\n\nFor columns that look like categorical/dimension keys:\n\n```sql\nSELECT\n    column_name,\n    COUNT(*) as frequency,\n    ROUND(COUNT(*) * 100.0 / SUM(COUNT(*)) OVER(), 2) as percentage\nFROM <table>\nGROUP BY column_name\nORDER BY frequency DESC\nLIMIT 20\n```\n\nThis reveals:\n- High-cardinality columns (likely IDs or unique values)\n- Low-cardinality columns (likely categories or status fields)\n- Skewed distributions (one value dominates)\n\n## Step 5: Sample Data\n\nGet representative rows:\n\n```sql\nSELECT *\nFROM <table>\nLIMIT 10\n```\n\nIf the table is large and you want variety, sample from different time periods or categories.\n\n## Step 6: Data Quality Assessment\n\nSummarize quality across dimensions:\n\n### Completeness\n- Which columns have NULLs? What percentage?\n- Are NULLs expected or problematic?\n\n### Uniqueness\n- Does the apparent primary key have duplicates?\n- Are there unexpected duplicate rows?\n\n### Freshness\n- When was data last updated? (MAX of timestamp columns)\n- Is the update frequency as expected?\n\n### Validity\n- Are there values outside expected ranges?\n- Are there invalid formats (dates, emails, etc.)?\n- Are there orphaned foreign keys?\n\n### Consistency\n- Do related columns make sense together?\n- Are there logical contradictions?\n\n## Step 7: Output Summary\n\nProvide a structured profile:\n\n### Overview\n2-3 sentences describing what this table contains, who uses it, and how fresh it is.\n\n### Schema\n| Column | Type | Nulls% | Distinct | Description |\n|--------|------|--------|----------|-------------|\n| ... | ... | ... | ... | ... |\n\n### Key Statistics\n- Row count: X\n- Date range: Y to Z\n- Last updated: timestamp\n\n### Data Quality Score\n- Completeness: X/10\n- Uniqueness: X/10\n- Freshness: X/10\n- Overall: X/10\n\n### Potential Issues\nList any data quality concerns discovered.\n\n### Recommended Queries\n3-5 useful queries for common questions about this data.\n"
}