Clean up a list in ONE command: remove duplicates (ignoring case and spacing if wanted), sort, count how many times each item occurs, trim spaces, change case, drop blank lines, keep only lines that match (or don't match) a word or pattern. Works on a pasted list saved as a .txt file or on one column of a CSV/Excel file. Use for: dedupe this list, sort these names, how many times does each value appear, remove duplicates from a column, clean up this e-mail list, keep only the rows containing X. Triggers: duplicates, dedupe, sort list, count occurrences, clean list, remove blanks, unique values, trim.
Say it in your own words; the assistant finds this recipe by itself. Examples:
Clean up a list in ONE command: remove duplicates (ignoring case and spacing if wanted), sort, count how many times each item occurs, trim spaces, change case, drop blank lines, keep only lines that match (or don't match) a word or pattern. Works on a pasted list saved as a .txt file or on one column of a CSV/Excel file. Use for: dedupe this list, sort these names, how many times does each value appear, remove duplicates from a column, clean up this e-mail list, keep only the rows containing X. Triggers: duplicates, dedupe, sort list, count occurrences, clean list, remove blanks, unique values, trim.
python3 /skills/list-cleanup/scripts/cleanup.py <verb> IN OUT [options] — IN is a .txt (one item per
line; save a pasted list to /work/in/list.txt first) or a .csv/.xlsx with --column NAME.
OUT keeps the input type (.txt → .txt, .csv → .csv; count always writes a CSV).
Every verb prints one OK … line with what it did, or ERROR ….
| Need | Command |
|---|---|
| Remove duplicates | cleanup.py dedupe /work/in/list.txt /work/out/list.txt --ignore-case --ignore-space |
| Sort (A→Z, or numbers) | cleanup.py sort /work/in/list.txt /work/out/sorted.txt [--reverse] [--numeric] [--ignore-case] |
| How often does each item occur | cleanup.py count /work/in/list.txt /work/out/counts.csv [--ignore-case] [--top 20] |
| Trim spaces, drop blank lines | cleanup.py trim /work/in/list.txt /work/out/list.txt |
| Change case | cleanup.py case /work/in/list.txt /work/out/list.txt --to upper|lower|title |
| Keep lines that match | cleanup.py keep /work/in/list.txt /work/out/list.txt --match "@gmail\\.com$" [--invert] [--ignore-case] |
| All of the usual cleanup | cleanup.py clean /work/in/list.txt /work/out/list.txt (trim + drop blanks + dedupe, order kept) |
For a spreadsheet column add --column Email: dedupe keeps the first row per value, sort sorts the
rows, count counts the column's values, keep filters rows. --match is a regular expression; plain
words work as-is.
Afterwards tell the person what changed (the OK line says how many lines were removed or kept) and give them the download link.
SKILL.mdscripts/__pycache__/cleanup.cpython-311.pycscripts/cleanup.pyWritten by Percerno, MIT licence.