Smart Tabular Analyzer
A browser-based CSV and Excel analysis tool for fast, privacy-friendly exploration of structured data.
Current release: v2.7.0
Language
English | 中文
Live Demo
Open the demo and choose 使用示例数据 ("Use Sample Data") to complete an analysis without preparing a file.
About
Smart Tabular Analyzer helps users understand structured files with unfamiliar or inconsistent schemas. After a CSV or Excel file is selected, the application previews the data, infers field types, checks common data quality issues, calculates statistics, renders charts, supports filtered and grouped analysis, produces data-grounded observations, and exports results.
The project is a static website with no backend service, database, account system, or build step. It does not require fixed column names.
Core Features
- Import CSV,
.xlsx, and .xls files, including multi-sheet workbook selection, Chinese field names and values, and CSV encoding options for Auto Detect, UTF-8, GBK, and GB18030. Excel cells formatted as percentages are shown as percentages.
- Skip title, note, and blank rows above the header automatically, and state in the status bar and reports which row the header was found on.
- Describe CSV parse warnings (such as a row with fewer cells than the header) in Chinese with the file line they are on; the quality page shows how many warnings there are and how many lines they cover, and the “more data columns than the header” and “unclosed quote” errors name their line too. A file separated by semicolons, tabs, or bars that comes out as a single column gets a hint.
- Preview the data page by page (50 rows a page, with the file row in the first column, following the filters) and summarize the dataset size, field composition, missing values, and duplicate rows.
- Infer Numeric, Categorical, Date, and ID fields, with manual correction to Numeric, Categorical, Date, ID, or Ignore when needed.
- Report missing values and rates, complete duplicate rows, IQR-based numeric outliers, and values that cannot be converted to numbers or dates, shown as format problems on the quality page with up to three samples per field and counted in the statistics table.
- Check fields of type ID for duplicate values: the quality page and reports list up to 10 duplicate values per field with how often each appears, the file rows it appears on (the Excel row, or the CSV line where the record starts), and how many of those rows are identical. ID columns whose values mostly repeat, such as a user ID in an event log, get a short note instead of a list.
- Find values such as N/A, NULL, or — that are often written for “no data” (suspected missing values): the quality page lists each one per field with its count, file rows, and how the missing count and the field type would change once it is confirmed. Nothing changes until you confirm; after confirming one value in one field (or all common values at once), it is treated as missing, so it no longer counts as a conversion failure and stays out of statistics, Top 10 lists, groups, filter options, and the ID duplicate check. Every confirmation can be undone. The original data, the data preview, and exported CSV files keep the original text, and reports list which values were treated as missing and which are still unconfirmed.
- Read numbers written with a unit, 万 / 亿, or full-width digits, column by column: a column of amounts such as 2200元 is read in 元 (yuan), 1.2万 is read as 12000 and 3亿 as 300000000, and full-width digits such as １２３ are read as 123. The recognized units are 元; kg, 克, 千克, 公斤, 斤, 吨; 件, 个, 箱, 台, 人, 次; 秒, 分钟, 小时, 天; 米, cm, mm, km; and 岁 (Latin letters in any case). Statistics, chart titles and axes, and reports show the unit after the field name, such as “金额（元）”; the data preview and exported CSV files keep the original text. The quality page lists which fields were read this way (“按单位读取的数字”, numbers read by unit) and points out category fields where most values, but not enough for a numeric field, can be read as numbers (“疑似数值列”, numeric-looking fields), with up to three values that could not be read.
- See the problem rows: next to each count on the quality page, “查看” (view) lists the rows behind it — missing values and format problems per field, outliers per numeric field, complete duplicate rows by group (the first row of each group marked as kept, as in the deduplicated export), every row of every duplicate ID value, suspected missing values per field and value (confirmed ones too), and CSV parse warnings. Each row shows its file row, a short note, and all original columns, with the problem cell highlighted, 50 rows a page, and a list to switch categories; “导出这一类” (export this category) writes every row of the category to CSV, with “文件行号” (file row) and “说明” (note) as the first two columns and the same BOM and formula-prefix neutralizing. The listed rows come from the same checks as the quality counts, use the original data, and do not follow the filters.
- Generate descriptive statistics, categorical Top 10 frequencies, numeric distributions (small integer ranges such as ratings are counted per value), and date trends that show record counts by default or the average or sum of a chosen numeric field.
- Apply categorical, numeric, and date filters, and run grouped analysis (one metric, one grouping field, optionally a date field).
- Show one data-grounded conclusion under the category chart and one under the date trend chart, each following the field selected in that chart and without inferring external business context; the trend conclusion compares record counts and flags a possibly incomplete last month or year, and the category conclusion says so instead of summarizing a field whose values are too scattered. Reports carry the same sentences.
- Export filtered or deduplicated rows to CSV in the browser while preserving field order and neutralizing spreadsheet-formula prefixes.
- Export the analysis as HTML or Markdown; reports also record the applied filters, and HTML reports embed the currently rendered charts for offline viewing.
Detailed processing rules and data flows are documented in Architecture.
Screenshots
Home
Upload CSV or Excel files, or start immediately with the built-in sample data.
Data Quality
Review missing values, duplicate rows, format problems, IQR-based numeric outliers, and duplicate ID values in one view.
Analysis Results
Compare numeric distributions, categorical Top 10 results, and date trends through interactive charts.
Privacy
CSV and Excel parsing, filtering, analysis, processed-data export, and report generation run locally in the browser. The application does not upload or store user files or generated reports. It does not send dataset contents to a third-party API or require an account.
The page loads pinned versions of PapaParse, Chart.js, and SheetJS from the repository's own vendor/ directory and verifies them with Subresource Integrity (SRI). It does not contact third-party CDN hosts, and uploaded file contents remain in the browser runtime. Because the libraries are hosted with the page, files you import can still be analyzed after the page has loaded, even without a network connection; the built-in sample data is fetched from the site and needs the connection.
Tech Stack
Quick Start
Use the Live Demo
- Open the Live Demo.
- Choose 使用示例数据 ("Use Sample Data"), or upload a
.csv, .xlsx, or .xls file.
- Review the inferred field types, correct them only when needed, and use the analysis navigation to explore or export the results.
Run Locally
Clone the repository and start a static server:
git clone https://github.com/wy-data-30/smart-tabular-analyzer.git
cd smart-tabular-analyzer
python -m http.server 8000
Then open:
http://localhost:8000
Opening index.html directly also works for file uploads, but a local server is recommended because some browsers restrict the sample-data fetch() request on file:// pages.
Testing and CI
The automated tests require Node.js 18 or later and use the built-in node:test runner without test dependencies.
Run the focused core data-processing suite:
npm test
Run the full import, export, report, boundary, and HTML integration regression suite:
npm run test:regression
GitHub Actions CI runs the full regression suite with Node.js 22 on every push and pull request. See the Testing Guide for suite coverage and troubleshooting.
Documentation
- Architecture: architecture, state, import, field inference, filtering, analysis, charts, and export data flows.
- Testing Guide: test commands, suite coverage, fixture guidance, and common failures.
- Maintenance Guide: core invariants, dependency updates, deployment, and troubleshooting.
- Release Checklist: automated and manual checks used before publication.
Project Structure
.
├── index.html
├── style.css
├── parsing.js # Missing values, number and date parsing
├── profiling.js # Field type detection and field profiles
├── format.js # Display formatting and HTML escaping
├── stats.js # Descriptive stats, IQR, grouping, trends
├── quality.js # Conversion failures, duplicates, ID duplicates, missing rates, problem rows
├── app.js # State, DOM map, refresh paths, rendering
├── importer.js # CSV / Excel import
├── report.js # Reports, CSV export and problem-row export
├── script.js # Startup: self-check and event bindings (loads last)
├── vendor/ # PapaParse, Chart.js, SheetJS and their licenses
├── examples/
│ └── sample-data.csv # Loaded by "Use Sample Data"
├── assets/
├── docs/
├── tests/ # Tests and fixtures (tests/fixtures/examples/ holds the other sample CSVs)
└── package.json
Known Limitations
- Each import is limited to 25 MB, 100,000 data rows, and 200 columns.
- Parsing and analysis run on the browser main thread and may temporarily block the interface near the import limits, especially on mobile devices.
- Field inference is heuristic, and strict date parsing may require manual confirmation for ambiguous columns.
- Title, note, and blank rows above the header are skipped automatically; the header row is looked for within 10 rows starting at the first non-empty row. A row can be skipped when it holds a single text cell, or two text cells where one contains a colon (such as “制表日期：…”). When rows are skipped, the status bar and the reports state “表头在第 N 行” (the header is on row N).
- If no header row can be confirmed, the error message suggests removing the title rows; choosing the header row manually is not supported yet.
- Multi-row headers are not supported, total rows such as “合计” are treated as data, and several tables on one worksheet are not supported.
- The header itself must still be unique and non-empty for every column that holds data, and Excel headers must be text cells; empty, headerless, protected, or structurally irregular worksheets may be rejected.
- Excel percentage formats are detected from the first 1,000 data rows below the header; other number formats such as currency are not read.
- ID duplicates compare values after trimming spaces, with case kept. Placeholder text such as N/A or — in an ID column is not treated as missing until it is confirmed on the quality page, so repeated unconfirmed placeholders are listed as duplicate values.
- Suspected missing values come from a fixed list and must match the whole cell (after trimming spaces and folding full-width characters); other text such as “缺考” or “未知” is not listed. Values that contain a zero-width space (U+200B) do not match.
- “无”, “暂无”, “-”, “/”, and “NA” can also be real values, so they are not included when confirming all common values. In accounting formats “-” usually means 0; check what the column means before confirming it.
- Excel formula error cells (#N/A, #DIV/0!, and so on) are imported as missing values directly and do not appear in the suspected missing values list.
- Numbers with units must be written as a whole cell of number, optional 万 / 亿, and optional unit. ￥2200元 (a currency sign and a unit), (2200元) (an accounting negative with a unit), 1.2 万 (a space before 万), and 2200元/件 are not read and still count as format problems. g, m, and 分 are not recognized as units, because they are ambiguous (4G / 5G, meters or minutes, points or cents).
- When one column mixes units, such as kg and 千克, or kg and 克, nothing is converted between them and the whole column is not read by unit; the quality page says which units appeared and how often.
- A field named in 万元 whose values are written like 1.2万 is still read in 元, as 12000.
- In accounting formats “-” usually means 0, but it is still listed as a suspected missing value and never read as 0.
License
This project is licensed under the MIT License. See LICENSE.
