esTS computes statistics of Spanish texts: basic statistics, readability, lexical diversity, lexical sophistication, style, phonostatistics, morphology, syntax and cohesion - by published formulas with the coefficients and the scales of their authors, and by the parts of speech and the features of Universal Dependencies.
The library works both with raw strings and with Doc objects of spaCy; most statistics need no trained model (see Installation). Try it without installing in the demo on Hugging Face Spaces: paste a text and get its readability, the metrics, the plots and the highlighting of its fragments. The notebooks show the library on its datasets - a short story through every tool, prose authors, sonnets and corpus measures; the first one opens in Colab.
- Object extraction - configurable sentence, word and character N-gram tokenizers that know the inverted marks, the dialogue dash and the abbreviations of Spanish, and the paragraphs of a text
- Syllables and stress - rule-based syllabification and the stressed syllable derived from the spelling, with no dictionary
- Basic statistics - counts of sentences, words, letters, syllables and punctuation marks by type, with distributions, normalized shares and the entropies of word lengths
- Readability metrics - Fernández Huerta, Szigriszt-Pazos with the INFLESZ scale, Gutiérrez de Polini, Crawford, Legibilidad µ, SOL, LIX and RIX, with a consensus grade, the school stages of Spain and reading time, and formulas of other languages with their own coefficients
- Lexical diversity metrics - TTR and its variations, MATTR, MSTTR, MTLD and MAMTLD, HD-D, Simpson's and Yule's indices, entropy, Orlov's Z, the burstiness of the lexemes, Zipf's and Heaps' laws
- Morphological statistics - parts of speech and fifteen grammatical features of Universal Dependencies, with the markers of Spanish: the moods, the non-finite forms,
ser against estar, the adverbs in -mente
- Corpus measures - keywords against a reference corpus, collocations, the dispersion of a word over the parts of a text, a KWIC concordance and the stylometry of authorship: Burrows's Delta with its variants, Zeta, the Mendenhall curve, the profile of the function words; the comparison of two corpora by 163 features of a text with effect sizes
- Visualizers - Zipf's law, literature fingerprinting, a word tree, lexical dispersion and keywords, a network of collocations, a dendrogram, PCA and MDS by Delta, the distances within and between authors, rolling Delta and Zeta, vocabulary growth, sentence lengths, and the highlighting of the fragments the statistics count, such as long sentences and passives
- Datasets - Spanish-language literature in the public domain: 150 works by 33 authors in four genres; 4,259 sonnets of the 15th-20th centuries with the metrical pattern and the rhyme of every line; a frequency dictionary of 83,785 lemmas by Google Books Ngram
- spaCy components - every statistics class as a component of a pipeline, the statistics attached to the
Doc in one pass
- Cohesion statistics - the overlap of nouns, arguments and content words between sentences, givenness and temporal cohesion in the manner of Coh-Metrix, the overlap with the next sentences and paragraphs of TAACO, with the density of 273 Spanish discourse markers by class and by place
- Lexical sophistication statistics - how rare the words of a text are in the language: the frequency, range and dispersion of the lemmas by a dictionary of Google Books Ngram, the frequency bands top-1000 to 10000, surprisal, perplexity and lexical density
- Style metrics - the SEO indicators of Advego and Text.ru (nausea, water content, spam score, naturalness by Zipf's law, keyword density) and the markers of the officialese style by the Spanish-language guides to plain language: verbal nouns, compound prepositions, parenthetical expressions and clichés
- Phonostatistics - the shares of the classes of sounds, consonant clusters, hiatuses, open syllables, hardness and the indices of alliteration and assonance, over the sounds of a rule-based transcription
- Verse statistics - the scansion of Spanish verse by its syllabic meter: the metrical syllables, the meter of a poem, the stress profile and the types of the endecasílabo, the rhyme in full and by assonance with its scheme, the strophes and the form of a poem
- Syntactic statistics - the dependency tree by distances, depth, clauses and coordination, the production units of L2SCA, prepositional chains, the order of the subject and the object, the similarity of adjacent sentences, with the constructions of the administrative style: the passive with
ser and with se, the participial and the gerund clauses, the chains of de, the split predicates
以上為 README 的開頭部分。其餘內容請閱讀原文。 檢視完整 README ↗
