Objeck Documentation

All Bundles

Bundle Natural language processing toolkit. Provides tokenization, text preprocessing, TF-IDF vectorization, cosine similarity, and sentiment analysis. Works with plain strings — no external model required. Compile with -lib nlp.

TF_IDF

Term Frequency-Inverse Document Frequency (TF-IDF) vectorizer for document analysis

Example

documents := ["The cat sat on the mat", "The dog sat on the log", "Cats and dogs are pets"];
tfidf := System.NLP.TF_IDF->New();
tfidf->Fit(documents);

test_doc := "The cat and the dog";
vector := tfidf->Transform(test_doc);
vocab := tfidf->GetVocabulary();

Fit #

Fits the TF-IDF model on a corpus of documents

method : public : Fit(documents:String[]) ~ Nil

Parameters

Name	Type	Description
documents	String	array of document strings

Example

docs := ["the cat sat on the mat", "the dog ran in the park", "cats and dogs are pets"];
tfidf := TF_IDF->New();
tfidf->Fit(docs);
"Vocabulary size: {$tfidf->GetVocabularySize()}"->PrintLine();

GetVocabulary #

Gets the vocabulary (word to index mapping)

method : public : GetVocabulary() ~ Hash<String,IntRef>

Return

Type	Description
Hash<String,IntRef>	vocabulary hash map

Example

vocab := tfidf->GetVocabulary();
keys := vocab->GetKeys()<String>;
each(k in keys) {
  k->PrintLine();
};

GetVocabularySize #

Gets the size of the vocabulary

method : public : GetVocabularySize() ~ Int

Return

Type	Description
Int	vocabulary size

New # constructor

Constructor

New()

Transform #

Transforms a document into a TF-IDF vector

method : public : Transform(document:String) ~ Float[]

Parameters

Name	Type	Description
document	String	input document string

Return

Type	Description
Float	TF-IDF vector as float array

Example

vec := tfidf->Transform("the cat chased the dog");
"Vector length: {$vec->Size()}"->PrintLine();