cevahir-ai
Cevahir Zihin: a mind that learns by living, not a Transformer. It keeps knowledge as evidence on ties, attends with a Seven-Place Focus and learns from every conversation. Also: a Turkish BPE tokenizer, a Transformer language-model core and cognitive systems.
Automatically checked · GitHub checked 2026-11-16
Overview
🇹🇷 Cevahir AI & Engine
Research archive DOI (always the latest version): 10.5281/zenodo.22864660 · Research reports
Latest update — November 2026: Cevahir Zihin, a mind that learns by living. Not a Transformer.
The new model in this repository, Cevahir Zihin (
zihin/, whatpython -m zihinruns), is not a neural network but a Living Evidence Mind. It has no weight matrices, embedding vectors, backpropagation or softmax attention. It keeps every relation as a tie and keeps each tie's evidence by where it came from: seen with its own eyes, told, imitated, its own act, inferred or imagined. It reads what to expect from a single rule:e = c·(f − ½) + ½. The earlier Transformer engine described below is still in the repository.
- Attention: at each moment only seven units enter the focus (the Seven-Place Focus). There is no context window: it never looks back over the past and reads only the ties of the units in focus, so the time per moment grows only slowly as the mind grows (about 2 ms on a small mind, about 4 ms at 3 million ties).
- Forgetting: while its ledger is kept, the evidence of any single experience can be taken back exactly (Exact Retraction); in the program this covers what you typed in the current session (
!unut).- Recall from one reading: a mind that read a set of documents once, given the first two words of a random sentence, said a sentence it had read that begins with them in 145 of 200, and the rest of that very sentence in 60 of the 61 whose two words begin only one sentence (Sequence Starts, Lived Exclusion, Exact Relations). The strongest small Transformer trained on the same documents for 50 minutes said a valid one in 41.
- Not knowing: it feels when it does not know (a measured inner state). How it says so is learned from the people it lives with and from its data; the core holds no sentence for it.
- Dreams: in sleep it dreams about the day's topics, and on later nights sorts which of its imaginings came out true.
- Learning from consequences: in a research test (six tiny functions, three candidate completions each) it learned from test results alone to choose the completion that passes; writing new code is still open.
There is no separate training phase; the model learns in every conversation. Please speak to it kindly.
Theory and architecture: docs/ZIHIN_THEORY.md · guide (Turkish): docs/ZIHIN_KILAVUZ.md · measurements and negative results: research/living_mind_2026_11_15/NOTES.md
Run Cevahir Zihin on your computer
It needs Python 3.11 or newer. Linux / macOS:
git clone https://github.com/myylogic/cevahir-ai
cd cevahir-ai
python3 -m venv .venv && . .venv/bin/activate
pip install torch numpy # optional: python-docx (Word), pypdf (PDF), pillow (images)
python -m zihin
Windows PowerShell:
git clone https://github.com/myylogic/cevahir-ai
cd cevahir-ai
pip install torch numpy
.\cevahir.ps1 # if scripts are blocked: powershell -ExecutionPolicy Bypass -File .\cevahir.ps1
Start it inside the cevahir-ai folder. The program speaks Turkish: it first asks "Seninle konuşan kim?" (who is
talking to you?; type your name). On first start the mind is empty and knows no language; it learns by living with
you. If you have files to teach it, type these at the program's prompt, not in the shell (each command has an English
alias):
!oku my_documents/ # !read: .txt .md .docx .pdf documents, .json/.tsv dialogues, source code
!ogret dialogues.tsv # !teach: each line question<TAB>answer, saved as UTF-8
!izle hello || hi # !watch: let it watch an exchange
!dogru / !yanlis # !good / !bad: how you received its last answer
!uyu !dusun <topic> !kendin # !sleep, !think, !introspect: dream, inner voice, introspection
!cik # !quit
While it is open it lives in its own time: it lives silent moments, thinks inwardly after ten seconds of silence and
sleeps after ten minutes (--yasamsiz turns this off); it says something only when its own decision produces words. The mind is saved at the end of every session to ~/.cevahir/zihin/cevahir.zihin
(Windows: %USERPROFILE%\.cevahir\zihin\cevahir.zihin; CEVAHIR_HOME changes the folder; --kaydetme does not
save).
The minds raised so far lived the author's private Turkish data and are not in this repository. Your copy learns the
language you teach it with !ogret and !oku. For a language other than Turkish, start a new mind with --dil diger
so that capital letters are lowered without Turkish rules (a mind keeps the rule it was raised with). After a dozen
pairs it answers the questions it was shown; how much data a new language needs for questions it has not seen has
not been measured. A mind file carries what it has read, so do not share a mind raised on private documents.
An open-source AI research engine: a new mind core that learns by living (Cevahir Zihin), alongside Turkish-specific tokenizer infrastructure, a Transformer language-model core and cognitive systems.
I'm Muhammed Yasin Yılmaz. I built Cevahir to bring data preparation, model learning and the use of a trained model in conversation into one codebase. It includes its own BPE pipeline, a configurable PyTorch Transformer decoder, training and checkpoint management, generation, cognitive workflows, conversational memory, and application services.
I want developers, especially young people in Türkiye, to have a foundation they can inspect, train on their own data and extend with new capabilities. I share the implemented modules, my completed model training runs and their outputs as this work develops.
My gift to Turkish youth. — Muhammed Yasin Yılmaz
I am continuing Cevahir as an open-source research and educational legacy, with productization no longer its goal. The engineering work and research on learning throughout a system's operating life continue together.
New core: Cevahir Zihin — a mind that learns while living (November 2026)
In November 2026 I started a new core, separate from the Transformer: Zihin (Turkish for "mind"). What I am after is not a fixed network but a mind that runs its own network while living, builds concepts from its experiences and measures what it builds. The Transformer engine and all earlier research remain below, unchanged.
Why this is not a Transformer:
| Transformer | Cevahir Zihin |
|---|---|
| train → freeze → use | learning is living: it learns at every moment, there is no separate training phase |
| fixed dimension, fixed vocabulary, embedding vectors | growing units and ties, open alphabet; no fixed dimension |
| next token by softmax | conviction: every belief has a strength and a source; if nothing accumulates to threshold, it stays silent |
| new knowledge damages old knowledge, retraining needed | new knowledge is added, old knowledge stays bit-identical |
| knowledge is spread over weights, a single example cannot be removed | while its ledger is kept, the evidence of a single experience can be retracted exactly from the core |
| no record of where a belief came from | every piece of evidence keeps its source (told, sensed, imitated, own act, inferred, imagined); trust in sources is learned |
Measured results (two generated worlds, 5 seeds; details in results and notes):
- A first hearing is weak and conviction grows with hearings (c 0.41 after one, 0.78 after four or more); a correction weighs old evidence instead of deleting it (120/120).
- It finds hidden classes by itself (Jaccard 1.00).
- On instances never told the asked dimension, generalisation improves with age: 93% / 81% (two domains) after 600 experiences, 100% / 90% after 1500. The earlier reading rule got worse with age; eight failed attempts and the cause are recorded.
- No forgetting: facts learned early are bit-identical hundreds of experiences later; living a second domain does not disturb the first (10/10).
- The evidence of a single experience can be retracted exactly (10/10); this is exact for the evidence, not for the structure the experience created.
- It imitates first, then detaches from what it copied by its outcomes, while keeping the knowledge that others act so (T7).
- Against a small Transformer on the identical stream of experiences (5 seeds × 2 worlds, identical probes; Zihin in its "members" configuration): on instances never told the asked dimension Zihin reaches 90% / 97% and 75% / 95%; a Transformer trained online in one pass 67% / 57% and 60% / 50%; one trained to convergence on the whole stream 77% / 67% and 53% / 53%. A second domain lowers the one-pass Transformer's accuracy on the first from 0.85 to 0.74 and leaves Zihin's at 1.00. The configuration the chat program runs was not in this comparison and did poorly in these worlds (held-out 0.57 and 0.04 after 600 experiences); on 15 November two causes were found: these worlds let it speak into their stream, so it lived its own words between theirs, and a relation reading (F12's "next") took the members' agreement on a value. Listening while the world speaks, as in a chat, and with Exact Relations (F28), Sequence Starts and Lived Exclusion, it reaches 0.78 and 0.81, with members, exceptions and the hidden classes all at 1.00 (the "members" configuration's 0.93 in the first world is not reached). Retracting one experience takes 20–77 ms in Zihin and is exact; the Transformer's exact counterpart is retraining from scratch. This comparison is in the small-data, learn-while-living regime; the Transformers are small and the gap may close with far more data.
- Chat: it says what it watched, answers questions it never watched by relations ("what does a bird do" → "a bird sings"), stays silent when it knows nothing, and stops giving a copied answer once told it was wrong.
- It sees and hears (first versions): image and sound files are perceived by plain pixel and signal arithmetic, with no trained network, and recognised by resemblance. New images 73% / 83% / 67% right, new sounds 93% / 97% / 93% (chance 33%; 0% with resemblance off).
- Imagination (first form): it imagines by carrying a thought over to a structurally similar concept and scores every step for reality against lived evidence (T8).
- Sleep and dreams: asleep, it dreams about the last topic and the words it is unsure of. A dream is a chain of imaginings linked by association, and an image it saw with a word is "seen" again in the dream. Then it weighs its imaginings against what it has lived, says which earlier dreams came out true or false, and learns from this how far to trust its own imagination. A dream found false is not deleted; real evidence outweighs it.
- Not knowing, and thinking: when its turn comes with no answer at once, a silent moment passes, in which it may perceive its not knowing as an inner state, and it decides again; what it says is only what its own decisions emit. How it says that it does not know, and how it asks, are learned from the people it lives with and from its data. Until 15 November 2026 the chat program also had coded reflexes that spoke for the mind (it asked back an unknown word with '?', said a thought for it, and spoke first after a silence); they were removed on that day, so a new mind is silent until it has lived how to answer. Until 15 November 2026 the chat program also held scripted lessons about the mind and about saying "bilmiyorum", written by the developer; they were removed from the core at the owner's decision, and minds raised before lived them.
- Online life: from the moment a session starts it lives in its own time. Its words come out as it says them. In silence it lives silent moments (and says something only if its own decision produces words), thinks inwardly after 10 seconds and sleeps after 10 minutes; these two timings are an innate rhythm that may later be read from the mind's own evidence.
- Turkish structure (opt-in): with
--parca cevahira mind lives each word with its root, suffix and stem pieces from the repository's own tokenizer infrastructure, so words built alike resemble each other. Raised with pieces from birth, a mind grew about five times as many ties per moment and ran out of memory, so pieces are off by default. - Learning code from its consequences (first test): on six tiny functions it first watches mostly wrong completions. Then the code it writes is actually run, and the test result comes back as a good/bad signal; when it hesitates it tries the alternative whose consequences it knows least. By round 4 it writes passing code for all six tasks; without the signal it stays at 0%, and without trying at 11%.
- Learning to learn (F24, off by default): from its own outcomes it learns how far a kind of "voice" can be believed (how many members speak, how much they agree), and it carries this into a new domain. Early learning speeds up in one direction (+0.058 ± 0.025; better in 7 of 10 lives) but not in the other; the result is mixed.
- Speed and memory: the core is 2–3.8x faster per moment with bit-identical results. Memory per tie went from 239 to 69 bytes: on a mind with 3 million ties the core shrank from 777 MB to 223 MB and the file from 930 MB to 377 MB. Old mind files are converted on load. The memory layout costs 13–32% per moment.
- A self: an innate "self" unit is present in every moment the mind speaks or thinks. By introspection it reads what it has said and thought, and how much, in its own life.
- Recall of what was read once (F26 Sequence Starts, F27 Lived Exclusion): how a word went on where a sequence began with it is now kept apart from how it went on anywhere, and what a situation was itself lived to be followed by counts against what a generalisation proposes there. A mind that lived only the documents (about five minutes on one core) was given the first two words of 200 random sentences: it said a read sentence beginning with them in 150 (before these readings 90), the rest of that very sentence in 89, and in 60 of the 61 sentences whose two words begin only that sentence. The duel's Transformers, trained on the same documents with the same units, said a valid sentence in 0 (1M, one pass), 10 (1M, 18 epochs) and 41 (7.4M, 50 minutes). With a one-word cue the 7.4M Transformer said a valid sentence more often than the mind (51 against 44): the mind stays silent when a word begins several sentences. Asked in the form the teacher dialogues used ("X nedir ?"), a mind raised on them and the documents now says 119 of 200 held-out answers exactly (59 before). The earlier duel had asked both sides the bare word, a form neither had lived; it is replaced by this test. With Exact Relations (F28, also on now) the two-word numbers are 90 exact and 145 valid (forks, where several read sentences go on from the same words, lose a tie the looser relation broke by accident) and the one-word valid count is 47. One data set and one sample of 200; notes.
- Meaning from the places words fill (F29 Analogy by Interchangeability): two words are interchangeable as far as they filled the same lived places, a place shared by few counting more; what a category's members lived carries over to a new instance through that interchangeability (the analogy rule of NARS). With F30 Corroboration (independent lived evidence about the same claim adds up, each lived context counted once), wordings never watched in the meaning world (3 seeds) with another greeting in front ("selam , adın ne ?", "hello , nasılsın ?") went from 0.17 to 1.00 answered, synonym wordings ("ismin ne ?") from 0.33 to 0.67, an extra word ("peki adın ne ?") from 0.33 to 0.67; words never met still carry nothing, so the hard wordings stay silent and no wrong answer appeared. A restructured question and a word form never met are still out of reach. With F31 Paradigmatic Relation and F32 Relational Binding (a frame's value may be what stands where its filler stands, and a place binds that stand-in as it binds its repeats), the definitions of four other words taught "selam ne demek ?" -> "selam , merhaba demektir" (meaning questions 0 -> 0.67 of 3 seeds). Recall, the teacher protocol and the generated worlds are unchanged; notes.
- One mind, many relationships (F33 Relational Memory): what is lived while someone the mind knows is present is also noted with them, and recall depends on who is there. What was lived with no one in particular is recalled with anyone, what was lived with the person present is theirs, and what was lived with others is recalled only as far as other people corroborate it. In a world of four people and two strangers (3 seeds), each person now gets their own greeting, favourite colour and cat's name (0 -> 1.00), a secret lived with one person is told to no one else (strangers 1.00 -> 0, left silent), and what all four taught is still known with a stranger; no persona, role or template per person. Trust is learned per person from whose word held (0.96 against 0.64 for someone whose facts the mind's own experience contradicted, where one trust for everyone stood at 0.57), and a changed habit takes over from the evidence itself (a new greeting after 6 encounters against the 8 of the old one, each also a failure of the old expectation); notes.
- Compositional generalisation (F34 Reply Relation): a frame's value may be what its filler, said on its own, was answered with, and several places of a context can be bound to the filler. On SCAN's "add primitive" split (Lake & Baroni 2018), made small for the mind's seven-item window (T-compose: four primitives, eleven templates, "jump" seen only alone), the mind answers 75 of 75 new commands ("jump twice" -> "i_jump i_jump", "walk and jump", "jump left twice"), where a Transformer trained from scratch on the same dialogues, exact on all its training commands, answers 4 of 75 and the mind without F34 none. On SCAN's own grammar, the 844 test commands that fit a window of 11: the mind 789 (93%), the Transformer 0 (it answers 1570 of its 1574 training commands). Every command whose words, end and answer fit the window is answered exactly (596 of 596); the 55 errors are answers whose last word is decided after the command's first word has left the window: with a window of 12 the mind answers all 844 exactly, and with a window of 13 all 1,188 test commands that fit 12 items. Longer commands are beyond the window (11% of the split's test commands fit 11 items, 29% fit 15; a working memory is next). Counting only the lived suffixes that keep every fixed word of a frame (which made the answers finish) silenced two greeting-fronted questions of the meaning world (1.00 -> 0.67); F38 Witness Reliability (off by default) brings them back (1.00): a shorter context is a witness of a claim as far as the mind has seen the longer contexts that end with it go on as it did, carried by NARS deduction, so it says nothing of a value it never went on with; answering with it loses no SCAN command, but a SCAN life lived with it learned otherwise (789 -> 767), so it stays off while a form that does not is measured; notes.
- Purpose (F35 Generalised Consequences): what an act would lead to is read from what similar paths led to, so what the mind learned from how its answers were received carries over to situations it never met (polite to things never handed to it: 0 -> 8 of 9, against the rude habit it watched), and with F33 what pleased one person becomes an expectation of that relationship (two people with opposite tastes each answered their way for new things, 9 of 9; without F33 their consequences cancel); notes.
- The layer below words (F36 Word Forms): a word never met is read as the lived word it shares its longest stem with, the stem being a boundary of the lived words themselves (Harris's successor variety): "merhabalar" is greeted as "merhaba" and "nasılsınız ?" answered as "nasılsın ?" (both 0 -> 1.00). An audit of what the mind can and cannot do, with the controlled experiment on "Yağmur nasıl yağar sence ?", is in DENETIM_2026_11_16.md (Turkish).
- Deduction with exceptions (F37 Inheritance, off by default until it is cheap on a large mind): facts read apart ("kartal bir kuştur .", "kuşlar uçar .") give a conclusion never taught ("kartal uçar") by the deduction of NARS, the class named by a stem the mind found in its own words, and an exception stated of itself ("penguen yüzer") wins over its class: T-deduce 48 of 48 (2 without), no wrong generalisation. Chains of two steps: a class word inherits from the classes it is said to be of, so "levrek bir balıktır .", "balıklar bir hayvandır ." and "hayvanlar solur ." read apart give "levrek solur": T-chain 27 of 27 (none without F37 or with one step), the nearer property 9 of 9, the exception 3 of 3, no wrong generalisation; an answer on the largest mind takes about eleven times as long with it. Causal relevance (F39 Replacement Witness, off by default): with clouds the common cause of the umbrella advice and the rain, the words alone lead the mind to rain on a sunny day with the advice; with Mill's method of difference read from its own life, the fine weather leads (still under the speaking line). Long answers, a first step (F40 Urge to Go On, off by default): two expected words that are both things known about the same item race against silence, not against each other (T-long: more facts said about things never asked, no more confabulated).
!nedenalso reports contradictions: when another reading expected another word (its own memory overriding a generalisation, for example).!nedenshows where each word of the last answer came from (lived evidence, a frame and its relation, a continuation, a form, similar situations, an expected consequence); notes.
Not yet: learning to learn gave only a small gain, in one direction (F24). Curiosity, imagination and dreams are first forms: the reality score cannot yet tell a plausible construction from a weak leap. Scaling is partly solved: the cost per moment now grows little with memory and memory per tie is 3.5x smaller, but memory still grows linearly with lived data; measurements and the plan are here. It has no voice of its own (no speech output). On real data its chat is mostly imitation; it answers questions about itself with the true things it was told, and its self unit does not steer what it says yet. The worlds are small and the Transformer comparison covers only the small-data regime. This is not a claim of consciousness or of general intelligence; neither can be measured for any machine today. The aim and the road map are explicit: a mind that learns by living, builds its own concepts, makes sense of what it acquired and knows itself. The design layers (deliberate focus, automatic reflexes, inner and outer voice, drives and will, knowing itself) are here, the vision here, the latest work (sleep and dreams, hearing) here.
Try it (full guide in Turkish: docs/ZIHIN_KILAVUZ.md; from PowerShell: .\cevahir.ps1):
python -m zihin # a new mind
!ogret dialogues.tsv # each line: utterance<TAB>answer (JSON and Word documents too: !oku)
!konus # chat mode
!dogru / !yanlis # how its last answer was received (good / bad)
!yas !neden !neden <text> => <word> !unut <n> !kaydet <name>
!gor me.jpg !dinle my_voice.wav # show an image / play a sound with the next utterance
!dusun [topic] !uyu !kendin # hear its inner voice / sleep, dream and sort its dreams / introspection
Read the engine as a book
Cevahir AI — Anatomy of an AI Engine: table of contents
The modular Turkish book connects AI fundamentals to the real Cevahir implementation: tokenization, neural computation, Transformer blocks, training, checkpoints, generation, memory, cognition, tools and open research. Each chapter follows actual callers, inputs, outputs, configuration gates and evidence. English reading guide · Source map · Keeping code and documentation aligned.
The research laboratory preserves successful and negative experiments. The research publication series presents nine reports by Muhammed Yasin Yılmaz, with English abstracts, full methods, evidence and limitations. The latest future-query study distinguishes the information needed for future queries and corrections. These bounded results do not solve the general living-learning problem. Version and DOI record.
What I'm building with Cevahir
With Cevahir, I want to bring language representation, model learning and model use into one extensible system. I want to understand and develop the whole path: how text becomes tokens, how a model learns from them, and how its responses use conversation history and tool results. I keep these stages open to inspection and modification in the source code.
A place to develop language representations. The BPE infrastructure developed specifically for Turkish connects normalization, vocabulary, merge rules, optional syllabification and morphology components to data preparation. Work on how language is represented can then reach model training through the same token identities.
Infrastructure that turns architectural choices into trainable models. Attention head layouts, positional encoding, normalization, residual structure and dense/expert FFN options feed into model construction. The training service connects that model to data; the training manager handles loss, gradients and optimization. Checkpoints and tokenizer identity provide the basis for using the resulting model with the same meaning later.
A cognitive layer that uses the learned model. Cevahir connects generation and candidate scoring to strategy selection, multiple candidate generation, tool use, memory retrieval and critic revision. This creates room to study and improve response construction without retraining the weights.
A path from inspectable internals to conversational applications. Model profiling, gradient/weight health checks and cognitive traces expose internal behavior. The unified Cevahir interface, conversation manager and application services connect the same engine to applications with users, sessions and history.
I use this foundation to develop Turkish language modeling and explore model and cognitive-system ideas. I've described how the parts work together in the system overview and their implementation behavior in the architecture contract.
System architecture
flowchart TD
Sources[Text, document and question-answer data] --> Tokens[TokenizerCore / BPE]
Tokens --> Cache[Prepared training data]
Cache --> Training[TrainingService / TrainingManager]
Training --> Core[CevahirNeuralNetwork]
Core --> Checkpoint[Weights, architecture and tokenizer identity]
Checkpoint --> Manager[ModelManager]
Manager --> Generation[Generation and candidate scoring]
Tokens --> Generation
User[User message] --> Chat[Cevahir / ChattingManager]
Chat --> Cognitive[Cognitive strategy and response processing]
Cognitive <--> Generation
Cognitive <--> Memory[Conversation memory and retrieval]
Cognitive --> Tools[Registered tools]
Tools --> Cognitive
Chat <--> Services[HTTP services and persistent sessions]
The main model path is Cevahir → ModelManager → CevahirNeuralNetwork. The training launcher selects the V3 training service when available, using the active V2 TrainingManager with training_backend="v2"; it retains a V2 service fallback. Names such as V4, V7 and V8 in older comments describe development history. The current model is a configurable core whose capabilities are selected through configuration.
Neural network core
The decoder combines embeddings, stacked Transformer layers, output normalization and vocabulary projection. Its configurable components include:
- MHA, MQA and GQA: the number of query and key/value heads can be configured separately.
- Position and attention: RoPE, linear/YaRN scaling options, causal attention, sliding-window masks, QK normalization and attention/output logit soft-caps.
- Feed-forward layers: dense FFN or top-k Mixture of Experts, with SwiGLU/GELU and router auxiliary loss in the training objective.
- Layer structure: RMSNorm/LayerNorm, pre/post normalization, parallel residuals, weight tying and stochastic depth.
- Memory and execution: incremental KV caching, bounded cache eviction with sink tokens, PyTorch SDPA, optional external Flash Attention, and gradient checkpointing.
Normal forward calls can use SDPA without materializing attention weights. return_attention_weights=True requests the diagnostic weights; collect_diagnostics=True requests detailed tensor statistics. Standard calls return (logits, attention_or_none); cached core calls include a third cache value. External Flash Attention and long-context configurations depend on the target hardware and need their own evaluation.
Tokenizer, data and training
TokenizerCore is the shared entry point to tokenizer infrastructure developed specifically for Turkish. BPE training, vocabulary/merge management, encoding and decoding use the same components. Turkish I/İ handling, syllabification, rule-based stem/suffix helpers and special-token management are part of this infrastructure. Syllabification and morphology are configurable; they do not run unconditionally on every inference call.
A tokenizer developed for Turkish does not necessarily need to be replaced to work with another language. The same vocabulary and merge rules can serve different languages within their text coverage. The tokenizer represents text; the model learns language through training data and weights. Cevahir's current character filters and coverage checks are described in the tokenizer guide.
The data loader carries documents and question-answer records into training with source information. Collection/conversion tools and subtitle processing help prepare text; their outputs enter through the selected data directory.
Training preparation reads supported TXT, DOCX and question-answer JSON data and builds tokenized input/target records. Prepared data can be reused across epochs. Cache identities include source content, the vocabulary, merge rules and encoding settings; V3 consumption also checks cache integrity.
The training system includes source-aware train/validation splitting, grouping of exact duplicate records, length-based batching, dynamic padding, gradient accumulation, precision selection, optimizer/scheduler integration, validation and checkpoint rotation. The active training manager records optimizer, scheduler, scaler, RNG and loop state for resuming at epoch boundaries. MoE auxiliary loss participates in optimization rather than being returned as unused metadata.
Generation, cognition and conversations
The unified interface exposes encoding, decoding, generation and cognitive processing. Generation includes temperature sampling, greedy decoding, top-k/top-p controls, repetition handling, EOS limits, beam search and incremental KV caching.
The cognitive system provides direct, think, debate and Tree of Thoughts workflows; context assembly, tool execution, critic stages, vector-memory integration, response caching and tracing. These are orchestration components whose response quality depends on the trained model, tools and retrieval setup. The built-in calculator executes restricted arithmetic; external tools can be registered through the tool interface. By default the layer runs a lean profile: each message gets one answer generation, at most one critic revision and one entropy probe, with no think/debate/ToT steps, self-consistency samples, constitutional critique or outbound fact-check requests. CognitiveManagerConfig.full() switches all of these back on, including claim lookups on en.wikipedia.org; with a counting fake model that costs 16–18 generation calls per plain message (about 70 generation and 52 scoring calls on a Tree of Thoughts route) instead of 2. When the model reports a violation of every constitutional principle, the full profile adds one suggestion call per principle and a revision: 15 more calls, 31–33 per plain message. Vector memory is on in both profiles: if the optional sentence-transformers package is installed, its encoder model is downloaded from Hugging Face on first use unless it is already cached. A failed entropy measurement now sends the message down the direct path and is no longer read as zero uncertainty: the answer's token budget and temperature use the middle of the entropy scale instead.
The cognitive module guide explains strategy execution, synchronous/asynchronous differences and open development boundaries.
ChattingManager manages conversation history and context. Memory entries, notes and summaries are scoped to users/sessions in the scoped cognitive flow. API services and database repositories connect the engine to authenticated sessions, stored conversations and user data.
Outputs from my training runs
The following screenshots come from my actual model-training runs and generation checks during training. They include prompts, generated responses and training-time output. I keep them here as a record of the training I have carried out with Cevahir.
I've also shared a training-data collection. Configure the data path and preparation settings for your own run. For inference, use a trained checkpoint together with the vocabulary, merges and BPE settings used for that checkpoint.
Development beyond V4
I continued developing the core beyond V4. Version notes in the source record these successive additions:
| Source label | Implementation present in the current code |
|---|---|
| V4 | RMSNorm, SwiGLU, KV caching, advanced checkpointing, quantization and MoE integration |
| V5 | GQA/MQA head layouts, sliding-window attention and YaRN RoPE scaling |
| V6 | PyTorch SDPA, QK-Norm, parallel residuals, attention/output logit soft-caps and scaled residual initialization |
| V7 | Stochastic depth, merged SwiGLU gate/up projection and KV management with sink tokens |
| V8 lower-layer notes | Fixes to boundaries such as KV capacity and MoE auxiliary-loss accounting |
These labels do not identify separately installed model packages. The V3 training service, active V2 training manager and V2 cognitive infrastructure each have their own development history. config_version=1 and architecture_version="cevahir-capabilities-1" identify configuration compatibility. See the neural core guide and architecture contract for the distinction.
Inspecting and developing the system
Model profiling exposes parameter distribution, memory usage and computational-cost estimates; health checks inspect gradients, weights and attention statistics. Cognitive trace and metrics interfaces expose the stages a response passes through. These tools help examine the effect of an architectural choice or cognitive processing step.
My recent infrastructure work has strengthened the connections between these parts: shared model configuration, MoE loss in the training objective, incremental attention caching, tokenizer/checkpoint identity and user/session-scoped memory. I've recorded the details in the lower-core contracts and lifecycle document. I track open work in the next development round.
Current research
The broader question is how operating experience can persistently, controllably and generally change a system's future computation and behavior. The book's research laboratory connects the subsequent work on acquired rules, representation growth, interference, state transport, learned update behavior and recurrent state discovery. The new correction-state experiment examines what must survive to revise earlier experience. The general problem remains open; retesting and computation routing are parts of that inquiry.
The in-network paradigm study (CEV-11) moved this question inside Cevahir's own decoder. In a 3.4M-parameter model trained on an invented Turkish world, a ROME-style rank-one write into one feed-forward projection, made from a single inflected sentence, reached the entity's other case forms written with an apostrophe 24.3 percentage points more often than the same forms written without it. The shortfall sat in forms where root and suffix fuse into one token, and the write also leaked to names sharing the last root token. All three preregistered primary hypotheses were supported in all eight confirmatory worlds, but they describe a write configuration that failed both preregistered leakage constraints in discovery. No secondary hypothesis was supported: variant-averaged keys added only 3.1 points (refuted), the prefix-averaging comparison was inconclusive, the predicted decline of the root-final key's similarity to its neighbors with depth was refuted, and the lifelong-ledger hypothesis was not run (the discovery lives had collapsed at every preregistered λ, and the confirmatory lives were deferred when compute was reassigned); the confirmatory binding-site map and causal tracing were not run either. In discovery, only the suffix-token write and the fine-tuning arms met both leakage constraints, and fine-tuning reached informal forms more often than the name-site writes (a descriptive observation, not a test). The model is tiny, the world synthetic and templated, one tokenizer was used, and each edit was applied alone.
I'm testing whether externally verified experience can help Cevahir choose how to spend computation on a request. The experience-conditioned computation record describes three optional mechanisms:
- Scoped feedback and strategy support: experience stays within the same user/session, model and tokenizer identity. Strategy preferences require support from distinct, externally evaluated examples; corrections and forgetting update that support.
research.mode="off"is the default. Budget, shadow and adaptive modes can be enabled explicitly. - Shared request compute caps: generation, scoring and entropy calls share limits on calls, reserved output tokens and input characters. These are execution limits, not measurements of actual token use or FLOPs.
- An explicit contextual MoE prior: a separately enabled, bounded routing bias can influence the existing router. It remains fixed while a KV cache is populated. The current connection selects researcher-defined profiles; it does not learn expert meanings from experience.
Small contract tests and a synthetic offline replay exercise these mechanisms without running a trained model. I have not established real task-quality, transfer or computation savings from them. The training screenshots above come from my earlier model runs, not these research checks.
The earlier representation witness audit produced a negative result for the proposed learning advantage: when both deterministic searches receive the same ordered candidate language and observations, filtering by training contradictions before the same full consistency check preserves the answer. All 15 completed synthetic comparisons returned the same compatible representations, fitted tables and predictions. Filtering sometimes changed the counted search work, but supplied no new learning advantage in this formulation. I have set aside that claim; the small audit remains separate from the main engine in research/.
Getting started
Run commands from the repository root. The lightweight checks have been recorded on Windows, Python 3.14.3 and PyTorch 2.10.0+cpu and on Linux, Python 3.13 and PyTorch 2.15 (CPU). Dependencies are grouped in root-level manifests: requirements.txt (model, tokenizer, training), requirements-dev.txt (tests, including the HTTP contract tests), requirements-api.txt (HTTP services and database) and requirements-optional.txt (monitoring, vector memory, data collection).
For the small core example below, create an environment and install the core dependencies. PowerShell:
python -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install --index-url https://download.pytorch.org/whl/cpu "torch>=2.5"
python -m pip install -r requirements.txt
On Linux/macOS, activate it with source .venv/bin/activate. The manifests give lower bounds, not a locked environment. Optional subsystems switch themselves off with a warning when their packages are missing.
Run a small model on CPU
This example constructs a small model and performs one forward pass. It uses synthetic token IDs and needs no dataset or checkpoint; it is a convenient starting point for exploring the core.
import torch
from model_management.config_schema import tiny_model_config
from model_management.model_manager import ModelManager
torch.set_num_threads(1)
torch.manual_seed(42)
manager = ModelManager(tiny_model_config())
manager.initialize(
build_optimizer=False, build_criterion=False, build_scheduler=False
)
logits, attention = manager.forward(
torch.tensor([[1, 5, 8, 2]]), inference=True
)
print(logits.shape) # torch.Size([1, 4, 128])
print(torch.isfinite(logits).all().item()) # True
Load your trained model
Use the exact tokenizer files and BPE settings from training. Modern checkpoints contain construction metadata, so an unbuilt ModelManager can build the model from the saved configuration:
from tokenizer_management.core.tokenizer_core import TokenizerCore
from model_management.model_manager import ModelManager
tokenizer = TokenizerCore({
"vocab_path": "data/vocab_lib/vocab.json",
"merges_path": "data/merges_lib/merges.txt",
"use_gpu": False,
# Supply bpe_config here if training used custom BPE settings.
})
manager = ModelManager({"device": "cpu"}, tokenizer=tokenizer)
manager.load("saved_models/checkpoints/last.pth", weights_only=True)
manager.eval_mode()
Replace the checkpoint path with your own. Raw/older weight files without construction metadata require an explicit matching model configuration. Legacy checkpoints without tokenizer identity emit a warning when a tokenizer is supplied; their token semantics cannot be verified automatically.
For the full interface, pass settings through CevahirConfig(model={...}, tokenizer={...}, load_model_path=...). A Cevahir instance builds its model before loading, so those architecture settings must match the checkpoint. load_model_path=None enables default-path auto-detection; load_model_path="" explicitly starts a new model. A selected missing or incompatible checkpoint raises an error. The terminal chat entry point is chat_pipeline.py, whose configuration must likewise match the trained model.
Train with your data
-
Choose the tokenizer. Reuse matching vocab/merges for an existing model. If creating a new tokenizer, set its paths and training options in tokenizer configuration, then run:
python tokenizer_management/train_bpe.py educationRetraining the tokenizer can change token IDs; existing model weights and prepared caches must remain paired with their original tokenizer.
-
Prepare the cache. Match the tokenizer options and
max_seq_lengthto your training configuration:python training_system/prepare_cache.py --data-dir education --no-clear-cache--no-clear-cachepreserves existing cache files; the script otherwise clears old caches. Use--helpfor sequence-length and encoding options. V3 training requires a compatible prepared cache. -
Configure and run training. Set paths, model dimensions, optimizer, batch size, epochs and device in
TRAIN_CONFIGin training_system/train.py:python training_system/train.pyThe checked-in preset is a substantial training configuration, including 100 epochs, batch size 64 and an eight-layer, 512-dimensional model. Adjust it for your hardware before running. The CPU example above is the lightweight entry point.
Architecture configuration is normalized by config_schema.py. Change the configuration supplied to the system; editing the default class definitions in model/cevahir.py is not required. Cache preparation and training still have separate entry-point settings, which must agree.
Application integration
The integrated Flask entry point is api.app_factory.create_app. It wires the model, ChattingManager, services, authentication, health checks and database access. Configure the database, an explicit JWT_SECRET_KEY, model path and matching tokenizer before starting the application. Model overrides can be supplied through CEVAHIR_MODEL_CONFIG in the factory configuration. The older api/app.py retains an obsolete configuration import and is not the recommended entry point.
Requests are attributed to a user only through a verified Authorization: Bearer <JWT> header. The API verifies tokens but does not issue them: there is no login, registration or token endpoint, so your own code must create tokens with api.security.jwt.create_access_token and the same JWT_SECRET_KEY. api/api_config.py selects the mode from FLASK_ENV; any other value is an error:
- Unset or
default: debug off, rate limiting on, CORS limited tohttp(s)://localhost,127.0.0.1and[::1]origins on any port. development: Flask'sDEBUGsetting on and rate limiting off for local work; CORS as in the default. AddingCEVAHIR_TRUST_X_USER_ID=1makes the unauthenticatedX-User-IDheader act as the caller's identity, with a warning logged on every use. Any caller can then act as any user, so the setting is accepted only indevelopmentandtesting; in other modes the configuration refuses to load.testing: for test runs:DEBUGon, rate limiting and metrics off;CEVAHIR_TRUST_X_USER_ID=1is accepted as indevelopment.production: refuses to load unlessPRODUCTION_SECRET_KEYandJWT_SECRET_KEYare two different values of at least 32 characters; placeholders such aschange-me-in-productionare rejected. Whether a value is random cannot be checked, so generate both, for example withpython -c "import secrets; print(secrets.token_urlsafe(48))". Rate limiting is always on, and browsers get cross-origin access only for origins listed inCORS_ORIGINS.
A comma-separated CORS_ORIGINS replaces the default origins; each entry is an exact origin (an IPv6 host goes in brackets, as in http://[::1]:3000) or a regular expression that must match the whole Origin. A non-matching Origin receives no CORS headers. *, and any pattern that matches every origin such as .*, is sent as a literal *, and credentials are never allowed. Rate limits are kept in process memory unless RATELIMIT_STORAGE_URI names a shared store. The unauthenticated /api/v3/health/detailed returns only generic component messages; error details go to the server log. python api/app_factory.py starts Flask's development server on 127.0.0.1:5000 with debug off in every mode, development included; --host/CEVAHIR_API_HOST, --port/PORT and --debug/CEVAHIR_API_DEBUG=1 change that, and debug is refused in production. Real deployments still need a production WSGI server and TLS in front of the API. These behaviors are covered by the contract tests in tests/evolution/test_api_security.py; they have not been checked against a deployed service or by a penetration test.
Repository map
| Directory | Responsibility |
|---|---|
| src/ | Neural decoder, attention, FFN/MoE, normalization and KV cache |
| tokenizer_management/ | BPE training, encoding/decoding, vocabulary and merge management |
| data_loader_management/ | Document and question-answer loading, chunking and source identity |
| data_processing/, dataset_subtitle/ | Data collection, document conversion and subtitle-to-text tools |
| training_system/ | Data preparation, cache, batching and training service |
| training_management/ | Training loop, optimization, monitoring and checkpoints |
| model_management/ | Configuration, construction, loading, saving, profiling and inference |
| model/ | Unified Cevahir interface and generation adapter |
| cognitive_management/ | Strategies, tools, critic, scoped memory and middleware |
| chatting_management/ | Sessions, conversation history and context |
| api/, database/ | HTTP services, authentication and persistence |
| benchmarks/, tests/, scripts/ | Measurements, behavior verification and standalone diagnostic tools |
| research/ | Isolated research experiments, including the deterministic representation witness audit |
| docs/ | Architecture, module guides and development history |
Verification and development status
I've shared examples from my model training above. In current engineering checks, I also use small CPU models to compare cached/full-sequence results, gradients, save/load behavior, tokenizer identity and state isolation. These synthetic loss or timing measurements do not measure the language quality of my trained model.
For a small lifecycle verification group and a separate core measurement:
python -m pip install -r requirements-dev.txt
python -m pytest tests/evolution/test_lower_layer_contracts.py tests/evolution/test_checkpoint_lifecycle.py -q
python -m pytest
python benchmarks/core.py --label local --output benchmarks/results/core_local.json
See benchmark instructions for broader verification. python -m pytest runs the tests/ suite: on 13 November 2026 it gave 529 passed, 11 skipped and 1 expected failure (Linux, Python 3.13, CPU, about 4 minutes). The skipped tests need a trained checkpoint or a generated training cache, which the repository does not ship; the expected failure pins a known checkpoint-loading bug. GitHub Actions runs this suite on every push and fails if a test leaves files in the checkout. Older test folders inside the packages (for example src/neural_network_module/test) still contain tests for removed modules and are not part of this run. I still have open work on bringing lossless Unicode tokenization into the main training path (a lossless byte-level BPE backend now exists in tokenizer_management/byte_bpe.py, separate from the shipped tokenizer), full-record training alignment checks, near-duplicate data separation, concurrent model replacement/generation, and retrieval/critic/ToT quality evaluation. GPU execution, distributed training, long-context quality and real compilation performance require separate validation; the presence of helper modules does not imply integration into the active training path.
Documentation, contribution and contact
- How the modules form a system
- Architecture and supported behavior
- Next major development round: findings and sequence
- Lower-core contracts
- Checkpoint, data and runtime lifecycle
- Experience-conditioned computation: mechanisms, evaluation and limits
- Independent capability discovery
- Representation witness audit: equivalence and the rejected learning claim
- Development log
- Module documentation
I welcome contributions to the core, data and tokenizer pipeline, model lifecycle, application integration and evaluations. Describe the affected flow, the resulting behavior and the verification appropriate to the change.
I share Cevahir under the Apache License 2.0.
Find me online: GitHub · X · Instagram
I updated this document to match the current source code on 20 September 2026.