The OHGOD Philosophy
One Hobbyist GPU, One Day
Six Observations Worth Addressing
"The best way to predict the future is to invent it."
— Alan Kay
Six Observations Worth Addressing
We're not here to tear down. We're here to build. But building
well requires seeing clearly.
The GPU is consumer-grade. The method isn't.
power.concentrate()
A handful of companies control the infrastructure that makes AI possible. Training frontier models costs hundreds of millions. The compute divide is real.
power.concentrate()
- OpenAI Plans to Lose $5 Billion This Year Only deep-pocketed companies can sustain AI
- OpenAI Completes Deal Valuing Company at $80B Massive valuations only possible for few players
- U.S. Antitrust Inquiries of Nvidia, Microsoft, OpenAI DOJ/FTC investigate Big Tech AI market dominance
- OpenAI May Go Bankrupt, Costs $700k/Day ChatGPT costs company $700k every single day
planet.burn()
Training large language models consumes extraordinary energy. By 2027, AI could consume 85-134 TWh annually—equivalent to Argentina, Netherlands, or Sweden combined.
planet.burn()
- AI Adoption in US Adds ~900k Tons COâ‚‚ Annually 2025 study finds significant carbon impact
- Three New York Cities' Worth of Power: AI Is Stressing the Grid AI data centers demanding massive power capacity
- AI, but at What Cost? Carbon Footprint Breakdown Training 100B parameter model = 30h of 737 flight
- Google's Carbon Emissions Surge 50% Due to AI Data center electricity up 17% in 2024
privacy.null()
LLMs are trained on the internet—your posts, your writings, your conversations—often without asking. Every interaction is a data-sharing operation.
privacy.null()
- Italy Fines OpenAI €15M After Privacy Probe GDPR violation: €15M fine for data collection
- Stack Overflow Users Deleting Answers After OpenAI Deal Users protest their content being sold to AI
- AI Companies Cause Most Traffic on Forums 4.8M Claude hits, 148k ChatGPT hits in 7 days
- Employees Feeding Sensitive Data to ChatGPT Raising security fears about corporate AI use
model.sealed()
Frontier AI companies disclose nothing—architectures, training data, failure modes all hidden. If we're being rational, OpenAI should be called ClosedAI. Secrecy for competitive advantage.
model.sealed()
- Foundation Model Transparency Index Stanford ranks AI companies on disclosure—most fail
- 2025 AI Index: Transparency Declining OpenAI dropped 14 points; now ranks 5th/6th
- GPT-4: 0% Architecture Disclosed Major release with zero technical transparency
- GPT-4o: Still No Details Pattern continues—secrecy for competitive advantage
replies.intelligent()
In the Middle Ages, the unexplained was God's work. Today, the unexplained is 'intelligence.' We just need to open the box.
replies.intelligent()
- Sycophancy Is the First LLM 'Dark Pattern' 100% of users affected by AI over-agreement
- Claude Says 'You're Absolutely Right!' About Everything LLMs agree even when users are wrong
- Hallucination Is Inevitable: Innate LLM Limitation Mathematical proof that LLMs must hallucinate
- NY Times Suit Wants OpenAI to Delete All GPT Instances Memorization enables copyright infringement claims
architecture.frozen()
The Transformer was invented in 2017. Eight years later, it still dominates—despite known limitations and promising alternatives.
architecture.frozen()
- 'Attention Is All You Need' Coauthor 'Sick' of Transformers After 8 years, even creators want alternatives
- DeepSeek-R1: Major Open Model Release New architecture challenges transformer dominance
- Meta Llama 3: Same Architecture, More Scale No major architectural changes from transformer
- The LLaMA Effect: Leak Sparked Open Source Alternatives Open weights enabling architectural experimentation
The Power Is Too Concentrated
The Observation
A handful of companies control the compute, models, and data that power AI. They publish scale to make it feel impossible. They never publish what actually makes it work. This isn't conspiracy; it's just economics. Training frontier models costs hundreds of millions of dollars. GPU computing is dominated by a handful of players. The chips that power everything flow through two or three companies.
"There is now a 'compute divide' between Big Tech and conventional R&D centers like universities, leading to increasing concentration of GenAI research in the hands of a few companies."
— Oxford Academic (2024)
Why This Matters
When only a few can build, only a few decide what gets built. Research narrows. Innovation follows money. And the rest of us become consumers of technology we can neither understand nor influence.
What Could Be Possible
OHGOD!! What if powerful AI can be trained on a consumer GPU in 24 hours? The compute moat might disappear. Universities won't need cloud credits to teach AI. Hobbyists could experiment and have fun.
The Planet Can't Afford This
The Observation
Training large language models consumes extraordinary amounts of energy. Training GPT-3 produced an estimated 502 tonnes of CO2. Every ChatGPT query uses roughly 15 times more energy than a Google search. Inference now accounts for more than half of total carbon footprint.
"Electricity demand from data centres worldwide is set to more than double by 2030 to around 945 terawatt-hours (TWh), slightly more than the entire electricity consumption of Japan today. AI will be the most significant driver of this increase, with electricity demand from AI-optimised data centres projected to more than quadruple by 2030. In the United States, power consumption by data centres is on course to account for almost half of the growth in electricity demand between now and 2030."
— IEA Energy and AI Report (2025)
Why This Matters
We love AI. We think it can do tremendous good. But we refuse to ignore the costs. Climate change is real. Energy consumption matters. And 'move fast and break things' isn't a philosophy that scales to planetary systems.
What Could Be Possible
OHGOD!! What if we could build efficiency into these foundation models? Training on a single GPU in 24 hours under someone's desk might mean people can take care of their inference needs right from home, with their own electricity.
Your Data Isn't Yours Anymore
The Observation
Researchers proved transformer LLMs are mathematically invertible. Hidden states aren't abstractions—they're your prompt in disguise. OWASP 2025 lists vector embedding weaknesses as a top vulnerability. Embedding inversion attacks recover names, diagnoses, and financials with 90%+ accuracy. 'Zero retention' is theater—the New York Times lawsuit forced OpenAI to hand over all logs anyway.
"You touch that thing with AI, your one sensitive document is like 5x-ed. Training sets, search indices, prompts, the model, the logs. And those other places? No one's paying attention to."
— Patrick Walsh, DEF CON 2024 - Exploiting Shadow Data in AI Models
Why This Matters
Privacy isn't about having something to hide. It's about the right to think, explore, and make mistakes without being recorded, analyzed, and subpoenaed. Every cloud AI interaction is a data-sharing operation you can't take back.
What Could Be Possible
OHGOD!! What if your data never had to leave your device? What if the model just ran locally? Your thoughts stay with yourself. No embeddings to invert. No need to be afraid to ask silly questions.
Nobody Knows What's Inside
The Observation
Stanford's Foundation Model Transparency Index (December 2025) scored major AI companies at just 41/100 on average—down 17 points from 2024. OpenAI dropped 14 points. Meta dropped 29. Both fell from 1st-2nd place in 2023 to 5th-6th in 2025. Only 30% of companies even responded to transparency requests (down from 74%). The lowest scorers? xAI and Midjourney at 14/100. This isn't secrecy for safety. It's secrecy for competitive advantage.
"Transparency is a vital precondition for public accountability, scientific innovation, and effective governance."
— The Foundation Model Transparency Index
Why This Matters
We're building critical infrastructure on foundations we cannot examine. When AI makes decisions about hiring, lending, healthcare, and justice, 'trust us' isn't good enough. Science requires verification. Democracy requires transparency. Even 'open' models like Meta's score poorly—openness doesn't guarantee transparency.
What Could Be Possible
OHGOD!! What if you could see exactly how your AI works? What if every architecture choice, training dataset, and failure mode was documented? No black boxes. No 'trust us.' You train it yourself, so you know what's inside. The industry scores 41/100 on transparency, we can do better.
The Intelligence of the Gaps
The Observation
Medieval scholars attributed the unexplained to God; we attribute impressive LLM outputs to 'intelligence.' But for transformers, we actually know a lot: induction heads drive in-context learning, FFN layers store facts as key-value memories, sparse autoencoders extract interpretable features, circuit tracing maps decision pathways. Hallucinations, sycophancy, memorization; these aren't mysterious emergent properties. They're statistical pattern-matching we should examine more.
"Just as 'God of the gaps' shrinks as science advances, our definition of 'intelligence' keeps shifting. There's a running joke in AI: intelligence is whatever computers haven't done yet."
— Marcel Salathé, 'AI and the God of the Gaps' (2024)
Why This Matters
72% of users trust AI for facts. 75% of those get misled. Hallucinations, sycophancy, memorization; these aren't mysteries. They're predictable outcomes we could fix with more experimenting.
What Could Be Possible
OHGOD!! What if 'intelligence' is just a label for what we haven't examined yet? Build small enough to train yourself, and you can call it what it actually is.
The Architecture Barely Changes
The Observation
The Transformer was invented in 2017. Eight years later, it still dominates—despite known limitations and promising alternatives. LLaMA-3 made no major architectural changes. Innovation requires experimentation, and experimentation requires affordable compute.
"Despite many small and creative innovations since the original transformer architecture, there have not been any significant 'breakthrough' discoveries that have led to much better leaderboard results."
— arXiv Survey (2024)
Why This Matters
If the Transformer isn't optimal (and evidence suggests it may not be), then we're investing billions scaling a suboptimal design. Mamba offers 5x throughput. RWKV proves RNNs can compete. Griffin matches Llama-2 on 6x fewer tokens.
What Could Be Possible
OHGOD!! What if there's a better architecture waiting to be found? If you can train something in a day on a consumer GPU with your own evaluation, maybe you'll find it.
The Experiments We Propose
The Hypothesis
There exists at least one architecture that achieves meaningful language model performance when trained on a single consumer GPU within 24 hours at the 200-500M parameter scale.
This is a hypothesis, not a promise. We don't know if it's true. That's why we're running the experiments.
Proposed Experimental Setup
Hardware Constraint
Training Constraint
If the Hypothesis Holds, It Implies:
Anyone can do AI research
Meaningful AI with minimal energy
Personal AI that stays personal
Fully transparent, verifiable models
Mechanisms over mysticism—AI we dissect
Architecture experiments anyone can run
The Evidence That Motivates Us
The Cramming Precedent
Geiping & Goldstein (2022)"How far can we get with a single GPU in just one day?"
Their answer for BERT: remarkably far. OHGOD extends this inquiry to decoder-only models—the architecture that powers modern chatbots.
SmolLM2: Already Working at Target Scale
Hugging FaceModels at our target size can perform. The question: can we get meaningful results with 1/40th to 1/80th of the training compute?
| Model | Params | Tokens | HellaSwag |
|---|---|---|---|
| SmolLM2-135M | 135M | 2T | 42.1% |
| SmolLM2-360M | 360M | 4T | 54.5% |
Gemma 3 270M: Extreme Overtraining Works
Google (2025)Google trained a 270M parameter model on 6 trillion tokens—1,111x the "optimal" ratio. It works. The relationship between model size and capability is more flexible than the scaling orthodoxy suggests.
The Call to Action
To Researchers
Question the scaling orthodoxy. Prove what's possible with constraints.
The assumption that intelligence requires scale is testable—not axiomatic. One GPU. One day. What can you build? The OHGOD constraint isn't a limitation—it's a research methodology that forces innovation over brute force.
Investigate:
- • Alternative architectures beyond transformers
- • Training dynamics under compute constraints
- • Memorization vs. generalization metrics
Explore:
- • Data efficiency and smart curation strategies
- • Benchmarks that measure understanding, not recall
- • Training techniques for consumer hardware
To Users
Demand AI that respects your autonomy.
You should not have to send personal data to remote servers, trust corporations with your conversations, or accept opacity as the price of capability.
- • Support open-source projects
- • Demand transparency
- • Reject cloud dependency
To Everyone
The democratization of AI is not inevitable. It must be fought for.
The current trajectory leads to a world where AI capability is a corporate monopoly—licensed, monitored, and controlled by entities whose interests diverge from yours.
OHGOD proposes an alternative: AI that belongs to individuals. AI that runs on your hardware. AI you can inspect, modify, and trust.
Our Commitments
1 Radical Transparency
- • All code will be released (open source)
- • All weights will be released (Hugging Face)
- • All training logs will be public
- • All technical information will be public
2 Reproducibility
- • Anyone with consumer hardware can verify our claims
- • No proprietary data
- • No secret sauce
- • No "trust us"
3 Honest Evaluation
- • Measuring what matters—understanding, not just benchmarks
- • Reporting what we find, not what we hoped to find
- • Pre-registered hypotheses
4 Open Science
- • Public amendment logs
- • Negative results published with same rigor
If the experiments fail, we'll document exactly how and why. That's what makes this science, not marketing.