Experiment 1: Open-Source Alignment Benchmark

Our first open-source alignment benchmark with methodology and results

T# Post-Mortem: The “Green Detection” Neuron Experiment — Lessons from the Lab

By the TinkerForge AI Research Team — July 2025

Introduction

As TinkerForge AI opened its doors, we set out to explore the fundamental building blocks of biological intelligence: neurons. Our founding experiment, chronicled here, was a bold attempt to capture the emergent, interactive, and adaptive nature of biological neurons — in code. Our goal? To see whether a simple, bio-inspired neural system could “learn” to detect green pixels, mimicking the learning and decision-making processes of real neurons.

This post-mortem is both a technical summary and a candid reflection on what we learned as a new research lab — about neurons, about code, and about the creative chaos of ambitious experiments.

The Vision and the Reality

We began with the idea that neurons are the basis of biological thinking, and that in artificial systems, vision appears to be the easiest thing to test. We decided to build out a project that might incorporate the learning process of an actual neuron and neural network using membrane potential, learning rates, activation rates, and more. But this proved to be much more complicated and nuanced when we created the entire system.

It was very easy to lose track of where we were, what code had been created, and how to rightfully tune the system in general. This introduced a great reason we don’t try to attack ALL problems at once: it’s complicated!! Keep it simple!

Thus, we ended up with a hodge-podge of great ideas, implemented in code… with no insightful way of tuning things because our own reach exceeded our grasp. In this way, the overall project was scrapped, and attempted to be recreated with a simpler format in experiment 2.

What We Actually Built: Technical Achievements

Despite the project’s complexities, our team accomplished a number of technical milestones:

1. Custom Bio-Inspired Neuron Model

• Each neuron tracks a specialty (e.g., preference for green, red, or blue), a membrane-like activation potential, and a dynamic threshold based on how well its specialty aligns with the global goal (“detect green”).
• Neighbor Communication: Neurons excite or inhibit each other via messages — a nod to biological networks.
• Adaptive Learning: Learning rates change dynamically. If a neuron consistently makes correct predictions, it “stabilizes” and its learning rate drops (reducing overshooting and mirroring synaptic plasticity).

2. Ensemble Monitoring and Emergent Clusters

• Clustering: Neurons are grouped by specialty using unsupervised clustering (e.g., k-means).
• Activity Tracking: We log which neurons fire for which inputs, and how their specialties evolve.
• Human-Readable Mapping: Our monitor module translates the raw numbers into interpretable summaries — e.g., “This cluster specializes in green!”

3. Goal-Driven Activation

• Neurons that are well-aligned with the current goal (“detect green”) receive an activation boost — reflecting how biological systems direct attention.
• Contextual Processing: Activation thresholds and firing depend not just on the input, but on neighbor signals, past activation, and current goals.

4. State Persistence and Experiment Logging

• All experiments, activations, and neuron states are logged in detail (logs/YYYY-MM-DD.json). This enables reproducible science and post-hoc analysis.
• The system can resume from previous states, allowing us to study emergent properties over time.

5. Interactive Experimentation and Visualization

• We created demo scripts for testing different network topologies (linear, ring, star) and for visualizing neuron activations and specialties.
• Human researchers can adjust parameters, rerun experiments, and generate human-readable summaries at each step.

Key Innovations and Biological Inspirations

• Neighbor Excitation/Inhibition: Mirroring synaptic signaling, neurons influence each other’s likelihood of firing.
• Dynamic Thresholds: Firing isn’t fixed — it varies based on goal alignment and recent activity.
• Specialty Emergence: Neurons develop emergent “preferences” (e.g., for green) through repeated exposure and learning.
• Adaptive Weight and Confidence Learning: Each neuron’s weight (akin to synaptic strength) is learnable and adapts with experience.
• Cluster Competition: Clusters compete for specialization, preventing any one group from dominating — a step toward more diverse, robust learning.

Lessons Learned

• Ambition vs. Execution: The desire to capture the full nuance of biological neurons led to a complex, evolving codebase. Each new feature (adaptive learning rates, cluster competition, goal-driven activation) made tuning and troubleshooting harder.
• Keep It Simple: The experiment reinforced the value of narrowing focus. Tackling too many axes of biological complexity at once quickly overwhelmed our ability to reason about the system.
• Transparency is Key: The need for clear logging, visualization, and human-readable summaries became apparent. Without these, interpreting emergent behavior was nearly impossible.
• Iteration Matters: This experiment was scrapped — but not wasted! The technical scaffolding and lessons learned directly informed our next iteration (“Experiment 2”) with a much simpler, more focused design.

What’s Next?

• Refined Experiments: Experiment 2 will focus on one or two axes of biological plausibility at a time (e.g., just specialty emergence and simple neighbor signaling).
• Better Tooling: More robust visualization and parameter tracking.
• Comparative Benchmarks: Testing against traditional neural nets (CNNs) and tracking robustness to noise and novelty.
• Open Collaboration: We’re inviting the wider research community to contribute new activation functions, clustering strategies, and visualization tools!

Conclusion

Our first experiment was a beautiful mess: ambitious, chaotic, and ultimately unsustainable in its initial form. But in true research spirit, it provided the foundation for a more disciplined, interpretable, and powerful approach. At TinkerForge AI, we’re committed to open, iterative, and honest science — and we hope our post-mortems are as valuable as our successes.

Stay tuned for Experiment 2 and beyond!

Interested in collaborating, or have feedback on our approach? Reach out to the TinkerForge AI team!